The nation’s Meteorological Agency issued a special weather warning for what it called “historic” rains.
Landslide warnings have been issued in over a quarter of the nation’s prefectures.
The agency also warned of rising rivers and strong winds.
National broadcaster NHK said at least 62 people were dead and 44 were still missing.
The Fire and Disaster Management Agency said by Saturday morning, more than 1.6million people had been ordered to evacuate their homes.
A further 3.1million more were advised to leave.
Helicopter footage showed people on their roofs waving for help in Kurashiki.
The footage also showed Japanese soldiers rescuing children by boat from a flooded river in Hiroshima.
Chief Cabinet Secretary Yoshihide Suga said around 48,000 police, firefighters and members of Japan’s Self-Defence Forces were responding to appeals for help.
The agency said in Motoyama, a town on Shikoku island, around 373 miles from the capital Tokyo, a resounding 23 inches of rain fell between Friday and Saturday morning.
An official at the Japanese Meteorological Agency said: “We’ve never experienced this kind of rain before.
“This is a situation of extreme danger.”
Prime Minister Shinzo Abe added: “There are still many people missing and others in need of help, we are working against time.”
Among the missing is a nine-year-old boy who is believed to be trapped in his house by a landslide that left at least three others dead.
NHK said people who have died include a man who fell from a bridge into a river in western Hiroshima city, and a 77-year-old man in Takashima, 35 miles east of Kyoto, who was swept into a canal as he tried to remove debris.
Four people in Ehime, Hiroshima and Yamaguchi were also in critical condition after being injured in landslides.