The protesters, carrying placards which read "Rise up against the restart" and "The nuclear era is over," lined the streets around Prime Minister Yoshihiko Noda's residence in central Tokyo as police watched on, according to an AFP photographer.
The main entrance to the residence was seen guarded by armoured vehicles and barricades of uniformed police.
Organisers quoted in local media estimated turnout exceeded 100,000 people, over double the turnout they estimated at a similar protest last week.
The demonstration had been called by liberal writers Takashi Hirose and Satoshi Kamata in an online message which spread on Twitter and Facebook in what was likened by a popular tabloid to the "Arab Spring," a wave of protests that topped governments in the Arab world last year.
The crowd blocked off a six-lane road and adjoining streets leading to the Prime Minister Yoshihiko Noda's official residence in central Tokyo. Police parked five armoured riot control buses in front of the entrance to prevent protesters entering the compound.
Several helicopters circled overhead as the sun went down on a clear, early summer evening.
The protest capped weeks of sporadic demonstrations and was the biggest gathering in central Tokyo since Noda said this month the restart of two reactors in western Japan was necessary to avoid damaging the economy.
All of the country's 50 nuclear reactors were taken off line after an earthquake and tsunami hit the Fukushima nuclear power plant on the northeast coast on March 11 last year, triggering the world's worst atomic accident since Chernobyl in 1986.
Nuclear power had previously supplied nearly 30 percent of Japan's electricity.
The first of the two Ohi reactors operated by Kansai Electric Power Co is scheduled to be reactivated on Sunday.
The crowd, including office workers, mothers with children and elderly people, chanted "oppose restarts" and "exit nuclear power".
The decision to restart the reactors as summer power-cuts loom was seen as a victory for Japan's still-powerful nuclear industry.
But Japanese people have grown wary of nuclear power since Fukushima, with surveys showing that about 70 percent want to abandon atomic energy even if not immediately.