If people took the time to look at the alternatives, nuculear isn't so bad. Sure, we get some very, very dangerous waste, but we get a small amount of it, and we know where it is. We can't say that about some of the other options.
Fear and misunderstanding mainly.
That said, nuclear energy does have a large set of downsides.
Nuclear power plants are ridiculously complex, and expensive to build. The technology is fairly new (despite it having been around since the 40s).
Once built, the reactors require around-the-clock monitoring to keep them under control, while still being hot enough to be productive. There are a ton of safety procedures that demand 100% adherence and perfect execution. Even then, there will be unforeseen problems that require a quick response to keep the reactor safe.
Even the best reactor designs won't protect you against all foreseeable problems. Fukushima is a great example. The Fukushima reactors were designed to handle quakes up to 7.0 - considered to be a once in a century event. However the quake that hit Fukushima was one of the strongest ever recorded in human history. This quake was followed up closely by a 2nd punch - a tsunami so high it flooded the Diesel powered water pumps. Without the pumps to keep the reactor cool, the cores literally melted, forming a swimming pool sized puddle of radioactive goop. There's really no way to recover from this. At best you'll have to rebuild the core. At worst, you'll have to tear down the whole building and start from scratch. Japan is assuming they'll have to do the former. The buildings are just too dangerous to dismantle without exposing the surroundings to highly radioactive materials.
On top of that, reactors produce radioactive waste. This is mainly in the form of spent uranium that is no longer radioactive enough for the reactor to use but is still dangerously radioactive to you or me. Worse still, it will remain this way for hundreds even thousands of years. So then you have transport and store this material in such a way that it'll be untouched until long after it's since become mostly harmless.
Fear of the unknown and the unseen. Whole countries run under nuclear power. Will there be mishaps, yup. Everything we make causes it's own issues. THis just happens to be a biggy when it goes bad.
It's pretty damn dangerous, just look at Chernobyl. I'm not saying it shouldn't be used (I think it should) but people are worried about nuclear fallout and the waste produced.
Most educated Americans are not scared of it, its the Left wing Media and Left wing politics that are creating mass hysteria.