You don't really have to specialize, no, but it is a great idea. Specializing is a great thing, because the higher the "Specialize <insert spell type here>" skill is, the less likely you are to fizzle a spell of that type, the less time it takes to cast a spell of that type, and I believe you randomly use less mana to cast a spell of that type.
So, to clarify.
Abjuration, Evocoation, Alteration, Conjuration and Divination:
The spell types that all casters have, and these all cap at the level+1 * 5 formula.
Specialize Evocation, Specialize Abjuration, Specialize Conjuration, Specialize Alteration and Specialize Divination are completely seperate skills that give the bonus highlighted above. When you hit 30 (or 20 for an int caster), put a training point into each of these, then, either train the one you like the most all the way to 51 to ensure that one is your "primary" specialize, the one that gets the most bonuses highlighted above. Or, leave them all at base and see which one goes up more as you play then base your decision that way.
My druid primarily solos, so I chose specialize evocation for DDs and quading, but alteration is also a fine choice for grouping because it improves heals.
When people talk about resetting them all to 50 they are referring to a quest in the Temple of Ro that allows you to reset your primary specialize down to 50 so that you can chose a new primary specialize.
Good luck!