yeah I don't think it is worth switching servers just to be on the one that might be a tiny bit more populated. They really are pretty even in size, and AFAIK none of them have under-population problems.
Also keep in mind you are restricted from doing another server transfer for some time once you do one so if, say, a old friend comes back and you want to be on the same server, or you make friends on the forums and want to join their LS on another server, you'll be restricted from changing servers.
I would only recommend changing servers if you have a good linkshell waiting.