It's already been said: world PvP is part of World of Warcraft. Occasional raids into major cities keep things interesting. You'll also find a lot of higher-level people tend to hang around the major cities anyway, so they can still repel invasions, and with linked city chat, reinforcements can come from the other cities.
I'd rather get them attacking a major city (except perhaps Darnassus- it's usually almost devoid of high-level players, and takes too long to fly to) than attacking the lowbie areas. I've seen high-level Horde raids on Sentinel Hill in Westfall of all places. They sit there and camp all the NPCs, including the quest givers, or kill the Defias Traitor over and over again, making it impossible for questing players to complete their quests. I've also seen raids into Auberdine during low server population periods- basically choosing one of the towns which is least likely to face opposition in. As soon as a few 60 Alliance arrive, they disappear.
Let's not forget the Alliance-side raids either, most commonly the Crossroads of all places. It takes longer to get to from Orgrimmar than Razor Hill, so gives Alliance players that raid towns more time to have fun at the expense of the people trying to quest there. I've also seen one small raid into Bloodhoof Village in Mulgore, but that takes a long time to get to- you have to pass Xroads to get to it. I did see a raid on Brill once- only 10 Alliance, and it's close to Undercity, so the people that planned it gave up rather quickly when a large mass of lvl 60s showed up. I was on an undead character that time, and one guy thought it funny to keep killing the NPC I needed to turn my quests in to. The biggest raids I ever saw was a 30-man Horde raid on Darnassus around 9-10pm server time, guaranteeing not many 60s would come right away, and a 40-man Alliance PUG raid on Xroads.
World PvP is part of the game, whether you're on a PvE server or PvP. How much of it affects you depends largely on whether you flag or not, especially if the raids are on friendly ground (which they usually are). As has been mentioned, you could always just move to another location until the carnage has been completed.
Edited, Today 9:41am by Wondroustremor