Actually, the part I like about fishing s that the best fish are usually found in places that are "hot", with mobs even if not playing PVP. You have to swap out your primary melee weapon for a fishing pole, which as a weapon isn't much. If a monster suddenly turns up (a Naga swimming by while you're fishing in his larder usually takes exception to your presence) its a scramble, not as bad for a hunter since your pet can hold him while you switch, but still a problem.
This goes double for fishing in Lake Wintergrasp, where you're autoflagged for PvP. Back when that area was hot, I set up my Outfitter "fishing" costume to have my fishing hat, my fishing pole, and every other slot filled with my PvP gear.