It would takes maybe five awful ratings to get your karma down that much. Since you don't have a lot of ratings either way, your karma can take a jump from just a few ratings.
As time goes along and you get rated more, it'll take more for your score to change.
Edited, Dec 8th 2006 9:14pm by Paskil