Well you can check if you have a virus or spyware, but yeah, the server is probably causing it.
Before I upgraded my video card I was getting 5-10 fps and after I upgraded to a Radeon 9250 with 128 megs of vram I get 20-25 fps. Except when I lag which is often. And I have a gig of ram.