I'd say that it's the 4.2 GHz, you realize that when more players are getting on the server the server is lagging more and more.
Changing the IP could solve the problem, but I'm not sure how long. I don't know how exactly the servers are running, can't tell more.