That could be due to the server not being a very fast computer. I have played on may servers with absolutly no lag whatsoever, then other where it seems like you are shooting blanks. So i'm guessing it's mostly a server hardware issue more than a netcode issue.
I would like to see some filter options in the browser personally. To only show populated servers, below a certian ping, gametype, and filter by patch version.