This is due to the fact that the other players are most likely taking a different route to the server and are not being affected by the problem. If you consistently notice high ping times, contact your ISP and discuss the problem with them. Be sure to have a list of the problem hosts or to actually be running UOtrace while you are speaking with them. There will be some routers that your data will pass through that are independent companies and your ISP may not have any influence over them. In other words, you are sending information to or through that host and the host is either not receiving the data or is not able to return it to you.
Any packet loss is a sign of some type of problem.
If you consistently notice packet loss, you should contact your provider to discuss possible solutions. The quality of your hardware components plays a factor in achieving optimal performance. This means that the various devices that make up your computer can affect how well your computer runs the Ultima Online application. You may be able to determine if this is the case by using a friend's computer, if one is available. Try to connect to your current ISP via their computer, and see if you notice any difference.