Be sure to ask exactly how many hops away from the backbone your provider is connected. The speed and size of their Web servers. The more CPU power dedicated to handling server requests, the better. High end servers running UNIX or NT are usually the performance leaders. Multiple diverse connections.
Internet traffic can take many different paths to reach a given destination.
Some paths are faster or more direct than others. Having multiple, diverse connections to the Internet is a key requirement. What this means is your data has more paths to choose from, and better odds of finding a faster route. Peering points are where traffic from the large backbone providers get transferred from one network to another. Like an interchange between large interstate highways, where traffic often slows to a crawl.