I promise we'll talk again about BGP real soon. In particular, about how to share the traffic load properly when multi-homed. In a month or two, I'll hopefully be able to give some real-world experience, as we're considering building some web caches to improve quality for our downstream ISP customers. When a national provider's network is having a very bad day, aggressive caching is a way to get the users zippier access to the data. Getting much more than that is trickier, but 40 to 45 percent is somewhat achievable.
More disk is necessary, but the key factor is a larger user population.
The more end users that are hitting on the cache, the greater likelihood that the cache will have a higher hit rate, because the chances are better in a larger user population that two people want to go to the same place on the Web. It's possible to set up hierarchies of caches using ICP, the Internet Cache Protocol, and to configure your cache to get the data requested by users of other caches in the cache hierarchy, thus simulating a larger user population and making your cache more effective. A simple modification allows you to simulate the larger user population. So this data doesn't necessarily need to be charged for at the full rate. Often, this prepopulation is done when bandwidth is cheaper or when the network is just less utilized.