halved the time between page loads, still greater than what's specified in robots.txt