mwrock/RequestReduce — Issue #52: Periodically check for new content
Author: mwrock
Created: 2011-10-22 09:32:57+00:00
State: open

Currently we only fetch new content upon a URL change. This makes content refreshes a bit painful forcing the use of explicit flushing if the change does not result in a url change. We are also forced to ignore near future expiring javascript. If we routinely checked and refreshed content, we could merge/minify near future javascript.

As we process content, persist the original urls and caching headers/etags. If not busy processing content, issue head requests for cached content every N minutes and refresh if content has been changed.