harvard-lil/perma — pull request #723 by jcushman (2014-11-05 16:22:23+00:00)
Title: Faster warc playback.

This hugely speeds up warc playback using our redis cache, avoiding the need for an http lookup, database hit, and filesystem hit for every single resource in the warc.

DEV NOTE: Now we have redis going, we can cache other stuff using Django's caching system. This could e.g. be a cleaner way of generating our stats page than the celery task -- just wrap the view in a `@cache` decorator. The upcoming auto-generated list of registrars is another plausible cache target -- anything slow to generate and rarely changing.

DEPLOYMENT NOTE: The 'CDX_SERVER_URL' setting is no longer necessary and can be deleted.