harvard-lil/perma — pull request #401 by jcushman (2014-01-19 20:27:28+00:00)
Title: Serve source view using WARC files if they exist. 

This adds a Python application based on pywb that runs alongside our
Django app (under the same wsgi application) and serves WARC files. WARC files will be used if
link.asset.warc_capture points to a “.warc” or “.warc.gz” file. (Until
we add WARC capture, the only way that would happen is if you manually
edit the database.)

For testing, if you have Matt's warc test file ( out.warc-20131220120520963-00000-21657-990ubunutu-8000.warc.gz ), you can create a Permalink for one of the links it contains ( e.g. http://edition.cnn.com/2013/11/05/opinion/annan-cordoso-humane-drug-policy/ ), put the warc test file in the assets folder for that link, and change its warc_capture entry in the database to the warc's filename. It should then load in under "Archived site" when you view the link.

Once you set warc_capture you can also browse any link in the wark directly at /warc/[guid]/[link], something like this:

/warc/1234-ABCD/http://edition.cnn.com/2013/11/05/opinion/annan-cordoso-humane-drug-policy/

DEPLOYMENT NOTE:

This depends on forked versions of three packages that aren’t in PyPI —
pywb, warc-tools, and cdx_writer. Hopefully the changes will be merged
upstream and they’ll be added to PyPI. In the meantime they’re living
in my repo and included as git submodules.

So, when you update for the first time, you need to run:

git submodule init
git submodule update

to pull down the libraries.