Add a README
Add asset support to OFT crawler
Allow a failure to dump a referer chain

* Lets us tracethe URLs that caused a problem
Stop truncating summaries

* anything after the second para is OK
* Fixes #7
Make sure we hang on to CASE_DETAIL urls

* We can pull them out of the doc using meta DC.identifier
Ah. Also hang onto our Set of original_urls

* Oops