Big changes, including schema changes.
Switch to readability_extract
Made scraping more robust. Improved syncing checkpoint logic. Added a command to rescrape old content.
Added pytz to requirements.txt
Added static assets.
Added PDF content-type guard.
Don't try to parse an empty string or None as HTML when the document is missing a title.
Don't try to parse an empty HTTP body.
Changed deny-PDF,default-allow policy to allow-html,default-deny policy. Catch invalid URL exceptions.
fix for accepted status code change
made congressional scraper compatible with overhaul, fixed broken speaker.gov scraper code
fixed date parse for speaker.gov
Added Romney for President scraper.
logging.notice => logging.info
Typo fix: ) => }
Allow source doctype to be blank
Added user_agent field to Source model
Added unicode logging to handle logging of errors related to URLs with unicode characters in them. Also added proper mime type parameter parsing.
Added search_fields to admins.
Added django-extensions