added language detection, text and html checks for content type
added ordered words field for analytics
moved dupefilter logic to nofilter.py
updated start urls, settings
added new dupefilter, updated code to use url hashes, updated scripts code to reflect that