scrapy/w3lib — pull request #1 by shane42 (2012-02-10 18:28:16+00:00)
Title: Add encoding functions

These are loosely based on the encoding in scrapy.

Main differences:
* tweaks to regular expressions for encoding detection in HTML. One regexp handles html and xml
* handle byte order marks
* better handling of character encoding overrides, with an updated list
* does not fall back to BeautifulSoup, instead the auto-detect is customizeable and disabled by default