add functions for determining the encoding of html

This is based on the encoding detection in scrapy
added BSD license file
extend BOM detection to utf-32 and perf. improvement
small improvement to documentation

The content type header parameter to html_to_unicode has been documented
more clearly.
Merge pull request #1 from shane42/master

Add encoding functions for converting html to unicode
correctly handle missing encoding in html header
added NEWS File
bumped version to 1.1
removed support for running setup.py test, in favour of using nosetests
added link to NEWS file to README
handle multiline comments and pre compile regular expression
Merge pull request #2 from dangra/multiline-html-comments

handle multiline comments and pre compile regular expression
normalized indentation to 4 spaces
removed use of deprecated urljoin_rfc() function
check for data length before looking for BOM
fix non returning value bug for encoding.to_unicode and reuse it elsewhere
Merge pull request #3 from dangra/encoding-detection-for-empty-body

Encoding detection for empty body
remove extra spacing from testfile
add two border test cases
bump version to 1.2