embolalia/willie — pull request #446 by NachE (2014-02-03 12:20:22+00:00)
Title: [web] [modules/url] Improve charset detect on url.py, new class on web.py

This try to improve the charset detect of a web page to properly decode document on find_title() function.

In some cases a page with not charset on headers force url.py find_title() to fail silently. Its because this function does not parse html for lookup correct charset.

This patch add a class on web.py that allow devels lookup tags and attributes on a html document. This new class is used by find_title() on url.py for search tag that allow content-type attribute and charset declaration. Once upon the charset is detected, the title can be decoded with a correct charset.