WordTokenizer.tokenize rewritten
doctests indent
fix unicode handling for a new tokenizer; add pounds char to rules
Merge branch 'speed_up_text_tokenizer' of https://github.com/chekunkov/webstruct into speed_up_text_tokenizer
Merge pull request #16 from scrapinghub/speed_up_text_tokenizer

Speed up text tokenizer