fix unicode handling for a new tokenizer; add pounds char to rules