enhanced tokenizer and updated README.md

enhanced tokenizer now includes @ and # in the tokens.

readme.md updated with some additional steps for first timers running the code.