cleaned up documentation; added example

for 2.4 release -- cleaned up documentation; added example in the Tokenizer class.

ready for review and merge.