tests and annotations
- added tests for the tokenizer util
- added ThreadSafe and NotThreadSafe annotations
- several misc typo fixes and cosmetics that I don't quite remember