refactored Java tokenizer to single pass