added stream comments to tokenizer