added type symbols, ids and comments to tokenize