added Tokenize function, fixed Scan algorithm, updated id pattern and added eof... -sai