Handle eof specially:

1. When found '\0', chech if real end of file has been reached.
2. When saving eof token, do not check for the end of token array