Handle arrows at end of file.
Reduce cyclomatic complexity.
Ignore generated tokens for detecting eof.
Merge pull request #149 from pathable/arrow-eof

Handle arrows at end of file.