Change the "reader" implementation to be reentrant

The non reentrant approach was a source of problem with nested parsing
that happened when reading Lua interfaces (ilua files).
Fix error with EOF token in the run-lexer application
Fix file reader to close the file when finished