refactor and fixes for newline token