I wanted to propose a different strategy which is very similar to how he
taught us last year.  This gives us a LOT of control over what tokens
are valid and invalid.  I've written WordToken and NumberToken to
demonstrate this strategy.  The Source.java file allows us to peek one
character ahead before consuming it.
Use Test.java and inputjason.txt to see some results