A new approach to implement the function next ()

Use Finite-state machine. Update state. Fix functions determine the type
of the token.