Changed parser semantics to only consume tokens when necessary.