Merge tokens and grammars in C
Improve tests for state machine transitions