Added test class for grammar; added identifyToken method.