Added support for extra tokens during lexicographic analysis.

Tokens currently supported are identifiers, int contstants, float constants
without exponent, string literals, +, -, *, /, >, <, >=, <=, ==, !=, &&, ||,
!, =, {, }, (, and ).
Fixed a slight mistake when trying to identify || operator.