Make lexer support multibyte and produce tokens with offset.

Multibyte is now supported by using [:blank:] character class
when the ruby version in use supports this. (It matches all Unicode
blank characters except CR LF). 

No other character classes are used when matching strings, which will
allow Unicode in strings (but not in bare words).

Lexer now also computes total offset per token, and offset from last
seen linebreak (or 0 if on the first line). These values are available
as :offset, and :pos in the token.
Correct precedence of '?' and recognize virtualized resource defaults.

The precedence of the '?' operator was wrong (only allowing
function calls and primary expressions as LHS).

Virtualized Resource Defaults are illegal, but needs to be recognized
and flagged as such rather than syntax errors.
Fix optional endcomma missing in function call expression

This was simply missing in the function call rules.
Add tests for optional endcomma in function calls.
Correct issue with optional lambda.

This is a follow on problem caused by adding optional endcomma
in argument list. The lambda setting was used with the value from
the wrong parse rule.
Checker did not check root.

Also fixes checker being fed a partial result wrapped in a factory
instance.
Make compiler unit test use ParserFactory to create parser.

The compiler test is not testing the parser and should use
the configured parser dictated by the ParserFactory.
Increase coverage of pops parser/transformation tests.
Modify intergation parser test to use ParserFactory.

This test should test the result of parsing, not unit test one
particular parser implementation.
Improve location processing (among other things adding line pos).

This contains a new feature; keeping track of the line position
to enable better reporting for syntax errors (e.g "line 13:3", 
vs just "line 13"). This is helpful to a user since a particular
punctuation character may appear many times on a line.

The puppet/parser classes have been updated to at least accept
a pos. (The "old" lexer does not produce pos information, and if
pos is nil, it is ignored later).

The egrammar implementation is refactored, and instead of extracting
line information separately, it simply passes on an instance of
the SourcePosAdapter which has all the required information.

The egrammar lexer computes offsets, and positions. It stores them
in the lexing_context until asked for a detailed position.

The lexer needs to be multibyte aware when computing pos, and it is now
updated to use multibyte regular expressions for whitespace (as a side
effect). Multibyte is supported for Ruby 1.9.3 and later.
Add unit test for eparser_adapter.

This is basically a copy of the parser unit test, but with removal
of tests that only check the parser implementation (internal calls
to "self" from the old parser).

There are also minor adjustments as the egrammar parser has better
information (e.g. the result contains "file" instead of "File" as type
is already detected and type internally has already done a downcase).