Rust: Change/simplify how the string tokens are handled.

Previously, only the string contents were stored in lexerState::token_str (i.e.
not including the delimeters). Now, the delimeters are stored as well, thus
preserving them. This also simplifies the code a bit.

A new function is added to handle the character storage, which is also now
applied for normal identifiers. To that end, the MAX_STRING_LENGTH was boosted
to 256 so that all reasonably sized identifiers may fit.
Rust: Add a test for the new string lexer.
Rust: Parse character literals.

These were omitted by mistake. Caused bugs like '"' being interpreted as a
start/end of a string, '}' as the end of a block etc.
Rust: Add test for character literals.
Rust: Update comment parsing.

Rust now allows CRLF line endings in source files, which doesn't actually
change any lexing here but the comment was wrong.

Also, the hashbang initial comment has a special case where #![ doesn't count
as a comment (but instead an inner attribute that just happens to be on the
first line).
Rust: Add new comment handling test.
Rust: Update test sources to be valid Rust code.
Rust: Add keyword
Merge pull request #306 from SiegeLord/more_rust_updates