seattlerb/ruby_parser — pull request #111 by whitequark (2013-04-06 12:23:53+00:00)
Title: Replace handwritten lexer with a Ragel-generated one

Yay, it's finally done!

The lexer should be complete and handle all valid 1.9 syntax. It will immediately fix a bunch of open issues: #20, #96, #98, #100, #102, #105, #106 and #108.

The resulting codebase fails five tests, but as you seem to be quite fond of names such as `"no2"` and `test_wtf_8`, I've no idea how (of if) to fix these and I'll just leave this to you.

Feel free to screw with my code, but note that in this current form, it's almost completely diff-compatible with what I have in [parser](http://github.com/whitequark/parser). As long as it remains so, it'll be easy for me to submit pull requests for any bugs found or features implemented (though both cases are quite unlikely at this point).