chriswailes/RLTK — pull request #22 by jamesotron (2013-03-14 21:46:36+00:00)
Title: Enable actions to return more than one token.

Hi Chris.

I'm not sure if you're interested in this, but I wanted to make Lexer actions able to return more than one token.

The language I am parsing is whitespace aware, and I wanted to emit more than one outdent token if required, eg

```
foo
bar
baz
```

should return `:VAR, :INDENT, :VAR, :INDENT, :VAR, :OUTDENT, :OUTDENT` from the lexer.

I did this by patching the lexer to allow the action to return an array of arrays, so instead of
`[ :OUTDENT, 1 ]` it can return `[[ :OUTDENT 1 ], [ :OUTDENT, 0 ]]`.

Feel free to merge if you think it's useful.