beelsebob/CoreParse — pull request #30 by siuying (2014-01-22 05:03:55+00:00)
Title: Add CPRegexpKeywordRecogniser tokenizer: a tokenizer that accept a NSRegularExpressionand return a CPKeywordToken.

This recognizer allow us to define a keyword by using a regular expression.

For example, following Tokenizer:
```
[tokenizer addTokenRecogniser:[CPIdentifierRecogniser identifierRecogniser]];
[tokenizer addTokenRecogniser:[CPRegexpKeywordRecogniser recogniserForRegexp:[NSRegularExpression regularExpressionWithPattern:@"[ \t\r\n\f]*(>)[ \t\r\n\f]*" options:0 error:nil]]];
[tokenizer addTokenRecogniser:[CPWhiteSpaceRecogniser whiteSpaceRecogniser]];
```

will match ``table ~ a`` as `` ``. This is much more powerful than standard keyword recognizer and can be handy in some use case (such as trimming spaces in token)