Araq/Nimrod — pull request #962 by barcharcraz (2014-02-27 03:27:00+00:00)
Title: added another version of initGeneralTokenizer

added a version of initGeneralTokenizer that accepts a cstring instead of a string. Also made the old version use this function in its implementation.

I use this when calling the highlighter from C#.