tokenizer now supports looking at previous match as well, as required by arcance javascript regex syntax