attempt to allow look-behind assertions in tokens

Instead of consuming matched text, offset is increased
and regex matching is tested from offset.