tenderlove/rkelly — pull request #14 by sasmith (2012-10-03 02:14:54+00:00)
Title: Updates tokenizer to correctly parse `/*comment*/index`

The tokenizer looks for the longest string it can match, and breaks ties based on COMMENTS > ... > REGEXP.

According to its matching rules, though `/*comment*/index` would make an eleven character comment (`/*comment*/`) or a twelve character regexp (`/*comment*/i`).

This change requires that the meat of regular expressions not start with a `*` so as to fix this problem.