Tokenizes a string that represents a regular expression.