Fixes #6255 - removed regex for token parsing

It appears that the regex expression is unreliable, I have added a token parsing mechanism in it's place.  Please let me know if there are any test cases that we should add to this.
@steverhoades any plans for this?