add testing for pattern matching.
add method in TextTokenStream to check if it has errors