The scanner now scans the given string part by part, finding only one token as a time. This is a correction over scanning the entire document and returning a list of tokens.