Takes a string of characters and turns it into a string of tokens.