Token Scanner Passes all 30 Test Cases

bugs have been squashed and now all 30 test cases pass. more comments
have been added.