simplified a bunch of stuff and moved some code from the scanner to the token