Include preprocessor added

Also cleaned up the tokenizer a bit.