Update of Tokenizer

Tokenizer has been updated to better handle the splitting of tokens. It
still does not have an output, however it no longer has a trailing \n
and handles < > |, \t and “ “ tokens.

Other changes:
- Removed C++ commenting (//)
- Added more printf() statements to track user entered information.
- Exit no longer checks for line break.