Implemented token class, cleaned up tokenizer, rewrote some logic in create_command_stream.