Adjusted tokenizer to ignore whitespace-only lines, fixed tests