macros for space and lf
tokenizer tests - same as in the prolog code
tests for unicode words