Add THREADS parameter to tests script
Change ne symbol to avoid conflicts with the ! sign
Finalize tokenizer