Abstracts tokenizer, adds tmp directory