avoid reading in full corpus when its tokenized form is cached