utilities for getting tokenized text out of squoia's preprocessed files