Add corpus-wide tokenization.
Update comments.