flatten improved to smash things into the same hierarchy as the source directories
The offsets file now indexes into the scleaner TREC corpus.
The offsets for the original TREC doc layout, picked straight from the five disks.