write batches in separate files, then consolidate at the end
minor