also store all raw data when re-processing
try to reduce reprocessing memory use