tweaks to data processing