make some improvements to data processing