further refining of data processing
added code comments