updated data harvesting process