add data mining logic to TODO and update progess