process while crawl + queries for clustering and plots