somw more work on the feedback function
some optimization to crawl.py