added reuiqrements, now using green threads instead of separate processes for scraping