added support for reddit API to get top 100 urls to crawl