Crawl all the links on a page.
Remove then callback from crawler js file. 
It's not crawler's responsibility to call afterSuccessCallback.
Remove the duplicate products.
Merge pull request #1 from nehachaudhary/crawl-all-links

Crawl all links