Crawl all links

crawler file should not handle the afterSucceccCallback. move it to app.js.
Remove duplicate products.