1. Add retry to url download function
2. Add crawling of all pages