finished minicrawler for now