Core for crawling sites with some info