Various improvements and additions in crawling, UI, settings