subcategory crawler now also saves the list
link_extraction now removes all links that are not from an allowed category