re-added crawlers and automation