adds method to kick off crawler plus minor fixes