added crawler capabilities