encoding the content crawlers