improve robot detection and handling