Revert "Check for an available slave early on"

fastgm from fishcooking spotted a regression in performance after this patch.