Don't fail over to next backends when not at top-level

That's counter-intuitive.