Fail if we start at an impossible location.