Stop searching for an empty reference slot once we've found one. We don't need to fill it from the back, and if we did, this would still be inefficient.