Don't smash the stack when we run out of reference slots.