Fix an edge case that caused the bug in #475 to sometimes reoccur