Cleanup old shuffle file metadata from memory