Clean up file/directory handles after closing