Free up file handles in workers