Add closed flag, do not leak FDs for closed servers anymore