Perform socket close in all cases.

This avoids "leaking" the file descriptors.