Close up listening servers after the tests

If we open up a ephemeral port to test on, at the end of the test we
should close it up.
Merge commit 'e1ebe92b284039d053c4083d71a688c20c77fea9'