Fix resource leaks on exception paths

Make it clear that the server socket isn't a leaked resource