Cleanup, and also track server connection closure to avoid connection limits