Don't try to close the protobuf server if it's not open

Fix #584