Change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: healthcheck nodeport allocation doesn't have roll-back, this may lead to nodeport leak. What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: /sig network /assign Environment: Kubernetes version (use ): Cloud provider or hardware configuration**: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig network /assign
Could you clarity "roll-back"? It seems that we basically in the same way.
I didn't mean the HealthCheck NodePort release process in . I meant the allocation process in and . Take for example, please see: There is a defer for common NodePorts in : Once fails, the above defer will finally exec. Then the allocated NodePorts will roll-back, i.e. these NodePorts will be released. But the HealthCheck NodePort allocation process doesn't use this roll-back. This will lead to leak. I run a simple test and see the HealthCheck NodePort leak. Will paste the result later.
Testing service manifest, . some simple testing code. See: . Note that I add this error intentionally: kube-apiserver with flag . . We will get below output. This is expected. again. We will get below output. Note the error . That means that when creating service fails at step 3, the NodePort and HealthCheck NodePort should be both released, but the was NOT.
Fortunately, several minute later, the leak was repaired and cleaned up. But anyway, we should release the HealthCheck NodePort when fails. We shouldn't depend on the later repair.
Thanks for tracing down, sent for fixing.
We're having this issue on service deletion. It looks like healthcheck nodeport is not deallocated immediately when deleting the service. This is causing issues for backup and restore of services with heathcheck node port.

Discussion:
// Handle ExternalTraiffc related fields during service creation. if utilfeature.DefaultFeatureGate.Enabled(features.ExternalTrafficLocalOnly) { if apiservice.NeedsHealthCheck(service) { <del> if err := rs.allocateHealthCheckNodePort(service); err != nil { </del> <ins> if err := rs.allocateHealthCheckNodePort(service, nodePortOp); err != nil { </ins> return nil, errors.NewInternalError(err) } }

// healthCheckNodePortUpdate handles HealthCheckNodePort allocation/release // and adjusts HealthCheckNodePort during service update if needed. <del> func (rs *REST) healthCheckNodePortUpdate(oldService, service *api.Service) (bool, error) { </del> <ins> func (rs *REST) healthCheckNodePortUpdate(oldService, service *api.Service, nodePortOp *portallocator.PortAllocationOperation) (bool, error) { </ins> neededHealthCheckNodePort := apiservice.NeedsHealthCheck(oldService) oldHealthCheckNodePort := oldService.Spec.HealthCheckNodePort

// Insert health check node port into the service's HealthCheckNodePort field if needed. case !neededHealthCheckNodePort && needsHealthCheckNodePort: glog.Infof("Transition to LoadBalancer type service with ExternalTrafficPolicy=Local") <del> if err := rs.allocateHealthCheckNodePort(service); err != nil { </del> <ins> if err := rs.allocateHealthCheckNodePort(service, nodePortOp); err != nil { </ins> return false, errors.NewInternalError(err) }

// Free the existing healthCheckNodePort and clear the HealthCheckNodePort field. case neededHealthCheckNodePort && !needsHealthCheckNodePort: glog.Infof("Transition to non LoadBalancer type service or LoadBalancer type service with ExternalTrafficPolicy=Global") <del> err := rs.serviceNodePorts.Release(int(oldHealthCheckNodePort)) if err != nil { glog.Warningf("error releasing service health check %s node port %d: %v", service.Name, oldHealthCheckNodePort, err) return false, errors.NewInternalError(fmt.Errorf("failed to free health check nodePort: %v", err)) } glog.Infof("Freed health check nodePort: %d", oldHealthCheckNodePort) </del> <ins> glog.V(4).Infof("Releasing healthCheckNodePort: %d", oldHealthCheckNodePort) nodePortOp.ReleaseDeferred(int(oldHealthCheckNodePort)) </ins> // Clear the HealthCheckNodePort field. service.Spec.HealthCheckNodePort = 0

// Handle ExternalTraiffc related updates. if utilfeature.DefaultFeatureGate.Enabled(features.ExternalTrafficLocalOnly) { <del> success, err := rs.healthCheckNodePortUpdate(oldService, service) </del> <ins> success, err := rs.healthCheckNodePortUpdate(oldService, service, nodePortOp) </ins> if !success || err != nil { return nil, false, err }

} // allocateHealthCheckNodePort allocates health check node port to service. <del> func (rs *REST) allocateHealthCheckNodePort(service *api.Service) error { </del> <ins> func (rs *REST) allocateHealthCheckNodePort(service *api.Service, nodePortOp *portallocator.PortAllocationOperation) error { </ins> healthCheckNodePort := service.Spec.HealthCheckNodePort if healthCheckNodePort != 0 { // If the request has a health check nodePort in mind, attempt to reserve it. <del> err := rs.serviceNodePorts.Allocate(int(healthCheckNodePort)) </del> <ins> err := nodePortOp.Allocate(int(healthCheckNodePort)) </ins> if err != nil { return fmt.Errorf("failed to allocate requested HealthCheck NodePort %v: %v", healthCheckNodePort, err) } <del> glog.Infof("Reserved user requested nodePort: %d", healthCheckNodePort) </del> <ins> glog.V(4).Infof("Reserved user requested healthCheckNodePort: %d", healthCheckNodePort) </ins> } else { // If the request has no health check nodePort specified, allocate any. <del> healthCheckNodePort, err := rs.serviceNodePorts.AllocateNext() </del> <ins> healthCheckNodePort, err := nodePortOp.AllocateNext() </ins> if err != nil { return fmt.Errorf("failed to allocate a HealthCheck NodePort %v: %v", healthCheckNodePort, err) } service.Spec.HealthCheckNodePort = int32(healthCheckNodePort) <del> glog.Infof("Reserved allocated nodePort: %d", healthCheckNodePort) </del> <ins> glog.V(4).Infof("Reserved allocated healthCheckNodePort: %d", healthCheckNodePort) </ins> } return nil }