Change:
What happened: Service Load balancer may not be cleaned up on type change if below sequence happen: type changes from to controller restarts/crashes before it gets a chance to process the update from 1. controller starts up again and sees the service doesn't want load balancer, hence skips it. load balancer resource sticks around even if it is no longer needed by the service. Note that this is a different issue than , as load balancer resource will still be cleaned up when the service object is deleted, just not when the type change happens. Anything else we need to know?: This is a corner case that is not covered by the initial service load balancer finalizer implementation (). This can probably be fixed in a trivial way --- make sure the handler takes in service with non-LoadBalancer type but has finalizer attached: I will put more thoughts later. cc
/sig network
Can i reserve this issue in advance? :) I'd like to take this issue as my first issue.
Sure thing, feel free to work on this and glad that you can help :)
Do you mean attach finalizer for service that non-LoadBalancer type in AddFunc? I'm new to kubernetes, hope you can guide me. :)
No worries. I actually meant should include service with non-LoadBalancer type && with finalizer attached. As we only attach finalizer to a service only when we are creating a load balancer for it, a finalizer presents on the service (no matter what type it has) indicates there is a load balancer associated. Hence it makes sense that we should clean up the load balancer if the service no longer wants one, even if the service has not been deleted.
To clarify further, "attach finalizer for service that non-LoadBalancer type in AddFunc" would not be what we want --- we shouldn't attach finalizer to service that we will not be creating load balancer for.
After go through service controller part code, i guess you mean add a condition for as follows: . Am i right? I'm not sure when service type changes if can be involved. Can you clarify this ?
if ok && (wantsLoadBalancer(svc) (svc.Spec.Type != v1.ServiceTypeLoadBalancer && servicehelper.HasLBFinalizer(svc))) . Yea this was what I meant. On service type change, will be involved instead of . That part is already handled. This issue is explicit for the case when controller fails to handle the type change update before it exiting (e.g. crashing, restart, etc.). Putting such detection to is more off a remedy, it means the next time service controller starts up, it can re-syncs all existing services and find out it still need to cleanup load balancer for this service, even if it doesn't have load balancer type anymore.
Thanks . I have sent a PR assined to you. I think it should be checked in function . It is the case that need to clean up whatever the reason is.

Discussion:
cache.ResourceEventHandlerFuncs{ AddFunc: func(cur interface{}) { svc, ok := cur.(*v1.Service) <ins> // Check cleanup here can provide a remedy when controller failed to handle // changes before it exiting (e.g. crashing, restart, etc.). </ins> if ok && (wantsLoadBalancer(svc) || needsCleanup(svc)) { s.enqueueService(cur) }

// needsCleanup checks if load balancer needs to be cleaned up as indicated by finalizer. func needsCleanup(service *v1.Service) bool { <del> return service.ObjectMeta.DeletionTimestamp != nil && servicehelper.HasLBFinalizer(service) </del> <ins> if !servicehelper.HasLBFinalizer(service) { return false } if service.ObjectMeta.DeletionTimestamp != nil { return true } // Service doesn't want loadBalancer but owns loadBalancer finalizer also need to be cleaned up. if service.Spec.Type != v1.ServiceTypeLoadBalancer { return true } return false </ins> } // needsUpdate checks if load balancer needs to be updated due to change in attributes.

} <ins> // Test cases: // index    finalizer    timestamp    wantLB  |  clean-up //   0         0           0            0     |   false    (No finalizer, no clean up) //   1         0           0            1     |   false    (Ignored as same with case 0) //   2         0           1            0     |   false    (Ignored as same with case 0) //   3         0           1            1     |   false    (Ignored as same with case 0) //   4         1           0            0     |   true //   5         1           0            1     |   false //   6         1           1            0     |   true    (Service is deleted, needs clean up) //   7         1           1            1     |   true    (Ignored as same with case 6) </ins> func TestNeedsCleanup(t *testing.T) { testCases := []struct { desc               string

expectNeedsCleanup bool }{ { <del> desc:               "service without finalizer without timestamp", </del> <ins> desc:               "service without finalizer", </ins> svc:                &v1.Service{}, expectNeedsCleanup: false, }, { <del> desc: "service without finalizer with timestamp", </del> <ins> desc: "service with finalizer without timestamp without LB", </ins> svc: &v1.Service{ ObjectMeta: metav1.ObjectMeta{ <del> DeletionTimestamp: &metav1.Time{ Time: time.Now(), }, </del> <ins> Finalizers: []string{servicehelper.LoadBalancerCleanupFinalizer}, }, Spec: v1.ServiceSpec{ Type: v1.ServiceTypeNodePort, </ins> }, }, <del> expectNeedsCleanup: false, </del> <ins> expectNeedsCleanup: true, </ins> }, { <del> desc: "service with finalizer without timestamp", </del> <ins> desc: "service with finalizer without timestamp with LB", </ins> svc: &v1.Service{ ObjectMeta: metav1.ObjectMeta{ Finalizers: []string{servicehelper.LoadBalancerCleanupFinalizer}, }, <ins> Spec: v1.ServiceSpec{ Type: v1.ServiceTypeLoadBalancer, }, </ins> }, expectNeedsCleanup: false, },

desc: "service with finalizer with timestamp", svc: &v1.Service{ ObjectMeta: metav1.ObjectMeta{ <ins> Finalizers: []string{servicehelper.LoadBalancerCleanupFinalizer}, </ins> DeletionTimestamp: &metav1.Time{ Time: time.Now(), }, <del> Finalizers: []string{servicehelper.LoadBalancerCleanupFinalizer, "unrelated"}, </del> }, }, expectNeedsCleanup: true,