Change:
} actualIDs := []string{} for _, healthDescription := range healthResponse.TargetHealthDescriptions { <del> if healthDescription.TargetHealth.Reason != nil { </del> <ins> if aws.StringValue(healthDescription.TargetHealth.State) == elbv2.TargetHealthStateEnumHealthy { actualIDs = append(actualIDs, *healthDescription.Target.Id) } else if healthDescription.TargetHealth.Reason != nil { </ins> switch aws.StringValue(healthDescription.TargetHealth.Reason) { case elbv2.TargetHealthReasonEnumTargetDeregistrationInProgress: // We don't need to count this instance in service if it is

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Using a pod like the nginx-ing-cntlr with an AWS Network Load Balancer (NLB), and then draining the node its running on to force a migration to another node pool, does not reliably deregister the drained nodes from the NLB Target group. This causes the nodeport used by the NLB to remain open, and the drained node ends up continuing to funnel traffic for other pods cause of being used. This leads to connection timeouts for clients, and downtime on migrations. Switching to a Classic ELB, instead of an NLB, with connection draining enabled does work, and the instances that are drained are practically removed immediately from the instances targeted by the ELB, as expected. There seems to be some sort of issue with NLB's as this is not happening, or happening rarely (only saw it once in many tests but couldn't recreate it). The node + LB set seems to be working as I see in the controller-mgr logs that the nodes are updated as I went from 11 nodes to 8, after attempting to drain + migrate a nodepool of 3 nodes with nginx-ing-cntlr all running on them: However, this update does not seem to be translating into deregistering instances from the NLB. And i've waited minutes & hours in case timeouts were playing a hand, but that didn't seem to help. What you expected to happen: On node draining, with being used, that the NLB would deregister the instances as expected. How to reproduce it (as minimally and precisely as possible): Launch a Deployment + NLB service on a node and drain it to force it over to another node pool. Watch the instances in the AWS NLB Target group never get out of "healthy" status or be drained + removed. Anything else we need to know?: Environment: AWS EKS Kubernetes version (use ): 1.12 & 1.13 Cloud provider or hardware configuration: AWS OS (e.g: ): AWS Linux 2 Kernel (e.g.

): 4.14.123-111.109.amzn2.x8664 Install tools: Network plugin and version (if this is a network-related bug): core-dns v1.2.6 Others:
/sig aws
Related:
/sig cloud-provider
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details