Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: For each NLB created, K8s creates an ingress rule on the node group's security group for each port exposed for each CIDR attached to the VPC to allow health checks to succeed. AWS limits the number of rules per security group to 60 by default. This quota is adjustable but multiplied by the quota for security groups per network interface cannot exceed 1000. I have a VPC with 16 CIDR blocks attached to it and a quota for rules per security group set to 80. Each NLB created adds 18 new rules (16 for Health Check, 1 for Client, and 1 for MTU) to the node group's security groups. This high number of unnecessary rules created makes it impossible to create more than 4 NLBs per node group (if each one has a single port exposed). What you expected to happen: For the health check to succeed, the NLB must be able to ping the node port for the service on the cluster's node group. So the node group's security group just needs an inbound rule for each ENI of the NLB, making the number of rules necessary on the cluster's node group directly proportional to the number of AZs the NLB is configured. How to reproduce it (as minimally and precisely as possible): Create a VPC on AWS with 8 CIDRs attached to it; Create a K8s Cluster; Create a Network Load Balancer with 6 ports exposed; If the default quota of rules per security group has not been changed, the NLB creation will fail with RulesPerSecurityGroupLimitExceeded. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: OS (e.g: ):Kernel (e.g. ):Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig cloud-provider
/area provider/aws

Related discussion:
text: return targetGroup, nil } <del> func (c *Cloud) getVpcCidrBlocks() ([]string, error) { vpcs, err := c.ec2.DescribeVpcs(&ec2.DescribeVpcsInput{ VpcIds: []*string{aws.String(c.vpcID)}, }) if err != nil { return nil, fmt.Errorf("error querying VPC for ELB: %q", err) } if len(vpcs.Vpcs) != 1 { return nil, fmt.Errorf("error querying VPC for ELB, got %d vpcs for %s", len(vpcs.Vpcs), c.vpcID) } cidrBlocks := make([]string, 0, len(vpcs.Vpcs[0].CidrBlockAssociationSet)) for _, cidr := range vpcs.Vpcs[0].CidrBlockAssociationSet { if aws.StringValue(cidr.CidrBlockState.State) != ec2.VpcCidrBlockStateCodeAssociated { continue } cidrBlocks = append(cidrBlocks, aws.StringValue(cidr.CidrBlock)) } return cidrBlocks, nil } </del> // updateInstanceSecurityGroupsForNLB will adjust securityGroup's settings to allow inbound traffic into instances from clientCIDRs and portMappings. // TIP: if either instances or clientCIDRs or portMappings are nil, then the securityGroup rules for lbName are cleared. <del> func (c *Cloud) updateInstanceSecurityGroupsForNLB(lbName string, instances map[InstanceID]*ec2.Instance, clientCIDRs []string, portMappings []nlbPortMapping) error { </del> <ins> func (c *Cloud) updateInstanceSecurityGroupsForNLB(lbName string, instances map[InstanceID]*ec2.Instance, subnetCIDRs []string, clientCIDRs []string, portMappings []nlbPortMapping) error { </ins> if c.cfg.Global.DisableSecurityGroupIngress { return nil }