Change:
/kind bug What happened: Since upgrading to Kube 1.7.6, I'm seeing my HPAs scale above it's maxReplicas. Because of this, it's rapidly creating pods, then terminating them, and repeat. What you expected to happen: I expected the HPA to never exceed maxReplicas. How to reproduce it (as minimally and precisely as possible): The HPA is logging: This seems to happen with the deployment's is low, and the HPA shows the deployment using more than 175% of it's If I increase the deployment's to better match actual usage (so that the deployment is using 120% or less), it occurs much less often. I deliberately set the low, to ensure services were over provisioned during the first roll out of HPA. I think the switch in after should be converted to individual if() statements. Anything else we need to know?: I am only setting on the deployment, not Looking at the code in , the version that worked (1.6.2) would only scale up and down by 1. The 1.7.6+ code is attempting to scale up by a larger amount. The new code seems appropriate in large clusters, where maxReplicas is in the tens or hundreds. It doesn't seem appropriate for clusters where maxReplicas is in the single digits. Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-07-21T19:06:19Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.6", GitCommit:"", GitTreeState:"clean", BuildDate:"2017-09-14T06:36:08Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration**: AWS m4.2xlarge, 5 nodes OS (e.g. from /etc/os-release):Kernel (e.g. ):Install tools: kops Version 1.7.0 Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
Yeah, I think the bug is that we have a case short-circuiting another case.
I think your right - we are not checking whether . I submitted a diff with a fix and regression test.

Discussion:
case desiredReplicas > scaleUpLimit: setCondition(hpa, autoscalingv2.ScalingLimited, v1.ConditionTrue, "ScaleUpLimit", "the desired replica count is increasing faster than the maximum scale rate") desiredReplicas = scaleUpLimit <ins> // Ensure that even if the scaleUpLimit is greater // than the maximum number of replicas, we only // set the max number of replicas as desired. if scaleUpLimit > hpa.Spec.MaxReplicas { setCondition(hpa, autoscalingv2.ScalingLimited, v1.ConditionTrue, "TooManyReplicas", "the desired replica count was more than the maximum replica count") desiredReplicas = hpa.Spec.MaxReplicas } </ins> case hpa.Spec.MinReplicas != nil && desiredReplicas < *hpa.Spec.MinReplicas: // make sure we aren't below our minimum var statusMsg string

tc.runTest(t) } <ins> func TestUpscaleCapGreaterThanMaxReplicas(t *testing.T) { tc := testCase{ minReplicas:     1, maxReplicas:     20, initialReplicas: 3, // desiredReplicas would be 24 without maxReplicas desiredReplicas:     20, CPUTarget:           10, reportedLevels:      []uint64{100, 200, 300}, reportedCPURequests: []resource.Quantity{resource.MustParse("0.1"), resource.MustParse("0.1"), resource.MustParse("0.1")}, useMetricsAPI:       true, expectedConditions: statusOkWithOverrides(autoscalingv2.HorizontalPodAutoscalerCondition{ Type:   autoscalingv2.ScalingLimited, Status: v1.ConditionTrue, Reason: "ScaleUpLimit", }, autoscalingv2.HorizontalPodAutoscalerCondition{ Type:   autoscalingv2.ScalingLimited, Status: v1.ConditionTrue, Reason: "TooManyReplicas", }), } tc.runTest(t) } </ins> func TestConditionInvalidSelectorMissing(t *testing.T) { tc := testCase{ minReplicas:         1,