Change:
/kind flake /sig apps This is an old flaking since , but the flaky rate is not that high. The rate in recent runs in is about 3/23 ~ 13%.
/priority backlog as the failure rate now is not high /cc old threads in and
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /priority backlog as the failure rate now is not high /cc old threads in and Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/priority backlog
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten still flaking
/triage accepted /assign

Discussion:
nodes, err := c.CoreV1().Nodes().List(ctx, metav1.ListOptions{}) framework.ExpectNoError(err) nodeCount := len(nodes.Items) <del> retryTimeout := dsRetryTimeout + time.Duration(nodeCount*30)*time.Second </del> <ins> // We disturb daemonset progress by randomly terminating pods. randomPodTerminationTimeout := 5 * time.Minute retryTimeout := dsRetryTimeout + randomPodTerminationTimeout + time.Duration(nodeCount*30)*time.Second </ins> ginkgo.By("Check that daemon pods surge and invariants are preserved during that rollout") <del> ageOfOldPod := make(map[string]time.Time) </del> <ins> nodeToAgeOfOldPod := make(map[string]map[string]time.Time) </ins> deliberatelyDeletedPods := sets.NewString() err = wait.PollUntilContextTimeout(ctx, dsRetryPeriod, retryTimeout, true, func(ctx context.Context) (bool, error) { podList, err := c.CoreV1().Pods(ds.Namespace).List(ctx, metav1.ListOptions{})

// if this is a pod in an older version AND there is a new version of this pod, record when // we started seeing this, otherwise delete the record (perhaps the node was drained) if nodesToVersions[pod.Spec.NodeName][newVersion] > 0 { <del> if _, ok := ageOfOldPod[string(pod.UID)]; !ok { ageOfOldPod[string(pod.UID)] = now </del> <ins> if _, ok := nodeToAgeOfOldPod[pod.Spec.NodeName][string(pod.UID)]; !ok { if _, ok := nodeToAgeOfOldPod[pod.Spec.NodeName]; !ok { nodeToAgeOfOldPod[pod.Spec.NodeName] = make(map[string]time.Time) } nodeToAgeOfOldPod[pod.Spec.NodeName][string(pod.UID)] = now </ins> } } else { <del> delete(ageOfOldPod, string(pod.UID)) </del> <ins> delete(nodeToAgeOfOldPod, pod.Spec.NodeName) </ins> } } // purge the old pods list of any deleted pods <del> for uid := range ageOfOldPod { if !podUIDs.Has(uid) { delete(ageOfOldPod, uid) </del> <ins> for node, uidToTime := range nodeToAgeOfOldPod { for uid := range uidToTime { if !podUIDs.Has(uid) { delete(uidToTime, uid) } } if len(uidToTime) == 0 { delete(nodeToAgeOfOldPod, node) </ins> } } deliberatelyDeletedPods = deliberatelyDeletedPods.Intersection(deletedPodUIDs)

} // invariant: the controller must react to the new pod becoming ready within a reasonable timeframe (2x grace period) <del> for uid, firstSeen := range ageOfOldPod { if now.Sub(firstSeen) > maxSurgeOverlap { errs = append(errs, fmt.Sprintf("An old pod with UID %s has been running alongside a newer version for longer than %s", uid, maxSurgeOverlap)) </del> <ins> for node, uidToTime := range nodeToAgeOfOldPod { for uid, firstSeenSinceNewVersionPod := range uidToTime { if now.Sub(firstSeenSinceNewVersionPod) > maxSurgeOverlap { errs = append(errs, fmt.Sprintf("An old pod with UID %s on a node %s has been running alongside a newer version for longer than %s", uid, node, maxSurgeOverlap)) } </ins> } }

} else { framework.Logf("Deleted pod %s prematurely", pod.Name) deliberatelyDeletedPods.Insert(string(pod.UID)) <ins> // If it is an old version we do not need to measure the controller reaction because we have done it instead. // If it is a new version, we have to reset the time to start counting the time for the replacement pod to reach readiness again. delete(nodeToAgeOfOldPod, pod.Spec.NodeName) </ins> } } }