Change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --It is not a secret that . In my opinion the main reason for the slowness is the flexibility of the inter-pod affinity/anti-affinity rules. However, in most practical scenarios users do not need all the flexibility. For example, people often do not use arbitrary labels in topology key of the affinity rules. They often use node, zone, or region as the topology. Node in particular is used often. That's why I am proposing a fast path for the case that topology key is Node. Today, the affinity/anti-affinity predicate does not have any distinction for topology keys and in the cluster regardless. The logic should recognize "" and scan only pods on the same node for this topology key. Is this a BUG REPORT or FEATURE REQUEST?: FEATURE REQUEST /kind feature /sig scheduling cc/
ref/
/assign
GitHub didn't allow me to assign the following users: misterikkit. Note that only can be assigned. <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc
We found some other simple optimizations that had a significant impact on the affinity predicate. I will upload PRs for those as well.
Reopening this as the performance of anti-affinity still doesn't seem to be satisfactory enough on large clusters (i.e 5k nodes) - and it's causing them to fail to startup. Ref: issue I can help with evaluating any performance improvements if needed.
The change sent improved performance of the predicate, but apparently it is not enough to make our large scale scalability tests happy.
This issue is not about general performance improvment of affinity/anti-affinity. It was to implement a particular improvment which is already implemented. There are other performance improvement issues. I am going to close this one.
Sounds good. Thanks.
How come with kops 1.10.0 I still see all kube-dns pods on the same node because the kube-dns deployment doesn't specify any anti-affinity rule?
I think you are correct. you can double-check with the option to . I have a v1.9 cluster in front of me, and kube-dns has no affinity rules. (I didn't use kops though)

Discussion:
// First return value indicates whether a matching pod exists on a node that matches the topology key, // while the second return value indicates whether a matching pod exists anywhere. // TODO: Do we really need any pod matching, or all pods matching? I think the latter. <del> func (c *PodAffinityChecker) anyPodMatchesPodAffinityTerm(pod *v1.Pod, allPods []*v1.Pod, node *v1.Node, term *v1.PodAffinityTerm) (bool, bool, error) { </del> <ins> func (c *PodAffinityChecker) anyPodMatchesPodAffinityTerm(pod *v1.Pod, pods []*v1.Pod, nodeInfo *schedulercache.NodeInfo, term *v1.PodAffinityTerm) (bool, bool, error) { </ins> if len(term.TopologyKey) == 0 { return false, false, fmt.Errorf("empty topologyKey is not allowed except for PreferredDuringScheduling pod anti-affinity") }

if err != nil { return false, false, err } <del> for _, existingPod := range allPods { </del> <ins> // Special case: When the topological domain is node, we can limit our // search to pods on that node without searching the entire cluster. if term.TopologyKey == kubeletapis.LabelHostname { pods = nodeInfo.Pods() } for _, existingPod := range pods { </ins> match := priorityutil.PodMatchesTermsNamespaceAndSelector(existingPod, namespaces, selector) if match { matchingPodExists = true

if err != nil { return false, matchingPodExists, err } <del> if priorityutil.NodesHaveSameTopologyKey(node, existingPodNode, term.TopologyKey) { </del> <ins> if priorityutil.NodesHaveSameTopologyKey(nodeInfo.Node(), existingPodNode, term.TopologyKey) { </ins> return true, matchingPodExists, nil } }

// Check all affinity terms. for _, term := range GetPodAffinityTerms(affinity.PodAffinity) { <del> termMatches, matchingPodExists, err := c.anyPodMatchesPodAffinityTerm(pod, filteredPods, node, &term) </del> <ins> termMatches, matchingPodExists, err := c.anyPodMatchesPodAffinityTerm(pod, filteredPods, nodeInfo, &term) </ins> if err != nil { errMessage := fmt.Sprintf("Cannot schedule pod %+v onto node %v, because of PodAffinityTerm %v, err: %v", podName(pod), node.Name, term, err) glog.Error(errMessage)

// Check all anti-affinity terms. for _, term := range GetPodAntiAffinityTerms(affinity.PodAntiAffinity) { <del> termMatches, _, err := c.anyPodMatchesPodAffinityTerm(pod, filteredPods, node, &term) </del> <ins> termMatches, _, err := c.anyPodMatchesPodAffinityTerm(pod, filteredPods, nodeInfo, &term) </ins> if err != nil || termMatches { glog.V(10).Infof("Cannot schedule pod %+v onto node %v, because of PodAntiAffinityTerm %v, err: %v", podName(pod), node.Name, term, err)