Change:
We know that nodeAffinity/nodeSelector is honored during the calculating of PodTopolgySpread, however, when some existing pods match the incoming pod's topologySpreadConstratints, while also excluded by the nodeAfffinity, things get a bit tricky. Raising this issue to discuss if we should read it as a bug. Here are the repro steps: you have a 3-node cluster, worker1 and worker2 belong to zoneA, and worker3 belongs to zoneB: make a deployment with 3 replicas. Suppose 2 land on worker2, and 1 land on worker. a new deployment with topologySpreadConstraints as well as nodeAffinity (not schedule to worker2): test pod will land on worker3. This behavior concerns me: it looks like the 2 pods on worker2 are taken into consideration, even though worker2 is excluded by nodeAffinity, so the pods on zoneA/zoneB distribute as 2:2. However, shouldn't we disregard the matching pods that are excluded by nodeAffinity? In that case, the test pod should land on worker1 to reach a pods distribution as 1:1. Please share your thoughts. /sig scheduling /kind bug
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
cc/
That is not the issue, the 2 pods on work2 aren't actually being taken into consideration; the problem is similar to what describes. What happened is that zoneA is now not "visible" at all, and so it is as if we have a single zone. The solution should apply to both cases.
I think we should consider it as a bug. Imagine in this situation, all of these pods are labeled with , then we want to apply a like this: is excluded in scheduling. But now when we try to calculate the skew, we will still calculate the pods in . Since : is , then we will schedule the new pod to or , but since they are resource exhausted, then it will be in pending state. But if we filtered out the , the pod will be scheduled to successfully. I think it's the right algorithm.
it's not the same as , even if worker2 becomes invisible, there are still 2 zones:
ah, I didn't notice existed. What part of the code is causing this behavior? I would have assumed that the spreading would happen between and . The following code shows that we should be skipping counting the pods on worker2
it's the that is passed to processNode(): We should pass on a filtered list of nodes to processNode(). included the fix in , and that bits should be extracted out to be a standalone PR, if we get to a consensus on this issue is a bug.
I still don't get where the bug is. I think iterating over all node is the right way and the line I pointed to above should filter out the nodes that don't match the affinity. Passing a filteredList (as in exclude nodes that were filtered by previous filters) is certainly not the right solution because we need to count pods on all nodes. Let alone the filteredlist is different based on how we order the filters, which should never be a factor in the semantics of the PodTopologySpread.
The bug lies in the preFilterState's calculation. For the above case, it's expected to be , but it ended up with . The code I posted above is in PodTopologySpread's PreFitler, not Filter.
ok, this is indeed a bug, we need to check the node affinity constraints again inside the second loop.
actually, doesn't seem efficient that we have two loops in the first place
It seems we are doing it right in the score function, excluding those nodes:
Feel the same. Let's fix the correctness issue first, and then improve the efficiency.
Do we know of a use case where this would be useful? It's kind of odd that there are matching pods in a node where we don't intend to schedule. Maybe during some sort of migration?
It's about clarifying/correcting the semantics of constraints (nodeAffinity, and the to-be- ones like Unschedulable taint) embedded in PodTopologySpread. So IMO we should fix the correctness issue, no matter there is a use case or not.
One thing to be aware of is whether a potential fix could render some pods unschedulable, thus be a breaking change. And that's where use cases weigh in.

Discussion:
TpKeyToCriticalPaths: make(map[string]*criticalPaths, len(constraints)), TpPairToMatchNum:     make(map[topologyPair]*int32, sizeHeuristic(len(allNodes), constraints)), } <ins> // Nodes that pass nodeAffinity check and carry all required topology keys will be // stored in `filteredNodes`, and be looped later to calculate preFilterState. var filteredNodes []*framework.NodeInfo </ins> requiredSchedulingTerm := nodeaffinity.GetRequiredNodeAffinity(pod) for _, n := range allNodes { node := n.Node()

pair := topologyPair{key: c.TopologyKey, value: node.Labels[c.TopologyKey]} s.TpPairToMatchNum[pair] = new(int32) } <ins> filteredNodes = append(filteredNodes, n) </ins> } processNode := func(i int) { <del> nodeInfo := allNodes[i] </del> <ins> nodeInfo := filteredNodes[i] </ins> node := nodeInfo.Node() for _, constraint := range constraints {

atomic.AddInt32(tpCount, int32(count)) } } <del> pl.parallelizer.Until(context.Background(), len(allNodes), processNode) </del> <ins> pl.parallelizer.Until(context.Background(), len(filteredNodes), processNode) </ins> // calculate min match for each topology pair for i := 0; i < len(constraints); i++ {

"node-b": framework.Unschedulable, }, }, <ins> { // In this unit test, NodeAffinity plugin is not involved, so node-b still fits name: "incoming pod has nodeAffinity, pods spread as 0/~2~/0/1, hence node-a fits", pod: st.MakePod().Name("p").Label("foo", ""). NodeAffinityNotIn("node", []string{"node-b"}). SpreadConstraint(1, "zone", v1.DoNotSchedule, st.MakeLabelSelector().Exists("foo").Obj()). Obj(), nodes: []*v1.Node{ st.MakeNode().Name("node-a").Label("zone", "zone1").Label("node", "node-a").Obj(), st.MakeNode().Name("node-b").Label("zone", "zone1").Label("node", "node-b").Obj(), st.MakeNode().Name("node-x").Label("zone", "zone2").Label("node", "node-x").Obj(), st.MakeNode().Name("node-y").Label("zone", "zone2").Label("node", "node-y").Obj(), }, existingPods: []*v1.Pod{ st.MakePod().Name("p-b1").Node("node-b").Label("foo", "").Obj(), st.MakePod().Name("p-b2").Node("node-b").Label("foo", "").Obj(), st.MakePod().Name("p-y1").Node("node-y").Label("foo", "").Obj(), }, wantStatusCode: map[string]framework.Code{ "node-a": framework.Success, "node-b": framework.Success, // in real case, it's Unschedulable "node-x": framework.Unschedulable, "node-y": framework.Unschedulable, }, }, </ins> } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) {