Change:
What happened: When a pod can't be scheduled and the cardinality of taints on nodes is high, the Note field in the event can become bigger than the 1024 allowed and the event isn't sent. The pod stays in a state of without en event which makes the User Experience very bad What you expected to happen: I would expect the Kube-scheduler to be able to send the event no matter how many different taints are on nodes How to reproduce it (as minimally and precisely as possible): add many different taints to nodes and then trying to schedule something which can't be successfully scheduled (higher CPU/memory than any node has) Anything else we need to know?: In this case the length of the message was 1334 characters. Environment: Kubernetes version (use ): 1.18.9 Cloud provider or hardware configuration: bare metal Possible solutions: make event size configurable, it's now hardcoded to 1024 characters make scheduler note aware of that limit and work around I, if length 1024 then truncate the string (so at least something is shown) produce N events splitting the text (preferred one): change event Note to be to something like: (the whole complete information would still be present in the pod status field. for debugging purposes)
/sig scheduling
/triage accepted /cc
I could provide the fix in case we define which solution is best
This is a consequence of I think we can just drop some reasons when we are building the status message if the message is too large. This comes from here /help
/remove-help /assign
how many different taints did you have? I think in all cases we should truncate the message as well:
Another remidiation is to supress outoutping nil fields (such as GenerateName:"", SelfLink:"", UID:"", ResourceVersion:"", etc.) in the EventRecorder side.
The size limit we're hitting is specifically on the field
Completely agreed
good issue~
/assign
/unassign

Discussion:
visibility = ["//visibility:public"], deps = [ "//pkg/api/v1/pod:go_default_library", <ins> "//pkg/apis/core/validation:go_default_library", </ins> "//pkg/scheduler/algorithmprovider:go_default_library", "//pkg/scheduler/apis/config:go_default_library", "//pkg/scheduler/apis/config/scheme:go_default_library",

"k8s.io/client-go/tools/cache" "k8s.io/klog/v2" podutil "k8s.io/kubernetes/pkg/api/v1/pod" <ins> "k8s.io/kubernetes/pkg/apis/core/validation" </ins> schedulerapi "k8s.io/kubernetes/pkg/scheduler/apis/config" "k8s.io/kubernetes/pkg/scheduler/apis/config/scheme" "k8s.io/kubernetes/pkg/scheduler/core"

} pod := podInfo.Pod <del> fwk.EventRecorder().Eventf(pod, nil, v1.EventTypeWarning, "FailedScheduling", "Scheduling", err.Error()) </del> <ins> msg := truncateMessage(err.Error()) fwk.EventRecorder().Eventf(pod, nil, v1.EventTypeWarning, "FailedScheduling", "Scheduling", msg) </ins> if err := updatePod(sched.client, pod, &v1.PodCondition{ Type:    v1.PodScheduled, Status:  v1.ConditionFalse,

} } <ins> // truncateMessage truncates a message if it hits the NoteLengthLimit. func truncateMessage(message string) string { max := validation.NoteLengthLimit if len(message) <= max { return message } suffix := " ..." return message[:max-len(suffix)] + suffix } </ins> func updatePod(client clientset.Interface, pod *v1.Pod, condition *v1.PodCondition, nominatedNode string) error { klog.V(3).InfoS("Updating pod condition", "pod", klog.KObj(pod), "conditionType", condition.Type, "conditionStatus", condition.Status, "conditionReason", condition.Reason) podCopy := pod.DeepCopy()