jiebi/CodeConvo huggingface dataset:
Code change:
// or the kubeadm preferred one for the desired Kubernetes version var desiredEtcdVersion *version.Version if cfg.Etcd.Local.ImageTag != "" { <del> desiredEtcdVersion, err = version.ParseSemantic(cfg.Etcd.Local.ImageTag) </del> <ins> desiredEtcdVersion, err = version.ParseSemantic( convertImageTagMetadataToSemver(cfg.Etcd.Local.ImageTag)) </ins> if err != nil { return true, errors.Wrapf(err, "failed to parse tag %q as a semantic version", cfg.Etcd.Local.ImageTag) }

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --When attempting to upgrade kubernetes using kubeadm upgrade apply, kubeadm fails to parse an etcd imagetag containing build metadata in the form of as a valid semver. An example upgrade produces an error similar to: Note this will occur even if the etcd version is unchanged. The +` character. This pattern is used to support custom kubernetes images, for instance kube-apiserver:v1.20.4+patch.0 will be converted to imagetag v1.20.4_patch.0 when running make quick-release-images. Add the following to kubeadm- Install kubernetes. Then upgrade kubernetes with kubeadm upgrade apply. I have tested a fix for this issue and will PR shortly. Kubernetes version (use ): N/A Cloud provider or hardware configuration: N/A OS (e.g: ): N/A Kernel (e.g. ): N/A Install tools: kubeadm Network plugin and version (if this is a network-related bug): N/A Others: N/A
/sig cluster-lifecycle
/area kubeadm
/triage accepted
Code change:
// Cannot be updated. // +optional WorkingDir string `json:"workingDir,omitempty" protobuf:"bytes,5,opt,name=workingDir"` <del> // List of ports to expose from the container. Exposing a port here gives // the system additional information about the network connections a // container uses, but is primarily informational. Not specifying a port here </del> <ins> // List of ports to expose from the container. Not specifying a port here </ins> // DOES NOT prevent that port from being exposed. Any port which is // listening on the default "0.0.0.0" address inside a container will be // accessible from the network. <ins> // Modifying this array with strategic merge patch may corrupt the data. // For more information See https://github.com/kubernetes/kubernetes/issues/108255. </ins> // Cannot be updated. // +optional // +patchMergeKey=containerPort

Related discussion:
text: The comments in says that : However there are several cases where this is false : When using a Service with a targetPort that references a name in the ports arrays of the pod. Some software (ab)uses this API to get which ports are used (See Although unfixable, my goal is to fix the documentation so other people don't have the same problem). The Azure Load Balancer implementation does this for instance. Some other network/security software might use this information. Also, is it possible to document/reference the above issue in the comment ? Since it is described as informational I expected for the network to work without this information. But it's definitely not the case. Not needed since it's mostly a doc issue. I'm interested in contributing this patch but would like some feedbacks before doing it. :) <details</details<details</details<details</details<details</details<details</details<details</details
/sig network
/triage accepted -- thoughts? I believe the doc are due for a refresh.
I'd take this one. /assign
I didn't see that you wanted to do this patch, I am very sorry and now I unassign it. /unassign
No worries But I'm still not sure where / what to add. Should I only fix the part saying it is only informational ? Is there anything else ? (cc
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
Code change:
const ( // StatusSuccess represents the successful completion of command. StatusSuccess = "Success" <del> // StatusFailed represents that the command failed. StatusFailure = "Failed" </del> // StatusNotSupported represents that the command is not supported. StatusNotSupported = "Not supported" )

Related discussion:
text: <!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): statusFailure Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): 4 Environment: Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: Others What happened: In FlexVolume code: The statusFailure constant should have string literal "Failure", not "Failed". Because in the test code or documentation everywhere, only "Failure" string is used, not "Failed" for the flexvolume cmd output status string. So this is a typo bug. Here is the test: here is the documentation: here is the example: What you expected to happen: The statusFailure constant should have string literal "Failure", not "Failed" at this line: How to reproduce it (as minimally and precisely as possible): code inspect: Anything else do we need to know: nothing else
Code change:
FlagTimeout      = "request-timeout" ) <ins> // RecommendedConfigOverrideFlags is a convenience method to return recommended flag names prefixed with a string of your choosing func RecommendedConfigOverrideFlags(prefix string) ConfigOverrideFlags { return ConfigOverrideFlags{ AuthOverrideFlags:    RecommendedAuthOverrideFlags(prefix), ClusterOverrideFlags: RecommendedClusterOverrideFlags(prefix), ContextOverrideFlags: RecommendedContextOverrideFlags(prefix), CurrentContext: FlagInfo{prefix + FlagContext, "", "", "The name of the kubeconfig context to use"}, Timeout:        FlagInfo{prefix + FlagTimeout, "", "0", "The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests."}, } } </ins> // RecommendedAuthOverrideFlags is a convenience method to return recommended flag names prefixed with a string of your choosing func RecommendedAuthOverrideFlags(prefix string) AuthOverrideFlags { return AuthOverrideFlags{

Related discussion:
text: Found in openshift, but I'm pretty sure the codepaths are the same. That means the token is getting ignored and in-cluster config is being used in preference to flags.
Is a similar issue?
Yep, kind of regression.
Sigh.
Yes, most likely the same cause.
I'm on it.
Unit test :)
Probably will want to cherrypick this into 1.5.
WIP in
Code change:
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/net" "k8s.io/apimachinery/pkg/util/runtime" <ins> "k8s.io/apimachinery/pkg/util/sets" </ins> "k8s.io/apimachinery/pkg/util/wait" corev1client "k8s.io/client-go/kubernetes/typed/core/v1" "k8s.io/client-go/tools/record"

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: I created several Workloads which require up to ten ports for both UDP and TCP protocol. Now my logs get spammed with . What you expected to happen: should consider -combination instead of only . How to reproduce it (as minimally and precisely as possible): Deploy any service with a node-port with both protocols, e.g.: Anything else we need to know?: This was already acknowledged by here: However, as this thread addresses a different bug and seems kind of stuck, i'd like to suggest that this might be addressed in a new context. Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Deployed via Rancher v2.4.3
/sig api-machinery
/remove-sig api-machinery /sig network
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry. ‍
How did you get into this situation? The allocator should not have granted the same node port twice?
Oh right, that issue. The only real way out is to de-allocate the NodePorts and then re-allocate them, I think
As this happens for all of my ports in the said Production Application (the one where my log gets spammed) as well as complete fresh installation (where i verified the bug), i think there's no "way out" except fixing the bug. As said, for me it's necessary to assign both protocols to the same port. This might be a niche-use-case, but checking the it's actually very likely for ports to allow tcp AND udp. I'm not sure about the history of this, but it might be that issue was the origin, which was adressed in PR ("Allow services which use same port, different protocol to use the same nodePort for both"). But this PR doesn't contain the adjustment of the said , so still considers the behaviour as invalid, while it's actually allowed and supported.
Code change:
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/net" "k8s.io/apimachinery/pkg/util/runtime" <ins> "k8s.io/apimachinery/pkg/util/sets" </ins> "k8s.io/apimachinery/pkg/util/wait" corev1client "k8s.io/client-go/kubernetes/typed/core/v1" "k8s.io/client-go/tools/record"

Related discussion:
text: /assign
Ahh, I thought you meant that you had accidentally been allocated the same nodePorts and I was very confused when you said "all of them". I have reproduced this issue, and I can see it by inspection. The obvious fix would be to collect ports split by protocol, but I think that's not right since we don't store it split. An even simpler fix would be for to build a . Then for each protocol, for each port, build that contains all the port numbers, but remove any duplicates ONLY IF they are duplicated across protocols. Duplicates in the same protocol should be retained (to report an error later). I don't have time to tackle this tonight (and write a test), but it seems like a good bug.
perhaps I can fix this problem /assign
things may be a little more complicated. Right now the is not aware of the protocol of the port so I guess simply rewriting function to return is not enough, the port allocator will still return on same node port even with different protocol. perhaps we need to refactor the to make it protocol-aware.
I don't think we need to do make the allocator be protocol aware. IIRC we allow you to use the same nodeport on different protocols ONLY within a single service. This means (I think) that we can do hyper-local deduplication as we collect the nodeports for each service. If we find the same nodeport number with different protocols, just remove the duplicates because it is allowed. I.e. a service that looks like: return - deduped. if ANY OTHER service tried to use it would be an error at the "rebuilt" allocator Likewise if a single Service claims to use the same nodeport number more than once on the same protocol, it should be an error: return - NOT deduped.
obviously, we want a great unit test here :P
ah got it! thanks for your detailed explanation!
Hi! I have filed a PR to resolve this issue, could you take a look?
Code change:
mkdir -p "${dst_dir}" local salt_dir="${KUBE_ROOT}/cluster/saltbase/salt" <ins> cp "${salt_dir}/cluster-autoscaler/cluster-autoscaler.manifest" "${dst_dir}/" </ins> cp "${salt_dir}/fluentd-es/fluentd-es.yaml" "${release_stage}/" cp "${salt_dir}/fluentd-gcp/fluentd-gcp.yaml" "${release_stage}/" cp "${salt_dir}/kube-registry-proxy/kube-registry-proxy.yaml" "${release_stage}/"

Related discussion:
text: For some reason they stopped working and we even don't create manifest file for cluster autoscaler. I used the following command to start the cluster:
It's possible it's related to
Code change:
if pod == nil { return framework.NewStatus(framework.Error, "pod cannot be nil") } <del> state.RLock() defer state.RUnlock() </del> if v, e := state.Read(framework.StateKey(pod.Name)); e == nil { if value, ok := v.(*stateData); ok && value.data == "never bind" { return framework.NewStatus(framework.Unschedulable, "pod is not permitted")

Related discussion:
text: What happened: kube-scheduler () crashes due to a panic while running : The race condition is triggered when an out-of-tree plugin, that writes to during its Filter phase, is registered through . Access to in general happens from a single goroutine and there are no practical concurrency issues, with the exception of the phase, that parallelizes calls for each node across plugins: A quick glance at the tip of the branch seems to indicate the bug is still present there. What you expected to happen: Ideally access to should be synchronized everywhere for correctness, including by all in-tree plugins, since there are no guarantees about how will call multiple plugins concurrently. It seems to me it is something that was considered at some point, given that has methods. Sadly, almost no in-tree plugins are synchronizing access. For now, I will try to work around the problem by doing a single write to during and storing a data bag structure where I will write state into during (with proper synchronization), instead of writing to directly. How to reproduce it (as minimally and precisely as possible): Install a scheduler plugin that writes to during its phase. In-tree (builtin) plugins are not synchronizing their access to . Environment: Kubernetes version (use ): v1.18.8 Cloud provider or hardware configuration: linux-amd64 (AWS) OS (e.g: ): Ubuntu 18.04.5 LTS (Bionic Beaver) Others: custom kube-scheduler binary built with an out-of-tree plugin using
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling
AFAICT all in-tree plugins write to the cycle state in PreFilter and then only read from cycle state in Filter. So this shouldn't be a problem with in-tree plugins as concurrent read access should be fine during RunFilterPlugins. But since your out-of-tree plugin writes in Filter, I can see why this can be a problem.
Code change:
if pod == nil { return framework.NewStatus(framework.Error, "pod cannot be nil") } <del> state.RLock() defer state.RUnlock() </del> if v, e := state.Read(framework.StateKey(pod.Name)); e == nil { if value, ok := v.(*stateData); ok && value.data == "never bind" { return framework.NewStatus(framework.Unschedulable, "pod is not permitted")

Related discussion:
text: Aside from adding Lock/Unlock in in-tree plugin, offering mutual exclusion on each key as a built-in feature within CycleState could be an alternative solution. This way, reads from in-tree plugins during Filter wouldn't interfere with writes during Filter from out-of-tree plugins.
cc
should we change the Read/Write CycleState methods to be thread safe?
Adding locks without a proper justification would just increase the latency. could you clarify your use case? Why do you need to write anything during Filter?
yeah, we should measure it, but I don't expect it to have significant effect...
Adding locks without a proper justification would just increase the latency. Without contention (which is the case with in-tree plugins), I don't think this is a concern.
Adapting source from : source: essentially, 11.3 ns/Lock-Unlock pair
I remember locks showing up in profiling for PreScore/Score for certain plugins. Although I have to admit that the accesses were not properly implemented. I'd like to hear an answer for before doing any experiments.
+1 to that I'd also like to understand the motivations / requirements of mutating CycleState in Filter().
My particular use-case is that we have resources being dynamically to/removed from Nodes, and those resources need to be allocated (and tracked) in the scheduling loop. Technically I could pre-compute all the available dynamic resources for all Nodes at the phase, and snapshot it, but that would require me to pre-compute everything for all Nodes on all scheduling cycles. Instead, what I'm currently doing is computing what dynamic resources can be utilized on-demand during the phase (kind of a lazy loading of dynamic resources for each pair), and saving state of what extra resources were considered during for each Node that was evaluated. Then, later in the scheduler plugin can lock-in the exact dynamic resources for the particular Node that was selected. I understand the concern with adding extra synchronization overhead without a real need. Even if my use-case is not good enough to justify it, there is another argument for it: technically, the lack of access synchronization on violates the . Even if in the end we decide synchronizing all access is not worth the overhead (for correctness), I would strongly encourage better documentation saying which plugin phases are allowed to modify , and perhaps even forcing a snapshot-read-only copy of to be used by plugins.
Using the scheduler to track resources is not a good idea.
Code change:
if pod == nil { return framework.NewStatus(framework.Error, "pod cannot be nil") } <del> state.RLock() defer state.RUnlock() </del> if v, e := state.Read(framework.StateKey(pod.Name)); e == nil { if value, ok := v.(*stateData); ok && value.data == "never bind" { return framework.NewStatus(framework.Unschedulable, "pod is not permitted")

Related discussion:
text: The scheduler should be stateless. Can't you track the resources with an external controller or with a device plugin and making it part of the Node spec?
there may be some misunderstanding in my poor choice of words. What I mean by "tracking" is that the scheduler needs to track usage and availability of these extra per-Node resources, to be able to allocate them during placement/scheduling. Exactly the same it does with volumes, cpus, memory, ephemeral disk, etc Doing computations in a controller outside of the scheduling loop would be similar to "precomputing" these resources. In my particular use case I chose a different way because: introduces another point of failure, if the external controller stops pre-computing extra resources and publishing them somewhere (CRD, Node spec, etc) scheduling halts (at least scheduling on these extra resources) Node resources in my use case are volatile (they are coming and going frequently), and the extra latency by an extra controller (vs doing all computations right in the scheduling loop) may be problematic for my use case. Even if you consider my use case not enough to add proper synchronization to , I'd still encourage making it explicit what scheduling phases are allowed to modify it (and even passing in a snapshot read-only version of to phases that are not expected to update it), to avoid confusion and race conditions.
Ok, if you are not tracking the existence of resources in the scheduler, I wouldn't suggest the external controller. Still, have you considered the device plugin framework? That way, the default kube-scheduler could keep track of resources usage. Otherwise, I still don't see why you would need to update the CycleState during Filter. As its name indicates, CycleState only lives during a scheduling cycle.
Regardless of the correctness of usage of CycleState to do what wants, I still think we need to do at least one of the following: that writing to CycleState during Filter will lead to data races, a read-only CycleState to Filter plugins (whether we do this at the interface level or if we simply drop Write requests during Filter is an implementation detail), or make CycleState concurrency safe CycleState as it exists right now allows plugin authors to shoot themselves in the foot IMO.
I'm leaning towards 1, but it's fair to continue the discussion before deciding.
Can you run the "controller" as a goroutine along with your scheduler plugin?
Code change:
if pod == nil { return framework.NewStatus(framework.Error, "pod cannot be nil") } <del> state.RLock() defer state.RUnlock() </del> if v, e := state.Read(framework.StateKey(pod.Name)); e == nil { if value, ok := v.(*stateData); ok && value.data == "never bind" { return framework.NewStatus(framework.Unschedulable, "pod is not permitted")

Related discussion:
text: So it's not an external controller binary, and hence don't introduce another point of failure. It's pretty like how the default scheduler manages its internal cache. But note that cache is only needed when you need a more efficient data structure. If a {key:value} map is good enough, spinning up a XYZInformer and obtain the latest XYZ resource on-demand is the most recommended way. I think it quite depends on how you design the internal XYZ cache. For example, it should only tweak the cache in an incremental manner - in other words, unchanged resource will be kept unmodified. You can check out the "doubly-pointer snapshot" of scheduler's cache. With above said, if I understand your case correctly, you may need to build a sophisticated XYZ cache, which runs in a separate goroutine and responds to your XYZ resource changes. And then upon each scheduling cycle, take a snapshot (may needs a specific snapshotting algorithm) in PreFilter() and WRITE to CycleState, and then in Filter() READ from CycleState. My personal option is to suggest all plugin developers to read from CycleState in Filter().
nailed it. What you describe is almost exactly what I’m doing today. There are some details particular to my use case (e.g. just tracking current state with an informer and an incremental cache is not enough, because there’s some logic involved in deciding if currently available resources are usable by each pair, and I currently do that computation in as pairs are evaluated). In any case, I just wanted to share my use case to provide some color, but I don’t want what I’m doing distracting us from the more important questions, mentioned by above. My personal preference would be as a very short term next step, then when someone has cycles to do it. I would be OK with punting on until a more strong need shows up.
Note that limiting to only reading from works for me, as long as I can still read my own data structure from it and do writes on it (i.e. don’t deepclone for the phase).
also adds overhead (from copying the state, which can be very expensive).
Could implement just by changing the interface passed to (e.g. ). That wouldn’t be backwards compatible, but very cheap.