jiebi/CodeConvo huggingface dataset:
Code change:
import ( "bytes" "encoding/binary" <ins> "errors" </ins> "fmt" "net" "os/exec"

Related discussion:
text: Hi, Since my upgrade to 1.18 version, I have errors in kube-proxy: I have ipv4/ipv6 dualstack enable. No problem with cluster and IPVS works despite errors. Do you have also this issue ? OS: RHEL7 Kubernetes version (use ):
/sig network
Related to cc
/cc
Taking one of the outputs it gives me the following addresses? can you share more details about your environment, Is it possible that the CoreDNS pod has the address ? what are your cluster-cidr and service-cidr? Indeed that new net.Utils change looks suspicious , a quick grep in the repo gives me some suspects, maybe the service/ipallocator? It's important to mention there were similar issues with IPVS before, interpreting IPv4 addresses as IPv6, has some experience with this kind of issues, maybe he can see something:) /cc
A bit tricky to trace since libnetwork/ipvs is moving around at the moment. Problem seems to be in; The "family" must be wrong. It is called from;
Please do a and attach the output.
Hi, Some informations: cluster-cidr and service-cidr: fd00:10:245:0:0:0:0:0/64,10.245.0.0/16 fd00:10:96:0:0:0:0:0/112,10.96.0.0/12 ipvsadm output:
Ref:
The commit in the ref above should be in v1.18.0 which is the version used in this issue. So the commit does not fix the problem.
Maybe something wrong around here Before , it looks like this: Now it looks like this: However is not initializedit is zero :(.
I do not get these faults. Has the netlink API changed? What kernel version are you using?
d.AddressFamily is supposed to be set from the "NetlinkRouteAttr" slice; It should definitely not be hard-coded to ipv4.
Made a trying to reproduce this in my environment but didn't got this error.
Code change:
import ( "bytes" "encoding/binary" <ins> "errors" </ins> "fmt" "net" "os/exec"

Related discussion:
text: I may try this on some older kernel like from CentOS 7 Added exactly the same IPVS configuration here: And when running this 'gist/program' pointing to the IPVS from the DNS (fd00:10:96::a port 53 UDP) got the expected result: Maybe trying to run this also in some other scenarios could clarify if this is something related to Netlink API changed between Kernels, something with the used library, etc.
After some study, I find that this issue most likely is caused by the low linux kernel version. I have reproduced this problem on . Accroding to the code, netlink will try to get attribute, but I find the kernel just does not support it :( This is the defined in (kernel 3.10) No attribute is defined! But in new kernel version, the is defined like : Obviously, the kernel has some attributes (IPVSDESTATTRADDRFAMILY, IPVSDESTATTRSTATS64...). That is why kube-proxy works well on systems with a higher version of the kernel. So we can address this issue by upgrading our linux kernel. Not sure which kernel version is the minimum requirement, maybe we should document something about this.
Here is the defined in :
Good finding. So, we can conclude that this really is kernel version dependent. It is hard to know what to do about it. We must check the family. I would propose that we make it work for ipv4 with some fallback and put a requirement on kernel version for ipv6-only/dual-stack. But the decision is others to make. /cc
/cc
The symbol in the kernel tree The for service family seems very old. Since the service must have the same family as the real-servers perhaps it can be used for compatibility with old kernels?
The commit that adds the symbol; Seem to go into linux-3.18
I did not know the minimum required for the kernel version, I was using the stable and official version of RHEL7. My kernel version is: Currently the max version in the repository is: According to the documentation (), I should switch to RHEL8 if I want a recent kernel :-
There is no minimum kernel version requirement afaik. So I think this must be fixed for ipv4-only clusters IMO.
Code change:
import ( "bytes" "encoding/binary" <ins> "errors" </ins> "fmt" "net" "os/exec"

Related discussion:
text: However ipv6-only and dual-stack are in "alpha" so a requirement on kernel version might be ok, but I can't say myself. A complication is that the problem is not in k8s itself but in a 3rd party lib.
ipv6-only is beta since 1.18 :smile:
/assign
I opened a PR to fix this in moby/ipvs If someone w/ a cluster on the older kernel version can test it that would be great, otherwise I will setup a cluster to validate this some time this week.
Instead of bootstraping a whole cluster right now (pretty busy here) I can use the Gist I've mentioned to check in a CentOS 7.1 if it faces the problem before and after the patch. At least to have some more test right now
with a small test, it worked fine. Environment: CentOS 7.1 - Kernel 3.10.0-229.el7.x86_64 + ipvsadm 1.2.1 Pre-loaded some IPv6 rules into ipvs: I've used code, searching for the following structure: I then call ` to get the IPVS object, and then try to fetch the destinations. The following are the results: Before patching the ipvs module: After patching the ipvs module (pinning to your repo/branch) Also I've made those tests with the structure but searching for TCP, and for kube-apiserver (that have only one destination) and it worked fine in all those scenarios. If wanted I can improve the gist to pass the parameters as command arguments instead of hardcoded (sorry, did this in a rush to test)
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry. ‍
/remove-triage unresolved
Thanks just waiting on now
same issue,sloved by update kernel to 5.X
I didn't want to wait, I updated to kernel-lt in the version 4.4 . The problem is solved on my side :)
Thanks for confirming. We'll still try to get in since I'm sure other folks with older kernels will run into this issue.
/retitle parseIP error with IPVS proxy
upgrade your system kernel to 4.xx edition
FYI: just merged, need to run some validation & cut a new release (v1.0.1).
Code change:
import ( "bytes" "encoding/binary" <ins> "errors" </ins> "fmt" "net" "os/exec"

Related discussion:
text: FYI
/retitle DualStack / IPv6-only: parseIP error with IPVS proxy on CentOS 7
v1.18 cherry-pick
Hi, it seems that you're using v1.18.0 and this has been corrected in v1.18.3, can you please update and check? Tks
Code change:
}, } if utilfeature.DefaultFeatureGate.Enabled(features.PodDisruptionConditions) { <ins> role.Rules = append(role.Rules, rbacv1helpers.NewRule("get").Groups(legacyGroup).Resources("pods").RuleOrDie()) </ins> role.Rules = append(role.Rules, rbacv1helpers.NewRule("patch").Groups(legacyGroup).Resources("pods/status").RuleOrDie()) } return role

Related discussion:
text: For 1.25 clusters with PodDIsruptionConditions enabled pods continue to run if the node is tainted with untolerated NoExecute taint. Also, the error is not logged. The pod running on the tainted node should be terminated (prep) Create a 1.25 cluster with the enabled (tested on kind) a job with a single Pod running long enough for the test (say 3min), the node with a custom taint (so that it is not tolerated by the Pod) with the NoExecute effect (nodeName No response <details</details<detailsreproducible on kind </details<details</details<details</details<details</details<details</details
/sig apps /triage accepted /priority important-soon
/assign
Code change:
"//pkg/controller/nodeipam/ipam/cidrset:go_default_library", "//pkg/controller/nodeipam/ipam/sync:go_default_library", "//pkg/controller/util/node:go_default_library", <ins> "//pkg/scheduler/algorithm:go_default_library", </ins> "//pkg/util/node:go_default_library", <ins> "//pkg/util/taints:go_default_library", </ins> "//vendor/github.com/golang/glog:go_default_library", "//vendor/k8s.io/api/core/v1:go_default_library", "//vendor/k8s.io/apimachinery/pkg/api/errors:go_default_library",

Related discussion:
text: forked from As from the discussion with and we have to start checking taints wherever we check for node.Spec.Unschedulable. Also overlaps on some parts with /sig node /assign
cc
No sure the discussion history :); personally, prefer to deprecate and replace it with taint.
- same here, the goal would be to deprecate its usage, start checking taints everywhere unschedulable is used. Also, as long as we as we have v1, we might have to keep checking for both.
great, there's same subtask in my mind: deprecated description to the taint for command with taint others ??
- yup, but for 4 it is more something like check taints wherever we checn for unschedulable, since we cannot just replace until v1 is gone.
, my major point is to handle it task by task to make sure we did not miss anything :).
yes we definitely should
- should we define a specific taint for unschedulable (key and effect) ? (i.e what happens when a user uses cordon/uncordon)
I'm not sure about having specific Taint for Unschedulable is a good idea. Pretty much any Taint with NoSchedule effect will do (and this is what we should be checking).
- I agree, but in case of we can add a taint with key or something like that with a effect. This might help us especially if the node has no taints.
, not sure whether we should handle specially; AFAIK, set this flag for node maintenance, this taint should not be tolerated.
The whole point of moving from to Taints is to make things more configurable. as they're decision makers here.
If we are discussing about deprecate/re-implement by Unschedulable taint, +1 as it make sense. Current cordon is causing confusion in community since you can always taint it instead, and tainting is what most Kubernetes setup tool is doing today.
I agree with that we should use NoSchedule taint. I also agree that generally no pod should tolerate NoSchedule taints, but they may want to do so if the desire, knowing that anything could happen to the node they are scheduled on. For example, today DS pods tolerate "Unschedulable" bit.
Code change:
"//pkg/controller/nodeipam/ipam/cidrset:go_default_library", "//pkg/controller/nodeipam/ipam/sync:go_default_library", "//pkg/controller/util/node:go_default_library", <ins> "//pkg/scheduler/algorithm:go_default_library", </ins> "//pkg/util/node:go_default_library", <ins> "//pkg/util/taints:go_default_library", </ins> "//vendor/github.com/golang/glog:go_default_library", "//vendor/k8s.io/api/core/v1:go_default_library", "//vendor/k8s.io/apimachinery/pkg/api/errors:go_default_library",

Related discussion:
text: They may as well want to tolerate "NoSchedule" taints.
/sig scheduling
Agree; it's up to 'user' to decide whether tolerates the taint, we just need to taint it and doc what's happened in node for taints :)
We should have a mechanism that allows a cluster admin to limit types of tolerations that users can set. This is needed to prevent abuse in multi-tenant clusters. For example, cluster admin may want to taint nodes with rare resources to prevent pods from certain group of users to land on them. Users shouldn't be able to tolerate such taints. That said, this is a separate topic of its own which is related to policy and should be discussed separately.
- we’re already facing this in production for our spare nodes. I’ll bring this up in our next sig-meeting, I’ll also file an issue for this
Code change:
# Use host IP instead of localhost in control plane kubeconfig files. export KUBECONFIG_USE_HOST_IP="${KUBECONFIG_USE_HOST_IP:-false}" <ins> # Optional: Install Pigz on Windows. # Pigz is a multi-core optimized version of unzip.exe. # It improves container image pull performance since most time is spent # unzipping the image layers to disk. export WINDOWS_ENABLE_PIGZ="${WINDOWS_ENABLE_PIGZ:-true}" </ins>

Related discussion:
text: <!-- Please only use this template for submitting enhancement requests --What would you like to be : pigz for GCE Windows Why is this needed: Pigz improves the image layer extraction performance for multi-core systems. Pigz is usually preinstalled on Linux. This enhancement is scoped specifically for the GKE Windows node scripts. For Context:
/assign me
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
GitHub didn't allow me to assign the following users: me. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign me Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig windows /assign jeremyje
s/GKE/GCE/? as in kube-up?
Replaced but I don't understand the kube-up part.
is how one creates a GCE cluster from this repo. Otherwise we'd be talking about something that isn't part of kubernetes.
Code change:
experimentalHostUserNamespaceDefaulting: utilfeature.DefaultFeatureGate.Enabled(features.ExperimentalHostUserNamespaceDefaultingGate), keepTerminatedPodVolumes:                keepTerminatedPodVolumes, nodeStatusMaxImages:                     nodeStatusMaxImages, <ins> lastContainerStartedTime:                newTimeCache(), </ins> } if klet.cloud != nil {

Related discussion:
text: These tests have flaked consistently since merged: in many cases, they are the only failure in a test run: Either the functionality introduced by that PR is not reliable, or the test is faulty. /sig node /priority critical-urgent /assign
/milestone v1.19
/assign
Looks like this is failing ~5% of pull-kubernetes-node-e2e runs
FYI
Looks like there might still be a race somewhere looking at the kubelet logs Success case: Fail case:
Looking at it more, it seems like is unexpectedly empty here Success: Fail:
Stacks of interest I think we might be dealing with a stale If there is a call before the container is to the Pod in the runtime and the is called within the 2 second default cache period, it seems possible that the in the could have no containers.
what do you think of this analysis? If so, any obvious solutions occur to you?
If I'm correct, didn't introduce this; just the e2e test exposed an existing bug in the kubelet.
That all looks correct. I'm taking a look at the proposed solutions
Code change:
if mapping.Key != strings.ToLower(mapping.Key) { allErrs = append(allErrs, field.Invalid(fldPath.Child("key"), mapping.Key, "key must be lowercase")) } <ins> if isKubernetesDomainPrefix(mapping.Key) { allErrs = append(allErrs, field.Invalid(fldPath.Child("key"), mapping.Key, "k8s.io, kubernetes.io and their subdomains are reserved for Kubernetes use")) } </ins> if seenExtraKeys.Has(mapping.Key) { allErrs = append(allErrs, field.Duplicate(fldPath.Child("key"), mapping.Key)) continue

Related discussion:
text: Disallow setting all and namespaced extra info in custom ways for now. /assign aramase enj /sig auth
/triage accepted
Code change:
"description": "Endpoint represents a single logical "backend" implementing a service.", "properties": { "addresses": { <del> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100.", </del> <ins> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267", </ins> "items": { "type": "string" },

Related discussion:
text: Given the number of changes currently in flight for kube-proxy, documentation seems like a safer initial approach to me.
/assign
can I have this editing only the first address(Address[0]) in Endpoint.Address to be used. The use of an array in this field is based on historical reasons. Isn't that right
thanks for volunteering to help with this one! I know has some big kube-proxy changes/fixes in flight right now, so it would be good to be sure you don't conflict with him. Otherwise, as long as isn't already working on this, I think you're free to have it.
I don't think we agreed that the array is for historical reasons only. We aren't currently using it, but we aren't ready to say that we won't use it in the future at this point. So, the docs should warn that kube-proxy currently ignores additional addresses, but it shouldn't imply that it's actually wrong to have additional addresses I'm not
thanks i will work on this
It was for a reason, but I think that reason was mixed-family slices (before we decided to break it up into single-family slices). What would it mean to have multiple addresses here? Are they assumed to be equivalent? Or do they have different context (like a split-horizon)? If the latter, there isn't any metadata here (i.e. it's not a struct like to allow new fields). I am inclined to believe it was meant as equivalent, and as such it's probably OK to document that assumption clearly. That said, I don't see how anyone could use multiple addresses in a meaningful way - they are either assumed equivalent or they lack enough metadata to distinguish their differences. ?? If we do document the equivalence, it should be generically worded. E.g. "These addresses are assumed to be equivalent and some consumers may choose to ignore all but the first value."
if it was meant as equivalent, why put multiple same addresses into Addresses? I don't understand what the purpose of this is.
The first version of the API had both IP families in a single slice, so it made sense to have multiple addresses. We changed that, though we didn't immediately remove the old support. So my best guess is that we should have converted this to a singular value when we did that, but we missed it.
Code change:
"description": "Endpoint represents a single logical "backend" implementing a service.", "properties": { "addresses": { <del> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100.", </del> <ins> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267", </ins> "items": { "type": "string" },

Related discussion:
text: On Wed, Nov 24, 2021, 5:45 PM Cyclinder
so we should change the API doc? Or It's just that documenting it will only use the first address in the , like you said above
I'd like some of the others to comment, but it's this a US holiday, so it might be a few days..
So maybe we should just declare that the array is essentially a bug, and only the first address is used? Maybe something should warn if you ever set additional addresses? (FTR, the context that I noticed this in was that I was adding unit tests, and was creating EndpointSlices with a single Endpoint and multiple Addresses, which doesn't work; you need multiple Endpoints with a single Address each.)
I think the simplest and most compat approach is to say "these are all assumed to be fungible and clients may choose to only use the first element". Thoughts?
I think the simplest and most compat approach is to say "these are all assumed to be fungible and clients may choose to only use the first element". Big +1 to that approach. I don't think it's necessarily invalid to set multiple addresses here, there may actually be some use cases for it, but adding this kind of statement should help clear things up significantly.
hmm as we push for multi homed pod this will make a lot of sense. There might also be a chance for community to provide a drop-in replacement for proxy that does much more than we do now. I think we should update the docs and don't modify anything for now.
Note that this API doesn't include per-IP metadata, so a multi-homed pod would be challenging with this mechanic. I think it's maybe time to reconsider multi-net more holistically. On Mon, Dec 13, 2021, 10:24 AM Khaled Henidak (Kal) <
:thinking:
If they're fungible, then "clients may choose to use only one of the elements" (or "only a subset"?). No reason to call out the first one specifically.
That's fair On Tue, Dec 14, 2021, 6:09 AM Dan Winship
I would like to add to this issue that it would makes sense to have kube-proxy reduce the subset of endpoints in the endpointslice based on the label of the node and the & attributes of the endpoint.
Code change:
"description": "Endpoint represents a single logical "backend" implementing a service.", "properties": { "addresses": { <del> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100.", </del> <ins> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267", </ins> "items": { "type": "string" },

Related discussion:
text: Taking only into consideration the first endpoint which is in the same zone of the node or has no zone dependency and is ready for serving. & Would it make sense to reopen this issue?
I don't follow entirely what you are suggesting, can you elaborate?
kube-proxy should understand which zone a node is in and prefer endpoints in the same zone.
but that is unrelated to this issue, topology is considered See KEPs related, topology-aware-routing was replaced by topology-aware-hints
thank you for clarifying. It seems ambiguous to me to have a zone attribute and a hint attribute. I fail to implement the hinting on a service in v1.23.8, but this is urelated to this issue.
heh, don't worry, this topic is complex and we went through different solutions, you have more historical context on the KEPs I linked above and this thread in the mailing list