jiebi/CodeConvo huggingface dataset:
An can contain up to 1000 , which can each contain up to 100 . Or at least, that's what the docs say, but actually, kube-proxy (or more specifically, ) only looks at the first Address in each Endpoint: (This is compatible with how works, since it would only generate multi-Address Endpoints if a pod had multiple of the same address family, which is not allowed. But it's not compatible with manually-generated EndpointSlices.) Either all are used, or else the docs state that only is used and the field is an array solely for historical reasons.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network /priority important-longterm /cc
All addresses in the slice must be of the same type, do you mean creating an Slice with different address types in the Addresses field?
I'm not sure if/how kube-proxy should handle an endpoint with more than 1 address. I think the goal of kube-proxy is to give each endpoint an equivalent probability of being selected. To extend that logic here, I think we'd need to take the probability assigned to an endpoint and divide it by the number of addresses attached to that endpoint. So if an endpoint has a 50% probability of being selected, and it provides 50 addresses, each should end up with a 1% probability of being selected. Of course with the way we need to do descending probabilities, the logic may get a bit complicated. This isn't as clear as it should be, but I don't think kube-proxy or any consumer of the EndpointSlice API needs to commit to supporting every value specified by that API. For example, many may filter by address type, Service, or some other means. I think it's also valid for a consumer to say they only support a single address per family per endpoint. So all of that to say, I'd support either documenting that kube-proxy only selects the first address in an endpoint, or updating kube-proxy to support more than one address per endpoint.
Given the number of changes currently in flight for kube-proxy, documentation seems like a safer initial approach to me.
/assign
can I have this editing only the first address(Address[0]) in Endpoint.Address to be used. The use of an array in this field is based on historical reasons. Isn't that right
thanks for volunteering to help with this one! I know has some big kube-proxy changes/fixes in flight right now, so it would be good to be sure you don't conflict with him. Otherwise, as long as isn't already working on this, I think you're free to have it.
I don't think we agreed that the array is for historical reasons only. We aren't currently using it, but we aren't ready to say that we won't use it in the future at this point. So, the docs should warn that kube-proxy currently ignores additional addresses, but it shouldn't imply that it's actually wrong to have additional addresses I'm not
thanks i will work on this
It was for a reason, but I think that reason was mixed-family slices (before we decided to break it up into single-family slices). What would it mean to have multiple addresses here? Are they assumed to be equivalent? Or do they have different context (like a split-horizon)? If the latter, there isn't any metadata here (i.e. it's not a struct like to allow new fields). I am inclined to believe it was meant as equivalent, and as such it's probably OK to document that assumption clearly. That said, I don't see how anyone could use multiple addresses in a meaningful way - they are either assumed equivalent or they lack enough metadata to distinguish their differences. ?? If we do document the equivalence, it should be generically worded. E.g. "These addresses are assumed to be equivalent and some consumers may choose to ignore all but the first value."
if it was meant as equivalent, why put multiple same addresses into Addresses? I don't understand what the purpose of this is.
The first version of the API had both IP families in a single slice, so it made sense to have multiple addresses. We changed that, though we didn't immediately remove the old support. So my best guess is that we should have converted this to a singular value when we did that, but we missed it.
On Wed, Nov 24, 2021, 5:45 PM Cyclinder
so we should change the API doc? Or It's just that documenting it will only use the first address in the , like you said above
I'd like some of the others to comment, but it's this a US holiday, so it might be a few days..
So maybe we should just declare that the array is essentially a bug, and only the first address is used? Maybe something should warn if you ever set additional addresses? (FTR, the context that I noticed this in was that I was adding unit tests, and was creating EndpointSlices with a single Endpoint and multiple Addresses, which doesn't work; you need multiple Endpoints with a single Address each.)
I think the simplest and most compat approach is to say "these are all assumed to be fungible and clients may choose to only use the first element". Thoughts?
I think the simplest and most compat approach is to say "these are all assumed to be fungible and clients may choose to only use the first element". Big +1 to that approach. I don't think it's necessarily invalid to set multiple addresses here, there may actually be some use cases for it, but adding this kind of statement should help clear things up significantly.
hmm as we push for multi homed pod this will make a lot of sense. There might also be a chance for community to provide a drop-in replacement for proxy that does much more than we do now. I think we should update the docs and don't modify anything for now.
Note that this API doesn't include per-IP metadata, so a multi-homed pod would be challenging with this mechanic. I think it's maybe time to reconsider multi-net more holistically. On Mon, Dec 13, 2021, 10:24 AM Khaled Henidak (Kal) <
:thinking:
If they're fungible, then "clients may choose to use only one of the elements" (or "only a subset"?). No reason to call out the first one specifically.
That's fair On Tue, Dec 14, 2021, 6:09 AM Dan Winship
I would like to add to this issue that it would makes sense to have kube-proxy reduce the subset of endpoints in the endpointslice based on the label of the node and the & attributes of the endpoint.
Taking only into consideration the first endpoint which is in the same zone of the node or has no zone dependency and is ready for serving. & Would it make sense to reopen this issue?
I don't follow entirely what you are suggesting, can you elaborate?
kube-proxy should understand which zone a node is in and prefer endpoints in the same zone.
but that is unrelated to this issue, topology is considered See KEPs related, topology-aware-routing was replaced by topology-aware-hints
thank you for clarifying. It seems ambiguous to me to have a zone attribute and a hint attribute. I fail to implement the hinting on a service in v1.23.8, but this is urelated to this issue.
heh, don't worry, this topic is complex and we went through different solutions, you have more historical context on the KEPs I linked above and this thread in the mailing list
/kind feature The CNI "capability arguments" parameter - currently used to pass port mappings - has a few new capabilities: bandwidth limiting and network address management. The dockershim's networking driver should use them. The network address is just passing the PodCIDR dynamically to the plugin, so it no longer has to retrieve it from the APIServer. Bandwidth limiting is already implemented in Kubenet. It is configured via annotations. There is no upgrade risk in providing these values; non-compliant plugins will simply ignore them. Administrators need to explicitly enable each capability individually in their CNI configuration file. To avoid skew between runtimes, the other runtimes should also be upgraded to pass these. We may also wish to look in to a way to add this more directly to the CRI, so we don't have this skew problem, but that's a different issue.
/sig network
/assign
fyi.
Looks like this was never done for CRI?
cc
The desciption of --all-namespaces is listing the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace." But when I used the following command: error: namespace may not be empty when retrieving a resource by name no namespace set on resource pods "counter" The error messages are unclear, and the second message isn't labeled as an error even.
And why couldn't use --all-namespaces flag?
You're asking for a specific pod name (counter) in all namespaces, which is a nonsensical query. The pod name is not a grep-like filter, it's a specific name. Yeah, the errors could be better. Describe could work, but potentially generates a LOT of information and API calls.
But I found the following API is supported now. Get The result is: has the more information. Do we need to support the following kubectl command(using the above API) or just return clear error messages? get pods counter --all-namespaces
I'd be ok with this being implemented via the field selector.
The issue is still here :). What's our expected result? I'd like to contribute this.
, after check the code, the URL was built by ; do you mean we should update it; or avoid to use , instead, build url () directly.
After checking the code, it's better to add fieldSelector to , is created for fieldSelector. Will submit a PR based on .
getting an object with a particular name across all namespaces seems like something we shouldn't encourage... it breaks the assumption of namespace independence.
, so we'll only update message for this issue? for example:
Disagree. Need easy way to troubleshouting the problem "why pods is stuck in Pending status". It's tiring for every pod to find who created it and in which namespace. In the logs there is only the name of the pod.
Now, with field selectors we can do To further my understanding, I'd ask:- doesn't itself break independence? One can search by label across all namespaces, it feels funny for by name to be special.
What would you like to be : A unit test that verifies the evaluation order of the nodes in the scheduler. Why is this needed: The scheduler evaluates the nodes in a specific order until it finds a pre-defined minimum number of feasible nodes. To give all nodes a fair chance of being evaluated, in each scheduling cycle, the scheduler starts evaluating from where it left off in the previous cycle. This is captured by the parameter in  We need to add a unit test that captures regressions related to this behavior to avoid bugs like /sig scheduling /priority important-soon /help
/assign
/remove-help
What happened: The Conformance Document currently has a static header that is displayed from a file. The release information in the header should reflect proper release number so that the document reflect the current state. What you expected to happen: Should show proper current release information How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: <!-- DO NOT EDIT BELOW THIS LINE --/kind bug
in reaction to comments is still hardcoded to mention v1.9 at the top, could we parameterize this?
/sig testing /area conformance /release-note-none
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-lifecycle rotten
/priority backlog
This seems to have been addressed at Should we just close this?
No, specifically did not address this issue, this should stay open. See
Replace the azure auth plugin deprecation with a in v1.24 xref: /assign /milestone v1.24 /triage accepted /sig auth
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten /lifecycle frozen
/assign
/priority important-soon /kind cleanup deprecation
Is this a BUG REPORT or FEATURE REQUEST?: /kind enhancement /sig apps Description: is a key function of DaemonSet; it's better to have more unit test case for it, e.g. node status, pod on the node. There's several test cases to the DaemonSet's finial decision, which are kind of integration test to me; prefer to provide more unit test case for this key function.
GitHub didn't allow me to assign the following users: mathspanda. Note that only can be assigned. <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
do you have any specific cases or shall I figure out something? I can take a look at this issue if you don't mind.
, , thanks very much for our effort on this enhancement; as you known, DaemonSet will use scheduler's predicates to decide which node can place pods; there's several predicates errors , the purpose of this issue is to trigger every error in unit test for code coverage.
Sure, thanks. I'll cover these predicates.
Also I've found interesting issue when I wanted to add test for predicate. It looks like is not handled or maybe I did something wrong, I couldn't find check. According following comments it should fail because AWS EBS forbids any two pods mounting the same volume ID : Test case: Any thoughts? is it correct behavior?
Using this script: I'm seeing some random errors. 1 - running the above script it fails for me on the near the bottom when it tries to create . Notice that I had already successfully created which of the same type: Sometimes if I wait a while (like over a minute) then the two commands at the bottom work. 2 - At the top, if I comment out the commands and uncomment the commands, it fails with: Which is interesting, because when I give it a totally unknown type I get this: which leads me to think that it does know about the Org1Type but fails late in the processing for some reason. I said these errors we random, because there have been times when I would run this script and then go back later and try some of the failing commands manually and they would work - almost like there's a timing/async issue going on - but since I couldn't reliably reproduce those all I can do is just mention it but can't provide a solid testcase to show it, yet. Additionally, I've seen cases where the (for one of the instances) would fail and then I would try again and it would complain that the resource I'm trying to create already exists. I'm wondering if this is due to the original create actually working but then failing on some kind of verification step afterwards and that's the error being shown - but that's just a guess. I'm running these tests with PR but I saw seeing similar results even from master. ping
Any chance addresses this?
I'm pretty sure I tried that PR w/o any luck but let me try again just to be
nope, still fails for me using . On a related question... is it normal for thirdPartyResource types to not be visible right away? I find that I have to add a in there to wait for these new types to be known. I understand that some things in kube need/should-be async, but I'm not sure why the creation of these would be.
Hm, I'd have to look at the code, I thought it should be synchronous, but maybe not? might know.
It is not synchronous. It is picked up by the thirdparty controller. It can take a little while.
is something that can be changed or does it need to be this way?
That would take a lot more effort I'd say.
Cannot comment about the feasibility, as I have not looked into that option yet.
Also note that if you're in a HA system, you may need to wait until it shows up on all apiserver replicas before using it... , Uday Ruddarraju wrote:
Is there a way we can tell if that has happened? It seems like we need some better way to check than "hit it until it works, and even then, maybe it does not".
An option is that we can wait until that really happen.
Yes I would prefer for certain operations to wait until they're really done before they return but as asked, how can we know for sure in this case?
I see some options: wait to return until it's ready to be called. return some value on where to check for it to be ready. resource is stubbed out immediately, and returns some 'not yet ready' status until it is. knows enough to 'keep trying' until some timeout. I'm not sure what the intention is going forward if in fact core k8s APIs are going to be switched to using this mechanism. It can't be left in such an inconsistent state without specification. This behavior is not in the design.
IMO, kubectl can try to check some condition until some timeout, but we still need to make sure what's the condition, I would hava to take a look at the thirdparty resource code and see what we can do.
I think doing it on the kubectl side would be a mistake because then the REST API clients would need to replicate that logic. If there are certain operations (like, IMO, creating a new 3rd party resource) that should be blocked then I'd prefer if the POST itself blocked until it was done.
The logical here is that when you create a new ThirdPartyResource resource, it will return just after the resource is stored in etcd, but the process to register a new RESTful resource path and add a new api group is asynchronous, this is what thirdparty_controller will do. Basically put this wait in kubectl is wrong, this makes kubectl create complicate. As for real use case, users can use discovery client to verify that the api group is registered.
Third party resource installation is not meant to be flipped on and off often. The only use case for that is testing your third party bring up.
And there it's sufficient to just keep listing the third party data path until it returns a success value. For extra certainty, look up the endpoints of the kubernetes service and make sure it appears on every endpoint. The add is an async operation. It returns success because the 3rd party resource spec has been saved. I definitely think we should document the need for this wait better but I don't think we should actually make the control plane do the wait for you.
That's just what I mean, we should document this
fwiw, the async nature is intentional. the system is purposely de-coupled, there are lots of other examples of this in the system (e.g. creating a replication controller doesn't create pods immediately) It's much more robust this way (of course it does put a little more burden on the client side) I could see adding a utility function somewhere to wait, but honestly it's basically
yep, I get that and for lots of things the async nature makes sense. But I'm not sure it makes sense for all. For example, I think people would understand that it takes a while to spin up containers, but I'm not so sure the delay for creating a new resource type (which is mainly updating etcd) is so easy to understand. I think, at an abstract level, a lot of this goes to the UX. Sure, every client could do that 404-loop you wrote, but is that really necessary in all cases and is that the best UX we can offer? From a UX perspective I think there are going to be some operations that should appear to be synchronous even if under the covers its async and the server does the 404-loop for you (or at least by default and have a flag to disable it). And let's not forget there's nothing stopping a client from doing the async work on their side by running their cmd in the background, on a new thread/process, and then checking for that thread to complete - if they really didn't want to wait around. So, net, I think we need to look at this from a user's perspective and not a Kube-developer's, who understands what's going on under the covers. From a user's perspective, for operation "X" would I normally expect that to take some time? If so, then an async UX is ok.
But if I would expect "X" to be immediate (meaning once I get back the 200 I should be able to take the next action on that resource because I should have gotten back a non-200 immediately if there was an error) then exposing it as an async op might not be the best UX. IMO, creating a TPR falls into that category.
The fact that there can be multiple apiservers means it really is an async operation: you need to wait for each apiserver to get the watch notification and install the handlers. Operations with that sort of dependency are never synchronous in our api. Is it problematic to just do the wait loop, or just wait 10 seconds or something? How often are you adding new 3rd party resources? , Doug Davis wrote:
(I'm not arguing that we shouldn't make things easy for users, rather that we are working on other things that are even worse for users and this is less important since it should be used infrequently.) , Daniel Smith wrote:
Just as background on my current experimentation with TPRs: I'm using a single hyperkube env, which I think has just a single API server. In this work, which is actually part of an education class, we're walking thru a script that registers some TPRs and then creates instances of them to show students about the concept and how to use them. I'm finding at after each I have to add some kind of or , that's a pretty poor UX. I can certainly understand in a multi-api-server env there may be more complex issues involved, but if we can at least improve it for the simple case I think that would help a lot. I would also note that even a !404-loop isn't sufficient in the HA case since you may be getting luck w.r.t. which apiServer you're hitting, right? And yes, I know there are a ton of bigger fish to fry and I do appreciate the discussion. :-)
It's unfortunately not really easier to do it for the single-master case, since we have to support the multi-master case. If we add an exception for the single master case that adds complexity and actually makes the problem a bit worse for the multi-master case.