jiebi/CodeConvo huggingface dataset:
In our Tanzu Supervisor Cluster k8s setup, we observe that the "should verify that a failing subpath expansion can be modified during the lifecycle of a container" conformance test fails. The root-cause seems to be that the test is updating the pod annotations, thereby resulting in removing the other annotations present in the pod object. Removing the custom annotations that we use causes the test to fail during pod deletion. Should this test perhaps use Patch instead of Update so that the annotations are merged and not replaced? Thanks ginkgo.By("updating the pod") podClient.Update(pod.ObjectMeta.Name, func(pod *v1.Pod) { pod.ObjectMeta.Annotations = map[string]string{"mysubpath": "mypath"} })
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig storage /sig conformance I personally agree that it should not remove existing annotations, but better ask the test owners
The label(s) cannot be applied, because the repository doesn't have them <detailsIn response to : /sig storage /sig conformance I personally agree that it should not remove existing annotations, but better ask the test owners Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
May I request the original author to weigh in on this? Thank you.
I dont have any issues with the code change to a merge but what would bother me is that conformance tests are being altered by outside influences which doesnt seem particularly right. I think this would be for the sig storage owners to decide
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at . /lifecycle stale
WRT , container registry credentials should be included in a docker file instead. We deprecated for this reason in , but is more tricky, because the file it references is not static. We need to figure out what the alternative is for dynamically providing pull credentials to nodes. /cc
I think we need to enable Managed Service Identity (MSI) for all instances before deprecating . To do this, we also need to grant VM access to Resource Group in Azure IAM (e.g. config Azure AD in acs-engine). WDYT?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
I think this is still active. Right, /remove-lifecycle stale
Thanks, this is still active
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
-- What's the status of this?
Sorry, no progress yet. Any ideas on this?
-- pinging again on this one.
We're not making any progress yet in this release. Still seeking the possible solutions for non-MSI cases, secret is one option for this. /milestone v1.15 Related and
None one is working on this. Let's move this out of 1.15 /milestone v1.16
thanks. This missed v1.15 feature timeline. LGTM of moving to v1.16.
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?
/milestone clear is preparing a KEP for this, but it's still not ready yet.
Will follow this KEP for implementation
This should be addressed by /milestone v1.18
Hello ! Bug Triage team here for the 1.18 release. This is a friendly reminder that code freeze is scheduled for 5 March, which is a few weeks from now. Is this issue still intended for 1.18? Thanks in advance!
C/O Code freeze for release 1.18 is starting in 5 March (less than 1 week from now) and we want to ensure that each issue has a chance to be finished on time. Is this still on track for this deadline?
/milestone v1.19 The work on the KEP hasn't started yet, so moving this to 1.19
Hello from Bug Triage team! Wanted to follow up on this issue with a friendly reminder that the code freeze for 1.19 is starting June 25th (about 4 weeks from now). As this issue is tagged for 1.19, is it still planned for this release?
/milestone v1.20
Hey Bug Triage here. Wanted to follow up on the status of this issue as we're approaching code freeze on 12.11.2020. This issue is tagged for 1.20, is it still planned for this release?
Hi Bug Triage here, seeing that there is not much recent activity here, and we are pretty late into code freeze, I am going to remove the milestone label. /milestone clear
/triage accepted /priority backlog I think we would be able to remove the flag once external kubelet credential provider promoted to GA
/kind deprecation
This issue has not been updated in over 1 year, and should be re-triaged. You can: Confirm that this issue is still relevant with (org members only) Close this issue with For more details on the triage process, see /remove-triage accepted
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available .
https://k8s- It's quite flaky.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
/assign
/sig networking
There are two failures in gci-gce-slow, so it's not specific to gke. https://k8s- Also in gce-slow: https://k8s-
/sig network
This is strange, seems like this test is trying to reach the test-netexec pod through servicePublicIP:serivcePort, but the request hit the gce default backend instead. cc
There is another type of failure: Seems to be the port collision between test service and fluentd daemonset, which changed to use few days ago ().
cc
Checked the kube-proxy logs and found this is not the case. Port was not allocated (although that could happen in theory) to any service during test:
This turned out to be an interesting corner case. It happens at the beginning after we set up the GCE network LoadBalancer for OnlyLocal service --- when the LB still forwarding traffic onto nodes that don't have endpoints on. For more context, when network LB forward traffic to nodes, packet arrives node with below src:dst: If no endpoints are on the node, and there happens to have a pod using hostPort with the same port number, packet will be DNAT to that pod IP instead. These are observations and I haven't traced down to the exact iptables entries. I'm changing service port in test to avoid collision for now. cc
At least on AWS, if the clock drifts 300 seconds (which it will), then we get the rather cryptic message: This is because AWS includes the timestamp in the request. Installing ntpd would stop this drifting from happening. If the drift does happen, the likely symptom is that EC2 calls would stop working: attach/detach volumes on the minions, instance sync and routing configuration on the master. I don't believe that GCE has the same problem.
From we can find that, hardcode "private" and "public", seems only support that network with such name. , it fetch the first ip. usually, it can include both v6 and v4 ip. what's the meaning to just fetch the first one ? is it v4 or v6 ?
ping and
Yes. Zoom out a level or two in the code, and you'll see we're trying to either a) setup an openstack loadbalancer to point to the minion node (a single address) using , or b) report to kubernetes all the addresses we know about the node and whether they are "internal" or "external" in . All we have at the starting point is the server name (from kubernetes), and we need to make up the rest of this information somehow. (I agree (a) should reuse the NodeAddresses() function from (b), but the latter is a newer function and the code hasn't yet been refactored in that way. This doesn't really change the substance of this discussion.) Re (1): The logic as currently implemented is: Assume any network named "private" is "internal" Assume any network named "public" is "external" Assume the AccessIPs (IPv4 and IPv6) are "external" Any other network addresses are ignored. I implemented the first two ("private" and "public") following the Rackspace public cloud precedent, and on the basis that users in other OpenStack clouds can use these names if they want to encourage an address other than the Access addresses. For the (a) use case, we just take the first address we find in order of the above; for (b) we report that full list. I preferred "private" to "public" since (again, at least on Rackspace public cloud), the traffic on "private" network is sometimes cheaper/free. I agree this isn't great, and please suggest something better if you have a use case in mind. Perhaps we should pull a list of network names that should be considered "internal" and "external" from the provider config? Perhaps I could assume that provider networks are public and non-provider networks are internal? I don't want to add too much configuration here if there's a good algorithm that will Just Work.
I think in practice it's probably ok to continue to prefer "internal" to "external" for loadbalancer setup, but I can contrive hypothetical cases that need more flexibility here too. Again, if you have a concrete need for such a thing, please file a bug so we have some idea of the flexibility required. Complicating all this is that Kubernetes never actually defined what an "internal" address was. just says "Generally the IP address of the node that is routable only within the cluster" which isn't a sufficiently strict definition. Looking at where Kubernetes actually uses it: just uses the first address reported (preferring InternalIPs to ExternalIPs) and the Kubernetes master configures an "SSH Tunnel Check" which just uses the first ExternalIP ... which kind of makes it pointless to ever even bother reporting anything beyond the first address. In both cases, it's assumed that this address is reachable by the Kubernetes master nodes, which makes the Internal vs External distinction a bit pointless. Re (2): is meant to return more than one IP, and the loadbalancer setup can only use a single IP for each node (or would unnecessarily track/healthceck multiple paths to the same backend for multiple addresses). If you're seeing single addresses returned by then you should file a bug. All the above treats IPv4 and IPv6 as equally important, if the OpenStack cloud has both, and the "first" address used for the loadbalancer will be an arbitrary choice between the two protocols (not quite true: will use AccessIPv4 before AccessIPv6). I think that's how it should be (the rest of the code should be protocol agnostic), but I wouldn't be surprised if there's some part of Kubernetes that doesn't support IPv6 properly. There certainly used to be lots of places that assumed IPv4 in Kubernetes, and GCE doesn't support IPv6 iirc so it gets less testing - if you've found something that breaks with IPv6, please file bugs. Now this bug: what would you like done here? Would you like the current situation to be documented further? Is the "private"/"public" naming thing too restrictive for your usecase? Something else?
Hi, if we talked about openstack, I expect it not bind to rackspace, although, the kubernetes called RK SDK, But from my understanding, seems RK itself SDK not bind network to "private" and "pubic". User can define public network with their names, like site-network, for example. or "testnet", And for private, it should be seem. For nova list, the addresses, it can have "OS-EXT-IPS:type":"fixed" or "OS-EXT-IPS:type":"floating" for floating, I think what you means "Public", for fixed, I think its what you means "Private". In other words, we'd better to find related information from other fields in nova or neutron infromation. Instead of bind to "private" and "public" name.
I this issue and wrote a patch that implements what suggests. I will send a PR soon
Glad to see some improvements here. And wish to use it if implemented
I am trying to bring up a kind cluster with containerd in order to run e2e test on mac + kind. Not sure whether this is a bug or I missed some prerequisites. Any help would be greatly appreciated :) building the image dra/node:latest works as expected:bringing up the kind cluster failed (seems that kubelet is unhealthy and healthchecked failed) below the control plane logs: below the logs of one worker: successfully bring up the kind cluster with containerd Tried on M2 macbook pro I followed NONE Darwin MacBook-Pro-de- 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:19:12 PDT 2022; root:xnu-8792.41.9~3/RELEASEARM64T6020 arm64 arm Darwin No response No response
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
what type of mac you're using? for m1/m2 mac, you need to replace the linux-amd64 with linux-arm64(your log seems like using m1/m2 mac which same as the problem I've met before)
thanks I am going to try that
ping me if you can run e2e on mac. I setup the cluster but failed to run the e2e test..
can you please share the exact command you tried and the failure ?
If you use a newer version of kind you might not need to clobber the included containerd, not sure what version is required And yes, on M1/M2 you need to use arm binaries/images.
this is the command that I tried !
It worked for me just fine on M2 machine with this obvious fix for the build script: Here is a test output:
I'll submit a PR with the fix. Updating the documentation would be also nice to have. I hope and can do that.
DRA tests fail on kind 0.18.0. Which Containerd version is used there?
It was the latest 1.6.x at release time. 1.6 is their LTS branch before 2.0 comes which has some tricky changes for us to deal with (notably the CRI hosts config deprecation which will break all our local registry users :/)
Unfortunately 1.6 doesn't have CDI support, which DRA depends on. The version used in the build script is 1.6.0-830-g34d078e99, but it was probably built from containerd master after 1.6 was released.
thanks for providing the details. you could run the e2e as shown by and find the result
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When starting pods with AFS volumes, there is an initial delay of five seconds until the pod is transitioning from the "Scheduled" state. The reason for this is that currently the volume mounting happens inside a wait.Poll which will initially wait a specified interval (currently 5 seconds) before execution. Event log for comparison with and without AFS volume attached (seconds since event): What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /assign /priority important-soon /sig cloud-provider /area provider/azure
see
I've been pulling my hair out trying get kubectl talk to my local cluster. It seems that kubectl merges the local file with values pulled down from the cloud. I've done the following: The I just set is silently overridden. Questions: is this documented anywhere? should kubectl merge in my GCE cluster when I set my cloud-provider to something else? should kubectl override my previously set current-context?
workaround is to use flag to override the config file value.
but not a good UX long term
Someone else hit this as well: . Better fix is to have the local-up script set a context, which I will try to add today. As for the overwriting behavior, it's documented in , but based on what you describe, the intended load order is not working; local .kubeconfig should overwrite global. cc
That sounds infuriating. I'll take a look this morning.
After this () has been merged and EnpointSlice feature is now enabled by default, we noticed Kube-proxy will not work on Windows. Only message I can see in the logs is: /sig windows
/cc /cc /cc
How about separating the above check into two if statements so that the log can pinpoint which field is false ?
It seems long term, EndpointSlice support should be so that is kept in sync with other implementations.
cc
Thanks for reporting this! Windows proxiers were still intended to be reading from Endpoints, but apparently that's not working as intended. EndpointSlices are moving back to disabled by default for 1.17, so that should fix the failing tests for now. I'll work on an actual fix for this as well, but it may not make it in until after KubeCon. /assign /sig network /priority important-soon
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry. ‍
/remove-triage unresolved
/assign cc
GitHub didn't allow me to assign the following users: jocelynberrendonner. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign cc Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
: Please let me know if you need any help on the fix! I'll be more than happy to review/help test the fix on Windows.
is sent as a fix. cc It's a test failure reported by the publishing robot.
The line it fails at:
It looks like a vendor copy vs gopath copy issue.
One possible fix is removing from the vendor/ directory. I'm testing locally.
Confirmed removing from the vendor/ directory fixed the test in robot.
We need to add the "issue filer" feature to the publishing robot, we can't rely on to report robot issues everytime :P I'm just not sure how long the publishing robot will be needed (i.e., when we will get rid of the staging/ directory) so not sure whether we should invest more time on the robot.
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: patched out the Size field (was PR ), it will cause issue again. See also and . What you expected to happen: PVC resizes of Cinder volumes successfully. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/kind bug /sig storage /sig openstack
/assign
We have a new vanity url, , which redirects to the GCS release bucket - e.g. Right now, the https endpoint doesn't work, since isn't in the list of domains on the certificate. Once it works, we should switch all references over to the nicer vanity URL. x-ref
So what it required to make this issues actionable? Where to add this for the certificate?
the issue tracking that is
This needs to be triaged as a release-blocker or not for 1.5
not a release blocker.
Is it appropriate to move this to the next milestone or clear the 1.5 milestone? (and remove the non-release-blocker tag as well)
can this be closed?
I forgot to get to this in 1.6, but it'd still be good to do.
do you want this in v1.7? is there a sig to own this?
doesn't need to block 1.7, no.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list .
/sig release
Our GKE scalability jobs (https://k8s-) are failing in the cluster teardown step (even though the e2e tests themselves passed). E.g failures: https://k8s- https://k8s- Seems like it's caused by some leaked network-related resources: /assign Krzysiek - Could you PTAL into this when you find time? It'll be great to have our GKE suite back to green. cc
I think this is related to an earlier bug we've seen (similar one for GCE): Basically service-controller is creating some cloud resources (like firewalls) for lb-type services which are not brought down properly when the k8s cluster is brought down.
Also another related issue - There should also be an internal bug for this somewhere. IMO we may be able to fix this in test-infra (without waiting for proper long-term fix on gke side).
I am adding clean-up method. This way all of the firewall rules will be cleaned up after tests. If this method has to remove any firewall rule, the teardown phase will return exception. It's not a fix for the bug - it only allows us to easy isolate tests leaking firewall rules.
Tests: should have session affinity work for LoadBalancer service with ESIPP on should be able to switch session affinity for LoadBalancer service with ESIPP on should have session affinity work for LoadBalancer service with ESIPP off should be able to switch session affinity for LoadBalancer service with ESIPP off leak resources related to the LoadBalancer. PR will fix this.
From After , Multi-AZ Clusters tests in 1.19 and 1.20 started timing out as createBalancedPodForNodes is extremely inefficient in scale: for each node (out of 5k), it lists all pods in a cluster serially (each list takes ~5s), which translates to ~7 hours. Optimizing this function by listing pods once and then in memory by Spec.NodeName /kind bug /priority critical-urgent /sig scheduling
/triage accepted
/assign
/reopen Looks that ComputeCPUMemFraction took only 12 minutes, but it stuck later, on It is scheduling one pod per each node: Which is waiting 5s to be ready: 5k nodes * 5s = ~7h
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign Can you do the pod creations in parallel?
Yes, good idea. I'll update to do that