jiebi/CodeConvo huggingface dataset:
The benchmark is basically stress testing the new connection handling of IPVS. But I'm not sure we should be optimizing for such a thing, when in reality connections will be much longer lived (like with gRPC, or http/1.1+). It's a trade off between improved load balancing (connreusemode=1 or 2) and connection creation latency (connreusemode=0).
I have a php program use unirest to request other sevice in the cluster. It not use keepalive.
I would prefer to improve the latency number and I think the scenario where one client send lots of requests at the same time is not a common case and mostly happen in test environment.
It might be possible to improve this in the kernel too. You can see the change here: bypasses the reuse checks and expiring logic for new connections.
We set in IPVS proxier in case other people don't know what kernel parameters they should tune. You are familiar with IPVS and linux kernel, but I think many users don't have enough context.
, I think this was because the the container already deleted, but kube-proxy need time to flush the rules, iptables mode should also have this problem.
Yes, I will test preStop recently, may solve the problem
Here the likely reason for the slow throughput is described: As setting to causes , is any of the options listed possible to do? I guess only option 2 might be worth looking at as conntrack is probably needed for the remaining iptables components.
Hello everyone We are very fortunate to tell you that this bug has been fixed by us and has been verified to work very well. The patch is being submitted to the Linux kernel community. You can also apply this patch to your own kernel, and then only need to set (default) and (default). As the sysctl switch is newly . You can adapt the kube-proxy by judging whether there is , if so, it means that the current kernel is the version that fixed this bug.
That Can solve the following problems: update, IPVS keeps scheduling traffic to the destroyed Pod IPVS traffic scheduling after scaled up or rolling update IPVS low throughput issue second connection delay in masque low throughputÂ Bench can fill up ipvs service proxy in seconds 1s latency in when upgrading from on RHEL 8.1 -proxy ipvs connreuse_mode setting causes errors with high load from single client Thank you. By Yang Yuxi TencentCloudContainerTeam
Following-up on comment above for posterity. The patch mentioned above didn't make it to the kernel but there are two recently merged patches worth highlighting. One of them fixes the 1 second delay issue and the other fixes dropped packets when stale connection entries in the IPVS table are used: 1) 2)
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
sorry wrong issue.
looks like this was recently in #. can you take a look please?
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
This requires persistent storage. Is this flaky on master? Any guidance on how to diagnose this? I did run the e2e tests on GCE.
and is this really a p0?
it's happening pretty frequently, but I don't think we care about examples tests as much.
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
who can assist me with this? I cannot recreate it ... Command I ran. Results that I am getting. Just ran that off of head, and it worked. I am rerunning as the full test suite.
look through the logs, from last test cassandra-2 was stuck in pending which prevents the petset from starting up (c*-3 not even created) you see events like and logs like
so the test is already slow. I am only giving it one cpu, giving it less will make it super slow. Any recommendations? .25 CPU? And where did you find this?
[FLAKE-PING] This flaky-test issue would love to have more attention.
This is going to sit for a bit. With my schedule this week, won't likely get to it.
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
[FLAKE-PING] This flaky-test issue would love to have more attention.
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
https://k8s- Failed: [] [Feature:Example] [] CassandraPetSet should create petset {Kubernetes e2e suite}
[FLAKE-PING] This flaky-test issue would love to have more attention.
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If this may be security issue, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: When trying to mount an Azure File mount it fails, it looks as though its putting a new line between the name of the Azure Storage Account and the Azure storage domain. There are no new line characters in the secret that holds the account name. The following is the secret: The following is the declaration within the section of the Kubernetes yaml: The following is the error that's shown in `kubectl describe po/pod_name: What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: azure OS (e.g. from /etc/os-release): Ubuntu 16.04.3 LTS Kernel (e.g. ): 4.13.0-1011-azure Install tools: Others:
/sig storage
/sig azure
It seems to me that does decode to , so it seems to be behaving correctly. Maybe it's a good idea to trim the return char for and fields after ?
is specified by user, so trim the is not a good idea in the code. Another simple way is to use kubectl create secret to create azure-secret, the accountname and key won't require base64 encoded, so you won't mess up with let me know if you have any question, thanks.
Thanks !
Bumped into this issue today too on our cluster. Can confirm command worked. Is there an easy way to time the /n off the end of the echo for before using base64? Thanks.
yes it's
Of course it is! Doh!
i think would've made better sense to trim the azurestorageaccountname value when receiving it... anyways the storage account name wont have n in them right? I think that just made better developer experience...
is not allowed in azure storage account name
so if azure storage account name does not allow in the name.. then why not just trim the name instead of error... Because devs would just and that would already have the n in the resulting base64-encoded string... right?
sounds reasonable /reopen
Reopened this issue.
<detailsIn response to : sounds reasonable /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
awesome. Thanks
so then this goes in to kube version 1.14.6 in AKS ? or how does this work?
It would only go into k8s v1.16.0 since it's not a critical bug, we don't want to do hot fix to the old release.
got it. thanks Andy.
Is this a BUG REPORT or FEATURE REQUEST? (choose one): bug Kubernetes version (use ): Environment: Cloud provider or hardware configuration: N/A OS (e.g. from /etc/os-release): RHE 7.2 Kernel (e.g. ): What happened: The node resource is empty when start local cluster by . What you expected to happen: There's one node in this local cluster. How to reproduce it (as minimally and precisely as possible):
Just check the script/code, it seems kubelet will not register until dynamic config is there ( enabled all alpha features by default).
I met same problem
What do you mean "dynamic config" ?
I think there's a PR that make kubelet can update configuration dynamically, similar to what we had done in scheduler. But did not find PR :(. I'm checking the code.
Should be this one: /cc /sig node
that shouldn't be the case. As long as you can get a client and a nodeName you should be able to register. What specific error are you seeing that suggests it's dynamic config? Also why does local-up-cluster enable all alpha features by default? That seems wrong to me.
Not sure why, but it really does:
I think this should be caused by /cc
, sorry for the disturbing :). Regarding all alpha features, I prefer to enable is by default: local-up-cluster is a kind of test environment for developer, enabling all alpha features maybe helpful to catch up potential feature interactions.
I would not expect all alpha features on by default in local up cluster
Neither would I. Developers, by default, shouldn't have to wonder if bugs are their fault or if they come from unstable alpha components. Users, by default, shouldn't have to deal with unstable alpha features either.
/ , got that; alpha components is unstable, do NOT cover everything for it.
met same problem
This shipped in 1.22 as deprecated in 1.22. As a hacky fix to unbreak tests on 1.23 dev, I bumped the deprecation policy one release: We will need to revert that before 1.23 ships and figure out a better solution. /kind cleanup /milestone v1.23 /sig node
/triage accepted
I want to try it. Can I take this issue?
I have explained how to fix this on PR:
/reopen reverted in as fix did not work
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
Hi, Im the bug triage shadow. Since we're in 1.23 milestone code freeze, could we move this issue to milestone 1.24?
this is critical for 1.23 and I think it just got forgotten... we need to get this fixed before test freeze
/priority important-soon
What happened: From : What you expected to happen: Metrics should register successfully. How to reproduce it (as minimally and precisely as possible): Run kubernetes-integration test Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
There are no sig labels on this issue. Please add a sig label by either: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. See the . The in method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/kind failing-test <!-- If this is a flake, please add: /kind flake --/sig network seems to be related.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/triage accepted
According to the : "The scheduler bypasses any nodes that don't have any topologySpreadConstraints[].topologyKey present." However, the is:  "The scheduler bypasses any nodes that any don't have all topologySpreadConstraints[].topologyKey present." This is a bug as the implementation is not following . We expect the code implementation to align with the documentation's description. Read the documentation and the code. /sig scheduling of the on the branch, and of the on the branch <detailsNA </details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
pr in case documentation is upheld
I dont't think the implementation of func is not as expected. Perhaps we can optimize the document a bit. /cc
I don't understand what the issue is trying to point out. -return true if the node has all required . -bypasses a node that don't have any topologySpreadConstraints[*].topologyKey. Nothing looks wrong?
I also think there is no problem with code logic. According to doc, if one node only have one label that equal to , this node should not be bypassed. This sentence may be ambiguous. In fact, a node with labels that match all should not be bypassed by scheduler.
So, where's the ambiguousness of the doc? Yes, and it matches the doc's explanation.
This is where there might be ambiguity. means that if node only have one matched label, this node will not be bypassed. But in fact the node labels must match all(not one of) topologySpreadConstraints[].topologyKey. Maybe I misunderstood it as well because English is not my native language. Ignore me if I'm wrong.
Okay, I got what you wanted to say. This is... an interesting English question.
For example, the sentence "I don't have any of those books"; I'd interpret it as "I have no book; I don't have book-a, don't have book-b... don't have book-z". On the other hand, the sentence "If any of the following conditions are not true, the alert is triggered", I'd interpret it as "if condition-a and b is true, but condition-c isn't true, the alert is triggered". I interpreted the topology spread's doc like the second one; if any of topologySpreadConstraints[].topologyKey aren't in the Node's label, the node is bypassed. I'm not sure which interpretation is natural for the native speakers (looks like ChatGPT has the same interpretation as me though), but at least we can say it's ambiguous. I guess it'd be clearer if we rephrase it to, for example, "The scheduler bypasses any nodes that is missing at least of of the required topologySpreadConstraints[].topologyKey in its label.". Feel free to raise a PR. /remove-kind bug /kind documentation
I agree that the sentence is a little ambiguous. But this is not a bug in the code. The implementation matches the original intent of the design. I suggest opening a bug in /website so that SIG Docs can advise on better wording.
/close
Closing this issue. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Thanks for your swift response! We chose to post this here because we were unsure about the expected behavior and wanted to hear developers' opinions. Now that we have clarity, so we will open a bug issue in the /website.
It would better to have e2e for iSCSI persistent volume. /sig storage
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-testing, kubernetes/test-infra and/or . /close
/kind bug What happened: kubectl became very slow, taking 30s to list pods etc. After going deep down the rabbit hole of debugging the controller components, I ran kubectl -v=8 which showed: What you expected to happen: Since caching is required for kubectl to respond in reasonable time and the issue isn't obvious, kubectl should at least print warnings in default log level. How to reproduce it (as minimally and precisely as possible): I got into the situation by running kubectl with sudo, but just chown'ing should be enough to reproduce this. Anything else we need to know?: Environment: kubectl v1.8.3
/sig cli
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale /lifecycle freeze
Seen in issue Logs from : At the very least, we should retry godep restore a couple times given the heavy network dependencies it has. The network dependencies are sorta necessary... but they change rarely so we could also do some clever caching of unmodified clones that we put in place ahead of time to dodge network traffic outside of new deps being . cc
We could probably do a single retry and avoid most flakes. Godep restore takes a long time though, so if we retry too many times we'll run into timeout issues (but maybe that's fine). See also , tools/godep
Actually, it looks like this is only being done for an excluded verify step (). We should not do this for an excluded step, and we should not do for every PR.
can you explain why staging/ needs to function?
verify- also runs :
eventually gets published to . IIRC, it needs because otherwise a later fails ().
If we can run the verify script in order, then we can run verify-staging-client- after verify-, then we can save one .
verify- exits out early if Godeps/ or vendor/ hasn't changed.
Hmm, ok. How often will fail?
I'm seeing ~.5% failure rate over the last few days, and averaging ~2 minutes for the godep restore. Do we have to check in _vendor for the staging client code at all?
We run to find all the dependencies of clients, including k8s packages, so _vendor is not the reason we need .
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --Patching a Kubernetes service failed with I the following yaml documents Create an external name service Change to to an ClusterIP Service Try to change it back to an external name service Kubernetes version (use ):Cloud provider or hardware configuration:
/sig node /sig api-machinery
Unsure if Service is part of networking /sig networking
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : Unsure if Service is part of networking Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network
/remove-sig node
/cc /triage accepted
It is part of networking :), actually this part of Services is being revisited these days /assign
ack.
/sig network
Please don't triage/accept issues unless you're SURE they are your sig's. This fell through cracks.
/assign
this does not repo on: Here is what i did: service (the same one above) the service and change to =OK the service and change to =OK I have also performed step 3 using and the result is also OK /triage needs-information
I was using client v1.21.0 - is it resolved in newer kubectl 1.21.x patches?
please do. And also please share the exact steps you used were you using kubectl edit? or replace? or patch?
It's in the description - kubectl apply
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /triage accepted - working on a fix. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/triage accepted working on a fix
to confirm does the merged PR allow you to now change the type from to ? Or does it still produce an error with the difference it is reported properly vs. INTERNAL_ERROR? If it's the former can we cherry pick this PR back to supported K8s versions?
MetricsGrabber tests failing when scheduler & controller-manager not visible. In managed offerings, like GKE, scheduler and controller-manager are not visible causing BeforeSuite of MetricsGrabber to fail. Test is failing then control plane pods can't be found (as on GKE): HasControlPlanePods was in /assign
/sig instrumentation
Just removing this check allows tests to run without problems (some skipped, one passing).
/triage accepted
But it doesn't have the Godeps/ folder. add the Godeps/. Ideally, we should not let the IntOrString pull in so much dependency to
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <bre.g., for API Machinery<br(2) specifying the label manually: <bre.g., for sig/scalability<br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list