jiebi/CodeConvo huggingface dataset:
Code change:
}, }, }, <del> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "udpTimeout", "udpStreamTimeout"}, </del> <ins> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "tcpBeLiberal", "udpTimeout", "udpStreamTimeout"}, </ins> }, }, Dependencies: []string{

Related discussion:
text: This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close [1]:
Closing this issue. <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Hi, are you still working on this? Same problem happened in our clusters, just because of asymmetrical routing.
/reopen
Reopened this issue. <detailsIn response to : /reopen </details
Also the command suggested by is helping us, but I still think this should be a bug of kube-proxy.
The problem here is: what happens if the solutions are exclusive or we regress? The addition of this rule fixed a legit bug that was documented and there is also a nice blog explaining the details Removing this rule without a repro and understanding what is the root cause of this issue is not really the best way to proceed. I think that people affected should try to find the root cause and explain the problem or just provide a way to reproduce it, so we can fix it
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at .
Code change:
}, }, }, <del> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "udpTimeout", "udpStreamTimeout"}, </del> <ins> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "tcpBeLiberal", "udpTimeout", "udpStreamTimeout"}, </ins> }, }, Dependencies: []string{

Related discussion:
text: /close [1]:
Closing this issue. <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue or PR with Mark this issue or PR as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The same problem happened in our cluster, the same as because of asymmetrical routing. the command suggested by doesn't work for us, and I delete the iptables rule and add new rule(refer to ), it works I think that makes sense I'll put up a reproduction of the gist link later on /reopen
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The main problem is how to be completely sure we don't regress, this comment from Tim is spot on
I'm trying to check that non-local connections will still reset without the INVALID Rule
I think the patch in is wrong; it changes the rule from "if INVALID then drop" to "if INVALID and from a pod, then drop" so that it doesn't affect non-kubernetes traffic, but I think the rule we actually need is "if INVALID and to a service, then drop". In particular, I believe we need to make sure that the rule applies to traffic with a cluster-external source and an in-cluster destination. It's possible that changing it to "if INVALID and to a pod, then drop" would also work. I skimmed and some linked bugs elsewhere but I was unable to confirm whether that rule would cover the reporters' use cases.
I think that is only INVALID and to a pod
In my case, cluster-external source to service is both ok with the INVALID rule or without the INVALID rule. It seems the rule doesn't affect on access service.
Code change:
}, }, }, <del> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "udpTimeout", "udpStreamTimeout"}, </del> <ins> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "tcpBeLiberal", "udpTimeout", "udpStreamTimeout"}, </ins> }, }, Dependencies: []string{

Related discussion:
text: I add this rule but it doesn't work
So, based on that, the bug happens when a reply packet fails to get un-DNAT'ed, causing the client to then send a RST to the server, which matches the source/destination IP that the server knows of. But this could happen if you had a service with external IPs too; in that case the reply packet would be from the external endpoint IP to the node IP (because the packet would have been masqueraded), and if conntrack didn't unmap it correctly then the node would send a RST to the external IP, breaking the connection in just the same way. In this case there is no good way kube-proxy can recognize the packet as being its own, other than by using conntrack, which presumably won't work if the packet is INVALID?
It seems that there is no better way to solve this for external sources accessing the cluster CIDR, is it right? If so, Can we have a flag to decide whether to create The invalid rule? It would be nice that It's up to the user to decide whether to create the rule.
There's related discussion in . I think we should create whatever iptables rules we can that will fix up our own traffic without interfering with other traffic, and also document the sysctl that admins can use to change conntrack behavior to work around the problem. So probably, we should change the "drop if INVALID" rule to "drop if INVALID and to a pod", and also address the problem in , but not try to solve the problem with external IPs I mentioned in because we don't have a good way of fixing that without potentially breaking other things. (And it's kind of way more of an edge case anyway.) Note that can currently output rules for "from a pod", but not for "to a pod", so that will need some tweaking. (FWIW will also eventually need the ability to output nftables rules rather than iptables rules so that's another thing to think about if someone is going to change its API.)
So in the absence of a way to fix the problem with external IPs, can we have a flag to decide whether to create this rule or not? If true (default), create the drop if INVALID and to a pod rule. Otherwise, We do not create the rule.
Code change:
}, }, }, <del> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "udpTimeout", "udpStreamTimeout"}, </del> <ins> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "tcpBeLiberal", "udpTimeout", "udpStreamTimeout"}, </ins> }, }, Dependencies: []string{

Related discussion:
text: I believe that even without this rule we can solve this problem by way of sysctl, right? I think we should do the following things We need a flag to disable the rule We should change the "drop if INVALID" rule to "drop if INVALID and to a pod" We document sysctl as a way to solve the problem.
If the rule is "drop if INVALID and to a pod", we shouldn't need a flag to disable it; it won't affect non-pod traffic, and it won't interfere with users that want to set the sysctl instead. (Setting the sysctl will cause packets to not be marked as INVALID so the rule we add just won't get hit.)
OK, drop if INVALID and to a pod is help us identify cluster traffic, right? We only drop cluster traffic if it's invalid, but this will not solve the problem of external IP. So we need a flag to disable the rule is drop if INVALID?
The same problem happened in our cluster, the same as because of asymmetrical routing None of the above methods solve the problem in our ENV. Are there any other workarounds?
Can you execute the following command on each node? I think this is a workaround.
Thank you for your reply I also tried removing this rule. However, k8s will reconfigure the deleted rules to the system at some point in time.
I worded that badly: I meant "external IP" as in "the endpoints are cluster-external IPs rather than pod IPs", not as in . So that would apply only to people using kube-proxy to load-balance traffic from clients outside the cluster to servers outside the cluster, which is a pretty weird use case. And nobody ever reported the packet RST bug for that case anyway. So I was arguing that we can just ignore that case. So: change the existing "drop if invalid" rule to "drop if invalid and to a pod" document the sysctl We don't need an option to disable the drop rule, because it won't affect non-k8s traffic, and it will automatically become a no-op if someone decides to use the sysctl instead.
Thank you for your response. I'd love to take this one. But I have a question, How to define k8s traffic?
Code change:
}, }, }, <del> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "udpTimeout", "udpStreamTimeout"}, </del> <ins> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "tcpBeLiberal", "udpTimeout", "udpStreamTimeout"}, </ins> }, }, Dependencies: []string{

Related discussion:
text: If a pod in a cluster, we call it an underlay pod, its IP is not allocated from cluster CIDR, this is a real underlay IP, when it accesses other underlay pods in the cluster or pods that allocate IP from cluster CIDR(We call it overlay pod), is this k8s traffic? In our cluster, multiple types of CNI pods are implemented through multus running in a cluster, because of the problem of asymmetrical routing and the rule that makes them inaccessible, it seems that the above method will not solve this problem.
For kube-proxy, this is done by the . If you are in a cluster where there is no simple way to discriminate pod from non-pod traffic, then you'd have to run kube-proxy with , and then kube-proxy will see that returns and so it won't try to distinguish local from non-local traffic, and will do some things (like internal/external traffic policy) slightly differently as a result. So in this case, if there was no functioning localDetector, it would also not create the "drop invalid" rule
Thanks for the details, this is probably the case we want.
/assign
I did not understand how you could get asymmetric routing in K8s, but reading up more carefully I saw: If I understand this correctly, this is a load-balancer outside K8s that load-balance directly to POD addresses. To get asymmetric routing some kind of Direct Server Return (DSR) must also be used. So, to get asymmetric routing: from an external source is directed to an external LB LB is configured with POD-IPs as targets and selects one packet is routed to a K8s node, e.g. with ECMP. But is may not be the node where the target POD is executing packet is handled by the POD, but the response packet may be directed to some other GW This will give asymmetric routing. But it's not the every-day K8s network setup for sure. Given the above it's fairly easy to reproduce the problem. You don't even have to setup a load-balancer. The biggest problem may be that you need two ways in to the cluster. Example: ! I setup direct routes that ensures asymmetric routing, which can be checked with on vm-201 and vm-202.
Code change:
}, }, }, <del> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "udpTimeout", "udpStreamTimeout"}, </del> <ins> Required: []string{"maxPerCore", "min", "tcpEstablishedTimeout", "tcpCloseWaitTimeout", "tcpBeLiberal", "udpTimeout", "udpStreamTimeout"}, </ins> }, }, Dependencies: []string{

Related discussion:
text: From I can now directly ping POD-IPs, but due to the problem in this issue, I can't make a TCP connection: The POD with address runs on . A similar setup as above can be used to verify a PR. Now, there is a simple work-around: use proxy-mode=ipvs The update with the rule never made it to the ipvs proxier (a rather common case :cry: ), but in this case it may save your day. The same as above with proxy-mode=ipvs; The server on 5001 responds with the POD name.
Yeah, Our production environment also solved this problem by switching to IPVS mode. In IPVS mode, there is no the DROP rule.
The sysctl solves the problem in , but not the problem in this issue. Those problems are kind of related, but rather distant.
The same affects outgoing traffic if packets are not masqueraded to the node-ip, as should be a common setup for ipv6, please see For Calico I think you can setup BGP for clusterCIDRs and route incoming packets to PODs to the correct K8s node, instead of using ECMP. This would solve the problem, and save the extra hop.
I met the same issue. KUBE-FORWARD drop my packages.
temptatively solution
The issue fixed by , I'll close it, you can feel free to open it if have any questions. /close
Closing this issue. <detailsIn response to : The issue fixed by , I'll close it, you can feel free to open it if have any questions. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Code change:
k8s-app: fluentd-es kubernetes.io/cluster-service: "true" version: v1.22 <del> # This annotation ensures that fluentd does not get evicted if the node # supports critical pod annotation based priority scheme. # Note that this does not guarantee admission on the nodes (#40573). </del> <ins> # This annotation ensures that fluentd does not get evicted if the node # supports critical pod annotation based priority scheme. # Note that this does not guarantee admission on the nodes (#40573). </ins> annotations: scheduler.alpha.kubernetes.io/critical-pod: '' <ins> scheduler.alpha.kubernetes.io/tolerations: '[{"key": "node.alpha.kubernetes.io/ismaster", "effect": "NoSchedule"}]' </ins> spec: containers: - name: fluentd-es

Related discussion:
text: Problem showed, that despite e2e tests for container's logs, we still can encounter regressions for system logs. We need to establish an e2e test for ingesting system components's logs: kubelet docker CC
broke ingesting logs from master and we didn't notice it at once Test should check that logs are present from all nodes, including master (when possible)
Moving to 1.7 as late to happen in 1.6. Feel free to switch back if this is incorrect.
/sig instrumentation do you want this in v1.7? (since you're listed as sig-instrumentation leads at and I can't find a team to ping)
No, thanks for noticing!
Fixed by
Code change:
metrics = []resettableCollector{ deprecatedRequestGauge, requestCounter, <ins> longRunningRequestsGauge, </ins> longRunningRequestGauge, requestLatencies, responseSizes,

Related discussion:
text: apiserverlongrunninggauge and apiserverregisterwatchers are kinda redundant . We discussed this in our SIG instrumentation meeting and while it's clear that apiserverlongrunninggauge, the name does not conform to instrumentation guidelines (it was created before those guidelines existed). What we want to do is deprecate both metrics and create a new one which actually does conform to our naming conventions probably something like . has volunteered to undertake this. /cc /sig instrumentation
/assign
GitHub didn't allow me to assign the following users: yan-lgtm. Note that only , repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
/triage accepted
Hello! Have you still working on this issue? I am new to the community, and can I try it?
Hey I the deprecate in If you want to add the new metric, feel free to do so.
Can I add the new metric? If I can, can you give me some advice about the new metric? Thank you~
/assign
Code change:
clientset-path cloud-config cloud-provider <ins> cloud-provider-gce-lb-src-cidrs </ins> cluster-cidr cluster-context cluster-dns

Related discussion:
text: From GCE email today: Current behavior: You are not required to add firewall rules for Network LB health checks today Action required: We are introducing three new ranges for health checking: 209.85.152.0/22, 209.85.204.0/22, and 35.191.0.0/16. You need to ensure that your firewall allows traffic from 209.85.152.0/22, 209.85.204.0/22, and 35.191.0.0/16. The health check probes to your backend instances will come from one of these ranges. We have until Jan 31, 2018 - 1 year.
/assign this issue explains the confusion I had several weeks ago when speaking with and you about L4 health checks. I didn't realize a firewall rule isn't currently required.
/assign
Is this done now?
Not yet, there is a WIP on .
/close
Code change:
dc := &DisruptionController{ kubeClient:   kubeClient, queue:        workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "disruption"), <del> recheckQueue: workqueue.NewNamedDelayingQueue("disruption-recheck"), </del> <ins> recheckQueue: workqueue.NewNamedDelayingQueue("disruption_recheck"), </ins> broadcaster:  record.NewBroadcaster(), } dc.recorder = dc.broadcaster.NewRecorder(scheme.Scheme, v1.EventSource{Component: "controllermanager"})

Related discussion:
text: What happened: From : What you expected to happen: Metrics should register successfully. How to reproduce it (as minimally and precisely as possible): Run kubernetes-integration test Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
There are no sig labels on this issue. Please add a sig label by either: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. See the . The in method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Code change:
By("Deleting provisioner daemonset") deleteProvisionerDaemonset(config) }) <ins> It("should discover dynamicly created local persistent volume mountpoint in discovery directory", func() { By("Starting a provisioner daemonset") createProvisionerDaemonset(config) By("Creating a volume in discovery directory") dynamicVolumePath := path.Join(config.discoveryDir, fmt.Sprintf("vol-%v", string(uuid.NewUUID()))) setupLocalVolumeProvisionerMountPoint(config, dynamicVolumePath, config.node0) By("Waiting for the PersistentVolume to be created") _, err := waitForLocalPersistentVolume(config.client, dynamicVolumePath) Expect(err).NotTo(HaveOccurred()) By("Deleting provisioner daemonset") deleteProvisionerDaemonset(config) By("Deleting volume in discovery directory") cleanupLocalVolumeProvisionerMountPoint(config, dynamicVolumePath, config.node0) }) </ins> }) Context("StatefulSet with pod anti-affinity", func() {

Related discussion:
text: <!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: Now that the local provisioner uses mount propagation, the provisioner e2e test should also test adding mount points after the provisioner has started
would you have time to look into this?
/assign
FYI:
Code change:
endLine(tabWriter) for _, plState := range cfgCtlr.priorityLevelStates { if plState.queues == nil { <del> tabPrint(tabWriter, row( plState.pl.Name, // 1 "<none>",        // 2 "<none>",        // 3 "<none>",        // 4 "<none>",        // 5 "<none>",        // 6 )) if includeRequestDetails { continueLine(tabWriter) tabPrint(tabWriter, row( "<none>", // 7 "<none>", // 8 "<none>", // 9 "<none>", // 10 "<none>", // 11 "<none>", // 12 "<none>", // 13 "<none>", // 14 )) } endLine(tabWriter) </del> continue } queueSetDigest := plState.queues.Dump(includeRequestDetails)

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Testing API Priority and Fairness, I issued this command in a rather idle cluster: and I got this result: That is, a list of one request with no information of any sort. That comes from , which invents one phantom request for each priority level that is exempt. We do not want or need a phantom request; is for listing requests in queues. What you expected to happen: I expected to get only queued requests. How to reproduce it (as minimally and precisely as possible): As shown above, with API Priority and Fairness enabled. Anything else we need to know?: Environment: Kubernetes version (use ): this is in Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/cc /cc /cc