jiebi/CodeConvo huggingface dataset:
Change:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: Which test(s) are flaking: Testgrid link: https://k8s- Reason for failure: Based on the execution conditions sometimes the value of could be 0 instead of expected Anything else we need to know: links to appreciated links to specific failures in spyglass appreciated <!-- Please see the deflaking doc () for more guidance! --
/sig node
This particular test also fails on CRIO with Fedora CoreOS platform because the return value of is slightly different on that platform.
thank you for opening this issue. Just to have more visibility in the future, and avoid duplicated work, could you please share issues like this one in #sig-node or with the mailist list for CI improvements ? I also see you opened so will assign you this issue /assign /triage accepted /priority important-longterm /cc
thank you for opening and working on this issue. You can check the for issues logged against tests that are producing non-deterministic results. And also it is worth noting that that logged has a for this problem along with the following specific failures
sorry I wasn't aware of CI Signal project board. Thanks for the link.
sorry my bad, I didn't realize there was already an issue for this.
More failures around not having NetworkStats and MemoryStats,
From the kubelet logs of one of the failed jobs. Open the failed job's and look for
helped me to get rid of errors related fsinfo not available. I do some more testing. It's just a temporary patch which looks for because that's what I am using. If it really works, I could make it more generic.

Discussion:
"strings" "time" <del> "k8s.io/api/core/v1" </del> <ins> v1 "k8s.io/api/core/v1" </ins> "k8s.io/apimachinery/pkg/api/resource" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" kubeletstatsv1alpha1 "k8s.io/kubelet/pkg/apis/stats/v1alpha1"

"Name":      gstruct.Ignore(), "StartTime": recent(maxStartAge), "CPU": ptrMatchAllFields(gstruct.Fields{ <del> "Time":                 recent(maxStatsAge), "UsageNanoCores":       bounded(10000, 2e9), </del> <ins> "Time": recent(maxStatsAge), // CRI stats provider tries to estimate the value of UsageNanoCores. This value can be // either 0 or between 10000 and 2e9. // Please refer, https://github.com/kubernetes/kubernetes/pull/95345#discussion_r501630942 // for more information. "UsageNanoCores":       gomega.SatisfyAny(gomega.BeZero(), bounded(10000, 2e9)), </ins> "UsageCoreNanoSeconds": bounded(10000000, 1e15), }), "Memory": ptrMatchAllFields(gstruct.Fields{
Change:
Which jobs are flaking: Which test(s) are flaking: https://k8s- https://k8s- Testgrid link: https://k8s- https://k8s- Reason for failure: Anything else we need to know: Logs suggest that pulling the image of first init container () of the hollow node pod fails: Further perusal shows it's related to rate limiting by from where the image is pulled - we receive lots of 429 errors when attempting to pull the image: Hollow node template: /assign /sig scalability
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
possible related to ? do we need to use a different container image registry or an image tag other than latest to avoid repulling repeatedly?
Do we really need busybox? Can we use some other image?
Re answering this question - what we use there is "sysctl" So we basically need to the smallest image that contains this sysctl.
do we need to use a different container image registry or an image tag other than latest to avoid repulling repeatedly? Do we really need busybox? Can we use some other image? Re answering this question - what we use there is "sysctl" Reason for adding init container to kubemark's hollow nodes in the first place is the outcome of this past discussion: Tl;dr NPD creates an instance per hollow node pod on a node and the default upper limit of those instances was insufficient, hence with sysctl call was introduced. This is done once per hollow node pod, so there's a bit of redundancy going on here (as this is enough to be run just once per node). We could either: 1) fetch busybox image from a different container registry, as Jordan suggested, 2) use a fixed tag other than to omit image force pulling by kubelet each time a hollow node pod is created, 3) revisit ideas mentioned in , 4) use some other image for the init container - any suggestions on that? No. 2 appeals to me most as this looks like the easiest way to get this sorted out.
No - that's a reason for creating inotify. You don't need busybox for that. You need an image that has systctl inside to use (see my comment above). But I agree that using something different than :latest can be the cheapest mitigation short term to do.
No - that's a reason for creating inotify. You don't need busybox for that. You need an image that has systctl inside to use (see my comment above). I understood what you meant in the first place, maybe sentences in my comment weren't phrased in a proper order :D. Anyways, here's the PR with a proposed fix:
Makes sense - sorry. Approved the PR.

Discussion:
spec: initContainers: - name: init-inotify-limit <del> image: busybox </del> <ins> image: busybox:1.32 </ins> command: ['sysctl', '-w', 'fs.inotify.max_user_instances=1000'] securityContext: privileged: true
Change:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: bazel-test-master Which test(s) are flaking: gotest: Testgrid link: Reason for failure: Anything else we need to know: links to appreciated links to specific failures in spyglass appreciated <!-- Please see the deflaking doc () for more guidance! --/sig cloud-provider
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig cloud-provider
ping !
Interestingly the expected test string was logged: Test:
Also unable to reproduce this, ran it a few thousand times:
Looks like a true flake. Does this need attention?
/assign
I cannot repro this failure either, while if it's really flacky, I could remove this ut since that's really legacy code, since it's azure blob disk code which is deprecated long time ago, remove ut is quite safe.
here is the PR to remove this flacky ut:
/assign

Discussion:
assert.True(t, strings.Contains(err.Error(), expectedErr)) assert.Empty(t, diskURI) } <del> func TestDeleteBlobDisk(t *testing.T) { ctrl := gomock.NewController(t) defer ctrl.Finish() b := GetTestBlobDiskController(t) b.common.cloud.BlobDiskController = &b mockSAClient := mockstorageaccountclient.NewMockInterface(ctrl) b.common.cloud.StorageAccountClient = mockSAClient fakeDiskURL := "fake" diskURL := "https://foo.blob./vhds/bar.vhd" err := b.DeleteBlobDisk(fakeDiskURL) expectedErr := fmt.Errorf("failed to parse vhd URI invalid vhd URI for regex https://(.*).blob./vhds/(.*): fake") assert.Equal(t, expectedErr, err) mockSAClient.EXPECT().ListKeys(gomock.Any(), b.common.resourceGroup, "foo").Return(storage.AccountListKeysResult{}, &retryError500) err = b.DeleteBlobDisk(diskURL) expectedErr = fmt.Errorf("no key for storage account foo, err Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: <nil>") assert.Equal(t, expectedErr, err) mockSAClient.EXPECT().ListKeys(gomock.Any(), b.common.resourceGroup, "foo").Return(storage.AccountListKeysResult{ Keys: &[]storage.AccountKey{ { KeyName: to.StringPtr("key1"), Value:   to.StringPtr("key1"), }, }, }, nil) b.accounts["foo"] = &storageAccountState{} err = b.DeleteBlobDisk(diskURL) expectedErrStr := "storage: service returned error: StatusCode=403, ErrorCode=AccountIsDisabled, ErrorMessage=The specified account is disabled." assert.Error(t, err) assert.True(t, strings.Contains(err.Error(), expectedErrStr)) } </del>
Change:
Which jobs are failing: ci-cri-containerd-node-e2e Which test(s) are failing: Summary API [NodeConformance] when querying /stats/summary should report resource usage through the stats api Since when has it been failing: 2020 10-12 09:04 PDT Since Testgrid link: Reason for failure: Capability set incorrectly. CAPNETRAW should be NETRAW. I1013 18:39:19.058] Oct 13 18:10:51.403: INFO: At 2020-10-13 18:09:34 +0000 UTC - event for stats-busybox-0: {kubelet tmp-node-e2e--cos-81--1210-0} Failed: Error: failed to create containerd task: OCI runtime create failed: starting container process caused: unknown capability "CAPCAPNET_RAW": unknown Anything else we need to know: I have no idea where this is documented. It's a best guess.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
it's a node_e2e test so /sig node
Seems like there's some dispute on who is doing the correct thing.
I would like to have a try.
please see discussion in

Discussion:
pods := getSummaryTestPods(f, numRestarts, pod0, pod1) f.PodClient().CreateBatch(pods) <ins> ginkgo.By("restarting the containers to ensure container metrics are still being gathered after a container is restarted") </ins> gomega.Eventually(func() error { for _, pod := range pods { err := verifyPodRestartCount(f, pod.Name, len(pod.Spec.Containers), numRestarts)

return nil }, time.Minute, 5*time.Second).Should(gomega.BeNil()) <del> // Wait for cAdvisor to collect 2 stats points </del> <ins> ginkgo.By("Waiting 15 seconds for cAdvisor to collect 2 stats points") </ins> time.Sleep(15 * time.Second) // Setup expectations.

maxStartAge = time.Hour * 24 * 365 // 1 year maxStatsAge = time.Minute ) <del> // fetch node so we can know proper node memory bounds for unconstrained cgroups </del> <ins> ginkgo.By("Fetching node so we can match against an appropriate memory limit") </ins> node := getLocalNode(f) memoryCapacity := node.Status.Capacity["memory"] memoryLimit := memoryCapacity.Value()

"Time":           recent(maxStatsAge), "AvailableBytes": fsCapacityBounds, "CapacityBytes":  fsCapacityBounds, <del> // we assume we are not running tests on machines < 10tb of disk </del> <ins> // we assume we are not running tests on machines more than 10tb of disk </ins> "UsedBytes":  bounded(e2evolume.Kb, 10*e2evolume.Tb), "InodesFree": bounded(1e4, 1e8), "Inodes":     bounded(1e4, 1e8),

Image: busyboxImage, SecurityContext: &v1.SecurityContext{ Capabilities: &v1.Capabilities{ <del> Add: []v1.Capability{"CAP_NET_RAW"}, </del> <ins> Add: []v1.Capability{"NET_RAW"}, </ins> }, }, Command: getRestartingContainerCommand("/test-empty-dir-mnt", 0, numRestarts, "echo 'some bytes' >/outside_the_volume.txt; ping -c 1 google.com; echo 'hello world' >> /test-empty-dir-mnt/file;"),
Change:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: sig-netwok jobs running serial tests Which test(s) are failing: [sig-network] DNS configMap nameserver.* Since when has it been failing: Seems this was the trigger Testgrid link: https://k8s-,%20master Reason for failure: It seems is trying to use the container Anything else we need to know:
/sig network /sig testing /cc /triage accepted
/assign

Discussion:
Command:       cmd, Namespace:     t.f.Namespace.Name, PodName:       t.utilPod.Name, <del> ContainerName: "util", </del> <ins> ContainerName: t.utilPod.Spec.Containers[0].Name, </ins> CaptureStdout: true, CaptureStderr: true, })
Change:
What happened: I was testing out the pod topology spread functionality in a local k8s cluster. I noticed that when I set the constraint to be pods are still successfully scheduled. What you expected to happen: Reading the documentation on , I expected pods not to be scheduled since the node should be "bypassed due to the absence of label". Did I miss something here? How to reproduce it (as minimally and precisely as possible): Create a pod with topologySpreadConstraints[0].topologyKey that does not match to any node labels, and set to . Anything else we need to know?: Thank you Environment: Kubernetes version (use ):Cloud provider or hardware configuration: It's a kind cluster created with image . OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scheduling
This sounds like a bug, I will look into it. /assign

Discussion:
} // However, "empty" preFilterState is legit which tolerates every toSchedule Pod. <del> if len(s.TpPairToMatchNum) == 0 || len(s.Constraints) == 0 { </del> <ins> if len(s.Constraints) == 0 { </ins> return nil }

}, }, { <ins> name: "pod cannot be scheduled as all nodes don't have label 'rack'", pod: st.MakePod().Name("p").Label("foo", "").SpreadConstraint( 1, "rack", v1.DoNotSchedule, st.MakeLabelSelector().Exists("foo").Obj(), ).Obj(), nodes: []*v1.Node{ st.MakeNode().Name("node-a").Label("zone", "zone1").Label("node", "node-a").Obj(), st.MakeNode().Name("node-x").Label("zone", "zone2").Label("node", "node-x").Obj(), }, wantStatusCode: map[string]framework.Code{ "node-a": framework.UnschedulableAndUnresolvable, "node-x": framework.UnschedulableAndUnresolvable, }, }, { </ins> name: "pods spread across nodes as 2/1/0/3, only node-x fits", pod: st.MakePod().Name("p").Label("foo", "").SpreadConstraint( 1, "node", v1.DoNotSchedule, st.MakeLabelSelector().Exists("foo").Obj(),
Change:
What happened: throws errors if no pods are found in v1.19.3, but returns success with empty response in v1.16.8 What you expected to happen: If there are no pods that return, no error should be thrown as the range should evaluate to "no op". How to reproduce it (as minimally and precisely as possible): Adapt the example from to get the command . Run this when there are pods in default namespace - all is well. Run this when there are no pods in default namespace and you get the error: Anything else we need to know?: This breaks all kinds of code that relies on jsonpath's earlier behaviour. I found an old report () that seems appropriate, but that is long since merged and closed. Environment: Kubernetes version (use ): 1.19.3 Cloud provider or hardware configuration: x86-64 OS (e.g: ): RedHat Enterprise Linux 7.6 (Maipo) Kernel (e.g. ): Install tools: n/a Network plugin and version (if this is a network-related bug): n/a Others: n/a
/sig cli
/cc
/sig api-machinery
/assign cc /triage accepted

Discussion:
if j.beginRange > 0 { j.beginRange-- j.inRange++ <del> for _, value := range results { </del> <ins> if len(results) > 0 { for _, value := range results { j.parser.Root.Nodes = nodes[i+1:] nextResults, err := j.FindResults(value.Interface()) if err != nil { return nil, err } fullResult = append(fullResult, nextResults...) } } else { // If the range has no results, we still need to process the nodes within the range // so the position will advance to the end node </ins> j.parser.Root.Nodes = nodes[i+1:] <del> nextResults, err := j.FindResults(value.Interface()) </del> <ins> _, err := j.FindResults(nil) </ins> if err != nil { return nil, err } <del> fullResult = append(fullResult, nextResults...) </del> } j.inRange--

testJSONPathSortOutput(randomPrintOrderTests, t) } <ins> func TestEmptyRange(t *testing.T) { var input = []byte(`{"items":[]}`) var emptyList interface{} err := json.Unmarshal(input, &emptyList) if err != nil { t.Error(err) } tests := []jsonpathTest{ {"empty range", `{range .items[*]}{.metadata.name}{end}`, &emptyList, "", false}, {"empty nested range", `{range .items[*]}{.metadata.name}{":"}{range @.spec.containers[*]}{.name}{","}{end}{"+"}{end}`, &emptyList, "", false}, } testJSONPath(tests, true, t) } </ins> func TestNestedRanges(t *testing.T) { var input = []byte(`{ "items": [
Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: While investigating I've found that kubectl doesn't handle sigterm with profiling. This way, when profiling a long run command (as wait, drain, etc) and interrupting the command in the middle (ctrl-c, as an example) the profile file is not available. What you expected to happen: Profiling file exists if the command exists correctly or if the command is interrupted with ctrl-c How to reproduce it (as minimally and precisely as possible): Execute a long run command (as kubectl drain, kubectl wait) with --profile=cpu and then interrupt the command before it finishes the execution Anything else we need to know?: Because kubectl deals with errors with an util (cmdutil.CheckErr) that in some specific time exists with os.Exit(1), it's not possible yet to also profile programs that exits. Environment: Kubernetes version (use ): kubectl compiled from upstream Cloud provider or hardware configuration: OS (e.g: ): Ubuntu 20.04
/sig cli /assign
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Discussion:
import ( "fmt" "os" <ins> "os/signal" </ins> "runtime" "runtime/pprof"

if err != nil { return err } <del> return pprof.StartCPUProfile(f) </del> <ins> err = pprof.StartCPUProfile(f) if err != nil { return err } </ins> // Block and mutex profiles need a call to Set{Block,Mutex}ProfileRate to // output anything. We choose to sample all events. case "block": runtime.SetBlockProfileRate(1) <del> return nil </del> case "mutex": runtime.SetMutexProfileFraction(1) <del> return nil </del> default: // Check the profile name is valid. if profile := pprof.Lookup(profileName); profile == nil {

} } <ins> // If the command is interrupted before the end (ctrl-c), flush the // profiling files c := make(chan os.Signal, 1) signal.Notify(c, os.Interrupt) go func() { <-c flushProfiling() os.Exit(0) }() </ins> return nil }
Change:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which test(s) are failing: Since when has it been failing: since upgrading golang: Reason for failure: Since Go 1.15, type casting from int to string using is not allowed, output: Anything else we need to know: related links: golang/go golang/go
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
the fix should be straight forward, so I can create a pull request to fix this. /sig testing /sig cli /assign
/assign

Discussion:
package create import ( <ins> "strconv" </ins> "testing" rbac "k8s.io/api/rbac/v1"

} for i, tc := range tests { <del> t.Run(string(i), func(t *testing.T) { </del> <ins> t.Run(strconv.Itoa(i), func(t *testing.T) { </ins> roleBinding, err := tc.options.createRoleBinding() if err != nil { t.Errorf("unexpected error:n%#vn", err)
Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Trying to connect to Service VIP from a POD is failing intermittently in Overlay network mode What you expected to happen: Frontend should be able to connect to Service VIP always. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: With a configuration like this: Node 1 has these POD's POD1 POD2 POD3 Node 2 has these POD's POD4 POD5 POD6 Service1 has these pod's as backend - POD1,POD2,POD5,POD6 Curl to Service1 from POD3 would fail if POD5 or POD6 is chosen for the connection Curl to Service1 from POD3 would pass if POD1 or POD2 is chosen for the connection Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network /sig windows

Discussion:
} } <del> if proxier.network.networkType == "Overlay" { </del> <ins> // For Overlay networks 'SourceVIP' on an Load balancer Policy can either be chosen as // a) Source VIP configured on kube-proxy (or) // b) Node IP of the current node // // For L2Bridge network the Source VIP is always the NodeIP of the current node and the same // would be configured on kube-proxy as SourceVIP // // The logic for choosing the SourceVIP in Overlay networks is based on the backend endpoints: // a) Endpoints are any IP's outside the cluster ==> Choose NodeIP as the SourceVIP // b) Endpoints are IP addresses of a remote node => Choose NodeIP as the SourceVIP // c) Everything else (Local POD's, Remote POD's, Node IP of current node) ==> Choose the configured SourceVIP if proxier.network.networkType == "Overlay" && !ep.GetIsLocal() { </ins> providerAddress := proxier.network.findRemoteSubnetProviderAddress(ep.IP()) isNodeIP := (ep.IP() == providerAddress)