jiebi/CodeConvo huggingface dataset:
Change:
description: Create a StatefulSet resource. Newly created StatefulSet resource MUST have a scale of one. Bring the scale of the StatefulSet resource up to two. StatefulSet scale MUST be at two replicas. <del> release: v1.16 </del> <ins> release: v1.16, v1.21 </ins> file: test/e2e/apps/statefulset.go - testname: StatefulSet, Rolling Update with Partition codename: '[sig-apps] StatefulSet [k8s.io] Basic StatefulSet functionality [StatefulSetBasic]

Discussion:
[ ] two weeks soak start date : testgrid-link [ ] two weeks soak end date : [ ] test promotion pr : kubernetes/kubernetes#? According to this APIsnoop query, there are still some remaining StatefulsetScale endpoints that are untested. with this query, you can filter untested endpoints by their category and eligibility for conformance. e.g below shows a query to find all conformance eligible untested, stable, apps endpoints sql-mode select distinct endpoint, right(useragent,73) AS useragent from testing.auditevent -- where useragent ilike '%subresource%' where endpoint ilike '%AppsV1NamespacedStatefulSetScale%' and releasedate::BIGINT round(((EXTRACT(EPOCH FROM NOW()))::numeric)1000,0) - and useragent like 'e2e%' order by endpoint limit 30; example endpoint | useragent -----------------------------------------|--------------------------------------------------------------------------- patchAppsV1NamespacedStatefulSetScale | [StatefulSetBasic] should have a working scale subresource [Conformance] readAppsV1NamespacedStatefulSetScale | [StatefulSetBasic] should have a working scale subresource [Conformance] replaceAppsV1NamespacedStatefulSetScale | [StatefulSetBasic] should have a working scale subresource [Conformance] (3 rows) If a test with these calls gets merged, test coverage will go up by 1 point* This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Change:
) type pullResult struct { <del> imageRef string err      error </del> <ins> imageRef     string err          error pullDuration time.Duration </ins> } type imagePuller interface {

Discussion:
While going over Kubernetes events I noticed that image pulls for some small images were taking several minutes according to the "Successfully pulled image" event. From investigating this is misleading as the actual pull time is much smaller but due to --serialize-image-pulls=true being the default in the observed cluster and other pods being initialized at the time, the reported pull time ends up accounting for time while waiting for other pods to finish pulling. I expected to see the time it takes to download the image itself from a registry. Customers seem to rely on this metric for this information and we have received some complaints as a result where customers noted slow pull times as a result of misunderstanding the current behavior of this metric. Start with a K8s cluster with --serialize-image-pulls=true (default) - For my check I used default minikube v1.26.1 pod-large where pod-large pulls any large image (the image cannot be locally cached) after step 1 (within seconds) create pod-small where pod-small pulls any small image Check the output for the event after running: Note that despite the small size, pull time for the image in pod-small is several minutes, we should instead see it take seconds (which is how long the pull actually takes). To note this is the case you can try pulling another small image like alpine that isnt cached locally and compare after the above are done. Here is the code that pulls an image in kubernetes and logs the event: As can be seen from it, it's not true pull time, just time from invoking the puller to receiving a result. The serial puller is using a Golang channel as a queue and will just wait until a previous image is done pulling as I noted before, thus it will end up recording the amount of time in the queue + pull time yielding a misleading metric. <details</details<detailsShould reproduce anywhere (tried it on minikube) but have personally observed this in Azure as well </details<details</details<details</details<details</details<details</details
/sig node
But means that kubelet can pull multi images at the same time. For kubelet, is by default true. is for promoting the use of parallel image pulls. The problem maybe with . I did some simple test with 3 ubuntu images as below. The time was collected from events.

-- --serialize-image-pulls=true -- -- image pulling serialize ubuntu:22.10 73.6 MiB 1m21.284220239s ubuntu:20.04 75.7 MiB 2m7.876426944s ubuntu:14.04 212.9 MiB 4m52.884660938s
You are right thanks for taking a look at this. It seems I got the two reversed in my head. The issue indeed happens when --serialize-image-pulls=true which is the default as you pointed out. I'll update the issue to reflect that
/triage accepted /priority important-longterm
Change:
} func expectResult(t *testing.T, w *worker, expectedResult results.Result, msg string) { <del> result, ok := resultsManager(w.probeManager, w.probeType).Get(testContainerID) </del> <ins> result, ok := resultsManager(w.probeManager, w.probeType).Get(w.containerID) </ins> if !ok { t.Errorf("[%s - %s] Expected result to be set, but was not set", w.probeType, msg) } else if result != expectedResult {

Discussion:
Happened in GKE suite:
Does it need a liveness probe, maybe?
to delegate
Saw
Can you take a look at this one? Thanks!
FWIW, I saw some fishy messages in in (node: 104.197.48.84) It seems like docker was giving conflicting view of the container status, i.e., whether the container was running or not (and whether kubelet should kill it). kubelet's first attempt to kill the container : All the exec well after the grace period still shows that the container is the "Running: true" state () Later, kubelet tried killing the container again and failed because container does not exist. Note that after the failed kill also shows that the container was in the Running state. At the same time, cadvisor observed the container being stopped but then detected the container and then tried adding it again, but failed to find the /cc UPDATE: According to supervisord, docker was not killed during this time.
I didn't find result for the first Killing, does that really return? What is the log level, I remember that you told me before the log level in GKE is 2?
AFAICT, the pod worker may have never returned because the specific pod worker didn't log anything beyond that point. I was looking at (docker stop hangs) to see if it's relevant. didn't print any error though.
Yeah, the last SyncPod log was which is before the first Killing. Then the pod worker was stuck in
This really looks similar with I tried to reproduce it following the comment. The container was not actually killed. Docker stop hangs. Docker ps shows the container is still running. But the difference is that when I tried to kill it with another , it still hung there.
I was able to reproduce this quite easily by running 30 pods with the same spec in the test and with :
From
My theory is that during may cause docker to leave the container in a unrecoverable state (yep, a bug). I can consistently reproduce this by simply creating (which always fails), and leave the cluster for < 10 min. The restart counts of some of the pods will stop increasing after sometime, and after ssh'ing in to the node, one can easily confirm the same symptoms posted above. If pods use the http probe, they do not have this issue.

For the workaround, we can instruct the probe workers to stop probing after detecting a liveness check failure, until the container has been restarted (with a new container ID). Note that this is just a workaround, as it doesn't stop other components or users to run during this window. I've verified that with this fix, there are no zombie docker containers anymore. Will upload a PR.
FWIW, I also tested it with docker 1.8.3, which has the same problem.
Thanks !
Change:
Properties: map[string]spec.Schema{ "maxUnavailable": { SchemaProps: spec.SchemaProps{ <del> Description: "The maximum number of pods that can be unavailable during the update. Value can be an absolute number (ex: 5) or a percentage of desired pods (ex: 10%). Absolute number is calculated from percentage by rounding up. This can not be 0 if MaxSurge is 0. By default, a fixed value of 1 is used. Example: when this is set to 30%, the old RC can be scaled down to 70% of desired pods immediately when the rolling update starts. Once new pods are ready, old RC can be scaled down further, followed by scaling up the new RC, ensuring that the total number of pods available at all times during the update is at least 70% of desired pods.", </del> <ins> Description: "The maximum number of pods that can be unavailable during the update. Value can be an absolute number (ex: 5) or a percentage of desired pods (ex: 10%). Absolute number is calculated from percentage by rounding down. This can not be 0 if MaxSurge is 0. By default, a fixed value of 1 is used. Example: when this is set to 30%, the old RC can be scaled down to 70% of desired pods immediately when the rolling update starts. Once new pods are ready, old RC can be scaled down further, followed by scaling up the new RC, ensuring that the total number of pods available at all times during the update is at least 70% of desired pods.", </ins> Ref:         spec.MustCreateRef("#/definitions/intstr.IntOrString"), }, },

Discussion:
API documentation says is rounded up, but it is actually rounded down: The code shows it is rounded down, not up: and
Yes, the documentation is obsolete. We changed maxUnavailable to round down so that we wouldn't accidentally infer 100% unavailability for non-100 percentages. For example, maxSurge=25% and maxUanavailable=25% with a Deployment of 1 replica meant that we could scale down 1 before thus leading in 100% maxUnavailable (implicit).
Change:
} // setting a threshold to 0% disables; non-empty map overrides default value (necessary due to omitempty) initialConfig.EvictionHard = map[string]string{"memory.available": "0%"} <ins> if initialConfig.FeatureGates == nil { initialConfig.FeatureGates = map[string]bool{} } </ins> initialConfig.FeatureGates[string(LSCIQuotaFeature)] = quotasRequested }) runEvictionTest(f, evictionTestTimeout, noPressure, noStarvedResource, logDiskMetrics, []podEvictSpec{

Discussion:
Seen in this job failing on multiple tests, this is just one of the stack traces it seems that for a long time :smile: it panics here I don't have a good understanding of those tests mechanics, but it seems a complex setup embedding functions and goroutines before the test setup, I suggest to move to something more simple No response_ /sig node
ping I don't have time for working on this unfortunately but happy to help on reviews, this can also be a good task for new contributors, also I don't consider a bad idea to rework the tests a bit to make them easier to read and simpler, I did that for sig-network and we have a pretty low flake rate since then
/assign /cc This probably came out of /triage accepted
/priority important-soon /milestone v1.23
Ooops - I'm assuming it's going through one of the paths where Features or whatever isn't initialized.
Change:
"-m", "mark", "!", "--mark", fmt.Sprintf("%s/%s", masqueradeMark, masqueradeMark), "-j", "RETURN"); err != nil { klog.Errorf("Failed to ensure filtering rule for %v: %v", KubePostroutingChain, err) <del> return </del> <ins> return false </ins> } // Clear the mark to avoid re-masquerading if the packet re-traverses the network stack. // We know the mark bit is currently set so we can use --xor-mark to clear it (without needing

Discussion:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: ci-kubernetes-e2e-gci-gce-alpha-features Which test(s) are failing: Networking should recreate its iptables rules if they are deleted [Disruptive] Since when has it been failing: Commit Testgrid link: Reason for failure: Anything else we need to know: Possibly related to this? It's listed in the git diff /sig network
/assign /cc Thanks for detecting it so quicklly
most probably is related to
/unassign /assign he has already submitted a fix
/triage accepted
Reviewed. Marking as resolved on the CI Signal Board. Once was merged, failures stopped and they have not re-occurred in the last 8 days (Oct 10th)
Yay
Change:
go_test( name = "go_default_test", <del> srcs = [ "controller_test.go", "patch_test.go", ], </del> <ins> srcs = ["controller_test.go"], </ins> embed = [":go_default_library"], deps = [ "//pkg/controller:go_default_library",

Discussion:
PR service patch methods to Once there is a release with that PR included, the code in service-controller and legacy-cloud-providers/gce can be updated to use those. /area platform/gce /assign
/sig network
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry. ‍
/remove-triage unresolved
Those labels are not set on the issue: <detailsIn response to : /remove-triage unresolved Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Created a PR to use these new functions in Service-controllers.
Change:
lastErr = err return false, nil } <ins> if warningWriter.WarningCount() != 1 { lastErr = fmt.Errorf("expected one warning, got: %v", warningWriter.WarningCount()) return false, nil } if !strings.Contains(recordedWarnings.String(), expectedWarning) { lastErr = fmt.Errorf("unexpected warning, expected: n%vn, got: n%v", expectedWarning, recordedWarnings.String()) return false, nil } lastErr = nil </ins> return true, nil <del> }); err != nil { </del> <ins> }); err != nil || lastErr != nil { </ins> t.Fatalf("failed to wait for apiserver handling webhook mutation: %v, last error: %v", err, lastErr) } <del> if lastErr != nil { t.Fatal(lastErr) } if warningWriter.WarningCount() != 1 { t.Errorf("expected one warning, got: %v", warningWriter.WarningCount()) } if !strings.Contains(recordedWarnings.String(), expectedWarning) { t.Errorf("unexpected warning, expected: n%vn, got: n%v", expectedWarning, recordedWarnings.String()) } recordedWarnings.Reset() </del> // Make sure reset happens in update requests pod, err = client.CoreV1().Pods("default").Update(context.TODO(), pod, metav1.UpdateOptions{})

Discussion:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --pull-kubernetes-integration TestMutatingWebhookResetsInvalidManagedFields Not sure. Example failures from recent PRs: (PR: ) (PR: ) /sig api-machinery /kind flake
There's even a few failures in the post-submit master integration tests job: (ignore the test/integration/job failures, I have an open PR to fix that).
cc
Thanks! /cc /wg api-expression
Oh no, what did I do :-/ I'll have a look. Might be that the admission webhook is not up quick enough. /assign
/triage accepted
I tried this locally and it passes, so maybe I can try stress testing this
Would increasing the timelimit here give more time to the admissionhook to come up ?
I think so. Looked at it a few days ago, I'm not sure how much we should increase it. I don't want to slow down the tests a lot more if we can avoid it. Do we know if the other admission tests (duplicate owner ref) have the same issue?
Can we make the webhook blocking so that the request will wait for it to be ready? Can we make sure that it's up-and-running somehow before performing the actual test? Having an arbitrary "wait" before we start the test seems wrong.
The poll and timeout are fine as is. lastErr is only getting set, never unset. If validateManagedFieldsAndDecode returns nil, lastErr should get cleared.
Hmm.. actually, the one failing is the Update request, after the Patch request has already been checked. The patch request passing means the webhook was effective and working, the update request should work consistently the first time. If the patch request really is ensuring the webhook was installed, then a timeout or retry on the update request is not the right approach.
But isnt the patch request alos failing to report the adequate warnings
this continues to be the flakiest test in the integration job in by a wide margin
So sorry for letting this slip for so long :-/ I'll work on it right when I get back home in an hour.
Change:
csipbv1 "github.com/container-storage-interface/spec/lib/go/csi" api "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/resource" <ins> "k8s.io/apimachinery/pkg/util/wait" </ins> "k8s.io/kubernetes/pkg/volume" "k8s.io/kubernetes/pkg/volume/csi/fake" volumetypes "k8s.io/kubernetes/pkg/volume/util/types"

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Kubelet dies with a panic exception if CSI plugin returns NodeId == "" and nil error. Exception log see at the end of the message What you expected to happen: Kubelet should just raise an error to the plugin and ignore the registration. How to reproduce it (as minimally and precisely as possible): Create a CSI plugin with only Node that would implement GetNodeInfo as follows: Register it as a DaemonSet and see kublet crush Anything else we need to know?: Environment: Kubernetes version (use ): 1.14.10-gke.17 Cloud provider or hardware configuration: GKE OS (e.g: ): Container-Optimized OS (cos) Kernel (e.g. ): - Install tools: Network plugin and version (if this is a network-related bug): Others: Related exception Related line of code: - no check whether error is not nil before calling Error() on it.
/sig storage
/sig node is this the version of the kubelet too? 14 is out of support. is the issue reproducible with a 1.15+ kubelet?
/assign
I checked the code - same defect still exists in current codebase.
thanks /priority important-soon
Wow, that was fast. Thanks!
Change:
if err != nil { return err, retry } <del> // Always update the cache upon success </del> <ins> // Always update the cache upon success. // NOTE: Since we update the cached service if and only if we successully // processed it, a cached service being nil implies that it hasn't yet // been successfully processed. </ins> cachedService.service = service s.cache.set(namespacedName.String(), cachedService) case cache.Deleted:

Discussion:
In , I found a logic error (when I had a broken cloud implementation): If createLoadBalancerIfNeeded returns err, then cachedService will have service = nil. This causes updateLoadBalancerHosts to panic. It would be easy to skip cacheService if service=nil, but I wasn't sure whether this should then be included in servicesToRetry or not. Another problem (by inspection) is that I think there's a race condition where updateLoadBalancerHosts can run immediately after getOrCreate, before the service field has been set.
/cc
Great catches, Justin, thanks! I believe that simply not processing cached services with a nil api.Service object in the node pool reconciler is the correct approach. A nil Service object implies that the load balancer reconciler still intends to process it, and the LB reconciler always uses the most up-to-date list of nodes. I'll send a PR ASAP.
Change:
# https://bazel-review.googlesource.com/#/c/10390/ http_archive( name = "io_bazel", <del> sha256 = "667d32da016b1e2f63cf345cd3583989ec4a165034df383a01996d93635753a0", </del> <ins> sha256 = "892a84aa1e7c1f99fb57bb056cb648745c513077252815324579a012d263defb", </ins> strip_prefix = "bazel-df2c687c22bdd7c76f3cdcc85f38fefd02f0b844", urls = ["https://github.com/ixdy/bazel/archive/df2c687c22bdd7c76f3cdcc85f38fefd02f0b844.tar.gz"], )

Discussion:
I don't see an existing issue, so opening one for tracking. PR builds are currently failing due to the SHA of a remote archive changing. ... $ sha256sum 27a13726ff8621cfa4592fbef116b253043c2b093c8814265dc4489a0e67d229 cc cc cc
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
[MILESTONENOTIFIER] Milestone Labels Complete Issue label settings: sig/release: Issue will be escalated to these SIGs if needed. priority/critical-urgent: Never automatically move out of a release milestone; continually escalate to contributor and SIG through all available channels. kind/bug: Fixes a bug discovered during the current release. <detailsAdditional instructions available <a href=""here</aThe commands available for adding these labels are documented <a href=""here</a</details
cc
/assign We can probably just bump the sha, but I'm trying to figure out what changed...
I found a copy of the which matched the sha256sum (thanks There's no diff when I compare the extracted trees. There's also no diff if I compare the output on each. The new and old gzip'd tarballs are slightly different sizes (old bytes, new bytes). If I ungzip them, they are the same size ( bytes) but they still have different sha256sums. My best guess at this point is that there was some library change recently which is creating slightly different versions of these tarballs, but I don't know how it's different. I also don't know why the archive tarballs for other repos don't seem to have changed.
go 1.8 vs 1.9 gzip library maybe?
It seems like the order of some files in the tarball may have changed.
Yeah, we're not alone:
I guess we should start mirroring these archives somewhere rather than depending on GitHub.
Change:
// AddWatchReactor appends a reactor to the end of the chain. func (c *Fake) AddWatchReactor(resource string, reaction WatchReactionFunc) { <ins> c.Lock() defer c.Unlock() </ins> c.WatchReactionChain = append(c.WatchReactionChain, &SimpleWatchReactor{resource, reaction}) } // PrependWatchReactor adds a reactor to the beginning of the chain. func (c *Fake) PrependWatchReactor(resource string, reaction WatchReactionFunc) { <ins> c.Lock() defer c.Unlock() </ins> c.WatchReactionChain = append([]WatchReactor{&SimpleWatchReactor{resource, reaction}}, c.WatchReactionChain...) }

Discussion:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI -- ci-kubernetes-generate-make-test-count10 pr:pull-kubernetes-unit ci-kubernetes-unit ci-kubernetes-generate-make-test-count10 TestAttacherMountDeviceWithInline links to appreciated links to specific failures in spyglass appreciated <!-- Please see the deflaking doc () for more guidance! --
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig sig-storage </details
/sig storage