jiebi/CodeConvo huggingface dataset:
Change:
# Delete the master replica pd (possibly leaked by kube-up if master create failed). # TODO(jszczepkowski): remove also possibly leaked replicas' pds <del> local -r replica-pd="${REPLICA_NAME:-${MASTER_NAME}}-pd" if gcloud compute disks describe "${replica-pd}" --zone "${ZONE}" --project "${PROJECT}" &>/dev/null; then </del> <ins> local -r replica_pd="${REPLICA_NAME:-${MASTER_NAME}}-pd" if gcloud compute disks describe "${replica_pd}" --zone "${ZONE}" --project "${PROJECT}" &>/dev/null; then </ins> gcloud compute disks delete  --project "${PROJECT}"  --quiet  --zone "${ZONE}"  <del> "${replica-pd}" </del> <ins> "${replica_pd}" </ins> fi # Delete disk for cluster registry if enabled

Discussion:
./hack/e2e- line 1192: local: `replica-pd=jenkins-us-central1-b-master-pd': not a valid identifier FYI - this looks related to your recent work? I don't see any obvious PR to blame, but I seem to remember you guys touching this part of the code recently? BTW, I deleted a bunch of leaked resources in the relevant test project on Friday (9/16), assuming that the test scripts would re-create whatever they needed. It's possible (but unlikely) that this has anything to do with that.
Here are some example logs to aid debugging: https://k8s-
I believe this to be the source of dysfunction: Which is turn caused by what you found, which causes the disk to not be cleaned up ahead of time: You can try this out in your local shell: git blames leads us to this recent commit: cc the author The root problem is unfortunatley very annoying: this is how bash tells you that variable names (in this case, $replica-pd) cannot contain dashes!
fix is up in
Awesome many thanks. I'll make sure it gets merged ASAP.
Change:
"github.com/GoogleCloudPlatform/kubernetes/pkg/api" "github.com/GoogleCloudPlatform/kubernetes/pkg/client" "github.com/GoogleCloudPlatform/kubernetes/pkg/client/record" <ins> "github.com/GoogleCloudPlatform/kubernetes/pkg/healthz" </ins> "github.com/GoogleCloudPlatform/kubernetes/pkg/master/ports" "github.com/GoogleCloudPlatform/kubernetes/pkg/util" "github.com/GoogleCloudPlatform/kubernetes/plugin/pkg/scheduler"

Discussion:
Found this by calling the "/validate" end point on the master: root curl 127.0.0.1:8080/validate <snip{ "component": "scheduler", "health": "failure", "healthCode": 1, "msg": "404 page not foundn", "err": "unhealthy http status code: 404 (404 Not Found)" }, </snipSeems like any requests to it return a 404: root netstat -p -l | grep kube-scheduler tcp 0 0 localhost: : LISTEN 4105/kube-scheduler root curl 127.0.0.1:/healthz 404 page not found root curl 127.0.0.1: 404 page not found
I'm not seeing this on my 0.14.1 cluster: $ curl http://127.0.0.1:8080/validate { "component": "scheduler", "health": "success", "msg": "ok", "err": "nil" }, $ curl http://127.0.0.1:/healthz ok
This is happening on HEAD. seems to have caused this. /cc
Could we rollback this PR?
is this only on the scheduler?
Yes. The change basically broke the /healthz mapping in kube-scheduler, so the health pings are failing. The "validate-" doesn't hit it, but the master's "/validate" does. This is causing the internal cluster validation checks to fail.
GKE relies on the /validate endpoint on the master (since it can't run the validate- script) and this has broken our GKE-CI build since it was merged this morning.
I think we can revert the PR and let profiling break for now. what do you think?
SGTM, sorry for the breakage.
Ahh, you guys already revert the commit? The fix should be very simple. Let me send PR shortly to fix this.
Thanks for the fix and sorry for the breakage
Change:
rgName := az.getLoadBalancerResourceGroup() allLBs, rerr := az.LoadBalancerClient.List(ctx, rgName) if rerr != nil { <ins> if rerr.IsNotFound() { return nil, nil } </ins> az.Event(service, v1.EventTypeWarning, "ListLoadBalancers", rerr.Error().Error()) klog.Errorf("LoadBalancerClient.List(%v) failure with err=%v", rgName, rerr) return nil, rerr.Error()

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --I recall that this issue was reported previously in It was fixed in the master back then with and it was cherry-picked into with However we still observe the same issue with the legacy azure cloud-controller-manager v1.19.9. legacy azure cloud controller-manager to be able to delete LoadBalancer when the underlying resource group is deleted. a LoadBalancer manually the underlying resource group the LoadBalancer created in step 1 sure that the deletion of the LoadBalancer fails Events of the Service: Kubernetes version (use ): v1.19.9 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig cloud-provider /area provider/azure /cc
Thanks for reporting the issue. It should be a bug inside cloud provider. /triage accepted
Change:
} // createPodOrFail creates a pod with the specified containerPorts. <del> func createPodOrFail(f *framework.Framework, ns, name string, labels map[string]string, containerPorts []v1.ContainerPort) { </del> <ins> func createPodOrFail(f *framework.Framework, ns, name string, labels map[string]string, containerPorts []v1.ContainerPort, args ...string) { </ins> ginkgo.By(fmt.Sprintf("Creating pod %s in namespace %s", name, ns)) <del> pod := e2epod.NewAgnhostPod(ns, name, nil, nil, containerPorts) </del> <ins> pod := e2epod.NewAgnhostPod(ns, name, nil, nil, containerPorts, args...) </ins> pod.ObjectMeta.Labels = labels // Add a dummy environment variable to work around a docker issue. // https://github.com/docker/docker/issues/14203

Discussion:
The test only validates that the API objects are created E2e tests must verify the behaviour, i.e. that the Service and Endpoints actually work sending traffic. The file is not well organized and there is a lot of duplicate code, one possibility is to use current Services helper has a function to test the reachability that iterates over all service ports However, something that needs to be addressed is that the pods that are backing the service doesn't have any process listening on those ports
/help wanted /sig network /priority important-soon /triage accepted
I think that there are more tests that doesn't verify the reachability , those should be modified too /help-wanted
/help-wanted
/assign
/triage accepted
/assign /unassign I will take on this since it was more difficult than I expected, and is a Conformance test.. I assigned you to , that is similar to this, but creating a new test from scratch
Change:
cmd.Flags().StringVar(&cfg.Networking.ServiceSubnet, "service-cidr", cfg.Networking.ServiceSubnet, "The range of IP address used for service VIPs") cmd.Flags().StringVar(&featureGatesString, "feature-gates", featureGatesString, "A set of key=value pairs that describe feature gates for various features. "+ "Options are:n"+strings.Join(features.KnownFeatures(&features.InitFeatureGates), "n")) <ins> cmd.Flags().Var(utilflag.NewMapStringString(&cfg.APIServerExtraArgs), "apiserver-extra-args", "A set of extra flags to pass to the API Server or override default ones in form of <flagname>=<value>") </ins> } if properties.use == "all" || properties.use == "controller-manager" { cmd.Flags().StringVar(&cfg.Networking.PodSubnet, "pod-network-cidr", cfg.Networking.PodSubnet, "The range of IP addresses used for the Pod network") <ins> cmd.Flags().Var(utilflag.NewMapStringString(&cfg.ControllerManagerExtraArgs), "controller-manager-extra-args", "A set of extra flags to pass to the Controller Manager or override default ones in form of <flagname>=<value>") } if properties.use == "all" || properties.use == "scheduler" { cmd.Flags().Var(utilflag.NewMapStringString(&cfg.SchedulerExtraArgs), "scheduler-extra-args", "A set of extra flags to pass to the Scheduler or override default ones in form of <flagname>=<value>") </ins> } cmd.Flags().StringVar(&cfgPath, "config", cfgPath, "Path to kubeadm config file (WARNING: Usage of a configuration file is experimental)")

Discussion:
/kind feature What happened: When calling , there is no way to override default flag values for the different components of the controlplane. This is surprising, as in the config structure (and in controlplane phase codebase itself), there are explicit settings for doing that (APIServerExtraArgs, ControllerManagerExtraArgs, SchedulerExtraArgs). What you expected to happen: Those config fields should be exposed as flags on and How to reproduce it (as minimally and precisely as possible): Try to deploy K8s using Kubeadm with a custom Admission Control setting (I'll follow up on this issue with a PR)
btw, I am not sure which sig owns kubeadm.
it's sig-cluster-lifecycle.
/sig sig-cluster-lifecycle
/sig cluster-lifecycle
Change:
framework.ExpectEqual(statusUpdated.Status.Conditions[len(statusUpdated.Status.Conditions)-1].Message, "Updated by an e2e test", "The Message returned was %v", statusUpdated.Status.Conditions[0].Message) framework.Logf("Status.Condition: %#v", statusUpdated.Status.Conditions[len(statusUpdated.Status.Conditions)-1]) }) <ins> ginkgo.It("should apply an update to a Namespace", func() { var err error var updatedNamespace *v1.Namespace ns := f.Namespace.Name cs := f.ClientSet ginkgo.By(fmt.Sprintf("Updating Namespace %q", ns)) err = retry.RetryOnConflict(retry.DefaultRetry, func() error { updatedNamespace, err = cs.CoreV1().Namespaces().Get(context.TODO(), ns, metav1.GetOptions{}) framework.ExpectNoError(err, "Unable to get Namespace %q", ns) updatedNamespace.Labels[ns] = "updated" updatedNamespace, err = cs.CoreV1().Namespaces().Update(context.TODO(), updatedNamespace, metav1.UpdateOptions{}) return err }) framework.ExpectNoError(err, "failed to update Namespace: %q", ns) framework.ExpectEqual(updatedNamespace.ObjectMeta.Labels[ns], "updated", "Failed to update namespace %q. Current Labels: %#v", ns, updatedNamespace.Labels) framework.Logf("Namespace %q now has labels, %#v", ns, updatedNamespace.Labels) }) </ins> }) func unstructuredToNamespace(obj *unstructured.Unstructured) (*v1.Namespace, error) {

Discussion:
[X] Test approval issue: [X] Test PR: 27 Aug 2022 [x] Two weeks soak end date: 9 Sept 2022 [x] Test promotion PR: According to this APIsnoop query, there are still a remaining Namespace endpoint which is untested. replaceCoreV1Namespace Using a number of existing e2e test practices a new has been created for 1 Namespace endpoint. The e2e logs for this test are listed below. This query shows the following endpoints are hit within a short period of running this e2e test If a test with these calls gets merged, test coverage will go up by 1 point This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/triage accepted
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Change:
"//staging/src/k8s.io/apiserver/pkg/authentication/serviceaccount:go_default_library", "//staging/src/k8s.io/client-go/kubernetes:go_default_library", "//staging/src/k8s.io/client-go/tools/cache:go_default_library", <ins> "//staging/src/k8s.io/client-go/tools/watch:go_default_library", </ins> "//staging/src/k8s.io/client-go/util/flowcontrol:go_default_library", "//staging/src/k8s.io/client-go/util/retry:go_default_library", "//staging/src/k8s.io/client-go/util/workqueue:go_default_library",

Discussion:
[x] Original approval issue: [x] Two weeks soak end date: This issue is created to allow edit by /wip /hold This issue proposes the writing of a new test for the API endponts , , , with the intention of promoting it to Conformance. This issue contains a very basic mock test in order to start the conversation about what a good e2e test would look like for this endpoint. The code below is verified to be hitting the intended endpoint, show as per the queries in APIsnoop's live view of the cluster's audit logs. According to this APIsnoop query, there are still some remaining Endpoints endpoints which are untested. a Service in a namespace all Endpoints in all Namespaces find the created Service(1) /update the Namespaced Endpoint for the Service(1) the Namespaced Endpoint for the Service(1) the Namespaced Endpoint for the Service(1) Namespaced Endpoint(1) via a Collection with a LabelSelector Discover useragents: If a test with these calls gets merged, Conformance coverage will go up by 4 points* /sig testing /sig architecture /area conformance
/reopen Wait for Promotion PR to Merge.
Reopened this issue. <detailsIn response to : /reopen Wait for Promotion PR to Merge. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Change:
} secondaryAllocatorCIDR := r.secondaryServiceIPs.CIDR() <del> if *(service.Spec.IPFamily) == api.IPv6Protocol && netutil.IsIPv6CIDR(&secondaryAllocatorCIDR) { </del> <ins> if (*(service.Spec.IPFamily) == api.IPv6Protocol) == netutil.IsIPv6CIDR(&secondaryAllocatorCIDR) { </ins> return r.secondaryServiceIPs }

Discussion:
What happened: When K8s is started in dual-stack mode with IPv6 as the "main" (or base) family services specified with ipFamily:IPv4 still get a ipv6 ; The is expicitly set to "10.0.0.0" using . What you expected to happen: Services with ipFamily:IPv4 shall get ipv4 ClusterIP addresses. How to reproduce it (as minimally and precisely as possible): Setup a K8s cluster in dual-stack mode with family IPv6 as base Define a service with ipFamily: IPv4 To make K8s think IPv6 is the main family the cue is to define ipv6 addresses to K8s binaries. Below are my settings as an example. I do not use but install "from scratch"; Master processes; is accessed with ipv4 which does not affect the K8s family. Worker processes; Anything else we need to know?: There are other problems with dual-stack/ipv6-base but may be caused by this one. For example does not work and some processes does not become "ready". Ipv6 traffic to ipv6 services and DNS works; Environment: Kubernetes version (use ):Cloud provider or hardware configuration: - OS (e.g: ): BusyBox based own setup. Kernel (e.g. ):Install tools: "from scratch" Network plugin and version (if this is a network-related bug): Others:
/sig network /cc
The problem seem to be; The problem occurs when an ipv6 range is specified first, even in an ipv4-based dual-stack cluster. But the ipv6 range must be specified first otherwise the "kubernetes" service gets an ipv4 address but an ipv6 backend; ~Perhaps the way to fix this problem is to make sure the service address is from the right family and require that the order to is "ipv4,ipv6".~ Bad idea!
There seem to be a option; But I can't see it in the help printout. I will try to use it...
:disappointed:
Here it where the kubernetes service address is set;
/assign
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry. ‍
Change:
if regional && utilfeature.DefaultFeatureGate.Enabled(features.GCERegionalPersistentDisk) { disk, err = gce.getRegionalDiskByName(diskName) if err != nil { <del> glog.V(5).Infof("Could not find regional PD named %q to Attach. Will look for a zonal PD", diskName) err = nil } else { mc = newDiskMetricContextRegional("attach", gce.region) </del> <ins> return err </ins> } <del> } if disk == nil { </del> <ins> mc = newDiskMetricContextRegional("attach", gce.region) } else { </ins> disk, err = gce.getDiskByName(diskName, instance.Zone) if err != nil { return err

Discussion:
/kind bug If a PV's zone annotation format indicates that it's a regional PD, but pdName corresponds to a regular PD, currently the GCE PD plugin allows attach to succeed. It should fail, because the mismatch is often caused by user error. /sig gcp /sig storage /assign
Can we have an admission controller that checks for this user error?
how would an out of tree volume plugin enforce this restriction? Shouldn't we try to model surfacing the error using the flows available to CSI plugins, since the in-tree plugins are planned to migrate to that?
cc I think the ValidateVolumeCapabilities call may be able work for this purpose. We would still need some sort of CSI admission controller/webhook to make the callout to the plugin for this though. Another alternative. Today we already have an admission controller that auto-fills in zone information for GCE PDs. It is being moved into the cloud-specific admission controllers, and we could potentially extend this there.
Today the PVLabel admission controller already overwrites the user-specified label with the correct label, but it's important to have a secondary check in the plugin. Consider this scenario: both a regional and a regular disk exist with name , and the user's PV has the zone label matching the regional disk. The admission controller then retains the user specified label since it's valid. Next, the user deletes the regional disk, and the PV is left unchanged. When a pod is created with this PV, the regular disk will be attached, even though the attach should fail. As for CSI, yeah it looks like we could use as the call is designed for verifying pre-provisioned volumes. And I agree, if there are cloud-specific admission controllers in the future, it'd be good to include this check there instead. When CSI begins to support topology, ideally should also have a topology field. Synced with offline.
Change:
${ETCD_CMD}  --name="etcd-$(hostname)"  --debug  <ins> --force-new-cluster  </ins> --data-dir=${DATA_DIRECTORY}  --listen-client-urls http://127.0.0.1:${ETCD_PORT}  --advertise-client-urls http://127.0.0.1:${ETCD_PORT} 

Discussion:
It appears that the API server is no longer able to talk to etcd after an upgrade. See https://k8s- for an example. Interesting bits from api server log:
I have debugged a lot of 2.3.7 -3.0.14 upgrades and adding to every invocation helps a ton. Any chance we can twiddle this knob when we run these tests? Perhaps via instance metadata?
shows etcd starting, and that 3.0.14 stays running for ~10 hours, and apparently appending to its log. It looks like someone is talking to it. The failure modes I saw in testing upgrades were generally that etcd would crash, which is not what we see here.
One thing just occurred to me. We should probably have a test suite like these upgrades that simply tries to upgrade etcd without changing the version of k8s. This would let us isolate etcd failures from failures of other types.
- I think there is still one flag fix in the "migration" script that you found but didn't create a PR for. Maybe this is also reason of this failure? Can you please finish the loop and finally create that PR so that we can have the script in head updated?
I'm reopening for verifying if the problem is fixed.
I have re-built . I'll see if I can abort the current run of https://k8s-
We are currently using "3.0.14-alpha.1" image. So if you want to try a different image, I think you should update manifest files to the new image. Or rebuild 3.0.14-alpha.1 image :) [Unless I'm missing something how the test works.]
D'oh. Thanks, I just assumed the default tag in the Makefile was correct.
Should I be looking into this job: https://k8s- From the logs that pasted at the beginning (and which appear in all logs) and seems that the problem is with certificated files, not with etcd itself. - did anything change around how certs are handled? Any ideas why those tests may be facing this issue?
Actually, looking at the log that is causing it to fail: I very strongly suspect that the victim of those failures is this PR: - can you please take a look if the hypothesis sounds reasonable to you?

I have seen something related, probably the same issue. I was testing 1.4.7, persisted the etcd data dir, restarted the cluster on master and wasn't able to access any resources created when the cluster was running in 1.4.7. I was able to see those resources in etcd by using curl directly on etcd. Using etcd 3.0.15
Ok, ya this was caused by . I tried to revert and got some nasty conflicts. I'm hoping that will be ready by the EOD and will fix this issue.
thanks!
can you confirm that your PR would have caused the same problem if was running etcd v2? I'm trying to rule out any interaction with etcd v3 or our scripts related to it.
yes, I don't think this problem is etcd related, as far as i can tell right now.
- is this issue still relevant? Or can we close as obsolete?
Change:
// volume *should* be attached to this node until it is safely unmounted. desiredVolumes := vm.desiredStateOfWorld.GetVolumesToMount() mountedVolumes := vm.actualStateOfWorld.GetGloballyMountedVolumes() <del> volumesToReportInUse := make( []v1.UniqueVolumeName, 0, /* len */ len(desiredVolumes)+len(mountedVolumes) /* cap */) desiredVolumesMap := make( map[v1.UniqueVolumeName]bool, len(desiredVolumes)+len(mountedVolumes) /* cap */) </del> <ins> volumesToReportInUse := make([]v1.UniqueVolumeName, 0, len(desiredVolumes)+len(mountedVolumes)) desiredVolumesMap := make(map[v1.UniqueVolumeName]bool, len(desiredVolumes)+len(mountedVolumes)) </ins> for _, volume := range desiredVolumes { if volume.PluginIsAttachable {

Discussion:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: is not sent in a stable sort order. this makes node status patch requests larger than needed, and makes debugging nodes harder than necessary. What you expected to happen: stable sort of volumesInUse How to reproduce it (as minimally and precisely as possible): always Anything else we need to know?: nope Environment: n/a
/sig node
- this is a good getting started item.
I've already submitted a PR in . PTAL. Thanks.
Change:
# Specialized tests which should be skipped by default for projects. GCE_DEFAULT_SKIP_TESTS=( "${REBOOT_SKIP_TESTS[@]}" <del> "AutoscalingsSuite" </del> "Reboot" "ServiceLoadBalancer" )

Discussion:
Per our email discussion while getting 1.1 out the door. It looks like this never happened, though, which means we're not running autoscaling tests against GCE on the 1.1 branch, (we were running it on GKE with the 1.1-features job, but I'm removing that job in ). I'd once again like to (strongly) suggest that we mirror test suites running against release branch to jobs running against master, so that we avoid this problem of accidentally not running tests when we think we are. I'd much rather have a perhaps-unnecessary slow/HPA/whatever suite than continually try to figure out where slow/HPA/whatever tests are running, depending on if I'm looking at release jobs versus jobs running against master. cc
+1 We should be running the autoscaling tests in slow or continuous.