jiebi/CodeConvo huggingface dataset:
Change:
"fmt" "net/http" "net/http/httptest" <del> "sync" </del> "testing" "time"

Discussion:
We have found that on 1k nodes cluster scheduling takes 60-200 ms and becomes the bottleneck. We want to have a standard way to reproduce scheduling latency metrics result and benchmark scheduler as simple and fast as possible. Only with that can we: 1. figure out underlying behaviors and potential bottlenecks quickly; 2. publish results in a standard way and prevent future performance degradation. We have the following goals: Goal 1: Save time on testing. More accurate scheduling latency. We mock up kubelets and only measure scheduling latency and do not care about the latency to start the pod. This is different from Kubemark -- scheduled pods won’t be run by mocked kubelets. We only need to set up the master components. Goal 2: Profiling runtime metrics and find out performance bottleneck. Write scheduler integration test. Take advantage of go test and profiling tools, like cpu-profiling, memory-profiling and block-profiling. This is very fine-grained metrics. Goal 3: Reproduce test result easily. We want to have a known place to do the performance related test for scheduler. Developers should just run one script to collect all the information they need. We have the following plans for writing scheduler tests: density test (by adding a new Go test) schedule 30k pods on 1000 (fake) nodes and 3k pods on 100 (fake) nodes print out scheduling rate every second let you learn the rate changes vs number of scheduled pods benchmark (by modifying the existing benchmark in integration or simply add a new one) make use of and report nanosecond/op. schedule b.N pods when the cluster has N nodes and P scheduled pods. Since it takes relatively long time to finish one round, b.N is small -- level of 10. We are new to k8s codebase. We also want to know where is the best place (directory) to commit in these tests? Any suggestions are welcomed!
/cc
This extends
So density already does this... the one option it looks like you may want is custom control , real or fake is irrelevant as the e2e's just need to point to an api-server. re: benchmarks, seems like we could simply add to the unit-tests.
After chatting with lets add this as an agenda item for the SIG meeting tomorrow.

+1 for adding benchmark (howeveer this should be integration test, not unit-test in my opinion)
cc The next SIG on scheduler meeting is on 12.07, isn't it? If there is one tomorrow, can I have the information please?
retract, +1 to . I was adding this to the list given the context, but it crosses over into both.
Slides from today's meeting:
Here are are the action item notes from the SIG meeting: Everyone agrees this is a good idea and long overdue, but the question is where would this live: New category of tests? (isolated/sandbox’d/component) The current testing structure allows to easily append. create new directory (component) + hack/script to execute Future "we could" gate on known good data, and fail performance regressions
/cc
/cc
Change:
if !fp.needFullSync { t.Errorf("Proxier did not fail on previous partial resync?") } <del> updatedPRFailures, err := testutil.GetCounterMetricValue(metrics.IptablesPartialRestoreFailuresTotal) </del> <ins> updatedPartialRestoreFailures, err := testutil.GetCounterMetricValue(metrics.IptablesPartialRestoreFailuresTotal) </ins> if err != nil { t.Errorf("Could not get partial restore failures metric: %v", err) } <del> if updatedPRFailures != prFailures+1.0 { t.Errorf("Partial restore failures metric was not incremented after failed partial resync (expected %.02f, got %.02f)", prFailures+1.0, updatedPRFailures) </del> <ins> if updatedPartialRestoreFailures != partialRestoreFailures+1.0 { t.Errorf("Partial restore failures metric was not incremented after failed partial resync (expected %.02f, got %.02f)", partialRestoreFailures+1.0, updatedPartialRestoreFailures) </ins> } // On retry we should do a full resync, which should succeed (and delete svc4)

Discussion:
When unit tests in are invoked with 1, it fails Unit tests should work for any . This is important for troubleshooting flaking tests. Discovered when I tried to add parallel execution as described in master N/A N/A N/A N/A N/A
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network /area kube-proxy
/cc
Change:
stream := &fakeUpgradeStream{} c.streams[headers.Get(api.PortHeader)] = stream <del> stream.data = c.portData[headers.Get(api.PortHeader)] </del> <ins> // only simulate data on the data stream for now, not the error stream if headers.Get(api.StreamType) == api.StreamTypeData { stream.data = c.portData[headers.Get(api.PortHeader)] } </ins> return stream, nil }

Discussion:
I0407 16:48:52. ] Forwarding from 6000 -6000 --- FAIL: TestForwardPorts (0.02s) 2: Error reading data: read tcp 127.0.0.1:5000: connection reset by peer FAIL coverage: 57.4% of statements FAIL 0.168s https://travis-
cc
Andy, can you take a look at this one?
I've been unable to reproduce locally (Fedora 21 VM) but I will look again. Dawn Chen wrote:
Change:
<ins> /* Copyright 2023 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package filters import ( "context" "net/http" "k8s.io/apiserver/pkg/endpoints/request" ) type taskKeyType int const taskKey taskKeyType = iota type Task struct { Func func() } func WithTask(parent context.Context, t *Task) context.Context { return request.WithValue(parent, taskKey, t) } func TaskFrom(ctx context.Context) *Task { t, _ := ctx.Value(taskKey).(*Task) return t } // WithRoutine returns an http.Handler that executes preparation of long running requests (i.e. watches) // in a separate Goroutine and then serves the long running request in the main Goroutine. Doing so allows // freeing stack memory used in preparation Goroutine for better memory efficiency. func WithRoutine(handler http.Handler, longRunning request.LongRunningRequestCheck) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { ctx := req.Context() requestInfo, _ := request.RequestInfoFrom(ctx) if !longRunning(req, requestInfo) { handler.ServeHTTP(w, req) return } req = req.WithContext(WithTask(ctx, &Task{})) panicCh := make(chan any, 1) go func() { defer func() { if r := recover(); r != nil { panicCh <- r } close(panicCh) }() handler.ServeHTTP(w, req) }() if p, ok := <-panicCh; ok { panic(p) } ctx = req.Context() if t := TaskFrom(ctx); t != nil && t.Func != nil { t.Func() } }) } </ins>

Discussion:
Optimize the memory usage of goroutine stack for serving watch requests. According to the of handling a watch request, the stack memory is 0x3768, which is very close to 16 KiB and Golang runtime will double it to 32 KiB once exceeded. In fact, it does exceed if we print the traceback deeper and take the fact that extra stack memory reserved by Golang itself into consideration. A naive optimization is to run as much as preparation handlers in a separate goroutine and then move the actual serving to another goroutine, so the former routine can free the stack memory earlier and make it more memory efficient. Given the extensive use of watch requests, 32 KiB per watch could add up to a very high amount of memory overhead for . A typical watch request commonly lasts for 30s or longer, which means this memory will be active for quite long. I see a great improvement in memory efficiency to do this optimization.
/cc
/sig api-machinery
/cc if you would like to take a look when you have time :) Thank you /triage accepted
Change:
and minor versions MUST only be an integer. release: v1.19 file: test/e2e/apimachinery/server_version.go <ins> - testname: ControllerRevision, resource lifecycle codename: '[sig-apps] ControllerRevision [Serial] should manage the lifecycle of a ControllerRevision [Conformance]' description: Creating a DaemonSet MUST succeed. Listing all ControllerRevisions with a label selector MUST find only one. After patching the ControllerRevision with a new label, the label MUST be found. Creating a new ControllerRevision for the DaemonSet MUST succeed. Listing the ControllerRevisions by label selector MUST find only two. Deleting a ControllerRevision MUST succeed. Listing the ControllerRevisions by label selector MUST find only one. After updating the ControllerRevision with a new label, the label MUST be found. Patching the DaemonSet MUST succeed. Listing the ControllerRevisions by label selector MUST find only two. Deleting a collection of ControllerRevision via a label selector MUST succeed. Listing the ControllerRevisions by label selector MUST find only one. The current ControllerRevision revision MUST be 3. release: v1.25 file: test/e2e/apps/controller_revision.go </ins> - testname: CronJob Suspend codename: '[sig-apps] CronJob should not schedule jobs when suspended [Slow] [Conformance]' description: CronJob MUST support suspension, which suppresses creation of new jobs.

Discussion:
[X] Test approval issue: [X] Test PR: 8 Jul 2022 [x] Two weeks soak end date: 22 Jul 2022 [x] Test promotion PR: According to this APIsnoop query, there are still some remaining Controller Revision endpoints which are untested. listAppsV1ControllerRevisionForAllNamespacesreadAppsV1NamespacedControllerRevisionpatchAppsV1NamespacedControllerRevisioncreateAppsV1NamespacedControllerRevisiondeleteAppsV1NamespacedControllerRevisionreplaceAppsV1NamespacedControllerRevisiondeleteAppsV1CollectionNamespacedControllerRevision Using a number of existing e2e test practices a new has been created for 7 ControllerRevision endpoints. The e2e logs for this test are listed below. This query shows the following endpoints are hit within a short period of running this e2e test If a test with these calls gets merged, test coverage will go up by 7 points This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/triage accepted
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Change:
func (n NamespacedName) String() string { return n.Namespace + string(Separator) + n.Name } <ins> // MarshalLog emits a struct containing required key/value pair func (n NamespacedName) MarshalLog() interface{} { return struct { Name, Namespace string }{ Name:      n.Name, Namespace: n.Namespace, } } </ins>

Discussion:
should implement . This ensures that it's use as value in a key/value pair emits a struct and not the string. /ref
This depends on logr 1.2.0, which will be in /wg structure-logging /sig instrumentation
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : This depends on logr 1.2.0, which will be in /sig instrumentation Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I'd like to work on this :) /assign
/wg structured-logging
/triage accepted
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale /lifecycle frozen
Change:
# to readonly. # The configured vars will only contain platforms allowed by the # KUBE_SUPPORTED* vars at the top of this file. <ins> declare -a -g KUBE_SERVER_PLATFORMS declare -a -g KUBE_CLIENT_PLATFORMS </ins> kube::golang::setup_platforms() { if [[ -n "${KUBE_BUILD_PLATFORMS:-}" ]]; then # KUBE_BUILD_PLATFORMS needs to be read into an array before the next

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: building Kubernetes release and it failed due to a variable not being set (). It appears a does not function as expected on (used Ubuntu 16). Additional details will be provided in comment below What you expected to happen: successful build of Kuberentes release or documented minimum supported OS releases (i.e., bash versions) How to reproduce it (as minimally and precisely as possible): using Ubuntu 16 (), run [] using , or (containing or subsequent cherry pick) Anything else we need to know?: It appears the issue is with how and create variables, where I believe does not create the variables with the expected scope (created using scope versus scope). This creates a minimum supported OS version when running from the best I can tell. Environment: Kubernetes version (use ):  Cloud provider or hardware configuration: N/A OS (e.g: ):Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): N/A Others: /sig release
Some of the details I mentioned above, modified the file to output some helpful information the diff of changes I made the results when running on Ubuntu 16 (bash 4.3), the variable appears to be scoped inside the function the results when running on Ubuntu 16 (bash 4.3) without the variable attribute (you can now see the variable is a variable, not locally scoped to the function ) And on Ubuntu 18 (bash 4.4) the variable is scoped with the attribute as you would expect and runs as expected.
I guess fixes this problem. cc
I don't think the is/was the only issue here. It was obviously as I was testing with , so only variables were being used and not scoped outside the function (in bash 4.3).
we are also getting the same error on bash version 4.3 cc
any update on this, are you fixing it??
Same question with , this issue impact OpenLab cloud-provider-openstack E2E testing job.
can you please try this?

it works with your fix , previously jobs were failing --- EDIT --- Hold on, I think I was too fast, it failed in a later stage don't know if is a problem or related to the script changes. Should we have the tar files as an output ?
Sorry I missed this yesterday. I will try as well. Otherwise, my assumption was to revert the use of a function, as that creates the scoping problem we've been seeing with variables in .
i think that would be not
Also both not should work now.
Looks like worked for me. Was able to run a using successfully.
ref: as the pr that appears to have caused this
Change:
framework.ExpectNoError(err) metrics := map[string]float64{} <del> for method, samples := range response { if metricNames.Has(method) { if len(samples) == 0 { return nil, fmt.Errorf("metric %s has no samples", method) } lastSample := samples[len(samples)-1] metrics[method] = float64(lastSample.Value) </del> <ins> for _, samples := range response { if len(samples) == 0 { continue </ins> } <del> } // Ensure all metrics were provided for name := range metricNames { if _, found := metrics[name]; !found { return nil, fmt.Errorf("metric %s not found", name) </del> <ins> // Find the *last* sample that has the label we are interested in. for i := len(samples) - 1; i >= 0; i-- { metricNameWithLabels := samples[i].Metric.String() if metricNames.Has(metricNameWithLabels) { metrics[metricNameWithLabels] = float64(samples[i].Value) break } </ins> } }

Discussion:
[It] [sig-storage] CSI Mock selinux on mount metrics SELinuxMount metrics [LinuxOnly][Feature:SELinux][Feature:SELinuxMountReadWriteOncePod][Serial] warning is not bumped on two Pods with the same context on RWO volume [sig-storage] [It] [sig-storage] CSI Mock selinux on mount metrics SELinuxMount metrics [LinuxOnly][Feature:SELinux][Feature:SELinuxMountReadWriteOncePod][Serial] warning is bumped on two Pods with a different context on RWO volume [sig-storage] Sample failure: October 25, 2023 It fails since The PR a label to some SELinux metric and the tests do not count with that. No response /sig storage
/assign /triage accepted
Change:
for i, pl := range plugins { weight := f.scorePluginWeight[pl.Name()] <del> nodeScoreList := pluginToNodeScores[i] </del> <ins> nodeScoreList := pluginToNodeScores[pl.Name()] </ins> score := nodeScoreList[index].Score if score > framework.MaxNodeScore || score < framework.MinNodeScore {

Discussion:
/sig scheduling /kind regression caused a performance degradation in the 5k nodes scalability tests ! The PR changed how to split the load among parallel threads. Previously, we would parallelize over the number of nodes. The PR does it over the number of plugins, which is much smaller. Performance to improve or remain the same https://perf- No response 28 N/A <details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign The quickest fix is to revert, but I'm open to other suggestions.
/priority critical-urgent
Can we just use another Parallelizer to handle the score process for each plugin? I ran the UT locally and it seems fine, but I'm not familar with
Depending on how many cores you have available, you might just overload the routines. I think it's better to revert the parallizer over plugins while keeping the change from map to slice. Although I don't expect a major impact for the latter. You might want to run a profiler for the benchmark with a higher number of nodes.
I'll create a revert version first :)
Let's revert this first since we reached the code freeze. We can do the refactor later.
Sorry for causing this trouble and thanks everyone for a prompt action
Change:
} pv := &v1.PersistentVolume{ ObjectMeta: metav1.ObjectMeta{ <del> // A.K.A InnerVolumeSpecName required to match for Unmount Name: volume.Name, </del> <ins> // Must be unique per disk as it is used as the unique part of the // staging path Name: fmt.Sprintf("%s-%s", AWSEBSDriverName, ebsSource.VolumeID), </ins> }, Spec: v1.PersistentVolumeSpec{ PersistentVolumeSource: v1.PersistentVolumeSource{

Discussion:
Minimal Repro: Create 2 different disks Create 2 Pods each using one of the disks Start both pods on the same node You will notice one of the disks gets ed to a path: , then both disks are ed from to a target path: which is similar but at least uses the pod . Both of these volumes are being treated as the same volume in the Kubelet even though they are definitely not the same volume. Note: it may not be "treated" as the same volume - it may just be that since the paths are the same the following volumes see that the path is already a mountpoint and skip the step. This is the more likely scenario based on observations. The value is the "name" of the inline-volume. These can obviously be the same across different pods - we should actually be placing it in a more unique directory (based off the translated maybe?). /cc
/sig storage
The inline volume example I'm using (ignore the fact that is a complete misnomer): I've confirmed that changing the "name" makes inline volumes work well since there is no longer a collision on the paths
/assign
Alright, problem is sort of twofold: The referred to above for inline-volumes is set here: As we can see from the comment: it is set to the because the in-tree CSI Plugin makes a lot of naming assumptions based on the which for PVC/PVs is the PV name (unique in a cluster). Primarily, during unmount the CSI Plugin tries to from a "well known ": which happens to be some path based on the which for the inline volume is just the ( above). So we run into the problem that if we use the directory is not sufficiently unique for different disks. And if we use something else like then the unmount pathway is unable to find the . I think using is a no-go. So we need to use something sufficiently unique (like ) for the ( above) and fix up some of the other codepaths to either use different "well-known paths" to write to, or to fix up path generation code so they give us the correct locations. I'm also going to check to see whether 's refactor fixes this issue (because it might since it translates things earlier)
OK. Really good news.

your refactor in its current state actually fixes this issue if we set the to something unique here: Looks like the early translation is also making the pathway from the "correct" dir for inline migration. So I will create a PR that fixes this issue that should merge after