jiebi/CodeConvo huggingface dataset:
Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: when run , I founded failed. What you expected to happen: Test passed How to reproduce it (as minimally and precisely as possible): Sudo version 1.8.6p7 Anything else we need to know?: Environment: Kubernetes version (use ): 1.17 Cloud provider or hardware configuration: OS (e.g: ):centos Kernel (e.g. ):Linux elk-141 4.9.6- SMP Wed Mar 1 18:39:23 CST 2017 x8664 x8664 x8664 GNU/Linu Install tools: Network plugin and version (if this is a network-related bug): Others:
/cc
/sig node

Discussion:
echo 'No need to refresh sudo credentials' else echo 'Updating sudo credentials' <del> sudo --validate || exit 1 </del> <ins> sudo -v || exit 1 </ins> fi # Do not use any network plugin by default. User could override the flags with

echo 'No need to refresh sudo credentials' else echo 'Updating sudo credentials' <del> sudo --validate || exit 1 </del> <ins> sudo -v || exit 1 </ins> fi # FOCUS is ginkgo focus to select which tests to run. By default, FOCUS is
Change:
What happened: has a typo, (should be ) This also shows up in the rendered documentation. What you expected to happen: Documentation has correct path for readyz endpoint. How to reproduce it (as minimally and precisely as possible): Easiest to see this wrong in the source code, I think. Anything else we need to know?: Environment: Kubernetes version (use ): v1.16 Cloud provider or hardware configuration: not applicable OS (e.g: ): not applicable Kernel (e.g. ): not applicable Install tools: Network plugin and version (if this is a network-related bug): Others: /sig docs /kind bug /priority backlog

Discussion:
fs.DurationVar(&s.ShutdownDelayDuration, "shutdown-delay-duration", s.ShutdownDelayDuration, ""+ "Time to delay the termination. During that time the server keeps serving requests normally and /healthz "+ <del> "returns success, but /readzy immediately returns failure. Graceful termination starts after this delay "+ </del> <ins> "returns success, but /readyz immediately returns failure. Graceful termination starts after this delay "+ </ins> "has elapsed. This can be used to allow load balancer to stop sending traffic to this server.") utilfeature.DefaultMutableFeatureGate.AddFlag(fs)
Change:
Same as an issue that occurred for v1.14.0: the failing test causes ginkgo to exit non-zero which causes the script to abort without running the routine. A recent PR to this script fixed a race but we didn't realize that now that the command was running in the foreground (and not the background) that the non-zero status would impact that script. Currently, Sonobuoy can get around this by just using the Golang based runner via . The Go-runner does not seem to have a similar vulnerability. This is going to be the default for Sonobuoy runs starting in Sonobuoy v0.16.0 if the PR gets merged. Honestly, I think that we may want to consider just removing the bash script all-together for v1.17.0 if the go runner works well for this entire cycle. See the original reporting of the issue and diagnosis here: /sig testing /area conformance /priority important-soon
/assign The PR in question that re-emerged this problem: FWIW Sonobuoy is adding an integration suite that will be able to test this going forward (soon). I think we need to fix the script (and get it cherrypicked into v1.16.x branches) I think that if the go runner proves stable over v1.16 then we default to it going forward We consider removing entirely if we think we can do that without impacting other workflows. Maybe that wont be possible due to other workflows though. cc
is probably never called due to , it can probably be fixed by adding a or before line 70.
Yes that is exactly why it is but in the past when I recommended removing/modifying the errexit code I've gotten pushback from a googler just due to convention: see Just will add something like to the lines that need this

Discussion:
# Entry provided via env var to simplify invocation. if [[ -n ${E2E_USE_GO_RUNNER:-} ]]; then set -x <del> /gorunner exit $? </del> <ins> /gorunner && ret=0 || ret=$? exit ${ret} </ins> fi # We get the TERM from kubernetes and handle it gracefully

) set -x <del> /usr/local/bin/ginkgo "${ginkgo_args[@]}" /usr/local/bin/e2e.test -- --disable-log-dump --repo-root=/kubernetes --provider="${E2E_PROVIDER}" --report-dir="${RESULTS_DIR}" --kubeconfig="${KUBECONFIG}" -v="${E2E_VERBOSITY}" > >(tee "${RESULTS_DIR}"/e2e.log) ret=$? </del> <ins> /usr/local/bin/ginkgo "${ginkgo_args[@]}" /usr/local/bin/e2e.test -- --disable-log-dump --repo-root=/kubernetes --provider="${E2E_PROVIDER}" --report-dir="${RESULTS_DIR}" --kubeconfig="${KUBECONFIG}" -v="${E2E_VERBOSITY}" > >(tee "${RESULTS_DIR}"/e2e.log) && ret=0 || ret=$? </ins> set +x saveResults exit ${ret}
Change:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: https://k8s- Which test(s) are failing: [] [sig-node] kubelet [] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. Since when has it been failing: 2019-09-26 Testgrid link: Reason for failure: Anything else we need to know: /cc
is there someone who can pick this up since it's constantly failing in the master-informing? Thanks :)
Could you help us here?
Looked into the test logs, and it seems like the in was accidentally removed in , which caused its value to be 0. As the current log is showing: And last succeeded test log is: I'll send a PR to fix this. /cc
It seems that this PR didn't help. This run already contains this PR and it failed:
/milestone v1.17 /priority critical-urgent
I0926 12:39:42.660] I0926 12:35:26. ] Trace[]: "Reflector ListAndWatch" (started: 2019-09-26 12:33:26. +0000 UTC m=+1132.) (total time: 2m0.000301191s):
Seems like the test is timing out creating the pods and reflector to watch those pods.
Looks like the section of code taking a long time is
Checked KAS log and seeing about 7000 requests of the form I1010 12:26:59. 1 ] GET /api/v1/nodes/gce-scale-cluster-minion-group-1-4p2m:/proxy/stats/summary: (9.753712ms) 200 [ (linux/amd64) kubernetes/ -- [] [sig-node] kubelet [] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. 146.148.73.158:] between replicationcontroller being created and the pod list/watch request.
The is only supposed to run if there are less than 10 nodes in the cluster. However, the situation doesn't hold with this change: (or ) was changed to . While the e2e test only uses 10 () nodes from the cluster, the resource monitor is querying all the nodes in the cluster. Thus the client qps limit was exceeded and caused the reflector to be throttled. I'll send a fix
/assign

Discussion:
nodeLabels = make(map[string]string) nodeLabels["kubelet_cleanup"] = "true" nodes, err := e2enode.GetBoundedReadySchedulableNodes(c, maxNodesToCheck) <ins> numNodes = len(nodes.Items) </ins> framework.ExpectNoError(err) nodeNames = sets.NewString() for i := 0; i < len(nodes.Items); i++ {

} // Start resourceMonitor only in small clusters. <del> if len(nodes.Items) <= maxNodesToCheck { </del> <ins> if numNodes <= maxNodesToCheck { </ins> resourceMonitor = e2ekubelet.NewResourceMonitor(f.ClientSet, e2ekubelet.TargetContainers(), containerStatsPollingInterval) resourceMonitor.Start() }
Change:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: https://k8s- Which test(s) are failing: [] [sig-node] kubelet [] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. Since when has it been failing: 2019-09-26 Testgrid link: Reason for failure: Anything else we need to know: /cc
is there someone who can pick this up since it's constantly failing in the master-informing? Thanks :)
Could you help us here?
Looked into the test logs, and it seems like the in was accidentally removed in , which caused its value to be 0. As the current log is showing: And last succeeded test log is: I'll send a PR to fix this. /cc
It seems that this PR didn't help. This run already contains this PR and it failed:
/milestone v1.17 /priority critical-urgent
I0926 12:39:42.660] I0926 12:35:26. ] Trace[]: "Reflector ListAndWatch" (started: 2019-09-26 12:33:26. +0000 UTC m=+1132.) (total time: 2m0.000301191s):
Seems like the test is timing out creating the pods and reflector to watch those pods.
Looks like the section of code taking a long time is
Checked KAS log and seeing about 7000 requests of the form I1010 12:26:59. 1 ] GET /api/v1/nodes/gce-scale-cluster-minion-group-1-4p2m:/proxy/stats/summary: (9.753712ms) 200 [ (linux/amd64) kubernetes/ -- [] [sig-node] kubelet [] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. 146.148.73.158:] between replicationcontroller being created and the pod list/watch request.
The is only supposed to run if there are less than 10 nodes in the cluster. However, the situation doesn't hold with this change: (or ) was changed to . While the e2e test only uses 10 () nodes from the cluster, the resource monitor is querying all the nodes in the cluster. Thus the client qps limit was exceeded and caused the reflector to be throttled. I'll send a fix
/assign

Discussion:
} } <ins> // While we only use a bounded number of nodes in the test. We need to know // the actual number of nodes in the cluster, to avoid running resourceMonitor // against large clusters. actualNodes, err := e2enode.GetReadySchedulableNodes(c) framework.ExpectNoError(err) </ins> // Start resourceMonitor only in small clusters. <del> if numNodes <= maxNodesToCheck { </del> <ins> if len(actualNodes.Items) <= maxNodesToCheck { </ins> resourceMonitor = e2ekubelet.NewResourceMonitor(f.ClientSet, e2ekubelet.TargetContainers(), containerStatsPollingInterval) resourceMonitor.Start() }
Change:
There's some practical limit on the number of concurrent access tokens that can be used in a Kubernetes cluster before it begins to spend all of its time in the authn stack verifying them. Absent fixes to it was < 500; with the fixes linked there it's still not infinitely scalable-- I still hit (what I believe to be) with ~ 5k unique tokens @ ~700QPS. This request is to this dimension an official name & definition, and list it with others, e.g. here: it it have a useful value (I think it should be 10k) /kind bug /sig scalability
/cc
oh and 4. add tests which verify it is as high as believed.
cc since projected tokens will increase the number of unique tokens
/sig auth
LRUs are not good at scaling and sharding has some major pitfalls as pointed out. I would suggest that we try to drop the LRU cache for successful authentication results in the token cache. The main benefit of using an LRU here is bound cache size. Entries in the token cache have a fast expiration independent of the LRU policy. The records probably take up O(2kb) (guesstimate) of memory. Given that these are successful attempts with presumably valid tokens, I don't think we are getting any benefit from the LRU. If we don't have a constraint on fixed cache size, we can back the token cache with a sync.Map and run a single background goroutine to periodically walk the map and remove expired entries. This would fix the problematic locking and eviction due to nonuniform distribution of tokens over shards. The failure cache is less important and we don't cache failures by default. If we consider this the "slowpath", we can probably leave it as. If that's not good enough, we have a couple options (with various tradeoffs) to improve it.
If off by default is not enough, more reasoning why the failed cache is not super useful: it might help with an accidental DoS but it doesn't prevent anything actively malicious DoS. I think we can focus our improvements on the success cache.
/assign He is already working on adding an explicit test for it.
has taken it over and is actively working on this. Unfortunately, I cannot assign this issue to him as he's not a k8s member yet
It's not closed - all testes must be submitted and appropriate documentation must be
Thanks everyone!!

Discussion:
"strings" "sync" "sync/atomic" <ins> "syscall" </ins> "time" "github.com/golang/glog"

return nil, fmt.Errorf("failed to select a host interface: %v", err) } <ins> err = setRLimit(64 * 1000) if err != nil { return nil, fmt.Errorf("failed to set open file handler limit: %v", err) } </ins> proxyPorts := newPortAllocator(pr) glog.V(2).Infof("Setting proxy IP to %v and initializing iptables", hostIP) return createProxier(loadBalancer, listenIP, iptables, hostIP, proxyPorts, udpIdleTimeout) } <ins> func setRLimit(limit uint64) error { return syscall.Setrlimit(syscall.RLIMIT_NOFILE, &syscall.Rlimit{Max: limit, Cur: limit}) } </ins> func createProxier(loadBalancer LoadBalancer, listenIP net.IP, iptables iptables.Interface, hostIP net.IP, proxyPorts PortAllocator, udpIdleTimeout time.Duration) (*Proxier, error) { // convenient to pass nil for tests.. if proxyPorts == nil {
Change:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: https://k8s- Which test(s) are failing: all nfs provisioner tests Since when has it been failing: 10/1/2019 Testgrid link: https://k8s- Reason for failure: tbd Anything else we need to know:
cc
fix on the way, got merged but when it rebased the version wasn't updated (needs to be v2.2.2 for that selflink thing)

Discussion:
configs[Mounttest] = Config{e2eRegistry, "mounttest", "1.0"} configs[MounttestUser] = Config{e2eRegistry, "mounttest-user", "1.0"} configs[Nautilus] = Config{e2eRegistry, "nautilus", "1.0"} <del> configs[NFSProvisioner] = Config{quayIncubator, "nfs-provisioner", "v2.2.0-k8s1.12"} </del> <ins> configs[NFSProvisioner] = Config{quayIncubator, "nfs-provisioner", "v2.2.2"} </ins> configs[Nginx] = Config{dockerLibraryRegistry, "nginx", "1.14-alpine"} configs[NginxNew] = Config{dockerLibraryRegistry, "nginx", "1.15-alpine"} configs[Nonewprivs] = Config{e2eRegistry, "nonewprivs", "1.0"}
Change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We have enabled in genericscheduler. However, the steps being traced seem outdated. In particular, the "snapshot" is expensive and should be traced. Other steps can be traced as well wherever appropriate, e.g., PreFilter. Why is this needed: Improve granularity of tracing. /sig scheduling
/assign

Discussion:
if err := podPassesBasicChecks(pod, g.pvcLister); err != nil { return result, err } <ins> trace.Step("Basic checks done") </ins> // Run "prefilter" plugins. preFilterStatus := g.framework.RunPreFilterPlugins(state, pod) if !preFilterStatus.IsSuccess() { return result, preFilterStatus.AsError() } <ins> trace.Step("Running prefilter plugins done") </ins> numNodes := g.cache.NodeTree().NumNodes() if numNodes == 0 {

if err := g.snapshot(); err != nil { return result, err } <ins> trace.Step("Snapshoting scheduler cache and node infos done") </ins> <del> trace.Step("Basic checks done") </del> startPredicateEvalTime := time.Now() filteredNodes, failedPredicateMap, filteredNodesStatuses, err := g.findNodesThatFit(state, pod) if err != nil { return result, err } <ins> trace.Step("Computing predicates done") </ins> // Run "postfilter" plugins. postfilterStatus := g.framework.RunPostFilterPlugins(state, pod, filteredNodes, filteredNodesStatuses)

FilteredNodesStatuses: filteredNodesStatuses, } } <del> trace.Step("Computing predicates done") </del> <ins> trace.Step("Running postfilter plugins done") </ins> metrics.SchedulingAlgorithmPredicateEvaluationDuration.Observe(metrics.SinceInSeconds(startPredicateEvalTime)) metrics.DeprecatedSchedulingAlgorithmPredicateEvaluationDuration.Observe(metrics.SinceInMicroseconds(startPredicateEvalTime)) metrics.SchedulingLatency.WithLabelValues(metrics.PredicateEvaluation).Observe(metrics.SinceInSeconds(startPredicateEvalTime))

if err != nil { return result, err } <del> trace.Step("Prioritizing done") </del> metrics.SchedulingAlgorithmPriorityEvaluationDuration.Observe(metrics.SinceInSeconds(startPriorityEvalTime)) metrics.DeprecatedSchedulingAlgorithmPriorityEvaluationDuration.Observe(metrics.SinceInMicroseconds(startPriorityEvalTime)) metrics.SchedulingLatency.WithLabelValues(metrics.PriorityEvaluation).Observe(metrics.SinceInSeconds(startPriorityEvalTime)) metrics.DeprecatedSchedulingLatency.WithLabelValues(metrics.PriorityEvaluation).Observe(metrics.SinceInSeconds(startPriorityEvalTime)) host, err := g.selectHost(priorityList) <del> trace.Step("Selecting host done") </del> <ins> trace.Step("Prioritizing done") </ins> return ScheduleResult{ SuggestedHost:  host, EvaluatedNodes: len(filteredNodes) + len(failedPredicateMap) + len(filteredNodesStatuses),
Change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Add a metric to track scheduler cache size. metric name Description -- -- schedulercachesize Number of elements in scheduler cache. Why is this needed: Gives better observability of scheduler cache. For, the number of "assumedpods" gives us an idea of how many pods are "assumed" and therefore being "bound". We can compare this with number of to make sure assume-bind works as expected. /sig scheduling
Is anyone already working on these metrics? If not I'd like to help out /assign
Thanks . Feel free to work on this one. Just a reminder, we may need feedback/approval from sig-instrumentation.

Discussion:
deps = [ "//pkg/features:go_default_library", "//pkg/scheduler/listers:go_default_library", <ins> "//pkg/scheduler/metrics:go_default_library", </ins> "//pkg/scheduler/nodeinfo:go_default_library", "//pkg/scheduler/nodeinfo/snapshot:go_default_library", "//pkg/util/node:go_default_library",

"k8s.io/klog" "k8s.io/kubernetes/pkg/features" schedulerlisters "k8s.io/kubernetes/pkg/scheduler/listers" <ins> "k8s.io/kubernetes/pkg/scheduler/metrics" </ins> schedulernodeinfo "k8s.io/kubernetes/pkg/scheduler/nodeinfo" nodeinfosnapshot "k8s.io/kubernetes/pkg/scheduler/nodeinfo/snapshot" )

} // cleanupAssumedPods exists for making test deterministic by taking time as input argument. <ins> // It also reports metrics on the cache size for nodes, pods, and assumed pods. </ins> func (cache *schedulerCache) cleanupAssumedPods(now time.Time) { cache.mu.Lock() defer cache.mu.Unlock() <ins> defer cache.updateMetrics() </ins> // The size of assumedPods should be small for key := range cache.assumedPods {

return n.info.Node(), nil } <ins> // updateMetrics updates cache size metric values for pods, assumed pods, and nodes func (cache *schedulerCache) updateMetrics() { metrics.CacheSize.WithLabelValues("assumed_pods").Set(float64(len(cache.assumedPods))) metrics.CacheSize.WithLabelValues("pods").Set(float64(len(cache.podStates))) metrics.CacheSize.WithLabelValues("nodes").Set(float64(len(cache.nodes))) } </ins>

}, []string{"result"}) <ins> CacheSize = metrics.NewGaugeVec( &metrics.GaugeOpts{ Subsystem:      SchedulerSubsystem, Name:           "scheduler_cache_size", Help:           "Number of nodes, pods, and assumed (bound) pods in the scheduler cache.", StabilityLevel: metrics.ALPHA, }, []string{"type"}) </ins> metricsList = []metrics.Registerable{ scheduleAttempts, SchedulingLatency,

SchedulerQueueIncomingPods, SchedulerGoroutines, PermitWaitDuration, <ins> CacheSize, </ins> } )
Change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : storage/v1.CSINode is being introduced in 1.17 via The storage version should switch to v1 in 1.18. Why is this needed: We need to wait until 1.18 to handle the HA upgrade case where a controller may be 1 version higher than the API server. /assign
These files will have to be updated:

Discussion:
resources := []schema.GroupVersionResource{ batch.Resource("cronjobs").WithVersion("v1beta1"), networking.Resource("ingresses").WithVersion("v1beta1"), <del> // TODO #83513 csinodes override can be removed in 1.18 apisstorage.Resource("csinodes").WithVersion("v1beta1"), </del> apisstorage.Resource("csidrivers").WithVersion("v1beta1"), }

etcdStorageData[gvr("storage.k8s.io", "v1beta1", "csinodes")] = StorageData{ Stub:             `{"metadata": {"name": "csini1"}, "spec": {"drivers": [{"name": "test-driver", "nodeID": "localhost", "topologyKeys": ["company.com/zone1", "company.com/zone2"]}]}}`, ExpectedEtcdPath: "/registry/csinodes/csini1", <ins> ExpectedGVK:      gvkP("storage.k8s.io", "v1", "CSINode"), </ins> } // k8s.io/kubernetes/pkg/apis/storage/v1 etcdStorageData[gvr("storage.k8s.io", "v1", "csinodes")] = StorageData{ Stub:             `{"metadata": {"name": "csini2"}, "spec": {"drivers": [{"name": "test-driver", "nodeID": "localhost", "topologyKeys": ["company.com/zone1", "company.com/zone2"]}]}}`, ExpectedEtcdPath: "/registry/csinodes/csini2", <del> ExpectedGVK:      gvkP("storage.k8s.io", "v1beta1", "CSINode"), </del> } }
Change:
What happened: When debugging timeouts related to the stopping of cgroups, the current logs from -tryKillingCgroupProcesses do not provide any correlation between the PIDs which were killed and the cgroup they belonged to. What you expected to happen: Log the cgroup name along with the PID How to reproduce it (as minimally and precisely as possible): N/A Anything else we need to know?: N/A Environment: Kubernetes version (use ): master
/sig node

Discussion:
"path" "strings" <del> "k8s.io/api/core/v1" </del> <ins> v1 "k8s.io/api/core/v1" </ins> "k8s.io/apimachinery/pkg/types" utilerrors "k8s.io/apimachinery/pkg/util/errors" utilfeature "k8s.io/apiserver/pkg/util/feature"

removed := map[int]bool{} for i := 0; i < 5; i++ { if i != 0 { <del> klog.V(3).Infof("Attempt %v failed to kill all unwanted process. Retyring", i) </del> <ins> klog.V(3).Infof("Attempt %v failed to kill all unwanted process from cgroup: %v. Retyring", i, podCgroup) </ins> } errlist = []error{} for _, pid := range pidsToKill { if _, ok := removed[pid]; ok { continue } <del> klog.V(3).Infof("Attempt to kill process with pid: %v", pid) </del> <ins> klog.V(3).Infof("Attempt to kill process with pid: %v from cgroup: %v", pid, podCgroup) </ins> if err := m.killOnePid(pid); err != nil { <del> klog.V(3).Infof("failed to kill process with pid: %v", pid) </del> <ins> klog.V(3).Infof("failed to kill process with pid: %v from cgroup: %v", pid, podCgroup) </ins> errlist = append(errlist, err) } else { removed[pid] = true } } if len(errlist) == 0 { <del> klog.V(3).Infof("successfully killed all unwanted processes.") </del> <ins> klog.V(3).Infof("successfully killed all unwanted processes from cgroup: %v", podCgroup) </ins> return nil } }