jiebi/CodeConvo huggingface dataset:
Change:
func (s *sourceEtcd) run() { boundPods := api.BoundPods{} <del> err := s.helper.ExtractToList(s.key, &boundPods) </del> <ins> err := s.helper.ExtractObj(s.key, &boundPods, false) </ins> if err != nil { glog.Errorf("etcd failed to retrieve the value for the key %q. Error: %v", s.key, err) return

Discussion:
I believe this is a regression with HEAD. Everything runs happily until I upgrade it to HEAD: influx-grafana/ influxdb kubernetes/heapster_influxdb kubernetes-minion- name=influxdb Running Then, I ssh into minion-2 and stop the kubelet. After this, all pods' status in cluster become to Unknown. Then I restart kubelet, and wait for long time (more than 20 mins), but all pods' status stay at Unknown:
I will take a look.
Now I suspect it is not caused by an unhealthy minion. I noticed that another issue filed by today (), all pods' status are Unknown too.
True problem is ExtractList instead of ExtractObj. This will always read an empty boundpods object: But the watch afterwards works correctly. So the weird behavior is that it kills everything when the watch expires. sorry I failed to notice this when reviewing your change! BoundPods isn't technically a list even though it looks like it. (the bound pods are not stored as sub-keys in etcd)
It turns out the issue is more serious than what I initially expected. Any pods scheduled to a node by apiserver, and running for a while will be killed by kubelet because a bug caused by But any changes to etcd key of that boundpods under /registry/nodes/<XX/boundpods will make watch result correct, and restarting previous killed boundPods. To prove my theory, I delete one pod php1 from my test cluster. Before deletion, all pods are unknown: Now after I delete php1 by running: delete -f testing/php1 Running: cluster/../cluster/gce/../../_output/dockerized/bin/linux/amd64/kubectl delete -f testing/php1 php1 All pods on same node (kubernetes-minion-) are restarted I think I know the root cause is, and will sent a fix soon.
I think you hit the same issue earlier.
maybe you can write a little test for this? :)
Change:
store Store // listerWatcher is used to perform lists and watches. listerWatcher ListerWatcher <del> // period controls timing between an unsuccessful watch ending and // the beginning of the next list. period time.Duration // The period at which ShouldResync is invoked </del> <ins> // backoff manages backoff of ListWatch backoffManager wait.BackoffManager </ins> resyncPeriod time.Duration // ShouldResync is invoked periodically and whenever it returns `true` the Store's Resync operation is invoked ShouldResync func() bool

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: We recently had a production issue where one of our large cluster (~3k nodes) has it's API server OOMKilled. This caused all kubelets' connection to master broken, and caused a global synchronization of kubelet's reconnecting to master. Due to the burst of API call (18K QPS, 7x than normal), API server went into even worse condition, and caused more replicas to get OOMKilled. Current reflector impl has fixed 1sec backoff before calling next , which shall be the root cause of such instability. What you expected to happen: reflectors to backoff with jitter How to reproduce it (as minimally and precisely as possible): a cluster with a few thousands of nodes api server and reboot it Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /assign /sig api-machinery scalability
/cc
Change:
volName string // the underlying plugin plugin *flexVolumePlugin <ins> // the metric plugin volume.MetricsProvider </ins> } // volume.Volume interface

Discussion:
kind feature What happened: Currently there is no mechanism to report metrics as the flex plugins are not enabled to collect metrics. All flex volumes lack this feature as compared to in-tree/built-in drivers even though it is possible to collect such metrics for most of the plugins. What you expected to happen: Depending on the availability of the metrics from the mounted FS, flex plugins should allow to report metrics for the FS. How to reproduce it (as minimally and precisely as possible): The default behavior that can be proved easily by just running a command similar to the following command on a node that is running a flex volume. This will show that there are no metrics collected for flex volumes. kubectl proxy $curl 127.0.0.1:8001/api/v1/nodes/<IP/proxy/stats/summary Environment**: Kubernetes version (use ): 1.12 Cloud provider or hardware configuration: local x86 OS (e.g. from /etc/os-release): Ubuntu 16.04 Kernel (e.g. ): Linux KubeStorage 4.4.0-130-generic -Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x8664 x8664 x86_64 GNU/Linux Install tools: Others: /sig storage
+1 Yes, it will be good, if flex volume can support atleast the default metrics( i.e ), which most of the volume plugins support now. Based upon flex plugin "capabilities" parameters, it can be decided, whether the plugin supports "default metrics" or "nil metrics".
Change:
if pod == nil { return nil } <del> tolerations, err := getTolerationListFromPod(pod) if err != nil { return nil } </del> <ins> tolerationsPreferNoSchedule := getAllTolerationPreferNoSchedule(pod.Spec.Tolerations) </ins> return &priorityMetadata{ nonZeroRequest: getNonZeroRequests(pod), <del> podTolerations: tolerations, </del> <ins> podTolerations: tolerationsPreferNoSchedule, </ins> affinity:       schedulercache.ReconcileAffinity(pod), } }

Discussion:
Kubernetes version (use kubectl version): master What happened: Unnecessary always nil return value for function getTolerationListFromPod. What you expected to happen: We need to remove error return value for getTolerationListFromPod function.
/cc
cc
Change:
"github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" <ins> "github.com/onsi/gomega/gstruct" </ins> ) // Validate PV/PVC, create and verify writer pod, delete the PVC, and validate the PV's

Discussion:
[x] test approval issue : [x] test pr : [x] Flake [x] two weeks soak end date : 31 Oct 2023 [X] test promotion pr : [X] remove endpoint from According to following APIsnoop query, there are three PersistentVolume and three PersistentVolumeClaim endpoints that are untested. Using a number of existing e2e test practices a new has been created to provide future Conformance coverage for the 6 endpoints. The e2e logs for this test are listed below. This query shows the following PV and PVC endpoints are hit within a short period of running this e2e test. If a test with these calls gets merged, test coverage will go up by 6 points This test is also created with the goal of conformance promotion. /sig testing /sig architecture /area conformance
/triage accepted /cc
/reopen For tracking the rest of the steps to Promotion
Reopened this issue. <detailsIn response to : /reopen For tracking the rest of the steps to Promotion Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Change:
_ "k8s.io/kubernetes/test/e2e/framework/providers/azure" _ "k8s.io/kubernetes/test/e2e/framework/providers/gce" _ "k8s.io/kubernetes/test/e2e/framework/providers/kubemark" <ins> _ "k8s.io/kubernetes/test/e2e/framework/providers/openstack" </ins> ) var (

Discussion:
What happened: When specifying "openstack" as cloud-provider of e2e test, the test is failed like What you expected to happen: We can run e2e test with openstack cloud provider. How to reproduce it (as minimally and precisely as possible): Just kick the following command: ` Anything else we need to know?: This issue doesn't happen on v1.13, so this issue seems to happen during this dev cycle for v1.14. Environment: Kubernetes version (use ): master for v1.14 Cloud provider or hardware configuration: OpenStack OS (e.g: ): Ubuntu 16.04.5 LTS Kernel (e.g. ): 4.4.0-142-generic Install tools: kubeadm Others:
/sig openstack /sig testing
The exception raises at
The reason of this issue comes from
Ideally it is nice to put a new directory for registering the cloud provider.
/assign
I was wondering we could remove "openstack" provider specific tests and conditions from e2e tests, but there are storage tests which are related to OpenStack in e2e tests. So it is nice to add registration for OpenStack at this time.
Change:
// DeleteRoute from the cloud environment. func (g *Cloud) DeleteRoute(ctx context.Context, clusterName string, route *cloudprovider.Route) error { <del> timeoutCtx, cancel := cloud.ContextWithCallTimeout() </del> <ins> timeoutCtx, cancel := context.WithTimeout(ctx, 1*time.Hour) </ins> defer cancel() mc := newRoutesMetricContext("delete")

Discussion:
What happened: Ref , there are some GCE wrapper functions that don't properly respect the passed in context, e.g: We need to figure out a better way to handle this, but I'm hesitate to change it right away without enough thinking. File this issue for tracking purpose. cc
/sig cloud-provider /area provider/gcp
How about checking the deadline of the passed in ctx: If the timeout is longer than what cloud.ContextWithCallTimeout expects, create context with 1 hour timeout.
We should consider () as well as deadline. How about monitor user's timeout like this? cancel our context when timeout.
If the time period till ctx.Deadline() is shorter than 1 hour, there is no need to call cloud.ContextWithCallTimeout(). We can use ctx directly in that case.
Another option may be better:
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Change:
return utilerrors.NewAggregate(errList) } <del> // Find process(es) with a specified name (exact match) </del> <ins> // Find process(es) with a specified name (regexp match) </ins> // and return their pid(s) func PidOf(name string) ([]int, error) { if len(name) == 0 {

Discussion:
The description of says: However the implementation will interpret the name as a regex. So for example "..." will match any three-letter string. This is not what I understand by "exact match".
Will open an PR fix this comment
Change:
SyncFrequency:      *syncFrequency, HTTPCheckFrequency: *httpCheckFrequency, } <del> my_kubelet.RunKubelet(*config, *manifestUrl, *etcdServers, *address, *port) </del> <ins> my_kubelet.RunKubelet(*config, *manifestUrl, *etcdServers, *address, *dockerEndpoint, *port) </ins> }

Discussion:
When using or with a key/value that has a multi-line string as value, that value will appear as quoted string in the log message, i.e. the log message will still be a single line. It would be nicer to print the value in text mode so that it goes across multiple lines, with clear indicators for start and end of the value. Indention could be used to avoid mistaking some embedded line as the start of a new log message. In JSON mode, the value should be formatted normally. /sig instrumentation /wg structured-logging Always printing as a single line can can make the log output unreadable ().
When adding something like this, can be reverted and implemented differently.
My original thinking was that only objects that explicitly ask for multi-line output get formatted that way. But that means that we need to review log calls very carefully. An alternative is to automatically switch from quoting strings to multi-line output based on some heuristic, for example when there's more than one line break.
Here's a potential implementation:
If structured logging is needed in any other case I can contribute, please inform if needed.
/triage accepted /assign /priority important-soon /area logging /cc /cc
We have the same problem as in kube-scheduler also in kubelet, it's just that no-one has seen and/or complained about the multi-line strings. With the benchmark + log analysis tool from I found the following multi-line string parameters in a kubelet log: <details</details
Change:
func makeEvent(ref *api.ObjectReference, reason, message string) *api.Event { t := util.Now() <ins> namespace := ref.Namespace if namespace == "" { namespace = api.NamespaceDefault } </ins> return &api.Event{ ObjectMeta: api.ObjectMeta{ Name:      fmt.Sprintf("%v.%x", ref.Name, t.UnixNano()), <del> Namespace: ref.Namespace, </del> <ins> Namespace: namespace, </ins> }, InvolvedObject: *ref, Reason:         reason,

Discussion:
Any node object related events from the kubelet does not get recorded in the apiserver. I digged a bit deeper and it looks like kubelet does not sets namespace to be empty and this results in the kubelet trying to post to `/api/<version/events', which a resource that is not exported by the apiserver.
cc
Change:
- ./hack/verify-description.sh - ./hack/travis/install-std-race.sh - ./hack/build-go.sh <ins> - GOPATH=$PWD/Godeps/_workspace:$GOPATH go install ./... </ins> script: - KUBE_RACE="-race" KUBE_COVER="y" KUBE_GOVERALLS_BIN="$HOME/gopath/bin/goveralls" KUBE_TIMEOUT='-timeout 300s' KUBE_COVERPROCS=8 ./hack/test-go.sh -- -p=2

Discussion:
Several times over the past few months I've run into HEAD throwing compilation errors when doing , which may not prevent proper functioning of kubernetes but makes some commands during development like very hard to do. It would be great if we could add to Travis CI to prevent this from happening.
See for discussion.
Fix is merged.
This should be re-opened. Travis still doesn't build all packages with or so HEAD still breaks. See for rationale.
Do you mind reopening for now?
Sure. the fix that was merged was for the specific problem, but this bug is about the general class of problems.
Change:
} go func() { defer util.HandleCrash() <del> glog.Fatal(readOnlyServer.ListenAndServe()) </del> <ins> for { if err := readOnlyServer.ListenAndServe(); err != nil { glog.Errorf("Unable to listen for read only traffic (%v); will try again.", err) } time.Sleep(15 * time.Second) } </ins> }() }

Discussion:
I'm not using publicaddressoverride. At boot I'm trying to launch using: /usr/bin/kube-apiserver --logtostderr=true --v=0 --etcdservers=http://10.13.137.165:4001 --address=0.0.0.0 --port=8080 --kubeletport= --allowprivileged=true --portalnet=10.254.0.0/16 But this gets launched before eth0 has an ip address. This used to work. Now apiserver just refused to start (complaining about 127.0.0.1 being invalid). I think it needs to just keep trying until it gets an address and can get to work....
Adding --publicaddressoverride= as suggested doesn't help.... Nov 05 12:43:55 localhost kube-apiserver[713]: F1105 12:43:55. ] listen tcp 10.13.137.165:7080: bind: cannot assign requested address Which is a fatal error. I can't bind to it. It isn't up yet.
Bisect says either commit or
If publicaddressoverride is specified, I can put the read-only server bind in a goroutine and retry periodically. If it's not, though, since we can't change the public address once started, we'd have to delay construction until a suitable address shows up. That's pretty ugly; is it an important case to support?
I personally think delaying construction is fine. Being able to launch the apiserver on any node from the init system and have it just work 99% of the time seems like a good thing....
OK. Fix coming in 3, 2, 1...
See if does the trick for you.
commented there. it fixed it.
Change:
// If non-nil, use this IP address for the node nodeIP net.IP <ins> // use this function to validate the kubelet nodeIP nodeIPValidator func(net.IP) error </ins> // If non-nil, this is a unique identifier for the node in an external database, eg. cloudprovider providerID string

Discussion:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If this may be security issue, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: When using the flag on the kubelet, the node's list no longer reports , , addresses. What you expected to happen: That I can explicitly set to an IP, but the other addresses will still show up. How to reproduce it (as minimally and precisely as possible): Create a new kops cluster, modify the kubelet to use the flag with the eth0 on the node. Run Anything else we need to know?: This change will probably take place in  This came up as a result of Environment: Kubernetes version (use ): 1.9.x Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): any Kernel (e.g. ): any Install tools: kops /sig network /sig node I'll make a PR for this change shortly
Change:
done } <ins> function kube::release::md5() { if which md5 >/dev/null 2>&1; then md5 -q "$1" else md5sum "$1" | awk '{ print $1 }' fi } </ins> # This will take binaries that run on master and creates Docker images # that wrap the binary in them. (One docker image per binary) function kube::release::create_docker_images_for_server() { # Create a sub-shell so that we don't pollute the outer environment ( <del> local binary_name; </del> <ins> local binary_name </ins> for binary_name in "${KUBE_DOCKER_WRAPPED_BINARIES[@]}"; do <del> echo "+++ Building docker image: ${binary_name}"; local docker_file_path="$1/Dockerfile"; local binary_file_path="$1/${binary_name}"; if [ -f ${docker_file_path} ]; then rm ${docker_file_path}; fi; printf " FROM scratch n ADD ${binary_name} /${binary_name} n ENTRYPOINT [ "/${binary_name}" ]n" >> ${docker_file_path}; local md5_sum=$(md5sum ${binary_file_path} | awk '{print $1}') local docker_image_tag=gcr.io/google_containers/$binary_name:$md5_sum docker build -t "${docker_image_tag}" ${1}; docker save ${docker_image_tag} > ${1}/${binary_name}.tar; echo $md5_sum > ${1}/${binary_name}.docker_tag; rm ${docker_file_path}; </del> <ins> kube::log::status "Starting Docker build for image: ${binary_name}" ( local docker_file_path="$1/${binary_name}.Dockerfile" local binary_file_path="$1/${binary_name}" if [ -f ${docker_file_path} ]; then rm ${docker_file_path} fi printf " FROM scratch n ADD ${binary_name} /${binary_name} n ENTRYPOINT [ "/${binary_name}" ]n" >> ${docker_file_path} local md5_sum=$(kube::release::md5 ${binary_file_path}) local docker_image_tag=gcr.io/google_containers/$binary_name:$md5_sum docker build -q -f "${docker_file_path}" -t "${docker_image_tag}" ${1} >/dev/null docker save ${docker_image_tag} > ${1}/${binary_name}.tar echo $md5_sum > ${1}/${binary_name}.docker_tag rm ${docker_file_path} ) & </ins> done <ins> wait || { kube::log::error "previous Docker build failed"; return 1; } kube::log::status "Docker builds done" </ins> ) }

Discussion:
Filing for This code is wrong The local needs to be on a separate line from the command execution, and we need to be resilient to not existing on OS X, like:
For once I was actually watching the output of a build and noticed that I was seeing an error about md5sum not being found. I'm building on a mac using boot2docker. The short_hash function () works fine on my mac as md5 does exist.
Actually, I'm just going to fix this as a follow-on to .
Thank you!
Change:
Name: "should-be-removable-mutating-webhook.k8s.io", Rules: []v1beta1.RuleWithOperations{{ Operations: []v1beta1.OperationType{v1beta1.Create}, <ins> // This will not match any real resources so this webhook should never be called. </ins> Rule: v1beta1.Rule{ <del> APIGroups:   []string{"*"}, APIVersions: []string{"*"}, Resources:   []string{"*"}, </del> <ins> APIGroups:   []string{""}, APIVersions: []string{"v1"}, Resources:   []string{"invalid"}, </ins> }, }}, ClientConfig: v1beta1.WebhookClientConfig{

Discussion:
This flaked 3/4 times in the last week. /priority failing-test /priority important-soon /sig api-machinery /kind flake
/assign
Pinging here for latest status on this. This test is now one of our top flakes failing pull-kubernetes-e2e-gce job 40 times.
Here's the latest test run results for this test to get more logs. https://k8s-
The error message is really similar to the one in I'm looking into if the fix for that will also be sufficient to fix this.
We may want to reopen this until the fix is shown to actually work /reopen
Thanks for taking care of this. I will monitor this test and ping you back here to close when appropriate.
No failure 2 days since the fix. the fix seemed to have worked. So please go ahead and close the issue. Thanks much.
/close
Change:
out.CloudProvider = in.CloudProvider out.NodeName = in.NodeName out.AuthorizationModes = *(*[]string)(unsafe.Pointer(&in.AuthorizationModes)) <ins> out.PrivilegedPods = in.PrivilegedPods </ins> out.Token = in.Token out.TokenTTL = (*v1.Duration)(unsafe.Pointer(in.TokenTTL)) out.APIServerExtraArgs = *(*map[string]string)(unsafe.Pointer(&in.APIServerExtraArgs))

Discussion:
Dear All: I'm not sure if I'm right. When i use whith .I found controller-manager logs Someone can help me? Thanks a lot.
There are no sig labels on this issue. Please by:<br(1) mentioning a sig: <br(2) specifying the label manually: <br<brNote: method (1) will trigger a notification to the team. You can find the team list and label list
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Please give more information about your set up cc
Sorry for report later. My openstack didn't support metadata service,but support config-drive service. More info:
Thanks Based on this, I think it's an openstack cloudprovider problem more than a kubeadm problem...
Thank you for help me. I will change the label.
/area cloudprovider
/remove-kind support
Should be fixed with you can specify a search order for metadata and config drive /close
Specifying a search order doesn't really solve the problem, especially if the metadata api isn't available which is what ZhenyangZhao said . I'm getting the same issue and since the config drive is my only option at the moment.
Like for this to be /open
can you please provide logs/versions for your problem in your environment? Thanks, Dims
Let me know if you need anything else Kubernetes version: 1.8.4 Docker version: 17.09.0-ce Linux: CentOS 7.4 Looking at the docker container log for the api-server (I don't have problems with the controller-manager):
and share the following:
Hi! Thank you for filing this issue. We appreciate your interest in Kubernetes and kubeadm. In order to make the kubernetes/kubernetes issue tracker more usable, we're routing issues to different repositories with different focuses. I think this issue belongs in the kubeadm repository. Please move this issue to the in order for the kubeadm team to be able to track this efficiently. If you disagree with my assessment of moving this issue, please speak up and/or reopen the issue. Finally, please try to reproduce the bug with the latest Kubernetes version, it might as well already have been fixed :) Thank you and have a good day!