jiebi/CodeConvo huggingface dataset:
Code change:
We have to spend times on figuring out what are the mount error code means from issues and . In , the error message is and is great for diagnosing mount problem. The console message better be returned rather than just error code.

Related discussion:
text: command := exec.Command("umount", target) output, err := command.CombinedOutput() if err != nil { <del> glog.Errorf("Unmount failed: %vnUnmounting arguments: %snOutput: %sn", err, target, string(output)) return err </del> <ins> return fmt.Errorf("Unmount failed: %vnUnmounting arguments: %snOutput: %sn", err, target, string(output)) </ins> } return nil }
Code change:
Heapster pod: And logs: cc p0 for triage
Did the required node scope change?
Its either the API on the console or monitoring write scopes on the node. , Mike Danese wrote:
I think this affects only GCE
I will fix this issue since is on vacation.
Need to cherry pick so i'll leave open
is out for review.
has been closed and the cherrypick here () has been merged so I'm closing this issue. Thanks everyone for the debugging.

Related discussion:
text: #   google         - Heapster, Google Cloud Monitoring, and Google Cloud Logging #   googleinfluxdb - Enable influxdb and google (except GCM) #   standalone     - Heapster only. Metrics available via Heapster REST API. <del> ENABLE_CLUSTER_MONITORING="${KUBE_ENABLE_CLUSTER_MONITORING:-googleinfluxdb}" </del> <ins> ENABLE_CLUSTER_MONITORING="${KUBE_ENABLE_CLUSTER_MONITORING:-influxdb}" </ins> # Optional: Enable node logging. ENABLE_NODE_LOGGING="${KUBE_ENABLE_NODE_LOGGING:-true}"
Code change:
Heapster pod: And logs: cc p0 for triage
Did the required node scope change?
Its either the API on the console or monitoring write scopes on the node. , Mike Danese wrote:
I think this affects only GCE
I will fix this issue since is on vacation.
Need to cherry pick so i'll leave open
is out for review.
has been closed and the cherrypick here () has been merged so I'm closing this issue. Thanks everyone for the debugging.

Related discussion:
text: AUTOSCALER_MIN_NODES="${KUBE_AUTOSCALER_MIN_NODES:-1}" AUTOSCALER_MAX_NODES="${KUBE_AUTOSCALER_MAX_NODES:-${NUM_MINIONS}}" TARGET_NODE_UTILIZATION="${KUBE_TARGET_NODE_UTILIZATION:-0.7}" <ins> ENABLE_CLUSTER_MONITORING=googleinfluxdb </ins> fi # Optional: Enable deployment experimental feature, not ready for production use.
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: # Kubernetes Cluster Admin Guide: Cluster Components <ins> **Table of Contents** <!-- BEGIN MUNGE: GENERATED_TOC --> - [Kubernetes Cluster Admin Guide: Cluster Components](#kubernetes-cluster-admin-guide-cluster-components) - [Master Components](#master-components) - [kube-apiserver](#kube-apiserver) - [etcd](#etcd) - [kube-controller-manager](#kube-controller-manager) - [kube-scheduler](#kube-scheduler) - [addons](#addons) - [DNS](#dns) - [User interface](#user-interface) - [Container Resource Monitoring](#container-resource-monitoring) - [Cluster-level Logging](#cluster-level-logging) - [Node components](#node-components) - [kubelet](#kubelet) - [kube-proxy](#kube-proxy) - [docker](#docker) - [rkt](#rkt) - [monit](#monit) - [fluentd](#fluentd) <!-- END MUNGE: GENERATED_TOC --> </ins> This document outlines the various binary components that need to run to deliver a functioning Kubernetes cluster.
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: Addon objects are created in the "kube-system" namespace. <del> Example addons are: * [DNS](http://releases.k8s.io/HEAD/cluster/addons/dns/) provides cluster local DNS. * [kube-ui](http://releases.k8s.io/HEAD/cluster/addons/kube-ui/) provides a graphical UI for the cluster. * [fluentd-elasticsearch](http://releases.k8s.io/HEAD/cluster/addons/fluentd-elasticsearch/) provides log storage. Also see the [gcp version](http://releases.k8s.io/HEAD/cluster/addons/fluentd-gcp/). * [cluster-monitoring](http://releases.k8s.io/HEAD/cluster/addons/cluster-monitoring/) provides monitoring for the cluster. </del> <ins> #### DNS While the other addons are not strictly required, all Kubernetes clusters should have [cluster DNS](dns.md), as many examples rely on it. Cluster DNS is a DNS server, in addition to the other DNS server(s) in your environment, which serves DNS records for Kubernetes services. Containers started by Kubernetes automatically include this DNS server in their DNS searches. #### User interface The kube-ui provides a read-only overview of the cluster state.  Access [the UI using kubectl proxy](../user-guide/connecting-to-applications-proxy.md#connecting-to-the-kube-ui-service-from-your-local-workstation) #### Container Resource Monitoring [Container Resource Monitoring](../user-guide/monitoring.md) records generic time-series metrics about containers in a central database, and provides a UI for browsing that data. #### Cluster-level Logging [Container Logging](../user-guide/monitoring.md) saves container logs to a central log store with search/browsing interface.  There are two implementations: * [Cluster-level logging to Google Cloud Logging]( docs/user-guide/logging.md#cluster-level-logging-to-google-cloud-logging) * [Cluster-level Logging with Elasticsearch and Kibana]( docs/user-guide/logging.md#cluster-level-logging-with-elasticsearch-and-kibana) </ins> ## Node components
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: `monit` is a lightweight process babysitting system for keeping kubelet and docker running. <ins> ### fluentd `fluentd` is a daemon which helps provide [cluster-level logging](#cluster-level-logging). </ins> <!-- BEGIN MUNGE: GENERATED_ANALYTICS --> [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/admin/cluster-components.md?pixel)]()
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: - [Scheduler pod template](#scheduler-pod-template) - [Controller Manager Template](#controller-manager-template) - [Starting and Verifying Apiserver, Scheduler, and Controller Manager](#starting-and-verifying-apiserver-scheduler-and-controller-manager) <del> - [Logging](#logging) - [Monitoring](#monitoring) - [DNS](#dns) </del> <ins> - [Starting Cluster Services](#starting-cluster-services) </ins> - [Troubleshooting](#troubleshooting) - [Running validate-cluster](#running-validate-cluster) - [Inspect pods and services](#inspect-pods-and-services)
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: - Otherwise, if taking the firewall-based security approach - `--api-servers=http://$MASTER_IP` - `--config=/etc/kubernetes/manifests` <del> - `--cluster-dns=` to the address of the DNS server you will setup (see [Starting Addons](#starting-addons).) </del> <ins> - `--cluster-dns=` to the address of the DNS server you will setup (see [Starting Cluster Services](#starting-cluster-services).) </ins> - `--cluster-domain=` to the dns domain prefix to use for cluster DNS addresses. - `--docker-root=` - `--root-dir=`
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: You should soon be able to see all your nodes by running the `kubectl get nodes` command. Otherwise, you will need to manually create node objects. <del> ### Logging **TODO** talk about starting Logging. ### Monitoring **TODO** talk about starting Monitoring. ### DNS **TODO** talk about starting DNS. </del> <ins> ### Starting Cluster Services You will want to complete your Kubernetes clusters by adding cluster-wide services.  These are sometimes called *addons*, and [an overview of their purpose is in the admin guide]( ../../docs/admin/cluster-components.md#addons). Notes for setting up each cluster service are given below: * Cluster DNS: * required for many kubernetes examples * [Setup instructions](http://releases.k8s.io/HEAD/cluster/addons/dns/) * [Admin Guide](../admin/dns.md) * Cluster-level Logging * Multiple implementations with different storage backends and UIs. * [Elasticsearch Backend Setup Instructions](http://releases.k8s.io/HEAD/cluster/addons/fluentd-elasticsearch/) * [Google Cloud Logging Backend Setup Instructions](http://releases.k8s.io/HEAD/cluster/addons/fluentd-gcp/). * Both require running fluentd on each node. * [User Guide](../user-guide/logging.md) * Container Resource Monitoring * [Setup instructions](http://releases.k8s.io/HEAD/cluster/addons/cluster-monitoring/) * GUI * [Setup instructions](http://releases.k8s.io/HEAD/cluster/addons/kube-ui/) cluster. </ins> ## Troubleshooting
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: # Connecting to applications: kubectl proxy and apiserver proxy <ins> <!-- BEGIN MUNGE: GENERATED_TOC --> - [Connecting to applications: kubectl proxy and apiserver proxy](#connecting-to-applications-kubectl-proxy-and-apiserver-proxy) - [Getting the apiserver proxy URL of kube-ui](#getting-the-apiserver-proxy-url-of-kube-ui) - [Connecting to the kube-ui service from your local workstation](#connecting-to-the-kube-ui-service-from-your-local-workstation) <!-- END MUNGE: GENERATED_TOC --> </ins> You have seen the [basics](accessing-the-cluster.md) about `kubectl proxy` and `apiserver proxy`. This guide shows how to use them together to access a service([kube-ui](ui.md)) running on the Kubernetes cluster from your workstation.
Code change:
has TODOs for how to bring up cluster addons like monitoring and logging. We should fill these in. requested by user:

Related discussion:
text: # Logging <ins> **Table of Contents** <!-- BEGIN MUNGE: GENERATED_TOC --> - [Logging](#logging) - [Logging by Kubernetes Components](#logging-by-kubernetes-components) - [Examining the logs of running containers](#examining-the-logs-of-running-containers) - [Cluster level logging to Google Cloud Logging](#cluster-level-logging-to-google-cloud-logging) - [Cluster level logging with Elasticsearch and Kibana](#cluster-level-logging-with-elasticsearch-and-kibana) - [Ingesting Application Log Files](#ingesting-application-log-files) - [Known issues](#known-issues) <!-- END MUNGE: GENERATED_TOC --> </ins> ## Logging by Kubernetes Components Kubernetes components, such as kubelet and apiserver, use the [glog](https://godoc.org/github.com/golang/glog) logging library.  Developer conventions for logging severity are described in [docs/devel/logging.md](../devel/logging.md).
Code change:
Currently watches only log when they complete, which can make analyzing kube-apiserver log files a bit tricky. (did component x never start watching after that list or did apiserver shut down?)

Related discussion:
text: } if (opts.Watch || forceWatch) && rw != nil { <ins> glog.Infof("Started to log from %v for %v", ctx, req.Request.URL.RequestURI()) </ins> watcher, err := rw.Watch(ctx, &opts) if err != nil { scope.err(err, res.ResponseWriter, req.Request)
Code change:
We have found that on 1k nodes cluster scheduling takes 60-200 ms and becomes the bottleneck. We want to have a standard way to reproduce scheduling latency metrics result and benchmark scheduler as simple and fast as possible. Only with that can we: 1. figure out underlying behaviors and potential bottlenecks quickly; 2. publish results in a standard way and prevent future performance degradation. We have the following goals: Goal 1: Save time on testing. More accurate scheduling latency. We mock up kubelets and only measure scheduling latency and do not care about the latency to start the pod. This is different from Kubemark -- scheduled pods won’t be run by mocked kubelets. We only need to set up the master components. Goal 2: Profiling runtime metrics and find out performance bottleneck. Write scheduler integration test. Take advantage of go test and profiling tools, like cpu-profiling, memory-profiling and block-profiling. This is very fine-grained metrics. Goal 3: Reproduce test result easily. We want to have a known place to do the performance related test for scheduler. Developers should just run one script to collect all the information they need. We have the following plans for writing scheduler tests: density test (by adding a new Go test) schedule 30k pods on 1000 (fake) nodes and 3k pods on 100 (fake) nodes print out scheduling rate every second let you learn the rate changes vs number of scheduled pods benchmark (by modifying the existing benchmark in integration or simply add a new one) make use of and report nanosecond/op. schedule b.N pods when the cluster has N nodes and P scheduled pods. Since it takes relatively long time to finish one round, b.N is small -- level of 10. We are new to k8s codebase. We also want to know where is the best place (directory) to commit in these tests? Any suggestions are welcomed!
/cc
This extends
So density already does this... the one option it looks like you may want is custom control , real or fake is irrelevant as the e2e's just need to point to an api-server. re: benchmarks, seems like we could simply add to the unit-tests.
After chatting with lets add this as an agenda item for the SIG meeting tomorrow.
+1 for adding benchmark (howeveer this should be integration test, not unit-test in my opinion)
cc The next SIG on scheduler meeting is on 12.07, isn't it? If there is one tomorrow, can I have the information please?
retract, +1 to . I was adding this to the list given the context, but it crosses over into both.
Slides from today's meeting:
Here are are the action item notes from the SIG meeting: Everyone agrees this is a good idea and long overdue, but the question is where would this live: New category of tests? (isolated/sandbox’d/component) The current testing structure allows to easily append. create new directory (component) + hack/script to execute Future "we could" gate on known good data, and fail performance regressions
/cc
/cc

Related discussion:
text: "fmt" "net/http" "net/http/httptest" <del> "sync" </del> "testing" "time"
Code change:
I'm running application with several env variables. One of them has a key ending with ==. Another variable has a comma separated list of email addresses. While perfectly valid on *nix and windows, we can't pass these values into Kubernetes run command. It fails with "error: invalid env:" Example command: The parsing in should be more robust
I volunteer myself to fix this.
Maybe they can be passed from file? I mean s'rsly.. you need to lits 20 variables by hand?? I've specified them in yaml file already, but kubectl-run have no way to specify it.. without this, deployment is totally impossible when you need to run migrations first!

Related discussion:
text: func parseEnvs(envArray []string) ([]api.EnvVar, error) { envs := []api.EnvVar{} for _, env := range envArray { <del> parts := strings.Split(env, "=") if len(parts) != 2 || !validation.IsCIdentifier(parts[0]) || len(parts[1]) == 0 { </del> <ins> pos := strings.Index(env, "=") if pos == -1 { </ins> return nil, fmt.Errorf("invalid env: %v", env) } <del> envVar := api.EnvVar{Name: parts[0], Value: parts[1]} </del> <ins> name := env[:pos] value := env[pos+1:] if len(name) == 0 || !validation.IsCIdentifier(name) || len(value) == 0 { return nil, fmt.Errorf("invalid env: %v", env) } envVar := api.EnvVar{Name: name, Value: value} </ins> envs = append(envs, envVar) } return envs, nil
Code change:
I'm running application with several env variables. One of them has a key ending with ==. Another variable has a comma separated list of email addresses. While perfectly valid on *nix and windows, we can't pass these values into Kubernetes run command. It fails with "error: invalid env:" Example command: The parsing in should be more robust
I volunteer myself to fix this.
Maybe they can be passed from file? I mean s'rsly.. you need to lits 20 variables by hand?? I've specified them in yaml file already, but kubectl-run have no way to specify it.. without this, deployment is totally impossible when you need to run migrations first!

Related discussion:
text: } } } <ins> func TestParseEnv(t *testing.T) { tests := []struct { envArray  []string expected  []api.EnvVar expectErr bool test      string }{ { envArray: []string{ "THIS_ENV=isOK", "HAS_COMMAS=foo,bar", "HAS_EQUALS=jJnro54iUu75xNy==", }, expected: []api.EnvVar{ { Name:  "THIS_ENV", Value: "isOK", }, { Name:  "HAS_COMMAS", Value: "foo,bar", }, { Name:  "HAS_EQUALS", Value: "jJnro54iUu75xNy==", }, }, expectErr: false, test:      "test case 1", }, { envArray: []string{ "WITH_OUT_EQUALS", }, expected:  []api.EnvVar{}, expectErr: true, test:      "test case 2", }, { envArray: []string{ "WITH_OUT_VALUES=", }, expected:  []api.EnvVar{}, expectErr: true, test:      "test case 3", }, { envArray: []string{ "=WITH_OUT_NAME", }, expected:  []api.EnvVar{}, expectErr: true, test:      "test case 4", }, } for _, test := range tests { envs, err := parseEnvs(test.envArray) if !test.expectErr && err != nil { t.Errorf("unexpected error: %v (%s)", err, test.test) } if test.expectErr && err != nil { continue } if !reflect.DeepEqual(envs, test.expected) { t.Errorf("nexpected:n%#vnsaw:n%#v (%s)", test.expected, envs, test.test) } } } </ins>
Code change:
Have not reproduced locally even after a few tens of runs.
which version of Go are you using locally? This probably only repros on 1.5.1.
This was from Jenkins runs.
This is caused because an attempts counter is being read/written to without synchronization. I think this is caused by the event creation order not being guaranteed. I am looking a little deeper to track this down.
Got a fix with 130k+ runs and no failures. Will send out PR shortly.

Related discussion:
text: eventCorrelator := NewEventCorrelator(util.RealClock{}) return eventBroadcaster.StartEventWatcher( func(event *api.Event) { <del> // Make a copy before modification, because there could be multiple listeners. // Events are safe to copy like this. eventCopy := *event event = &eventCopy result, err := eventCorrelator.EventCorrelate(event) if err != nil { util.HandleError(err) } if result.Skip { return } tries := 0 for { if recordEvent(sink, result.Event, result.Patch, result.Event.Count > 1, eventCorrelator) { break } tries++ if tries >= maxTriesPerEvent { glog.Errorf("Unable to write event '%#v' (retry limit exceeded!)", event) break } // Randomize the first sleep so that various clients won't all be // synced up if the master goes down. if tries == 1 { time.Sleep(time.Duration(float64(sleepDuration) * randGen.Float64())) } else { time.Sleep(sleepDuration) } } </del> <ins> recordToSink(sink, event, eventCorrelator, randGen) </ins> }) } <ins> func recordToSink(sink EventSink, event *api.Event, eventCorrelator *EventCorrelator, randGen *rand.Rand) { // Make a copy before modification, because there could be multiple listeners. // Events are safe to copy like this. eventCopy := *event event = &eventCopy result, err := eventCorrelator.EventCorrelate(event) if err != nil { util.HandleError(err) } if result.Skip { return } tries := 0 for { if recordEvent(sink, result.Event, result.Patch, result.Event.Count > 1, eventCorrelator) { break } tries++ if tries >= maxTriesPerEvent { glog.Errorf("Unable to write event '%#v' (retry limit exceeded!)", event) break } // Randomize the first sleep so that various clients won't all be // synced up if the master goes down. if tries == 1 { time.Sleep(time.Duration(float64(sleepDuration) * randGen.Float64())) } else { time.Sleep(sleepDuration) } } } </ins> func isKeyNotFoundError(err error) bool { statusErr, _ := err.(*errors.StatusError) // At the moment the server is returning 500 instead of a more specific
Code change:
Have not reproduced locally even after a few tens of runs.
which version of Go are you using locally? This probably only repros on 1.5.1.
This was from Jenkins runs.
This is caused because an attempts counter is being read/written to without synchronization. I think this is caused by the event creation order not being guaranteed. I am looking a little deeper to track this down.
Got a fix with 130k+ runs and no failures. Will send out PR shortly.

Related discussion:
text: import ( "encoding/json" "fmt" <del> "runtime" </del> <ins> "math/rand" </ins> "strconv" "testing" "time"
Code change:
Have not reproduced locally even after a few tens of runs.
which version of Go are you using locally? This probably only repros on 1.5.1.
This was from Jenkins runs.
This is caused because an attempts counter is being read/written to without synchronization. I think this is caused by the event creation order not being guaranteed. I am looking a little deeper to track this down.
Got a fix with 130k+ runs and no failures. Will send out PR shortly.

Related discussion:
text: } func TestWriteEventError(t *testing.T) { <del> ref := &api.ObjectReference{ Kind:       "Pod", Name:       "foo", Namespace:  "baz", UID:        "bar", APIVersion: "version", } </del> type entry struct { timesToSendError int <del> attemptsMade     int </del> attemptsWanted   int err              error }
Code change:
Have not reproduced locally even after a few tens of runs.
which version of Go are you using locally? This probably only repros on 1.5.1.
This was from Jenkins runs.
This is caused because an attempts counter is being read/written to without synchronization. I think this is caused by the event creation order not being guaranteed. I am looking a little deeper to track this down.
Got a fix with 130k+ runs and no failures. Will send out PR shortly.

Related discussion:
text: err:              fmt.Errorf("A weird error"), }, } <del> done := make(chan struct{}) eventBroadcaster := NewBroadcaster() defer eventBroadcaster.StartRecordingToSink( &testEventSink{ </del> <ins> eventCorrelator := NewEventCorrelator(util.RealClock{}) randGen := rand.New(rand.NewSource(time.Now().UnixNano())) for caseName, ent := range table { attempts := 0 sink := &testEventSink{ </ins> OnCreate: func(event *api.Event) (*api.Event, error) { <del> if event.Message == "finished" { close(done) return event, nil } item, ok := table[event.Message] if !ok { t.Errorf("Unexpected event: %#v", event) return event, nil } item.attemptsMade++ if item.attemptsMade < item.timesToSendError { return nil, item.err </del> <ins> attempts++ if attempts < ent.timesToSendError { return nil, ent.err </ins> } return event, nil }, <del> }, ).Stop() clock := &util.FakeClock{time.Now()} recorder := recorderWithFakeClock(api.EventSource{Component: "eventTest"}, eventBroadcaster, clock) for caseName := range table { clock.Step(1 * time.Second) recorder.Event(ref, api.EventTypeNormal, "Reason", caseName) runtime.Gosched() } recorder.Event(ref, api.EventTypeNormal, "Reason", "finished") <-done for caseName, item := range table { if e, a := item.attemptsWanted, item.attemptsMade; e != a { t.Errorf("case %v: wanted %v, got %v attempts", caseName, e, a) </del> <ins> } ev := &api.Event{} recordToSink(sink, ev, eventCorrelator, randGen) if attempts != ent.attemptsWanted { t.Errorf("case %v: wanted %d, got %d attempts", caseName, ent.attemptsWanted, attempts) </ins> } } }