jiebi/CodeConvo huggingface dataset:
Code change:
glog.Infof("Running tests for APIVersion: %s", apiVersion) <del> manifestURL := ServeCachedManifestFile() apiServerURL, configFilePath := startComponents(manifestURL, apiVersion) </del> <ins> firstManifestURL := ServeCachedManifestFile(testPodSpecFile) secondManifestURL := ServeCachedManifestFile(testManifestFile) apiServerURL, configFilePath := startComponents(firstManifestURL, secondManifestURL, apiVersion) </ins> // Ok. we're good to go. glog.Infof("API Server started on %s", apiServerURL)

Related discussion:
text: Currently it assumes it can read ContainerManifest - it should try both
Why not hold on this until we have Kubelet-read-Pod in?
Also the HTTP source has the same problem.
It should read Pod. Changed the title.
Ideally, I'd like all Pods created directly via Kubelet to be to etcd and surfaced via the API. If updates are made via the API, I'd like those to propagate back to the node. We can discuss whether the config file should be updated or ignored. At some point, I'd like Kubelet to persistently cache pods received from the apiserver, perhaps by just writing them all out as config files.
is the issue to sync local pods back to apiserver.
this is somewhat related to other issues we talked about.
Detail: We want locally specified pods to be able to use YAML config files, but we also want to rip YAML out of the API server . The config-file behavior should be as similar to the kubectl experience. Kubelet could use parts of the kubectl library to translate YAML to JSON. Vision: Eventually, I'd like Kubelet to use the apiserver library and support exactly the same Pod API as the master, at which time I'd like kubectl be able to create pods directly on Kubelets, as well as reading from config files. I also expect to have a per-node controller . I'd like to be able to use the per-node controller to update pods started via config files (or direct kubectl-kubelet pods), overriding those original files. For availability and self-hosting, we'll eventually need local checkpointing , ideally just using serialized JSON for each object. If Kubelet used apiserver, this could be done using a custom registry implementation.
I think that is a prerequisite for it.
If noone picks it up, I'm going to start working on it tomorrow.
thanks, wojtek-t
fixed it for http channel
Code change:
// buildELBSecurityGroupList returns list of SecurityGroups which should be // attached to ELB created by a service. List always consist of at least <del> // 1 member which is an SG created for this service. Extra groups can be </del> <ins> // 1 member which is an SG created for this service or a SG from the Global config. Extra groups can be </ins> // specified via annotation func (c *Cloud) buildELBSecurityGroupList(serviceName types.NamespacedName, loadBalancerName, annotation string) ([]string, error) { var err error

Related discussion:
text: /kind bug What happened: When setting an elbSecurityGroup in for AWS, and deploying multiple services with differing ports, then the security group rules flap in AWS, causing services to become inaccessible (only one will be correctly configured at a time). What you expected to happen: I expect that the configured elbSecurityGroup's rules will not be touched at all. It must be pre-configured to permit access. Alternatively, all ports for all services are queried and managed together. How to reproduce it (as minimally and precisely as possible): See above. Anything else we need to know?: Most people are not running large clusters which may hit the AWS limits, so they won't use this option, nor see this issue. Environment: Kubernetes version (use ): 1.7.1 Cloud provider or hardware configuration**: AWS OS (e.g. from /etc/os-release): CentOS 7 Kernel (e.g. ): 4.10.13-1-ARCH Install tools: Custom Others:
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
/sig aws
This is fixed by
Code change:
if !apierrors.IsBadRequest(err) { t.Errorf("expected HTTP status: BadRequest, got: %#v", apierrors.ReasonForError(err)) } <del> if err.Error() != expectedError { </del> <ins> if !strings.Contains(err.Error(), expectedError) { </ins> t.Errorf("expected %#v, got %#v", expectedError, err.Error()) } }

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Below k8s UT fail when run with master-golang. TestCheckpointStateRestore/RestorecheckpointwithinvalidJSON error:=== RUN TestCheckpointStateRestore/RestorecheckpointwithinvalidJSON I1124 05:38:14. ] [cpumanager] initializing new in-memory state store unexpected error while creatng checkpointState: could not restore state from checkpoint: json: unexpected end of JSON input, please drain this node and delete the CPU manager checkpoint file "/tmp/cpumanagerstatetest479660663/cpumanagercheckpointtest" before restarting Kubelet --- FAIL: TestCheckpointStateRestore/RestorecheckpointwithinvalidJSON (0.00s) TestStrategicMergePatchInvalid error:=== RUN TestStrategicMergePatchInvalid expected "invalid character 'b' looking for beginning of value", got "json: invalid character 'b' looking for beginning of value" --- FAIL: TestStrategicMergePatchInvalid (0.00s) TestJSONPatch error:=== RUN TestJSONPatch invalid-syntax: expected error invalid character 'i' looking for beginning of value, but got json: invalid character 'i' looking for beginning of value --- FAIL: TestJSONPatch (0.00s) TestV4ErrorDecoder expandless error:=== RUN TestV4ErrorDecoder wrong error for message "{": want="error stream protocol error: unexpected end of JSON input in "{"", got="error stream protocol error: json: unexpected end of JSON input in "{"" --- FAIL: TestV4ErrorDecoder (0.00s What you expected to happen: The tests should pass with no error. How to reproduce it (as minimally and precisely as possible): a stable version of golang installed master go from upstream git clone cd go/src cd ../.. cp -r ./go /usr/local export PATH=/usr/local/go/bin:$PATH kubernetes repo and run the unit tests. () Anything else we need to know?: These failures are occurring due to mismatch of error messages between upstream go and the k8s tests.
Code change:
if !apierrors.IsBadRequest(err) { t.Errorf("expected HTTP status: BadRequest, got: %#v", apierrors.ReasonForError(err)) } <del> if err.Error() != expectedError { </del> <ins> if !strings.Contains(err.Error(), expectedError) { </ins> t.Errorf("expected %#v, got %#v", expectedError, err.Error()) } }

Related discussion:
text: There is an addition "json:" prefix to SyntaxError messages after this below commit: Environment: Kubernetes version (use ): latest source code from upstream Cloud provider or hardware configuration: OS (e.g: ): "CentOS Linux 8 (Core)" Kernel (e.g. ): Linux rajalakshmi-workspace 4.18.0-193.19.1.el82.ppc64le SMP Mon Sep 14 14:48:04 UTC 2020 ppc64le ppc64le ppc64le GNU/Linux Install tools: [root go]# go version go version devel + Fri Nov 20 00:54:43 2020 +0000 linux/ppc64le [root go]# Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig testing
will fix it in Go 1.16 as release-blocker.
I would fix the test here to be more tolerant. exact string matches on error messages aren't really an API… I would not expect that to block the golang release
Agree too, we can think about having a substring match to avoid these issues instead!
/assign Lets see if we can fix this in the k8s tests
Code change:
func mountToRootfs(m *configs.Mount, c *mountConfig) error { rootfs := c.root <del> mountLabel := c.label mountFd := c.fd dest, err := securejoin.SecureJoin(rootfs, m.Destination) if err != nil { return err } </del> <ins> // procfs and sysfs are special because we need to ensure they are actually // mounted on a specific path in a container without any funny business. </ins> switch m.Device { case "proc", "sysfs": // If the destination already exists and is not a directory, we bail <del> // out This is to avoid mounting through a symlink or similar -- which </del> <ins> // out. This is to avoid mounting through a symlink or similar -- which </ins> // has been a "fun" attack scenario in the past. // TODO: This won't be necessary once we switch to libpathrs and we can //       stop all of these symlink-exchange attacks. <ins> dest := filepath.Clean(m.Destination) if !strings.HasPrefix(dest, rootfs) { // Do not use securejoin as it resolves symlinks. dest = filepath.Join(rootfs, dest) } </ins> if fi, err := os.Lstat(dest); err != nil { if !os.IsNotExist(err) { return err } <del> } else if fi.Mode()&os.ModeDir == 0 { </del> <ins> } else if !fi.IsDir() { </ins> return fmt.Errorf("filesystem %q must be mounted on ordinary directory", m.Device) } if err := os.MkdirAll(dest, 0o755); err != nil { return err } <del> // Selinux kernels do not support labeling of /proc or /sys </del> <ins> // Selinux kernels do not support labeling of /proc or /sys. </ins> return mountPropagate(m, rootfs, "", nil) <ins> } mountLabel := c.label mountFd := c.fd dest, err := securejoin.SecureJoin(rootfs, m.Destination) if err != nil { return err } switch m.Device { </ins> case "mqueue": if err := os.MkdirAll(dest, 0o755); err != nil { return err

Related discussion:
text: opencontainers/runc has fixed three CVEs: Latest runc is pulled by k/k Check runc version in file No response n/a n/a <details</details<details</details<details</details<details</details
/assign /sig security architecture release
/triage accepted
Code change:
} } <ins> type buffer struct { b  bytes.Buffer rw sync.RWMutex } func (b *buffer) String() string { b.rw.RLock() defer b.rw.RUnlock() return b.b.String() } func (b *buffer) Write(p []byte) (n int, err error) { b.rw.Lock() defer b.rw.Unlock() return b.b.Write(p) } </ins> func TestSecretUpdated(t *testing.T) { datacenter := "0.0.0.0" secretName := "vccreds"

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: What you expected to happen: No data races How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
for the record as we tend to lose the logs in the gcs bucket
/sig instrumentation
Code change:
handler = genericfilters.WithCORS(handler, c.CorsAllowedOriginList, nil, nil, nil, "true") <ins> // WithWarningRecorder must be wrapped by the timeout handler // to make the addition of warning headers threadsafe handler = genericapifilters.WithWarningRecorder(handler) </ins> // WithTimeoutForNonLongRunningRequests will call the rest of the request handling in a go-routine with the // context with deadline. The go-routine can keep running, while the timeout logic will return a timeout to the client. handler = genericfilters.WithTimeoutForNonLongRunningRequests(handler, c.LongRunningFunc)

Related discussion:
text: I am doing v1.28.3 kube-apiserver testing in the test environment, and I often encounter kube-apiserver panic, especially when kube-apiserver call webhook fails. The following are the captured panic logs: and: kube-apiserver does not panic even if calling webhook times out or fails Deploy a webhook, remove the service corresponding to the webhook, and then create pods in batches to trigger webhook calls No response <details</details<detailsprivate cloud </details<details</details<details</details<details</details<details</details
/sig api-machinery
somewhat related to
The scenarios that trigger panic are similar(the apiserver fails to call webhook), but the log shows that the location of panic is different.
/assign For context of  Thank you. /triage accepted
again, full stack logs:
I'll try to test the mr:
Let me try to explain the panic - both goroutines are trying to write to the of the first goroutine: This request has timed out, and consequently the filter is trying to send a response to the caller. This is the client facing goroutine from the timeout filter. Note that the call to is protected by a mutex, that means that if the other goroutine is using the delegated object via the adapter, then this panic should not happen. second goroutine: let's check the other goroutine, it is also trying to write to the of the object: Now uses an unprotected version of the ResponseWriter object: The filter precedes the filter in the execution order, so the in is not using the thread safe implementation. This is the order today: one quick resolution is to move the after the timeout filter: , but there could be code that precedes that needs to write warning? Also note that the panic mentioned in is not as severe, it does not cause the apiserver to crash. For a more proper solution, i think we would want any thoughts?
Code change:
handler = genericfilters.WithCORS(handler, c.CorsAllowedOriginList, nil, nil, nil, "true") <ins> // WithWarningRecorder must be wrapped by the timeout handler // to make the addition of warning headers threadsafe handler = genericapifilters.WithWarningRecorder(handler) </ins> // WithTimeoutForNonLongRunningRequests will call the rest of the request handling in a go-routine with the // context with deadline. The go-routine can keep running, while the timeout logic will return a timeout to the client. handler = genericfilters.WithTimeoutForNonLongRunningRequests(handler, c.LongRunningFunc)

Related discussion:
text: /cc
I tend to remove the timeout filter, the sooner the better, and I think the Apiserver panic is the P0-level bug, directly affecting the availability of the cluster I have cherry-picked and tested it in my cluster, and I still need to continue to observe for 1-2 days
I opened a minimal WIP fix for the warning writer in since the timeout filter removal is a much bigger change to reason through
thanks a lot, I will try to test and cherry-pick the above minimal WIP fix in
sorry, I completely overlooked , I only saw and that looked too big to take safely right now I prefer to
Can we backport the fix to 1.28?
Why 1.28 specifically? I'm not aware of anything that changed in 1.28 that made this more or less prevalent. If this meets backport criteria, going to all active branches could make sense (1.27+)
Code change:
testRootDir := makeTempDirOrDie("kubelet_integ_1.", "") configFilePath := makeTempDirOrDie("config", testRootDir) glog.Infof("Using %s as root dir for kubelet #1", testRootDir) <del> kcfg := kubeletapp.SimpleKubelet(cl, &fakeDocker1, machineList[0], testRootDir, manifestURL, "127.0.0.1", 10250, api.NamespaceDefault, empty_dir.ProbeVolumePlugins(), nil, cadvisorInterface, configFilePath) </del> <ins> kcfg := kubeletapp.SimpleKubelet(cl, &fakeDocker1, machineList[0], testRootDir, firstManifestURL, "127.0.0.1", 10250, api.NamespaceDefault, empty_dir.ProbeVolumePlugins(), nil, cadvisorInterface, configFilePath) </ins> kcfg.PodStatusUpdateFrequency = 1 * time.Second kubeletapp.RunKubelet(kcfg) // Kubelet (machine)

Related discussion:
text: Currently it assumes it can read ContainerManifest - it should try both
Why not hold on this until we have Kubelet-read-Pod in?
Also the HTTP source has the same problem.
It should read Pod. Changed the title.
Ideally, I'd like all Pods created directly via Kubelet to be to etcd and surfaced via the API. If updates are made via the API, I'd like those to propagate back to the node. We can discuss whether the config file should be updated or ignored. At some point, I'd like Kubelet to persistently cache pods received from the apiserver, perhaps by just writing them all out as config files.
is the issue to sync local pods back to apiserver.
this is somewhat related to other issues we talked about.
Detail: We want locally specified pods to be able to use YAML config files, but we also want to rip YAML out of the API server . The config-file behavior should be as similar to the kubectl experience. Kubelet could use parts of the kubectl library to translate YAML to JSON. Vision: Eventually, I'd like Kubelet to use the apiserver library and support exactly the same Pod API as the master, at which time I'd like kubectl be able to create pods directly on Kubelets, as well as reading from config files. I also expect to have a per-node controller . I'd like to be able to use the per-node controller to update pods started via config files (or direct kubectl-kubelet pods), overriding those original files. For availability and self-hosting, we'll eventually need local checkpointing , ideally just using serialized JSON for each object. If Kubelet used apiserver, this could be done using a custom registry implementation.
I think that is a prerequisite for it.
If noone picks it up, I'm going to start working on it tomorrow.
thanks, wojtek-t
fixed it for http channel
Code change:
"regexp" "strings" "syscall" <ins> "github.com/golang/glog" </ins> ) const MOUNT_MS_BIND = syscall.MS_BIND

Related discussion:
text: 07-02 and 13-02 No response No response /sig
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scalability
I might be related to the npd.
The same test failures happening again on 25.02:
/sig node
/sig node /cc The restart of caused the flake. I also opened to track it in NPD repo.
I missed this log. It is caused by oom. Adding some memory may help. I submitted to fix it. Does mean killing metrics-server?
/remove-sig node Sorry, I misread the log. The NPD found the metric server is OOM. It is about the metrics server, not the NPD. Sorry for disturb. I may add some memory for metric-server and check why metric server memory changed. I heard that metrics server recently released a new version. I will check if it is related.
/sig node
OOM metrics shows the metric server is OOMed. This is odd, that only metrics-server-nanny is with a memory limit. It uses whose command is . metrics-server container has no memory limit, but it got killed by OOM. Does this indicate that the node is OOM? (edited)
The last run failed in node-problem-detector ContainerStateTerminated. I0227 05:03:31. ] PodStartupLatency: perc50: 14m29.018886827s, perc90: 28m44.625518807s, perc99: 32m2.19303496s; threshold 1h0m0s I0227 05:03:32. ]
Code change:
"regexp" "strings" "syscall" <ins> "github.com/golang/glog" </ins> ) const MOUNT_MS_BIND = syscall.MS_BIND

Related discussion:
text: Loaded restart count threshold overrides: map[coredns:2 fluentd-gcp:999 konnectivity-agent:2 kube-controller-manager:1 kube-proxy:2 kube-scheduler:1 l7-lb-controller:1 metadata-proxy:2 prometheus-to-sd-exporter:2] I0227 05:03:32. ] ClusterOOMsTracker: gathering cluster OOMs tracking measurement E0227 05:03:32. ] TestMetrics: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0] I0227 05:03:32. ] Step "[step: 29] gathering measurements" ended W0227 05:03:32. ] Got errors during step execution: [measurement call TestMetrics - TestMetrics error: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0]] I0227 05:03:32. ] Waiting for the chaos monkey subroutine to end... I0227 05:03:32. ] Chaos monkey ended. I0227 05:03:32. ] I0227 05:03:32. ] Stopping DnsLookupLatency probe... I0227 05:03:47. ] Stopping InClusterNetworkLatency probe... I0227 05:04:22. ] Resources cleanup time: 50.539700215s E0227 05:04:22. ]
E0227 05:04:22. ] Test Finished E0227 05:04:22. ] Test: E0227 05:04:22. ] Status: Fail E0227 05:04:22. ]
Code change:
"regexp" "strings" "syscall" <ins> "github.com/golang/glog" </ins> ) const MOUNT_MS_BIND = syscall.MS_BIND

Related discussion:
text: Errors: [measurement call TestMetrics - TestMetrics error: [action gather failed for SystemPodMetrics measurement: restart counts violation: RestartCount(node-problem-detector-tgl8h, node-problem-detector)=1, want <= 0; RestartCount(node-problem-detector-vpsmz, node-problem-detector)=1, want <= 0]] E0227 05:04:22. ]
I0227 05:04:22. ]
I0227 05:04:22. ] Running testing/huge- I0227 05:04:22. ]
Code change:
{ cfg: &kubeadmapi.MasterConfiguration{}, expected: api.Volume{ <del> Name: "pki", </del> <ins> Name: "k8s", </ins> VolumeSource: api.VolumeSource{ HostPath: &api.HostPathVolumeSource{ Path: kubeadmapi.GlobalEnvParams.KubernetesDir},

Related discussion:
text: <!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--What keywords did you search in Kubernetes issues before filing this one? controller /etc/ssl/certs kubeadm /etc/ssl/certs Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): 5 custom built master I also use a custom built hyperkube image instead of the single app images for apiserver, controller, ... Environment: Cloud provider or hardware configuration: azure OS (e.g. from /etc/os-release): CentOS 7.2 Kernel (e.g. ): Linux ma-kub8ms0 3.10.0-327.36.3.el7.x8664 SMP Mon Oct 24 16:09:20 UTC 2016 x8664 x8664 x8664 GNU/Linux Install tools: kubeadm Others: deployed with Azure RM template and ansible What happened: controller is not able to do https, including calls to the Azure RM API What you expected to happen: https should work as usual How to reproduce it (as minimally and precisely as possible): Use kubeadm to install kubernetes on CentOS exec into the controller and try "curl " Anything else do we need to know: It looks like caused this issue. On CentOS, I see these files on the host OS: I suspect the symlinks to /etc/pki/... being the problem here, as they are not available inside the hyperkube Debian image. I also tried to verify this with It seems to not be based on Debian and thus gives a different error: I see 2 possible solutions: of host mounting /etc/ssl/certs into the controller pod, the ca certificates could be installed into the image.