jiebi/CodeConvo huggingface dataset:
Code change:
nodeStatusUpdateRetry = 5 // Location of container logs. <del> containerLogsDir = "/var/log/containers" </del> <ins> ContainerLogsDir = "/var/log/containers" </ins> // max backoff period, exported for the e2e test MaxContainerBackOff = 300 * time.Second

Related discussion:
text: Cluster logging assumes kubelet stores container log at a specific directory with specific name. Currently the path is: After the CRI work, the path will be: However, we don't have test coverage for this. We don't even know whether the log files are there or not. We should add a node e2e test for this, basically: 1) Create a container which prints logs 2) Check whether the expected log file is created. 3) Check whether the logs are collected in the file as expected. And it would be better to launch a pod to do the check instead of accessing local file system directly, which makes containerizing easier. /cc because you are collecting missing test. /cc
I will try to add a node-e2e test there
I think it's ok to add this for CRI. But there maybe problems on today's interface for rkt. If I'm not wrong, rkt does't store logs at same location with docker. cc/ Is this true?
Thanks! :)
After gets merged, we need to update the test to verify the cri log path.
Code change:
"//vendor/k8s.io/apiserver/pkg/server:go_default_library", "//vendor/k8s.io/apiserver/pkg/util/feature:go_default_library", "//vendor/k8s.io/apiserver/pkg/util/proxy:go_default_library", <del> "//vendor/k8s.io/client-go/informers/core/v1:go_default_library", </del> "//vendor/k8s.io/client-go/listers/core/v1:go_default_library", "//vendor/k8s.io/client-go/pkg/version:go_default_library", "//vendor/k8s.io/client-go/rest:go_default_library",

Related discussion:
text: <!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Currently APIServiceRegistrationController watch service change and find the related APIServices related if any. Then it AddAPIService/RemoveAPIService for APIAggregator. I dig into the kube-aggregator and find it no need totally, because update APIServices status when related service or endpoint change. So if does not watch services, it will not miss any event for APIServices. What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/cc
/kind enhancement /sig api-machinery
/assign
that does not follow. AvailableConditionController modifies the APIService object. APIServiceRegistrationController must still watch the APIService objects to observe those changes.
oh, I misread the description. You're saying APIServiceRegistrationController doesn't need to watch v1.Service objects
Code change:
"//pkg/util/metrics:go_default_library", "//staging/src/k8s.io/api/apps/v1:go_default_library", "//staging/src/k8s.io/api/core/v1:go_default_library", <ins> "//staging/src/k8s.io/apimachinery/pkg/api/equality:go_default_library", </ins> "//staging/src/k8s.io/apimachinery/pkg/api/errors:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/labels:go_default_library",

Related discussion:
text: Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: a cluster, And node1 with resource 1 DaemonSet, request resource 4 run , There is no pod created. resource to the node: run , There is still no pod created. What you expected to happen: In step 5, There is one pod created. How to reproduce it (as minimally and precisely as possible): see above description Anything else we need to know?: Environment: Kubernetes version (use ):1.10.5 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release):CentOS7 Kernel (e.g. ): Install tools: Others: /sig app
xref
Thanks! is not work here! Because: DaemonSetsController didn't watch the changes.(we use to updateNode.) we restart kubelet, The DaemonSetsController watch the changes, but it seems that the node info is not updated, and still failed to create pod:
I found it is a bug here, ignore the changes. I made a fix , PTAL! Thanks!
Any workarounds for old cluster versions? Up: looks like in my case helped but after some delay.
, which version of k8s are you using? If it's sunset, you have to merge the pr yourself :). I think restart kubelet trigger delete/add to make sure it sync .
Sorry, I didn't say
I think we still support 1.12 for now :) let's cherry pick it if not and wait for next 1.12 release.
Code change:
}, } <ins> func addOrUpdateUnschedulablePod(p *PriorityQueue, pod *v1.Pod) { p.lock.Lock() defer p.lock.Unlock() p.unschedulableQ.addOrUpdate(pod) } func getUnschedulablePod(p *PriorityQueue, pod *v1.Pod) *v1.Pod { p.lock.Lock() defer p.lock.Unlock() return p.unschedulableQ.get(pod) } </ins> func TestPriorityQueue_Add(t *testing.T) { q := NewPriorityQueue(nil) if err := q.Add(&medPriorityPod); err != nil {

Related discussion:
text: Which jobs are failing: pull-kubernetes-bazel-test Which test(s) are failing: bazel test //pkg/scheduler/internal/queue:godefaulttest Since when has it been failing: Testgrid link: Reason for failure: WARNING: DATA RACE Read at 0x00c000230db0 by goroutine 47: runtime.mapiterinit() +0x0 (PriorityQueue).flushUnschedulableQLeftover() +0x17a (PriorityQueue).flushUnschedulableQLeftover-fm() +0x41 () +0x61 () +0xce () +0x5a Previous write at 0x00c000230db0 by goroutine 45: runtime.mapassignfaststr() +0x0 (UnschedulablePodsMap).addOrUpdate() +0x30e () +0x601 testing.tRunner() +0x162 Goroutine 47 (running) created at: (PriorityQueue).run() +0x172 () +0xe0b () +0x50 () +0x5a5 testing.tRunner() +0x162 Goroutine 45 (running) created at: testing.(T).Run() +0x659 testing.runTests.func1() +0xa8 testing.tRunner() +0x162 testing.runTests() +0x4ee testing.(M).Run() +0x2ee () bazel-out/k8- +0x2fb Anything else we need to know: I'm very sorry about that. UnschedulablePodsMap is not thread-safe. In , lock will be acquired before operating UnschedulablePodsMap. But in , lock isn‘t be acquired. In the previous case, only a single thread runs so it's ok. But after add goroutine , there will be data race. So I need to acquire lock before operating UnschedulablePodsMap.
/sig testing /sig scheduling
Code change:
func NewManager(topology []cadvisorapi.Node, topologyPolicyName string, topologyScopeName string, topologyPolicyOptions map[string]string) (Manager, error) { klog.InfoS("Creating topology manager with policy per scope", "topologyPolicyName", topologyPolicyName, "topologyScopeName", topologyScopeName) <ins> // When policy is none, the scope is not relevant, so we can short circuit here. if topologyPolicyName == PolicyNone { return &manager{scope: NewNoneScope()}, nil } </ins> opts, err := NewPolicyOptions(topologyPolicyOptions) if err != nil { return nil, err

Related discussion:
text: Follow up from: See details there. When is , there is no need to discover the NUMA topology. /sig node /priority backlog /kind feature /help-wanted
/cc
/triage accepted
I would like to take this up also can I get more brief information about it? code review comments from asking that inside the NewManager() implementation, we need to skip NewPolicyOptions and NewNUMAInfo but when the NumaTopology comes kubelet issue also comes. I need some more information to start with this!
/cc
When there is no topology manger policy specified, NewManager should be a no-op. Just skip all checks.
Exactly. Just to add: we may need few iterations over the change to get a clean flow in the call.
/assign
Code change:
kubectl set resources deployment nginx --limits=cpu=0,memory=0 --requests=cpu=0,memory=0 # Print the result (in yaml format) of updating nginx container limits from a local, without hitting the server <del> kubectl set resources -f path/to/file.yaml --limits=cpu=200m,memory=512Mi --local -o yaml`) </del> <ins> kubectl set resources -f path/to/file.yaml --limits=cpu=200m,memory=512Mi --dry-run -o yaml`) </ins> ) // ResourcesOptions is the start of the data required to perform the operation. As new fields are added, add them here instead of

Related discussion:
text: from https://k8s-
Really suspect this is
This is putting us into test-go post-commit run is broken
cc:
marked this p0 - if a fix isn't easy let's try to identify what introduced this and revert.
This flake on: 2016-10-30 07:19 PDT First instance: : It is flaking frequently, blocking PRs from getting merged, and slowing the submit queue ( at 75%). As on-call, I'm going to try and identify the cluprit and revert. Can use all help in doing so.
can you investigate whether your additional tests could cause this (by breaking the deployment of those pods, for instance) , Saad Ali wrote:
PR looks most suspect to me. It mucked with , and the errors we are seeing in each of the failures is related to : That PR was merged at Oct 30, 2016, 6:18 AM PDT and the first flake in was at Oct 30, 2016, 7:19 AM PDT. Thoughts? CC
I'm going to initiate a revert of to see if that fixes the issue
IMO we can just change the flaky test to not get pods but some other resources, such as deployments. We can't really expect to clean up pods immediately.
I'm going to proceed with revert (). Please feel free to send a follow up PR to handle this more gracefully.
Going to leave this opened until we have a few hours of runs with the revert. Want to verify, with some degree of certainty, that the revert took care of this flake.
We had 15 green runs of in a row. Considering this flake fixed.
Code change:
Spec: v1.PodSpec{ Containers: []v1.Container{ { <del> Name:  name, Image: "gcr.io/google_containers/busybox:1.24", VolumeMounts: []v1.VolumeMount{ { Name:      "tmpfs", MountPath: "/tmp", }, }, Command: cmd, }, }, RestartPolicy: v1.RestartPolicyNever, Volumes: []v1.Volume{ { Name:         "tmpfs", VolumeSource: v1.VolumeSource{HostPath: &v1.HostPathVolumeSource{Path: "/tmp"}}, </del> <ins> Name:      name, Image:     image, Lifecycle: lifecycle, </ins> }, }, },

Related discussion:
text: The prestop and poststart exec tests volume mount /tmp from the host and attempt to create a file there from inside the pod. This is blocked by selinux. Failure:
we need to have a mechanism to say that a test should not be run if selinux is enabled (if legitimately required) or fix each test (where appropriate)
Code change:
genericfeatures.APIResponseCompression: {Default: true, PreRelease: featuregate.Beta}, <ins> genericfeatures.APIServerIdentity: {Default: true, PreRelease: featuregate.Beta}, genericfeatures.APIServerTracing: {Default: true, PreRelease: featuregate.Beta}, genericfeatures.ConsistentListFromCache: {Default: false, PreRelease: featuregate.Alpha}, genericfeatures.CustomResourceValidationExpressions: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.31 genericfeatures.EfficientWatchResumption: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, </ins> genericfeatures.KMSv1: {Default: false, PreRelease: featuregate.Deprecated}, genericfeatures.KMSv2: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.31 genericfeatures.KMSv2KDF: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.31 <del> genericfeatures.ValidatingAdmissionPolicy: {Default: false, PreRelease: featuregate.Beta}, genericfeatures.CustomResourceValidationExpressions: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.31 </del> genericfeatures.OpenAPIEnums: {Default: true, PreRelease: featuregate.Beta}, <ins> genericfeatures.RemainingItemCount: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, </ins> genericfeatures.ServerSideApply: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.29 genericfeatures.ServerSideFieldValidation: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.29 <ins> genericfeatures.StorageVersionAPI: {Default: false, PreRelease: featuregate.Alpha}, genericfeatures.StorageVersionHash: {Default: true, PreRelease: featuregate.Beta}, </ins> genericfeatures.StructuredAuthenticationConfiguration: {Default: false, PreRelease: featuregate.Alpha}, genericfeatures.StructuredAuthorizationConfiguration: {Default: false, PreRelease: featuregate.Alpha}, genericfeatures.UnauthenticatedHTTP2DOSMitigation: {Default: true, PreRelease: featuregate.Beta}, <ins> genericfeatures.ValidatingAdmissionPolicy: {Default: false, PreRelease: featuregate.Beta}, genericfeatures.WatchBookmark: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, genericfeatures.WatchList: {Default: false, PreRelease: featuregate.Alpha}, </ins> genericfeatures.ZeroLimitedNominalConcurrencyShares: {Default: false, PreRelease: featuregate.Beta}, // inherited features from apiextensions-apiserver, relisted here to get a conflict if it is changed

Related discussion:
text: This avoids drift between Kubernetes components and allows for cluster management tools that pass uniform feature gates to all components. xref:
/assign
Hey I f you stuck anywhere. Please free to involve me in this.
Code change:
return NewDelayingQueueWithCustomClock(clock.RealClock{}, "") } <ins> // NewDelayingQueueWithCustomQueue constructs a new workqueue with ability to // inject custom queue Interface instead of the default one func NewDelayingQueueWithCustomQueue(q Interface, name string) DelayingInterface { return newDelayingQueue(clock.RealClock{}, q, name) } </ins> // NewNamedDelayingQueue constructs a new named workqueue with delayed queuing ability func NewNamedDelayingQueue(name string) DelayingInterface { return NewDelayingQueueWithCustomClock(clock.RealClock{}, name)

Related discussion:
text: <!-- Please only use this template for submitting enhancement requests --What would you like to be : Current constructors: Could we export a new constructor like to inject custom queue. Why is this needed: In my project, i want to create a with a custom queue based on workqueue.Interface but have no way to do that.
/sig api-machinery
thanks for the issue! We've reviewed it in the bug triage, and the consensus was that to ask you to send a PR with this and discuss it there. Would you be able to do it? thanks!
Yes, i will do that.
Code change:
echo "COPY nsswitch.conf /etc/" >> "${docker_file_path}" fi <del> # provide `--pull` argument to `docker build` if `KUBE_BUILD_PULL_LATEST_IMAGES` # is set to y or Y; otherwise try to build the image without forcefully # pulling the latest base image. local -a docker_build_opts=() if [[ "${KUBE_BUILD_PULL_LATEST_IMAGES}" =~ [yY] ]]; then docker_build_opts+=("--pull") fi "${DOCKER[@]}" build "${docker_build_opts[@]}" -q -t "${docker_image_tag}" "${docker_build_path}" >/dev/null </del> <ins> "${DOCKER[@]}" build ${docker_build_opts} -q -t "${docker_image_tag}" "${docker_build_path}" >/dev/null </ins> # If we are building an official/alpha/beta release we want to keep # docker images and tag them appropriately. local -r release_docker_image_tag="${KUBE_DOCKER_REGISTRY-$docker_registry}/${binary_name}-${arch}:${KUBE_DOCKER_IMAGE_TAG-$docker_tag}"

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: when I run , found bound variable err. What you expected to happen: Build success How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ):Linux paasn1 4.14.78-300.el7.Centos.x8664 SMP Sat Jan 5 15:49:43 CST 2019 x8664 x8664 x8664 GNU/Linux Install tools: Network plugin and version (if this is a network-related bug): Others: GNU bash version 4.2.46(2)-release (x8664-redhat-linux-gnu) Copyright (C) 2011 Free Software Foundation, Inc.
/sig release
Code change:
t.Errorf("unexpected object: %v", rc2) t.FailNow() } <del> if test.expectPullPolicy != rc2.Spec.Template.Spec.InitContainers[0].ImagePullPolicy { t.Errorf("expected ImagePullPolicy: %s, got: %s", test.expectPullPolicy, rc2.Spec.Template.Spec.InitContainers[0].ImagePullPolicy, ) </del> <ins> if err := assertInitContainers(rc2.Spec.Template.Spec.InitContainers, test.expected, test.validators); err != nil { t.Errorf("test %v failed: %v", test.name, err) </ins> } } }

Related discussion:
text: What happened: kubelet doesn't seem to be able to manage containers started with CRI-O any more I'm testing kubelet + CRI-O on an Ubuntu 18.04 server installation, inside a qemu/kvm VM managed by Vagrant (see ), in order to be able to test k8s + KubeVirt + KataContainers, which are all compatible with CRI-O kubelet will obsess on starting new containers, which looks like and endless process, and the cluster never gets ready. For instance here are some syslogs : What you expected to happen: It used to work (see asciinema recording at ) a few days ago, but I fear some recent changes broke it. How to reproduce it (as minimally and precisely as possible): Testing with vagrant up --provider=libvirt on Linux with a clone of that commit: vagrant ssh into the VM and ps -edf f or tail -f /var/log/syslog would do... Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: qemu/kvm VM +OS (e.g: ): Ubuntu 18.04.2 LTS Kernel (e.g. ): Linux kubernetes-vagrant-01 4.15.0-48-generic -Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x8664 x8664 x8664 GNU/Linux Install tools: kubeadm as in Network plugin and version (if this is a network-related bug): Others:
/sig node
I can confirm this. If I start cri-o from a systemd service it fails with the above mentioned error message. Starting crio with without systemd works just fine. systemd file:
Thanks for your detailed report :) Just want to do some quick version confirming - you've been using k8s 1.14.1 both when it did and did not work? Any other version changes?
Not sure for before, when it used to work, but according to the asciinema recording mentioned above (), it seems the same. However, not sure for CRI-O, underlying OS stuff (systemd, kernel, etc.). Sorry.
I was looking for some information for days. I have exactly the same issue when I use podman + systemd on 18.04.2.
Code change:
t.Errorf("unexpected object: %v", rc2) t.FailNow() } <del> if test.expectPullPolicy != rc2.Spec.Template.Spec.InitContainers[0].ImagePullPolicy { t.Errorf("expected ImagePullPolicy: %s, got: %s", test.expectPullPolicy, rc2.Spec.Template.Spec.InitContainers[0].ImagePullPolicy, ) </del> <ins> if err := assertInitContainers(rc2.Spec.Template.Spec.InitContainers, test.expected, test.validators); err != nil { t.Errorf("test %v failed: %v", test.name, err) </ins> } } }

Related discussion:
text: It is the same for both 14.15 and 14.18 kernels. Podman without systemd runs fine. Podman with systemd fails at runc with the same error podman[]: open /dev/stderr: no such device or address podman[]: NAME: runc - Open Container Initiative runtime podman[]: runc is a command line client for running applications packaged according to ... podman[]: time="2019-05-11T13:04:59-07:00" level=error msg="open /dev/stderr: no such device or addressn" podman[]: open /dev/stderr: no such device or address
I've tried to reproduce it on 18.04.2 using Podman from but I am not able to reproduce it. Do you have any reproducer? How did you install Podman?
I used ceph unit files to run ceph in podman rather than docker. Systemd unit successfully starts docker but fails with podman. I can successfully run the same podman container run command from the command line. How were you trying to replicate this? Can you post your unit file?
Hi, I'am having the same problem with Kubernetes with CRI-O. After joining a second worker node it fails to start flannel. gives the following error: What I've tried: Installing previous version of Kubernetes (from to ) Installing previous version of crio (from to ) Building runc myself from their github page and selecting this in the Running it on Ubunut 18.04.2 Issue: (the one giuseppe linked)
I wonder if the issue is with systemd and runc. In both cases, crio and podman, I see runc help output which tells me that something is not being passed to runc that it needs.
I've only used so it is different than your test and you confirmed it works from the CLI for you as well. Could you share your service file so I test it?
I've tried systemd and cgroupfs but no luck so far.
so it seems that runc fails to re-open . That seems like a runc issue, as it doesn't really need to reopen it. To fix the failure, you could use something like:
Just an example.
Code change:
t.Errorf("unexpected object: %v", rc2) t.FailNow() } <del> if test.expectPullPolicy != rc2.Spec.Template.Spec.InitContainers[0].ImagePullPolicy { t.Errorf("expected ImagePullPolicy: %s, got: %s", test.expectPullPolicy, rc2.Spec.Template.Spec.InitContainers[0].ImagePullPolicy, ) </del> <ins> if err := assertInitContainers(rc2.Spec.Template.Spec.InitContainers, test.expected, test.validators); err != nil { t.Errorf("test %v failed: %v", test.name, err) </ins> } } }

Related discussion:
text: I use ceph-ansible to install cluster so it may need to install additional daemons like mon before running mgr but the error, in this case, should be different, [Unit] Description=Ceph Manager [Service] EnvironmentFile=-/etc/environment ExecStartPre=-/usr/bin/podman stop ceph-mgr-storage2n1-la ExecStartPre=-/usr/bin/podman rm ceph-mgr-storage2n1-la ExecStart=/usr/bin/podman run --rm --net=host  --memory=16G  --cpus=4  -v /var/lib/ceph:/var/lib/ceph:z  -v /etc/ceph:/etc/ceph:z  -v /var/run/ceph:/var/run/ceph:z  -v /etc/localtime:/etc/localtime:ro  -e CLUSTER=ceph  -e CEPHDAEMON=MGR  -e CONTAINERIMAGE=  {{ cephmgrdockerextraenv }}  --name=ceph-mgr-storage2n1-la  ExecStopPost=-/usr/bin/podman stop ceph-mgr-storage2n1-la Restart=always RestartSec=10s TimeoutStartSec=120 TimeoutStopSec=15 [Install] WantedBy=multi- delighted <:
BTW, I projectatomic ppa and used apt-get -y podman to install. My podman version is 1.3.0 Sasha Litvak — You are receiving this because you commented. Reply to this email directly, view it on GitHub , or mute the thread
Corrected file.