jiebi/CodeConvo huggingface dataset:
Code change:
What happened: removed reporting of the OutOfDisk condition in the kubelet. A 1.12 kubelet upgraded to a 1.13 kubelet leaves the previously reported OutOfDisk condition in place, and it eventually gets switched to Unknown status. What you expected to happen: If this condition is no longer reported, it should be removed by the kubelet. How to reproduce it (as minimally and precisely as possible): a 1.12 kubelet, let it report OutOfDisk condition status. to 1.13 the condition is no longer updated a while, the node controller flips the status to Unknown Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: <!-- DO NOT EDIT BELOW THIS LINE --/kind bug /sig node
The fix for this should be picked to 1.13
/assign happened to have a fix handy, if no one is working on it.
after upgrade kubelet should ignore this condition . or change it to 'DiskPressure'
Better not. What if both condition and exist?
should be picked to 1.13 also, I think.
13.2 doesn't have this patch; would it be possible to make it for 1.13.3?
Unfortunately this was not mentioned in the release notes (or I missed it), got hit by this too. 13.2 is affected too, would be good for me to see this getting fixed in 1.13.3

Related discussion:
text: return nil } } <ins> // RemoveOutOfDiskCondition removes stale OutOfDisk condition // OutOfDisk condition has been removed from kubelet in 1.12 func RemoveOutOfDiskCondition() Setter { return func(node *v1.Node) error { var conditions []v1.NodeCondition for i := range node.Status.Conditions { if node.Status.Conditions[i].Type != v1.NodeOutOfDisk { conditions = append(conditions, node.Status.Conditions[i]) } } node.Status.Conditions = conditions return nil } } </ins>
Code change:
What happened: removed reporting of the OutOfDisk condition in the kubelet. A 1.12 kubelet upgraded to a 1.13 kubelet leaves the previously reported OutOfDisk condition in place, and it eventually gets switched to Unknown status. What you expected to happen: If this condition is no longer reported, it should be removed by the kubelet. How to reproduce it (as minimally and precisely as possible): a 1.12 kubelet, let it report OutOfDisk condition status. to 1.13 the condition is no longer updated a while, the node controller flips the status to Unknown Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: <!-- DO NOT EDIT BELOW THIS LINE --/kind bug /sig node
The fix for this should be picked to 1.13
/assign happened to have a fix handy, if no one is working on it.
after upgrade kubelet should ignore this condition . or change it to 'DiskPressure'
Better not. What if both condition and exist?
should be picked to 1.13 also, I think.
13.2 doesn't have this patch; would it be possible to make it for 1.13.3?
Unfortunately this was not mentioned in the release notes (or I missed it), got hit by this too. 13.2 is affected too, would be good for me to see this getting fixed in 1.13.3

Related discussion:
text: } } <ins> func TestRemoveOutOfDiskCondition(t *testing.T) { now := time.Now() var cases = []struct { desc       string inputNode  *v1.Node expectNode *v1.Node }{ { desc: "should remove stale OutOfDiskCondition from node status", inputNode: &v1.Node{ Status: v1.NodeStatus{ Conditions: []v1.NodeCondition{ *makeMemoryPressureCondition(false, now, now), { Type:   v1.NodeOutOfDisk, Status: v1.ConditionFalse, }, *makeDiskPressureCondition(false, now, now), }, }, }, expectNode: &v1.Node{ Status: v1.NodeStatus{ Conditions: []v1.NodeCondition{ *makeMemoryPressureCondition(false, now, now), *makeDiskPressureCondition(false, now, now), }, }, }, }, } for _, tc := range cases { t.Run(tc.desc, func(t *testing.T) { // construct setter setter := RemoveOutOfDiskCondition() // call setter on node if err := setter(tc.inputNode); err != nil { t.Fatalf("unexpected error: %v", err) } // check expected node assert.True(t, apiequality.Semantic.DeepEqual(tc.expectNode, tc.inputNode), "Diff: %s", diff.ObjectDiff(tc.expectNode, tc.inputNode)) }) } } </ins> // Test Helpers: // sortableNodeAddress is a type for sorting []v1.NodeAddress
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: When using the function from Python's Kuberneets API, we noticed that Pods are terminated bythe cluster without respecting their preStop hooks or terminationGracePeriodSeconds seconds. What you expected to happen: When using the eviction API, just as when using delete, Pods should be allowed to run their preStop hooks, as well as be given terminationGracePeriodSeconds before being terminated with SIGKILL. How to reproduce it (as minimally and precisely as possible): the following pod: the pod logs in another window to evict the pod. Here is a sample curl equivalent of what (1.13) is doing - happy path: Here is a sample curl equivalent of what Python's is doing (when not passed ) - broken path: The log output generated by the guinea pod on the happy path looks like this: The "Got signal 15" message shows up only after the preStop hook is finished, so ~10 seconds after the evict call is made What happens on the broken path is: We seem to be getting 2x SIGTERM, there are no messages indicating continued processing after the signal, and the signal itself is registered immediately after the eviction call is made, suggesting that preStop didn't run. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CentOS Linux release 7.5.1804 Kernel (e.g. ): 3.10.0-862.14.4.el7.x8664 SMP Wed Sep 26 15:12:11 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: in-house Others:
/sig api-machinery
this is because of this code: invoked from here: it is passing in the eviction delete options field as-is, meaning nil gets propagated and defaulted to mean delete immediately. when calling delete APIs normally, delete options defaults to a non-nil empty set of options:
Fixed in
patch looks legit. Thanks for the quick reaction and for the pointers!
I've a elasticsearch stateful set which have a preHook which never completes and the pod is never killed despite terminationGracePeriodSeconds being set. Is it related ? If so can this be considered to be cherry-picked on v1.13.x release branch ?

Related discussion:
text: go_test( name = "go_default_test", <del> srcs = ["storage_test.go"], </del> <ins> srcs = [ "eviction_test.go", "storage_test.go", ], </ins> embed = [":go_default_library"], deps = [ "//pkg/apis/core:go_default_library", <ins> "//pkg/apis/policy:go_default_library", "//pkg/client/clientset_generated/internalclientset/fake:go_default_library", </ins> "//pkg/registry/registrytest:go_default_library", "//pkg/securitycontext:go_default_library", "//staging/src/k8s.io/api/core/v1:go_default_library",
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: When using the function from Python's Kuberneets API, we noticed that Pods are terminated bythe cluster without respecting their preStop hooks or terminationGracePeriodSeconds seconds. What you expected to happen: When using the eviction API, just as when using delete, Pods should be allowed to run their preStop hooks, as well as be given terminationGracePeriodSeconds before being terminated with SIGKILL. How to reproduce it (as minimally and precisely as possible): the following pod: the pod logs in another window to evict the pod. Here is a sample curl equivalent of what (1.13) is doing - happy path: Here is a sample curl equivalent of what Python's is doing (when not passed ) - broken path: The log output generated by the guinea pod on the happy path looks like this: The "Got signal 15" message shows up only after the preStop hook is finished, so ~10 seconds after the evict call is made What happens on the broken path is: We seem to be getting 2x SIGTERM, there are no messages indicating continued processing after the signal, and the signal itself is registered immediately after the eviction call is made, suggesting that preStop didn't run. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CentOS Linux release 7.5.1804 Kernel (e.g. ): 3.10.0-862.14.4.el7.x8664 SMP Wed Sep 26 15:12:11 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: in-house Others:
/sig api-machinery
this is because of this code: invoked from here: it is passing in the eviction delete options field as-is, meaning nil gets propagated and defaulted to mean delete immediately. when calling delete APIs normally, delete options defaults to a non-nil empty set of options:
Fixed in
patch looks legit. Thanks for the quick reaction and for the pointers!
I've a elasticsearch stateful set which have a preHook which never completes and the pod is never killed despite terminationGracePeriodSeconds being set. Is it related ? If so can this be considered to be cherry-picked on v1.13.x release branch ?

Related discussion:
text: // At this point there was either no PDB or we succeeded in decrementing // Try the delete <del> _, _, err = r.store.Delete(ctx, eviction.Name, eviction.DeleteOptions) </del> <ins> deletionOptions := eviction.DeleteOptions if deletionOptions == nil { // default to non-nil to trigger graceful deletion deletionOptions = &metav1.DeleteOptions{} } _, _, err = r.store.Delete(ctx, eviction.Name, deletionOptions) </ins> if err != nil { return nil, err }
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: When using the function from Python's Kuberneets API, we noticed that Pods are terminated bythe cluster without respecting their preStop hooks or terminationGracePeriodSeconds seconds. What you expected to happen: When using the eviction API, just as when using delete, Pods should be allowed to run their preStop hooks, as well as be given terminationGracePeriodSeconds before being terminated with SIGKILL. How to reproduce it (as minimally and precisely as possible): the following pod: the pod logs in another window to evict the pod. Here is a sample curl equivalent of what (1.13) is doing - happy path: Here is a sample curl equivalent of what Python's is doing (when not passed ) - broken path: The log output generated by the guinea pod on the happy path looks like this: The "Got signal 15" message shows up only after the preStop hook is finished, so ~10 seconds after the evict call is made What happens on the broken path is: We seem to be getting 2x SIGTERM, there are no messages indicating continued processing after the signal, and the signal itself is registered immediately after the eviction call is made, suggesting that preStop didn't run. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CentOS Linux release 7.5.1804 Kernel (e.g. ): 3.10.0-862.14.4.el7.x8664 SMP Wed Sep 26 15:12:11 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: in-house Others:
/sig api-machinery
this is because of this code: invoked from here: it is passing in the eviction delete options field as-is, meaning nil gets propagated and defaulted to mean delete immediately. when calling delete APIs normally, delete options defaults to a non-nil empty set of options:
Fixed in
patch looks legit. Thanks for the quick reaction and for the pointers!
I've a elasticsearch stateful set which have a preHook which never completes and the pod is never killed despite terminationGracePeriodSeconds being set. Is it related ? If so can this be considered to be cherry-picked on v1.13.x release branch ?

Related discussion:
text: } createNode2NormalPodEviction := func(client clientset.Interface) func() error { return func() error { <ins> zero := int64(0) </ins> return client.Policy().Evictions("ns").Evict(&policy.Eviction{ TypeMeta: metav1.TypeMeta{ APIVersion: "policy/v1beta1",
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: When using the function from Python's Kuberneets API, we noticed that Pods are terminated bythe cluster without respecting their preStop hooks or terminationGracePeriodSeconds seconds. What you expected to happen: When using the eviction API, just as when using delete, Pods should be allowed to run their preStop hooks, as well as be given terminationGracePeriodSeconds before being terminated with SIGKILL. How to reproduce it (as minimally and precisely as possible): the following pod: the pod logs in another window to evict the pod. Here is a sample curl equivalent of what (1.13) is doing - happy path: Here is a sample curl equivalent of what Python's is doing (when not passed ) - broken path: The log output generated by the guinea pod on the happy path looks like this: The "Got signal 15" message shows up only after the preStop hook is finished, so ~10 seconds after the evict call is made What happens on the broken path is: We seem to be getting 2x SIGTERM, there are no messages indicating continued processing after the signal, and the signal itself is registered immediately after the eviction call is made, suggesting that preStop didn't run. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CentOS Linux release 7.5.1804 Kernel (e.g. ): 3.10.0-862.14.4.el7.x8664 SMP Wed Sep 26 15:12:11 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: in-house Others:
/sig api-machinery
this is because of this code: invoked from here: it is passing in the eviction delete options field as-is, meaning nil gets propagated and defaulted to mean delete immediately. when calling delete APIs normally, delete options defaults to a non-nil empty set of options:
Fixed in
patch looks legit. Thanks for the quick reaction and for the pointers!
I've a elasticsearch stateful set which have a preHook which never completes and the pod is never killed despite terminationGracePeriodSeconds being set. Is it related ? If so can this be considered to be cherry-picked on v1.13.x release branch ?

Related discussion:
text: Name:      "node2normalpod", Namespace: "ns", }, <ins> DeleteOptions: &metav1.DeleteOptions{GracePeriodSeconds: &zero}, </ins> }) } } createNode2MirrorPodEviction := func(client clientset.Interface) func() error { return func() error { <ins> zero := int64(0) </ins> return client.Policy().Evictions("ns").Evict(&policy.Eviction{ TypeMeta: metav1.TypeMeta{ APIVersion: "policy/v1beta1",
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: When using the function from Python's Kuberneets API, we noticed that Pods are terminated bythe cluster without respecting their preStop hooks or terminationGracePeriodSeconds seconds. What you expected to happen: When using the eviction API, just as when using delete, Pods should be allowed to run their preStop hooks, as well as be given terminationGracePeriodSeconds before being terminated with SIGKILL. How to reproduce it (as minimally and precisely as possible): the following pod: the pod logs in another window to evict the pod. Here is a sample curl equivalent of what (1.13) is doing - happy path: Here is a sample curl equivalent of what Python's is doing (when not passed ) - broken path: The log output generated by the guinea pod on the happy path looks like this: The "Got signal 15" message shows up only after the preStop hook is finished, so ~10 seconds after the evict call is made What happens on the broken path is: We seem to be getting 2x SIGTERM, there are no messages indicating continued processing after the signal, and the signal itself is registered immediately after the eviction call is made, suggesting that preStop didn't run. Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CentOS Linux release 7.5.1804 Kernel (e.g. ): 3.10.0-862.14.4.el7.x8664 SMP Wed Sep 26 15:12:11 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: in-house Others:
/sig api-machinery
this is because of this code: invoked from here: it is passing in the eviction delete options field as-is, meaning nil gets propagated and defaulted to mean delete immediately. when calling delete APIs normally, delete options defaults to a non-nil empty set of options:
Fixed in
patch looks legit. Thanks for the quick reaction and for the pointers!
I've a elasticsearch stateful set which have a preHook which never completes and the pod is never killed despite terminationGracePeriodSeconds being set. Is it related ? If so can this be considered to be cherry-picked on v1.13.x release branch ?

Related discussion:
text: Name:      "node2mirrorpod", Namespace: "ns", }, <ins> DeleteOptions: &metav1.DeleteOptions{GracePeriodSeconds: &zero}, </ins> }) } }
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: I'm running a K8s 1.13 cluster and a partial netsplit happened due to a defective linecard dropping only TCP packets above a certain sizes between two masters, but the third could still communicate with both (so the fault made the network non-transitive). etcd was run on the physical network, but Kubernetes traffic runs on a Wireguard overlay for security and routing is provided by OSPF (so internal communication was not affected since it was tunelled over UDP). Etcd experienced some temporary issues due to the connections dropping only certain packets (above ~1000 bytes), but quickly dropped one host out of the quorum, at which point it was degraded but all the remaining hosts could communicate freely. Each master node has an etcd instance and an apiserver. All apiservers are anycasted inside the overlay to balance the load and provide redundancy and the announcement is coupled to the health check, so in case any apiserver fails no traffic will be routed to it. The problem was that the apiserver did not fail its healthcheck even though all etcd write requests timed out. The non-working apiserver was still accessible and got routed its share of the anycast traffic since that was routed over the UDP overlay. Thus ~1/3 of all workloads and nodes were stuck on a non-working apiserver until I nuked the non-working apiserver. TLDR: One etcd server started timing out all writes due to a non-transitive netsplit and the Kubernetes apiserver healthcheck did not fail, causing 1/3 of K8s clients to not have a working control plane until manual intervention. What you expected to happen: The health check should've failed (in my opinion, I'm happy to discuss if you think otherwise). How to reproduce it (as minimally and precisely as possible): Set up 3 etcds with corresponding apiservers, introduce a split between two (shouldn't have to be a packet size limit, a full disconnect should reproduce cleanly), observe that all three report healthy even though at least one of them no longer works. Anything else we need to know?: Environment: Kubernetes version (use ): v1.13.0 Cloud provider or hardware configuration: Physical x86_64 servers OS (e.g. from /etc/os-release): Custom Kernel (e.g. ): Custom, based on 4.19. No patches affect this functionality. Install tools: Custom Others:
/sig api-machinery (maybe cluster-ops?)
cc
Looks like the etcdctl endpoint health command does a random read after fetching the list of members - - we should probably do the same?
Looks sane and would have prevented my issue. Should I PR this?
sure. thanks
/assign

Related discussion:
text: import ( "context" "fmt" <ins> "path" </ins> "sync" "sync/atomic" "time"
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: azure file remount on Windows in same node would fail What you expected to happen: If user delete a pod with azure file mount in deployment and it would probably schedule a pod on same node, azure file mount will fail since command would fail if file share is already mounted on the node: It's related to bug: Error logs would be like following: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): v1.9-v1.12, master Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: /assign
/sig windows
/sig azure
/milestone v1.14 cc This may be good to backport to 1.12 or 1.13 too if customers are using those versions.

Related discussion:
text: } } <del> // Mount : mounts source to target as NTFS with given options. </del> <ins> // Mount : mounts source to target with given options. // currently only supports cifs(smb), bind mount(for disk) </ins> func (mounter *Mounter) Mount(source string, target string, fstype string, options []string) error { target = normalizeWindowsPath(target) if source == "tmpfs" { <del> klog.V(3).Infof("azureMount: mounting source (%q), target (%q), with options (%q)", source, target, options) </del> <ins> klog.V(3).Infof("mounting source (%q), target (%q), with options (%q)", source, target, options) </ins> return os.MkdirAll(target, 0755) }
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!--What happened: azure file remount on Windows in same node would fail What you expected to happen: If user delete a pod with azure file mount in deployment and it would probably schedule a pod on same node, azure file mount will fail since command would fail if file share is already mounted on the node: It's related to bug: Error logs would be like following: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): v1.9-v1.12, master Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: /assign
/sig windows
/sig azure
/milestone v1.14 cc This may be good to backport to 1.12 or 1.13 too if customers are using those versions.

Related discussion:
text: return err } <del> klog.V(4).Infof("azureMount: mount options(%q) source:%q, target:%q, fstype:%q, begin to mount", </del> <ins> klog.V(4).Infof("mount options(%q) source:%q, target:%q, fstype:%q, begin to mount", </ins> options, source, target, fstype) <del> bindSource := "" </del> <ins> bindSource := source </ins> // tell it's going to mount azure disk or azure file according to options if bind, _, _ := isBind(options); bind {