jiebi/CodeConvo huggingface dataset:
Code change:
We fixed invalid device path in bunch of places via - but call still could use invalid device path. That check can sometimes use invalid path and hence can cause UnmountDevice operation to fails even though device has already been unmounted. /sig storage /sig openstack /assign
Actually this does not affect version 1.10 onwards. Not sure if 1.9 is still supported. I think I will close this for now. /close
I just noticed that 1.9 is still supported and hence it is probably worth fixing there for now.
How come the check can "sometimes" use invalid device path?
So on Cinder for example - call would return something like as attach device path. But if you see the linked PR(), will do a and use that device path for global mount. In general it is always a good idea to use device path returned by as single source of truth. This applies to something like EBS with NVMe too where device is discovered via call (but nvme disks aren't directly affected by this bug because returned device path on EBS is bogus).
So on EBS with NVMe for example - check will always fail (but still return success) because it will use something like as device path which does not actually exist on the node.
You said the issue only happens "sometimes". Can you clarify why it only happens occasionally?
doesn't the linked PR make it clear? detection based on the drive serial number (aka It's cinder ID) on the kubelet itself. As for why - Cinder sometimes returns wrong device path on call, I don't know. But it happens and we have worked around it.
Fixed by

Related discussion:
text: } volumeObj.globallyMounted = globallyMounted <ins> volumeObj.devicePath = devicePath </ins> asw.attachedVolumes[volumeName] = volumeObj return nil }
Code change:
We fixed invalid device path in bunch of places via - but call still could use invalid device path. That check can sometimes use invalid path and hence can cause UnmountDevice operation to fails even though device has already been unmounted. /sig storage /sig openstack /assign
Actually this does not affect version 1.10 onwards. Not sure if 1.9 is still supported. I think I will close this for now. /close
I just noticed that 1.9 is still supported and hence it is probably worth fixing there for now.
How come the check can "sometimes" use invalid device path?
So on Cinder for example - call would return something like as attach device path. But if you see the linked PR(), will do a and use that device path for global mount. In general it is always a good idea to use device path returned by as single source of truth. This applies to something like EBS with NVMe too where device is discovered via call (but nvme disks aren't directly affected by this bug because returned device path on EBS is bogus).
So on EBS with NVMe for example - check will always fail (but still return success) because it will use something like as device path which does not actually exist on the node.
You said the issue only happens "sometimes". Can you clarify why it only happens occasionally?
doesn't the linked PR make it clear? detection based on the drive serial number (aka It's cinder ID) on the kubelet itself. As for why - Cinder sometimes returns wrong device path on call, I don't know. But it happens and we have worked around it.
Fixed by

Related discussion:
text: } // Act <del> err = asw.MarkDeviceAsMounted(generatedVolumeName) </del> <ins> err = asw.MarkDeviceAsMounted(generatedVolumeName, devicePath) </ins> // Assert if err != nil {
Code change:
We fixed invalid device path in bunch of places via - but call still could use invalid device path. That check can sometimes use invalid path and hence can cause UnmountDevice operation to fails even though device has already been unmounted. /sig storage /sig openstack /assign
Actually this does not affect version 1.10 onwards. Not sure if 1.9 is still supported. I think I will close this for now. /close
I just noticed that 1.9 is still supported and hence it is probably worth fixing there for now.
How come the check can "sometimes" use invalid device path?
So on Cinder for example - call would return something like as attach device path. But if you see the linked PR(), will do a and use that device path for global mount. In general it is always a good idea to use device path returned by as single source of truth. This applies to something like EBS with NVMe too where device is discovered via call (but nvme disks aren't directly affected by this bug because returned device path on EBS is bogus).
So on EBS with NVMe for example - check will always fail (but still return success) because it will use something like as device path which does not actually exist on the node.
You said the issue only happens "sometimes". Can you clarify why it only happens occasionally?
doesn't the linked PR make it clear? detection based on the drive serial number (aka It's cinder ID) on the kubelet itself. As for why - Cinder sometimes returns wrong device path on call, I don't know. But it happens and we have worked around it.
Fixed by

Related discussion:
text: continue } if volume.pluginIsAttachable { <del> err = rc.actualStateOfWorld.MarkDeviceAsMounted(volume.volumeName) </del> <ins> err = rc.actualStateOfWorld.MarkDeviceAsMounted(volume.volumeName, volume.devicePath) </ins> if err != nil { glog.Errorf("Could not mark device is mounted to actual state of world: %v", err) continue
Code change:
We fixed invalid device path in bunch of places via - but call still could use invalid device path. That check can sometimes use invalid path and hence can cause UnmountDevice operation to fails even though device has already been unmounted. /sig storage /sig openstack /assign
Actually this does not affect version 1.10 onwards. Not sure if 1.9 is still supported. I think I will close this for now. /close
I just noticed that 1.9 is still supported and hence it is probably worth fixing there for now.
How come the check can "sometimes" use invalid device path?
So on Cinder for example - call would return something like as attach device path. But if you see the linked PR(), will do a and use that device path for global mount. In general it is always a good idea to use device path returned by as single source of truth. This applies to something like EBS with NVMe too where device is discovered via call (but nvme disks aren't directly affected by this bug because returned device path on EBS is bogus).
So on EBS with NVMe for example - check will always fail (but still return success) because it will use something like as device path which does not actually exist on the node.
You said the issue only happens "sometimes". Can you clarify why it only happens occasionally?
doesn't the linked PR make it clear? detection based on the drive serial number (aka It's cinder ID) on the kubelet itself. As for why - Cinder sometimes returns wrong device path on call, I don't know. But it happens and we have worked around it.
Fixed by

Related discussion:
text: MarkVolumeAsUnmounted(podName volumetypes.UniquePodName, volumeName v1.UniqueVolumeName) error // Marks the specified volume as having been globally mounted. <del> MarkDeviceAsMounted(volumeName v1.UniqueVolumeName) error </del> <ins> MarkDeviceAsMounted(volumeName v1.UniqueVolumeName, devicePath string) error </ins> // Marks the specified volume as having its global mount unmounted. MarkDeviceAsUnmounted(volumeName v1.UniqueVolumeName) error
Code change:
We fixed invalid device path in bunch of places via - but call still could use invalid device path. That check can sometimes use invalid path and hence can cause UnmountDevice operation to fails even though device has already been unmounted. /sig storage /sig openstack /assign
Actually this does not affect version 1.10 onwards. Not sure if 1.9 is still supported. I think I will close this for now. /close
I just noticed that 1.9 is still supported and hence it is probably worth fixing there for now.
How come the check can "sometimes" use invalid device path?
So on Cinder for example - call would return something like as attach device path. But if you see the linked PR(), will do a and use that device path for global mount. In general it is always a good idea to use device path returned by as single source of truth. This applies to something like EBS with NVMe too where device is discovered via call (but nvme disks aren't directly affected by this bug because returned device path on EBS is bogus).
So on EBS with NVMe for example - check will always fail (but still return success) because it will use something like as device path which does not actually exist on the node.
You said the issue only happens "sometimes". Can you clarify why it only happens occasionally?
doesn't the linked PR make it clear? detection based on the drive serial number (aka It's cinder ID) on the kubelet itself. As for why - Cinder sometimes returns wrong device path on call, I don't know. But it happens and we have worked around it.
Fixed by

Related discussion:
text: // Update actual state of world to reflect volume is globally mounted markDeviceMountedErr := actualStateOfWorld.MarkDeviceAsMounted( <del> volumeToMount.VolumeName) </del> <ins> volumeToMount.VolumeName, devicePath) </ins> if markDeviceMountedErr != nil { // On failure, return error. Caller will log and retry. return volumeToMount.GenerateErrorDetailed("MountVolume.MarkDeviceAsMounted failed", markDeviceMountedErr)
Code change:
We fixed invalid device path in bunch of places via - but call still could use invalid device path. That check can sometimes use invalid path and hence can cause UnmountDevice operation to fails even though device has already been unmounted. /sig storage /sig openstack /assign
Actually this does not affect version 1.10 onwards. Not sure if 1.9 is still supported. I think I will close this for now. /close
I just noticed that 1.9 is still supported and hence it is probably worth fixing there for now.
How come the check can "sometimes" use invalid device path?
So on Cinder for example - call would return something like as attach device path. But if you see the linked PR(), will do a and use that device path for global mount. In general it is always a good idea to use device path returned by as single source of truth. This applies to something like EBS with NVMe too where device is discovered via call (but nvme disks aren't directly affected by this bug because returned device path on EBS is bogus).
So on EBS with NVMe for example - check will always fail (but still return success) because it will use something like as device path which does not actually exist on the node.
You said the issue only happens "sometimes". Can you clarify why it only happens occasionally?
doesn't the linked PR make it clear? detection based on the drive serial number (aka It's cinder ID) on the kubelet itself. As for why - Cinder sometimes returns wrong device path on call, I don't know. But it happens and we have worked around it.
Fixed by

Related discussion:
text: // Update actual state of world to reflect volume is globally mounted markDeviceMappedErr := actualStateOfWorld.MarkDeviceAsMounted( <del> volumeToMount.VolumeName) </del> <ins> volumeToMount.VolumeName, devicePath) </ins> if markDeviceMappedErr != nil { // On failure, return error. Caller will log and retry. return volumeToMount.GenerateErrorDetailed("MapVolume.MarkDeviceAsMounted failed", markDeviceMappedErr)
Code change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: After running (on master branch), a new file named left as a surprise. and the file is not matched by git ignore so it's tracked by git which is really disturbing. I tried to find out where it's created but failed. Any one has a context w/ it? Environment: OS (e.g. from /etc/os-release): Darwin Kernel Version 17.6.0: Tue May 8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASEX8664 x86_64
/sig instrumentation
the only piece of code in local-up-cluster that touches the file is
can't reproduce this. Two questions: Can you see this everytime? What is the SHA of the commit on master when you tried this? Can check with that too.
finally i realize that the cause is command is built receiving different flags between Linux and OSX. on Linux, you'll find that for linux sed the is explained as: '--in-place[=SUFFIX]' This option specifies that files are to be edited in-place. GNU 'sed' does this by creating a temporary file and sending output to this file rather than to the standard output.(1). so the is optional. but for OSX, it is: Edit files in-place, saving backups with the specified extension. If a zero-length extension is given, no backup will be saved. It is not recommended to give a zero- length extension when in-place editing files, as you risk corruption or partial content in situations where disk space is exhausted, etc. the difference is that on OSX, flag is requiring a variable , so when we're using on OSX in the script: the which is supposed to be received as a flag will be a suffix for the backup file. then the magic happened.will raise a fix for this. /assign
linking the difference between and

Related discussion:
text: set -e source "${KUBE_ROOT}/hack/lib/init.sh" <ins> kube::util::ensure-gnu-sed </ins> function usage { echo "This script starts a local kube cluster. "
Code change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: After running (on master branch), a new file named left as a surprise. and the file is not matched by git ignore so it's tracked by git which is really disturbing. I tried to find out where it's created but failed. Any one has a context w/ it? Environment: OS (e.g. from /etc/os-release): Darwin Kernel Version 17.6.0: Tue May 8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASEX8664 x86_64
/sig instrumentation
the only piece of code in local-up-cluster that touches the file is
can't reproduce this. Two questions: Can you see this everytime? What is the SHA of the commit on master when you tried this? Can check with that too.
finally i realize that the cause is command is built receiving different flags between Linux and OSX. on Linux, you'll find that for linux sed the is explained as: '--in-place[=SUFFIX]' This option specifies that files are to be edited in-place. GNU 'sed' does this by creating a temporary file and sending output to this file rather than to the standard output.(1). so the is optional. but for OSX, it is: Edit files in-place, saving backups with the specified extension. If a zero-length extension is given, no backup will be saved. It is not recommended to give a zero- length extension when in-place editing files, as you risk corruption or partial content in situations where disk space is exhausted, etc. the difference is that on OSX, flag is requiring a variable , so when we're using on OSX in the script: the which is supposed to be received as a flag will be a suffix for the backup file. then the magic happened.will raise a fix for this. /assign
linking the difference between and

Related discussion:
text: #   foo: true #   bar: false for gate in $(echo ${FEATURE_GATES} | tr ',' ' '); do <del> echo $gate | sed -e 's/(.*)=(.*)/  1: 2/' </del> <ins> echo $gate | ${SED} -e 's/(.*)=(.*)/  1: 2/' </ins> done fi >>/tmp/kube-proxy.yaml
Code change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: After running (on master branch), a new file named left as a surprise. and the file is not matched by git ignore so it's tracked by git which is really disturbing. I tried to find out where it's created but failed. Any one has a context w/ it? Environment: OS (e.g. from /etc/os-release): Darwin Kernel Version 17.6.0: Tue May 8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASEX8664 x86_64
/sig instrumentation
the only piece of code in local-up-cluster that touches the file is
can't reproduce this. Two questions: Can you see this everytime? What is the SHA of the commit on master when you tried this? Can check with that too.
finally i realize that the cause is command is built receiving different flags between Linux and OSX. on Linux, you'll find that for linux sed the is explained as: '--in-place[=SUFFIX]' This option specifies that files are to be edited in-place. GNU 'sed' does this by creating a temporary file and sending output to this file rather than to the standard output.(1). so the is optional. but for OSX, it is: Edit files in-place, saving backups with the specified extension. If a zero-length extension is given, no backup will be saved. It is not recommended to give a zero- length extension when in-place editing files, as you risk corruption or partial content in situations where disk space is exhausted, etc. the difference is that on OSX, flag is requiring a variable , so when we're using on OSX in the script: the which is supposed to be received as a flag will be a suffix for the backup file. then the magic happened.will raise a fix for this. /assign
linking the difference between and

Related discussion:
text: function start_kubedns { if [[ "${ENABLE_CLUSTER_DNS}" = true ]]; then cp "${KUBE_ROOT}/cluster/addons/dns/kube-dns/kube-dns.yaml.in" kube-dns.yaml <del> sed -i -e "s/{{ pillar['dns_domain'] }}/${DNS_DOMAIN}/g" kube-dns.yaml sed -i -e "s/{{ pillar['dns_server'] }}/${DNS_SERVER_IP}/g" kube-dns.yaml </del> <ins> ${SED} -i -e "s/{{ pillar['dns_domain'] }}/${DNS_DOMAIN}/g" kube-dns.yaml ${SED} -i -e "s/{{ pillar['dns_server'] }}/${DNS_SERVER_IP}/g" kube-dns.yaml </ins> # TODO update to dns role once we have one. # use kubectl to create kubedns addon ${KUBECTL} --kubeconfig="${CERT_DIR}/admin.kubeconfig" --namespace=kube-system create -f kube-dns.yaml
Code change:
/kind bug What happened: After upgrading 1.11.1 -1.12.0 , when a node gets terminated in AWS, the controller-manager doesn't delete the node in the API. This has lead to several side effect for us : pods don't get GC Prometheus alerts firing because 100 of nodes are in the NotReady state for a long period What you expected to happen: kube-controller-manager deletes the node when it stops reporting, after checking with the cloud provider. How to reproduce it (as minimally and precisely as possible): Looking at the code, I believe this is what introduced the behaviour change : - The node gets tainted with because it's in a Terminated state. But it doesn't get removed because it still "exists" in the cloud provider. The behaviour is probably correct for a shutdown node, but in AWS a terminated node can't be restarted so there's no good reason to keep it there ? Environment: Kubernetes version : 1.2.0 Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CoreOS (sorry for the all the sigs, not sure who it belongs to) /sig aws /sig cloud-provider /sig cluster-lifecycle
as I see it should delete node if instance is in terminated state have you tested that change in that row? I do not see what could be the issue there. Or could it be that the is not in place for all binaries in 1.12.0?
I might be wrong, but I think that part of the code never gets executed once the node is tainted :
good finding. So we should remove terminated state from this row. Then it does not add shutdown taint if node is terminated - instead it will delete node. also this should return false.. because if it returns true node is not deleted
I had the same issue. After I scale down my worker node instance number from 3 to 1. One node is stuck in status and never got deleted. However, one interesting thing is one of the other node is removed successfully. do you see similar behavior too?
- can we make sure this gets cherry picked for 1.12 releases ? I think we already missed the cut for 1.12.2

Related discussion:
text: } if len(instances) == 0 { glog.Warningf("the instance %s does not exist anymore", providerID) <del> return true, nil </del> <ins> // returns false, because otherwise node is not deleted from cluster // false means that it will continue to check InstanceExistsByProviderID return false, nil </ins> } if len(instances) > 1 { return false, fmt.Errorf("multiple instances found for instance: %s", instanceID)
Code change:
/kind bug What happened: After upgrading 1.11.1 -1.12.0 , when a node gets terminated in AWS, the controller-manager doesn't delete the node in the API. This has lead to several side effect for us : pods don't get GC Prometheus alerts firing because 100 of nodes are in the NotReady state for a long period What you expected to happen: kube-controller-manager deletes the node when it stops reporting, after checking with the cloud provider. How to reproduce it (as minimally and precisely as possible): Looking at the code, I believe this is what introduced the behaviour change : - The node gets tainted with because it's in a Terminated state. But it doesn't get removed because it still "exists" in the cloud provider. The behaviour is probably correct for a shutdown node, but in AWS a terminated node can't be restarted so there's no good reason to keep it there ? Environment: Kubernetes version : 1.2.0 Cloud provider or hardware configuration: AWS OS (e.g. from /etc/os-release): CoreOS (sorry for the all the sigs, not sure who it belongs to) /sig aws /sig cloud-provider /sig cluster-lifecycle
as I see it should delete node if instance is in terminated state have you tested that change in that row? I do not see what could be the issue there. Or could it be that the is not in place for all binaries in 1.12.0?
I might be wrong, but I think that part of the code never gets executed once the node is tainted :
good finding. So we should remove terminated state from this row. Then it does not add shutdown taint if node is terminated - instead it will delete node. also this should return false.. because if it returns true node is not deleted
I had the same issue. After I scale down my worker node instance number from 3 to 1. One node is stuck in status and never got deleted. However, one interesting thing is one of the other node is removed successfully. do you see similar behavior too?
- can we make sure this gets cherry picked for 1.12 releases ? I think we already missed the cut for 1.12.2

Related discussion:
text: if instance.State != nil { state := aws.StringValue(instance.State.Name) // valid state for detaching volumes <del> if state == ec2.InstanceStateNameStopped || state == ec2.InstanceStateNameTerminated { </del> <ins> if state == ec2.InstanceStateNameStopped { </ins> return true, nil } }
Code change:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: e2e test "[sig-network] Network [It] should set TCP CLOSEWAIT timeout" requires the test target OS supports /proc/net/nfconntrack, but Ubuntu 16.04 doesn't support it. Then the test fails today like the following: What you expected to happen: The test succeeded or was skipped. How to reproduce it (as minimally and precisely as possible): Just kick Anything else we need to know?: The existence of /proc/net/nfconntrack depends on the Linux kernel config CONFIGNFCONNTRACKPROCFS, but that is disabled on Ubuntu 16.04. In addition, the test is skipped if the node number is less than 2. So I guess that is one of reasons this issue was not present until today. Environment: Kubernetes version (use ): v1.11.3 Cloud provider or hardware configuration: OpenStack OS (e.g. from /etc/os-release): Ubuntu 16.04.4 LTS Kernel (e.g. ): 4.4.0-134-generic Install tools: kubeadm Others:
/sig network
Some kube-proxy functionalities depends on conntrack, is it possible to run a conformance cluster without having conntrack enable?

Related discussion:
text: LogSSHResult(result) if result.Code != 0 || err != nil { <del> return nil, fmt.Errorf("failed running %q: %v (exit code %d)", cmd, err, result.Code) </del> <ins> return nil, fmt.Errorf("failed running %q: %v (exit code %d, stderr %v)", cmd, err, result.Code, result.Stderr) </ins> } return &result, nil