Change:
/kind bug Using , if a block device is symlinked to the discovery directory e.g. , then MapVolume will fail with since the containerized kubelet cannot see the path on host. is mounted by containerized kubelet, so if before we try to map the device we set devicePath to the evaluation of symlink and end up with a devicePath of , containerized kubelet will see the device and successfully map the device. Are there any issues with this solution? This is on Kubernetes 1.10
/sig storage
ah yes containerized kubelet has many issues when dealing with symlinks. Check out Mounter.GetMode(), it calls a nsenter EvalSymlinks which should be able to do what you need. But actually, one thing to consider, is do you need to do the losetup on the host, or does inside the container work?
Yes it appears to work from inside the container, on the host I run which disappears after I delete the pod using the device. My understanding of running losetup in MapVolume is not that great but I suppose that as long as that /dev/loop0 is created we are fine. Thanks for the quick response, I'll create a PR using that function and tag you
Does it create it on the host or in the containerized kubelet? What happens when containerized kubelet is restarted? The purpose of losetup is to do "refcounting" on the block device to prevent possible naming collisions when devices are attached/detached under unhealthy conditions. Ie, if a device showed up as /dev/sdb and you gave that to a pod, and then for some reason it become unhealthy and disappeared (while the pod is still running), another device (maybe even another volume type) could get attached to the node and reuse /dev/sdb. Now the pod has access to a device it's not supposed to. With filesystem volumes, the refcounting is done with mount points, but for block device, loopback is the equivalent.
It is created on the host (and also visible in the container), and stays around through restart. I understand now, thank you.
Actually after I restart it becomes /vdb instead of /dev/vdb in the losetup output, idk what that is about. There is no such thing as /vdb and /dev/vdb is still intact of course. Seems inconsequential since it is disappearing after all pods using the device are deleted, i.e. it's still serving as a refcounter

Discussion:
return m.wrappedMounter.ExistsPath(pathname) } <ins> func (m *execMounter) EvalHostSymlinks(pathname string) (string, error) { return m.wrappedMounter.EvalHostSymlinks(pathname) } </ins> func (m *execMounter) PrepareSafeSubpath(subPath Subpath) (newHostPath string, cleanupAction func(), err error) { return m.wrappedMounter.PrepareSafeSubpath(subPath) }

return true, errors.New("not implemented") } <ins> func (m *execMounter) EvalHostSymlinks(pathname string) (string, error) { return "", errors.New("not implemented") } </ins> func (mounter *execMounter) PrepareSafeSubpath(subPath Subpath) (newHostPath string, cleanupAction func(), err error) { return subPath.Path, nil, nil }

return false, nil } <ins> func (f *FakeMounter) EvalHostSymlinks(pathname string) (string, error) { return pathname, nil } </ins> func (f *FakeMounter) PrepareSafeSubpath(subPath Subpath) (newHostPath string, cleanupAction func(), err error) { return subPath.Path, nil, nil }

// Will operate in the host mount namespace if kubelet is running in a container. // Error is returned on any other error than "file not found". ExistsPath(pathname string) (bool, error) <ins> // EvalHostSymlinks returns the path name after evaluating symlinks. // Will operate in the host mount namespace if kubelet is running in a container. EvalHostSymlinks(pathname string) (string, error) </ins> // CleanSubPaths removes any bind-mounts created by PrepareSafeSubpath in given // pod volume directory. CleanSubPaths(podDir string, volumeName string) error

return utilfile.FileExists(pathname) } <ins> func (mounter *Mounter) EvalHostSymlinks(pathname string) (string, error) { return filepath.EvalSymlinks(pathname) } </ins> // formatAndMount uses unix utils to format and mount the given disk func (mounter *SafeFormatAndMount) formatAndMount(source string, target string, fstype string, options []string) error { readOnly := false

return true, errors.New("not implemented") } <ins> func (mounter *Mounter) EvalHostSymlinks(pathname string) (string, error) { return "", unsupportedErr } </ins> func (mounter *Mounter) PrepareSafeSubpath(subPath Subpath) (newHostPath string, cleanupAction func(), err error) { return subPath.Path, nil, unsupportedErr }

return utilfile.FileExists(pathname) } <ins> // EvalHostSymlinks returns the path name after evaluating symlinks func (mounter *Mounter) EvalHostSymlinks(pathname string) (string, error) { return filepath.EvalSymlinks(pathname) } </ins> // check whether hostPath is within volume path // this func will lock all intermediate subpath directories, need to close handle outside of this func after container started func lockAndCheckSubPath(volumePath, hostPath string) ([]uintptr, error) {

return utilfile.FileExists(kubeletpath) } <ins> func (mounter *NsenterMounter) EvalHostSymlinks(pathname string) (string, error) { return mounter.ne.EvalSymlinks(pathname, true) } </ins> func (mounter *NsenterMounter) CleanSubPaths(podDir string, volumeName string) error { return doCleanSubPaths(mounter, podDir, volumeName) }

return true, errors.New("not implemented") } <ins> func (*NsenterMounter) EvalHostSymlinks(pathname string) (string, error) { return "", errors.New("not implemented") } </ins> func (*NsenterMounter) SafeMakeDir(pathname string, base string, perm os.FileMode) error { return nil }

return volumeToMount.GenerateError("MapVolume failed", fmt.Errorf("Device path of the volume is empty")) } <ins> // When kubelet is containerized, devicePath may be a symlink at a place unavailable to // kubelet, so evaluate it on the host and expect that it links to a device in /dev, // which will be available to containerized kubelet. If still it does not exist, // AttachFileDevice will fail. If kubelet is not containerized, eval it anyway. mounter := og.GetVolumePluginMgr().Host.GetMounter(blockVolumePlugin.GetPluginName()) devicePath, err = mounter.EvalHostSymlinks(devicePath) if err != nil { return volumeToMount.GenerateError("MapVolume.EvalHostSymlinks failed", err) } </ins> // Map device to global and pod device map path volumeMapPath, volName := blockVolumeMapper.GetPodDeviceMapPath() mapErr = blockVolumeMapper.MapDevice(devicePath, globalMapPath, volumeMapPath, volName, volumeToMount.Pod.UID)