Repository: canonical/cloud-init

Issue:
FreeBSD: get_tmp_exec_path() still fails
# Bug report

This is a clean boot on Vultr (with Vultr as the only datasource)

```python-traceback
2023-06-15 17:50:18,413 - handlers.py[DEBUG]: finish: init-local/search-Vultr: FAIL: no local data found from DataSourceVultr
2023-06-15 17:50:18,413 - util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceVultr.DataSourceVultr'> failed
2023-06-15 17:50:18,413 - util.py[DEBUG]: Getting data from <class 'cloudinit.sources.DataSourceVultr.DataSourceVultr'> failed
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/cloudinit/sources/__init__.py", line 1009, in find_source
    if s.update_metadata_if_supported(
  File "/usr/local/lib/python3.9/site-packages/cloudinit/sources/__init__.py", line 895, in update_metadata_if_supported
    result = self.get_data()
  File "/usr/local/lib/python3.9/site-packages/cloudinit/sources/__init__.py", line 437, in get_data
    return_value = self._check_and_get_data()
  File "/usr/local/lib/python3.9/site-packages/cloudinit/sources/__init__.py", line 366, in _check_and_get_data
    return self._get_data()
  File "/usr/local/lib/python3.9/site-packages/cloudinit/sources/DataSourceVultr.py", line 56, in _get_data
    self.metadata = self.get_metadata()
  File "/usr/local/lib/python3.9/site-packages/cloudinit/sources/DataSourceVultr.py", line 98, in get_metadata
    tmp_dir=self.distro.get_tmp_exec_path(),
  File "/usr/local/lib/python3.9/site-packages/cloudinit/distros/__init__.py", line 989, in get_tmp_exec_path
    if not util.has_mount_opt(tmp_dir, "noexec"):
  File "/usr/local/lib/python3.9/site-packages/cloudinit/util.py", line 2748, in has_mount_opt
    *_, mnt_opts = get_mount_info(path, get_mnt_opts=True)
TypeError: cannot unpack non-iterable NoneType object
2023-06-15 17:50:18,421 - main.py[DEBUG]: No local datasource found
```

Even after #2146, this still fails.
As explained in that PR, on Linux with `parse_mount_info()`, we're allowed to return a parent mount for a non-existent directory / mountpoint.
On BSD, or with `parse_mount()` we are not allowed to do that.
After adding options parsing to `parse_mount()`, due to this limitation, `get_tmp_exec_path()` still fails with the same kind of exception on FreeBSD.

## Steps to reproduce the problem

on FreeBSD,

```python
meena@defbix ~/s/cloud-init (main)> sudo -H ipython
In [1]: from cloudinit.distros import freebsd
In [2]: f = freebsd.Distro('freebsd', {}, {})
In [3]: f.get_tmp_exec_path()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[3], line 1
----> 1 f.get_tmp_exec_path()

File /home/meena/src/cloud-init/cloudinit/distros/__init__.py:989, in Distro.get_tmp_exec_path(self)
    987 def get_tmp_exec_path(self) -> str:
    988     tmp_dir = temp_utils.get_tmp_ancestor(needs_exe=True)
--> 989     if not util.has_mount_opt(tmp_dir, "noexec"):
    990         return tmp_dir
    991     return os.path.join(self.usr_lib_exec, "cloud-init", "clouddir")

File /home/meena/src/cloud-init/cloudinit/util.py:2748, in has_mount_opt(path, opt)
   2747 def has_mount_opt(path, opt: str) -> bool:
-> 2748     *_, mnt_opts = get_mount_info(path, get_mnt_opts=True)
   2749     return opt in mnt_opts.split(",")

TypeError: cannot unpack non-iterable NoneType object

In [4]: 
```

## Environment details
- Cloud-init version: HEAD
- Operating System Distribution: FreeBSD
- Cloud provider, platform or installer type: Vultr



Discussion:
cc @CalvoM @holmanb 
Thanks @igalic for reporting this. Could you please attach the [full logs](https://canonical-cloud-init.readthedocs-hosted.com/en/latest/howto/bugs.html#collect-logs)?
full logs: [cloud-init.tar.gz](https://github.com/canonical/cloud-init/files/11949118/cloud-init.tar.gz)

n.b.: The code, as-is, right now, is wrong. Here's the `mount` output from my test / dev machine:

```
% mount
meena@defbix ~> mount
zroot/ROOT/PkgBase on / (zfs, local, noatime, nfsv4acls)
devfs on /dev (devfs)
/dev/gpt/efiboot0 on /boot/efi (msdosfs, local)
zroot/home on /home (zfs, local, noatime, nfsv4acls)
zroot/poudriere on /poudriere (zfs, local, noatime, nfsv4acls)
zroot/var/log on /var/log (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/mail on /var/mail (zfs, local, nfsv4acls)
zroot/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
zroot on /zroot (zfs, local, noatime, nfsv4acls)
zroot/usr/src on /usr/src (zfs, local, noatime, nfsv4acls)
zroot/var/crash on /var/crash (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/tmp on /var/tmp (zfs, local, noatime, nosuid, nfsv4acls)
zroot/usr/ports on /usr/ports (zfs, local, noatime, nosuid, nfsv4acls)
zroot/var/audit on /var/audit (zfs, local, noatime, noexec, nosuid, nfsv4acls)
```

and here's a series of attempts to get something sensible out of the code:

```python
In [27]: util.parse_mount("/", get_mnt_opts=True)
Out[27]: ('zroot/ROOT/PkgBase', 'zfs', '/', 'local,noatime,nfsv4acls')

In [28]: util.parse_mount("/var", get_mnt_opts=True)
Out[28]: ('zroot/var/audit', 'zfs', '/', 'local,noatime,nfsv4acls')

In [29]: util.parse_mount("/var/tmp", get_mnt_opts=True)
Out[29]: ('zroot/var/tmp', 'zfs', '/var/tmp', 'local,noatime,nosuid,nfsv4acls')

In [25]: tmp_dir
Out[25]: '/var/tmp/cloud-init'

In [26]: util.parse_mount(tmp_dir, get_mnt_opts=True)

```

why does querying `/var` return `zroot/var/audit` with a mountpoint of `/`??
given that `/var` doesn't exist, the mountpoint `/` is correct, but the "device" `zroot/var/audit` is bogus.

next, we have the correct result for `/var/tmp`.
but the query for `/var/tmp/cloud-init` doesn't return anything at all… maybe because it's a level too deep?
I don't know. yet.

Resolving patch:
diff --git a/cloudinit/util.py b/cloudinit/util.py
--- a/cloudinit/util.py
+++ b/cloudinit/util.py
@@ -2677,6 +2677,8 @@ def get_device_info_from_zpool(zpool):
 
 
 def parse_mount(path, get_mnt_opts=False):
+    """Return the mount information for PATH given the lines ``mount(1)``
+    This function is compatible with ``util.parse_mount_info()``"""
     (mountoutput, _err) = subp.subp(["mount"])
 
     # there are 2 types of mount outputs we have to parse therefore
@@ -2738,21 +2740,27 @@ def parse_mount(path, get_mnt_opts=False):
         # continue finding the real device like '/dev/da0'.
         # this is only valid for non zfs file systems as a zpool
         # can have gpt labels as disk.
+        # It also doesn't really make sense for NFS.
         devm = re.search("^(/dev/.+)[sp]([0-9])$", devpth)
         if not devm and is_FreeBSD() and fs_type not in ["zfs", "nfs"]:
-            devpth = get_freebsd_devpth(path)
+            # don't duplicate the effort of finding the mountpoint in
+            # ``get_freebsd_devpth()`` by passing it the ``path``
+            # instead only resolve the ``devpth``
+            devpth = get_freebsd_devpth(devpth)
+        match_devpth = devpth
 
         if match_mount_point == path:
             break
 
-    if mount_point and path in mount_point:
-        if get_mnt_opts:
-            if devpth and fs_type and match_mount_point and mount_options:
-                return (devpth, fs_type, match_mount_point, mount_options)
-        else:
-            if devpth and fs_type and match_mount_point:
-                return (devpth, fs_type, match_mount_point)
-    return None
+    if not match_mount_point or match_mount_point not in path:
+        # return early here, so we can actually read what's happening below
+        return None
+    if get_mnt_opts:
+        if match_devpth and fs_type and match_mount_point and mount_options:
+            return (match_devpth, fs_type, match_mount_point, mount_options)
+    else:
+        if match_devpth and fs_type and match_mount_point:
+            return (match_devpth, fs_type, match_mount_point)
 
 
 def get_mount_info(path, log=LOG, get_mnt_opts=False):


Test patch:
diff --git a/tests/unittests/test_util.py b/tests/unittests/test_util.py
--- a/tests/unittests/test_util.py
+++ b/tests/unittests/test_util.py
@@ -2201,9 +2201,9 @@ def test_parse_mount_with_ext(self, mount_out):
         # this one exists in mount_parse_ext.txt
         ret = util.parse_mount("/sys/kernel/debug")
         self.assertEqual(("none", "debugfs", "/sys/kernel/debug"), ret)
-        # this one does not even exist in mount_parse_ext.txt
-        ret = util.parse_mount("/not/existing/mount")
-        self.assertIsNone(ret)
+        # this one does not exist in mount_parse_ext.txt
+        ret = util.parse_mount("/var/tmp/cloud-init")
+        self.assertEqual(("/dev/mapper/vg00-lv_var", "ext4", "/var"), ret)
 
     @mock.patch("cloudinit.subp.subp")
     def test_parse_mount_with_zfs(self, mount_out):
@@ -2217,9 +2217,9 @@ def test_parse_mount_with_zfs(self, mount_out):
         # this one is the root, valid and also exists in mount_parse_zfs.txt
         ret = util.parse_mount("/")
         self.assertEqual(("vmzroot/ROOT/freebsd", "zfs", "/"), ret)
-        # this one does not even exist in mount_parse_ext.txt
-        ret = util.parse_mount("/not/existing/mount")
-        self.assertIsNone(ret)
+        # this one does not exist in mount_parse_ext.txt
+        ret = util.parse_mount("/var/tmp/cloud-init")
+        self.assertEqual(("vmzroot/var/tmp", "zfs", "/var/tmp"), ret)
 
 
 class TestIsX86(helpers.CiTestCase):