Repository: certbot/certbot

Issue:
Create directories with secure permissions regardless of the umask and python version
In Python 3.7, a significant change was made to `os.makedirs`. From the [os.makedirs documentation](https://docs.python.org/3/library/os.html#os.makedirs), that change was:

> Changed in version 3.7: The mode argument no longer affects the file permission bits of newly-created intermediate-level directories.

The reason this change is relevant for us can be seen below. Starting on a machine with Certbot installed using Python 3.7+ but never having run it, as an unprivileged user with `sudo` privileges you can run:
```
# Let Certbot initially create files with an empty umask
$ sudo sh -c 'umask 0 && certbot certificates'
...
# Then use Certbot to obtain a certificate
...
$ rm -rf /etc/letsencrypt/renewal-hooks/pre
$ mkdir /etc/letsencrypt/renewal-hooks/pre
$ echo -e '#!/bin/bash\necho I am malicious\necho I am running as user: $(id -u)' > /etc/letsencrypt/renewal-hooks/pre/hook.sh
$ chmod +x /etc/letsencrypt/renewal-hooks/pre/hook.sh
$ sudo certbot renew --force-renewal
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/example.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Plugins selected: Authenticator standalone, Installer None
Running pre-hook command: /etc/letsencrypt/renewal-hooks/pre/hook.sh
Output from pre-hook command hook.sh:
I am malicious
I am running as user: 0

Renewing an existing certificate
```
The reason this is possible is because `/etc/letsencrypt/renewal-hooks` is world writeable due to being created as an intermediate-level directory by `os.makedirs` with a umask of 0.

While most/all UNIX systems should be protected due to the default umask, I don't think we should fully rely on this.

To fix this, I think we should update `certbot.util.make_or_verify_dir` to temporarily set the `umask` before calling `filesystem.makedirs`. I think the umask value should be:
```
temporary_umask = current_umask | mode_given_to_make_or_verify_dir
```
This should give us consistent behavior around creating files like this on all Python versions.


Discussion:
I just corrected what I think the temporary umask should be in my initial post.
We discussed that on live, but wanted a confirmation: does it make sense to implement this fixing logic on `filesystem.makedirs` instead of `certbot.util.make_or_verify_dir`. The benefit I see is that it would make all makedirs operations consistent in all Certbot codebase. The drawback is that it does not follow anymore the logic of the local Python distribution on dirs that do not need a specific security strategy.
I'm personally a little torn, initially favored doing it in `certbot.util.make_or_verify_dir` to avoid deviating from the Python standard library on UNIX systems, and I verified that approach is safe with all of our current uses of `filesystem.makedirs`.

With that said, I could see not fixing `filesystem.makedirs` introducing bugs in the future and it is the behavior of the standard library in Python < 3.7 so fixing it there is probably for the best.

Thanks for asking about this.

Resolving patch:
diff --git a/certbot/certbot/compat/filesystem.py b/certbot/certbot/compat/filesystem.py
--- a/certbot/certbot/compat/filesystem.py
+++ b/certbot/certbot/compat/filesystem.py
@@ -237,8 +237,19 @@ def makedirs(file_path, mode=0o777):
                      will be applied if ``None``
     """
     if POSIX_MODE:
-        return os.makedirs(file_path, mode)
+        # Since Python 3.7, os.makedirs does not set the given mode to the intermediate directories
+        # that could be created in the process. To keep things safe and consistent on all
+        # Python versions, we set the umask accordingly to have all directories (intermediate and
+        # leaf) created with the given mode.
+        current_umask = os.umask(0)
+        try:
+            os.umask(current_umask | 0o777 ^ mode)
+            return os.makedirs(file_path, mode)
+        finally:
+            os.umask(current_umask)
 
+    # TODO: Windows does not support umask. A specific PR (#7967) is handling this, and will need
+    #       to add appropriate umask call for the Windows part of the logic below.
     orig_mkdir_fn = os.mkdir
     try:
         # As we know that os.mkdir is called internally by os.makedirs, we will swap the function in


Test patch:
diff --git a/certbot/tests/compat/filesystem_test.py b/certbot/tests/compat/filesystem_test.py
--- a/certbot/tests/compat/filesystem_test.py
+++ b/certbot/tests/compat/filesystem_test.py
@@ -1,6 +1,7 @@
 """Tests for certbot.compat.filesystem"""
 import contextlib
 import errno
+import stat
 import unittest
 
 try:
@@ -280,6 +281,26 @@ def test_makedirs_switch_os_mkdir(self):
         self.assertEqual(original_mkdir, std_os.mkdir)
 
 
+# TODO: This test can be used both by Linux and Windows once on #7967
+@unittest.skipUnless(POSIX_MODE, reason='Needs umask to succeed, and Windows does not have it')
+class LinuxMkdirTests(test_util.TempDirTestCase):
+    """Unit tests for Linux mkdir + makedirs functions in filesystem module"""
+    def test_makedirs_correct_permissions(self):
+        path = os.path.join(self.tempdir, 'dir')
+        subpath = os.path.join(path, 'subpath')
+
+        previous_umask = os.umask(0o022)
+
+        try:
+            filesystem.makedirs(subpath, 0o700)
+
+            import os as std_os  # pylint: disable=os-module-forbidden
+            assert stat.S_IMODE(std_os.stat(path).st_mode) == 0o700
+            assert stat.S_IMODE(std_os.stat(subpath).st_mode) == 0o700
+        finally:
+            os.umask(previous_umask)
+
+
 class CopyOwnershipAndModeTest(test_util.TempDirTestCase):
     """Tests about copy_ownership_and_apply_mode, copy_ownership_and_mode and has_same_ownership"""
     def setUp(self):