Repository: certbot/certbot

Issue:
Webroot breaks for non-root due to chown
Hi,

Today I'm trying out letsencrypt and stumbled upon that using webroot breaks when not using root due to a chown on line 72 of letsencrypt/plugins/webroot.py:

```
 49     def prepare(self):  # pylint: disable=missing-docstring
 50         path_map = self.conf("map")
 51
 52         if not path_map:
 53             raise errors.PluginError("--{0} must be set".format(
 54                 self.option_name("path")))
 55         for name, path in path_map.items():
 56             if not os.path.isdir(path):
 57                 raise errors.PluginError(path + " does not exist or is not a directory")
 58             self.full_roots[name] = os.path.join(path, challenges.HTTP01.URI_ROOT_PATH)
 59
 60             logger.debug("Creating root challenges validation dir at %s",
 61                          self.full_roots[name])
 62             try:
 63                 os.makedirs(self.full_roots[name])
 64                 # Set permissions as parent directory (GH #1389)
 65                 # We don't use the parameters in makedirs because it
 66                 # may not always work
 67                 # https://stackoverflow.com/questions/5231901/permission-problems-when-creating-a-dir-with-os-makedirs-python
 68                 stat_path = os.stat(path)
 69                 filemode = stat.S_IMODE(stat_path.st_mode)
 70                 os.chmod(self.full_roots[name], filemode)
 71                 # Set owner and group, too
 72                 os.chown(self.full_roots[name], stat_path.st_uid,
 73                           stat_path.st_gid)
 74
 75             except OSError as exception:
 76                 if exception.errno != errno.EEXIST:
 77                     raise errors.PluginError(
 78                         "Couldn't create root for {0} http-01 "
 79                         "challenge responses: {1}", name, exception)
```

It tries to set the ownership of the directory to the owner of the parent. So currently to bypass this issue either the webroot must be chown'd to the letsencrypt user, or the script needs to be run as root. Neither of which are satisfactory solutions. Is chown at all neccesary when letsencrypt is not run as root?



Discussion:
Probably the right answer is that `chown` should be wrapped in a try: except: pass branch.  @lbeltrame, are you interested in taking this?

That's really sad news :( In the meantime you can use https://github.com/kuba/simp_le - it's a client for webroot only, without any root requirements.

Il 06 dicembre 2015 22:00:18 CET, Peter Eckersley notifications@github.com ha scritto:

> Probably the right answer is that `chown` should be wrapped in a try:
> except: pass branch.  @lbeltrame, are you interested in taking this?

I can take a look tomorrow hopefully (yay for public holidays).

## 

Luca Beltrame
GPG key ID: 6E1A4E79

More correct is probably to just create the directories with the right owner (by temporarily changing the uid and gid if root) and permissions (by using os.makedirs' mode option with optionally temporarily changing the umask as well as explained in the linked discussion on stackoverflow).

Otherwise one easily gets race conditions if other users have write permissions to the webroot or a parent directory.  Using `fchown` and `fchmod` instead of `chown` and `chmod` is also a good idea.

Also os.makedirs can create multiple directories, but it looks like the permissions are only changed for the last one?

The easier solution for `_perform_single` (which also uses chmod/chown) is probably to
- Create the file writable-by-owner & world-readable (w/o any chmod call; this might require changing the umask temporarily).
- Do not care about the owner: users with write access to the parent directory can remove it anyway and there is not much interesting to do besides that.

In my opinion, I would not change the umask, I'm expecting whoever set the directory to have made it with correct permissions: the client should only be concerned with the files and directory it creates itself (unless that's what you mean). 

``` python

with open(path, "w") as validation_file:
    old_umask = os.umask(022)
    validation_file.write(validation.encode())
    os.umask(old_umask)
```

Would something of the sort suffice for `_perform_single`?

@aburch How would you define the "right" owner for a directory? The UID of the process?

FYI, I am trying to implement the suggestions, but I want to make sure I get things right, hence the questions. 

Luca Beltrame notifications@github.com writes:

> ``` python
> 
> with open(path, "w") as validation_file:
>             old_umask = os.umask(022)
>             validation_file.write(validation.encode())
>             os.umask(old_umask)
> ```
> 
> Would something of the sort suffice for `_perform_single`?

No, you need to change the umask before creating the file. Something
like

``` python

old_umask = os.umask(0o022)
try:
    with open(path, "wx") as validation_file:
        validation_file.write(validation.encode())
finally:
    os.umask(old_umask)
```

This also fails if the file already exists ("x" in open, needs
Python3.3+) and reverts the umask even if an exception is raised.

Note: I didn't test the code snippet above.

Ansgar

Luca Beltrame notifications@github.com writes:

> @aburch How would you define the "right" owner for a directory? The UID of the process?

Hmm, that's not easy to answer.

For myself, I plan to run letsencrypt as a dedicated service user.  So
it can only create files as its own user anyway.  I think that is also
how most distributions expect services to work.

I understand that you also want to provide a fully magic mode.  This is
always hard to get right; the best approximation is probably to use the
owner of the given webroot (if letsencrypt is started as root).  Maybe
provide a command-line option if this is not the case: for example an
admin could have created .../.well-known/acme-challenges manually and
set appropriate permissions which might not match the one for the
webroot itself.  (Though in this case I would expect the admin to not
run letsencrypt as root anyway.)

In any case, I would recommend avoiding chmod/chown.  For root, it
should be possible to to switch user with `os.seteuid` and `os.setegid`
and revert to the original user later.  This avoids calling `chmod` and
`chown` which are potentially unsafe to use.

Ansgar

This looks a little more complicated than what I expected (FYI, I'm not hacking on letsencrypt, just fixing this bug which had been preventing myself from using it), I'll try to post a WIP PR and hopefully gather some feedback.

Also, I think `seteuid` and `setegid` will only work on Linux and UNIX-like OSes, while I expect the letsencrypt client to work also on WIndows on the long term (Can someone from LE confirm this?).


Resolving patch:
diff --git a/letsencrypt/plugins/webroot.py b/letsencrypt/plugins/webroot.py
--- a/letsencrypt/plugins/webroot.py
+++ b/letsencrypt/plugins/webroot.py
@@ -2,7 +2,6 @@
 import errno
 import logging
 import os
-import stat
 
 import zope.interface
 
@@ -59,24 +58,38 @@ def prepare(self):  # pylint: disable=missing-docstring
 
             logger.debug("Creating root challenges validation dir at %s",
                          self.full_roots[name])
+
+            # Change the permissions to be writable (GH #1389)
+            # Umask is used instead of chmod to ensure the client can also
+            # run as non-root (GH #1795)
+            old_umask = os.umask(0o022)
+
             try:
-                os.makedirs(self.full_roots[name])
-                # Set permissions as parent directory (GH #1389)
-                # We don't use the parameters in makedirs because it
-                # may not always work
+                # This is coupled with the "umask" call above because
+                # os.makedirs's "mode" parameter may not always work:
                 # https://stackoverflow.com/questions/5231901/permission-problems-when-creating-a-dir-with-os-makedirs-python
-                stat_path = os.stat(path)
-                filemode = stat.S_IMODE(stat_path.st_mode)
-                os.chmod(self.full_roots[name], filemode)
-                # Set owner and group, too
-                os.chown(self.full_roots[name], stat_path.st_uid,
-                          stat_path.st_gid)
+                os.makedirs(self.full_roots[name], 0o0755)
+
+                # Set owner as parent directory if possible
+                try:
+                    stat_path = os.stat(path)
+                    os.chown(self.full_roots[name], stat_path.st_uid,
+                             stat_path.st_gid)
+                except OSError as exception:
+                    if exception.errno == errno.EACCES:
+                        logger.debug("Insufficient permissions to change owner and uid - ignoring")
+                    else:
+                        raise errors.PluginError(
+                            "Couldn't create root for {0} http-01 "
+                            "challenge responses: {1}", name, exception)
 
             except OSError as exception:
                 if exception.errno != errno.EEXIST:
                     raise errors.PluginError(
                         "Couldn't create root for {0} http-01 "
                         "challenge responses: {1}", name, exception)
+            finally:
+                os.umask(old_umask)
 
     def perform(self, achalls):  # pylint: disable=missing-docstring
         assert self.full_roots, "Webroot plugin appears to be missing webroot map"
@@ -87,26 +100,26 @@ def _path_for_achall(self, achall):
             path = self.full_roots[achall.domain]
         except IndexError:
             raise errors.PluginError("Missing --webroot-path for domain: {1}"
-                        .format(achall.domain))
+                                     .format(achall.domain))
         if not os.path.exists(path):
             raise errors.PluginError("Mysteriously missing path {0} for domain: {1}"
-                        .format(path, achall.domain))
+                                     .format(path, achall.domain))
         return os.path.join(path, achall.chall.encode("token"))
 
     def _perform_single(self, achall):
         response, validation = achall.response_and_validation()
+
         path = self._path_for_achall(achall)
         logger.debug("Attempting to save validation to %s", path)
-        with open(path, "w") as validation_file:
-            validation_file.write(validation.encode())
-
-        # Set permissions as parent directory (GH #1389)
-        parent_path = self.full_roots[achall.domain]
-        stat_parent_path = os.stat(parent_path)
-        filemode = stat.S_IMODE(stat_parent_path.st_mode)
-        # Remove execution bit (not needed for this file)
-        os.chmod(path, filemode & ~stat.S_IEXEC)
-        os.chown(path, stat_parent_path.st_uid, stat_parent_path.st_gid)
+
+        # Change permissions to be world-readable, owner-writable (GH #1795)
+        old_umask = os.umask(0o022)
+
+        try:
+            with open(path, "w") as validation_file:
+                validation_file.write(validation.encode())
+        finally:
+            os.umask(old_umask)
 
         return response
 


Test patch:
diff --git a/letsencrypt/plugins/webroot_test.py b/letsencrypt/plugins/webroot_test.py
--- a/letsencrypt/plugins/webroot_test.py
+++ b/letsencrypt/plugins/webroot_test.py
@@ -1,9 +1,10 @@
 """Tests for letsencrypt.plugins.webroot."""
+import errno
 import os
 import shutil
+import stat
 import tempfile
 import unittest
-import stat
 
 import mock
 
@@ -35,7 +36,6 @@ def setUp(self):
         self.config = mock.MagicMock(webroot_path=self.path,
                                      webroot_map={"thing.com": self.path})
         self.auth = Authenticator(self.config, "webroot")
-        self.auth.prepare()
 
     def tearDown(self):
         shutil.rmtree(self.path)
@@ -48,7 +48,7 @@ def test_more_info(self):
     def test_add_parser_arguments(self):
         add = mock.MagicMock()
         self.auth.add_parser_arguments(add)
-        self.assertEqual(0, add.call_count) # became 0 when we moved the args to cli.py!
+        self.assertEqual(0, add.call_count)  # args moved to cli.py!
 
     def test_prepare_bad_root(self):
         self.config.webroot_path = os.path.join(self.path, "null")
@@ -70,17 +70,33 @@ def test_prepare_reraises_other_errors(self):
         self.assertRaises(errors.PluginError, self.auth.prepare)
         os.chmod(self.path, 0o700)
 
+    @mock.patch("letsencrypt.plugins.webroot.os.chown")
+    def test_failed_chown_eacces(self, mock_chown):
+        mock_chown.side_effect = OSError(errno.EACCES, "msg")
+        self.auth.prepare()  # exception caught and logged
+
+    @mock.patch("letsencrypt.plugins.webroot.os.chown")
+    def test_failed_chown_not_eacces(self, mock_chown):
+        mock_chown.side_effect = OSError()
+        self.assertRaises(errors.PluginError, self.auth.prepare)
+
     def test_prepare_permissions(self):
+        self.auth.prepare()
 
         # Remove exec bit from permission check, so that it
         # matches the file
         self.auth.perform([self.achall])
-        parent_permissions = (stat.S_IMODE(os.stat(self.path).st_mode) &
-                              ~stat.S_IEXEC)
+        path_permissions = stat.S_IMODE(os.stat(self.validation_path).st_mode)
+        self.assertEqual(path_permissions, 0o644)
 
-        actual_permissions = stat.S_IMODE(os.stat(self.validation_path).st_mode)
+        # Check permissions of the directories
+
+        for dirpath, dirnames, _ in os.walk(self.path):
+            for directory in dirnames:
+                full_path = os.path.join(dirpath, directory)
+                dir_permissions = stat.S_IMODE(os.stat(full_path).st_mode)
+                self.assertEqual(dir_permissions, 0o755)
 
-        self.assertEqual(parent_permissions, actual_permissions)
         parent_gid = os.stat(self.path).st_gid
         parent_uid = os.stat(self.path).st_uid
 
@@ -88,6 +104,7 @@ def test_prepare_permissions(self):
         self.assertEqual(os.stat(self.validation_path).st_uid, parent_uid)
 
     def test_perform_cleanup(self):
+        self.auth.prepare()
         responses = self.auth.perform([self.achall])
         self.assertEqual(1, len(responses))
         self.assertTrue(os.path.exists(self.validation_path))