Change:
assert keyhash_old == keyhash_new with subtest("Can request certificates for vhost + aliases (apache-httpd)"): <del> switch_to(webserver, "httpd-aliases") webserver.wait_for_unit("acme-finished-c.example.test.target") </del> <ins> try: switch_to(webserver, "httpd-aliases") webserver.wait_for_unit("acme-finished-c.example.test.target") except Exception as err: _, output = webserver.execute( "cat /var/log/httpd/*.log && ls -al /var/lib/acme/acme-challenge" ) print(output) raise err </ins> check_issuer(webserver, "c.example.test", "pebble") check_connection(client, "c.example.test") check_connection(client, "d.example.test")

Discussion:
Describe the bug An recent change on release-20.09 broke acme on my system: It's trying to chown a path that's currently owned by root: I track nixpkgs versions in my setup, so the working state is and the broken state is . To Reproduce (Not sure if this is easily reproducible, but...) Have a config like and do an upgrade from NixOS 20.09 @ git commit to . Expected behavior That services that used to work don't get permission error and fail on the stable branch. Additional context I rolled back to the known good version, that failed now(!), I rolled forward again and manually fixed up the permissions. Now I'm getting this error: And later: I manually restarted the acme service. Now everything seems fine. Notify maintainers <!-- Please @ people who are in the list of the offending package or module. If in doubt, check for whoever last touched something. --CC CC due to their acme commits on release-20.09. Metadata Maintainer information:
seems to have removed a log message that I would have found helpful here: That changes the error from a "this module is broken" to "this module is guiding me to do a manual data migration". That's a big deal. Luckily did add , which was very helpful.
I'm glad you found the old error message - that is indeed the issue here. .well-known + .well-known/acme-challenge need to be owned by acme + the configured group. I will add back in the echo in my next PR. Thanks for pointing that out.
In a way it's weird that the code tries to do as non-root user, because it'll always fail if the ownership is not already correct. But maybe the point is just to verify the ownership? Thanks, adding back that message will help. Maybe even print the command the user should run ( )?
Verifying the owner is correct, but setting the group is the main intention since it can be changed between rebuilds (and will work in normal circumstances). I guess would be clearer here now that I think about it, and would still raise and appropriate error if the owner is not acme. I'll add it to the list Thanks!
From an end-user perspective, I wonder why the data migration isn't happening automatically.

Why cannot NixOS run the needed command as root? Is there risk of data loss?
I stumbled upon this today. I'm slightly confused: Why does the module need ownership of ? Shouldn't ownership of be enough? I had the ownership of the acme-challenge directory correct, but .well-known was owned by root, so it broke for me. But my understanding of the challenge is that it should only write/delete files in the acme-challenge subdirectory.
The reason .well-known is also chown'ed is that lego (the ACME client that the module uses) tries to create acme-challenge every time, and will throw a permissions error before checking if it exists. I thought I had a copy of the error in comments somewhere but I can't find it now, so I will test it once again to make sure. As for a more permanent fix, I think I will move these ownership fixes into the acme-fixperms.service which runs as root.
After some testing and some , I found that I don't actually need to fix .well-known any more. I'm almost certain this wasn't the case at one point, but alas removing it will resolve issues here. I have stripped the problematic part of the renewal script down to its bare minimum - it now only runs the following commands: The chgrp is required as the configured group can be changed between runs. Arguably this could be removed too if you assume that acme will always own this folder, and the changes I've made . (Set UMask to 0022 for the service) make it world readable. In an abundance of caution, I am going to keep it around for now.
I completely wiped my directory but starting acme fails with: This is on 20.09. does not exist yet when this is run. Is this the same issue in a dfiferent flavor? i don't have any acme related state on the machine and this is the entire config:
Let's leave this open until the fix is backported to release-20.09.
Sorry it took me so long... busy week. PR is open. If someone like could give it a test and see if it fixes their issues that would be great.
Backports merged, closing issue.

Describe the bug The subtests for apache httpd in fail: : Removing these subtests makes the test pass: To Reproduce Steps to reproduce the behavior: Expected behavior Test success. Screenshots Additional context Notify maintainers Metadata Please run and paste the result. Maintainer information:
The cause is that the permissions on the configured webroot directory are wrong when switching from Nginx to httpd. Taking from a debug statement: This subsequently causes 403 errors in httpd with the following error in the log: There are 3 solutions, sorted by my own preference to implement: Change the UMask to 0022 (currently 0023) so that any group can read the folder. Fix the permissions in the acme module when renewals are run. Fix the permissions in the test suite. Keeping in mind, I feel that changing the UMask would be the best option that would "Just work" without affecting the potentially sensitive group permissions on the webroot. Also - it's a web directory which will effectively be readable by anyone (AKA anyone that visits the page) anyway. I'm not in favour of the last option as I feel fixing it for all users is important, and I would rather have this thread + take the flak for somewhat open permissions than break people's setups entirely.
That seems to align with the purpose of the challenge directory. If you change the umask, make sure it doesn't affect other files and directories.
I've some tests for the resulting file's permissions. Also I would like to highlight the work did a while back in to determine where/what permissions lego applies to files. 0023 was a bit stingy on my part from the beginning. It should've stayed as 0022 all along ;P