Repository: certbot/certbot

Issue:
apache authenticator should authenticate all duplicate vhosts
See https://community.letsencrypt.org/t/cant-renew-certificate/153274/3?u=_az. I've seen this a few other times as well.

The Apache authenticator needs to authenticate all of the virtual hosts which could match the requested domain, including duplicates and HTTPS ones.

#6729 fixed this for the most part, but there are seemingly still some edge cases left over.

Let's investigate and fix those.


Discussion:
Another one today https://community.letsencrypt.org/t/renewal-on-a-roundcube-config/153393/5?u=_az.

So I think what's happening is that `ServerName`, omitted in the `000-default.conf` file on Debian, is being given a runtime fallback value based on:

1. `getnameinfo` for the address of the machine's `hostname`, if it contains a `.`, or
2. the machine's `hostname` if it contains a `.`, or
3. the numeric string format of the bind address

If this dynamically determined `ServerName` happens to overlap with the requested certificate, there's a good chance that authorization will fail, because the authenticator *should have*, but *didn't*, modify this vhost.

I think we should fix this if we can because it is happening fairly regularly.

I'm not sure whether it's viable to detect (1) because that detection may generate DNS queries, which is probably not a good thing to do in the authenticator.

But, I think we could reasonably detect (2). What I'm thinking is that we can detect if `chall.domain` matches the server's hostname, and if so, include any nameless vhosts in the vhosts we modify:

https://github.com/certbot/certbot/blob/78261dbae2571d9a52d6fee7ffc8802f1104aec9/certbot-apache/certbot_apache/_internal/http_01.py#L95-L101

Doing so looks to be pretty free. I also believe it would have prevented both of the support threads I've linked so far in this thread.
What if we just always modified any nameless vhosts that are listening on the right port to respond to the challenge? This seems simpler and would cover all cases, however, there may be downsides or problems with this approach that I'm not thinking of.
Thanks. 

>What if we just always modified any nameless vhosts that are listening on the right port to respond to the challenge

Now that I read the authenticator more closely, it seems that this type of eager coverage is something the authenticator already does. When no vhosts match by name, we are scooping up every vhost on that port and configuring it (`_relevant_vhosts`).

So now I feel pretty good about extending that eagerness to the initial case: if we did match some vhosts by name, let's tack on all the nameless ones too. Otherwise, fallthrough to `_relevant_vhosts` as before. 

Resolving patch:
diff --git a/certbot-apache/certbot_apache/_internal/http_01.py b/certbot-apache/certbot_apache/_internal/http_01.py
--- a/certbot-apache/certbot_apache/_internal/http_01.py
+++ b/certbot-apache/certbot_apache/_internal/http_01.py
@@ -95,10 +95,10 @@ def prepare_http01_modules(self):
     def _mod_config(self):
         selected_vhosts: List[VirtualHost] = []
         http_port = str(self.configurator.config.http01_port)
+
+        # Search for VirtualHosts matching by name
         for chall in self.achalls:
-            # Search for matching VirtualHosts
-            for vh in self._matching_vhosts(chall.domain):
-                selected_vhosts.append(vh)
+            selected_vhosts += self._matching_vhosts(chall.domain)
 
         # Ensure that we have one or more VirtualHosts that we can continue
         # with. (one that listens to port configured with --http-01-port)
@@ -107,9 +107,13 @@ def _mod_config(self):
             if any(a.is_wildcard() or a.get_port() == http_port for a in vhost.addrs):
                 found = True
 
-        if not found:
-            for vh in self._relevant_vhosts():
-                selected_vhosts.append(vh)
+        # If there's at least one elgible VirtualHost, also add all unnamed VirtualHosts
+        # because they might match at runtime (#8890)
+        if found:
+            selected_vhosts += self._unnamed_vhosts()
+        # Otherwise, add every Virtualhost which listens on the right port
+        else:
+            selected_vhosts += self._relevant_vhosts()
 
         # Add the challenge configuration
         for vh in selected_vhosts:
@@ -167,6 +171,10 @@ def _relevant_vhosts(self):
 
         return relevant_vhosts
 
+    def _unnamed_vhosts(self) -> List[VirtualHost]:
+        """Return all VirtualHost objects with no ServerName"""
+        return [vh for vh in self.configurator.vhosts if vh.name is None]
+
     def _set_up_challenges(self):
         if not os.path.isdir(self.challenge_dir):
             old_umask = filesystem.umask(0o022)


Test patch:
diff --git a/certbot-apache/tests/http_01_test.py b/certbot-apache/tests/http_01_test.py
--- a/certbot-apache/tests/http_01_test.py
+++ b/certbot-apache/tests/http_01_test.py
@@ -125,6 +125,18 @@ def test_configure_multiple_vhosts(self):
                 domain="duplicate.example.com", account_key=self.account_key)]
         self.common_perform_test(achalls, vhosts)
 
+    def test_configure_name_and_blank(self):
+        domain = "certbot.demo"
+        vhosts = [v for v in self.config.vhosts if v.name == domain or v.name is None]
+        achalls = [
+            achallenges.KeyAuthorizationAnnotatedChallenge(
+                challb=acme_util.chall_to_challb(
+                    challenges.HTTP01(token=((b'a' * 16))),
+                    "pending"),
+                domain=domain, account_key=self.account_key),
+        ]
+        self.common_perform_test(achalls, vhosts)
+
     def test_no_vhost(self):
         for achall in self.achalls:
             self.http.add_chall(achall)