Repository: certbot/certbot (Python)

Issue:
CSR's SAN field not following the user's desired order when requesting interactively, due to _scrub_checklist_input not preversing order
When a user requests a cert for multiple domains from the interactive interface, the resulting cert's common name is not always the first item from the user's input. It turns out that the CSR's SAN is not listed in the order specified by the user. I have a proposed change for the method `_scrub_checklist_input` to fix this.

I was requesting a cert for the 4th and 9th items listed by `certbot -i nginx`. Let's say my 4th item was `example.org` and the 9th item was `www.example.org`, the SAN in the CSR turned out to be `DNS:www.example.org, DNS:example.org`, when I really wanted `DNS:example.org, DNS:www.example.org`. Although the resulting cert worked perfectly, it had two minor issues: the CN of the resulting request was `www.example.org`, and the name of the directory used to store the certs was also `www.example.org`. That was somewhat surprising to me since I explicitly wanted `example.org` to be the first item (and therefore chosen as CN and the directory name).

Since certbot did not have the same problem if the interactively-chosen items were `1,2` or `2,3,4`, it puzzled me why the sequence `4,9` gave me a different result.

After looking into the code, I found that the tags returned by [the display module's `_scrub_checklist_input` method](https://github.com/certbot/certbot/blob/443ec2200f8c4eaf2cf28623392d4a7588f0ed44/certbot/certbot/_internal/display/obj.py#L332) did not preserve the order of the requested indices. It happens to work most of the time, until it doesn't:

```python
>>> list(set([1,2]))    # ok
[1, 2]
>>> list(set([2,3,4]))  # ok
[2, 3, 4]
>>> list(set([4,9]))    # not so: wanted [4, 9]
[9, 4]
```

[This discussion thread](https://community.letsencrypt.org/t/domain-ordering-not-respected-unexpected-certificate-subject/195337/23) mentioned how the CN is picked up from a CSR's SAN, and that's how I realized that the issue I ran into had something to do with how the certbot client generated the CSR for me.

If the user's input is `example.org` then `www.example.org`, I thought it would be best for the CSR's SAN to be listed in that order.

It it helps, I'm more than happy to make a PR. I have [a commit in my fork](https://github.com/lukhnos/certbot/commit/d6be2cecc6d46d86d74eb1f84fc696e9b02ca4a7) that has a new test case that demonstrates the issue as well as a change to `certbot/certbot/_internal/display/obj.py` from deduping with `set` to deduping with `dict`, since dictionaries preserve insertion order as of Python 3.7.

Thank you.




Discussion:
Thanks for the in-depth writeup! Your change looks good to me, I'd be happy to review it if you opened a PR.
> Thanks for the in-depth writeup! Your change looks good to me, I'd be happy to review it if you opened a PR.

I appreciate it: https://github.com/certbot/certbot/pull/9944




Patch:
diff --git a/AUTHORS.md b/AUTHORS.md
index d08b0b79f62..78ed21215bb 100644
--- a/AUTHORS.md
+++ b/AUTHORS.md
@@ -166,6 +166,7 @@ Authors
 * [Luca Ebach](https://github.com/lucebac)
 * [Luca Olivetti](https://github.com/olivluca)
 * [Luke Rogers](https://github.com/lukeroge)
+* [Lukhnos Liu](https://github.com/lukhnos)
 * [Maarten](https://github.com/mrtndwrd)
 * [Mads Jensen](https://github.com/atombrella)
 * [Maikel Martens](https://github.com/krukas)
diff --git a/certbot/CHANGELOG.md b/certbot/CHANGELOG.md
index b718aa6c0c4..45e4334bb24 100644
--- a/certbot/CHANGELOG.md
+++ b/certbot/CHANGELOG.md
@@ -15,7 +15,10 @@ Certbot adheres to [Semantic Versioning](https://semver.org/).
 
 ### Fixed
 
-*
+* Fixed a bug in Certbot where a CSR's SANs did not always follow the order of
+  the domain names that the user requested interactively. In some cases, the
+  resulting cert's common name might seem picked up randomly from the SANs
+  when it should be the first item the user had in mind.
 
 More details about these changes can be found on our GitHub repo.
 
diff --git a/certbot/certbot/_internal/display/obj.py b/certbot/certbot/_internal/display/obj.py
index 54c248b0cc9..39882f3fb6d 100644
--- a/certbot/certbot/_internal/display/obj.py
+++ b/certbot/certbot/_internal/display/obj.py
@@ -328,8 +328,9 @@ def _scrub_checklist_input(self, indices: Iterable[Union[str, int]],
         except ValueError:
             return []
 
-        # Remove duplicates
-        indices_int = list(set(indices_int))
+        # Remove duplicates. dict is used instead of set, since dict perserves
+        # insertion order as of Python 3.7
+        indices_int = list(dict.fromkeys(indices_int).keys())
 
         # Check all input is within range
         for index in indices_int:


Test patch:
diff --git a/certbot/certbot/_internal/tests/display/obj_test.py b/certbot/certbot/_internal/tests/display/obj_test.py
index be9bd324233..fa157e6456e 100644
--- a/certbot/certbot/_internal/tests/display/obj_test.py
+++ b/certbot/certbot/_internal/tests/display/obj_test.py
@@ -194,6 +194,25 @@ def test_scrub_checklist_input_valid(self):
                 self.displayer._scrub_checklist_input(list_, TAGS))
             assert set_tags == exp[i]
 
+    def test_scrub_checklist_maintain_indices_order(self):
+        # pylint: disable=protected-access
+        source_tags = ["T1", "T2", "T3", "T4", "T5", "T6", "T7", "T8", "T9"]
+        indices = [
+            ["4", "9"],
+            ["9", "4"],
+            ["4", "9", "4"],
+            ["9", "4", "9"],
+        ]
+        exp = [
+            ["T4", "T9"],
+            ["T9", "T4"],
+            ["T4", "T9"],
+            ["T9", "T4"],
+        ]
+        for i, list_ in enumerate(indices):
+            tags = self.displayer._scrub_checklist_input(list_, source_tags)
+            assert tags == exp[i]
+
     @mock.patch("certbot._internal.display.util.input_with_timeout")
     def test_directory_select(self, mock_input):
         args = ["msg", "/var/www/html", "--flag", True]