Repository: certbot/certbot

Issue:
CSR's SAN field not following the user's desired order when requesting interactively, due to _scrub_checklist_input not preversing order
When a user requests a cert for multiple domains from the interactive interface, the resulting cert's common name is not always the first item from the user's input. It turns out that the CSR's SAN is not listed in the order specified by the user. I have a proposed change for the method `_scrub_checklist_input` to fix this.

I was requesting a cert for the 4th and 9th items listed by `certbot -i nginx`. Let's say my 4th item was `example.org` and the 9th item was `www.example.org`, the SAN in the CSR turned out to be `DNS:www.example.org, DNS:example.org`, when I really wanted `DNS:example.org, DNS:www.example.org`. Although the resulting cert worked perfectly, it had two minor issues: the CN of the resulting request was `www.example.org`, and the name of the directory used to store the certs was also `www.example.org`. That was somewhat surprising to me since I explicitly wanted `example.org` to be the first item (and therefore chosen as CN and the directory name).

Since certbot did not have the same problem if the interactively-chosen items were `1,2` or `2,3,4`, it puzzled me why the sequence `4,9` gave me a different result.

After looking into the code, I found that the tags returned by [the display module's `_scrub_checklist_input` method](https://github.com/certbot/certbot/blob/443ec2200f8c4eaf2cf28623392d4a7588f0ed44/certbot/certbot/_internal/display/obj.py#L332) did not preserve the order of the requested indices. It happens to work most of the time, until it doesn't:

```python
>>> list(set([1,2]))    # ok
[1, 2]
>>> list(set([2,3,4]))  # ok
[2, 3, 4]
>>> list(set([4,9]))    # not so: wanted [4, 9]
[9, 4]
```

[This discussion thread](https://community.letsencrypt.org/t/domain-ordering-not-respected-unexpected-certificate-subject/195337/23) mentioned how the CN is picked up from a CSR's SAN, and that's how I realized that the issue I ran into had something to do with how the certbot client generated the CSR for me.

If the user's input is `example.org` then `www.example.org`, I thought it would be best for the CSR's SAN to be listed in that order.

It it helps, I'm more than happy to make a PR. I have [a commit in my fork](https://github.com/lukhnos/certbot/commit/d6be2cecc6d46d86d74eb1f84fc696e9b02ca4a7) that has a new test case that demonstrates the issue as well as a change to `certbot/certbot/_internal/display/obj.py` from deduping with `set` to deduping with `dict`, since dictionaries preserve insertion order as of Python 3.7.

Thank you.




Discussion:

Resolving patch:
diff --git a/certbot/certbot/_internal/display/obj.py b/certbot/certbot/_internal/display/obj.py
--- a/certbot/certbot/_internal/display/obj.py
+++ b/certbot/certbot/_internal/display/obj.py
@@ -328,8 +328,9 @@ def _scrub_checklist_input(self, indices: Iterable[Union[str, int]],
         except ValueError:
             return []
 
-        # Remove duplicates
-        indices_int = list(set(indices_int))
+        # Remove duplicates. dict is used instead of set, since dict perserves
+        # insertion order as of Python 3.7
+        indices_int = list(dict.fromkeys(indices_int).keys())
 
         # Check all input is within range
         for index in indices_int:


Test patch:
diff --git a/certbot/certbot/_internal/tests/display/obj_test.py b/certbot/certbot/_internal/tests/display/obj_test.py
--- a/certbot/certbot/_internal/tests/display/obj_test.py
+++ b/certbot/certbot/_internal/tests/display/obj_test.py
@@ -194,6 +194,25 @@ def test_scrub_checklist_input_valid(self):
                 self.displayer._scrub_checklist_input(list_, TAGS))
             assert set_tags == exp[i]
 
+    def test_scrub_checklist_maintain_indices_order(self):
+        # pylint: disable=protected-access
+        source_tags = ["T1", "T2", "T3", "T4", "T5", "T6", "T7", "T8", "T9"]
+        indices = [
+            ["4", "9"],
+            ["9", "4"],
+            ["4", "9", "4"],
+            ["9", "4", "9"],
+        ]
+        exp = [
+            ["T4", "T9"],
+            ["T9", "T4"],
+            ["T4", "T9"],
+            ["T9", "T4"],
+        ]
+        for i, list_ in enumerate(indices):
+            tags = self.displayer._scrub_checklist_input(list_, source_tags)
+            assert tags == exp[i]
+
     @mock.patch("certbot._internal.display.util.input_with_timeout")
     def test_directory_select(self, mock_input):
         args = ["msg", "/var/www/html", "--flag", True]