Repository: certbot/certbot

Issue:
Bug in FQDN detection: installer wrongly interprets _ 
## Problem

I'm attempting to get a certificate for a new domain with a following format:

`wkoszek_nc2.x.mycompany.com`

However the client seems to get confused about `_`. Upon removing the `_` from the domain name, it works fine. For example, this works OK:

`wkoszeknc2.x.mycompany.com`
## To reproduce the problem:

Use this command to reproduce:

`# ./certbot-auto certonly --standalone --email wkoszek@mycompany.com -d wkoszek_nc2.x.mycompany.com
`

I'm getting this:

```
Checking for new version...
Requesting root privileges to run certbot...
   /root/.local/share/letsencrypt/bin/letsencrypt certonly --standalone --email wkoszek@mycompany.com -d wkoszek_nc2.x.mycompany.com
Requested domain wkoszek_nc2.x.mycompany.com is not a FQDN
```

Installer thinks bare TLD is not a valid FQDN
If you attempt to enter either "[ai](http://ai.)" or "[ai.](http://ai.)" (same with [to](http://to.)), the installer complains about it not being a valid FQDN. It is a valid FQDN, and the installer should allow it to proceed with validation. Oddly enough, the installer _does_ treat `user@tld` properly for email entry.

Other TLDs such as IO used to also have websites hosted directly on their TLD. [ai](http://ai.) and [to](http://to.) are just two such examples of websites being hosted directly on a TLD.



Discussion:
Relevant code: https://github.com/certbot/certbot/blob/master/certbot/le_util.py#L343

The regex was taken from here: http://www.mkyong.com/regular-expressions/domain-name-regular-expression-example/

While fixing this it would also be nice to handle #3056 

Fun things I learned while digging in to this FQDN's _can_ have underscores, but hostnames _cannot_. While the label used to identify the host in some network is usually the hostname, it doesn't have to be. http://stackoverflow.com/a/2183140/1609380

Do we even have to have this check at all? I don't understand why that would be necessary.

If you can reach the domain, it's fine. If not - you would get a different error warning from a networking code, which would find a real problem: inability to communicate.

Not sure about `_`, since Bonjour has some underscores used for its domain names: https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/NetServices/Articles/domainnames.html

@wkoszek I'm not sure what we have to check and why.

The only restriction I can find on a fqdn is that each label be between 1 and 63 octets. And that the full domain name be less than 255 octets. Your are allowed to have "any binary label" in that space.

I cannot find extra restrictions looking through the [DANE rfc](https://tools.ietf.org/html/rfc6698), or the [ca policy rfc](https://tools.ietf.org/html/rfc3647). But those are large documents and I'm probably missing something.

However we currently disallow a few things seemingly arbitrarily, like non-ascii unicode. changing this may break downstream code.

For now I can change it to allow underscores.



Resolving patch:
diff --git a/certbot/util.py b/certbot/util.py
--- a/certbot/util.py
+++ b/certbot/util.py
@@ -423,14 +423,17 @@ def enforce_domain_sanity(domain):
         # It wasn't an IP address, so that's good
         pass
 
-    # FQDN checks from
-    # http://www.mkyong.com/regular-expressions/domain-name-regular-expression-example/
-    #  Characters used, domain parts < 63 chars, tld > 1 < 64 chars
-    #  first and last char is not "-"
-    fqdn = re.compile("^((?!-)[A-Za-z0-9-]{1,63}(?<!-)\\.)+[A-Za-z]{2,63}$")
-    if not fqdn.match(domain):
-        raise errors.ConfigurationError("Requested domain {0} is not a FQDN"
-                                        .format(domain))
+    # FQDN checks according to RFC 2181: domain name should be less than 255
+    # octets (inclusive). And each label is 1 - 63 octets (inclusive).
+    # https://tools.ietf.org/html/rfc2181#section-11
+    msg = "Requested domain {0} is not a FQDN because ".format(domain)
+    labels = domain.split('.')
+    for l in labels:
+        if not 0 < len(l) < 64:
+            raise errors.ConfigurationError(msg + "label {0} is too long.".format(l))
+    if len(domain) > 255:
+        raise errors.ConfigurationError(msg + "it is too long.")
+
     return domain
 
 


Test patch:
diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py
--- a/certbot/tests/cli_test.py
+++ b/certbot/tests/cli_test.py
@@ -342,11 +342,11 @@ def test_check_config_sanity_domain(self):
         # FQDN
         self.assertRaises(errors.ConfigurationError,
                           self._call,
-                          ['-d', 'comma,gotwrong.tld'])
+                          ['-d', 'a' * 64])
         # FQDN 2
         self.assertRaises(errors.ConfigurationError,
                           self._call,
-                          ['-d', 'illegal.character=.tld'])
+                          ['-d', (('a' * 50) + '.') * 10])
         # Wildcard
         self.assertRaises(errors.ConfigurationError,
                           self._call,
diff --git a/certbot/tests/display/ops_test.py b/certbot/tests/display/ops_test.py
--- a/certbot/tests/display/ops_test.py
+++ b/certbot/tests/display/ops_test.py
@@ -248,9 +248,9 @@ def test_filter_names_cancel(self, mock_util):
     def test_get_valid_domains(self):
         from certbot.display.ops import get_valid_domains
         all_valid = ["example.com", "second.example.com",
-                     "also.example.com"]
-        all_invalid = ["xn--ls8h.tld", "*.wildcard.com", "notFQDN",
-                       "uniçodé.com"]
+                     "also.example.com", "under_score.example.com",
+                     "justtld"]
+        all_invalid = ["xn--ls8h.tld", "*.wildcard.com", "uniçodé.com"]
         two_valid = ["example.com", "xn--ls8h.tld", "also.example.com"]
         self.assertEqual(get_valid_domains(all_valid), all_valid)
         self.assertEqual(get_valid_domains(all_invalid), [])
@@ -276,19 +276,18 @@ def test_choose_manually(self, mock_util):
         mock_util().input.return_value = (display_util.OK,
                                           "xn--ls8h.tld")
         self.assertEqual(_choose_names_manually(), [])
-        # non-FQDN and no retry
-        mock_util().input.return_value = (display_util.OK,
-                                          "notFQDN")
-        self.assertEqual(_choose_names_manually(), [])
-        # Two valid domains
+        # Valid domains
         mock_util().input.return_value = (display_util.OK,
                                           ("example.com,"
+                                           "under_score.example.com,"
+                                           "justtld,"
                                            "valid.example.com"))
         self.assertEqual(_choose_names_manually(),
-                         ["example.com", "valid.example.com"])
+                         ["example.com", "under_score.example.com",
+                          "justtld", "valid.example.com"])
         # Three iterations
         mock_util().input.return_value = (display_util.OK,
-                                          "notFQDN")
+                                          "uniçodé.com")
         yn = mock.MagicMock()
         yn.side_effect = [True, True, False]
         mock_util().yesno = yn