Repository: certbot/certbot

Issue:
Make nginx a first class plugin!



Discussion:
It may be ready, but it will take work and testing that we shouldn't be doing in this release.
So, oddly enough, we will probably have to make nginx a second-class plugin to make it a first-class plugin.

That is, certbot-auto currently auto-installs nginx and apache. We need it to not do that, because then we change the workflow of `run` from automatically choosing apache to asking the user to interactively select between apache and nginx.

So, doing this will require:

- delete `hidden = True`
- update website instructions to recommend plugin
- change from Alpha to Beta
- have certbot-auto not install nginx
\* have certbot-auto not install nginx, unless the `--nginx` flag is passed.
> then we change the workflow of run from automatically choosing apache to asking the user to interactively select between apache and nginx

Why is adding a prompt unacceptable? (Is there a reason this backwards compatibility must be maintained?)

It seems like the critical use case for preserving backwards compatibility would be renewal, but in the case of renewal, wouldn't the renewal conf file have a reference to the apache plugin, allowing the prompt to be avoided?
@ohemorange so if I'm following you correctly, the case we're most worried about is users who:

1. Have both apache and nginx installed on a server
2. Are running some kind of renewal script that is currently non-interactive, but doesn't use `renew` or `-n`
3. Are using the Apache plugin successfully, but without providing `--apache`, and instead are having that plugin selected for them as a default?
@zjs: both `certonly` and `run` work perfectly well for renewal, and people use those in their crontabs. if you do it that way, it doesn't read from the renewal conf file and instead uses only command line arguments.
@pde 

1. Do you mean the servers, or the plugins? Because the servers don't matter, and the question is if we should be installing the nginx plugin in one case, and in another case yes, they've already installed both plugins.
2. Yup! This should probably only be `run` / no subcommand.
3. Yup!
> @zjs: both certonly and run work perfectly well for renewal, and people use those in their crontabs. if you do it that way, it doesn't read from the renewal conf file and instead uses only command line arguments.

I see. That... complicates things.
Based on in-person discussions, we decided to split this into a slower rollout. Right now, we can do the more straightforward step of updating our documentation to say we support the Nginx plugin.
https://docs.google.com/spreadsheets/d/1cudoHzxjl1TEUbV7OCDW1bXcivzLnS2zGzqEnehJFq0/edit?usp=sharing
As noted in the spreadsheet above, different install methods lead to different versions of certbot. Which should we tell people to use nginx for? I'm leaning towards 0.13.0, personally.
(Questions from IRC, in case Thunderbird eats the logs:)

- It looks like our packages are pulled from the same source for CentOS 7 and RHEL 7. Is this accurate?
- Also, it looks like OpenBSD and FreeBSD both don't package either apache or nginx plugins. Is this accurate?
- Also, how do we contact our maintainers of certbot apache packages to get them to add nginx packages for: gentoo, fedora 23+, centOS/RHEL 7?
- Also, can older versions of Ubuntu not use ppas? Why do we say to use certbot-auto for ubuntu (other)?
> It looks like our packages are pulled from the same source for CentOS 7 and RHEL 7. Is this accurate?

Yes. They come from [EPEL](https://fedoraproject.org/wiki/EPEL) 7 which is made by Fedora. How you enable the EPEL repository on those two systems is a bit different though.

> Also, it looks like OpenBSD and FreeBSD both don't package either apache or nginx plugins. Is this accurate?

That looks accurate. If you search the FreeBSD site for Certbot packages, you get [this](https://www.freebsd.org/cgi/ports.cgi?query=certbot&stype=all). If you install Certbot on FreeBSD you get this message:
```
The letsencrypt plugins to support apache and nginx certificate installation
will be made available soon in the following ports:

 * Apache plugin: security/py-certbot-apache
 * Nginx plugin: security/py-certbot-nginx
```
Neither of the ports are available at this time though.

I haven't played with Certbot on OpenBSD, but looking around [their mirrors](https://www.openbsd.org/ftp.html) I could only find a `certbot` package which I confirmed doesn't include the Apache or Nginx plugins.

> Also, how do we contact our maintainers of certbot apache packages to get them to add nginx packages for: gentoo, fedora 23+, centOS/RHEL 7?

I e-mailed you their contact info.

> Also, can older versions of Ubuntu not use ppas? Why do we say to use certbot-auto for ubuntu (other)?

Correct. Our PPA contains packages for:

* Ubuntu 14.04 Trusty
* Ubuntu 16.04 Xenial
* Ubuntu 16.10 Yakkety

At this time, the only other version of Ubuntu that is still supported by Canonical is Ubuntu 12.04 but that is EOL'd this month. Ubuntu 17.04 comes out this month as well and we have no packages built for this version yet.
Thanks! Another question: What's the normal schedule for Ubuntu versions to get updated?
The normal schedule is packages are only updated in subsequent releases of Ubuntu. There is a [stable release updates](https://wiki.ubuntu.com/StableReleaseUpdates) process where you can request an update in certain cases. We're currently going through this to get newer versions in 16.04 and 16.10 and this is blocked on adding a lockfile because the package will start shipping with a cronjob when there wasn't one before.
Oh I actually meant when are packages updated in sid, so that the ppas get them?
As in, what's the relationship of experimental, sid, and stretch?
Oh. To provide slightly more context, Debian's primary focus is always stable. They're trying to maintain the current stable release and build the next one. The next stable release is Stretch which is currently frozen and not accepting updates. Packages can be uploaded to sid at any time, but:

1. Uploads to sid often slow down when Debian testing (named Stretch in this case) is frozen due to the aforementioned focus on stable.
2. Harlan had some technical issues on his end preventing him from uploading packages for a bit. I'm not sure the status of those issues.

Experimental isn't a full distribution and serves as a staging ground for packages sometimes. It can basically be ignored for Certbot's purposes.
Oh! A lot of the conversation I've heard but didn't understand make a lot more sense now, thanks. (Also oops, realized I meant Debian above, not Ubuntu.)

Ok so given that there might be a delay before newer versions get into sid (and therefore into the ppas), there's a question now of do we expose Nginx for people who have 0.11.1, or wait for it to get updated before telling people to use it? I'm leaning towards the latter, because of the thought of how many issues will get opened with bugs that have already been fixed, and (relatedly) how many people it just won't work for.
I'm also leaning towards the latter for the reasons you mentioned as well as continuing our slower rollout of Nginx rather than telling the whole world about it in a single update.
Realized I missed one: is there a process for getting updated in jessie-backports?
We should check with @hlieberman but I think that may be a hard one. My understanding is `jessie-backports` can't contain anything newer than what's in Stretch which is frozen and according to Debian policy will only receive security/bug fixes for 5+ years. Peter's plan is to try and get them to do an update for us every 3 - 6 months though which they haven't agreed to yet.
Note: there's currently a message under Advanced about installing the nginx plugin from master, if you really want to. To keep it around, the code would have to get icky, and plus we should maybe not be explicitly telling people to do something like that right in the instructions.

Resolving patch:
diff --git a/certbot-nginx/certbot_nginx/configurator.py b/certbot-nginx/certbot_nginx/configurator.py
--- a/certbot-nginx/certbot_nginx/configurator.py
+++ b/certbot-nginx/certbot_nginx/configurator.py
@@ -87,8 +87,6 @@ class NginxConfigurator(common.Plugin):
 
     description = "Nginx Web Server plugin - Alpha"
 
-    hidden = True
-
     DEFAULT_LISTEN_PORT = '80'
 
     @classmethod
diff --git a/certbot/plugins/selection.py b/certbot/plugins/selection.py
--- a/certbot/plugins/selection.py
+++ b/certbot/plugins/selection.py
@@ -108,11 +108,19 @@ def choose_plugin(prepared, question):
     opts = [plugin_ep.description_with_name +
             (" [Misconfigured]" if plugin_ep.misconfigured else "")
             for plugin_ep in prepared]
+    names = set(plugin_ep.name for plugin_ep in prepared)
 
     while True:
         disp = z_util(interfaces.IDisplay)
-        code, index = disp.menu(
-            question, opts, force_interactive=True)
+        if "CERTBOT_AUTO" in os.environ and names == set(("apache", "nginx")):
+            # The possibility of being offered exactly apache and nginx here
+            # is new interactivity brought by https://github.com/certbot/certbot/issues/4079,
+            # so set apache as a default for those kinds of non-interactive use
+            # (the user will get a warning to set --non-interactive or --force-interactive)
+            apache_idx = [n for n, p in enumerate(prepared) if p.name == "apache"][0]
+            code, index = disp.menu(question, opts, default=apache_idx)
+        else:
+            code, index = disp.menu(question, opts, force_interactive=True)
 
         if code == display_util.OK:
             plugin_ep = prepared[index]


Test patch:
diff --git a/certbot/plugins/selection_test.py b/certbot/plugins/selection_test.py
--- a/certbot/plugins/selection_test.py
+++ b/certbot/plugins/selection_test.py
@@ -1,4 +1,5 @@
 """Tests for letsencrypt.plugins.selection"""
+import os
 import sys
 import unittest
 
@@ -115,6 +116,7 @@ def setUp(self):
                                                                False))
         self.mock_apache = mock.Mock(
             description_with_name="a", misconfigured=True)
+        self.mock_apache.name = "apache"
         self.mock_stand = mock.Mock(
             description_with_name="s", misconfigured=False)
         self.mock_stand.init().more_info.return_value = "standalone"
@@ -146,3 +148,26 @@ def test_more_info(self, mock_util):
     def test_no_choice(self, mock_util):
         mock_util().menu.return_value = (display_util.CANCEL, 0)
         self.assertTrue(self._call() is None)
+
+    @test_util.patch_get_utility("certbot.plugins.selection.z_util")
+    def test_new_interaction_avoidance(self, mock_util):
+        mock_nginx = mock.Mock(
+            description_with_name="n", misconfigured=False)
+        mock_nginx.init().more_info.return_value = "nginx plugin"
+        mock_nginx.name = "nginx"
+        self.plugins[1] = mock_nginx
+        mock_util().menu.return_value = (display_util.CANCEL, 0)
+
+        unset_cb_auto = os.environ.get("CERTBOT_AUTO") is None
+        if unset_cb_auto:
+            os.environ["CERTBOT_AUTO"] = "foo"
+        try:
+            self._call()
+        finally:
+            if unset_cb_auto:
+                del os.environ["CERTBOT_AUTO"]
+
+        self.assertTrue("default" in mock_util().menu.call_args[1])
+
+if __name__ == "__main__":
+    unittest.main()  # pragma: no cover