Repository: certbot/certbot

Issue:
"IndexError: list index out of range" while setup for nginx
Affects version certbot 0.10.0.dev0 and propaby below.
System: Arch linux.

Log output:

```
2016-12-11 13:18:22,545:DEBUG:certbot.main:Root logging level set at 20
2016-12-11 13:18:22,549:INFO:certbot.main:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2016-12-11 13:18:22,552:DEBUG:certbot.main:certbot version: 0.10.0.dev0
2016-12-11 13:18:22,552:DEBUG:certbot.main:Arguments: ['--nginx']
2016-12-11 13:18:22,555:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#script,PluginEntryPoint#standalone,PluginEntryPoint#manual,PluginEntryPoint#webroot,PluginEntryPoint#nginx,PluginEntryPoint#null)
2016-12-11 13:18:22,557:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2016-12-11 13:18:25,558:DEBUG:certbot.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin - Alpha
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator
Initialized: <certbot_nginx.configurator.NginxConfigurator object at 0xb6931d8c>
Prep: True
2016-12-11 13:18:25,563:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_nginx.configurator.NginxConfigurator object at 0xb6931d8c> and installer <certbot_nginx.configurator.NginxConfigurator object at 0xb6931d8c>
2016-12-11 13:18:25,658:DEBUG:certbot.main:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.10.0.dev0', 'console_scripts', 'certbot')()
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 818, in main
    return config.func(config, plugins)
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 550, in run
    domains, certname = _find_domains_or_certname(config, installer)
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 288, in _find_domains_or_certname
    domains = display_ops.choose_names(installer)
  File "/usr/lib/python2.7/site-packages/certbot/display/ops.py", line 103, in choose_names
    domains = list(installer.get_all_names())
  File "/usr/lib/python2.7/site-packages/certbot_nginx/configurator.py", line 391, in get_all_names
    for vhost in self.parser.get_vhosts():
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 134, in get_vhosts
    servers = self._get_raw_servers()
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 114, in _get_raw_servers
    lambda x, y: srv.append((x[1], y)))
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 364, in _do_for_subarray
    _do_for_subarray(item, condition, func, path + [index])
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 364, in _do_for_subarray
    _do_for_subarray(item, condition, func, path + [index])
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 360, in _do_for_subarray
    if condition(entry):
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 113, in <lambda>
    _do_for_subarray(tree, lambda x: x[0] == ['server'],
IndexError: list index out of range
```

nginx configuration

```
user http;
worker_processes  1;

error_log  /var/log/nginx/error.log;
error_log  /var/log/nginx/error.log  notice;
error_log  /var/log/nginx/error.log  info;

#pid        logs/nginx.pid;


events {
	worker_connections  1024;
}


http {
	include       mime.types;
	default_type  application/octet-stream;

	log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
		'$status $body_bytes_sent "$http_referer" '
		'"$http_user_agent" "$http_x_forwarded_for"';

	access_log  /var/log/nginx/access.log  main;

	sendfile        on;

	keepalive_timeout  120;

	server {
		listen 80;
		server_name subdomain.myserver.com;
	
		rewrite ^ https://$server_name$request_uri permanent;
	}

	server {
		listen 443 ssl;
		server_name subdomain.myserver.com;

		add_header Strict-Transport-Security "max-age=31536000" always;

		ssl_certificate      /etc/letsencrypt/live/subdomain.myserver.com/fullchain.pem;
		ssl_certificate_key  /etc/letsencrypt/live/subdomain.myserver.com/privkey.pem;

		ssl_session_cache    shared:SSL:1m;
		ssl_session_timeout  5m;

		ssl_ciphers  HIGH:!aNULL:!MD5;
		ssl_prefer_server_ciphers  on;

		include sites-enabled/*;
	}

}
```


Discussion:
Does this nginx config work for you with a non-Let's Encrypt cert? Looks like you have your `include sites-enabled/*;` inside a server block, meaning you have nested server blocks, which isn't valid.
Yes, this works, Certs obtained via the webroot method work, too.
There are some nested location-blocks in these files.
Would you mind posting the (redacted) contents of the files inside your `sites-enabled` folder?
Hi @stertingen, could you provide some more information?
I'm very sorry that I cannot provide more information, it works by magic now. Maybe something was messed with my sites-enabled stuff; I don'k know. Shame on me.
I've the same issue with **nearly the same traceback**. I'm on **Arch Linux** too.

In my scenario I have successfully downloaded the initial/first certificate for a domain with a [default config](https://wiki.archlinux.org/index.php/Let%E2%80%99s_Encrypt#Webroot) for **nginx**. With that certificate I configured (see below) my nginx with HTTPS. Everything is fine up to here, HTTPS works. ;)

Now I've noticed that the certificate which I issued does not cover the **`www.`-subdomain** of the domain, which I need. So I though that I can rerun `certbot` with the `-d`-flag (specify domains) to **expand** my certificate for that `www.`-subdomain. This results in the traceback posted below.

`/etc/nginx/nginx.conf`:

```
worker_processes  auto;

events {
}

http {
	gzip on;
	include /etc/nginx/conf.d/http.conf;
	include /etc/nginx/conf.d/https.conf;
}
```

`/etc/nginx/conf.d/http.conf`:

```
server {
	listen 80;
	server_name very-fancy-domain.com;
	return 301 https://very-fancy-domain.com$request_uri;
}
```

`/etc/nginx/conf.d/https.conf`:

```
server {
	listen 443 ssl http2;
	server_name very-fancy-domain.com;
	# the following is more or less pasted from https://cipherli.st/
	ssl_certificate /etc/letsencrypt/live/very-fancy-domain.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/very-fancy-domain.com/privkey.pem;
	ssl_trusted_certificate /etc/letsencrypt/live/very-fancy-domain.com/chain.pem;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_prefer_server_ciphers on;
	ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
	ssl_ecdh_curve secp384r1; # Requires nginx >= 1.1.0
	ssl_session_cache shared:SSL:10m;
	ssl_session_tickets off; # Requires nginx >= 1.5.9
	ssl_stapling on; # Requires nginx >= 1.3.7
	ssl_stapling_verify on; # Requires nginx => 1.3.7
	add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
	add_header X-Frame-Options DENY;
	add_header X-Content-Type-Options nosniff;
	location / {
		root /srv/http;
		index index.html;
	}
	include /etc/nginx/conf.d/acme-challenge.conf; # this may not be needed, I've no idea if it is needed
}
```

`/etc/nginx/conf.d/acme-challenge.conf` (is this location/config needed or will `certbot` configure this automatically?):

```
location ^~ /.well-known {
	allow all;
	alias /var/lib/letsencrypt/.well-known/;
	default_type text/plain;
	try_files $uri =404;
}
```

I'm trying to expand my certificate with the following command:

```
certbot --debug --nginx -d very-fancy-domain.com,www.very-fancy-domain.com
```

On my machine I've installed [`certbot-0.13.0-2`](https://www.archlinux.org/packages/community/any/certbot/) and [`certbot-nginx-0.13.0-1`](https://www.archlinux.org/packages/community/any/certbot-nginx/).

```
[root@very-fancy-domain.com ~]# certbot --debug --nginx -d very-fancy-domain.com,www.very-fancy-domain.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/very-fancy-domain.com.conf)

It contains these names: very-fancy-domain.com

You requested these names for the new certificate: very-fancy-domain.com,
www.very-fancy-domain.com

Do you want to expand and replace this existing certificate with the new
certificate?
-------------------------------------------------------------------------------
(E)xpand/(C)ancel: E
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for very-fancy-domain.com
tls-sni-01 challenge for www.very-fancy-domain.com
Cleaning up challenges
2017/04/19 15:44:55 [notice] 2093#2093: signal process started
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.13.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 755, in main
    return config.func(config, plugins)
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 597, in run
    certname, lineage)
  File "/usr/lib/python2.7/site-packages/certbot/main.py", line 77, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/usr/lib/python2.7/site-packages/certbot/renewal.py", line 296, in renew_cert
    new_certr, new_chain, new_key, _ = le_client.obtain_certificate(domains)
  File "/usr/lib/python2.7/site-packages/certbot/client.py", line 285, in obtain_certificate
    self.config.allow_subset_of_names)
  File "/usr/lib/python2.7/site-packages/certbot/auth_handler.py", line 74, in get_authorizations
    resp = self._solve_challenges()
  File "/usr/lib/python2.7/site-packages/certbot/auth_handler.py", line 115, in _solve_challenges
    resp = self.auth.perform(self.achalls)
  File "/usr/lib/python2.7/site-packages/certbot_nginx/configurator.py", line 799, in perform
    sni_response = chall_doer.perform()
  File "/usr/lib/python2.7/site-packages/certbot_nginx/tls_sni_01.py", line 55, in perform
    vhost = self.configurator.choose_vhost(achall.domain)
  File "/usr/lib/python2.7/site-packages/certbot_nginx/configurator.py", line 230, in choose_vhost
    matches = self._get_ranked_matches(target_name)
  File "/usr/lib/python2.7/site-packages/certbot_nginx/configurator.py", line 253, in _get_ranked_matches
    vhost_list = self.parser.get_vhosts()
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 134, in get_vhosts
    servers = self._get_raw_servers()
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 114, in _get_raw_servers
    lambda x, y: srv.append((x[1], y)))
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 364, in _do_for_subarray
    _do_for_subarray(item, condition, func, path + [index])
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 364, in _do_for_subarray
    _do_for_subarray(item, condition, func, path + [index])
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 360, in _do_for_subarray
    if condition(entry):
  File "/usr/lib/python2.7/site-packages/certbot_nginx/parser.py", line 113, in <lambda>
    _do_for_subarray(tree, lambda x: x[0] == ['server'],
IndexError: list index out of range
```
Hi @NIPE-SYSTEMS, thanks for the report!

Would you mind pasting the contents of `/etc/letsencrypt/renewal/very-fancy-domain.com.conf`?

It looks like you ran Certbot the first time using the `certonly` subcommand and manually edited your configuration files, is that correct?
Repro'd! Putting together a fix now.
(It's the empty `events` block, which we should be able to handle but currently don't. You shouldn't need to include any sort of `acme-challenge.conf` manually.)

Resolving patch:
diff --git a/certbot-nginx/certbot_nginx/parser.py b/certbot-nginx/certbot_nginx/parser.py
--- a/certbot-nginx/certbot_nginx/parser.py
+++ b/certbot-nginx/certbot_nginx/parser.py
@@ -110,7 +110,7 @@ def _get_raw_servers(self):
             srv = servers[filename]  # workaround undefined loop var in lambdas
 
             # Find all the server blocks
-            _do_for_subarray(tree, lambda x: x[0] == ['server'],
+            _do_for_subarray(tree, lambda x: len(x) >= 2 and x[0] == ['server'],
                              lambda x, y: srv.append((x[1], y)))
 
             # Find 'include' statements in server blocks and append their trees


Test patch:
diff --git a/certbot-nginx/certbot_nginx/tests/parser_test.py b/certbot-nginx/certbot_nginx/tests/parser_test.py
--- a/certbot-nginx/certbot_nginx/tests/parser_test.py
+++ b/certbot-nginx/certbot_nginx/tests/parser_test.py
@@ -125,13 +125,13 @@ def test_get_vhosts(self):
                                  False, True,
                                  set(['localhost',
                                       r'~^(www\.)?(example|bar)\.']),
-                                 [], [9, 1, 9])
+                                 [], [10, 1, 9])
         vhost2 = obj.VirtualHost(nparser.abs_path('nginx.conf'),
                                  [obj.Addr('somename', '8080', False, False),
                                   obj.Addr('', '8000', False, False)],
                                  False, True,
                                  set(['somename', 'another.alias', 'alias']),
-                                 [], [9, 1, 12])
+                                 [], [10, 1, 12])
         vhost3 = obj.VirtualHost(nparser.abs_path('sites-enabled/example.com'),
                                  [obj.Addr('69.50.225.155', '9000',
                                            False, False),
@@ -186,7 +186,7 @@ def test_add_server_directives(self):
                                      None, None, None,
                                      set(['localhost',
                                            r'~^(www\.)?(example|bar)\.']),
-                                     None, [9, 1, 9])
+                                     None, [10, 1, 9])
         nparser.add_server_directives(mock_vhost,
                                       [['foo', 'bar'], ['\n ', 'ssl_certificate', ' ',
                                                         '/etc/ssl/cert.pem']],
diff --git a/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/nginx.conf b/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/nginx.conf
--- a/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/nginx.conf
+++ b/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/nginx.conf
@@ -14,6 +14,9 @@ events {
     worker_connections  1024;
 }
 
+empty {
+}
+
 include foo.conf;
 
 http {