Repository: certbot/certbot

Issue:
nginx parser confused by some unquoted substitutions in proxy_set_header directives
## My operating system is (include version):
Debian GNU/Linux 9 (stretch)


## I installed Certbot with (certbot-auto, OS package manager, pip, etc):
wget https://dl.eff.org/certbot-auto


## I ran this command and it produced this output:
While trying to run `/certbot-auto renew --post-hook "nginx -s reload"`
I get this error:
```
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/XXX.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for XXX.org
tls-sni-01 challenge for www.XXX.org
Cleaning up challenges
2018/09/14 21:45:38 [notice] 8286#8286: signal process started
Attempting to renew cert (XXX.org) from /etc/letsencrypt/renewal/XXX.org.conf produced an unexpected error: Could not automatically find a matching server block for XXX.org. Set the `server_name` directive to use the Nginx installer.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/XXX/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/XXX/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Running post-hook command: nginx -s reload
Error output from nginx:
2018/09/14 21:45:39 [notice] 8289#8289: signal process started

1 renew failure(s), 0 parse failure(s)
```
I think the real error is shown in the debug log:
```
2018-09-14 21:45:37,250:DEBUG:certbot_nginx.parser:Could not parse file: /etc/ng
inx/conf.d/default.conf due to Expected {Group:({[<SPC><TAB><CR><LF>] "#" rest o
f line}) | Group:(Forward: ...) | Group:({[<SPC><TAB><CR><LF>] {Combine:({{quote
d string, starting with " ending with " | quoted string, starting with ' ending 
with '} ")" [Re:('(\\$\\{)|[^{;\\s]')]...}) | Combine:({Re:('[^{};\\s\'\\"]') [R
e:('(\\$\\{)|[^{;\\s]')]...}) | quoted string, starting with " ending with " | q
uoted string, starting with ' ending with '} [{<SPC><TAB><CR><LF> {Combine:({{qu
oted string, starting with " ending with " | quoted string, starting with ' endi
ng with '} ")" [Re:('(\\$\\{)|[^{;\\s]')]...}) | Combine:({Re:('[^{};\\s\'\\"]')
 [Re:('(\\$\\{)|[^{;\\s]')]...}) | quoted string, starting with " ending with " 
| quoted string, starting with ' ending with '}}]... [<SPC><TAB><CR><LF>] Suppre
ss:(";")})} (at char 2372), (line:77, col:1)
```

The nginx config file is (masked):
```
server {
    listen       80 default_server;
    listen  [::]:80 default_server;

    server_name  XXX.org www.XXX.org;

    listen       443 ssl http2 default_server;
    listen  [::]:443 ssl http2 default_server;

	ssl_certificate /etc/letsencrypt/live/XXX.org/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/XXX.org/privkey.pem;


    # from https://cipherli.st/
    # and https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers "XXX";
    ssl_ecdh_curve secp384r1;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_stapling on;
    ssl_stapling_verify on;
    resolver 8.8.8.8 8.8.4.4 valid=300s;
    resolver_timeout 5s;
    # Disable preloading HSTS for now.  You can use the commented out header line that includes
    # the "preload" directive if you understand the implications.
    #add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
    add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
    # add_header X-Frame-Options SAMEORIGIN;
    # add_header X-Content-Type-Options nosniff;

    ssl_dhparam /etc/ssl/certs/dhparam.pem;

    access_log  /var/log/nginx/log/host.access.log  main;

    server_name_in_redirect off;

    location /ddd {
		return 302 $request_uri/;
    }

    location /ddd/ {
		proxy_pass       http://XXX:91/;
		proxy_set_header Host $http_host;
		proxy_set_header X-Origin-URI $uri;
		proxy_set_header Forwarded "for=$remote_addr;proto=$scheme";
    }

    location /ccc {
		return 302 $request_uri/;
    }

    location /bbb/ {
		proxy_pass       http://XXX:90/;
		proxy_set_header Host $http_host;
		proxy_set_header X-Origin-URI $uri;
		proxy_set_header Forwarded "for=$remote_addr;proto=$scheme";
    }

    location /aaa/ {
    	proxy_pass       http://XXX:89/;
		proxy_set_header Host $http_host;
		proxy_set_header X-Origin-URI $uri;
		proxy_set_header Forwarded "for=$remote_addr;proto=$scheme";
    }

    location / {
    	proxy_pass	 http://XXX:88/;
		proxy_set_header Host $http_host;
		proxy_set_header X-Origin-URI ${scheme}://${http_host}/$request_uri;
		proxy_set_header Forwarded "for=$remote_addr;proto=$scheme";
    }

}
```

Any ideas what might be going on?


Discussion:
Any thoughts on this, @ohemorange?

My first question might be about the lines

```
                proxy_set_header X-Origin-URI ${scheme}://${http_host}/$request_uri;
                proxy_set_header Forwarded "for=$remote_addr;proto=$scheme";
```

They just subjectively look difficult to parse.

@duglin, do you recall what you may have changed in this configuration file between the time when you originally obtained your certificate and now? Since this is a renewal, it looks like you once had an nginx configuration that Certbot *was* able to parse, but it's having trouble with your current one.
I just played with this briefly using the nginx parser roundtrip compatibility test (which uses our parser purely syntactically to attempt to parse configuration files). Indeed, the `X-Origin-URI` line is the problem for our parser; removing it causes the parsing to succeed. I also found that putting the value of the header in quotes like

```
                proxy_set_header X-Origin-URI "${scheme}://${http_host}/$request_uri";
```

will make the parsing succeed, at least syntactically.

I didn't check (1) whether this file is correctly parsed by nginx itself without the quotes, (2) whether it's correctly parsed by nginx with the quotes, or (3) whether Certbot would do the right thing to the file in other respects if the quotes were present.
@schoen thanks a ton!  Adding the quotes seems to have fixed it.

I do remember seeing this issue back when I first starting using certbot but can't remember how I got around it. I did look at an old version of the config from that timeframe but don't see any edits around that line so go figure.

Anyway, thanks again for the help! Should I close this issue or keep it open as a reminder to tweak the parser to support that line w/o quotes?
Can I assume that that line works properly in nginx and has the behavior that you intended? If so, I think we should keep the issue open because we aim to match nginx's parser in terms of which configuration files we can accept.
@schoen yup - nginx is happy with the line w/o the quotes.
Great, we'll keep the issue open then. Thanks for reporting it!
@schoen and thank you for helping me get around it!
Based on my testing, without the quotes, the offending line is also not valid Nginx. Was it working for you? As in, would Nginx run on this config file?
yes I was able to start nginx and even tried it with the -t option to test it - neither showed an error.
I'm on `nginx version: nginx/1.13.12`

```
$ nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

$ grep "X-Origin-URI" default.conf 
		proxy_set_header X-Origin-URI $uri;
		proxy_set_header X-Origin-URI $uri;
		proxy_set_header X-Origin-URI $uri;
		proxy_set_header X-Origin-URI ${scheme}://${http_host}/$request_uri;
```
Ah! You're using a newer version of Nginx than I am. Your version contains this change: https://github.com/nginx/nginx/commit/0ad556fe59ad132dc4d34dea9e80f2ff2c3c1314#diff-44ce6e0a027219f5b99513de6d08642a

Thanks for catching this, time to update our parser.
ah good catch!!

Resolving patch:
diff --git a/certbot-nginx/certbot_nginx/nginxparser.py b/certbot-nginx/certbot_nginx/nginxparser.py
--- a/certbot-nginx/certbot_nginx/nginxparser.py
+++ b/certbot-nginx/certbot_nginx/nginxparser.py
@@ -26,7 +26,7 @@ class RawNginxParser(object):
     dquoted = QuotedString('"', multiline=True, unquoteResults=False, escChar='\\')
     squoted = QuotedString("'", multiline=True, unquoteResults=False, escChar='\\')
     quoted = dquoted | squoted
-    head_tokenchars = Regex(r"[^{};\s'\"]") # if (last_space)
+    head_tokenchars = Regex(r"(\$\{)|[^{};\s'\"]") # if (last_space)
     tail_tokenchars = Regex(r"(\$\{)|[^{;\s]") # else
     tokenchars = Combine(head_tokenchars + ZeroOrMore(tail_tokenchars))
     paren_quote_extend = Combine(quoted + Literal(')') + ZeroOrMore(tail_tokenchars))


Test patch:
diff --git a/certbot-nginx/certbot_nginx/tests/nginxparser_test.py b/certbot-nginx/certbot_nginx/tests/nginxparser_test.py
--- a/certbot-nginx/certbot_nginx/tests/nginxparser_test.py
+++ b/certbot-nginx/certbot_nginx/tests/nginxparser_test.py
@@ -271,6 +271,8 @@ def test_weird_blocks(self):
             location ~ ^/users/(.+\.(?:gif|jpe?g|png))$ {
               alias /data/w3/images/$1;
             }
+
+            proxy_set_header X-Origin-URI ${scheme}://${http_host}/$request_uri;
         """
         parsed = loads(test)
         self.assertEqual(parsed, [[['if', '($http_user_agent', '~', 'MSIE)'],
@@ -281,7 +283,8 @@ def test_weird_blocks(self):
             [['return', '403']]], [['if', '($args', '~', 'post=140)'],
             [['rewrite', '^', 'http://example.com/']]],
             [['location', '~', '^/users/(.+\\.(?:gif|jpe?g|png))$'],
-            [['alias', '/data/w3/images/$1']]]]
+            [['alias', '/data/w3/images/$1']]],
+            ['proxy_set_header', 'X-Origin-URI', '${scheme}://${http_host}/$request_uri']]
         )
 
     def test_edge_cases(self):
@@ -289,10 +292,6 @@ def test_edge_cases(self):
         parsed = loads(r'"hello\""; # blah "heh heh"')
         self.assertEqual(parsed, [['"hello\\""'], ['#', ' blah "heh heh"']])
 
-        # empty var as block
-        parsed = loads(r"${}")
-        self.assertEqual(parsed, [[['$'], []]])
-
         # if with comment
         parsed = loads("""if ($http_cookie ~* "id=([^;]+)(?:;|$)") { # blah )
             }""")
@@ -342,10 +341,9 @@ def test_edge_cases(self):
         ])
 
         # variable weirdness
-        parsed = loads("directive $var;")
-        self.assertEqual(parsed, [['directive', '$var']])
+        parsed = loads("directive $var ${var} $ ${};")
+        self.assertEqual(parsed, [['directive', '$var', '${var}', '$', '${}']])
         self.assertRaises(ParseException, loads, "server {server_name test.com};")
-        self.assertRaises(ParseException, loads, "directive ${var};")
         self.assertEqual(loads("blag${dfgdfg};"), [['blag${dfgdfg}']])
         self.assertRaises(ParseException, loads, "blag${dfgdf{g};")