Repository: certbot/certbot

Issue:
Failed Apache Parsing
From the community forums

Hi all,

I'm trying to get some certificates using the Let's Encrypt client for two domains on my Linux web server. The web server is Apache 2.4 under Debian 8 on a VPS. I've had the VPS for quite a long time - the initial install was in 2003 - and it's been upgraded through a number of Debian releases, so the Apache 2 configuration is not quite standard.

The Let's Encrypt installer isn't playing the game on my installation, and I assume something to do with the way I have Apache setup is responsible for it, but I would like to debug the problem. Unfortunately, the error message and the matching Python code is not really enough for me to figure out what is going wrong. The error message I get from "letsencrypt auth" or "letsencrypt -d ssl.zzap.org auth" is:

An unexpected error occurred. Please see the logfiles in /var/log/letsencrypt for more details.

```
The log file says the following:
2015-09-17 04:37:46,230:DEBUG:letsencrypt.cli:Root logging level set at 30
2015-09-17 04:37:46,231:INFO:letsencrypt.cli:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2015-09-17 04:37:46,231:DEBUG:letsencrypt.cli:Arguments: ['-d', 'ssl.zzap.org', 'auth']
2015-09-17 04:37:46,232:DEBUG:letsencrypt.cli:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#manual,PluginEntryPoint#standalone)
2015-09-17 04:37:47,377:DEBUG:letsencrypt.cli:Exiting abnormally:
Traceback (most recent call last):
File "/usr/local/bin/letsencrypt", line 9, in 
load_entry_point('letsencrypt==0.1', 'console_scripts', 'letsencrypt')()
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/cli.py", line 920, in main
return args.func(args, config, plugins)
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/cli.py", line 312, in auth
config, args.authenticator, plugins)
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/display/ops.py", line 98, in pick_authenticator
config, default, plugins, question, (interfaces.IAuthenticator,))
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/display/ops.py", line 72, in pick_plugin
verified.prepare()
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/plugins/disco.py", line 197, in prepare
return [plugin_ep.prepare() for plugin_ep in self._plugins.itervalues()]
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/plugins/disco.py", line 98, in prepare
self._initialized.prepare()
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/configurator.py", line 144, in prepare
self.aug, self.conf("server-root"), self.conf("ctl"))
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/parser.py", line 54, in init
self.init_modules()
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/parser.py", line 72, in init_modules
matches = self.find_dir("LoadModule")
...skipping...
return args.func(args, config, plugins)
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/cli.py", line 312, in auth
config, args.authenticator, plugins)
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/display/ops.py", line 98, in pick_authenticator
config, default, plugins, question, (interfaces.IAuthenticator,))
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/display/ops.py", line 72, in pick_plugin
verified.prepare()
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/plugins/disco.py", line 197, in prepare
return [plugin_ep.prepare() for plugin_ep in self._plugins.itervalues()]
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt/plugins/disco.py", line 98, in prepare
self._initialized.prepare()
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/configurator.py", line 144, in prepare
self.aug, self.conf("server-root"), self.conf("ctl"))
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/parser.py", line 54, in init
self.init_modules()
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/parser.py", line 72, in init_modules
matches = self.find_dir("LoadModule")
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/parser.py", line 304, in find_dir
exclude))
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/letsencrypt_apache/parser.py", line 282, in find_dir
"%s//*[self::directive=~regexp('%s')]" % (start, regex))
File "/usr/local/src/letsencrypt/venv/local/lib/python2.7/site-packages/augeas.py", line 415, in match
raise RuntimeError("Error during match procedure!", path)
RuntimeError: ('Error during match procedure!', u"/files/etc/apache2/sites-enabled///*[self::directive=~regexp('([Ll][Oo][Aa][Dd][Mm][Oo][Dd][Uu][Ll][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]")
```

It looks like it can't find the LoadModule command, but this is in /etc/apache2/mods-enabled/ssl.load (for the SSL module). I tried adding it to /etc/apache2/sites-enabled/all, which contains all my site configuration, but that didn't make any difference. I also tried changing my site configuration to include domain names instead of IP addresses under the VirtualHost commands, but that didn't make any difference. Beyond this, I tried looking through the Python code, but I don't really know Python particularly well and the best I could do was figure out where the error message is (.../letsencrypt/letsencrypt-apache/letsencrypt_apache/parser.py) but not how to fix it, or even bypass the offending piece of code.

Does anyone here know how I would at least figure out what the client is looking for that it's not finding, or how to fix or work around this problem?

Thanks,

Regards,

Simon.



Discussion:
It'd be easier if user provided his config files, including mods-enabled.

Responded on community-forums with...

Hi Simon,

Thank you so much for reporting this. I filed an issue on the GitHub repo and assigned myself to it. https://github.com/letsencrypt/letsencrypt/issues/781

The reason is that it should have looked up //\* rather than ///*

I am guessing this is due to an atypical include statement.

I will try to start trying to reproduce the bug, but it might help if you could tell me what the include statement is in apache2.conf or httpd.conf

Something like...
IncludeOptional sites-enabled/*.conf
or
Include sites-enabled/

Feel free to respond on the GitHub repo.

Thanks again,
James

Sorry... (duplicating the posts)

It might also be a great help if you run letshelp-letsencrypt. It copies your config, removes all sensitive data from it, confirms the action with you, bundles the copy into a tar ball which you can send us.

It will get placed into our testing infrastructure and then we will never commit a change that doesn't work on your configuration again.

James

After about 20 minutes of modifying Include statements with tons of backslashes... I finally was able to recreate the bug.

It was more basic than I had assumed.

Breaks: `Include /etc/apache2/sites-enabled/`  or `Include /etc/apache2/sites-enabled//`

Doesn't Break: `Include /etc/apache2/sites-enabled` or `/etc/apache2/sites-enabled/*`  or `Include /etc/apache2/sites-enabled///////*`  or `Include sites-enabled/`  or `Include sites-enabled`

I am not correctly parsing when a backslash tails the Include statement and is a full path.

ref to community: https://community.letsencrypt.org/t/installer-throwing-error-message/1120


Resolving patch:
diff --git a/letsencrypt-apache/letsencrypt_apache/parser.py b/letsencrypt-apache/letsencrypt_apache/parser.py
--- a/letsencrypt-apache/letsencrypt_apache/parser.py
+++ b/letsencrypt-apache/letsencrypt_apache/parser.py
@@ -394,6 +394,8 @@ def _get_include_path(self, arg):
         if not arg.startswith("/"):
             # Normpath will condense ../
             arg = os.path.normpath(os.path.join(self.root, arg))
+        else:
+            arg = os.path.normpath(arg)
 
         # Attempts to add a transform to the file if one does not already exist
         if os.path.isdir(arg):


Test patch:
diff --git a/letsencrypt-apache/letsencrypt_apache/tests/complex_parsing_test.py b/letsencrypt-apache/letsencrypt_apache/tests/complex_parsing_test.py
--- a/letsencrypt-apache/letsencrypt_apache/tests/complex_parsing_test.py
+++ b/letsencrypt-apache/letsencrypt_apache/tests/complex_parsing_test.py
@@ -98,6 +98,9 @@ def test_include_complex(self):
     def test_include_fullpath(self):
         self.verify_fnmatch(os.path.join(self.config_path, "test_fnmatch.conf"))
 
+    def test_include_fullpath_trailing_slash(self):
+        self.verify_fnmatch(self.config_path + "//")
+
     def test_include_variable(self):
         self.verify_fnmatch("../complex_parsing/${fnmatch_filename}")