internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: certbot/certbot

Issue:
Travis: pep8: not found
In #2138 it was discovered that `pep8` is not run because it's not installed at the right time.



Discussion:

Resolving patch:
diff --git a/acme/setup.py b/acme/setup.py
--- a/acme/setup.py
+++ b/acme/setup.py
@@ -36,17 +36,18 @@
     install_requires.append('ndg-httpsclient')
     install_requires.append('pyasn1')
 
+dev_extras = [
+    'nose',
+    'pep8',
+    'tox',
+]
+
 docs_extras = [
     'Sphinx>=1.0',  # autodoc_member_order = 'bysource', autodoc_default_flags
     'sphinx_rtd_theme',
     'sphinxcontrib-programoutput',
 ]
 
-testing_extras = [
-    'nose',
-    'tox',
-]
-
 
 setup(
     name='acme',
@@ -76,8 +77,8 @@
     include_package_data=True,
     install_requires=install_requires,
     extras_require={
+        'dev': dev_extras,
         'docs': docs_extras,
-        'testing': testing_extras,
     },
     entry_points={
         'console_scripts': [
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -64,7 +64,12 @@ def read_file(filename, encoding='utf8'):
 dev_extras = [
     # Pin astroid==1.3.5, pylint==1.4.2 as a workaround for #289
     'astroid==1.3.5',
+    'coverage',
+    'nose',
+    'nosexcover',
+    'pep8',
     'pylint==1.4.2',  # upstream #248
+    'tox',
     'twine',
     'wheel',
 ]
@@ -76,14 +81,6 @@ def read_file(filename, encoding='utf8'):
     'sphinxcontrib-programoutput',
 ]
 
-testing_extras = [
-    'coverage',
-    'nose',
-    'nosexcover',
-    'pep8',
-    'tox',
-]
-
 setup(
     name='letsencrypt',
     version=version,
@@ -119,7 +116,6 @@ def read_file(filename, encoding='utf8'):
     extras_require={
         'dev': dev_extras,
         'docs': docs_extras,
-        'testing': testing_extras,
     },
 
     tests_require=install_requires,


Test patch:
Repository: certbot/certbot

Issue:
VersionConflict: ConfigArgParse 0.9.3
Hello,

I tried renewing our certificates today, and I got an error from letsencrypt-auto. The error message was:

VersionConflict: (ConfigArgParse 0.9.3 (/home/bogdanb/.local/share/letsencrypt/lib/python2.7/site-packages), Requirement.parse('ConfigArgParse>=0.10.0'))

Please see the two attached files for the logs:
[execution.log.txt](https://github.com/letsencrypt/letsencrypt/files/97381/execution.log.txt)
[letencrypt.log.txt](https://github.com/letsencrypt/letsencrypt/files/97383/letencrypt.log.txt)

This is after I had deleted the ~/.local folder where letsencrypt stores its downloads. (I had the same exception before.) This is on an Ubuntu 15.04 machine:

$ uname -a
Linux qotilabs-dime 2.6.32-042stab112.15 #1 SMP Tue Oct 20 17:22:56 MSK 2015 x86_64 x86_64 x86_64 GNU/Linux



Discussion:
I believe this is due to the limitations of the current `letsencrypt-auto` script. The script does not update itself and it appears that `letsencrypt-auto` scripts downloaded before [December 22nd](https://github.com/letsencrypt/letsencrypt/pull/1908) don't play nicely with the new 0.2.0 release.

You should be able to fix this by deleting `~/.local/share/letsencrypt`, doing a `git pull` to get a new version of `letsencrypt-auto`, and then running the script again.

If there's nothing we can put into `setup.py` to leave both py26 and old copies of leauto unbroken, we should consider vendorising ConfigArgParse.  But I'm still a bit confused about how this error wasn't caught by [test_leauto_upgrades.sh](https://github.com/letsencrypt/letsencrypt/blob/master/tests/letstest/scripts/test_leauto_upgrades.sh).

Oh, no, I do see the bug there.  That script uses the old pip release, but the newer version of `letsencrypt-auto` :( :( :(

A few possible strategies:
- vendorize ConfigArgParse (but a lot of folks want us not to do this, especially for easier packaging)
- relax the versioned dependency to `>= 0.9.3`.  If that would break anything, try to work around it in Python?


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,7 +33,10 @@ def read_file(filename, encoding='utf8'):
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [
     'acme=={0}'.format(version),
-    'ConfigArgParse>=0.10.0',  # python2.6 support, upstream #17
+    # We technically need ConfigArgParse 0.10.0 for Python 2.6 support, but
+    # saying so here causes a runtime error against our temporary fork of 0.9.3
+    # in which we added 2.6 support (see #2243), so we relax the requirement.
+    'ConfigArgParse>=0.9.3',
     'configobj',
     'cryptography>=0.7',  # load_pem_x509_certificate
     'parsedatetime',


Test patch:
Repository: certbot/certbot

Issue:
.keepdir mistaken for renewer config
When running letsencrypt for a renewal, it seems that the client confuses the .keepdir file installed to prevent the directory for being deleted for a configuration file.

The command: `letsencrypt certonly --webroot --webroot-path /var/www/html -d ...` ends up erroring with:

```

┌──────────────────────────────────────────────────────────────────────┐
│ Renewal configuration file /etc/letsencrypt/renewal/.keepdir is      │
│ broken. Skipping.                                                    │
│                                                                      │
└──────────────────────────────────────────────────────────────────────┘
```

Why is the .keepdir file distributed?



Discussion:
It actually also happened to me (the [original reporter over at Debian](https://bugs.debian.org/812223)) with e.g. `letsencrypt certonly --standalone -d …`

The solution for this is to have `storage.py` only pick up files specifically ending in `.conf`.


Resolving patch:
diff --git a/letsencrypt/cli.py b/letsencrypt/cli.py
--- a/letsencrypt/cli.py
+++ b/letsencrypt/cli.py
@@ -194,6 +194,8 @@ def _find_duplicative_certs(config, domains):
     le_util.make_or_verify_dir(configs_dir, mode=0o755, uid=os.geteuid())
 
     for renewal_file in os.listdir(configs_dir):
+        if not renewal_file.endswith(".conf"):
+            continue
         try:
             full_path = os.path.join(configs_dir, renewal_file)
             candidate_lineage = storage.RenewableCert(full_path, cli_config)
diff --git a/letsencrypt/renewer.py b/letsencrypt/renewer.py
--- a/letsencrypt/renewer.py
+++ b/letsencrypt/renewer.py
@@ -172,6 +172,8 @@ def main(cli_args=sys.argv[1:]):
                                constants.CONFIG_DIRS_MODE, uid)
 
     for renewal_file in os.listdir(cli_config.renewal_configs_dir):
+        if not renewal_file.endswith(".conf"):
+            continue
         print("Processing " + renewal_file)
         try:
             # TODO: Before trying to initialize the RenewableCert object,


Test patch:
diff --git a/letsencrypt/tests/renewer_test.py b/letsencrypt/tests/renewer_test.py
--- a/letsencrypt/tests/renewer_test.py
+++ b/letsencrypt/tests/renewer_test.py
@@ -68,6 +68,13 @@ def setUp(self):
         config.write()
         self.config = config
 
+        # We also create a file that isn't a renewal config in the same
+        # location to test that logic that reads in all-and-only renewal
+        # configs will ignore it and NOT attempt to parse it.
+        junk = open(os.path.join(self.tempdir, "renewal", "IGNORE.THIS"), "w")
+        junk.write("This file should be ignored!")
+        junk.close()
+
         self.defaults = configobj.ConfigObj()
         self.test_rc = storage.RenewableCert(config.filename, self.cli_config)
Repository: certbot/certbot

Issue:
Update _get_ranked_matches comment to describe ordering by SSL too
(Slightly related to #2059)

One technique for managing Nginx vhosts is to have two config files per server name, one for HTTP and one for HTTPS. Both files would contain the same server_name, and the Nginx configurator would pick one arbitrarily to install the certificate in.

Instead, configurator.py's `_get_ranked_matches` should use the SSL status of a vhost as part of its ranking, and preferentially install certificates in a vhost which already has SSL enabled.



Discussion:
isn't this what the vhost.ssl statements in the elif block are for?
They seem to come from parser.py's _parse_server() function which checks the addr directive for ssl

Maybe it's just not working because it's looking in the wrong place?

Yep, you're right. I was reading the comment, which doesn't mention ordering by SSL. Let's make this ticket about updating the comment, which should be easy. :-)


Resolving patch:
diff --git a/letsencrypt-nginx/letsencrypt_nginx/configurator.py b/letsencrypt-nginx/letsencrypt_nginx/configurator.py
--- a/letsencrypt-nginx/letsencrypt_nginx/configurator.py
+++ b/letsencrypt-nginx/letsencrypt_nginx/configurator.py
@@ -239,6 +239,7 @@ def choose_vhost(self, target_name):
 
     def _get_ranked_matches(self, target_name):
         """Returns a ranked list of vhosts that match target_name.
+        The ranking gives preference to SSL vhosts.
 
         :param str target_name: The name to match
         :returns: list of dicts containing the vhost, the matching name, and


Test patch:
Repository: certbot/certbot

Issue:
webroot failure gives a not-very-useful error
If webroot renewal (or probably initial auth) fails due to the lack of a running web server that's pointed at that webroot, the error message returned to the user is very generic and doesn't really mention the possibility that a suitable web server was unavailable. Users might find it hard to diagnose this problem on this basis.



Discussion:
Moving this into the milestone for now as it is an easy fix and might be useful for people running `renew` off of `0.4.0`.


Resolving patch:
diff --git a/letsencrypt/auth_handler.py b/letsencrypt/auth_handler.py
--- a/letsencrypt/auth_handler.py
+++ b/letsencrypt/auth_handler.py
@@ -480,6 +480,9 @@ def is_preferred(offered_challb, satisfied,
     return True
 
 
+_ACME_PREFIX = "urn:acme:error:"
+
+
 _ERROR_HELP_COMMON = (
     "To fix these errors, please make sure that your domain name was entered "
     "correctly and the DNS A record(s) for that domain contain(s) the "
@@ -490,7 +493,9 @@ def is_preferred(offered_challb, satisfied,
     "connection":
         _ERROR_HELP_COMMON + " Additionally, please check that your computer "
         "has a publicly routable IP address and that no firewalls are preventing "
-        "the server from communicating with the client.",
+        "the server from communicating with the client. If you're using the "
+        "webroot plugin, you should also verify that you are serving files "
+        "from the webroot path you provided.",
     "dnssec":
         _ERROR_HELP_COMMON + " Additionally, if you have DNSSEC enabled for "
         "your domain, please ensure that the signature is valid.",
@@ -540,11 +545,13 @@ def _generate_failed_chall_msg(failed_achalls):
 
     """
     typ = failed_achalls[0].error.typ
+    if typ.startswith(_ACME_PREFIX):
+        typ = typ[len(_ACME_PREFIX):]
     msg = ["The following errors were reported by the server:"]
 
     for achall in failed_achalls:
         msg.append("\n\nDomain: %s\nType:   %s\nDetail: %s" % (
-            achall.domain, achall.error.typ, achall.error.detail))
+            achall.domain, typ, achall.error.detail))
 
     if typ in _ERROR_HELP:
         msg.append("\n\n")


Test patch:
diff --git a/letsencrypt/tests/auth_handler_test.py b/letsencrypt/tests/auth_handler_test.py
--- a/letsencrypt/tests/auth_handler_test.py
+++ b/letsencrypt/tests/auth_handler_test.py
@@ -437,9 +437,12 @@ def setUp(self):
             "chall": acme_util.HTTP01,
             "uri": "uri",
             "status": messages.STATUS_INVALID,
-            "error": messages.Error(typ="tls", detail="detail"),
+            "error": messages.Error(typ="urn:acme:error:tls", detail="detail"),
         }
 
+        # Prevent future regressions if the error type changes
+        self.assertTrue(kwargs["error"].description is not None)
+
         self.http01 = achallenges.KeyAuthorizationAnnotatedChallenge(
             # pylint: disable=star-args
             challb=messages.ChallengeBody(**kwargs),
Repository: certbot/certbot

Issue:
Fix missing help_label in noninteractive menu



Discussion:

Resolving patch:
diff --git a/letsencrypt/display/util.py b/letsencrypt/display/util.py
--- a/letsencrypt/display/util.py
+++ b/letsencrypt/display/util.py
@@ -446,7 +446,7 @@ def notification(self, message, height=10, pause=False):
                 line=os.linesep, frame=side_frame, msg=message))
 
     def menu(self, message, choices, ok_label=None, cancel_label=None,
-             default=None, cli_flag=None):
+             help_label=None, default=None, cli_flag=None):
         # pylint: disable=unused-argument,too-many-arguments
         """Avoid displaying a menu.
 


Test patch:
Repository: certbot/certbot

Issue:
Deprecate apache-ctl argument
`--apache-ctl` argument was removed. This argument should still be added as a deprecated argument so it still works for people.



Discussion:

Resolving patch:
diff --git a/letsencrypt-apache/letsencrypt_apache/configurator.py b/letsencrypt-apache/letsencrypt_apache/configurator.py
--- a/letsencrypt-apache/letsencrypt_apache/configurator.py
+++ b/letsencrypt-apache/letsencrypt_apache/configurator.py
@@ -106,7 +106,9 @@ def add_parser_arguments(cls, add):
         add("handle-sites", default=constants.os_constant("handle_sites"),
             help="Let installer handle enabling sites for you." +
                  "(Only Ubuntu/Debian currently)")
-        le_util.add_deprecated_argument(add, "init-script", 1)
+        le_util.add_deprecated_argument(add, argument_name="ctl", nargs=1)
+        le_util.add_deprecated_argument(
+            add, argument_name="init-script", nargs=1)
 
     def __init__(self, *args, **kwargs):
         """Initialize an Apache Configurator.


Test patch:
Repository: certbot/certbot

Issue:
letsencrypt fails with "AttributeError: 'NoneType' object has no attribute 'lower'"
letsencrypt failed on our server when parsing our apache2.4 config. It turned out that the config files where the error happens are standard apache2 modules.
The error message is `"AttributeError: 'NoneType' object has no attribute 'lower'" in letsencrypt_apache/parser.py", line 311`.

We created pull request #2105 to demonstrate this behavior and to help with TTD ;-).

Thanks for providing this great project!



Discussion:
~~So this is a duplicate of #2105, right?~~ no I guess it's a bug to go with that PR :)

Assigning this to 0.4.0 though it would be great to get a fix for the imminent 0.3.0 release.

Yes, that's the bug report corresponding to the pull request with the test showing the problem.

this is a fun bug!

```
(Pdb) matches
[u'/files/tmp/tmp2pLutJtemp/debian_apache_2_4/two_vhost_80/apache2/mods-enabled/dav_svn.load/IfModule/directive', u'/files/tmp/tmp2pLutJtemp/debian_apache_2_4/two_vhost_80/apache2/mods-enabled/dav_svn.load/IfModule/IfModule/directive', u'/files/tmp/tmp2pLutJtemp/debian_apache_2_4/two_vhost_80/apache2/mods-enabled/authz_svn.load/directive', u'/files/tmp/tmp2pLutJtemp/debian_apache_2_4/two_vhost_80/apache2/mods-enabled/authz_svn.load/IfModule/directive', u'/files/tmp/tmp2pLutJtemp/debian_apache_2_4/two_vhost_80/apache2/mods-enabled/dav.load/IfModule/directive']
(Pdb) matches = self.aug.match("%s//*[self::directive=~regexp('%s')]" % (start, regex))
(Pdb) matches
[u'/files/tmp/tmp2pLutJtemp/debian_apache_2_4/two_vhost_80/apache2/mods-enabled/authz_svn.load/directive', u'/files/tmp/tmp2pLutJtemp/debian_apache_2_4/two_vhost_80/apache2/mods-enabled/authz_svn.load/IfModule/directive']
```

it seems that we're doing something to get rid of some of our matches so that when we later call get on things that were previously in the match list, they're no longer there


Resolving patch:
diff --git a/letsencrypt-apache/letsencrypt_apache/parser.py b/letsencrypt-apache/letsencrypt_apache/parser.py
--- a/letsencrypt-apache/letsencrypt_apache/parser.py
+++ b/letsencrypt-apache/letsencrypt_apache/parser.py
@@ -477,7 +477,7 @@ def _parse_file(self, filepath):
         # Note: This works for augeas globs, ie. *.conf
         if use_new:
             inc_test = self.aug.match(
-                "/augeas/load/Httpd/incl [. ='%s']" % filepath)
+                "/augeas/load/Httpd['%s' =~ glob(incl)]" % filepath)
             if not inc_test:
                 # Load up files
                 # This doesn't seem to work on TravisCI


Test patch:
Repository: certbot/certbot

Issue:
Version must be set on CSR before marshaling
If you happen to have a very old version of OpenSSL (looking at you Travis) and you don't set the CSR Version an empty integer (0 length byte slice) will be inserted into the version field which causes Boulder to choke as this is an invalid DER encoding. On newer versions if the version isn't set OpenSSL will sneakily just insert a `0` value and continue, meaning we hadn't stumbled across this before.

The version should be set using `req.set_version(2)` (2 = X509v3... because) in [`make_csr`](https://github.com/letsencrypt/letsencrypt/blob/fd1ebb2d9f6a0030307028aad92ceee586fc06c5/letsencrypt/crypto_util.py#L95). Fixing this will allow us to unblock moving Boulder to Golang 1.6 (which we should build a Boulder instance from and run a test-farm thingies against just to check!)



Discussion:

Resolving patch:
diff --git a/letsencrypt/crypto_util.py b/letsencrypt/crypto_util.py
--- a/letsencrypt/crypto_util.py
+++ b/letsencrypt/crypto_util.py
@@ -118,6 +118,7 @@ def make_csr(key_str, domains):
             value=", ".join("DNS:%s" % d for d in domains)
         ),
     ])
+    req.set_version(2)
     req.set_pubkey(pkey)
     req.sign(pkey, "sha256")
     return tuple(OpenSSL.crypto.dump_certificate_request(method, req)


Test patch:
Repository: certbot/certbot

Issue:
Expected a numeric value for http01_port
Forum user Randall describes this problem when using `letsencrypt renew`:

> ...The error was `Expected a numeric value for http01_port. ...` 
> Editing the file /etc/letsencrypt/renewal/[my.domain].conf file solved this problem. I changed `http01_port = None`  to  `http01_port = 80`

https://community.letsencrypt.org/t/help-us-test-renewal-with-letsencrypt-renew/10562/5

@bmw said that this would have been fixed (in terms of renewal config files correctly specifying `http01_port`) back in November, but people who have older lineages than that may have this problem when renewing with `standalone`. That could conceivably be a pretty large number of users!



Discussion:
For reference, this was the commit that changed the behavior: bb6736f36cfe68fc31595954965635d337df1d24

Looks like we had a similar problem before when `renewer.py` tried to cast the value to an int.

A number of people are still having trouble with this (we just had another report on the forum).


Resolving patch:
diff --git a/letsencrypt/cli.py b/letsencrypt/cli.py
--- a/letsencrypt/cli.py
+++ b/letsencrypt/cli.py
@@ -806,12 +806,18 @@ def _restore_required_config_elements(config, renewalparams):
     # int-valued items to add if they're present
     for config_item in INT_CONFIG_ITEMS:
         if config_item in renewalparams and not _set_by_cli(config_item):
-            try:
-                value = int(renewalparams[config_item])
-                setattr(config.namespace, config_item, value)
-            except ValueError:
-                raise errors.Error(
-                    "Expected a numeric value for {0}".format(config_item))
+            config_value = renewalparams[config_item]
+            # the default value for http01_port was None during private beta
+            if config_item == "http01_port" and config_value == "None":
+                logger.info("updating legacy http01_port value")
+                int_value = flag_default("http01_port")
+            else:
+                try:
+                    int_value = int(config_value)
+                except ValueError:
+                    raise errors.Error(
+                        "Expected a numeric value for {0}".format(config_item))
+            setattr(config.namespace, config_item, int_value)
 
 
 def _restore_plugin_configs(config, renewalparams):


Test patch:
diff --git a/letsencrypt/tests/cli_test.py b/letsencrypt/tests/cli_test.py
--- a/letsencrypt/tests/cli_test.py
+++ b/letsencrypt/tests/cli_test.py
@@ -710,6 +710,12 @@ def test_renew_with_bad_int(self):
         self._test_renew_common(renewalparams=renewalparams, error_expected=True,
                                 assert_oc_called=False)
 
+    def test_renew_with_nonetype_http01(self):
+        renewalparams = {'authenticator': 'webroot',
+                         'http01_port': 'None'}
+        self._test_renew_common(renewalparams=renewalparams, error_expected=False,
+                                assert_oc_called=True)
+
     def test_renew_with_bad_domain(self):
         renewalparams = {'authenticator': 'webroot'}
         names = ['*.example.com']
Repository: certbot/certbot

Issue:
letsencrypt-apache on Gentoo should read /etc/conf.d/apache
In Gentoo, Define's are passed by the initscript as -D arguments read in /etc/conf.d/apache

LE seems to ignore that. As a result, since the "Listen 443" directive is inside IfDefine blocks, it is systematically overlooked by LE since it doesn't know about the active Define directives.

LE will therefore add a "Listen 443" temporary directive, which will cause apache to fail with a "could not bind to address 0.0.0.0:443" error. LE in turn will fail with "urn:acme:error:connection" since apache is not running during the challenge.



Discussion:
Yeah, you are right. Gentoo is a bit of a special snowflake regarding the Apache Defines. I'd like to ping @osirisinferi for insight on this issue, as we've had some discussions about this exact thing in the past.

I'll start looking into remedying this.

My local solution was removing the `IfDefine` blocks for `mod_ssl.so` in `httpd.conf`. Ofcourse, that's not a permanent solution for Apache on Gentoo, but at least the LE client will work.

As for a more permanent solution, that's probably going to cost some time and code (parse `/etc/conf.d/apache2` and implement all the `-D`'s in the invocations of all the apache binaries/scripts). As far as I remember, in the previous issue the conclusion was a sort of `wont-fix`, because it's a single distribution with probably a disproportionate amount of effort to fix it.

I might have something this time, and I've begun to implement it. Going to run some tests tomorrow and we'll see how we'll fare.

I'd be happy to test too!


Resolving patch:
diff --git a/letsencrypt-apache/letsencrypt_apache/constants.py b/letsencrypt-apache/letsencrypt_apache/constants.py
--- a/letsencrypt-apache/letsencrypt_apache/constants.py
+++ b/letsencrypt-apache/letsencrypt_apache/constants.py
@@ -42,7 +42,7 @@
     vhost_root="/etc/apache2/vhosts.d",
     vhost_files="*.conf",
     version_cmd=['/usr/sbin/apache2', '-v'],
-    define_cmd=['/usr/sbin/apache2', '-t', '-D', 'DUMP_RUN_CFG'],
+    define_cmd=['apache2ctl', 'virtualhosts'],
     restart_cmd=['apache2ctl', 'graceful'],
     conftest_cmd=['apache2ctl', 'configtest'],
     enmod=None,


Test patch: