internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: deis/deis

Issue:
Remove scheduler timeouts / Taking too long to pull a run container fails with 'failed to create container'
Debugging Deis on Azure led us to initially suspect an Azure-specific issue, but we have seen this on other platforms. Any time a `deis run` takes longer than 60 seconds to complete the `docker pull`, the client errs with `failed to create container`.

This leads to a deeper issue with our timeouts -- the router uses a [20 minute timeout](https://github.com/deis/deis/blob/f03078180bd830cec7a23bd67acfc19e5f966f42/router/image/templates/nginx.conf#L152-L153). In the fleet scheduling logic, we have various loops which perform actions:
- [_wait_for_container](https://github.com/deis/deis/blob/f03078180bd830cec7a23bd67acfc19e5f966f42/controller/scheduler/fleet.py#L163)
- [_wait_for_destroy](https://github.com/deis/deis/blob/f03078180bd830cec7a23bd67acfc19e5f966f42/controller/scheduler/fleet.py#L180)
- [run](https://github.com/deis/deis/blob/f03078180bd830cec7a23bd67acfc19e5f966f42/controller/scheduler/fleet.py#L216)
- [wait for launch](https://github.com/deis/deis/blob/f03078180bd830cec7a23bd67acfc19e5f966f42/controller/scheduler/fleet.py#L260)
- [wait for start](https://github.com/deis/deis/blob/f03078180bd830cec7a23bd67acfc19e5f966f42/controller/scheduler/fleet.py#L269)
- [wait for completion](https://github.com/deis/deis/blob/f03078180bd830cec7a23bd67acfc19e5f966f42/controller/scheduler/fleet.py#L282)

We really shouldn't be "budgeting" the 20 minutes across these timeouts, as we have no idea how long any one will take at any one time. For example, on Azure, a `docker pull` for my run container took 5 minutes, but our pollwait loop only waits for 60s. Letting each of these loops run indefinitely and instead letting the worker/router handle the total request timeout is probably preferred here.



Discussion:

Resolving patch:
diff --git a/controller/scheduler/fleet.py b/controller/scheduler/fleet.py
--- a/controller/scheduler/fleet.py
+++ b/controller/scheduler/fleet.py
@@ -257,7 +257,8 @@ def _do_ssh(cmd):
             return rc, output
 
         # wait for container to launch
-        for _ in range(60):
+        # we loop indefinitely here, as we have no idea how long the docker pull will take
+        while True:
             rc, _ = _do_ssh('docker inspect {name}'.format(**locals()))
             if rc == 0:
                 break


Test patch:
Repository: deis/deis

Issue:
Graceful shutdown with SIGTERM
On Heroku, processes are given a [10 second](https://devcenter.heroku.com/articles/error-codes#r12-exit-timeout) grace period to exit following a `SIGTERM` before they are `SIGKILL`d during the standard [shutdown procedure](https://devcenter.heroku.com/articles/dynos#shutdown). I understand that this is the [default behavior](https://docs.docker.com/reference/commandline/cli/#stop) of `docker stop <id>`. 

What's the status of graceful shutdowns & restarts with deis? I'm not familiar enough with fleet to know if it is signalling containers automatically when a container [is destroyed](https://github.com/deis/deis/blob/v1.3.1/controller/scheduler/fleet.py#L192). I take it that `stop` is [not being called explicitly](https://github.com/deis/deis/blob/v1.3.1/controller/scheduler/fleet.py#L188)?



Discussion:
there's no explicit `deis ps:stop` command yet, which is why that's unimplemented, but it appears that containers are killed quite abrubtly when we destroy containers: https://github.com/deis/deis/blob/v1.3.1/controller/scheduler/fleet.py#L327


Resolving patch:
diff --git a/controller/scheduler/fleet.py b/controller/scheduler/fleet.py
--- a/controller/scheduler/fleet.py
+++ b/controller/scheduler/fleet.py
@@ -325,8 +325,10 @@ def attach(self, name):
     {"section": "Service", "name": "ExecStartPre", "value": '''/bin/sh -c "IMAGE=$(etcdctl get /deis/registry/host 2>&1):$(etcdctl get /deis/registry/port 2>&1)/{image}; docker pull $IMAGE"'''},  # noqa
     {"section": "Service", "name": "ExecStartPre", "value": '''/bin/sh -c "docker inspect {name} >/dev/null 2>&1 && docker rm -f {name} || true"'''},  # noqa
     {"section": "Service", "name": "ExecStart", "value": '''/bin/sh -c "IMAGE=$(etcdctl get /deis/registry/host 2>&1):$(etcdctl get /deis/registry/port 2>&1)/{image}; port=$(docker inspect -f '{{{{range $k, $v := .ContainerConfig.ExposedPorts }}}}{{{{$k}}}}{{{{end}}}}' $IMAGE | cut -d/ -f1) ; docker run --name {name} {memory} {cpu} {hostname} -P -e PORT=$port $IMAGE {command}"'''},  # noqa
+    {"section": "Service", "name": "ExecStop", "value": '''/usr/bin/docker stop {name}'''},
     {"section": "Service", "name": "ExecStop", "value": '''/usr/bin/docker rm -f {name}'''},
     {"section": "Service", "name": "TimeoutStartSec", "value": "20m"},
+    {"section": "Service", "name": "TimeoutStopSec", "value": "10"},
     {"section": "Service", "name": "RestartSec", "value": "5"},
     {"section": "Service", "name": "Restart", "value": "on-failure"},
 ]


Test patch:
Repository: deis/deis

Issue:
Proposal ref(controller): disable swap usage if there is a memory limit
If the app uses `deis limits:set -m type=XXX -a XXX` add the flag `--memory-swap=-1`

```
docker help run
....
  --memory-swap=""           Total memory usage (memory + swap), set '-1' to disable swap (format: <number><optional unit>, where unit = b, k, m or g)
```

without that if the node uses swap it will not use the specified limit



Discussion:
I think disabling swap is reasonable. Is this in Docker 1.4.1 which we're on now?

@carmstrong no, it's available since 1.5.0

> no, it's available since 1.5.0

Let's revisit this once we get that in stable (should be soon).


Resolving patch:
diff --git a/controller/deis/settings.py b/controller/deis/settings.py
--- a/controller/deis/settings.py
+++ b/controller/deis/settings.py
@@ -6,6 +6,7 @@
 import os.path
 import random
 import string
+import semantic_version
 import sys
 import tempfile
 
@@ -340,3 +341,12 @@
 if os.path.exists('/templates/confd_settings.py'):
     sys.path.append('/templates')
     from confd_settings import *  # noqa
+
+
+DOCKER_VERSION = os.environ.get('DOCKER_VERSION', '1.4.1')
+
+DISABLE_SWAP = ""
+
+if (semantic_version.validate(DOCKER_VERSION) and
+        semantic_version.Version(DOCKER_VERSION) >= semantic_version.Version('1.5.0')):
+    DISABLE_SWAP = "--memory-swap=-1"
diff --git a/controller/scheduler/fleet.py b/controller/scheduler/fleet.py
--- a/controller/scheduler/fleet.py
+++ b/controller/scheduler/fleet.py
@@ -129,7 +129,7 @@ def _create_container(self, name, image, command, unit, **kwargs):
         # prepare memory limit for the container type
         mem = kwargs.get('memory', {}).get(l['c_type'], None)
         if mem:
-            l.update({'memory': '-m {}'.format(mem.lower())})
+            l.update({'memory': '-m {} {} '.format(mem.lower(), settings.DISABLE_SWAP)})
         else:
             l.update({'memory': ''})
         # prepare memory limit for the container type


Test patch:
Repository: deis/deis

Issue:
deis auth:cancel is not deleting the user account 
[stacktrace failed call](https://gist.github.com/smothiki/9820dd7486e9654e3c77)

Its actually throwing 500 but we never caught it .

```
mothikis-MacBook-Pro:tests smothiki$ deis auth:cancel
Please log in again in order to cancel this account
username: test
password: 
Logged in as test
Cancel account "test" at http://deis.smothiki.io? (y/N) y
Account cancelled
```



Discussion:
looks related to https://github.com/deis/deis/pull/2865 in a sense, though really the client should've caught that 500 error


Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -768,11 +768,15 @@ def auth_cancel(self, args):
                 confirm = raw_input(
                     "Cancel account \"{}\" at {}? (y/N) ".format(username, controller))
             if confirm in ['y', True]:
-                self._dispatch('delete', '/v1/auth/cancel')
-                self._settings['controller'] = None
-                self._settings['token'] = None
-                self._settings.save()
-                self._logger.info('Account cancelled')
+                response = self._dispatch('delete', '/v1/auth/cancel')
+                if response.status_code == requests.codes.no_content:
+                    self._settings['controller'] = None
+                    self._settings['token'] = None
+                    self._settings.save()
+                    self._logger.info('Account cancelled')
+                else:
+                    self._logger.info('Account not changed')
+                    raise ResponseError(response)
             else:
                 self._logger.info('Account not changed')
 


Test patch:
Repository: deis/deis

Issue:
Domain name validation: w3.domain.com
I tried to add a domain to my app: `deis domains:add w3.domain.com`
and got the following error :
`{u'domain': [u'Hostname does not look valid.']}`
w3.domain.com is a valid domain.



Discussion:
That's interesting. [RFC 1123](http://tools.ietf.org/html/rfc1123#page-13) allows each hostname label to start and end with a digit, which is what the validation should allow.

This should be as easy as [adding tests](https://github.com/deis/deis/blob/master/controller/api/tests/test_domain.py#L51-L58) for this hostname to validate this, then amending the checks at https://github.com/deis/deis/blob/master/controller/api/serializers.py#L254 to be permissive for these domains.

To confirm, are you on v1.4.1?

Yes @bacongobbler, I am indeed on v1.4.1

In this case, it should be as simple as adding the tests and removing the `or label[-1].isdigit()` clause.

> In this case, it should be as simple as adding the tests and removing the or label[-1].isdigit() clause.

:+1: 


Resolving patch:
diff --git a/controller/api/serializers.py b/controller/api/serializers.py
--- a/controller/api/serializers.py
+++ b/controller/api/serializers.py
@@ -251,7 +251,8 @@ def validate_domain(self, value):
         allowed = re.compile("^(?!-)[a-z0-9-]{1,63}(?<!-)$", re.IGNORECASE)
         for label in labels:
             match = allowed.match(label)
-            if not match or '--' in label or label[-1].isdigit() or label.isdigit():
+            if not match or '--' in label or label.isdigit() or \
+               len(labels) == 1 and any(char.isdigit() for char in label):
                 raise serializers.ValidationError('Hostname does not look valid.')
         if models.Domain.objects.filter(domain=value).exists():
             raise serializers.ValidationError(


Test patch:
diff --git a/controller/api/tests/test_domain.py b/controller/api/tests/test_domain.py
--- a/controller/api/tests/test_domain.py
+++ b/controller/api/tests/test_domain.py
@@ -54,6 +54,7 @@ def test_manage_domain(self):
             'domain',
             'not.too.loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong',
             '3com.com',
+            'w3.example.com',
             'MYDOMAIN.NET',
         ]
         for domain in test_domains:
Repository: deis/deis

Issue:
(builder) Can't set task name /dev/mapper/docker-202:25-3667816-pool
If the builder is restarted on the same node the following (fatal) error is thrown:
`Can't set task name /dev/mapper/docker-202:25-3667816-pool`

I'm a bit unfamiliar with how the device mapper works. I suspect the error may be occurring because a loopback device is being setup in ExecStartPre but not cleaned up in ExecStopPost.
https://github.com/deis/deis/blob/master/deisctl/units/deis-builder.service#L10

This is likely related:
https://github.com/docker/docker/issues/4639



Discussion:

Resolving patch:
diff --git a/contrib/ec2/gen-json.py b/contrib/ec2/gen-json.py
--- a/contrib/ec2/gen-json.py
+++ b/contrib/ec2/gen-json.py
@@ -14,7 +14,7 @@
   Type=oneshot
   RemainAfterExit=yes
   ExecStart=/usr/sbin/wipefs -f /dev/xvdb
-  ExecStart=/usr/sbin/mkfs.ext4 /dev/xvdb
+  ExecStart=/usr/sbin/mkfs.ext4 -i 4096 -b 4096 /dev/xvdb
   ExecStart=/bin/touch /etc/ephemeral-volume-formatted
 '''
 MOUNT_EPHEMERAL_VOLUME = '''
@@ -47,7 +47,7 @@
   Type=oneshot
   RemainAfterExit=yes
   ExecStart=/usr/sbin/wipefs -f /dev/xvdf
-  ExecStart=/usr/sbin/mkfs.btrfs -f /dev/xvdf
+  ExecStart=/usr/sbin/mkfs.ext4 -i 4096 -b 4096 /dev/xvdf
   ExecStart=/bin/touch /etc/docker-volume-formatted
 '''
 MOUNT_DOCKER_VOLUME = '''
@@ -59,7 +59,7 @@
   [Mount]
   What=/dev/xvdf
   Where=/var/lib/docker
-  Type=btrfs
+  Type=ext4
 '''
 
 new_units = [


Test patch:
Repository: deis/deis

Issue:
Deis domains:remove removing all domains
Hello and Good day guys and thanks for the great work

Am experiencing a strange behaviour in v1.5
when i add two domains to my app using the 
`deis domains:add my1.example.com`
`deis domains:add my2.example.com`
When I delete one
`deis domains:remove my2.example.com`
both domains are deleted
Is this an expected behaviour?



Discussion:
@babarinde What version of Deis are you running? This sounds like #3308 which should be fixed in Deis v1.5.0 (and v1.5.1).

@carmstrong Deis v1.5.0 upgraded from v1.4

Note to self: uncomment the skipped test we wrote to validate the fix I'm going to write.


Resolving patch:
diff --git a/controller/api/views.py b/controller/api/views.py
--- a/controller/api/views.py
+++ b/controller/api/views.py
@@ -218,7 +218,8 @@ class DomainViewSet(AppResourceViewSet):
     serializer_class = serializers.DomainSerializer
 
     def get_object(self, **kwargs):
-        return self.get_queryset(**kwargs)
+        qs = self.get_queryset(**kwargs)
+        return qs.get(domain=self.kwargs['domain'])
 
 
 class CertificateViewSet(BaseDeisViewSet):


Test patch:
diff --git a/controller/api/tests/test_domain.py b/controller/api/tests/test_domain.py
--- a/controller/api/tests/test_domain.py
+++ b/controller/api/tests/test_domain.py
@@ -7,7 +7,6 @@
 from __future__ import unicode_literals
 
 import json
-import unittest
 
 from django.contrib.auth.models import User
 from django.test import TestCase
@@ -102,7 +101,6 @@ def test_delete_domain_does_not_remove_latest(self):
         with self.assertRaises(Domain.DoesNotExist):
             Domain.objects.get(domain=test_domains[0])
 
-    @unittest.skip(".get_object() returns all domains instead of the requested domain")
     def test_delete_domain_does_not_remove_others(self):
         """https://github.com/deis/deis/issues/3475"""
         self.test_delete_domain_does_not_remove_latest()
Repository: deis/deis

Issue:
Use python 2.7.9 or urllib3 cert setup code in client
We should fix the way we build the `deis` CLI to deal with the potential issue mentioned in #3514 to ensure that `urllib3` can't run into that cert handling bug.

Refs #3512.



Discussion:

Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -72,6 +72,7 @@
 import requests
 from tabulate import tabulate
 from termcolor import colored
+import urllib3.contrib.pyopenssl
 
 __version__ = '1.6.0-dev'
 
@@ -80,6 +81,7 @@
 
 
 locale.setlocale(locale.LC_ALL, '')
+urllib3.contrib.pyopenssl.inject_into_urllib3()
 
 
 class Session(requests.Session):
diff --git a/client/setup.py b/client/setup.py
--- a/client/setup.py
+++ b/client/setup.py
@@ -27,6 +27,11 @@
     KWARGS = {'scripts': ['deis']}
 
 
+with open('requirements.txt') as f:
+    required = f.read().splitlines()
+    required = [r for r in required if r.strip() and not r.startswith('#')]
+
+
 setup(name='deis',
       version='1.6.0-dev',
       license=APACHE_LICENSE,
@@ -56,10 +61,6 @@
           ('.', ['README.rst']),
       ],
       long_description=LONG_DESCRIPTION,
-      install_requires=[
-          'docopt==0.6.2', 'python-dateutil==2.4.2',
-          'PyYAML==3.11', 'requests==2.5.1',
-          'tabulate==0.7.4', 'termcolor==1.1.0'
-      ],
+      install_requires=required,
       zip_safe=True,
       **KWARGS)


Test patch:
diff --git a/tests/bin/build-deis-cli.sh b/tests/bin/build-deis-cli.sh
deleted file mode 100755
--- a/tests/bin/build-deis-cli.sh
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/sh
-#
-# Creates a python virtual environment and builds the `deis` client binary with it.
-
-virtualenv --system-site-packages venv
-. venv/bin/activate
-pip install docopt==0.6.2 python-dateutil==2.4.1 PyYAML==3.11 requests==2.5.1 git+https://github.com/pyinstaller/pyinstaller@7413317 termcolor==1.1.0
-make -C client/ client
Repository: deis/deis

Issue:
Error with `deis certs:remove`
Getting the following error when trying to remove a cert.

```
$ deis certs:remove '*.brandfolder.com'
Removing *.brandfolder.com... 405 METHOD NOT ALLOWED
Detail:
Method 'DELETE' not allowed.
```



Discussion:
can confirm with the following patch. We don't support wildcard certificates right now, but you should be able to delete them!

```
diff --git a/controller/api/tests/test_certificate.py b/controller/api/tests/test_certificate.py
index 18fa891..9f54985 100644
--- a/controller/api/tests/test_certificate.py
+++ b/controller/api/tests/test_certificate.py
@@ -110,3 +110,11 @@ thejiQz0ThCMBw7QMpVOiSvYAlQG0ATsRYwdTDqENIWKlerOLCSuxmbqe8XeDKhq
         url = '/v1/certs/autotest.example.com'
         response = self.client.delete(url, HTTP_AUTHORIZATION='token {}'.format(self.token))
         self.assertEqual(response.status_code, 204)
+        # deleting a wildcard cert should work too (even though they're unsupported right now)
+        # https://github.com/deis/deis/issues/3533
+        Certificate.objects.create(owner=self.user,
+                                   common_name='*.example.com',
+                                   certificate=self.autotest_example_com_cert)
+        url = '/v1/certs/*.example.com'
+        response = self.client.delete(url, HTTP_AUTHORIZATION='token {}'.format(self.token))
+        self.assertEqual(response.status_code, 204)
```

I was just hacking up the same test to see, thanks @bacongobbler. Will fix.


Resolving patch:
diff --git a/controller/api/urls.py b/controller/api/urls.py
--- a/controller/api/urls.py
+++ b/controller/api/urls.py
@@ -82,7 +82,7 @@
         views.AdminPermsViewSet.as_view({'delete': 'destroy'})),
     url(r'^admin/perms/?',
         views.AdminPermsViewSet.as_view({'get': 'list', 'post': 'create'})),
-    url(r'^certs/(?P<common_name>[-_.\w]+)/?'.format(settings.APP_URL_REGEX),
+    url(r'^certs/(?P<common_name>[-_*.\w]+)/?'.format(settings.APP_URL_REGEX),
         views.CertificateViewSet.as_view({'get': 'retrieve', 'delete': 'destroy'})),
     url(r'^certs/?',
         views.CertificateViewSet.as_view({'get': 'list', 'post': 'create'})),


Test patch:
diff --git a/controller/api/tests/test_certificate.py b/controller/api/tests/test_certificate.py
--- a/controller/api/tests/test_certificate.py
+++ b/controller/api/tests/test_certificate.py
@@ -110,3 +110,11 @@ def test_delete_certificate(self):
         url = '/v1/certs/autotest.example.com'
         response = self.client.delete(url, HTTP_AUTHORIZATION='token {}'.format(self.token))
         self.assertEqual(response.status_code, 204)
+        # deleting a wildcard cert should work too (even though they're unsupported right now)
+        # https://github.com/deis/deis/issues/3533
+        Certificate.objects.create(owner=self.user,
+                                   common_name='*.example.com',
+                                   certificate=self.autotest_example_com_cert)
+        url = '/v1/certs/*.example.com'
+        response = self.client.delete(url, HTTP_AUTHORIZATION='token {}'.format(self.token))
+        self.assertEqual(response.status_code, 204)
Repository: deis/deis

Issue:
fix(client): releases list date format is too wide
When listing release history that includes older events, we fall back to locale-specific date/time.  When this happens, the width of the command output is wrong/ugly:

```
$ deis releases
=== example-go Releases
v9      2 minutes ago            gabrtv deleted limit for cmd
v8      Sun Aug  3 20:24:45 2014 MDT gabrtv deleted limit for worker
v7      Sun Aug  3 20:12:00 2014 MDT gabrtv added limit for worker
v6      Sun Aug  3 20:10:51 2014 MDT gabrtv changed nothing
v5      Sun Aug  3 20:09:49 2014 MDT gabrtv changed limit for cmd
v4      Sun Aug  3 20:07:33 2014 MDT gabrtv added limit for cmd
v3      Sun Aug  3 20:07:13 2014 MDT gabrtv deployed gabrtv/example-go
v2      Sun Aug  3 20:03:49 2014 MDT gabrtv deployed gabrtv/example-go
v1      Sun Aug  3 20:03:43 2014 MDT gabrtv created initial release
```



Discussion:
@gabrtv is this still an issue in master?

I suspect it is.  We can modify the timestamp on a `Release` to test.


Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -2205,7 +2205,7 @@ def releases_list(self, args):
             data = response.json()
             for item in data['results']:
                 item['created'] = readable_datetime(item['created'])
-                self._logger.info("v{version:<6} {created:<24} {summary}".format(**item))
+                self._logger.info("v{version:<6} {created:<28} {summary}".format(**item))
         else:
             raise ResponseError(response)
 


Test patch:
Repository: deis/deis

Issue:
deis config:pull and config:push fail on empty lines
Whenever an empty line is present in the .env file (like at the end), the two commands `deis config:pull` and `deis config:push` fail in a cryptic way.

Just skip empty lines, please.

```
$ deis --version
1.6.0
```

It behaves like this:

```
[cimnine@local myapp]$ cat .env
bla

[cimnine@local myapp]$ deis config:push
Usage: deis config:push [options]
[cimnine@local myapp]$ deis config:pull
Traceback (most recent call last):
  File "<string>", line 2466, in <module>
  File "<string>", line 2462, in main
  File "<string>", line 2395, in _dispatch_cmd
  File "<string>", line 1332, in config_pull
IndexError: list index out of range
```



Discussion:

Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -1302,6 +1302,21 @@ def config_unset(self, args):
         else:
             raise ResponseError(response)
 
+    def _read_config_from_path(self, path):
+        env_dict = {}
+        with open(path, 'r') as f:
+            for line in f:
+                line = line.strip()
+                if not line:
+                    continue
+                if re.match(r'.+=.+', line) is None:
+                    self._logger.warning('could not parse config line: "%s"', line)
+                    continue
+                k, v = line.split('=', 1)
+                env_dict[k] = v
+
+        return env_dict
+
     def config_pull(self, args):
         """
         Extract all environment variables from an application for local use.
@@ -1327,10 +1342,7 @@ def config_pull(self, args):
             app = self._session.app
             try:
                 # load env_dict from existing .env, if it exists
-                with open('.env') as f:
-                    for line in f.readlines():
-                        k, v = line.split('=', 1)[0], line.split('=', 1)[1].strip('\n')
-                        env_dict[k] = v
+                env_dict = self._read_config_from_path('.env')
             except IOError:
                 pass
         response = self._dispatch('get', "/v1/apps/{}/config".format(app))
@@ -1374,8 +1386,8 @@ def config_push(self, args):
 
         # read from .env
         try:
-            with open(args.get('--path'), 'r') as f:
-                self._config_set(app, dictify([line.strip() for line in f]))
+            env_dict = self._read_config_from_path(args.get('--path'))
+            self._config_set(app, env_dict)
         except IOError:
             self._logger.error('could not read env from ' + args.get('--path'))
             sys.exit(1)


Test patch: