internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: deis/deis

Issue:
`deis destroy` did not destroy runtime layer in Opscode management console
On Friday, I performed a `deis destroy` on the NodeJS example application. I didn't check the Opscode Management Console to see if everything had closed down. On Monday, I opened up the Opscode console and the runtime layer was still there (screenshot below).

![screen shot 2013-08-26 at 4 05 29 pm](https://f.cloud.github.com/assets/2786310/1029626/3907f580-0e9c-11e3-9077-d6b740269619.png)



Discussion:
@mboersma This looks to me like the API calls failed due to an intermittent Hosted Chef issue.  Knife commands typically have a retry of 5 attempts to workaround this type of thing.  We need to add that.

If for some reason we can't remove the Chef objects, we need to raise an exception.


Resolving patch:
diff --git a/celerytasks/chef.py b/celerytasks/chef.py
--- a/celerytasks/chef.py
+++ b/celerytasks/chef.py
@@ -11,6 +11,7 @@
 import httplib
 import json
 import re
+import time
 import urlparse
 
 from chef_rsa import Key
@@ -22,7 +23,7 @@ def ruby_b64encode(value):
     """
     b64 = base64.b64encode(value)
     for i in xrange(0, len(b64), 60):
-        yield b64[i:i+60]
+        yield b64[i:i + 60]
 
 
 class UTC(datetime.tzinfo):
@@ -116,12 +117,20 @@ def __init__(self, server_url, client_name, client_key):
         self.conn = httplib.HTTPSConnection(self.hostname)
         self.conn.connect()
 
-    def request(self, verb, path, body=''):
+    def request(self, verb, path, body='', attempts=5, interval=5):
         url = self.path + path
         headers = create_authorization(
             self.headers, verb, url, self.client_key, self.client_name, body)
-        self.conn.request(verb, url, body=body, headers=headers)
-        resp = self.conn.getresponse()
+        # retry all chef api requests
+        for _ in range(attempts):
+            self.conn.request(verb, url, body=body, headers=headers)
+            resp = self.conn.getresponse()
+            if resp.status != 500:
+                break
+            time.sleep(interval)
+        else:
+            errmsg = 'Chef API requests failed: {}'.format(path)
+            raise RuntimeError(errmsg)
         return resp.read(), resp.status
 
     def create_databag(self, name):


Test patch:
Repository: deis/deis

Issue:
Update AMIs in all EC2 regions
Our images are behind on many security updates and should be re-published as v0.0.7 versions.



Discussion:

Resolving patch:
diff --git a/api/models.py b/api/models.py
--- a/api/models.py
+++ b/api/models.py
@@ -196,14 +196,14 @@ class Flavor(UuidAuditedModel):
     # Deis-optimized EC2 amis -- with 3.8 kernel, chef 11 deps,
     # and large docker images (e.g. buildstep) pre-installed
     IMAGE_MAP = {
-        'ap-northeast-1': 'ami-a57aeca4',
-        'ap-southeast-1': 'ami-e03a72b2',
-        'ap-southeast-2': 'ami-bd801287',
-        'eu-west-1': 'ami-d9d3cdad',
-        'sa-east-1': 'ami-a7df7bba',
-        'us-east-1': 'ami-e85a2081',
-        'us-west-1': 'ami-ac6942e9',
-        'us-west-2': 'ami-b55ac885',
+        'ap-northeast-1': 'ami-6da8356c',
+        'ap-southeast-1': 'ami-a66f24f4',
+        'ap-southeast-2': 'ami-d5f66bef',
+        'eu-west-1': 'ami-acbf5adb',
+        'sa-east-1': 'ami-f9fd5ae4',
+        'us-east-1': 'ami-69f3bc00',
+        'us-west-1': 'ami-f0695cb5',
+        'us-west-2': 'ami-ea1e82da',
     }
 
     class Meta:


Test patch:
Repository: deis/deis

Issue:
`deis register` should ask for password confirmation
Currently, the deis client (latest dev version) does not ask to confirm for your password. We should have a password confirmation prompt to ensure that they didn't spell it incorrectly.



Discussion:
+1

+9

Basic stuff.

We might mimic the prompts of `python manage.py createsuperuser` since we're really doing a Django user registration.


Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -574,6 +574,10 @@ def auth_register(self, args):
         password = args.get('--password')
         if not password:
             password = getpass('password: ')
+            confirm = getpass('password (confirm): ')
+            if password != confirm:
+                print('Password mispatch, aborting registration.')
+                return False
         email = args.get('--email')
         if not email:
             email = raw_input('email: ')


Test patch:
Repository: deis/deis

Issue:
Update AMIs in all EC2 regions
Our images are behind on some kernel and security updates and should be re-published as v0.1.0 versions. It's also kind of a performance optimization since we do apt-get upgrade during bootstrap.



Discussion:

Resolving patch:
diff --git a/provider/ec2.py b/provider/ec2.py
--- a/provider/ec2.py
+++ b/provider/ec2.py
@@ -17,14 +17,14 @@
 # Deis-optimized EC2 amis -- with 3.8 kernel, chef 11 deps,
 # and large docker images (e.g. buildstep) pre-installed
 IMAGE_MAP = {
-    'ap-northeast-1': 'ami-6da8356c',
-    'ap-southeast-1': 'ami-a66f24f4',
-    'ap-southeast-2': 'ami-d5f66bef',
-    'eu-west-1': 'ami-acbf5adb',
-    'sa-east-1': 'ami-f9fd5ae4',
-    'us-east-1': 'ami-69f3bc00',
-    'us-west-1': 'ami-f0695cb5',
-    'us-west-2': 'ami-ea1e82da',
+    'ap-northeast-1': 'ami-d95ac4d8',
+    'ap-southeast-1': 'ami-1823694a',
+    'ap-southeast-2': 'ami-e56af7df',
+    'eu-west-1': 'ami-7447a003',
+    'sa-east-1': 'ami-334bec2e',
+    'us-east-1': 'ami-493d6a20',
+    'us-west-1': 'ami-0e2b1f4b',
+    'us-west-2': 'ami-72e27c42',
 }
 
 


Test patch:
Repository: deis/deis

Issue:
500 error isn't helpful when Rackspace image is missing
There is code in providers/rackspace.py, for example, that formats and raises an explicit and helpful EnvironmentError in build_node(). All the CLI ever sees is 500 INTERNAL SERVER ERROR, however. Somewhere between Django, Celery, and the CLi some informative detail is lost.



Discussion:
Django has good justifications for not leaking 500 error content in general. Instead we should probably handle this case-by-case, raising EnvironmentErrors or the like and then returning those as 40x HTTP responses explicitly from views.


Resolving patch:
diff --git a/api/views.py b/api/views.py
--- a/api/views.py
+++ b/api/views.py
@@ -174,8 +174,8 @@ def scale(self, request, **kwargs):
         formation = self.get_object()
         try:
             databag = models.Node.objects.scale(formation, new_structure)
-        except models.Layer.DoesNotExist as e:
-            return Response(str(e), status=status.HTTP_400_BAD_REQUEST)
+        except (models.Layer.DoesNotExist, EnvironmentError) as err:
+            return Response(str(err), status=status.HTTP_400_BAD_REQUEST)
         return Response(databag, status=status.HTTP_200_OK,
                         content_type='application/json')
 
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -1823,7 +1823,7 @@ def main():
     except ResponseError as err:
         resp = err.message
         print('{} {}'.format(resp.status_code, resp.reason))
-        print(resp.text)
+        print(resp.json() or resp.text)
         sys.exit(1)
 
 


Test patch:
Repository: deis/deis

Issue:
Refresh EC2 AMIs for 0.1.1 release
There are a few package updates, esp. Docker 0.6.4. Having the AMIs up-to-date speeds node scaling time somewhat, and it's a slow but mostly-automated process so let's get in the habit of doing this each release.

(Ditto for Rackspace, Vagrant .box images on S3, and DigitalOcean when we get there for each of them...)



Discussion:
ok,i like,favor you!


Resolving patch:
diff --git a/provider/ec2.py b/provider/ec2.py
--- a/provider/ec2.py
+++ b/provider/ec2.py
@@ -17,14 +17,14 @@
 # Deis-optimized EC2 amis -- with 3.8 kernel, chef 11 deps,
 # and large docker images (e.g. buildstep) pre-installed
 IMAGE_MAP = {
-    'ap-northeast-1': 'ami-d95ac4d8',
-    'ap-southeast-1': 'ami-1823694a',
-    'ap-southeast-2': 'ami-e56af7df',
-    'eu-west-1': 'ami-7447a003',
-    'sa-east-1': 'ami-334bec2e',
-    'us-east-1': 'ami-493d6a20',
-    'us-west-1': 'ami-0e2b1f4b',
-    'us-west-2': 'ami-72e27c42',
+    'ap-northeast-1': 'ami-0b45de0a',
+    'ap-southeast-1': 'ami-eaf9b3b8',
+    'ap-southeast-2': 'ami-e9970bd3',
+    'eu-west-1': 'ami-c021c1b7',
+    'sa-east-1': 'ami-b5da7ca8',
+    'us-east-1': 'ami-a30b57ca',
+    'us-west-1': 'ami-30e3d475',
+    'us-west-2': 'ami-ca63fafa',
 }
 
 


Test patch:
Repository: deis/deis

Issue:
Update chef_version in provisioning scripts
I see in the digitalocean support that @bacongobbler removed the --bootstrap-version=11.4.4 and things still seem to work with more current Chef (11.6.2). This wasn't the case before--the apt cookbook failed--so we had pinned it at a working version.

Let's retest that we're compatible with Chef 11.6.x and then remove --bootstrap-version from the provisioning scripts if so.



Discussion:
oops, didn't catch that. Shouldn't we want to pin to a specific version of Chef just for stability reasons, and then update when it's been thoroughly tested? Or are we pinning stuff at a time when the code base is more stable?

I guess it would be smarter to pin it at a more current version, or we'll get burned at some point.


Resolving patch:
diff --git a/cm/chef.py b/cm/chef.py
--- a/cm/chef.py
+++ b/cm/chef.py
@@ -21,7 +21,7 @@
 CHEF_INSTALL_TYPE = 'gems'
 CHEF_RUBY_VERSION = '1.9.1'
 CHEF_ENVIRONMENT = '_default'
-CHEF_CLIENT_VERSION = '11.4.4'
+CHEF_CLIENT_VERSION = '11.6.2'
 
 # load chef config using CHEF_CONFIG_PATH
 try:


Test patch:
Repository: deis/deis

Issue:
Refresh EC2 AMIs for 0.2.0 release
Having the AMIs up-to-date speeds node scaling time somewhat, and it's a slow but mostly-automated process so let's do this each release.

(Ditto for Rackspace, Vagrant .box images on S3, and DigitalOcean when we get there for each of them...)



Discussion:

Resolving patch:
diff --git a/provider/ec2.py b/provider/ec2.py
--- a/provider/ec2.py
+++ b/provider/ec2.py
@@ -17,14 +17,14 @@
 # Deis-optimized EC2 amis -- with 3.8 kernel, chef 11 deps,
 # and large docker images (e.g. buildstep) pre-installed
 IMAGE_MAP = {
-    'ap-northeast-1': 'ami-0b45de0a',
-    'ap-southeast-1': 'ami-eaf9b3b8',
-    'ap-southeast-2': 'ami-e9970bd3',
-    'eu-west-1': 'ami-c021c1b7',
-    'sa-east-1': 'ami-b5da7ca8',
-    'us-east-1': 'ami-a30b57ca',
-    'us-west-1': 'ami-30e3d475',
-    'us-west-2': 'ami-ca63fafa',
+    'ap-northeast-1': 'ami-39bfda38',
+    'ap-southeast-1': 'ami-c0613492',
+    'ap-southeast-2': 'ami-9741ddad',
+    'eu-west-1': 'ami-39bc5e4e',
+    'sa-east-1': 'ami-0775d31a',
+    'us-east-1': 'ami-fa99c193',
+    'us-west-1': 'ami-802412c5',
+    'us-west-2': 'ami-0e7be33e',
 }
 
 


Test patch:
Repository: deis/deis

Issue:
Invalid URL u'/api/auth/register': No schema supplied
Just cloned the repo fresh 30 minutes ago and provisioned a new controller on EC2 and `pip install --upgrade deis` and I got this error when running `deis register`:

``` bash
wintermute:deis andrewdeandrade$ deis --version
Deis CLI 0.1.1
wintermute:deis andrewdeandrade$ deis register ec2-54-241-44-134.us-west-1.compute.amazonaws.com
username: myuser
password: 
password (confirm): 
email: myuser@example.com
Invalid URL u'/api/auth/register': No schema supplied
Usage: deis auth:register <controller> [options]
```



Discussion:
@malandrew you need http:// in front of the URL.. but we should have told you that, so I'm marking this as a bug. :)

Also, the deis CLI should be reporting itself as "0.2.0" if it's up-to-date, although the older client is likely to work anyway.

Should it just assume http? Also what about https support? It looks like this register process is sending both a username and password in the clear.

Furthermore, the security groups being set up on EC2 are wide open. It would be nice if the provision process used either http://checkip.amazonaws.com/ or http://ifconfig.me/ to get the developer's IP address and used `/32` CIDR notation to restrict access to only the IP a developer is working from.


Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -568,6 +568,8 @@ def auth_register(self, args):
         --email=EMAIL          provide an email address
         """
         controller = args['<controller>']
+        if not urlparse.urlparse(controller).scheme:
+            controller = "http://{}".format(controller)
         username = args.get('--username')
         if not username:
             username = raw_input('username: ')
@@ -635,6 +637,8 @@ def auth_login(self, args):
         Usage: deis auth:login <controller> [--username=<username> --password=<password>]
         """
         controller = args['<controller>']
+        if not urlparse.urlparse(controller).scheme:
+            controller = "http://{}".format(controller)
         username = args.get('--username')
         headers = {}
         if not username:


Test patch:
Repository: deis/deis

Issue:
Erroneous prompt after `deis nodes:scale <formationName> runtime=0`
The prompt after `deis nodes:scale <formationName> runtime=0` says you can use `deis create --formation=<formationName>` to create an application, but in truth, if you do this, it will throw you an error.

Maybe it should say "Use `deis nodes:scale <formationName> runtime=1` to create a runtime layer so that you can create an application."

```
deis nodes:scale clojureFormation runtime=0
Scaling nodes... but first, coffee!
done in 20s

Use `deis create --formation=clojureFormation` to create an application
```



Discussion:

Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -1535,8 +1535,11 @@ def nodes_scale(self, args):
         """
         formation = args.get('<formation>')
         body = {}
+        runtimes = True
         for type_num in args.get('<type=num>'):
             typ, count = type_num.split('=')
+            if (typ, count) == ('runtime', '0'):
+                runtimes = False
             body.update({typ: int(count)})
         print('Scaling nodes... but first, coffee!')
         try:
@@ -1551,7 +1554,9 @@ def nodes_scale(self, args):
             progress.join()
         if response.status_code == requests.codes.ok:  # @UndefinedVariable
             print('done in {}s\n'.format(int(time.time() - before)))
-            print('Use `deis create --formation={}` to create an application'.format(formation))
+            if runtimes:
+                print('Use `deis create --formation={}` to create an application'.format(
+                      formation))
         else:
             raise ResponseError(response)
 


Test patch:
Repository: deis/deis

Issue:
Error provisioning new controller: ".host_nodes_dir"
I provisioned a new controller on DigitalOcean and hit this error during initial registration. We need to loosen up the Vagrant config requirements so this doesn't show up.

``` console
[2013-11-07 16:02:37,088: INFO/MainProcess] consumer: Connected to amqp://guest@127.0.0.1:5672//.

==> /var/log/deis/server.log <==
    provider = import_provider_module(provider_type)
  File "/opt/deis/controller/provider/__init__.py", line 8, in import_provider_module
    tasks = importlib.import_module('provider.' + provider_type)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/opt/deis/controller/provider/vagrant.py", line 18, in <module>
    HOST_NODES_DIR = open('/home/vagrant/.host_nodes_dir').read().strip()
IOError: [Errno 2] No such file or directory: u'/home/vagrant/.host_nodes_dir'
WARNING Forbidden (CSRF cookie not set.): /api/auth/register
2013-11-07 16:29:55 [28407] [DEBUG] Closing connection. 
```



Discussion:

Resolving patch:
diff --git a/provider/vagrant.py b/provider/vagrant.py
--- a/provider/vagrant.py
+++ b/provider/vagrant.py
@@ -7,6 +7,7 @@
 from api.ssh import exec_ssh, connect_ssh
 
 import json
+import logging
 import string
 import subprocess
 import uuid
@@ -14,20 +15,25 @@
 from api.models import Layer
 from api.models import Node
 
+logger = logging.getLogger(__name__)
+
 # Collect details for connecting to the host machine
-HOST_NODES_DIR = open('/home/vagrant/.host_nodes_dir').read().strip()
-PKEY = open('/home/vagrant/.ssh/id_rsa').read()
+try:
+    HOST_NODES_DIR = open('/home/vagrant/.host_nodes_dir').read().strip()
+    PKEY = open('/home/vagrant/.ssh/id_rsa').read()
+except IOError as err:
+    logger.warn(err)
 
 
 def seed_flavors():
-    """Seed the database with default flavors for each Rackspace region.
+    """Seed the database with default flavors for vagrant.
 
     :rtype: list of dicts containing flavor data
     """
     flavors = []
     for m in ['512', '1024', '2048']:
         flavors.append({
-            'id': 'vagrant-{}'.format(m),
+            'id': "vagrant-{}".format(m),
             'provider': 'vagrant',
             'params': json.dumps({
                 'memory': m


Test patch:
Repository: deis/deis

Issue:
404 on cm.destroy should be a warning.
It's all too easy to run into the case where something errored in formations:create and we have a Django database entry for the formation, but no Chef node. The recent merge makes it impossible to delete the Formation in that case. If we instead treat the Chef 404 as a warning, this will work.



Discussion:

Resolving patch:
diff --git a/cm/chef.py b/cm/chef.py
--- a/cm/chef.py
+++ b/cm/chef.py
@@ -142,11 +142,11 @@ def purge_node(node):
     client = _get_client()
     node_id = node['id']
     body, status = client.delete_node(node_id)
-    if status != 200:
-        raise RuntimeError('Could not purge node {node_id}: {body}'.format(**locals()))
+    if status not in [200, 404]:
+        raise RuntimeError("Could not purge node {node_id}: {body}".format(**locals()))
     body, status = client.delete_client(node_id)
-    if status != 200:
-        raise RuntimeError('Could not purge node client {node_id}: {body}'.format(**locals()))
+    if status not in [200, 404]:
+        raise RuntimeError("Could not purge node client {node_id}: {body}".format(**locals()))
 
 
 def converge_controller():
@@ -312,6 +312,6 @@ def _purge(databag_name, item_name):
     """
     client = _get_client()
     body, status = client.delete_databag_item(databag_name, item_name)
-    if status == 200 or status == 404:
+    if status in [200, 404]:
         return body, status
     raise RuntimeError('Could not purge {item_name}: {body}'.format(**locals()))


Test patch:
Repository: deis/deis

Issue:
`deis run` generates ugly error if app code not yet pushed
I ran `deis run ls -la` after I had created the app, but before I had pushed the code with `git push deis master`. Here is the error I received:

```
ben$ example-python-flask > deis run ls -la
Warning: non-zero return code 255
lxc-start: No such file or directory - failed to mount '/opt/deis/runtime/slugs/hushed-sailfish-1/app' on '/usr/lib/lxc/root///app'
lxc-start: failed to setup the mount entries for '5f4f4d932501338fa2062d52e5893dfbd3933fa09102c67493a169a2a87ee479'
lxc-start: failed to setup the container
lxc-start: invalid sequence number 1. expected 2
lxc-start: failed to spawn '5f4f4d932501338fa2062d52e5893dfbd3933fa09102c67493a169a2a87ee479'
lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/cpuset//lxc/5f4f4d932501338fa2062d52e5893dfbd3933fa09102c67493a169a2a87ee479'
```



Discussion:

Resolving patch:
diff --git a/api/tasks.py b/api/tasks.py
--- a/api/tasks.py
+++ b/api/tasks.py
@@ -53,6 +53,8 @@ def converge_node(node):
 @task
 def run_node(node, command):
     output, rc = CM.run_node(node.flat(), command)
+    if rc != 0 and 'failed to setup the container' in output:
+        output = '\033[35mPlease run `git push deis master` first.\033[0m\n' + output
     return output, rc
 
 


Test patch:
Repository: deis/deis

Issue:
`des auth:cancel` leaves entry in deis-users databag
We saw this when setting up the CLI tests. Need to hook up a cm.purge_user to the appropriate model signal.



Discussion:

Resolving patch:
diff --git a/api/models.py b/api/models.py
--- a/api/models.py
+++ b/api/models.py
@@ -15,6 +15,7 @@
 from django.conf import settings
 from django.contrib.auth.models import User
 from django.db import models
+from django.db.models.signals import post_delete
 from django.db.models.signals import post_save
 from django.dispatch import receiver
 from django.dispatch.dispatcher import Signal
@@ -818,10 +819,15 @@ def _user_publish(self):
     CM.publish_user(self.flat(), self.calculate())
 
 
+def _user_purge(self):
+    CM.purge_user(self.flat())
+
+
 # attach to built-in django user
 User.flat = _user_flat
 User.calculate = _user_calculate
 User.publish = _user_publish
+User.purge = _user_purge
 
 # define update/delete callbacks for synchronizing
 # models with the configuration management backend
@@ -836,8 +842,14 @@ def _publish_user_to_cm(**kwargs):
         return
     kwargs['instance'].publish()
 
+
+def _purge_user_from_cm(**kwargs):
+    kwargs['instance'].purge()
+
+
 # use django signals to synchronize database updates with
 # the configuration management backend
 post_save.connect(_publish_to_cm, sender=App, dispatch_uid='api.models')
 post_save.connect(_publish_to_cm, sender=Formation, dispatch_uid='api.models')
 post_save.connect(_publish_user_to_cm, sender=User, dispatch_uid='api.models')
+post_delete.connect(_purge_user_from_cm, sender=User, dispatch_uid='api.models')
diff --git a/cm/chef.py b/cm/chef.py
--- a/cm/chef.py
+++ b/cm/chef.py
@@ -236,6 +236,17 @@ def publish_user(user, data):
     _publish('deis-users', user['username'], data)
 
 
+def purge_user(user):
+    """
+    Purge a user from configuration management.
+
+    :param app: a dict containing the username of the user
+    :returns: a tuple of (body, status) from the underlying HTTP response
+    :raises: RuntimeError
+    """
+    _purge('deis-users', user['username'])
+
+
 def publish_app(app, data):
     """
     Publish an app to configuration management.


Test patch: