internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: deis/deis

Issue:
Some object API failures still create DB entries
When I tried to deploy a dev cluster, I tried doing so without running `deis providers:discover` and `deis keys:add` first, which was fine. However, after resolving that issue, I get an error saying that this layer was already created:

```
(venv)><> deis layers:create dev nodes digitalocean-new-york-1 --proxy=y --runtime=y
Creating nodes layer... 400 BAD REQUEST
{u'__all__': [u'Layer with this Formation and Id already exists.']}
```

Looking aside from the raw unicode strings, this should not be happening. It should create the layer only if the run was complete.

I have two proposals to fix this:
- catch the user with his pants down before running `layers:create` ("did you forget to run `deis keys:add`/`deis providers:discover`?")
- rollback the DB transaction on a server error

Happy to contribute this fix.



Discussion:
We have a similar issue on formations and nodes create operations. Need to investigate default transaction semantics in views, maybe use django's transaction.atomic() decorator.

https://docs.djangoproject.com/en/dev/topics/db/transactions/#tying-transactions-to-http-requests
ATOMIC_REQUESTS = True
New in 1.6, this used to be handled by TransactionMiddleware is my excuse for not knowing about it. :)


Resolving patch:
diff --git a/api/views.py b/api/views.py
--- a/api/views.py
+++ b/api/views.py
@@ -9,7 +9,6 @@
 from Crypto.PublicKey import RSA
 from django.contrib.auth.models import AnonymousUser
 from django.contrib.auth.models import User
-from django.db import transaction
 from django.utils import timezone
 from guardian.shortcuts import assign_perm
 from guardian.shortcuts import get_objects_for_user
@@ -626,11 +625,10 @@ def rollback(self, request, *args, **kwargs):
         if version < 1:
             return Response(status=status.HTTP_404_NOT_FOUND)
         prev = app.release_set.get(version=version)
-        with transaction.atomic():
-            summary = "{} rolled back to v{}".format(request.user, version)
-            app.release_set.create(owner=request.user, version=last_version + 1,
-                                   build=prev.build, config=prev.config,
-                                   summary=summary)
+        summary = "{} rolled back to v{}".format(request.user, version)
+        app.release_set.create(owner=request.user, version=last_version + 1,
+                               build=prev.build, config=prev.config,
+                               summary=summary)
         # publish release to registry as new docker image
         if settings.REGISTRY_URL:
             repository_path = "{}/{}".format(app.owner.username, app.id)
diff --git a/deis/settings.py b/deis/settings.py
--- a/deis/settings.py
+++ b/deis/settings.py
@@ -329,3 +329,6 @@
     from .local_settings import *  # @UnusedWildImport # noqa
 except ImportError:
     pass
+
+# Ensure that atomic DB transactions per view are enabled
+DATABASES['default']['ATOMIC_REQUESTS'] = True


Test patch:
Repository: deis/deis

Issue:
Make EC2 AMIs for 0.6.0 release



Discussion:

Resolving patch:
diff --git a/controller/provider/ec2.py b/controller/provider/ec2.py
--- a/controller/provider/ec2.py
+++ b/controller/provider/ec2.py
@@ -17,14 +17,14 @@
 # Deis-optimized EC2 amis -- with 3.8 kernel, chef 11 deps,
 # and large docker images (e.g. buildstep) pre-installed
 IMAGE_MAP = {
-    'ap-northeast-1': 'ami-59007158',
-    'ap-southeast-1': 'ami-4cb4e51e',
-    'ap-southeast-2': 'ami-6d5bc257',
-    'eu-west-1': 'ami-c3ef12b4',
-    'sa-east-1': 'ami-3b45e626',
-    'us-east-1': 'ami-d90408b0',
-    'us-west-1': 'ami-9c3906d9',
-    'us-west-2': 'ami-8c6a05bc',
+    'ap-northeast-1': 'ami-ae85ec9e',
+    'ap-southeast-1': 'ami-904919c2',
+    'ap-southeast-2': 'ami-a9db4393',
+    'eu-west-1': 'ami-01eb1576',
+    'sa-east-1': 'ami-d3cc6ece',
+    'us-east-1': 'ami-51382c38',
+    'us-west-1': 'ami-ec0d33a9',
+    'us-west-2': 'ami-a085ec90',
 }
 
 


Test patch:
Repository: deis/deis

Issue:
nodes:scale fails on Vagrant
Looks like a path issue when we shifted around the project in #641. The problem that exists here though is that the `contrib` directory no longer exists in the `deis/controller` image, since it only adds the `controller/` path as per https://github.com/opdemand/deis/blob/master/controller/Dockerfile#L19. Should we move `contrib` to `controller/contrib`, so that the directory is present in the docker image?

```
><> deis nodes:scale dev runtime=1
Scaling nodes... but first, coffee!
400 BAD REQUEST
[Errno 2] No such file or directory: u'/app/deis/contrib/vagrant/util/nodes_vagrantfile_template.rb'
```

Note that the source code now exists at `/app` in the container, so there will have to be another PR to change the path hardcoded at https://github.com/opdemand/deis/blob/master/controller/provider/vagrant.py#L82.



Discussion:
The `contrib` dir really ought to stay in the project root I think. Can we work around this with an additional ADD command in the Dockerfile?

There's no way to be able to add directories outside of the relative directory by design.

```
Step 8 : ADD .. /deis
2014/03/23 09:20:59 Forbidden path outside the build context: .. (/mnt/sda1/tmp)
```


Resolving patch:
diff --git a/controller/provider/vagrant.py b/controller/provider/vagrant.py
--- a/controller/provider/vagrant.py
+++ b/controller/provider/vagrant.py
@@ -17,12 +17,12 @@
 
 logger = logging.getLogger(__name__)
 
-CONTRIB_PATH = '/app/deis/contrib/vagrant'
+VAGRANT_UTIL_PATH = '/app/provider/vagrant-util'
 
 # Collect details for connecting to the host machine
 try:
-    HOST_NODES_DIR = open('{}/.host_nodes_dir'.format(CONTRIB_PATH)).read().strip()
-    PKEY = open('{}/util/ssh_keys/id_rsa_vagrant-deis-controller'.format(CONTRIB_PATH)).read()
+    HOST_NODES_DIR = open('{}/.host_nodes_dir'.format(VAGRANT_UTIL_PATH)).read().strip()
+    PKEY = open('{}/ssh_keys/id_rsa_vagrant-deis-controller'.format(VAGRANT_UTIL_PATH)).read()
 except IOError as err:
     logger.warn(err)
 
@@ -79,7 +79,7 @@ def build_node(node):
 
     # Create a new Vagrantfile from a template
     node['params'].setdefault('memory', '512')
-    template = open('/app/deis/contrib/vagrant/util/nodes_vagrantfile_template.rb')
+    template = open('{}/nodes_vagrantfile_template.rb'.format(VAGRANT_UTIL_PATH))
     raw = string.Template(template.read())
     ip_addr = '192.168.61.' + str(Node.objects.all().count() + 100)
     result = raw.substitute({


Test patch:
Repository: deis/deis

Issue:
Client: Allow creating apps without adding a git remote
I'm using multiple apps to run the same code with different environment settings (staging, beta, prod). Today creating those apps is a bit cumbersome as deis requires a `git remote` to be successfully created.

A flag to disable the git remote creation when creating an app would be really helpful.



Discussion:
As a workaround for the time being, you can rename the remotes to your liking before running `deis create` e.g. `git remote rename deis deis-prod`.

Yes, I use `deis create --id=x && git remote rm deis` today as I don't need the deis remote, because travis is deploying the actual app now for us https://github.com/travis-ci/dpl/pull/120


Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -417,11 +417,12 @@ def apps_create(self, args):
         If no ID is provided, one will be generated automatically.
         If no cluster is provided, a cluster named "dev" will be used.
 
-        Usage: deis apps:create [<id> --cluster=<cluster>] [options]
+        Usage: deis apps:create [<id> --cluster=<cluster> --no-remote] [options]
 
         Options
 
         --cluster=CLUSTER      target cluster to host application [default: dev]
+        --no-remote            do not create a 'deis' git remote
         """
         try:
             self._session.git_root()  # check for a git repository
@@ -460,9 +461,12 @@ def apps_create(self, args):
             hostname = urlparse.urlparse(self._settings['controller']).netloc.split(':')[0]
             git_remote = "ssh://git@{hostname}:2222/{app_id}.git".format(**locals())
             try:
-                subprocess.check_call(
-                    ['git', 'remote', 'add', '-f', 'deis', git_remote],
-                    stdout=subprocess.PIPE)
+                if args.get('--no-remote'):
+                    print('remote available at {}'.format(git_remote))
+                else:
+                    subprocess.check_call(
+                        ['git', 'remote', 'add', '-f', 'deis', git_remote],
+                        stdout=subprocess.PIPE)
             except subprocess.CalledProcessError:
                 sys.exit(1)
             print('Git remote deis added')


Test patch:
Repository: deis/deis

Issue:
services remain listed in fleetctl after app is deleted
``` console
$ deis destroy

 !    WARNING: Potentially Destructive Action
 !    This command will destroy the application: bolder-farmland
 !    To proceed, type "bolder-farmland" or re-run this command with --confirm=bolder-farmland

> bolder-farmland
Destroying bolder-farmland... done in 0s
Git remote deis removed
$ fleetctl list-units
UNIT                        LOAD    ACTIVE  SUB DESC                    MACHINE
bolder-farmland_v2.web.1-announce.service   -   -   -   bolder-farmland_v2.web.1 announce   -
bolder-farmland_v2.web.1-log.service        -   -   -   bolder-farmland_v2.web.1 log        -
bolder-farmland_v2.web.1.service        -   -   -   bolder-farmland_v2.web.1        -
bolder-farmland_v3.web.1-announce.service   -   -   -   bolder-farmland_v3.web.1 announce   -
bolder-farmland_v3.web.1-log.service        -   -   -   bolder-farmland_v3.web.1 log        -
bolder-farmland_v3.web.1.service        -   -   -   bolder-farmland_v3.web.1        -
deis-builder.service                loaded  active  running deis-builder                304bff27.../172.17.8.100
deis-cache.service              loaded  active  running deis-cache              304bff27.../172.17.8.100
deis-controller.service             loaded  active  running deis-controller             304bff27.../172.17.8.100
deis-database.service               loaded  active  running deis-database               304bff27.../172.17.8.100
deis-logger.service             loaded  active  running deis-logger             304bff27.../172.17.8.100
deis-registry.service               loaded  active  running deis-registry               304bff27.../172.17.8.100
deis-router.service             loaded  active  running deis-router             304bff27.../172.17.8.100
```

We need to wire up `FleetClient.destroy()` to `api.models.Container` destruction.



Discussion:

Resolving patch:
diff --git a/controller/api/models.py b/controller/api/models.py
--- a/controller/api/models.py
+++ b/controller/api/models.py
@@ -121,9 +121,10 @@ def create(self, *args, **kwargs):
         build = Build.objects.create(owner=self.owner, app=self, image=settings.DEFAULT_BUILD)
         Release.objects.create(version=1, owner=self.owner, app=self, config=config, build=build)
 
-    def destroy(self, *args, **kwargs):
+    def delete(self, *args, **kwargs):
         for c in self.container_set.all():
             c.destroy()
+        return super(App, self).delete(*args, **kwargs)
 
     def deploy(self, release):
         tasks.deploy_release.delay(self, release).get()


Test patch:
Repository: deis/deis

Issue:
applications are failing to start
My fault for not properly testing #834.

```
core@deis-1 ~ $ sudo systemctl status decent-painting_v2.web.1.service
● decent-painting_v2.web.1.service - decent-painting_v2.web.1
   Loaded: loaded (/run/systemd/system/decent-painting_v2.web.1.service; static)
   Active: inactive (dead)

Apr 29 21:49:32 deis-1 systemd[1]: Starting decent-painting_v2.web.1...
Apr 29 21:49:34 deis-1 docker[27101]: Pulling repository 10.0.2.15:5000/bacongobbler/decent-painting
Apr 29 21:49:46 deis-1 systemd[1]: Started decent-painting_v2.web.1.
Apr 29 21:49:46 deis-1 docker[27713]: cat: invalid option -- 'f'
Apr 29 21:49:46 deis-1 docker[27713]: Try `cat --help' for more information.
Apr 29 21:49:47 deis-1 docker[27754]: decent-painting_v2.web.1
core@deis-1 ~ $ sudo systemctl cat decent-painting_v2.web.1.service
# /run/systemd/system/decent-painting_v2.web.1.service

[Unit]
Description=decent-painting_v2.web.1
After=docker.service
Requires=docker.service

[Service]
ExecStartPre=/usr/bin/docker pull 10.0.2.15:5000/bacongobbler/decent-painting:v2
ExecStart=-/usr/bin/docker run --name decent-painting_v2.web.1 -P -e PORT=5000 10.0.2.15:5000/bacongobbler/decent-painting:v2 cat Procfile | grep ^web | cut -f 1 -d ' ' --complement | sh -
ExecStop=-/usr/bin/docker rm -f decent-painting_v2.web.1
```

Seems to be a pipe failure in systemd unit files, so I'm gonna patch this up.



Discussion:

Resolving patch:
diff --git a/controller/api/models.py b/controller/api/models.py
--- a/controller/api/models.py
+++ b/controller/api/models.py
@@ -258,7 +258,11 @@ def _get_scheduler(self):
     def _get_command(self):
         c_type = self.type
         if c_type:
-            return "cat Procfile | grep ^{c_type} | cut -f 1 -d ' ' --complement | sh -"
+            # handle special case for Dockerfile deployments
+            if c_type == 'cmd':
+                return ''
+            else:
+                return 'start {c_type}'
         else:
             return ''
 


Test patch:
Repository: deis/deis

Issue:
Make EC2 use contrib/coreos/user-data
We need all providers to use the same user-data file, as that's where our customizations live. Even though the contents of user-data need to be injected into the payload for EC2 as a string, it shouldn't be hard to parse the file locally and string substitute it into deis.template at runtime.



Discussion:

Resolving patch:
diff --git a/contrib/ec2/gen-json.py b/contrib/ec2/gen-json.py
new file mode 100755
--- /dev/null
+++ b/contrib/ec2/gen-json.py
@@ -0,0 +1,13 @@
+#!/usr/bin/env python
+import json
+import os
+
+template = json.load(open("deis.template",'r'))
+
+with open('../coreos/user-data','r') as f:
+  lines = f.readlines()
+
+template['Resources']['CoreOSServerLaunchConfig']['Properties']['UserData']['Fn::Base64']['Fn::Join'] = [ '', lines ]
+template['Parameters']['ClusterSize']['Default'] = str(os.getenv('DEIS_NUM_INSTANCES', 3))
+
+print json.dumps(template)


Test patch:
Repository: deis/deis

Issue:
Client: Allow running a command without git remote
At the moment `deis apps:run` requires a git remote to be configured. This makes managing multiple apps a more complicated.

It would be nice, if  `deis apps:run` could also respects a provided `--app` parameter. 



Discussion:
Good catch. All application-specific commands should have a `--app=<appname>` option.


Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -600,7 +600,9 @@ def apps_run(self, args):
 
         Usage: deis apps:run <command>...
         """
-        app = self._session.app
+        app = args.get('--app')
+        if not app:
+            app = self._session.app
         body = {'command': ' '.join(sys.argv[2:])}
         response = self._dispatch('post',
                                   "/api/apps/{}/run".format(app),


Test patch:
Repository: deis/deis

Issue:
Cannot delete cluster
I created a cluster with a bad key and when I tried to delete it the client threw a 500 error.

Logs below:

```
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: self.pre_delete(obj)
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: File "/app/api/views.py", line 201, in pre_delete
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: cluster.destroy()
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: File "/app/api/models.py", line 99, in destroy
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: return tasks.destroy_cluster.delay(self).get()
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: File "/usr/local/lib/python2.7/dist-packages/celery/result.py", line 152, in get
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: interval=interval)
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: File "/usr/local/lib/python2.7/dist-packages/celery/backends/base.py", line 186, in wait_for
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: raise result
May 07 23:36:58 coreos-test-3ee2b1c1-0845-41db-9fd5-d421a1cfea76 sh[5108]: AttributeError: 'App' object has no attribute 'destroy'
```



Discussion:
Thanks for the bug report, I'll try to reproduce this. Are you using Deis from GitHub master?


Resolving patch:
diff --git a/controller/api/models.py b/controller/api/models.py
--- a/controller/api/models.py
+++ b/controller/api/models.py
@@ -134,6 +134,9 @@ def deploy(self, release):
             self.save()
             self.scale()
 
+    def destroy(self, *args, **kwargs):
+        return self.delete(*args, **kwargs)
+
     def scale(self, **kwargs):
         """Scale containers up or down to match requested."""
         requested_containers = self.structure.copy()


Test patch:
diff --git a/controller/api/tests/test_cluster.py b/controller/api/tests/test_cluster.py
--- a/controller/api/tests/test_cluster.py
+++ b/controller/api/tests/test_cluster.py
@@ -45,6 +45,12 @@ def test_cluster(self):
         response = self.client.get('/api/clusters')
         self.assertEqual(response.status_code, 200)
         self.assertEqual(len(response.data['results']), 1)
+        # ensure we can delete the cluster with an app
+        # see https://github.com/deis/deis/issues/927
+        url = '/api/apps'
+        body = {'cluster': 'autotest'}
+        response = self.client.post(url, json.dumps(body), content_type='application/json')
+        self.assertEqual(response.status_code, 201)
         url = '/api/clusters/{cluster_id}'.format(**locals())
         response = self.client.get(url)
         self.assertEqual(response.status_code, 200)
Repository: deis/deis

Issue:
Better document `clusters:create` parameters (specifically --auth)
We need to better document the parameters to `clusters:create`. It's easy for users to supply the wrong SSH key (insecure Vagrant key? My pubkey? A generated deis key?). We've also had people wonder if all hosts need to be specified. In the README we don't really elaborate on any of the values we're providing, and the docs are short on this, too.

We need to provide better documentation around this.



Discussion:
:+1: 

:thumbsup: 


Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -797,6 +797,13 @@ def clusters_create(self, args):
 
         Usage: deis clusters:create <id> <domain> --hosts=<hosts> --auth=<auth> [options]
 
+        Parameters:
+
+        <id>             a name for the cluster
+        <domain>         a domain under which app hostnames will live
+        <hosts>          a comma-separated list of cluster members
+        <auth>           a path to an SSH private key used to connect to cluster members
+
         Options:
 
         --type=TYPE      cluster type [default: coreos]


Test patch:
Repository: deis/deis

Issue:
fix(client): return HTTP 204 response for no logs
Referring to the OP in #989:

The 404 here is misleading. We found the endpoint, but no logs were found. We should be returning `HTTP 204 NO CONTENT`, not `HTTP 404 NOT FOUND`.

fix(client): return HTTP 204 response for no logs
Referring to the OP in #989:

The 404 here is misleading. We found the endpoint, but no logs were found. We should be returning `HTTP 204 NO CONTENT`, not `HTTP 404 NOT FOUND`.



Discussion:

Resolving patch:
diff --git a/controller/api/views.py b/controller/api/views.py
--- a/controller/api/views.py
+++ b/controller/api/views.py
@@ -299,7 +299,7 @@ def logs(self, request, **kwargs):
             logs = app.logs()
         except EnvironmentError:
             return Response("No logs for {}".format(app.id),
-                            status=status.HTTP_404_NOT_FOUND,
+                            status=status.HTTP_204_NO_CONTENT,
                             content_type='text/plain')
         return Response(logs, status=status.HTTP_200_OK,
                         content_type='text/plain')


Test patch:
diff --git a/controller/api/tests/test_app.py b/controller/api/tests/test_app.py
--- a/controller/api/tests/test_app.py
+++ b/controller/api/tests/test_app.py
@@ -79,7 +79,7 @@ def test_app_actions(self):
             os.remove(path)
         url = '/api/apps/{app_id}/logs'.format(**locals())
         response = self.client.post(url)
-        self.assertEqual(response.status_code, 404)
+        self.assertEqual(response.status_code, 204)
         self.assertEqual(response.data, 'No logs for {}'.format(app_id))
         # write out some fake log data and try again
         with open(path, 'w') as f:
Repository: deis/deis

Issue:
bug(client): deis run requires remote
Executing `deis run --app example` still requires a remote to be setup.

```
Could not find deis remote in `git remote -v`
Usage: deis <command> [<args>...]
```



Discussion:

Resolving patch:
diff --git a/client/deis.py b/client/deis.py
--- a/client/deis.py
+++ b/client/deis.py
@@ -579,12 +579,12 @@ def apps_run(self, args):
         """
         Run a command inside an ephemeral app container
 
-        Usage: deis apps:run <command>...
+        Usage: deis apps:run [--app=<app> --] <command>...
         """
         app = args.get('--app')
         if not app:
             app = self._session.app
-        body = {'command': ' '.join(sys.argv[2:])}
+        body = {'command': ' '.join(args.get('<command>'))}
         response = self._dispatch('post',
                                   "/api/apps/{}/run".format(app),
                                   json.dumps(body))
@@ -1611,11 +1611,10 @@ def main():
         method = getattr(cli, cmd)
     else:
         raise DocoptExit('Found no matching command, try `deis help`')
-    # re-parse docopt with the relevant docstring unless it needs sys.argv
-    if cmd not in ('apps_run',):
-        docstring = trim(getattr(cli, cmd).__doc__)
-        if 'Usage: ' in docstring:
-            args.update(docopt(docstring))
+    # re-parse docopt with the relevant docstring
+    docstring = trim(getattr(cli, cmd).__doc__)
+    if 'Usage: ' in docstring:
+        args.update(docopt(docstring))
     # dispatch the CLI command
     _dispatch_cmd(method, args)
 


Test patch: