internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Miserlou/Zappa

Issue:
CLI always exits with return code 130
Independent of the actual success of a CLI command, the CLI always returns code 130. This causes successful deployments through CIs to fail.



Discussion:

Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -901,7 +901,11 @@ def handle(): # pragma: no cover
     try:
         cli = ZappaCLI()
         sys.exit(cli.handle())
-    except (KeyboardInterrupt, SystemExit): # pragma: no cover
+    except SystemExit as e: # pragma: no cover
+        if cli.zip_path:
+            cli.remove_uploaded_zip()
+        sys.exit(e.code)
+    except KeyboardInterrupt: # pragma: no cover
         if cli.zip_path: # Remove the Zip from S3 upon failure.
             cli.remove_uploaded_zip()
         sys.exit(130)


Test patch:
Repository: Miserlou/Zappa

Issue:
Targets of event rules are duplicated if delete permission is missing
I am experimenting with minimal permissions for the IAM user used by zappa. If the permission for deleting a target/rule is missing, the targets in a schedule are duplicated without that an error is raised. This means, the lambda function gets triggered multiple times simultaneously. 

The issue can be reproduced by restricting the event permissions of the Zappa AWS user to the policy below and run `zappa update`.

```
{
  "Action": [
    "events:PutRule",
    "events:PutTargets",
    "events:ListRules"
  ],
  "Resource": "*",
  "Effect": "Allow"
}
```



Discussion:
Damn, that's a great bug. 

Can you confirm that the function is actually executed multiple times concurrently?

I guess the issue is that we should raise a warning when this occurs, or do you expect another behavior?

Yes, it seems that it executes concurrently. See Cloudwatch graph below from the Lambda execution, at the end I had 5 targets running every minute.
![image](https://cloud.githubusercontent.com/assets/2604142/17990812/10684d58-6aee-11e6-9e0f-3cb47c1f9110.png)

I'd expect to see the AWS error, for example if you remove the `ListRules` permission, Zappa nicely tells you that this permission is missing. For the delete targets, this error must be swallowed somehow. I had a brief look in the code but I could not find where it actually happens, will have another look now.

Ah, dang. Okay, good bug report. This code got messed with recently. Is this against 0.21.0 or master head?

It is against master head.

Found the issue: https://github.com/Miserlou/Zappa/blob/master/zappa/zappa.py#L1275
I was missing the permission `ListTargetsByRule`, this error is swallowed and skips the actual delete call. 
A possible solution could be to check if the `ClientError` is an `AccessDeniedException` and if so, raise it.

EDIT: Added missing words.

Excellent sleuthing! Sounds like a good solution to me if you can properly type damn boto3 errors, want to send a PR? 😄 

Working on it.


Resolving patch:
diff --git a/zappa/zappa.py b/zappa/zappa.py
--- a/zappa/zappa.py
+++ b/zappa/zappa.py
@@ -1275,8 +1275,12 @@ def delete_rule(self, rule_name):
         try:
             targets = self.events_client.list_targets_by_rule(Rule=rule_name)
         except botocore.exceptions.ClientError as e:
-            logger.debug('No target found for this rule: {} {}'.format(rule_name, e.message))
-            return
+            error_code = e.response['Error']['Code']
+            if error_code == 'AccessDeniedException':
+                raise
+            else:
+                logger.debug('No target found for this rule: {} {}'.format(rule_name, e.message))
+                return
 
         if 'Targets' in targets and targets['Targets']:
             response = self.events_client.remove_targets(Rule=rule_name, Ids=[x['Id'] for x in targets['Targets']])


Test patch:
Repository: Miserlou/Zappa

Issue:
It is not possible to use an up-to-date boto3 version
Zappa filters out the boto3 and botocore package. Unfortunately, the available boto3 version in Lambda is quite old and is lacking some features.



Discussion:

Resolving patch:
diff --git a/zappa/zappa.py b/zappa/zappa.py
--- a/zappa/zappa.py
+++ b/zappa/zappa.py
@@ -179,7 +179,7 @@
 
 ZIP_EXCLUDES = [
     '*.exe', '*.DS_Store', '*.Python', '*.git', '.git/*', '*.zip', '*.tar.gz',
-    '*.hg', '*.egg-info', 'botocore*', 'pip', 'docutils*', 'boto3*', 'setuputils*'
+    '*.hg', '*.egg-info', 'pip', 'docutils*', 'setuputils*'
 ]
 
 ##


Test patch:
Repository: Miserlou/Zappa

Issue:
certify can fail for lets_encrypt_key in S3
The `zappa certify` command fails

```
ClientError: An error occurred (404) when calling the HeadObject operation: Not Found
```

given the setting

```
        "lets_encrypt_key": "s3://my-bucket/my-folder/server.key",
```

because the key in the S3 URI is extracted incorrectly if it has slashes.



Discussion:

Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -947,8 +947,7 @@ def certify(self):
             return
 
         if 's3://' in account_key_location:
-            bucket = account_key_location.split('s3://')[1].split('/')[0]
-            key_name = account_key_location.split('s3://')[1].split('/')[1]
+            bucket, key_name = account_key_location.split('s3://')[1].split('/', 1)
             self.zappa.s3_client.download_file(bucket, key_name, '/tmp/account.key')
         else:
             from shutil import copyfile


Test patch:
Repository: Miserlou/Zappa

Issue:
`zappa init` fails with syntax error
```
$ zappa init
Traceback (most recent call last):
  File "/usr/local/bin/zappa", line 7, in <module>
    from zappa.cli import handle
  File "/usr/local/lib/python3.5/site-packages/zappa/cli.py", line 854
    except ImportError, e:
                      ^
SyntaxError: invalid syntax
```

For better forward compatibility with Python 3, this should be `except ImportError as e:`. See https://www.python.org/dev/peps/pep-3110/ for more details.

(After digging through other issues, I now see that zappa doesn't fully support Python 3.)



Discussion:

Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -411,7 +411,7 @@ def update(self):
             except botocore.client.ClientError:
                 click.echo(click.style("Failed", fg="red") + " to " + click.style("manage IAM roles", bold=True) + "!")
                 click.echo("You may " + click.style("lack the necessary AWS permissions", bold=True) + " to automatically manage a Zappa execution role.")
-                click.echo("To fix this, see here: " + click.style("https://github.com/Miserlou/Zappa#using-custom-aws-iam-roles-and-policie", bold=True))
+                click.echo("To fix this, see here: " + click.style("https://github.com/Miserlou/Zappa#using-custom-aws-iam-roles-and-policies", bold=True))
                 sys.exit(-1)
 
         # Create the Lambda Zip,
@@ -890,13 +890,13 @@ def init(self, settings_file="zappa_settings.json"):
         try: # pragma: no cover
             import django
             has_django = True
-        except ImportError, e:
+        except ImportError as e:
             has_django = False
 
         try: # pragma: no cover
             import flask
             has_flask = True
-        except ImportError, e:
+        except ImportError as e:
             has_flask = False
 
         print('')


Test patch:
diff --git a/tests/test_handler.py b/tests/test_handler.py
--- a/tests/test_handler.py
+++ b/tests/test_handler.py
@@ -37,5 +37,5 @@ def test_run_function(self):
         try:
             LambdaHandler.run_function(unsupported, 'e', 'c')
             self.fail('Exception expected')
-        except RuntimeError, e:
+        except RuntimeError as e:
             pass
\ No newline at end of file
Repository: Miserlou/Zappa

Issue:
Tail broken
With the newest version only, tail is broken. Environment is no longer parsed, PR incoming :)


Discussion:

Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -263,7 +263,7 @@ def positive_int(s):
         # Log Tailing
         ##
         tail_parser = subparsers.add_parser(
-            'tail', help='Tail deployment logs.'
+            'tail', parents=[env_parser], help='Tail deployment logs.'
         )
         tail_parser.add_argument(
             '--no-color', action='store_true',


Test patch:
Repository: Miserlou/Zappa

Issue:
zappa unschedule <env> doesn't work
<!--- Provide a general summary of the issue in the Title above -->
## Context
```bash
(venv) rich@rich-vbox:~/Projects/test$ zappa unschedule dev
usage: zappa [-h] [-v] [-a APP_FUNCTION] [-s SETTINGS_FILE]
             {certify,deploy,init,invoke,manage,rollback,schedule,status,tail,undeploy,unschedule,update}
             ...
zappa: error: unrecognized arguments: dev
(venv) rich@rich-vbox:~/Projects/test$ zappa unschedule
Calling unschedule for environment dev..
Unscheduling..
Unscheduled test-dev-zappa-keep-warm-handler.keep_warm_callback.
```
<!--- Provide a more detailed introduction to the issue itself, and why you consider it to be a bug -->
<!--- Also, please make sure that you are running Zappa _from a virtual environment_ and are using Python 2.7 --> 
Looks like you can unschedule without providing an environment but it will unschedule if you don't provide one.
## Expected Behavior
<!--- Tell us what should happen -->

## Actual Behavior
<!--- Tell us what happens instead -->

## Possible Fix
<!--- Not obligatory, but suggest a fix or reason for the bug -->

## Steps to Reproduce
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug include code to reproduce, if relevant -->
1.
2.
3.
4.

## Context
<!--- How has this bug affected you? What were you trying to accomplish? -->

## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Zappa version used:
* Operating System and Python version:
* Link to your project (optional):
* Your `zappa_settings.py`: 



Discussion:

Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -324,7 +324,8 @@ def positive_int(s):
         ##
         # Unschedule
         ##
-        subparsers.add_parser('unschedule', help='Unschedule functions.')
+        subparsers.add_parser('unschedule', parents=[env_parser],
+                              help='Unschedule functions.')
 
         ##
         # Updating


Test patch:
Repository: Miserlou/Zappa

Issue:
Region us-west-1 is available, but PyPI Zappa (0.35.2) doesn't include it.  
<!--- Provide a general summary of the issue in the Title above -->
## Context

Lambda/API Gateway region `us-west-1` became available [11/27/2016](https://aws.amazon.com/about-aws/whats-new/2016/11/aws-lambda-available-in-us-west-n-california/), but Zappa 0.35.2 gives warnings saying ```Warning! AWS Lambda may not be available in this AWS Region!
Warning! AWS API Gateway may not be available in this AWS Region!``` when doing a `zappa deploy` to `us-west-1`.  

Adding `us-west-1` to zappa.py: `API_GATEWAY_REGIONS` & `LAMBDA_REGIONS` resolves the issue and allows deployment to `us-west-1`.

PR to follow.




Discussion:

Resolving patch:
diff --git a/zappa/zappa.py b/zappa/zappa.py
--- a/zappa/zappa.py
+++ b/zappa/zappa.py
@@ -188,11 +188,19 @@
 REDIRECT_RESPONSE_TEMPLATE = ""
 
 # Latest list: https://docs.aws.amazon.com/general/latest/gr/rande.html#apigateway_region
-API_GATEWAY_REGIONS = ['us-east-1', 'us-east-2', 'us-west-2', 'eu-central-1', 'eu-west-1', 'ap-northeast-1',
-                       'ap-northeast-2', 'ap-southeast-1', 'ap-southeast-2']
+API_GATEWAY_REGIONS = ['us-east-1', 'us-east-2',
+                       'us-west-1', 'us-west-2',
+                       'eu-central-1',
+                       'eu-west-1', 'eu-west-2',
+                       'ap-northeast-1', 'ap-northeast-2',
+                       'ap-southeast-1', 'ap-southeast-2']
 # Latest list: https://docs.aws.amazon.com/general/latest/gr/rande.html#lambda_region
-LAMBDA_REGIONS = ['us-east-1', 'us-east-2', 'us-west-2', 'eu-central-1', 'eu-west-1', 'ap-northeast-1',
-                  'ap-northeast-2', 'ap-southeast-1', 'ap-southeast-2']
+LAMBDA_REGIONS = ['us-east-1', 'us-east-2',
+                  'us-west-1', 'us-west-2',
+                  'eu-central-1',
+                  'eu-west-1', 'eu-west-2',
+                  'ap-northeast-1', 'ap-northeast-2',
+                  'ap-southeast-1', 'ap-southeast-2']
 
 # We never need to include these.
 # Related: https://github.com/Miserlou/Zappa/pull/56


Test patch:
Repository: Miserlou/Zappa

Issue:
ap-south-1 not included in core.py
ap-south-1 has API gateway and lambda, but zappa hasn't included it.


Discussion:

Resolving patch:
diff --git a/zappa/core.py b/zappa/core.py
--- a/zappa/core.py
+++ b/zappa/core.py
@@ -154,7 +154,8 @@
                        'eu-central-1',
                        'eu-west-1', 'eu-west-2',
                        'ap-northeast-1', 'ap-northeast-2',
-                       'ap-southeast-1', 'ap-southeast-2']
+                       'ap-southeast-1', 'ap-southeast-2',
+                       'ap-south-1']
 
 # Latest list: https://docs.aws.amazon.com/general/latest/gr/rande.html#lambda_region
 LAMBDA_REGIONS = ['us-east-1', 'us-east-2',
@@ -162,7 +163,8 @@
                   'eu-central-1',
                   'eu-west-1', 'eu-west-2',
                   'ap-northeast-1', 'ap-northeast-2',
-                  'ap-southeast-1', 'ap-southeast-2']
+                  'ap-southeast-1', 'ap-southeast-2',
+                  'ap-south-1']
 
 # We never need to include these.
 # Related: https://github.com/Miserlou/Zappa/pull/56


Test patch:
Repository: Miserlou/Zappa

Issue:
Make project_name A Required Setting
The project_name currently defaults to pwd. When have multiple working environments for developers or CI servers this can lead to deployment issues that are not straight forward to troubleshoot. If it stays as an optional setting, maybe update the warning from:

Warning! Couldn't get function - have you deployed yet?

to

Warning! Couldn't get function - It is not deployed or was deployed under a different project_name.



Discussion:
Did you set the project up using `init`? It could be put in there but I don't think it needs to be a required setting.

I did not use init, just made the JSON. I think putting it in init, updating that warning, and possibly adding a warning on deploy as well if project_name isn't specified in the settings. I would think most prod-level deployments will come across this scenario at some point so it would be good to communicate the project_name importance.


Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -1587,6 +1587,7 @@ def init(self, settings_file="zappa_settings.json"):
                 env_name: {
                     's3_bucket': env_bucket,
                     'runtime': 'python3.6' if sys.version_info[0] == 3 else 'python2.7'
+                    'project_name': self.get_project_name()
                 }
             }
 
@@ -1886,7 +1887,7 @@ def load_settings(self, settings_file=None, session=None):
             # If the name is invalid, this will throw an exception with message up stack
             self.project_name = validate_name(self.stage_config['project_name'])
         else:
-            self.project_name = slugify.slugify(os.getcwd().split(os.sep)[-1])[:15]
+            self.project_name = self.get_project_name()
 
         # The name of the actual AWS Lambda function, ex, 'helloworld-dev'
         # Assume that we already have have validated the name beforehand.
@@ -2339,6 +2340,9 @@ def is_http_log_entry(self, string):
 
         return False
 
+    def get_project_name(self):
+        return slugify.slugify(os.getcwd().split(os.sep)[-1])[:15]
+
     def colorize_log_entry(self, string):
         """
         Apply various heuristics to return a colorized version of a string.


Test patch:
Repository: Miserlou/Zappa

Issue:
Bad "END" token coloring in zappa manage diffsettings
Zappa 0.44.3

I ran `zappa manage dev diffsettings` to get my settings diff output. In those settings I have configured the following:

`REST_FRAMEWORK = {'DEFAULT_RENDERER_CLASSES': ('rest_framework.renderers.JSONRenderer',)} 

This is what it looks like in the output:

![image](https://user-images.githubusercontent.com/235410/31518178-1390b2b4-afa7-11e7-87e7-45bcad513080.png)

The "END" is being parsed and tokenized the same way as `[END] RequestId: ...`.


Discussion:
Ha!

Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -1269,12 +1269,20 @@ def colorize_invoke_command(self, string):
             try:
                 for token in ['START', 'END', 'REPORT', '[DEBUG]']:
                     if token in final_string:
-                        format_string = '{}' if token == '[DEBUG]' else '[{}]'
-                        final_string = final_string.replace(token, click.style(
+                        format_string = '[{}]'
+                        # match whole words only
+                        pattern = r'\b{}\b'
+                        if token == '[DEBUG]':
+                            format_string = '{}'
+                            pattern = re.escape(token)
+                        repl = click.style(
                             format_string.format(token),
                             bold=True,
                             fg='cyan'
-                        ))
+                        )
+                        final_string = re.sub(
+                            pattern.format(token), repl, final_string
+                        )
             except Exception: # pragma: no cover
                 pass
 


Test patch:
diff --git a/tests/tests.py b/tests/tests.py
--- a/tests/tests.py
+++ b/tests/tests.py
@@ -867,6 +867,14 @@ def test_cli_colorize_invoke_command(self):
         colorized_string = zappa_cli.colorize_invoke_command(plain_string)
         self.assertEqual(final_string, colorized_string)
 
+    def test_cli_colorize_whole_words_only(self):
+        zappa_cli = ZappaCLI()
+        plain_string = "START RESTART END RENDER report [DEBUG] TEXT[DEBUG]TEXT"
+        final_string = "\x1b[36m\x1b[1m[START]\x1b[0m RESTART \x1b[36m\x1b[1m[END]\x1b[0m RENDER report \x1b[36m\x1b[1m[DEBUG]\x1b[0m TEXT\x1b[36m\x1b[1m[DEBUG]\x1b[0mTEXT"
+
+        colorized_string = zappa_cli.colorize_invoke_command(plain_string)
+        self.assertEqual(final_string, colorized_string)
+
     def test_cli_colorize_invoke_command_bad_string(self):
         zappa_cli = ZappaCLI()
         plain_string = "Hey, I'm a plain string, won't be colorized"
Repository: Miserlou/Zappa

Issue:
Failed to generate or install certificate! 
**My Domain SSL has expired. So I tried to create a new SSL key by doing** 

   > $ openssl genrsa 2048 > account.key; 

**And Added my Domain name and Path to zappa_settings.json file. 
I tried to run** 

  > $ zappa certify production

**This is what the Error I am getting**

> $ zappa certify production
Calling certify for stage production..
Are you sure you want to certify? [y/n] y
Certifying domain xxxx.domain.com ..
Error registering: 400 {
  "type": "urn:acme:error:malformed",
  "detail": "Provided agreement URL [https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf] does not match current agreement URL [https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf]",
  "status": 400
}
Failed to generate or install certificate! :(

==============
I find the link [https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf](https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf)  has nothing but "404 Not Found" Error. what is the problem ? And also do we have to run $ zappa certify everytime we need to
 renew the certificate ? 





Discussion:

Resolving patch:
diff --git a/zappa/letsencrypt.py b/zappa/letsencrypt.py
--- a/zappa/letsencrypt.py
+++ b/zappa/letsencrypt.py
@@ -239,7 +239,7 @@ def register_account():
     LOGGER.info("Registering account...")
     code, result = _send_signed_request(DEFAULT_CA + "/acme/new-reg", {
         "resource": "new-reg",
-        "agreement": "https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf",
+        "agreement": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
     })
     if code == 201:  # pragma: no cover
         LOGGER.info("Registered!")


Test patch:
Repository: Miserlou/Zappa

Issue:
Non-string value in zappa_settings.environment_variables caused AttributeError
Dear Zappatistas, 

I came across this minor thing: I had the following in my zappa_settings.json
`"environment_variables": {"FLASK_DEBUG": 0}`
I tried `zappa update dev` and got back 
```
ValueError("The following environment variables are not strings: {}".format(", ".join(non_strings)))
ValueError: The following environment variables are not strings: FLASK_DEBUG
``` 
followed by 
```
print("Error: {}".format(e.message))
AttributeError: 'ValueError' object has no attribute 'message'
```

It worked fine with:
`"environment_variables": {"FLASK_DEBUG": "0"}`

## Expected Behavior
Should properly report error without crashing. 

## Actual Behavior
Throws an exception about the exception. 

## Possible Fix
in cli.py line 1478
replace             
```raise ValueError("The following environment variables are not strings: {}".format(", ".join(non_strings)))```
with 
```raise ValueError("The following environment variables are not strings: {}".format(", ".join([str(non_string) for non_string in non_strings])))```

Also, it seems like ValueError does not have a message attribute, but can be printed like so str(e)

## Your Environment
* Zappa version used: 0.45.1
* Operating System and Python version: Windows10 Python3.6




Discussion:
Python 3 problem (works in Python 2). Will PR a fix.

Resolving patch:
diff --git a/zappa/cli.py b/zappa/cli.py
--- a/zappa/cli.py
+++ b/zappa/cli.py
@@ -525,7 +525,10 @@ def dispatch_command(self, command, stage):
         try:
             self.load_settings(self.vargs.get('settings_file'))
         except ValueError as e:
-            print("Error: {}".format(e.message))
+            if hasattr(e, 'message'):
+                print("Error: {}".format(e.message))
+            else:
+                print(str(e))
             sys.exit(-1)
         self.callback('settings')
 
@@ -1660,10 +1663,10 @@ def init(self, settings_file="zappa_settings.json"):
                 'project_name': self.get_project_name()
             }
         }
-        
+
         if profile_region:
           zappa_settings[env]['aws_region'] = profile_region
-        
+
         if has_django:
             zappa_settings[env]['django_settings'] = django_settings
         else:


Test patch: