Repository: Miserlou/Zappa

Issue:
Error Status Codes are Injected into Response Body
Given the following Flask App:

``` python
from flask import Flask, request, abort, make_response
app = Flask(__name__)

@app.route('/')
def index():
    abort(make_response("my custom error", 500))

```

When I run this app locally and hit its endpoint, the output I see is:

```
my custom error
```

When I deploy this app using zappa and hit its endpoint, the output I see is:

```
500my custom error
```

What's going on? Different versions of Flask, perhaps?



Discussion:
I should note that 

```
my custom error
```

is the expected/desired output.

This is because of how the handler wraps content so it can be found by api gateway and translated into the correct response. This is the reason custom 404 pages also have the same issue (there is a <!DOCTYPE html>404 injected at the top).

This adds the content to your response:
https://github.com/Miserlou/Zappa/blob/master/zappa/handler.py#L195-L196

This matches the content:
https://github.com/Miserlou/Zappa/blob/master/zappa/zappa.py#L726-L740

This is a high priority for me, but I haven't nailed down exactly how I want it to go yet. Ideally we can strip the flag in API gateway before sending it on to the requester. If not, we'll have to get more clever about what is being added to the response that is both detectable by the api gateway regex and does not impede generating responses.

Yeah, you've stumbled onto Zappa's primary hack. If you can come up with a better solution, I'd love to hear it. API Gateway's capabilities may have improved to the point where this is no longer possible, but I don't know yet.

Hello All, I believe that I might have a possible solution for this. I believe that we can still leverage the lambda error string matching if we change the format of the error message. I suggest that, instead of using base 64 encoding on the entire error message string, we only base 64 encode the body and then URL encode the error message with the status. Thus a resulting error message from lambda may look like this:

`status=404&body=<base64 string>`

On the API Gateway, the Lambda Error Regex would now look like this:

`status=404.*`

Lastly, the Body Mapping Template would be:

```
#set($message = $util.urlDecode($input.path('$.errorMessage')))
$util.base64Decode($message.body)
```

The resulting payload would only contain the actual body without any need for the prefix:

`u"<!DOCTYPE html>" + unicode(response.status_code) + unicode('<meta charset="utf-8" />')`

I am going to run a quick test using this method and see if it will work. If so, I can apply the patch and submit a PR. Let me know if you have any thoughts. Thanks!

Very excited stuff - looking forward to hearing the results of your experiments!

To follow up, I was able to successfully remove the `<!DOCTYPE html>XXX` prefix from the response body, however I did so in a slightly different way than described above. I was having issues getting the `$util.urlDecode` function to unpack the string properly and getting access the properties directly. I decided to try using a JSON array to pass the status followed by the body as a base64 encoded string. Thus the python code to send the errorMessage looks something like this:

```
errorMessage = json.dumps([
    unicode(response.status_code),
    base64.b64encode(response.data.encode('utf-8'))])
```

The Lambda Error Regex:

```
\["400".*
```

The Body Mapping Template:

```
#set($_body = $util.parseJson($input.path('$.errorMessage'))[1])
$util.base64Decode($_body)
```

I will be submitting a PR very shortly, but be advised that this change actually causes some issues. There are many API Gateways already setup to use the legacy Regex and Body Mapping Templates. Introducing a new pattern will cause all the old ones to fail upon the next update of the Lambda Function. I might suggest using a flag to enable this new mapping so that we don't break existing deployments. Thoughts?

Excited for this.

Nice, very much looking forward to this! I don't think $util.parseJson was available when I got started..

I don't mind requiring a redeployment as long as our version is < 1.0.0, but my mind can be swayed..

Any motion on that PR @randuhmm? :D


Resolving patch:
diff --git a/zappa/handler.py b/zappa/handler.py
--- a/zappa/handler.py
+++ b/zappa/handler.py
@@ -8,6 +8,7 @@
 import os
 import json
 import inspect
+import collections
 
 import boto3
 from werkzeug.wrappers import Response
@@ -326,8 +327,10 @@ def handler(self, event, context):
                 # The DOCTYPE ensures that the page still renders in the browser.
                 exception = None
                 if response.status_code in ERROR_CODES:
-                    content = u"<!DOCTYPE html>" + unicode(response.status_code) + unicode('<meta charset="utf-8" />') + response.data.encode('utf-8')
-                    exception = base64.b64encode(content)
+                    content = collections.OrderedDict()
+                    content['http_status'] = response.status_code
+                    content['content'] = base64.b64encode(response.data.encode('utf-8'))
+                    exception = json.dumps(content)
                 # Internal are changed to become relative redirects
                 # so they still work for apps on raw APIGW and on a domain.
                 elif 300 <= response.status_code < 400 and hasattr(response, 'Location'):
diff --git a/zappa/zappa.py b/zappa/zappa.py
--- a/zappa/zappa.py
+++ b/zappa/zappa.py
@@ -163,7 +163,7 @@
 }"""
 
 RESPONSE_TEMPLATE = """#set($inputRoot = $input.path('$'))\n$inputRoot.Content"""
-ERROR_RESPONSE_TEMPLATE = """#set($inputRoot = $input.path('$.errorMessage'))\n$util.base64Decode($inputRoot)"""
+ERROR_RESPONSE_TEMPLATE = """#set($_body = $util.parseJson($input.path('$.errorMessage'))['content'])\n$util.base64Decode($_body)"""
 REDIRECT_RESPONSE_TEMPLATE = ""
 
 API_GATEWAY_REGIONS = ['us-east-1', 'us-west-2', 'eu-west-1', 'eu-central-1', 'ap-northeast-1', 'ap-southeast-2']
@@ -1423,7 +1423,7 @@ def selection_pattern(status_code):
         if status_code in ['301', '302']:
             pattern = 'https://.*|/.*'
         elif status_code != '200':
-            pattern = base64.b64encode("<!DOCTYPE html>" + str(status_code)) + '.*'
+            pattern = '\{"http_status": ' + str(status_code) + '.*'
             pattern = pattern.replace('+', r"\+")
 
         return pattern


Test patch:
diff --git a/tests/tests.py b/tests/tests.py
--- a/tests/tests.py
+++ b/tests/tests.py
@@ -229,12 +229,12 @@ def test_redirect_pattern(self):
                 self.assertRegexpMatches(url, pattern)
 
     def test_b64_pattern(self):
-        head = '<!DOCTYPE html>'
+        head = '\{"http_status": '
 
         for code in ['400', '401', '402', '403', '404', '500']:
             pattern = Zappa.selection_pattern(code)
 
-            document = base64.b64encode(head + code + random_string(50))
+            document = head + code + random_string(50)
             self.assertRegexpMatches(document, pattern)
 
             for bad_code in ['200', '301', '302']: