Repository: getmoto/moto

Issue:
moto lambda response Payload is the log content, not function response data
See all details in https://github.com/aio-libs/aiobotocore/issues/793
- https://github.com/aio-libs/aiobotocore/pull/796 demonstrates the problem

Although I don't know how to reconcile that with the moto unit tests, i.e.
- https://github.com/spulec/moto/blob/master/tests/test_awslambda/test_lambda.py#L95-L125
- this appears to distinguish between the logs and payload


Discussion:
Hi @dazza-codes, it looks like that issue was fixed in this PR: https://github.com/spulec/moto/pull/2669

Can you repro this issue with the latest version? (https://pypi.org/project/moto/#history)

## pre-release fix notes

It could be fixed in prereleases:
- used `poetry add --dev moto=1.3.15.dev620 -E server` to update to the latest prerelease
- ran the test (as noted below) and it passes now

The new response looks like:
```python
(Pdb) pprint(func.response)
{'LogResult': 'eyJzdGF0dXNDb2RlIjoyMDAsImJvZHkiOnsiaSI6MX19',
 'Payload': <StreamingBody at 0x7f185f8a1a08 for ClientResponseContentProxy at 0x7f185fc57448>,
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '33',
                                      'content-type': 'text/html; '
                                                      'charset=utf-8',
                                      'date': 'Wed, 15 Apr 2020 01:34:09 GMT',
                                      'server': 'Werkzeug/1.0.1 Python/3.6.10',
                                      'x-amz-crc32': '2900392314',
                                      'x-amz-log-result': 'eyJzdGF0dXNDb2RlIjoyMDAsImJvZHkiOnsiaSI6MX19',
                                      'x-amzn-requestid': '6T4R3THB0O9XL210P9KHZTC7EENUNCEWS0HC1BR5A0UCEKIUQWC6'},
                      'HTTPStatusCode': 200,
                      'RequestId': '6T4R3THB0O9XL210P9KHZTC7EENUNCEWS0HC1BR5A0UCEKIUQWC6',
                      'RetryAttempts': 0},
 'StatusCode': 200}

```

The only nit-picks are:
- the content-type is not JSON, whereas AWS-lambda is JSON.
- the JSON body is in both the 'Payload' and the 'LogResult' now, so that's a reverse of the issue (and while that's better than the reverse, it still means something is missing, now it's the logs)


## stable release bug (1.3.14)

Code and tests to replicate this are in https://gitlab.com/dazza-codes/aio-aws/-/merge_requests/8

### released versions in venv
```
$ pip freeze | grep -i oto
aiobotocore==0.12.0
boto==2.49.0
boto3==1.12.3
botocore==1.15.3
moto==1.3.14
```

### test failure debugging

```python
tests/test_aio_aws_lambda.py:154: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
aio_aws/aio_aws_lambda.py:163: in invoke
    await self.read_response()  # updates self.content
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = AWSLambdaFunction(name='lambda_dev', type='RequestResponse', log_type='None', context=None, payload=b'{"i": 1}', quali...s\tBilled Duration: 100 ms\tMemory Size: 128 MB\tMax Memory Used: 26 MB\t\x1b[0m\n\n{"statusCode":200,"body":{"i":1}}')

    async def read_response(self):
        """
        Asynchronous coroutine to read a lambda response; this
        updates the ``content`` or the ``error`` values.
    
        :raises: botocore.exceptions.ClientError, botocore.exceptions.ParamValidationError
        """
    
        if self.response and response_success(self.response):
    
            log_result = self.response.get("LogResult")
            if log_result:
                self.logs = base64.b64decode(log_result)
    
            response_payload = self.response.get("Payload")
            if response_payload:
    
                async with response_payload as stream:
                    data = await stream.read()
    
                body = data.decode()
    
>               assert False  # for debugging
E               AssertionError

aio_aws/aio_aws_lambda.py:207: AssertionError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/dlweber/src/aio-aws/aio_aws/aio_aws_lambda.py(207)read_response()
-> assert False
(Pdb) self.logs
b'\x1b[32mSTART RequestId: e1c62ace-c0ff-194f-8896-1a80ed362f7b Version: 1\x1b[0m\nevent: {\'i\': 1}\n\x1b[32mEND RequestId: e1c62ace-c0ff-194f-8896-1a80ed362f7b\x1b[0m\n\x1b[32mREPORT RequestId: e1c62ace-c0ff-194f-8896-1a80ed362f7b\tInit Duration: 532.85 ms\tDuration: 3.41 ms\tBilled Duration: 100 ms\tMemory Size: 128 MB\tMax Memory Used: 26 MB\t\x1b[0m\n\n{"statusCode":200,"body":{"i":1}}'
(Pdb) body
'\x1b[32mSTART RequestId: e1c62ace-c0ff-194f-8896-1a80ed362f7b Version: 1\x1b[0m\nevent: {\'i\': 1}\n\x1b[32mEND RequestId: e1c62ace-c0ff-194f-8896-1a80ed362f7b\x1b[0m\n\x1b[32mREPORT RequestId: e1c62ace-c0ff-194f-8896-1a80ed362f7b\tInit Duration: 532.85 ms\tDuration: 3.41 ms\tBilled Duration: 100 ms\tMemory Size: 128 MB\tMax Memory Used: 26 MB\t\x1b[0m\n\n{"statusCode":200,"body":{"i":1}}'
(Pdb) 
```



When it runs against an actual AWS lambda function (i.e. live test), it has a very different response that works.

```
Test async lambda functions
[INFO]  2020-04-15T01:11:05.215Z  aio-aws:invoke:165  AWS Lambda (lambda_dev) invoked OK

{'FunctionName': 'lambda_dev', 'InvocationType': 'RequestResponse', 'LogType': 'None', 'Payload': b'{"i": 0}'}

{'ResponseMetadata': {'RequestId': '8fd8d972-f00c-4e5f-95c7-723d896fe133', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Wed, 15 Apr 2020 01:11:05 GMT', 'content-type': 'application/json', 'content-length': '63', 'connection': 'keep-alive', 'x-amzn-requestid': '8fd8d972-f00c-4e5f-95c7-723d896fe133', 'x-amzn-remapped-content-length': '0', 'x-amz-executed-version': '$LATEST', 'x-amzn-trace-id': 'root=1-5e965f29-b530e6c8868245f2385652d0;sampled=0'}, 'RetryAttempts': 0}, 'StatusCode': 200, 'ExecutedVersion': '$LATEST', 'Payload': <StreamingBody at 0x7f24ee2dd948 for ClientResponseContentProxy at 0x7f24ee2dd908>}

{'statusCode': 200, 'body': '"0, 1, 2, 3, 4, 5, 6, 7, 8, 9"'}

None
1 lambdas finished in 0.30 seconds.
```

Things to note:
- `StatusCode` is 200 from AWS, 202 from moto
- `content-type` is json from AWS, html from moto
- with `LogType == None`, AWS has no logs, moto has logs (same data in the body/content too)


Resolving patch:
diff --git a/moto/awslambda/models.py b/moto/awslambda/models.py
--- a/moto/awslambda/models.py
+++ b/moto/awslambda/models.py
@@ -379,6 +379,7 @@ def _invoke_lambda(self, code, event=None, context=None):
             event = dict()
         if context is None:
             context = {}
+        output = None
 
         try:
             # TODO: I believe we can keep the container running and feed events as needed
@@ -394,7 +395,7 @@ def _invoke_lambda(self, code, event=None, context=None):
 
             env_vars.update(self.environment_vars)
 
-            container = output = exit_code = None
+            container = exit_code = None
             log_config = docker.types.LogConfig(type=docker.types.LogConfig.types.JSON)
             with _DockerDataVolumeContext(self) as data_vol:
                 try:
@@ -455,24 +456,28 @@ def _invoke_lambda(self, code, event=None, context=None):
 
             # We only care about the response from the lambda
             # Which is the last line of the output, according to https://github.com/lambci/docker-lambda/issues/25
-            output = output.splitlines()[-1]
-            return output, False
+            resp = output.splitlines()[-1]
+            logs = os.linesep.join(
+                [line for line in self.convert(output).splitlines()[:-1]]
+            )
+            return resp, False, logs
         except BaseException as e:
             traceback.print_exc()
-            return "error running lambda: {}".format(e), True
+            logs = os.linesep.join(
+                [line for line in self.convert(output).splitlines()[:-1]]
+            )
+            return "error running lambda: {}".format(e), True, logs
 
     def invoke(self, body, request_headers, response_headers):
-        payload = dict()
 
         if body:
             body = json.loads(body)
 
         # Get the invocation type:
-        res, errored = self._invoke_lambda(code=self.code, event=body)
+        res, errored, logs = self._invoke_lambda(code=self.code, event=body)
         if request_headers.get("x-amz-invocation-type") == "RequestResponse":
-            encoded = base64.b64encode(res.encode("utf-8"))
+            encoded = base64.b64encode(logs.encode("utf-8"))
             response_headers["x-amz-log-result"] = encoded.decode("utf-8")
-            payload["result"] = response_headers["x-amz-log-result"]
             result = res.encode("utf-8")
         else:
             result = res
@@ -1044,9 +1049,9 @@ def invoke(self, function_name, qualifier, body, headers, response_headers):
         if fn:
             payload = fn.invoke(body, headers, response_headers)
             response_headers["Content-Length"] = str(len(payload))
-            return response_headers, payload
+            return payload
         else:
-            return response_headers, None
+            return None
 
 
 def do_validate_s3():
diff --git a/moto/awslambda/responses.py b/moto/awslambda/responses.py
--- a/moto/awslambda/responses.py
+++ b/moto/awslambda/responses.py
@@ -178,7 +178,7 @@ def _invoke(self, request, full_url):
         function_name = unquote(self.path.rsplit("/", 2)[-2])
         qualifier = self._get_param("qualifier")
 
-        response_header, payload = self.lambda_backend.invoke(
+        payload = self.lambda_backend.invoke(
             function_name, qualifier, self.body, self.headers, response_headers
         )
         if payload:
@@ -187,6 +187,8 @@ def _invoke(self, request, full_url):
             elif request.headers.get("X-Amz-Invocation-Type") == "DryRun":
                 status_code = 204
             else:
+                if request.headers.get("X-Amz-Log-Type") != "Tail":
+                    del response_headers["x-amz-log-result"]
                 status_code = 200
             return status_code, response_headers, payload
         else:


Test patch:
diff --git a/tests/test_awslambda/test_lambda.py b/tests/test_awslambda/test_lambda.py
--- a/tests/test_awslambda/test_lambda.py
+++ b/tests/test_awslambda/test_lambda.py
@@ -43,6 +43,7 @@ def _process_lambda(func_str):
 def get_test_zip_file1():
     pfunc = """
 def lambda_handler(event, context):
+    print("custom log event")
     return event
 """
     return _process_lambda(pfunc)
@@ -112,18 +113,29 @@ def test_invoke_requestresponse_function():
         FunctionName="testFunction",
         InvocationType="RequestResponse",
         Payload=json.dumps(in_data),
+        LogType="Tail",
     )
 
     success_result["StatusCode"].should.equal(200)
-    result_obj = json.loads(
-        base64.b64decode(success_result["LogResult"]).decode("utf-8")
-    )
+    logs = base64.b64decode(success_result["LogResult"]).decode("utf-8")
 
-    result_obj.should.equal(in_data)
+    logs.should.contain("START RequestId:")
+    logs.should.contain("custom log event")
+    logs.should.contain("END RequestId:")
 
     payload = success_result["Payload"].read().decode("utf-8")
     json.loads(payload).should.equal(in_data)
 
+    # Logs should not be returned by default, only when the LogType-param is supplied
+    success_result = conn.invoke(
+        FunctionName="testFunction",
+        InvocationType="RequestResponse",
+        Payload=json.dumps(in_data),
+    )
+
+    success_result["StatusCode"].should.equal(200)
+    assert "LogResult" not in success_result
+
 
 @mock_lambda
 def test_invoke_requestresponse_function_with_arn():
@@ -152,11 +164,6 @@ def test_invoke_requestresponse_function_with_arn():
     )
 
     success_result["StatusCode"].should.equal(200)
-    result_obj = json.loads(
-        base64.b64decode(success_result["LogResult"]).decode("utf-8")
-    )
-
-    result_obj.should.equal(in_data)
 
     payload = success_result["Payload"].read().decode("utf-8")
     json.loads(payload).should.equal(in_data)