Repository: localstack/localstack

Issue:
Lambda event body illegally quoted
When quoting the event body of a lambda event, quotes are [first escaped manually](https://github.com/localstack/localstack/blob/6ff360533f66a170dcb4b4314d4e729236de24e5/localstack/services/awslambda/lambda_executors.py#L168), [then escaped by shlex](https://github.com/localstack/localstack/blob/6ff360533f66a170dcb4b4314d4e729236de24e5/localstack/services/awslambda/lambda_executors.py#L286) (or pipes is python 2.7).

This first pass of escapes produces illegal JSON. To be precise, it produces JSON that is load-able by Python, but unmarshal-able by Go, which does not include `\'` in its list of valid esscape sequences (per the JSON spec).

To get around this issue, we currently need to ensure that none of our values used for local testing include a single quote character, otherwise our DynamoDB streams consumers will be _poisoned_ until a forceful restart and purge of the data. For a complete report, our troubles end [here](https://github.com/aws/aws-lambda-go/blob/master/lambda/handler.go#L113).

To illustrate this issue we need to first generate the escaped event the same way it is generated in localstack:

```python
>>> import json, shlex
>>> print(shlex.quote(json.dumps({'test': 'it\'s illegal'}).replace("'", "\\'")))
'{"test": "it\'"'"'s illegal"}'
```

Now, if we test to see how this is interpreted in the shell, we get something interesting.

```bash
$ export TEST='{"test": "it\'"'"'s illegal"}' && env | grep TEST
TEST={"test": "it\'s illegal"}
```

This seems like valid JSON, but the `\'`is an illegal escape and cannot be decoded universally. The solution may be as simple as removing the superfluous `string.replace` call indicated above.


Discussion:

Resolving patch:
diff --git a/localstack/services/awslambda/lambda_executors.py b/localstack/services/awslambda/lambda_executors.py
--- a/localstack/services/awslambda/lambda_executors.py
+++ b/localstack/services/awslambda/lambda_executors.py
@@ -165,12 +165,11 @@ def _execute(self, func_arn, func_details, event, context=None, version=None, as
             LOG.warning('Empty event body specified for invocation of Lambda "%s"' % func_arn)
             event = {}
         event_body = json.dumps(event)
-        event_body_escaped = event_body.replace("'", "\\'")
 
         docker_host = config.DOCKER_HOST_FROM_CONTAINER
 
         # amend the environment variables for execution
-        environment['AWS_LAMBDA_EVENT_BODY'] = event_body_escaped
+        environment['AWS_LAMBDA_EVENT_BODY'] = event_body
         environment['HOSTNAME'] = docker_host
         environment['LOCALSTACK_HOSTNAME'] = docker_host
         if context:


Test patch:
diff --git a/tests/integration/test_integration.py b/tests/integration/test_integration.py
--- a/tests/integration/test_integration.py
+++ b/tests/integration/test_integration.py
@@ -297,7 +297,7 @@ def test_kinesis_lambda_forward_chain(self):
             event_source_arn=get_event_source_arn(TEST_CHAIN_STREAM2_NAME), runtime=LAMBDA_RUNTIME_PYTHON27)
 
         # publish test record
-        test_data = {'test_data': 'forward_chain_data_%s' % short_uid()}
+        test_data = {'test_data': 'forward_chain_data_%s with \'quotes\\"' % short_uid()}
         data = clone(test_data)
         data[lambda_integration.MSG_BODY_MESSAGE_TARGET] = 'kinesis:%s' % TEST_CHAIN_STREAM2_NAME
         kinesis.put_record(Data=to_bytes(json.dumps(data)), PartitionKey='testId', StreamName=TEST_CHAIN_STREAM1_NAME)
diff --git a/tests/integration/test_lambda.py b/tests/integration/test_lambda.py
--- a/tests/integration/test_lambda.py
+++ b/tests/integration/test_lambda.py
@@ -216,7 +216,7 @@ def test_invocation_with_qualifier(self):
         self.assertIn('Version', response)
 
         # invoke lambda function
-        data_before = b'{"foo": "bar"}'
+        data_before = b'{"foo": "bar with \'quotes\\""}'
         result = self.lambda_client.invoke(
             FunctionName=lambda_name,
             Payload=data_before,
@@ -264,7 +264,7 @@ def test_upload_lambda_from_s3(self):
         )
 
         # invoke lambda function
-        data_before = b'{"foo": "bar"}'
+        data_before = b'{"foo": "bar with \'quotes\\""}'
         result = self.lambda_client.invoke(
             FunctionName=lambda_name, Payload=data_before)
         data_after = json.loads(result['Payload'].read())
@@ -346,13 +346,13 @@ def test_nodejs_lambda_running_in_docker(self):
         )
         result = self.lambda_client.invoke(
             FunctionName=TEST_LAMBDA_NAME_CUSTOM_RUNTIME,
-            Payload=b'{"text":"Hello"}')
+            Payload=b'{"text":"bar with \'quotes\\""}')
         result_data = result['Payload'].read()
 
         self.assertEqual(result['StatusCode'], 200)
         self.assertEqual(
             to_str(result_data).strip(),
-            """Echoing request: '{"text": "Hello"}'""")
+            """Echoing request: '{"text": "bar with \'quotes\\""}'""")
 
         # assert that logs are present
         expected = ['.*Custom Runtime Lambda handler executing.']