Repository: localstack/localstack

Issue:
Kinesis putRecords error
It seems that putRecords causes the following error:

```
Starting mock Kinesis (http port 4568)...
Starting mock S3 (http port 4572)...
Starting mock Firehose service (http port 4573)...
Starting mock Lambda service (http port 4574)...
Listening at http://:::4565
* Running on http://0.0.0.0:4563/ (Press CTRL+C to quit)
127.0.0.1 - - [08/May/2018 13:52:25] "GET / HTTP/1.1" 200 -
Ready.
r127.0.0.1 - - [08/May/2018 13:56:43] "PUT /prd1541-qa1-vf-sms-send-stream-archive HTTP/1.1" 200 -
127.0.0.1 - - [08/May/2018 13:56:43] "HEAD /prd1541-qa1-vf-sms-send-stream-archive HTTP/1.1" 200 -
2018-05-08T13:56:43:ERROR:localstack.services.generic_proxy: Error forwarding request: 'Records' Traceback (most recent call last):
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 215, in forward
    updated_response = self.proxy.update_listener.return_response(**kwargs)
  File "/opt/code/localstack/localstack/services/kinesis/kinesis_listener.py", line 49, in return_response
    response_records = response_body['Records']
KeyError: 'Records'

2018-05-08T13:56:43:ERROR:localstack.services.generic_proxy: Error forwarding request: 'Records' Traceback (most recent call last):
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 215, in forward
    updated_response = self.proxy.update_listener.return_response(**kwargs)
  File "/opt/code/localstack/localstack/services/kinesis/kinesis_listener.py", line 49, in return_response
    response_records = response_body['Records']
KeyError: 'Records'

2018-05-08T13:56:43:ERROR:localstack.services.generic_proxy: Error forwarding request: 'Records' Traceback (most recent call last):
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 215, in forward
    updated_response = self.proxy.update_listener.return_response(**kwargs)
  File "/opt/code/localstack/localstack/services/kinesis/kinesis_listener.py", line 49, in return_response
    response_records = response_body['Records']
KeyError: 'Records'
```

I'm using the latest localstack image and my `Kinesis.putRecords` call goes through the Javascript AWS SDK: 

```
...
  const result = await client.putRecords({
    StreamName,
    Records: records.map(data => ({
      Data: new Buffer(JSON.stringify(data)),
      PartitionKey: 'testPartition'
    }))
  }).promise();
...
```

It seems like there's a mismatch int the expected format in `kinesis_listener.py`.


Discussion:
@ewaltman I was getting this same error, but tracked it down to a mistake on my end. I was using an invalid value for `explicitHashKey`. Could that be your issue too?

```
Value 'test' at 'records.1.member.explicitHashKey' failed to satisfy constraint: Member must satisfy regular expression pattern: 0|([1-9]\d{0,38});
```
Hey @mrobinsn - I'm not using that param anywhere :/ is this field being set anywhere by localstack?

Thanks for reporting @ewaltman . How exactly are you starting LocalStack? Can you confirm that this issue persists with the latest version on `master`? Any chance you can provide a complete, self-contained example to reproduce this issue? Thanks
Any error response coming back from kinesis will show up as this error in localstack because the response JSON doesn't include the "records" array if an error occurred.  

I bumped into this same error when I forgot to re-create my stream after restarting localstack.  

_**Example error JSON from kinesis...**_

`{"__type":"ResourceNotFoundException","message":"Stream my-stream under account 000000000000 not found."}`
so localstack is buggy in the assumption that this line will work? https://github.com/localstack/localstack/blob/master/localstack/services/kinesis/kinesis_listener.py#L55

Resolving patch:
diff --git a/localstack/services/kinesis/kinesis_listener.py b/localstack/services/kinesis/kinesis_listener.py
--- a/localstack/services/kinesis/kinesis_listener.py
+++ b/localstack/services/kinesis/kinesis_listener.py
@@ -84,18 +84,19 @@ def return_response(self, method, path, data, headers, response):
         elif action == ACTION_PUT_RECORDS:
             event_records = []
             response_body = json.loads(to_str(response.content))
-            response_records = response_body['Records']
-            records = data['Records']
-            for index in range(0, len(records)):
-                record = records[index]
-                event_record = {
-                    'data': record['Data'],
-                    'partitionKey': record['PartitionKey'],
-                    'sequenceNumber': response_records[index].get('SequenceNumber')
-                }
-                event_records.append(event_record)
-            stream_name = data['StreamName']
-            lambda_api.process_kinesis_records(event_records, stream_name)
+            if 'Records' in response_body:
+                response_records = response_body['Records']
+                records = data['Records']
+                for index in range(0, len(records)):
+                    record = records[index]
+                    event_record = {
+                        'data': record['Data'],
+                        'partitionKey': record['PartitionKey'],
+                        'sequenceNumber': response_records[index].get('SequenceNumber')
+                    }
+                    event_records.append(event_record)
+                stream_name = data['StreamName']
+                lambda_api.process_kinesis_records(event_records, stream_name)
         elif action == ACTION_UPDATE_SHARD_COUNT:
             # Currently kinesalite, which backs the Kinesis implementation for localstack, does
             # not support UpdateShardCount:


Test patch: