Repository: serverless/serverless (JavaScript)
Date: 2019-12-17 16:52:49+00:00
Task type: Feature

Issue:
expose Bisect on Function Error prop for Kinesis Streams
# Feature Proposal

https://aws.amazon.com/about-aws/whats-new/2019/11/aws-lambda-supports-failure-handling-features-for-kinesis-and-dynamodb-event-sources/

## Description

<!-- Please use https://forum.serverless.com, StackOverflow or other forums for Q&A -->
<!-- Please answer ALL the question below. Otherwise we probably have to close the issue due to missing information -->

1. What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.
1. **Optional:** If there is additional config how would it look

Similar or dependent issues:

- https://github.com/serverless/serverless/pull/7024



Issue discussion:
@WIZARDISHUNGRY Are you working on an implementation for this? If not, I'd like to give it a go
No, go for it

Solution patch:
diff --git a/docs/providers/aws/events/streams.md b/docs/providers/aws/events/streams.md
index f36aa98317f..3a04e5a5e6b 100644
--- a/docs/providers/aws/events/streams.md
+++ b/docs/providers/aws/events/streams.md
@@ -110,6 +110,26 @@ functions:
           batchWindow: 10
 ```
 
+## Setting BisectBatchOnFunctionError
+
+This configuration provides the ability to recursively split a failed batch and retry on a smaller subset of records, eventually isolating the metadata causing the error.
+
+**Note:** Serverless only sets this property if you explicitly add it to the stream configuration (see example below).
+
+[Related AWS documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventsourcemapping.html#cfn-lambda-eventsourcemapping-bisectbatchonfunctionerror)
+
+**Note:** The `stream` event will hook up your existing streams to a Lambda function. Serverless won't create a new stream for you.
+
+```yml
+functions:
+  preprocess:
+    handler: handler.preprocess
+    events:
+      - stream:
+          arn: arn:aws:kinesis:region:XXXXXX:stream/foo
+          bisectBatchOnFunctionError: true
+```
+
 ## Setting the MaximumRetryAttempts
 
 This configuration sets up the maximum number of times to retry when the function returns an error.
diff --git a/lib/plugins/aws/package/compile/events/stream/index.js b/lib/plugins/aws/package/compile/events/stream/index.js
index 09684548e1f..ff266ef3558 100644
--- a/lib/plugins/aws/package/compile/events/stream/index.js
+++ b/lib/plugins/aws/package/compile/events/stream/index.js
@@ -203,6 +203,10 @@ class AwsCompileStreamEvents {
               streamResource.Properties.MaximumRetryAttempts = event.stream.maximumRetryAttempts;
             }
 
+            if (event.stream.bisectBatchOnFunctionError) {
+              streamResource.Properties.BisectBatchOnFunctionError = true;
+            }
+
             const newStreamObject = {
               [streamLogicalId]: streamResource,
             };


Test patch:
diff --git a/lib/plugins/aws/package/compile/events/stream/index.test.js b/lib/plugins/aws/package/compile/events/stream/index.test.js
index fe7cb8b2b20..b61ee8a1248 100644
--- a/lib/plugins/aws/package/compile/events/stream/index.test.js
+++ b/lib/plugins/aws/package/compile/events/stream/index.test.js
@@ -379,6 +379,12 @@ describe('AwsCompileStreamEvents', () => {
               {
                 stream: 'arn:aws:dynamodb:region:account:table/baz/stream/3',
               },
+              {
+                stream: {
+                  arn: 'arn:aws:dynamodb:region:account:table/buzz/stream/4',
+                  bisectBatchOnFunctionError: true,
+                },
+              },
             ],
           },
         };
@@ -475,6 +481,40 @@ describe('AwsCompileStreamEvents', () => {
           awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
             .Resources.FirstEventSourceMappingDynamodbBaz.Properties.Enabled
         ).to.equal('True');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBaz.Properties.BisectBatchOnFunctionError
+        ).to.equal(undefined);
+
+        // event 4
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBuzz.Type
+        ).to.equal('AWS::Lambda::EventSourceMapping');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBuzz.DependsOn
+        ).to.equal('IamRoleLambdaExecution');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBuzz.Properties.EventSourceArn
+        ).to.equal(awsCompileStreamEvents.serverless.service.functions.first.events[3].stream.arn);
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBuzz.Properties.BatchSize
+        ).to.equal(10);
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBuzz.Properties.StartingPosition
+        ).to.equal('TRIM_HORIZON');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBuzz.Properties.Enabled
+        ).to.equal('True');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingDynamodbBuzz.Properties.BisectBatchOnFunctionError
+        ).to.equal(true);
       });
 
       it('should allow specifying DynamoDB and Kinesis streams as CFN reference types', () => {
@@ -749,6 +789,12 @@ describe('AwsCompileStreamEvents', () => {
               {
                 stream: 'arn:aws:kinesis:region:account:stream/baz',
               },
+              {
+                stream: {
+                  arn: 'arn:aws:kinesis:region:account:stream/buzz',
+                  bisectBatchOnFunctionError: true,
+                },
+              },
             ],
           },
         };
@@ -856,6 +902,44 @@ describe('AwsCompileStreamEvents', () => {
           awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
             .Resources.FirstEventSourceMappingKinesisBaz.Properties.Enabled
         ).to.equal('True');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBaz.Properties.BisectBatchOnFunctionError
+        ).to.equal(undefined);
+
+        // event 4
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.Type
+        ).to.equal('AWS::Lambda::EventSourceMapping');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.DependsOn
+        ).to.equal('IamRoleLambdaExecution');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.Properties.EventSourceArn
+        ).to.equal(awsCompileStreamEvents.serverless.service.functions.first.events[3].stream.arn);
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.Properties.BatchSize
+        ).to.equal(10);
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.Properties.ParallelizationFactor
+        ).to.equal(undefined);
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.Properties.StartingPosition
+        ).to.equal('TRIM_HORIZON');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.Properties.Enabled
+        ).to.equal('True');
+        expect(
+          awsCompileStreamEvents.serverless.service.provider.compiledCloudFormationTemplate
+            .Resources.FirstEventSourceMappingKinesisBuzz.Properties.BisectBatchOnFunctionError
+        ).to.equal(true);
       });
 
       it('should add the necessary IAM role statements', () => {