Repository: serverless/serverless (JavaScript)
Date: 2019-03-04 19:20:24+00:00
Task type: Feature

Issue:
Add support for asynchronous lambda invocation with integration type AWS
# This is a Feature Proposal

## Description

API Gateway supports asynchronous lambda invocation when integration type is AWS. Described here:
https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-lambda.html
Currently it is not possible to configure this in serverless directly and requires a resource override to enable. This feature proposal is to add support for integration type AWS, asynchronous lambda invocation without using an resource override. 

* What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.

Asynchronous lambda invocation allows for a quick response back (e.g. 202 Accepted) while allowing the lambda to continue running. This would be desired if you wish to kick off a lambda that runs longer then the API GW 30 second timeout, or if the application needs a quick response without the need for direct success for failure of the longer running lambda.  

* If there is additional config how would it look

Add new integration type `AWS_ASYNC` that configures the `X-Amz-Invocation-Type` correctly:
```yaml
  lambda:
    handler: index.handler
    events:
      - http:
          path: users
          method: post
          integration: AWS_ASYNC
```
* Cloud Formation Information

`AWS_ASYNC` would set the integration type to AWS and add ` "integration.request.header.X-Amz-Invocation-Type": "'Event'"` to `Integration { RequestParameters: {} }` similar to as follows:
```
          "ApiGatewayMethodUsersPost": {
            "Type": "AWS::ApiGateway::Method",
            "Properties": {
              "HttpMethod": "POST",
              "RequestParameters": {
              },
              "Integration": {
                "IntegrationHttpMethod": "POST",
                "Type": "AWS",
                "RequestParameters": {
                  "integration.request.header.X-Amz-Invocation-Type": "'Event'"
                }
```
## Additional Data

* ***Serverless Framework Version you're using***:
serverless/serverless as of 4/28/2018
* ***Operating System***:
Ubuntu 16.04
* ***Stack Trace***:
N/A
* ***Provider Error messages***:
N/A


Issue discussion:
@jjkirby Developed the following resource override to use as a work-around until the feature proposal is complete:
```yaml
resources:
   Resources:
     ApiGatewayMethodUsersPost:
       Type: AWS::ApiGateway::Method
       Properties:
         HttpMethod: POST
         Integration:
           IntegrationHttpMethod: POST
           Type: "AWS"
           RequestParameters:
              "integration.request.header.X-Amz-Invocation-Type": "'Event'"
```
@jjkirby If you find there is a need for additional resource overrides please add that information here. Thanks
Or just add an element like async which does the header settings under the hood such as:

```
lambda:
    handler: index.handler
    events:
      - http:
          path: users
          method: post
          integration: AWS
          async: true
```
@jjkirby @bsdkurt As the asynchronous invocation is only valid with AWS type integrations, it could be encoded into a new integration type: `AWS_ASYNC`. Having a separate `async` property could make people believe that it is independent from the integration and should do something in any case.
I've updated the feature proposal with suggestions from above.
My org is also in need of this feature. What help is needed?
Hi @defionscode. I wrote the feature proposal after helping jjkirby on slack with how to figure out what resource override was needed to get this to work. I am not currently using asynchronous lambda's and wont have time to implement this, but thought it would be helpful to the community to document the current work-around and propose how to get this integrated.

What is needed now is someone to develop the feature with the unit tests and documentation and then make a pull request for review.
I am not a Node guy but I will  be willing to help (where I can)  since I started all of this
This would be really nice - I went to the doc trying to figure out how to set up api gateway "event" invocationType and saw that unfortunately it is not possible yet. My support to this story.
Is there something that needs to be done other than adding the resource override to serverless.yml to get the workaround suggested by @bsdkurt to work? When I do this, I still get a timeout for requests, indicating that the function is not being handled asynchronously.
@scottb can you post your serverless.yml file?


@scott2b  Here are my snippets:

My plugins:
  - serverless-pseudo-parameters
  - serverless-reqvalidator-plugin
  - serverless-aws-documentation

One of the keys is double quote around single quote on request parameter Event and WHEN-NO _MATCH on pass through
API definition:

```
 events:
        - http:
           path: sfNotify
           method: POST
           cors: true
           private: true
           integration: aws
           reqValidatorName: onlyBody
           documentation:
              tags:
                - notify
              summary: Post a flight event notification generated by a third party
              description: ''
              requestModels:
                 "application/json": sfNotifyPostRequest
           request:
             passThrough: WHEN_NO_MATCH
             parameters:
               headers:
                'X-Amz-Invocation-Type': "'Event'"
```

Then my resources:

```
resources:
   Resources:
     GatewayResponse:
         Type: 'AWS::ApiGateway::GatewayResponse'
         Properties:
           ResponseParameters:
             gatewayresponse.header.Access-Control-Allow-Origin: "'*'"
           ResponseType: INVALID_API_KEY
           RestApiId:
             Ref: 'ApiGatewayRestApi'
           StatusCode: '403'
     # Using CF Request Validator
     onlyBody:
       Type: "AWS::ApiGateway::RequestValidator"
       Properties:
         Name: 'only-body'
         RestApiId:
           Ref: ApiGatewayRestApi
         ValidateRequestBody: true
         ValidateRequestParameters: false
     ApiGatewayMethodSfnotifyPost:
       Type: AWS::ApiGateway::Method
       Properties:
         HttpMethod: POST
         Integration:
           IntegrationHttpMethod: POST
           Type: "AWS"
           RequestParameters:
              "integration.request.header.X-Amz-Invocation-Type": "'Event'"

```
Is there anyway to override the resource while still keeping the very easy to use Api object in the events of the lambda functions that use that specific API gateway?

```
"Events": {
    "MyApiGateway": {
        "Type": "Api",
         "Properties": {
             "Method": "get",
                 "Path": "/mypath",
             }
     }
},
```

Hi guys, is there any update on this issue? Thanks. :)

Solution patch:
diff --git a/docs/providers/aws/events/apigateway.md b/docs/providers/aws/events/apigateway.md
index 3449c6a836c..f3c2912c09a 100644
--- a/docs/providers/aws/events/apigateway.md
+++ b/docs/providers/aws/events/apigateway.md
@@ -483,6 +483,21 @@ functions:
               - nickname
 ```
 
+### Using asyncronous integration
+
+Use `async: true` when integrating a lambda function using event invocation. This lets API Gateway to return immediately while the lambda continues running. If not othewise speficied integration type will be `AWS`.
+
+```yml
+functions:
+  create:
+    handler: posts.create
+    events:
+      - http:
+          path: posts/create
+          method: post
+          async: true  # default is false
+```
+
 ### Catching Exceptions In Your Lambda Function
 
 In case an exception is thrown in your lambda function AWS will send an error message with `Process exited before completing request`. This will be caught by the regular expression for the 500 HTTP status and the 500 status will be returned.
diff --git a/lib/plugins/aws/package/compile/events/apiGateway/lib/method/integration.js b/lib/plugins/aws/package/compile/events/apiGateway/lib/method/integration.js
index d9e76d3fded..de7da698cbe 100644
--- a/lib/plugins/aws/package/compile/events/apiGateway/lib/method/integration.js
+++ b/lib/plugins/aws/package/compile/events/apiGateway/lib/method/integration.js
@@ -92,9 +92,8 @@ module.exports = {
         IntegrationResponses: this.getIntegrationResponses(http),
       });
     }
-
-    if ((type === 'AWS' || type === 'HTTP' || type === 'HTTP_PROXY') &&
-      (http.request && !_.isEmpty(http.request.parameters))) {
+    if (((type === 'AWS' || type === 'HTTP' || type === 'HTTP_PROXY') &&
+      (http.request && !_.isEmpty(http.request.parameters))) || http.async) {
       _.assign(integration, {
         RequestParameters: this.getIntegrationRequestParameters(http),
       });
@@ -192,6 +191,11 @@ module.exports = {
         parameters[`integration.${key.substring('method.'.length)}`] = key;
       });
     }
+
+    if (http.async) {
+      parameters['integration.request.header.X-Amz-Invocation-Type'] = "'Event'";
+    }
+
     return parameters;
   },
 
diff --git a/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.js b/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.js
index a480f3978e9..f0b724b6829 100644
--- a/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.js
+++ b/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.js
@@ -385,6 +385,11 @@ module.exports = {
       }
       return normalizedIntegration;
     }
+
+    if (http.async) {
+      return 'AWS';
+    }
+
     return 'AWS_PROXY';
   },
 


Test patch:
diff --git a/lib/plugins/aws/package/compile/events/apiGateway/lib/method/index.test.js b/lib/plugins/aws/package/compile/events/apiGateway/lib/method/index.test.js
index d558150bd98..705e477ce50 100644
--- a/lib/plugins/aws/package/compile/events/apiGateway/lib/method/index.test.js
+++ b/lib/plugins/aws/package/compile/events/apiGateway/lib/method/index.test.js
@@ -379,6 +379,47 @@ describe('#compileMethods()', () => {
     });
   });
 
+  it('should add request parameter when async config is used', () => {
+    awsCompileApigEvents.validated.events = [
+      {
+        functionName: 'First',
+        http: {
+          path: 'users/create',
+          method: 'post',
+          async: true,
+        },
+      },
+    ];
+    return awsCompileApigEvents.compileMethods().then(() => {
+      expect(
+        awsCompileApigEvents.serverless.service.provider.compiledCloudFormationTemplate
+          .Resources.ApiGatewayMethodUsersCreatePost.Properties.Integration
+          .RequestParameters['integration.request.header.X-Amz-Invocation-Type']
+      ).to.equal("'Event'");
+    });
+  });
+
+  it('should add request parameter when integration type is AWS_PROXY and async', () => {
+    awsCompileApigEvents.validated.events = [
+      {
+        functionName: 'First',
+        http: {
+          path: 'users/create',
+          method: 'post',
+          integration: 'AWS_PROXY',
+          async: true,
+        },
+      },
+    ];
+    return awsCompileApigEvents.compileMethods().then(() => {
+      expect(
+        awsCompileApigEvents.serverless.service.provider.compiledCloudFormationTemplate
+          .Resources.ApiGatewayMethodUsersCreatePost.Properties.Integration
+          .RequestParameters['integration.request.header.X-Amz-Invocation-Type']
+      ).to.equal("'Event'");
+    });
+  });
+
   it('should set authorizer config for AWS_IAM', () => {
     awsCompileApigEvents.validated.events = [
       {
diff --git a/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.test.js b/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.test.js
index 54a77792660..e8fb9792993 100644
--- a/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.test.js
+++ b/lib/plugins/aws/package/compile/events/apiGateway/lib/validate.test.js
@@ -1563,6 +1563,27 @@ describe('#validate()', () => {
     expect(validated.events[0].http.integration).to.equal('MOCK');
   });
 
+  it('should support async AWS integration', () => {
+    awsCompileApigEvents.serverless.service.functions = {
+      first: {
+        events: [
+          {
+            http: {
+              method: 'GET',
+              path: 'users/list',
+              async: true,
+            },
+          },
+        ],
+      },
+    };
+
+    const validated = awsCompileApigEvents.validate();
+    expect(validated.events).to.be.an('Array').with.length(1);
+    expect(validated.events[0].http.integration).to.equal('AWS');
+    expect(validated.events[0].http.async);
+  });
+
   it('should show a warning message when using request / response config with LAMBDA-PROXY', () => {
     awsCompileApigEvents.serverless.service.functions = {
       first: {