Repository: serverless/serverless (JavaScript)
Date: 2019-07-12 01:44:23+00:00
Task type: Bug Fix

Issue:
Cross region SNS trigger
<!--
1. If you have a question and not a bug/feature request please ask it at http://forum.serverless.com
2. Please check if an issue already exists so there are no duplicates
3. Check out and follow our Guidelines: https://github.com/serverless/serverless/blob/master/CONTRIBUTING.md
4. Fill out the whole template so we have a good overview on the issue
5. Do not remove any section of the template. If something is not applicable leave it empty but leave it in the Issue
6. Please follow the template, otherwise we'll have to ask you to update it
-->

# This is a (Bug Report)

## Description

When trying to deploy a lambda with an SNS trigger in a different region the cloudformation script fails.

* What went wrong?

Cloudformation fails to create SNS event subscription for lambda with error:

`invalid parameter: TopicArn`

* What did you expect should have happened?

Expected the lambda to deploy (via cloudformation) with a trigger on an SNS topic in a different region.

* What was the config you used?

```
service: testservice
provider:
  name: aws
  runtime: nodejs6.10
  region: us-west-1

functions:
  hello:
    handler: handler.run
    events: 
      - sns: arn:aws:sns:eu-west-1:xxxxxxxxxxx:stresstest
```

* What stacktrace or error message from your provider did you see?
```
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (2.41 MB)...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - stresser-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - CREATE_COMPLETE - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - CREATE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - CREATE_IN_PROGRESS - AWS::SNS::Subscription - HelloSnsSubscriptionStresstest
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - CREATE_FAILED - AWS::SNS::Subscription - HelloSnsSubscriptionStresstest
CloudFormation - CREATE_FAILED - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - UPDATE_ROLLBACK_IN_PROGRESS - AWS::CloudFormation::Stack - stresser-dev
CloudFormation - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - stresser-dev
CloudFormation - DELETE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - DELETE_COMPLETE - AWS::SNS::Subscription - HelloSnsSubscriptionStresstest
CloudFormation - DELETE_COMPLETE - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - DELETE_COMPLETE - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - DELETE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - DELETE_COMPLETE - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - DELETE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - DELETE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - DELETE_COMPLETE - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - DELETE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - UPDATE_ROLLBACK_COMPLETE - AWS::CloudFormation::Stack - stresser-dev
```

Similar or dependent issues:
* #2183

## Additional Data

```
  Your Environment Information -----------------------------
     OS:                 darwin
     Node Version:       7.10.0
     Serverless Version: 1.14.0
```



Issue discussion:
Thanks for opening @matthisk 👍 

🤔 right now Serverless assumes that everything will be deployed into one region. @eahefnawy do you have any idea if there's a possible solution / workaround available for this?
I have the same problem.  I need to subscribe to an us-east-1 ARN for the AWS marketplace but my whole infrastructure runs in us-west-2
@pmuens I don't think this is actually a Serverless issue. I looked into this once before and IIRC it is purely a CloudFormation issue. That seems to also be indicated by the error coming back from CloudFormation. I think the issue will need to be raised with AWS directly.
Interestingly, even when I manually subscribed a Lambda function in a different region than the SNS topic, I had weird behavior. Here's what happened:

```
$ aws sns subscribe \
   --topic-arn arn:aws:sns:us-east-1:123456789012:TOPIC_NAME \
   --protocol lambda \
   --notification-endpoint arn:aws:lambda:us-west-2:987654321098:function:function-name

# That worked:
{
    "SubscriptionArn": "arn:aws:sns:us-east-1:123456789012:TOPIC_NAME:c9b553d2-3d77-4a99-ad22-52bdb744bc06"
}
```

When I went to the SNS topic in the web console, I could see the cross-account, cross-region subscription.

But then when I went to the Lambda function in the web console, it showed me this:

![image](https://user-images.githubusercontent.com/171934/28035506-dc6af678-6582-11e7-8d4b-78d3d57cadff.png)

Do we even know for sure if AWS supports cross-region SNS->Lambda subscriptions? I could not find any documentation saying they did.
Interesting 🤔 Thanks for the update and investigation on this @jthomerson 👍 

> Do we even know for sure if AWS supports cross-region SNS->Lambda subscriptions?

Not entirely sure about that. The first time I encountered it was this issue.
@jthomerson 
Q: Do my AWS Lambda functions need to be in the same region as my Amazon SNS usage?
You can subscribe your AWS Lambda functions to an Amazon SNS topic in any region.
https://aws.amazon.com/sns/faqs/

We need this on our project.  What are the next steps for follow up with AWS?  I'm sure you guys can pull more strings than we can to get this worked out with CloudFormation
@srg-avai Yes, you can subscribe cross-region like the FAQ says. However, you can not do it (or could not at the time that I wrote that post) via CloudFormation. Thus, I ended up having to make a custom resource that did the cross-region subscription, and using that custom CloudFormation resource rather than using the built-in CF SNS subscription resource. Obviously, that's not an option for the SLS team (since we'd all have to deploy custom resources to use SLS), but maybe their connections / weight with AWS could get the issue noticed if someone on their team reported it to AWS.
@jthomerson thank you, do you mind providing some more detail?  i'm interested in the custom resource approach... could you share an example to head start me (and anyone in the future with this prob) on this? 
fwiw anyone else reading this... the approach jthomerson describes above is as follows:
- implement a custom resource - use the resources example on the serverless docs and then search AWS docs for custom resources
-  the custom resource is a lambda which cloud formation calls mid-deployment, you do what you need to do in the lambda (e.g. subscribe a lambda to a topic in another region), and then you post to an endpoint (provided by cloudfront on the request) when you're done - with a success or failure property (all described in AWS custom resources docs)
I have created a Gist showing how to implement the custom resource with SLS:

https://gist.github.com/jscattergood/00a2ea6a80fe41a74c5c7efed1b238b4


This is now supported in cloudformation, however, you have to add target region within cfn.
Example:
`  SNSSampleSubscription:
    Type: AWS::SNS::Subscription

    Properties:
      Endpoint: !Sub 'arn:aws:lambda:us-east-2:2222222:function:sample_lambda'
      Protocol: lambda
      TopicArn: !Sub 'arn:aws:sns:us-east-1:1111111:osbf-pull-ami'
      Region: !Sub 'us-east-1'`

In above example, lambda is in us-east-2, and it is subscribing to an sns in us-east-1. The "Region" allows you to do that.
> This is now supported in cloudformation, however, you have to add target region within cfn.

I was able to modify my `serverless.yml` with the `Region` parameter and I can see that the subscription is created successfully against the correct topic.

serverless.yml
```yml
resources:
  Resources
    SubNameFromGeneratedCfn:
      Properties:
        Region: eu-west-1
```

I also had to ensure that `serverless-psuedo-parameters` didn't auto replace the region so just a heads up if you are using that:

```yml
custom:
  pseudoParameters:
    skipRegionReplace: true
```
Hi,
I can confirm the "Region" option is working, although I have not seen this option in the documentation. 

My "LambdaFunction" is created in a EU region and I am subscribing tot the "AmazonIpSpaceChanged" SNS notification in "us-east-1"

See CFN  json sniped below.

Regards,
Frank

`
    "LambdaAmazonIpSpaceChangedSubscription" : {

      "Type" : "AWS::SNS::Subscription",

      "Properties" : {

        "Endpoint" : {"Fn::GetAtt" : ["LambdaFunction", "Arn"] },

        "Protocol" : "lambda",

        "TopicArn" : "arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged",

        "Region": "us-east-1"

      }

    }
`


Whilst there is now a workaround available it would be great if there was first class support for this on the sns event in serverless.
The sns topic is in the region eu-west-1 (not created using cloud formation) so permission is added manually in topic policy and lambda is in different region us-west-2 and using cloud formation. How to provide permission in this case? As I am getting same error "invalid parameter: TopicArn".

{
      "Sid": "Subscribe-to-topic",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::xxxxx:root"
      },
      "Action": "sns:Subscribe",
      "Resource": "arn:aws:sns:eu-west-1:xxxxx:publish-notifier"
}
@Imran99   Could you please share you `serverless.yml` ?
Hey all - sorry if this is just me being dense, but this took me a while to figure out and I want to add a couple clarifications in case they help out others who run into this issue.


My lambda functions are in us-east-2, but I wanted to use a SNS queue in us-east-1 as a trigger.

I originally tried this in my serverless.yml file, but I got the "invalid parameter: TopicArn" error (like the original poster).  
```
handleBounceOrComplaintEmailSNS:
  handler: server.handleBounceOrComplaintEmailSNS
  events:
    - sns:
        arn: arn:aws:sns:us-east-1:0005555555555:email-bounces-complaints
```

I didn't know enough about serverless or cloudformation to know what @Imran99 meant by _SubNameFromGeneratedCfn_, so it took a lot of head scratching and searching.  
After staring at my _.serverless/cloudformation-template-update-stack.json_ for a while (and several failed attempts), I finally sorted it out.  By searching for my SNS's ARN in _.serverless/cloudformation-template-update-stack.json_, I found that the _SubNameFromGeneratedCfn_ Imran99 was referring to was HandleBounceOrComplaintEmailSNSSnsSubscriptionemailbouncescomplaints (in my template).  By using that name under resources, I was able get it working!  So these two components of my serverless.yml file are: 


```
handleBounceOrComplaintEmailSNS:
  handler: server.handleBounceOrComplaintEmailSNS
  events:
    - sns:
        arn: arn:aws:sns:us-east-1:0005555555555:email-bounces-complaints

resources:  
  Resources:
    HandleBounceOrComplaintEmailSNSSnsSubscriptionemailbouncescomplaints:
      Type: AWS::SNS::Subscription
      Properties:
        Region: us-east-1
```

Bottom line: use Imran99's workaround, but note you'll need to search _.serverless/cloudformation-template-update-stack.json_ to figure out _SubNameFromGeneratedCfn_.  Thank you all for the workaround!
> Whilst there is now a workaround available it would be great if there was first class support for this on the sns event in serverless.

@dschep why was this issue closed? I do not see any messages from maintainers as to whether this issue will be addressed. There isn't a pending PR either.

Solution patch:
diff --git a/docs/providers/aws/events/sns.md b/docs/providers/aws/events/sns.md
index 738f5e9b4b8..fdc442a166b 100644
--- a/docs/providers/aws/events/sns.md
+++ b/docs/providers/aws/events/sns.md
@@ -62,6 +62,7 @@ functions:
 ```
 
 Or with intrinsic CloudFormation function like `Fn::Join` or `Fn::GetAtt`.
+**Note:** The arn can be in a different region to enable cross region invocation
 
 ```yml
 functions:
diff --git a/lib/plugins/aws/package/compile/events/sns/index.js b/lib/plugins/aws/package/compile/events/sns/index.js
index 4d065c677a5..b1fe069945d 100644
--- a/lib/plugins/aws/package/compile/events/sns/index.js
+++ b/lib/plugins/aws/package/compile/events/sns/index.js
@@ -3,10 +3,10 @@
 const _ = require('lodash');
 
 class AwsCompileSNSEvents {
-  constructor(serverless) {
+  constructor(serverless, options) {
     this.serverless = serverless;
     this.provider = this.serverless.getProvider('aws');
-
+    this.options = options;
     this.hooks = {
       'package:compileEvents': this.compileSNSEvents.bind(this),
     };
@@ -49,8 +49,8 @@ class AwsCompileSNSEvents {
           if (event.sns) {
             let topicArn;
             let topicName;
+            let region;
             let displayName = '';
-
             if (typeof event.sns === 'object') {
               if (event.sns.arn) {
                 topicArn = event.sns.arn;
@@ -64,6 +64,9 @@ class AwsCompileSNSEvents {
                   if (topicArn.indexOf('arn:') === 0) {
                     const splitArn = topicArn.split(':');
                     topicName = splitArn[splitArn.length - 1];
+                    if (splitArn[3] !== this.options.region) {
+                      region = splitArn[3];
+                    }
                   } else {
                     throw new this.serverless.classes.Error(
                       this.invalidPropertyErrorMessage(functionName, 'arn')
@@ -132,6 +135,7 @@ class AwsCompileSNSEvents {
                     Protocol: 'lambda',
                     Endpoint: endpoint,
                     FilterPolicy: event.sns.filterPolicy,
+                    Region: region,
                   },
                 },
               });


Test patch:
diff --git a/lib/plugins/aws/package/compile/events/sns/index.test.js b/lib/plugins/aws/package/compile/events/sns/index.test.js
index c312459c48c..18b72af715e 100644
--- a/lib/plugins/aws/package/compile/events/sns/index.test.js
+++ b/lib/plugins/aws/package/compile/events/sns/index.test.js
@@ -11,9 +11,12 @@ describe('AwsCompileSNSEvents', () => {
 
   beforeEach(() => {
     serverless = new Serverless();
+    const options = {
+      region: 'some-region',
+    };
     serverless.service.provider.compiledCloudFormationTemplate = { Resources: {} };
     serverless.setProvider('aws', new AwsProvider(serverless));
-    awsCompileSNSEvents = new AwsCompileSNSEvents(serverless);
+    awsCompileSNSEvents = new AwsCompileSNSEvents(serverless, options);
   });
 
   describe('#constructor()', () => {
@@ -273,6 +276,39 @@ describe('AwsCompileSNSEvents', () => {
       ).to.equal('AWS::Lambda::Permission');
     });
 
+    it('should create a cross region subscription when SNS topic arn in a different region than provider', () => {
+      awsCompileSNSEvents.serverless.service.functions = {
+        first: {
+          events: [
+            {
+              sns: {
+                arn: 'arn:aws:sns:some-other-region:accountid:foo',
+              },
+            },
+          ],
+        },
+      };
+
+      awsCompileSNSEvents.compileSNSEvents();
+      expect(
+        Object.keys(
+          awsCompileSNSEvents.serverless.service.provider.compiledCloudFormationTemplate.Resources
+        )
+      ).to.have.length(2);
+      expect(
+        awsCompileSNSEvents.serverless.service.provider.compiledCloudFormationTemplate.Resources
+          .FirstSnsSubscriptionFoo.Type
+      ).to.equal('AWS::SNS::Subscription');
+      expect(
+        awsCompileSNSEvents.serverless.service.provider.compiledCloudFormationTemplate.Resources
+          .FirstSnsSubscriptionFoo.Properties.Region
+      ).to.equal('some-other-region');
+      expect(
+        awsCompileSNSEvents.serverless.service.provider.compiledCloudFormationTemplate.Resources
+          .FirstLambdaPermissionFooSNS.Type
+      ).to.equal('AWS::Lambda::Permission');
+    });
+
     it('should throw an error when the arn an object and the value is not a string', () => {
       awsCompileSNSEvents.serverless.service.functions = {
         first: {