Repository: serverless/serverless (JavaScript)
Date: 2018-01-07 16:35:47+00:00
Task type: Bug Fix

Issue:
ServerlessError: An error occurred: ServerlessDeploymentBucket - API: s3:setBucketAccelerateConfiguration Access Denied.
<!--
1. If you have a question and not a bug/feature request please ask it at http://forum.serverless.com
2. Please check if an issue already exists so there are no duplicates
3. Check out and follow our Guidelines: https://github.com/serverless/serverless/blob/master/CONTRIBUTING.md
4. Fill out the whole template so we have a good overview on the issue
5. Do not remove any section of the template. If something is not applicable leave it empty but leave it in the Issue
6. Please follow the template, otherwise we'll have to ask you to update it
-->

# This is a Bug Report
## Description

For bug reports:
* What went wrong?
The deploy failed with error `ServerlessError: An error occurred: ServerlessDeploymentBucket - API: s3:setBucketAccelerateConfiguration Access Denied.`. We've been using serverless with the same IAM user for the past few months, and this is the first time it's happened.
* What did you expect should have happened?
I'm guessing this has something to do with the new feature added for bucketAcceleration. I would expect that the addition of that feature would not break current deploys -- I don't want bucket acceleration, so I shouldn't have to give that permission to my IAM user.
* What was the config you used?
* What stacktrace or error message from your provider did you see?
```ServerlessError: An error occurred: ServerlessDeploymentBucket - API: s3:setBucketAccelerateConfiguration Access Denied.
    at provider.request.then (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/lib/monitorStack.js:112:33)
From previous event:
    at AwsDeploy.monitorStack (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/lib/monitorStack.js:26:12)
    at provider.request.then (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/lib/updateStack.js:84:30)
From previous event:
    at AwsDeploy.update (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/lib/updateStack.js:84:8)
From previous event:
    at AwsDeploy.BbPromise.bind.then (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/lib/updateStack.js:101:12)
From previous event:
    at AwsDeploy.updateStack (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/lib/updateStack.js:95:8)
From previous event:
    at AwsDeploy.BbPromise.bind.then (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:135:39)
From previous event:
    at Object.AwsDeploy.hooks.aws:deploy:deploy:updateStack [as hook] (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:131:10)
    at BbPromise.reduce (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/classes/PluginManager.js:368:55)
From previous event:
    at PluginManager.invoke (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/classes/PluginManager.js:368:22)
    at PluginManager.spawn (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/classes/PluginManager.js:386:17)
    at AwsDeploy.BbPromise.bind.then (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:101:48)
From previous event:
    at Object.AwsDeploy.hooks.deploy:deploy [as hook] (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:97:10)
    at BbPromise.reduce (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/classes/PluginManager.js:368:55)
From previous event:
    at PluginManager.invoke (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/classes/PluginManager.js:368:22)
    at PluginManager.run (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/classes/PluginManager.js:399:17)
    at variables.populateService.then (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/Serverless.js:102:33)
    at tryOnImmediate (timers.js:543:15)
    at processImmediate [as _immediateCallback] (timers.js:523:5)
From previous event:
    at Serverless.run (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/lib/Serverless.js:89:74)
    at serverless.init.then (/opt/circleci/nodejs/v6.1.0/lib/node_modules/serverless/bin/serverless:42:50)
```
For feature proposals:
* What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.
* If there is additional config how would it look

Similar or dependent issues:
* #12345

## Additional Data

* ***Serverless Framework Version you're using***: 1.25.0
* ***Operating System***:
* ***Stack Trace***:
* ***Provider Error messages***:



Issue discussion:
Hi @dperszyk, thank you for repoting :+1: :smile: 

>I'm guessing this has something to do with the new feature added for bucketAcceleration.

I think so.
Which commands did you run when facing the error? `sls deploy`?
Additionally, could you tell us policies list that the IAM user is attached so that we can reproduce it on our end?
Hi @horike37 !

Thanks for responding so quickly. 
I ran `sls deploy --stage MYENV --aws-profile MYAWSPROFILE`.

Here's the policy for the IAM user, but with the resource names changed: 

```        Statement=[
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("s3", "GetObject"),
                        Action("s3", "PutObject"),
                        Action("s3", "ListBucket"),
                        Action("s3", "DeleteObject"),
                        Action("s3", "DeleteBucket"),
                        Action("s3", "ListBucketVersions"),
                        Action("s3", "GetBucketNotification"),
                        Action("s3", "PutBucketNotification")
                    ],
                    Resource=[
                        "arn:aws:s3:::mylambdas*serverlessdeploymentbucket*"
                    ]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("s3", "CreateBucket"),
                    ],
                    Resource=[
                        "arn:aws:s3:::*"
                    ]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("iam", "CreateRole"),
                        Action("iam", "PassRole"),
                        Action("iam", "AttachRolePolicy"),
                        Action("iam", "CreatePolicy"),
                        Action("apigateway", "GET"),
                        Action("apigateway", "POST"),
                        Action("apigateway", "PUT"),
                        Action("apigateway", "DELETE")
                    ],
                    Resource=["*"]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("cloudformation", "List*"), 
                        Action("cloudformation", "Get*"), 
                        Action("cloudformation", "PreviewStackUpdate"),
                        Action("cloudformation", "ValidateTemplate")
                    ],
                    Resource=[
                        "*"
                    ]
                ), 
                Statement(
                    Effect=Allow, 
                    Action=[
                        Action("cloudformation", "CreateStack"),
                        Action("cloudformation", "CreateUploadBucket"),
                        Action("cloudformation", "DeleteStack"),
                        Action("cloudformation", "DescribeStackEvents"),
                        Action("cloudformation", "DescribeStackResource"),
                        Action("cloudformation", "DescribeStackResources"),
                        Action("cloudformation", "UpdateStack"),
                        Action("cloudformation", "DescribeStacks")
                    ], 
                    Resource=[
                        "arn:aws:cloudformation:*:*:stack/mylambdas-*/*"
                    ]
                ),
                Statement(
                    Effect=Allow, 
                    Action=[
                        Action("lambda", "*")
                    ], 
                    Resource=[
                        "*"
                    ]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("iam", "*")
                    ],
                    Resource=["arn:aws:iam::*:role/mylambdas-*-*-myLambdaRole"]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("sqs", "*")
                    ],
                    Resource=[
                        "arn:aws:sqs:*:*:mylambdas-*-*"
                    ]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("kinesis", "*")
                    ],
                    Resource=["arn:aws:kinesis:*:*:stream/mylambdas-*-*"]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("events", "*")
                    ],
                    Resource=["*"]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("dynamodb", "*")
                    ],
                    Resource=[
                        "*"
                    ]
                ),
                Statement(
                    Effect=Allow,
                    Action=[
                        Action("cloudwatch", "GetMetricsStatistics")
                    ],
                    Resource=["*"]
                ), 
                Statement(
                    Effect=Allow, 
                    Action=[
                        Action("logs", "CreateLogGroup"),
                        Action("logs", "DeleteLogGroup"), 
                        Action("logs", "CreateLogStream"), 
                        Action("logs", "PutLogEvents"),
                        Action("logs", "DescribeLogStreams"),
                        Action("logs", "DescribeLogGroups"),
                        Action("logs", "FilterLogEvents")
                    ], 
                    Resource=[
                        "*"
                    ]
                )
            ]
```

Let me know if you need anything else. Thanks again!
Thank you for the detail @dperszyk :+1: 

I took a deep debugging.
The root cause is that `AccelerateConfiguration` is needed within generated CloudFormation template even if you deploy without `--aws-s3-accelerate`.
Here is `cloudformation-template-create-stack.json` of this case. As a result, CF needs `PutAccelerateConfiguration` policy whether with the config or not.

```
"Resources": {
    "ServerlessDeploymentBucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "AccelerateConfiguration": {
          "AccelerationStatus": "Suspended"
        }
      }
    }
  }
```

Imo, That's a breaking change for users who set up minimum IAM policy.
It would be good to remove `AccelerateConfiguration` statement if you deploy without the config if it is possible.

cc: @alexcasalboni 
@horike37 I think you are right. You can find the related code [here](https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/lib/generateCoreTemplate.js#L52).

The `Suspended` value was intended to turn the feature off as soon as you stop using `--aws-s3-accelerate`. This is, in fact, problematic without `s3:PutAccelerateConfiguration` permissions.

As a quick fix, we could simply disable the `Suspended` value and only use `Enabled`. Please note that if we do NOT force the `Suspended` value, we won't disable Transfer Acceleration when you deploy without `--aws-s3-accelerate` (we discussed this case in #4293). That's objectively less annoying than this breaking change.

What do you think?
@alexcasalboni 

>Please note that if we do NOT force the Suspended value, we won't disable Transfer Acceleration >when you deploy without --aws-s3-accelerate (we discussed this case in #4293). 

Ah, I see. TBH, I can't think of any best solution 😅 😅 😅 
It would be good to collect some feedback from someone else of the community.

cc: @serverless/framework-maintainers 
Hi @horike37 , @alexcasalboni . I would also go without the automatic disable. Having clean minimal permissions is rated higher for me.
What about having a separate switch to disable the acceleration on demand again on the deployment bucket? So --aws-s3-accelerate would turn it on and `--no-aws-s3-accelerate` would be the symmetric part to turn it off again explicitly.

Solution patch:
diff --git a/docs/providers/aws/cli-reference/deploy.md b/docs/providers/aws/cli-reference/deploy.md
index 66204c690a9..8bdeced0998 100644
--- a/docs/providers/aws/cli-reference/deploy.md
+++ b/docs/providers/aws/cli-reference/deploy.md
@@ -25,7 +25,8 @@ serverless deploy
 - `--verbose` or `-v` Shows all stack events during deployment, and display any Stack Output.
 - `--function` or `-f` Invoke `deploy function` (see above). Convenience shortcut - cannot be used with `--package`.
 - `--conceal` Hides secrets from the output (e.g. API Gateway key values).
-- `--aws-s3-accelerate` Enables S3 Transfer Acceleration making uploading artifacts much faster. You can read more about it [here](http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html). **Note: When using Transfer Acceleration, additional data transfer charges may apply**
+- `--aws-s3-accelerate` Enables S3 Transfer Acceleration making uploading artifacts much faster. You can read more about it [here](http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html). It requires additional `s3:PutAccelerateConfiguration` permissions. **Note: When using Transfer Acceleration, additional data transfer charges may apply.**
+- `--no-aws-s3-accelerate` Explicitly disables S3 Transfer Acceleration). It also requires additional `s3:PutAccelerateConfiguration` permissions.
 
 ## Artifacts
 
diff --git a/lib/plugins/aws/package/lib/generateCoreTemplate.js b/lib/plugins/aws/package/lib/generateCoreTemplate.js
index 09a20d4ddee..fae3c200071 100644
--- a/lib/plugins/aws/package/lib/generateCoreTemplate.js
+++ b/lib/plugins/aws/package/lib/generateCoreTemplate.js
@@ -25,6 +25,14 @@ module.exports = {
 
     const bucketName = this.serverless.service.provider.deploymentBucket;
     const isS3TransferAccelerationEnabled = this.provider.isS3TransferAccelerationEnabled();
+    const isS3TransferAccelerationDisabled = this.provider.isS3TransferAccelerationDisabled();
+
+    if (isS3TransferAccelerationEnabled && isS3TransferAccelerationDisabled) {
+      const errorMessage = [
+        'You cannot enable and disable S3 Transfer Acceleration at the same time',
+      ].join('');
+      return BbPromise.reject(new this.serverless.classes.Error(errorMessage));
+    }
 
     if (bucketName) {
       return BbPromise.bind(this)
@@ -45,17 +53,25 @@ module.exports = {
         });
     }
 
-    this.serverless.service.provider.compiledCloudFormationTemplate
-      .Resources.ServerlessDeploymentBucket.Properties = {
-        AccelerateConfiguration: {
-          AccelerationStatus:
-            isS3TransferAccelerationEnabled ? 'Enabled' : 'Suspended',
-        },
-      };
-
     if (isS3TransferAccelerationEnabled) {
+      // enable acceleration via CloudFormation
+      this.serverless.service.provider.compiledCloudFormationTemplate
+        .Resources.ServerlessDeploymentBucket.Properties = {
+          AccelerateConfiguration: {
+            AccelerationStatus: 'Enabled',
+          },
+        };
+      // keep track of acceleration status via CloudFormation Output
       this.serverless.service.provider.compiledCloudFormationTemplate
       .Outputs.ServerlessDeploymentBucketAccelerated = { Value: true };
+    } else if (isS3TransferAccelerationDisabled) {
+      // explicitly disable acceleration via CloudFormation
+      this.serverless.service.provider.compiledCloudFormationTemplate
+        .Resources.ServerlessDeploymentBucket.Properties = {
+          AccelerateConfiguration: {
+            AccelerationStatus: 'Suspended',
+          },
+        };
     }
 
     const coreTemplateFileName = this.provider.naming.getCoreTemplateFileName();
diff --git a/lib/plugins/aws/provider/awsProvider.js b/lib/plugins/aws/provider/awsProvider.js
index 4c79f14bef1..bcc78e58372 100644
--- a/lib/plugins/aws/provider/awsProvider.js
+++ b/lib/plugins/aws/provider/awsProvider.js
@@ -317,6 +317,10 @@ class AwsProvider {
     return !!this.options['aws-s3-accelerate'];
   }
 
+  isS3TransferAccelerationDisabled() {
+    return !!this.options['no-aws-s3-accelerate'];
+  }
+
   disableTransferAccelerationForCurrentDeploy() {
     delete this.options['aws-s3-accelerate'];
   }


Test patch:
diff --git a/lib/plugins/aws/package/lib/generateCoreTemplate.test.js b/lib/plugins/aws/package/lib/generateCoreTemplate.test.js
index a089d536950..e4cb5eba22f 100644
--- a/lib/plugins/aws/package/lib/generateCoreTemplate.test.js
+++ b/lib/plugins/aws/package/lib/generateCoreTemplate.test.js
@@ -61,14 +61,45 @@ describe('#generateCoreTemplate()', () => {
 
     return expect(awsPlugin.generateCoreTemplate()).to.be.fulfilled
       .then(() => {
+        const template = awsPlugin.serverless.service.provider.compiledCloudFormationTemplate;
         expect(
-          awsPlugin.serverless.service.provider.compiledCloudFormationTemplate
-            .Outputs.ServerlessDeploymentBucketName.Value
+          template.Outputs.ServerlessDeploymentBucketName.Value
         ).to.equal(bucketName);
         // eslint-disable-next-line no-unused-expressions
         expect(
-          awsPlugin.serverless.service.provider.compiledCloudFormationTemplate
-            .Resources.ServerlessDeploymentBucket
+          template.Resources.ServerlessDeploymentBucket
+        ).to.not.exist;
+      });
+  });
+
+  it('should use a custom bucket if specified, even with S3 transfer acceleration', () => {
+    const bucketName = 'com.serverless.deploys';
+
+    awsPlugin.serverless.service.provider.deploymentBucket = bucketName;
+    awsPlugin.provider.options['aws-s3-accelerate'] = true;
+
+    const coreCloudFormationTemplate = awsPlugin.serverless.utils.readFileSync(
+      path.join(
+        __dirname,
+        'core-cloudformation-template.json'
+      )
+    );
+    awsPlugin.serverless.service.provider
+      .compiledCloudFormationTemplate = coreCloudFormationTemplate;
+
+    return expect(awsPlugin.generateCoreTemplate()).to.be.fulfilled
+      .then(() => {
+        const template = awsPlugin.serverless.service.provider.compiledCloudFormationTemplate;
+        expect(
+          template.Outputs.ServerlessDeploymentBucketName.Value
+        ).to.equal(bucketName);
+        // eslint-disable-next-line no-unused-expressions
+        expect(
+          template.Resources.ServerlessDeploymentBucket
+        ).to.not.exist;
+        // eslint-disable-next-line no-unused-expressions
+        expect(
+          template.Outputs.ServerlessDeploymentBucketAccelerated
         ).to.not.exist;
       });
   });
@@ -80,11 +111,6 @@ describe('#generateCoreTemplate()', () => {
           .Resources.ServerlessDeploymentBucket
         ).to.be.deep.equal({
           Type: 'AWS::S3::Bucket',
-          Properties: {
-            AccelerateConfiguration: {
-              AccelerationStatus: 'Suspended',
-            },
-          },
         });
     })
   );
@@ -114,4 +140,36 @@ describe('#generateCoreTemplate()', () => {
         expect(template.Outputs.ServerlessDeploymentBucketAccelerated.Value).to.equal(true);
       });
   });
+
+  it('should explicitly disable S3 Transfer Acceleration, if requested', () => {
+    sinon.stub(awsPlugin.provider, 'request').resolves();
+    sinon.stub(serverless.utils, 'writeFileSync').resolves();
+    serverless.config.servicePath = './';
+    awsPlugin.provider.options['no-aws-s3-accelerate'] = true;
+
+    return awsPlugin.generateCoreTemplate()
+      .then(() => {
+        const template = serverless.service.provider.coreCloudFormationTemplate;
+        expect(template.Resources.ServerlessDeploymentBucket).to.be.deep.equal({
+          Type: 'AWS::S3::Bucket',
+          Properties: {
+            AccelerateConfiguration: {
+              AccelerationStatus: 'Suspended',
+            },
+          },
+        });
+      });
+  });
+
+  it('should explode if transfer acceleration is both enabled and disabled', () => {
+    sinon.stub(awsPlugin.provider, 'request').resolves();
+    sinon.stub(serverless.utils, 'writeFileSync').resolves();
+    serverless.config.servicePath = './';
+    awsPlugin.provider.options['aws-s3-accelerate'] = true;
+    awsPlugin.provider.options['no-aws-s3-accelerate'] = true;
+
+    return expect(
+      awsPlugin.generateCoreTemplate()
+    ).to.be.rejectedWith(serverless.classes.Error, /at the same time/);
+  });
 });