Repository: serverless/serverless (JavaScript)
Date: 2017-06-16 07:21:01+00:00
Task type: Feature

Issue:
Deploy fails when using deploymentBucket / SSE support
I'm trying to use `serverless deploy` with a pre-existing bucket for the deployment package.  When I add the `deploymentBucket` value in the provider I get the following error:  **self signed certificate in certificate chain**

If I remove `deploymentBucket` the CF stack attempts to create but I don't have permissions to create buckets (nor do I want to).

Can I use the issue described here #2189 to define a pre-existing target S3 bucket where packages are uploaded to?

```
$ serverless deploy -v
 
  Serverless Error ---------------------------------------
 
     self signed certificate in certificate chain
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
 
  Your Environment Information -----------------------------
     OS:                 darwin
     Node Version:       7.2.1
     Serverless Version: 1.3.0
```

Here is a snippit from my serverless.yml
```
provider:
  name: aws
  runtime: python2.7
  stage: dev
  region: us-west-2
  deploymentBucket: my-cool-bucket
  profile: sdrad-workflow-dev
```
Any ideas?


Issue discussion:
I think this might be caused by the requirement on our buckets for server side encryption.  I'll try and test later with a bucket without this policy.

```
        {
            "Sid": "DenyNoEncryption",
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::hli-workflow-sdrad-pdx/*",
            "Condition": {
                "StringNotEquals": {
                    "s3:x-amz-server-side-encryption": "AES256"
                }
            }
        },
```
@gmetzker thanks for the update!

Yes, I've also found these links:
https://forums.aws.amazon.com/message.jspa?messageID=604354
http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_server-certs.html

Seems like something has to be configured on your end?! Would be great if you could report back once you've hopefully resolved this problem 👍 
It turns out this encryption policy was causing my issue.  When I use a bucket that is less strict I don't get this error.

It seems like it might be helpful to support the various S3 encryption options.  Maybe on the provider there we should implement a section that specifies all the standard ss3/kms options for the s3 uploads.

```
deploymentBucketOptions:
  deploymentBucket: {bucket name}
  sse: {AES256|aws:kms}
  sseKmsKeyId:  {kms keyid for serverside encryption}
  sseCustomerKey:  {Customer Provided Encryption Key use for server-side encryption}

```

@gmetzker thanks for getting back and investigating further!
+1 for SSE KMS key ID for S3 uploads.
Supporting S3 encryption is crucial for devs at my job. We have two teams blocked by this issue and it's a showstopper for adoption by additional teams. Should we fork, or is this feature on the near-future roadmap?
@MetaThis thanks for responding! Right now there's no direct plan to add the feature in the upcoming version.

Seems like an important enterprise-level feature though.

/cc @brianneisler @eahefnawy 
I took a rough stab at this. I don't have the time to create a pull request and add tests, but hopefully someone can iterate on this. This expands on https://github.com/serverless/serverless/issues/2888#issuecomment-266600618.

https://github.com/serverless/serverless/blob/master/lib/plugins/aws/deploy/lib/uploadArtifacts.js#L15-L20

```diff
const params = {
  Bucket: this.bucketName,
  Key: `${this.serverless.service.package.artifactDirectoryName}/${compiledTemplateFileName}`,
  Body: body,
  ContentType: 'application/json',
+ ServerSideEncryption: this.serverless.service.provider.deploymentBucketOptions.sse || '',
+ SSEKMSKeyId: this.serverless.service.provider.deploymentBucketOptions.sseKmsKeyId || '',
+ SSECustomerKey: this.serverless.service.provider.deploymentBucketOptions.sseCustomerKey || '',
};
```

https://github.com/serverless/serverless/blob/master/lib/plugins/aws/deploy/lib/uploadArtifacts.js#L32-L37

```diff
const params = {
  Bucket: this.bucketName,
  Key: `${this.serverless.service.package.artifactDirectoryName}/${fileName}`,
  Body: fs.createReadStream(artifactFilePath),
  ContentType: 'application/zip',
+ ServerSideEncryption: this.serverless.service.provider.deploymentBucketOptions.sse || '',
+ SSEKMSKeyId: this.serverless.service.provider.deploymentBucketOptions.sseKmsKeyId || '',
+ SSECustomerKey: this.serverless.service.provider.deploymentBucketOptions.sseCustomerKey || '',
};
```

Solution patch:
diff --git a/docs/providers/aws/guide/deploying.md b/docs/providers/aws/guide/deploying.md
index 2550bd36282..e412ccc1ad8 100644
--- a/docs/providers/aws/guide/deploying.md
+++ b/docs/providers/aws/guide/deploying.md
@@ -61,6 +61,9 @@ The Serverless Framework translates all syntax in `serverless.yml` to a single A
   serverless deploy --stage production --region eu-central-1
   ```
 
+* You can specify your own S3 bucket which should be used to store all the deployment artifacts.
+  The `deploymentBucket` config which is nested under `provider` lets you e.g. set the `name` or the `serverSideEncryption` method for this bucket
+
 Check out the [deploy command docs](../cli-reference/deploy.md) for all details and options.
 
 ## Deploy Function
diff --git a/docs/providers/aws/guide/serverless.yml.md b/docs/providers/aws/guide/serverless.yml.md
index 547eadea194..dd28f42c2dd 100644
--- a/docs/providers/aws/guide/serverless.yml.md
+++ b/docs/providers/aws/guide/serverless.yml.md
@@ -31,7 +31,9 @@ provider:
   profile: production # The default profile to use with this service
   memorySize: 512 # Overwrite the default memory size. Default is 1024
   timeout: 10 # The default is 6
-  deploymentBucket: com.serverless.${self:provider.region}.deploys # Deployment bucket name. Default is generated by the framework
+  deploymentBucket:
+    name: com.serverless.${self:provider.region}.deploys # Deployment bucket name. Default is generated by the framework
+    serverSideEncryption: AES256 # when using server-side encryption
   role: arn:aws:iam::XXXXXX:role/role # Overwrite the default IAM role which is used for all functions
   cfnRole: arn:aws:iam::XXXXXX:role/role # ARN of an IAM role for CloudFormation service. If specified, CloudFormation uses the role's credentials
   versionFunctions: false # Optional function versioning
diff --git a/docs/providers/aws/guide/services.md b/docs/providers/aws/guide/services.md
index c5c67356fcb..ee776f9e22a 100644
--- a/docs/providers/aws/guide/services.md
+++ b/docs/providers/aws/guide/services.md
@@ -94,7 +94,9 @@ provider:
   region: us-east-1 # Overwrite the default region used. Default is us-east-1
   profile: production # The default profile to use with this service
   memorySize: 512 # Overwrite the default memory size. Default is 1024
-  deploymentBucket: com.serverless.${self:provider.region}.deploys # Overwrite the default deployment bucket
+  deploymentBucket:
+    name: com.serverless.${self:provider.region}.deploys # Overwrite the default deployment bucket
+    serverSideEncryption: AES256 # when using server-side encryption
   versionFunctions: false # Optional function versioning
   stackTags: # Optional CF stack tags
    key: value
diff --git a/lib/classes/Service.js b/lib/classes/Service.js
index 49707a6e601..8bb225bc81b 100644
--- a/lib/classes/Service.js
+++ b/lib/classes/Service.js
@@ -7,7 +7,6 @@ const BbPromise = require('bluebird');
 const semver = require('semver');
 
 class Service {
-
   constructor(serverless, data) {
     this.serverless = serverless;
 
@@ -110,6 +109,7 @@ class Service {
           that.serviceObject = { name: serverlessFile.service };
           that.service = serverlessFile.service;
         }
+
         that.custom = serverlessFile.custom;
         that.plugins = serverlessFile.plugins;
         that.resources = serverlessFile.resources;
diff --git a/lib/plugins/aws/deploy/lib/uploadArtifacts.js b/lib/plugins/aws/deploy/lib/uploadArtifacts.js
index 6c7ad61d24d..36065bca007 100644
--- a/lib/plugins/aws/deploy/lib/uploadArtifacts.js
+++ b/lib/plugins/aws/deploy/lib/uploadArtifacts.js
@@ -1,5 +1,7 @@
 'use strict';
 
+/* eslint-disable no-use-before-define */
+
 const fs = require('fs');
 const BbPromise = require('bluebird');
 const filesize = require('filesize');
@@ -12,13 +14,18 @@ module.exports = {
     const compiledTemplateFileName = 'compiled-cloudformation-template.json';
 
     const body = JSON.stringify(this.serverless.service.provider.compiledCloudFormationTemplate);
-    const params = {
+    let params = {
       Bucket: this.bucketName,
       Key: `${this.serverless.service.package.artifactDirectoryName}/${compiledTemplateFileName}`,
       Body: body,
       ContentType: 'application/json',
     };
 
+    const deploymentBucketObject = this.serverless.service.provider.deploymentBucketObject;
+    if (deploymentBucketObject) {
+      params = setServersideEncryptionOptions(params, deploymentBucketObject);
+    }
+
     return this.provider.request('S3',
       'putObject',
       params,
@@ -29,13 +36,18 @@ module.exports = {
   uploadZipFile(artifactFilePath) {
     const fileName = artifactFilePath.split(path.sep).pop();
 
-    const params = {
+    let params = {
       Bucket: this.bucketName,
       Key: `${this.serverless.service.package.artifactDirectoryName}/${fileName}`,
       Body: fs.createReadStream(artifactFilePath),
       ContentType: 'application/zip',
     };
 
+    const deploymentBucketObject = this.serverless.service.provider.deploymentBucketObject;
+    if (deploymentBucketObject) {
+      params = setServersideEncryptionOptions(params, deploymentBucketObject);
+    }
+
     return this.provider.request('S3',
       'putObject',
       params,
@@ -84,3 +96,23 @@ module.exports = {
       .then(this.uploadFunctions);
   },
 };
+
+function setServersideEncryptionOptions(putParams, deploymentBucketOptions) {
+  const encryptionFields = [
+    ['serverSideEncryption', 'ServerSideEncryption'],
+    ['sseCustomerAlgorithim', 'SSECustomerAlgorithm'],
+    ['sseCustomerKey', 'SSECustomerKey'],
+    ['sseCustomerKeyMD5', 'SSECustomerKeyMD5'],
+    ['sseKMSKeyId', 'SSEKMSKeyId'],
+  ];
+
+  const params = putParams;
+
+  encryptionFields.forEach((element) => {
+    if (deploymentBucketOptions[element[0]]) {
+      params[element[1]] = deploymentBucketOptions[element[0]];
+    }
+  }, this);
+
+  return params;
+}
diff --git a/lib/plugins/aws/provider/awsProvider.js b/lib/plugins/aws/provider/awsProvider.js
index 3d0f48cef34..24c2e38545a 100644
--- a/lib/plugins/aws/provider/awsProvider.js
+++ b/lib/plugins/aws/provider/awsProvider.js
@@ -120,6 +120,21 @@ class AwsProvider {
     if (timeout) {
       AWS.config.httpOptions.timeout = parseInt(timeout, 10);
     }
+
+    // Support deploymentBucket configuration as an object
+    const provider = this.serverless.service.provider;
+    if (provider && provider.deploymentBucket) {
+      if (_.isObject(provider.deploymentBucket)) {
+        // store the object in a new variable so that it can be reused later on
+        provider.deploymentBucketObject = provider.deploymentBucket;
+        if (provider.deploymentBucket.name) {
+          // (re)set the value of the deploymentBucket property to the name (which is a string)
+          provider.deploymentBucket = provider.deploymentBucket.name;
+        } else {
+          provider.deploymentBucket = null;
+        }
+      }
+    }
   }
 
   request(service, method, params) {
@@ -194,6 +209,13 @@ class AwsProvider {
     impl.addEnvironmentProfile(result, `AWS_${stageUpper}`);
 
     result.region = this.getRegion();
+
+    const deploymentBucketObject = this.serverless.service.provider.deploymentBucketObject;
+    if (deploymentBucketObject && deploymentBucketObject.serverSideEncryption
+      && deploymentBucketObject.serverSideEncryption === 'aws:kms') {
+      result.signatureVersion = 'v4';
+    }
+
     return result;
   }
 


Test patch:
diff --git a/lib/plugins/aws/deploy/lib/uploadArtifacts.test.js b/lib/plugins/aws/deploy/lib/uploadArtifacts.test.js
index dedac9f6c1e..9f6d0180990 100644
--- a/lib/plugins/aws/deploy/lib/uploadArtifacts.test.js
+++ b/lib/plugins/aws/deploy/lib/uploadArtifacts.test.js
@@ -59,6 +59,37 @@ describe('uploadArtifacts', () => {
         awsDeploy.provider.request.restore();
       });
     });
+
+    it('should upload to a bucket with server side encryption bucket policy', () => {
+      awsDeploy.serverless.service.provider.compiledCloudFormationTemplate = { key: 'value' };
+      awsDeploy.serverless.service.provider.deploymentBucketObject = {
+        serverSideEncryption: 'AES256',
+      };
+
+      const putObjectStub = sinon
+        .stub(awsDeploy.provider, 'request').resolves();
+
+      return awsDeploy.uploadCloudFormationFile().then(() => {
+        expect(putObjectStub.calledOnce).to.be.equal(true);
+        expect(putObjectStub.calledWithExactly(
+          'S3',
+          'putObject',
+          {
+            Bucket: awsDeploy.bucketName,
+            Key: `${awsDeploy.serverless.service.package
+              .artifactDirectoryName}/compiled-cloudformation-template.json`,
+            Body: JSON.stringify(awsDeploy.serverless.service.provider
+              .compiledCloudFormationTemplate),
+            ContentType: 'application/json',
+            ServerSideEncryption: 'AES256',
+          },
+          awsDeploy.options.stage,
+          awsDeploy.options.region
+        )).to.be.equal(true);
+
+        awsDeploy.provider.request.restore();
+      });
+    });
   });
 
   describe('#uploadZipFile()', () => {
@@ -92,6 +123,35 @@ describe('uploadArtifacts', () => {
         awsDeploy.provider.request.restore();
       });
     });
+
+    it('should upload to a bucket with server side encryption bucket policy', () => {
+      const tmpDirPath = testUtils.getTmpDirPath();
+      const artifactFilePath = path.join(tmpDirPath, 'artifact.zip');
+      serverless.utils.writeFileSync(artifactFilePath, 'artifact.zip file content');
+      awsDeploy.serverless.service.provider.deploymentBucketObject = {
+        serverSideEncryption: 'AES256',
+      };
+      const putObjectStub = sinon
+        .stub(awsDeploy.provider, 'request').resolves();
+
+      return awsDeploy.uploadZipFile(artifactFilePath).then(() => {
+        expect(putObjectStub.calledOnce).to.be.equal(true);
+        expect(putObjectStub.calledWithExactly(
+          'S3',
+          'putObject',
+          {
+            Bucket: awsDeploy.bucketName,
+            Key: `${awsDeploy.serverless.service.package.artifactDirectoryName}/artifact.zip`,
+            Body: sinon.match.object.and(sinon.match.has('path', artifactFilePath)),
+            ContentType: 'application/zip',
+            ServerSideEncryption: 'AES256',
+          },
+          awsDeploy.options.stage,
+          awsDeploy.options.region
+        )).to.be.equal(true);
+        awsDeploy.provider.request.restore();
+      });
+    });
   });
 
   describe('#uploadFunctions()', () => {
diff --git a/lib/plugins/aws/provider/awsProvider.test.js b/lib/plugins/aws/provider/awsProvider.test.js
index 53d38858df4..f4b8e7705c4 100644
--- a/lib/plugins/aws/provider/awsProvider.test.js
+++ b/lib/plugins/aws/provider/awsProvider.test.js
@@ -60,6 +60,54 @@ describe('AwsProvider', () => {
       // clear env
       delete process.env.AWS_CLIENT_TIMEOUT;
     });
+
+    describe('when checking for the deploymentBucket config', () => {
+      it('should do nothing if the deploymentBucket config is not used', () => {
+        serverless.service.provider.deploymentBucket = undefined;
+
+        const newAwsProvider = new AwsProvider(serverless, options);
+
+        expect(newAwsProvider.serverless.service.provider.deploymentBucket).to.equal(undefined);
+      });
+
+      it('should do nothing if the deploymentBucket config is a string', () => {
+        serverless.service.provider.deploymentBucket = 'my.deployment.bucket';
+
+        const newAwsProvider = new AwsProvider(serverless, options);
+
+        expect(newAwsProvider.serverless.service.provider.deploymentBucket)
+          .to.equal('my.deployment.bucket');
+      });
+
+      it('should save the object and use the name for the deploymentBucket if provided', () => {
+        const deploymentBucketObject = {
+          name: 'my.deployment.bucket',
+          serverSideEncryption: 'AES256',
+        };
+        serverless.service.provider.deploymentBucket = deploymentBucketObject;
+
+        const newAwsProvider = new AwsProvider(serverless, options);
+
+        expect(newAwsProvider.serverless.service.provider.deploymentBucket)
+          .to.equal('my.deployment.bucket');
+        expect(newAwsProvider.serverless.service.provider.deploymentBucketObject)
+          .to.deep.equal(deploymentBucketObject);
+      });
+
+      it('should save the object and nullify the name if it is not provided', () => {
+        const deploymentBucketObject = {
+          serverSideEncryption: 'AES256',
+        };
+        serverless.service.provider.deploymentBucket = deploymentBucketObject;
+
+        const newAwsProvider = new AwsProvider(serverless, options);
+
+        expect(newAwsProvider.serverless.service.provider.deploymentBucket)
+          .to.equal(null);
+        expect(newAwsProvider.serverless.service.provider.deploymentBucketObject)
+          .to.deep.equal(deploymentBucketObject);
+      });
+    });
   });
 
   describe('#request()', () => {
@@ -418,6 +466,15 @@ describe('AwsProvider', () => {
       const credentials = newAwsProvider.getCredentials();
       expect(credentials.credentials.profile).to.equal('notDefault');
     });
+
+    it('should set the signatureVersion to v4 if the serverSideEncryption is aws:kms', () => {
+      newAwsProvider.serverless.service.provider.deploymentBucketObject = {
+        serverSideEncryption: 'aws:kms',
+      };
+
+      const credentials = newAwsProvider.getCredentials();
+      expect(credentials.signatureVersion).to.equal('v4');
+    });
   });
 
   describe('#getRegion()', () => {