Repository: serverless/serverless (JavaScript)
Date: 2016-11-20 00:16:13+00:00
Task type: Feature

Issue:
Design Environment Variable Support
# This is a Feature Proposal

## Description

As environment variables are an often requested feature for Serverless and AWS I want to start the design process, so we can start with implementing support for Environment Variables when they become ready at AWS at some point in the future. This is also important to make sure we capture existing use cases and our design reflects that.

There are three main use cases for Environment variables:

1. Expose Service resources created by Serverless to the lambda function (e.g. S3 bucket created as an event)
2. Expose custom data from the development team as an environment variable
3. Configure environment variables for use with our future SDK so we can do service discovery in an easy way

## Proposal for config syntax

For this first proposal I want to focus on giving users the ability to define environment variables on a provider or function level. In the future we can then add additional auto-generated Environment variables.

We will create an `environment` config parameter that allows you to set environment variables on your service or lambda function. If you set it on a provider level every function will get the environment variable. You can use the full power of the Variable system to set those parameters. Of course as those should eventually be put into the CF template you should be able to use any built-in Cloudformation functions as well (e.g. reference the ARN of a custom resource for an environment variable)

```
provider:
  environment:
    SOMEKEY: value

functions:
  hello:
    environment:
      SOMEKEY: othervalue
      otherkey: somevalue
      VARIABLEKEY: ${self:custom.variablekey}}
      S3BUCKET:
        Ref: S3Bucket
resources:
  Resources:
    S3Bucket:
      Type: "AWS::S3::Bucket"
```

Future updates will include automatically adding resources created from Events to the Environment and further automated setup features.

Similar or dependent issues:
*



Issue discussion:
Great, if you need some help let me know. Cheers

It would be great if there was an easy way to add a reference to a function for when you are invoking them. 

@flomotlik that looks great. How will this support external secrets? i.e. database password, JWT secret

Right now they're built into the code package. But ideally they would be set within the lambda environment. This would have to be implemented by AWS though.

> How will this support external secrets? i.e. database password, JWT secret

The idea would be that in your serverless.yml you reference something external through a variable (e.g. ENV or local production config file that isn't commited to the repo) during the deployment and it will set the environment variables upon deployment. AWS has to support setting environment variables through Cloudformation for that, so we can only discuss design here, implementation would have to happen as soon as they support it. I just want to make sure we start discussing this so when they come out with it at some point we have some common understanding in the community about it.

@andymac4182 Not sure what you mean exactly?

Looks great for me! But we'll have global environments too, right? Some values can be general.

The design looks exactly how it should be! I really hope they(AWS) implement it like [cfn-apigateway-stage-variables](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-stage.html#cfn-apigateway-stage-variables).

Not sure about:

``` yaml
otherkey: somevalue
```

Would this default to `process.env.OTHERKEY` within node(depends on AWS)?

http://docs.aws.amazon.com/lambda/latest/dg/env_variables.html

ENV variable support in Lambda!

In the example configuration above, is the word 'environment' standing in for an example environment name? For instance, would there be a section in here for staging and production?

What I'm used to doing is creating a dotenv file ala https://github.com/bkeepers/dotenv. This is a pretty common way of doing things and there are bindings that more or less do things this way for any language you might want to use. The property of this approach that most strongly appeals to me is that environment-specific configuration can go in an environment specific file. If I have a team and only some of the team members are working on the staging environment, I can share with those team members a .env.staging file and all they need to do is drop it in the project root (usually gitignored). I can't tell if this use case is accounted for in the proposal, but if the idea is that everything should be mixed into one big yaml file or segmented into separate files based on something other than the environment, then it would not be meeting my particular needs.

I'd also be interested to hear more about use case 3 above. I don't understand where that is going.

My main use case, and this seems pretty common, is that I have a service dependency (for example stripe.com). I want to run with a test token in my staging environment and a live token in my production environment. So I need to set STRIPE_TOKEN=test-xyz or STRIPE_TOKEN=live-xyz depending on the current environment.

Let's moveeeeee 🎉 http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-environment.html

Engage!

![687474703a2f2f33332e6d656469612e74756d626c722e636f6d2f34636163336533666463383832383765623533333632653131613733316665352f74756d626c725f6e616b74323336693366317261313175386f385f3430302e676966](https://cloud.githubusercontent.com/assets/2228236/20454397/3d36bffa-ae40-11e6-990a-af16feab923b.gif)

Let's go!

On Sat, Nov 19, 2016, 7:09 AM Piotr Gasiorowski notifications@github.com
wrote:

> Engage!
> 
> [image:
> 687474703a2f2f33332e6d656469612e74756d626c722e636f6d2f34636163336533666463383832383765623533333632653131613733316665352f74756d626c725f6e616b74323336693366317261313175386f385f3430302e676966]
> https://cloud.githubusercontent.com/assets/2228236/20454397/3d36bffa-ae40-11e6-990a-af16feab923b.gif
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> https://github.com/serverless/serverless/issues/2673#issuecomment-261702705,
> or mute the thread
> https://github.com/notifications/unsubscribe-auth/ACIX8rmDQmDCMRcrDy9-pbEs8LOF_WX5ks5q_rzdgaJpZM4KsoTc
> .
> 
> ## 
> 
> Best Regards,

_Wallison Marra_
wallison@iset.com.br
_www.iset.com.br http://www.iset.com.br_
iSET INC

## 

---

_AVISO_: A informação contida neste e-mail, bem como em qualquer de seus 
anexos, é CONFIDENCIAL e destinada ao uso exclusivo do(s) destinatário(s) 
acima referido(s), podendo conter informações sigilosas e/ou legalmente 
protegidas. Caso você não seja o destinatário desta mensagem, informamos 
que qualquer divulgação, distribuição ou cópia deste e-mail e/ou de 
qualquer de seus anexos é absolutamente proibida. Solicitamos que o 
remetente seja comunicado imediatamente, respondendo esta mensagem, e que o 
original desta mensagem e de seus anexos, bem como toda e qualquer cópia 
e/ou impressão realizada a partir destes, sejam permanentemente apagados 
e/ou destruídos. Informações adicionais sobre nossa empresa podem ser 
obtidas no site http://www.iset.com.br/.

_NOTICE_: The information contained in this e-mail and any attachments 
thereto is CONFIDENTIAL and is intended only for use by the recipient named 
herein and may contain legally privileged and/or secret information.
If you are not the e-mail´s intended recipient, you are hereby notified 
that any dissemination, distribution or copy of this e-mail, and/or any 
attachments thereto, is strictly prohibited. Please immediately notify the 
sender replying to the above mentioned e-mail address, and permanently 
delete and/or destroy the original and any copy of this e-mail and/or its 
attachments, as well as any printout thereof. Additional information about 
our company may be obtained through the site http://www.iset.com.br/.

Let's do this! 💃 💯 🎉 


Solution patch:
diff --git a/docs/providers/aws/guide/functions.md b/docs/providers/aws/guide/functions.md
index 6ead0c38da8..81166a2054b 100644
--- a/docs/providers/aws/guide/functions.md
+++ b/docs/providers/aws/guide/functions.md
@@ -220,19 +220,38 @@ Then, when you run `serverless deploy`, VPC configuration will be deployed along
 
 ## Environment Variables
 
-We're working on great environment variable support. Until then, you'll be able to use the following tools for different languages to set environment variables and make them available to your code.
+You can add Environment Variable configuration to a specific function in `serverless.yml` by adding an `environment` object property in the function configuration. This object should contain a a key/value collection of string:string:
 
-## Javascript
-
-You can use [dotenv](https://www.npmjs.com/package/dotenv) to load files with environment variables. Those variables can be set during your CI process or locally and then packaged and deployed together with your function code.
+```yml
+# serverless.yml
+service: service-name
+provider: aws
 
-## Python
+functions:
+  hello:
+    handler: handler.hello
+    environment:
+      TABLE_NAME: tableName
+```
 
-You can use [python-dotenv](https://github.com/theskumar/python-dotenv) to load files with environment variables. Those variables can be set during your CI process or locally and then packaged and deployed together with your function code.
+Or if you want to apply Environment Variable configuration to all functions in your service, you can add the configuration to the higher level `provider` object, and overwrite these service level config at the function level. For example:
 
-## Java
+```yml
+# serverless.yml
+service: service-name
+provider:
+  name: aws
+  environment:
+    TABLE_NAME: tableName1
 
-For Java the easiest way to set up environment like configuration is through [property files](https://docs.oracle.com/javase/tutorial/essential/environment/properties.html). While those will not be available as environment variables they are very commonly used configuration mechanisms throughout Java.
+functions:
+  hello: # this function will overwrite the service level environment config above
+    handler: handler.hello
+    environment:
+      TABLE_NAME: tableName2
+  users: # this function will inherit the service level environment config above
+    handler: handler.users
+```
 
 ## Log Group Resources
 
diff --git a/lib/plugins/aws/deploy/compile/functions/index.js b/lib/plugins/aws/deploy/compile/functions/index.js
index 7a2c28dd7c4..44c61a827f6 100644
--- a/lib/plugins/aws/deploy/compile/functions/index.js
+++ b/lib/plugins/aws/deploy/compile/functions/index.js
@@ -81,6 +81,23 @@ class AwsCompileFunctions {
       newFunction.Properties.Description = functionObject.description;
     }
 
+    if (functionObject.environment || this.serverless.service.provider.environment) {
+      newFunction.Properties.Environment = {};
+      newFunction.Properties.Environment.Variables = Object.assign(
+        {},
+        this.serverless.service.provider.environment,
+        functionObject.environment
+      );
+
+      Object.keys(newFunction.Properties.Environment.Variables).forEach((key) => {
+        // taken from the bash man pages
+        if (!key.match(/^[A-Za-z_][a-zA-Z0-9_]*$/)) {
+          const errorMessage = 'Invalid characters in environment variable';
+          throw new this.serverless.classes.Error(errorMessage);
+        }
+      });
+    }
+
     if ('role' in functionObject) {
       newFunction.Properties.Role = this.compileRole(functionObject.role);
     } else if ('role' in this.serverless.service.provider) {


Test patch:
diff --git a/lib/plugins/aws/deploy/compile/functions/tests/index.js b/lib/plugins/aws/deploy/compile/functions/tests/index.js
index 7bb65192602..10792eca604 100644
--- a/lib/plugins/aws/deploy/compile/functions/tests/index.js
+++ b/lib/plugins/aws/deploy/compile/functions/tests/index.js
@@ -360,6 +360,170 @@ describe('AwsCompileFunctions', () => {
       };
     });
 
+    it('should create a function resource with environment config', () => {
+      awsCompileFunctions.serverless.service.functions = {
+        func: {
+          handler: 'func.function.handler',
+          name: 'new-service-dev-func',
+          environment: {
+            test1: 'test1',
+            test2: 'test2',
+          },
+        },
+      };
+
+      awsCompileFunctions.serverless.service.provider.environment = {
+        providerTest1: 'providerTest1',
+      };
+
+      const compliedFunction = {
+        Type: 'AWS::Lambda::Function',
+        Properties: {
+          Code: {
+            S3Key: `${awsCompileFunctions.serverless.service.package.artifactDirectoryName}/${
+              awsCompileFunctions.serverless.service.package.artifact}`,
+            S3Bucket: { Ref: 'ServerlessDeploymentBucket' },
+          },
+          FunctionName: 'new-service-dev-func',
+          Handler: 'func.function.handler',
+          MemorySize: 1024,
+          Role: { 'Fn::GetAtt': ['IamRoleLambdaExecution', 'Arn'] },
+          Runtime: 'nodejs4.3',
+          Timeout: 6,
+          Environment: {
+            Variables: {
+              test1: 'test1',
+              test2: 'test2',
+              providerTest1: 'providerTest1',
+            },
+          },
+        },
+      };
+
+      awsCompileFunctions.compileFunctions();
+
+      expect(
+        awsCompileFunctions.serverless.service.provider.compiledCloudFormationTemplate
+          .Resources.FuncLambdaFunction
+      ).to.deep.equal(compliedFunction);
+
+      awsCompileFunctions.serverless.service.functions = {
+        func: {
+          handler: 'func.function.handler',
+        },
+      };
+    });
+
+    it('should create a function resource with function level environment config', () => {
+      awsCompileFunctions.serverless.service.functions = {
+        func: {
+          handler: 'func.function.handler',
+          name: 'new-service-dev-func',
+          environment: {
+            test1: 'test1',
+          },
+        },
+      };
+
+      const compliedFunction = {
+        Type: 'AWS::Lambda::Function',
+        Properties: {
+          Code: {
+            S3Key: `${awsCompileFunctions.serverless.service.package.artifactDirectoryName}/${
+              awsCompileFunctions.serverless.service.package.artifact}`,
+            S3Bucket: { Ref: 'ServerlessDeploymentBucket' },
+          },
+          FunctionName: 'new-service-dev-func',
+          Handler: 'func.function.handler',
+          MemorySize: 1024,
+          Role: { 'Fn::GetAtt': ['IamRoleLambdaExecution', 'Arn'] },
+          Runtime: 'nodejs4.3',
+          Timeout: 6,
+          Environment: {
+            Variables: {
+              test1: 'test1',
+            },
+          },
+        },
+      };
+
+      awsCompileFunctions.compileFunctions();
+
+      expect(
+        awsCompileFunctions.serverless.service.provider.compiledCloudFormationTemplate
+          .Resources.FuncLambdaFunction
+      ).to.deep.equal(compliedFunction);
+
+      awsCompileFunctions.serverless.service.functions = {
+        func: {
+          handler: 'func.function.handler',
+        },
+      };
+    });
+
+    it('should create a function resource with provider level environment config', () => {
+      awsCompileFunctions.serverless.service.functions = {
+        func: {
+          handler: 'func.function.handler',
+          name: 'new-service-dev-func',
+        },
+      };
+
+      awsCompileFunctions.serverless.service.provider.environment = {
+        providerTest1: 'providerTest1',
+      };
+
+      const compliedFunction = {
+        Type: 'AWS::Lambda::Function',
+        Properties: {
+          Code: {
+            S3Key: `${awsCompileFunctions.serverless.service.package.artifactDirectoryName}/${
+              awsCompileFunctions.serverless.service.package.artifact}`,
+            S3Bucket: { Ref: 'ServerlessDeploymentBucket' },
+          },
+          FunctionName: 'new-service-dev-func',
+          Handler: 'func.function.handler',
+          MemorySize: 1024,
+          Role: { 'Fn::GetAtt': ['IamRoleLambdaExecution', 'Arn'] },
+          Runtime: 'nodejs4.3',
+          Timeout: 6,
+          Environment: {
+            Variables: {
+              providerTest1: 'providerTest1',
+            },
+          },
+        },
+      };
+
+      awsCompileFunctions.compileFunctions();
+
+      expect(
+        awsCompileFunctions.serverless.service.provider.compiledCloudFormationTemplate
+          .Resources.FuncLambdaFunction
+      ).to.deep.equal(compliedFunction);
+
+      awsCompileFunctions.serverless.service.functions = {
+        func: {
+          handler: 'func.function.handler',
+        },
+      };
+    });
+
+    it('should throw an error if environment variable has invalid name', () => {
+      awsCompileFunctions.serverless.service.functions = {
+        func: {
+          handler: 'func.function.handler',
+          name: 'new-service-dev-func',
+          environment: {
+            '1test1': 'test1',
+            test2: 'test2',
+          },
+        },
+      };
+
+      expect(() => awsCompileFunctions.compileFunctions()).to.throw(Error);
+    });
+
     it('should consider function based config when creating a function resource', () => {
       awsCompileFunctions.serverless.service.functions = {
         func: {