Repository: serverless/serverless (JavaScript)
Date: 2019-04-02 04:38:18+00:00
Task type: Bug Fix

Issue:
invoke local docker does not set environment variables configured in serverless.yml
<!--
1. If you have a question and not a bug report please ask first at http://forum.serverless.com
2. Please check if an issue already exists. This bug may have already been documented
3. Check out and follow our Guidelines: https://github.com/serverless/serverless/blob/master/CONTRIBUTING.md
4. Fill out the whole template so we have a good overview on the issue
5. Do not remove any section of the template. If something is not applicable leave it empty but leave it in the Issue
6. Please follow the template, otherwise we'll have to ask you to update it
-->

# This is a Bug Report

## Description

* What went wrong?
`sls invoke local --docker` does not pass environment variables configured in `serverless.yml` to the docker container running lambda

* What did you expect should have happened?
Environment variables configured in `serverless.yml` should be available for lambda function running in the docker container

* What was the config you used?

`serverless.yml`
```yml
service: env-var-bug

provider:
  name: aws
  runtime: python2.7
  environment:
    SOME_ENV_VAR: some-value

functions:
  hello:
    handler: handler.hello
```

`handler.py`
```py
import os

def hello(event, context):
    return os.environ['SOME_ENV_VAR']
```

* What stacktrace or error message from your provider did you see?

```
➜  env-var-bug sls invoke local -f hello
"some-value"
➜  env-var-bug sls invoke local -f hello --docker
Serverless: Building Docker image...
START RequestId: d3116be1-e3f9-402b-a460-468115ddd24e Version: $LATEST

'SOME_ENV_VAR': KeyError
Traceback (most recent call last):
  File "/var/task/handler.py", line 5, in hello
    return os.environ['SOME_ENV_VAR']
  File "/usr/lib64/python2.7/UserDict.py", line 40, in __getitem__
    raise KeyError(key)
KeyError: 'SOME_ENV_VAR'

END RequestId: d3116be1-e3f9-402b-a460-468115ddd24e
REPORT RequestId: d3116be1-e3f9-402b-a460-468115ddd24e Duration: 7 ms Billed Duration: 100 ms Memory Size: 1536 MB Max Memory Used: 14 MB


{"stackTrace": [["/var/task/handler.py", 5, "hello", "return os.environ['SOME_ENV_VAR']"], ["/usr/lib64/python2.7/UserDict.py", 40, "__getitem__", "raise KeyError(key)"]], "errorType": "KeyError", "errorMessage": "'SOME_ENV_VAR'"}


  Error --------------------------------------------------

  1

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           10.11.0
     Serverless Version:     1.39.1
```

Similar or dependent issues:
* #5863

## Additional Data

* ***Serverless Framework Version you're using***:  1.39.1
* ***Operating System***:  mac OSX High Sierra
* ***Stack Trace***: Provided Above
* ***Provider Error messages***: Provided Above



Issue discussion:
Same issue occurs when environment variables are passed as command line argument as well 
```
sls invoke local -f hello -e SOME_ENV_VAR=foo --docker
``` 

Solution patch:
diff --git a/lib/plugins/aws/invokeLocal/index.js b/lib/plugins/aws/invokeLocal/index.js
index 4f312ae379a..cc75d1dadba 100644
--- a/lib/plugins/aws/invokeLocal/index.js
+++ b/lib/plugins/aws/invokeLocal/index.js
@@ -103,6 +103,12 @@ class AwsInvokeLocal {
     });
   }
 
+  getConfiguredEnvVars() {
+    const providerEnvVars = this.serverless.service.provider.environment || {};
+    const functionEnvVars = this.options.functionObj.environment || {};
+    return _.merge(providerEnvVars, functionEnvVars);
+  }
+
   loadEnvVars() {
     const lambdaName = this.options.functionObj.name;
     const memorySize = Number(this.options.functionObj.memorySize)
@@ -130,10 +136,9 @@ class AwsInvokeLocal {
       lambdaDefaultEnvVars.AWS_PROFILE = profileOverride;
     }
 
-    const providerEnvVars = this.serverless.service.provider.environment || {};
-    const functionEnvVars = this.options.functionObj.environment || {};
+    const configuredEnvVars = this.getConfiguredEnvVars();
 
-    _.merge(process.env, lambdaDefaultEnvVars, providerEnvVars, functionEnvVars);
+    _.merge(process.env, lambdaDefaultEnvVars, configuredEnvVars);
 
     return BbPromise.resolve();
   }
@@ -315,6 +320,16 @@ class AwsInvokeLocal {
         this.serverless.config.servicePath, '.serverless', 'invokeLocal', 'artifact'));
   }
 
+  getEnvVarsFromOptions() {
+    const envVarsFromOptions = {};
+    // Get the env vars from command line options in the form of --env KEY=value
+    _.concat(this.options.env || [])
+    .forEach(itm => {
+      const splitItm = _.split(itm, '=');
+      envVarsFromOptions[splitItm[0]] = splitItm.slice(1, splitItm.length).join('=') || '';
+    });
+    return envVarsFromOptions;
+  }
 
   invokeLocalDocker() {
     const handler = this.options.functionObj.handler;
@@ -329,10 +344,16 @@ class AwsInvokeLocal {
         .then((results) => new BbPromise((resolve, reject) => {
           const imageName = results[2];
           const artifactPath = results[3];
+          const configuredEnvVars = this.getConfiguredEnvVars();
+          const envVarsFromOptions = this.getEnvVarsFromOptions();
+          const envVars = _.merge(configuredEnvVars, envVarsFromOptions);
+          const envVarsDockerArgs = _.flatMap(envVars,
+            (value, key) => ['--env', `${key}=${value}`]
+          );
           const dockerArgsFromOptions = this.getDockerArgsFromOptions();
           const dockerArgs = _.concat([
             'run', '--rm', '-v', `${artifactPath}:/var/task`,
-          ], dockerArgsFromOptions, [
+          ], envVarsDockerArgs, dockerArgsFromOptions, [
             imageName, handler, JSON.stringify(this.options.data),
           ]);
           const docker = spawn('docker', dockerArgs);


Test patch:
diff --git a/lib/plugins/aws/invokeLocal/index.test.js b/lib/plugins/aws/invokeLocal/index.test.js
index f767643bb36..72c52e6e66e 100644
--- a/lib/plugins/aws/invokeLocal/index.test.js
+++ b/lib/plugins/aws/invokeLocal/index.test.js
@@ -1153,6 +1153,9 @@ describe('AwsInvokeLocal', () => {
       serverless.setProvider('aws', new AwsProvider(serverless, options));
       awsInvokeLocalMocked = new AwsInvokeLocalMocked(serverless, options);
 
+      serverless.service.provider.environment = {
+        providerVar: 'providerValue',
+      };
       awsInvokeLocalMocked.options = {
         stage: 'dev',
         function: 'first',
@@ -1161,8 +1164,12 @@ describe('AwsInvokeLocal', () => {
           name: 'hello',
           timeout: 4,
           runtime: 'nodejs8.10',
+          environment: {
+            functionVar: 'functionValue',
+          },
         },
         data: {},
+        env: 'commandLineEnvVar=commandLineEnvVarValue',
         'docker-arg': '-p 9292:9292',
       };
 
@@ -1199,6 +1206,12 @@ describe('AwsInvokeLocal', () => {
           '--rm',
           '-v',
           'servicePath:/var/task',
+          '--env',
+          'providerVar=providerValue',
+          '--env',
+          'functionVar=functionValue',
+          '--env',
+          'commandLineEnvVar=commandLineEnvVarValue',
           '-p',
           '9292:9292',
           'sls-docker',
@@ -1209,6 +1222,48 @@ describe('AwsInvokeLocal', () => {
     );
   });
 
+  describe('#getEnvVarsFromOptions', () => {
+    it('returns empty object when env option is not set', () => {
+      delete awsInvokeLocal.options.env;
+
+      const envVarsFromOptions = awsInvokeLocal.getEnvVarsFromOptions();
+
+      expect(envVarsFromOptions).to.be.eql({});
+    });
+
+    it('returns empty object when env option empty', () => {
+      awsInvokeLocal.options.env = '';
+
+      const envVarsFromOptions = awsInvokeLocal.getEnvVarsFromOptions();
+
+      expect(envVarsFromOptions).to.be.eql({});
+    });
+
+    it('returns key value for option separated by =', () => {
+      awsInvokeLocal.options.env = 'SOME_ENV_VAR=some-value';
+
+      const envVarsFromOptions = awsInvokeLocal.getEnvVarsFromOptions();
+
+      expect(envVarsFromOptions).to.be.eql({ SOME_ENV_VAR: 'some-value' });
+    });
+
+    it('returns key with empty value for option without =', () => {
+      awsInvokeLocal.options.env = 'SOME_ENV_VAR';
+
+      const envVarsFromOptions = awsInvokeLocal.getEnvVarsFromOptions();
+
+      expect(envVarsFromOptions).to.be.eql({ SOME_ENV_VAR: '' });
+    });
+
+    it('returns key with single value for option multiple =s', () => {
+      awsInvokeLocal.options.env = 'SOME_ENV_VAR=value1=value2';
+
+      const envVarsFromOptions = awsInvokeLocal.getEnvVarsFromOptions();
+
+      expect(envVarsFromOptions).to.be.eql({ SOME_ENV_VAR: 'value1=value2' });
+    });
+  });
+
   describe('#getDockerArgsFromOptions', () => {
     it('returns empty list when docker-arg option is absent', () => {
       delete awsInvokeLocal.options['docker-arg'];