Repository: serverless/serverless (JavaScript)
Date: 2018-07-26 20:02:03+00:00
Task type: Bug Fix

Issue:
Breaking change in 1.28.0: Environment variable ... must contain string
# This is a Bug Report

## Description

After updating today to 1.28.0, Serverless (or a dependency) now expects all environment variables to be strings. This sounds reasonable, but it's a breaking change so I'm making people aware.

In YAML, if a string is unquoted then it gets interpreted as a number if it only contains digits. E.g.

```
AWS_ACCOUNT_ID: 1234567890
```
is different to 
```
AWS_ACCOUNT_ID: '1234567890'
```

Our CI pipeline uses YamlDotNet to generate the Yaml which contains the environment variables. This has an  [open issue](https://github.com/aaubry/YamlDotNet/issues/237) where it cannot serialize numeric strings properly.

Running `serverless deploy` now gives 

```
Serverless: Packaging service...

  Serverless Error ---------------------------------------

  Environment variable AWS_ACCOUNT_ID must contain string

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information -----------------------------
     OS:                     win32
     Node Version:           9.11.1
     Serverless Version:     1.28.0
```

Working around for us is not trivial. If it's easy for Serverless to resolve this breaking change, it may help others. I've solved this for now by fixing Serverless version to 1.27.1


Similar or dependent issues:

## Additional Data

* 1.28.0:
* win64:




Issue discussion:
Seconded. 
this obviously applies to empty vars too. we use SSM to populate envars and sometimes want that var to be empty +1
Thirded.
Same here... fourthed?
+1 , locked my sls install to previous version for now 
+1, Thanks!
+1 had to downgrade to 1.27.3, serious breaking bug.
+1 (also thumbed up 👍 the initial comment).  We had multiple pipelines begin failing late last week, and traced it back to this breaking update.  May make us rethink our pinning serverless versus vs. leading latest, but hoping 1.28.1 is coming soon. Also wonder regarding any future alignment to semver versioning for the package? We'd love to understand when breaking changes are coming (2.x) to make a decision, and let us pin to 1.x for all backward-compat feature enhancements, security patches, etc.
This bug was unintentionally introduced by #4890.
My possible solution to the problem is here

- Remove [only contain string check](https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/compile/functions/index.js#L265-L267) when `sls deploy`. This cause the breaking.
- Add single-quotation to env value itself just before deploying to avoid the Lambda API error if it contain only number when `sls deploy function`

We should cut a patch release as soon as possible.

cc: @serverless/framework-maintainers 
@horike37 Agree. We should put that into 1.28.1. I will create a patch branch and cherry pick as soon as a solution is ready.

> Add single-quotation to env value itself just before deploying to avoid the Lambda API error if it contain only number when sls deploy function

Is this enough, or do we have to cover cases where an env var is set to an object too?
@horike37 @HyperBrain 
I'm trying to add environment variable to lambda function, no matter what I do, I keep running into variable must be string error message.  What should the syntax look like to force env param into string?

so far I've tried below syntax, but they all failed

    environment: 
        SAMPLE_PARAM: "${opt:dxc-aws-account,self:provider.custom.dxc-aws-account}"

    environment: 
        SAMPLE_PARAM: '${opt:dxc-aws-account,self:provider.custom.dxc-aws-account}'

    environment: 
        SAMPLE_PARAM: {Fn::Join : ['', [${opt:dxc-aws-account,self:provider.custom.dxc-aws-account}]]}
this is the combination that seem to work for me...

    environment: 
        SAMPLE_PARAM: {Fn::Join : ['', ["${opt:dxc-aws-account,self:provider.custom.dxc-aws-account}"]]}
This also happens during `sls package` not just `sls deploy`.
> Add single-quotation to env value itself just before deploying to avoid the Lambda API error if it contain only number when sls deploy function

Don't forget `null`, `true`, `false`, and any other non-number & non-string field types.
@HyperBrain @dschep 
>Is this enough, or do we have to cover cases where an env var is set to an object too?
>Don't forget null, true, false, and any other non-number & non-string field types.

Thank you for the advice. Yes, we should cover those cases with checking the behavior of previous versions.

@mzvrzg

Thank you for testing and sharing the result 👍 

I've had to downgrade to serverless 1.27.3 due to this too.

We use environment variables as optional overrides so quite often had them blank.
I've been stung by this issue too, I worked around the issue by adding an empty string to the initialisation list:

eg

```
environment:
  MESSAGE: ${env.MESSAGE, ""}
```

We are using Node.js so an empty string should be falsy enough to allow default configuration to take over.
I'm having this issue too. I had to downgrade:
```
npm i -g serverless@1.27.3
```
Even when using a string, this can cause issues with libraries that are expecting a number or some other data type.
@jamesdixon that doesn't sound right to me.  These values should be read from `process.env` which always returns a string from what I can tell.  Thus I would expect those libraries to coerce the string values into the required type.   

However, I still think having a breaking change in 1.28 is not ideal!
@mr-beerkiss you could be right. however, if it's left up to the library, we can't assume. either way, i agree...let's go back to the way it was!
@mr-beerkiss the breaking change is that configuration has to be rewritten (even though all env vars are passed as strings, we need to add `"`) I believe it should be a warning not an error. I'm with @jamesdixon, we should change it.
@timgivois I 100% agree the breaking change is in Serverless, and I would even go so far as to say it shouldn't even be a warning and should quietly coerce all primitive types to a string.    
In 1.27.3 it was possible to refer to environment variables in the serverless.yml file which were not set to any value. 

This allowed for environment variables to be used to override default settings in our CI and CD pipelines. Specifically, this was done in the environment section of the lambda configuration:
```yml
    lambda:
       ...
        environment:
            OVERRIDE_ENV_VAR: ${env:OVERRIDE_ENV_VAR}
```

If the override_env_var was not set, then this was not set on the lambda. This is really useful in complex environments. Losing this feature is going to leave us pinned to 1.27.3

Is it necessary to enforce a value?

I can see that there are other use cases where it might be useful to enforce a value, but this does seem to me to be an example where not doing so is useful. It's not an uncommon pattern to use environment varibales in this way and to have code which checks for the existance or none existance of an env var.  

Solution patch:
diff --git a/lib/plugins/aws/deployFunction/index.js b/lib/plugins/aws/deployFunction/index.js
index 2106b2b9e6a..daef82b2e0e 100644
--- a/lib/plugins/aws/deployFunction/index.js
+++ b/lib/plugins/aws/deployFunction/index.js
@@ -164,9 +164,9 @@ class AwsDeployFunction {
             const errorMessage = 'Invalid characters in environment variable';
             throw new this.serverless.classes.Error(errorMessage);
           }
+
           if (!_.isString(params.Environment.Variables[key])) {
-            const errorMessage = `Environment variable ${key} must contain strings`;
-            throw new this.serverless.classes.Error(errorMessage);
+            params.Environment.Variables[key] = _.toString(params.Environment.Variables[key]);
           }
         });
       }
diff --git a/lib/plugins/aws/package/compile/functions/index.js b/lib/plugins/aws/package/compile/functions/index.js
index 90f68b80b95..614419f1055 100644
--- a/lib/plugins/aws/package/compile/functions/index.js
+++ b/lib/plugins/aws/package/compile/functions/index.js
@@ -260,11 +260,13 @@ class AwsCompileFunctions {
             return false;   // break loop with lodash
           }
           const value = newFunction.Properties.Environment.Variables[key];
-          const isCFRef = _.isObject(value) &&
-            !_.some(value, (v, k) => k !== 'Ref' && !_.startsWith(k, 'Fn::'));
-          if (!isCFRef && !_.isString(value)) {
-            invalidEnvVar = `Environment variable ${key} must contain string`;
-            return false;
+          if (_.isObject(value)) {
+            const isCFRef = _.isObject(value) &&
+              !_.some(value, (v, k) => k !== 'Ref' && !_.startsWith(k, 'Fn::'));
+            if (!isCFRef) {
+              invalidEnvVar = `Environment variable ${key} must contain string`;
+              return false;
+            }
           }
         }
       );


Test patch:
diff --git a/lib/plugins/aws/deployFunction/index.test.js b/lib/plugins/aws/deployFunction/index.test.js
index 526607c92d1..6b61d129b18 100644
--- a/lib/plugins/aws/deployFunction/index.test.js
+++ b/lib/plugins/aws/deployFunction/index.test.js
@@ -353,19 +353,35 @@ describe('AwsDeployFunction', () => {
       expect(() => awsDeployFunction.updateFunctionConfiguration()).to.throw(Error);
     });
 
-    it('should fail when using non-string values as environment variables', () => {
-      options.functionObj = {
-        name: 'first',
-        description: 'change',
-        environment: {
-          COUNTER: 6,
-        },
-      };
-
-      awsDeployFunction.options = options;
-
-      expect(() => awsDeployFunction.updateFunctionConfiguration()).to.throw(Error);
-    });
+    it('should transform to string values when using non-string values as environment variables',
+      () => {
+        options.functionObj = {
+          name: 'first',
+          description: 'change',
+          environment: {
+            COUNTER: 6,
+          },
+        };
+
+        awsDeployFunction.options = options;
+        return expect(awsDeployFunction.updateFunctionConfiguration()).to.be.fulfilled
+          .then(() => {
+            expect(updateFunctionConfigurationStub.calledOnce).to.be.equal(true);
+            expect(updateFunctionConfigurationStub.calledWithExactly(
+              'Lambda',
+              'updateFunctionConfiguration',
+              {
+                FunctionName: 'first',
+                Description: 'change',
+                Environment: {
+                  Variables: {
+                    COUNTER: '6',
+                  },
+                },
+              }
+            )).to.be.equal(true);
+          });
+      });
 
     it('should inherit provider-level config', () => {
       options.functionObj = {
diff --git a/lib/plugins/aws/package/compile/functions/index.test.js b/lib/plugins/aws/package/compile/functions/index.test.js
index c743626b74a..296c1cdb8f6 100644
--- a/lib/plugins/aws/package/compile/functions/index.test.js
+++ b/lib/plugins/aws/package/compile/functions/index.test.js
@@ -1456,7 +1456,7 @@ describe('AwsCompileFunctions', () => {
       return expect(awsCompileFunctions.compileFunctions()).to.be.rejectedWith(Error);
     });
 
-    it('should throw an error if environment variable is not a string', () => {
+    it('should accept an environment variable with a not-string value', () => {
       awsCompileFunctions.serverless.service.functions = {
         func: {
           handler: 'func.function.handler',
@@ -1467,7 +1467,13 @@ describe('AwsCompileFunctions', () => {
         },
       };
 
-      return expect(awsCompileFunctions.compileFunctions()).to.be.rejectedWith(Error);
+      return expect(awsCompileFunctions.compileFunctions()).to.be.fulfilled
+      .then(() => {
+        expect(awsCompileFunctions.serverless.service.provider
+          .compiledCloudFormationTemplate
+          .Resources.FuncLambdaFunction.Properties.Environment.Variables.counter
+        ).to.equal(18);
+      });
     });
 
     it('should accept an environment variable with CF ref and functions', () => {