Repository: serverless/serverless (JavaScript)
Date: 2019-10-21 23:23:25+00:00
Task type: Bug Fix

Issue:
The always restApi logs enable when using an SSM parameter in the configuration.
# Bug Report

## Description

<!-- Please use https://forum.serverless.com, StackOverflow or other forums for Q&A -->
<!-- Please answer ALL the question below. Otherwise we probably have to close the issue due to missing information -->

1. What did you do?

I enable logs depending on the stage using SSM parameters.

1. What happened?

The API Gateway logs enable for every stage. The SSM values for `dev` and `test` stages were `false`, and `staging` and `prod` were `true`.

1. What should've happened?

API Gateway logs should have been disabled for `dev` and `test`, and enabled for `staging` and `prod`.

1. What's the content of your `serverless.yml` file?

```yaml
provider:
  logs:
    restApi:
      accessLogging:     ${ssm:/${self:provider.env}/API_GW_LOGS}
      executionLogging:  ${ssm:/${self:provider.env}/API_GW_LOGS}
      fullExecutionData: ${ssm:/${self:provider.env}/API_GW_LOGS}
```

1. What's the output you get when you use the `SLS_DEBUG=*` environment variable (e.g. `SLS_DEBUG=* serverless deploy`)

Not captured.

Similar or dependent issues:

- #12345



Issue discussion:
There are lots of non-string configuration (boolean or numeric) in severless framework, and unfortunately most of them does not accept string value from `$opt` or `$ssm`...

It is recommended to [use `$file` variable and use booleans/numbers in those files](https://github.com/serverless/serverless/pull/6561#issuecomment-523006447).
Thanks @miguel-a-calles-mba for report.

For resources like `process.env` where value can only be a string (or not be set at at all). I believe the best convention to express booleans is to use var names as `ENABLE_*` or `DISABLE_*`, and resolve _boolean_ result basing on var existence (an empty string also may indicate _falsy_ value).

In JavaScript `Boolean("false") === true`, and I believe in it's best to stick to that characteristics when parsing YAML. Otherwise we may end with some nasty side effects.

@medikoo and @exoego,
Would this type of change help with the overall problem?
https://github.com/serverless/serverless/pull/6787
It passed unit tests.
I'm testing the `sls deploy` in my development environment.
>Would this type of change help with the overall problem?

I'm not sure if it's not a good idea to translate `"false"` to `false`. To avoid confusion and unwanted side effects we should rather follow how coercion in JS language works.

Anyway, let's focus exactly on your problem:
- Is _string_ the only possible value type of SSM parameter? (is there no way to store booleans or numbers?) 
- If it's the case, can you make a `API_GW_LOGS` an empty string (`''`) when you're after `false` and e.g. (`'1'`) when you're after truthy value?
> * Is _string_ the only possible value type of SSM parameter? (is there no way to store booleans or numbers?)

Correct. It only returns strings.

> * If it's the case, can you make a `API_GW_LOGS` an empty string (`''`) when you're after `false` and e.g. (`'1'`) when you're after truthy value?

Thanks. I'll give it a try, but I don't think SSM allows empty strings.


> Thanks. I'll give it a try, but I don't think SSM allows empty strings.

Ok, let us know
SSM will not accept an empty string. I tried setting the SSM value as a single space string and a zero string to no avail.
I'm not fond of this workaround, but it works. Is there a cleaner way to do this?

```yaml
provider:
  logs:
    restApi:
      accessLogging:     ${self:custom.restApiLogs.${ssm:/${self:provider.env}/API_GW_LOGS}}
      executionLogging:  ${self:custom.restApiLogs.${ssm:/${self:provider.env}/API_GW_LOGS}}
      fullExecutionData: ${self:custom.restApiLogs.${ssm:/${self:provider.env}/API_GW_LOGS}}

custom:
  restApiLogs:
    true: true
    false: false
```
This is a little cleaner, but alternate suggestions are welcomed. Thanks.

```yaml
provider:
  logs:
    restApi:
      accessLogging:     ${self:custom.${ssm:/${self:provider.env}/API_GW_LOGS}}
      executionLogging:  ${self:custom.${ssm:/${self:provider.env}/API_GW_LOGS}}
      fullExecutionData: ${self:custom.${ssm:/${self:provider.env}/API_GW_LOGS}}

custom:
  true: true
  false: false
```
Indeed I double checked, and the only available types are _string_ and _string list_.

Probably cleanest way to approach it then, is to depend on existence of a parametr, and set resolver as:

```yaml
accessLogging:  ${ssm:/${self:provider.env}/API_GW_LOGS, false}
```

If `API_GW_LOGS` exists, it'll resolve to truthy value, if it doesn't it'll be `false`.

Some other idea worth exploring could be to support some extra decorators in variable resolvers, through that we could mark the intention of converting string `'true'` and `'false'` to boolean as proposed in your PR. That may look as:

```yaml
accessLogging:  ${ssm:/${self:provider.env}/API_GW_LOGS|strToBool}
# or
accessLogging:  ${@strToBool(ssm:/${self:provider.env}/API_GW_LOGS)}
```

Still such feature will require an implementation which may not be that trivial (variables handling is already pretty complicated)


Solution patch:
diff --git a/docs/providers/aws/guide/variables.md b/docs/providers/aws/guide/variables.md
index 6076cbeda8d..08078cf12ba 100644
--- a/docs/providers/aws/guide/variables.md
+++ b/docs/providers/aws/guide/variables.md
@@ -46,6 +46,9 @@ You can define your own variable syntax (regex) if it conflicts with CloudFormat
 - [CloudFormation stack outputs](#reference-cloudformation-outputs)
 - [Properties exported from Javascript files (sync or async)](#reference-variables-in-javascript-files)
 - [Pseudo Parameters Reference](#pseudo-parameters-reference)
+- [Read String Variable Values as Boolean Values](#read-string-variable-values-as-boolean-values)
+
+## Casting string variables to boolean values
 
 ## Recursively reference properties
 
@@ -657,3 +660,27 @@ Resources:
         - Ref: 'AWS::AccountId'
         - 'log-group:/aws/lambda/*:*:*'
 ```
+
+## Read String Variable Values as Boolean Values
+
+In some cases, a parameter expect a `true` or `false` boolean value. If you are using a variable to define the value, it may return as a string (e.g. when using SSM variables) and thus return a `"true"` or `"false"` string value.
+
+To ensure a boolean value is returned, read the string variable value as a boolean value. For example:
+
+```yml
+provider:
+  tracing:
+    apiGateway: ${strToBool(${ssm:API_GW_DEBUG_ENABLED})}
+```
+
+These are examples that explain how the conversion works:
+
+```plaintext
+${strToBool(true)} => true
+${strToBool(false)} => false
+${strToBool(0)} => false
+${strToBool(1)} => true
+${strToBool(2)} => Error
+${strToBool(null)} => Error
+${strToBool(anything)} => Error
+```
diff --git a/lib/classes/Variables.js b/lib/classes/Variables.js
index 6f248b408fa..469aa79bdf9 100644
--- a/lib/classes/Variables.js
+++ b/lib/classes/Variables.js
@@ -55,6 +55,7 @@ class Variables {
     this.cfRefSyntax = RegExp(/^(?:\${)?cf(\.[a-zA-Z0-9-]+)?:/g);
     this.s3RefSyntax = RegExp(/^(?:\${)?s3:(.+?)\/(.+)$/);
     this.ssmRefSyntax = RegExp(/^(?:\${)?ssm:([a-zA-Z0-9_.\-/]+)[~]?(true|false|split)?/);
+    this.strToBoolRefSyntax = RegExp(/^(?:\${)?strToBool\(([a-zA-Z0-9_.\-/]+)\)/);
 
     this.variableResolvers = [
       { regex: this.slsRefSyntax, resolver: this.getValueFromSls.bind(this) },
@@ -81,6 +82,10 @@ class Variables {
         isDisabledAtPrepopulation: true,
         serviceName: 'SSM',
       },
+      {
+        regex: this.strToBoolRefSyntax,
+        resolver: this.getValueStrToBool.bind(this),
+      },
       { regex: this.deepRefSyntax, resolver: this.getValueFromDeep.bind(this) },
     ];
   }
@@ -822,6 +827,27 @@ class Variables {
       });
   }
 
+  getValueStrToBool(variableString) {
+    return BbPromise.try(() => {
+      if (_.isString(variableString)) {
+        const groups = variableString.match(this.strToBoolRefSyntax);
+        const param = groups[1].trim();
+        if (/^(true|false|0|1)$/.test(param)) {
+          if (param === 'false' || param === '0') {
+            return false;
+          }
+          // truthy or non-empty strings
+          return true;
+        }
+        throw new this.serverless.classes.Error(
+          'Unexpected strToBool input; expected either "true", "false", "0", or "1".'
+        );
+      }
+      // Cast non-string inputs
+      return Boolean(variableString);
+    });
+  }
+
   getDeepIndex(variableString) {
     const deepIndexReplace = RegExp(/^deep:|(\.[^}]+)*$/g);
     return variableString.replace(deepIndexReplace, '');


Test patch:
diff --git a/lib/classes/Variables.test.js b/lib/classes/Variables.test.js
index 5162f0ee4b5..6bf869b3ac6 100644
--- a/lib/classes/Variables.test.js
+++ b/lib/classes/Variables.test.js
@@ -2378,6 +2378,70 @@ module.exports = {
     });
   });
 
+  describe('#getValueStrToBool()', () => {
+    const errMessage = 'Unexpected strToBool input; expected either "true", "false", "0", or "1".';
+    beforeEach(() => {
+      serverless.variables.service = {
+        service: 'testService',
+        provider: serverless.service.provider,
+      };
+      serverless.variables.loadVariableSyntax();
+    });
+    it('regex for "true" input', () => {
+      expect(serverless.variables.strToBoolRefSyntax.test('${strToBool(true)}')).to.equal(true);
+    });
+    it('regex for "false" input', () => {
+      expect(serverless.variables.strToBoolRefSyntax.test('${strToBool(false)}')).to.equal(true);
+    });
+    it('regex for "0" input', () => {
+      expect(serverless.variables.strToBoolRefSyntax.test('${strToBool(0)}')).to.equal(true);
+    });
+    it('regex for "1" input', () => {
+      expect(serverless.variables.strToBoolRefSyntax.test('${strToBool(1)}')).to.equal(true);
+    });
+    it('regex for "null" input', () => {
+      expect(serverless.variables.strToBoolRefSyntax.test('${strToBool(null)}')).to.equal(true);
+    });
+    it('regex for truthy input', () => {
+      expect(serverless.variables.strToBoolRefSyntax.test('${strToBool(anything)}')).to.equal(true);
+    });
+    it('regex for empty input', () => {
+      expect(serverless.variables.strToBoolRefSyntax.test('${strToBool()}')).to.equal(false);
+    });
+    it('true (string) should return true (boolean)', () => {
+      return serverless.variables.getValueStrToBool('strToBool(true)').should.become(true);
+    });
+    it('false (string) should return false (boolean)', () => {
+      return serverless.variables.getValueStrToBool('strToBool(false)').should.become(false);
+    });
+    it('1 (string) should return true (boolean)', () => {
+      return serverless.variables.getValueStrToBool('strToBool(1)').should.become(true);
+    });
+    it('0 (string) should return false (boolean)', () => {
+      return serverless.variables.getValueStrToBool('strToBool(0)').should.become(false);
+    });
+    it('truthy string should throw an error', () => {
+      return serverless.variables
+        .getValueStrToBool('strToBool(anything)')
+        .catch(err => err.message)
+        .should.become(errMessage);
+    });
+    it('null (string) should throw an error', () => {
+      return serverless.variables
+        .getValueStrToBool('strToBool(null)')
+        .catch(err => err.message)
+        .should.become(errMessage);
+    });
+    it('strToBool(true) as an input to strToBool', () => {
+      const input = serverless.variables.getValueStrToBool('strToBool(true)');
+      return serverless.variables.getValueStrToBool(input).should.become(true);
+    });
+    it('strToBool(false) as an input to strToBool', () => {
+      const input = serverless.variables.getValueStrToBool('strToBool(false)');
+      return serverless.variables.getValueStrToBool(input).should.become(true);
+    });
+  });
+
   describe('#getDeeperValue()', () => {
     it('should get deep values', () => {
       const valueToPopulateMock = {