Repository: serverless/serverless (JavaScript)
Date: 2020-01-28 17:18:22+00:00
Task type: Bug Fix

Issue:
SSM warning is now throwing an error
# Bug Report

## Description


1. What did you do? upgrade serverless
1. What happened? 
```
Serverless Error ---------------------------------------
  ParameterNotFound
```
1. What should've happened? 
```
 Serverless Warning
 A valid SSM parameter to satisfy the declaration .... could not be found
```
1. What's the content of your `serverless.yml` file?
my stages are:

```
int1:
  basicAuthUsername: ${ssm:/app/int/basic_auth_user}
uat:
  basicAuthUsername: ${ssm:/app/uat/basic_auth_user}
prod:
  basicAuthUsername: ${ssm:/app/prod/basic_auth_user}
```
int1 and uat live in the same aws account prod is in a different account. This did just spit out a warning saying this doesn't exist now i cannot deploy the app. 

I think just not evaluating other stages would fix this issue. ie. when i do `sls deploy --stage int` why should the prod ssm get executed? 





Issue discussion:
Also getting this issue: confirmed this change is specific to serverless `1.61.3` as `1.61.2` works.
@jeremiahlukus thanks for report.

Do you mean that you have `int1`, `uat` and `prod` configuration put top level in `serverless.yml` or are they nested at some property?

I don't think there's any convention internally that would recognize such settings as _stage_ specific, and variables are resolved for whole config unconditionally.

What you can do, is to prepare different config files per stage e.g. `serverless.dev.yml`, `serverless.prod.yml` etc. and via `--config serverless.{stage}.yml` decide which stage to deploy.
We are experiencing the same issue with pretty much the same setup. We have multiple development stages in the same account which can share some SSM variables and using custom variables was the only way we found how to do it. Production is on a different account so deploying to any of the stages now always errors out.

Separating config to separate file might be the more correct way(?) but seems highly inconvenient at least to us. We have on average 1-2 custom variables in service but 5-6 different stages and not sure how even configure it to our deployment pipeline (seed.run).

 
@tume at this point there's no concept of stage specific variables in `serverless.yml`, in all cases, there'll be attempt to resolve all from a config.

Still you can silence such errors by providing a fallback as:

```yaml
prod:
  basicAuthUsername: ${ssm:/app/prod/basic_auth_user, ''}
```
They are nested in custom these are our stages. There are only a couple of vars that are different it seems to be kind waste to make an entirely new yml file. 

Yes you can provide the fall back but you get no warning which is pretty unfortunate.
@jeremiahlukus you may open request to support concept of _stage specific configuration_.

It'll be good to start with some spec on how they should be configured, and if it'll feel right, we'd be definitely open of PR's to support such functionality.
I don't think it matters whether this is about stage-specific configuration; the main issue is that in a patch-version change, missing SSM parameters were changed from being a warning to a hard error. Either this should be reverted or version `1.61.3` should be yanked and replaced with a `2.0.0` release.
@clar-cmp indeed, thanks for further explaining that. Fix will be issued shortly

Solution patch:
diff --git a/lib/classes/Variables.js b/lib/classes/Variables.js
index e007044e98f..4b47521a021 100644
--- a/lib/classes/Variables.js
+++ b/lib/classes/Variables.js
@@ -808,36 +808,34 @@ class Variables {
         },
         { useCache: true }
       ) // Use request cache
-      .then(response => {
-        const plainText = response.Parameter.Value;
-        const type = response.Parameter.Type;
-        // Only if Secrets Manager. Parameter Store does not support JSON.
-        // We cannot parse StringList types, so don't try
-        if (type !== 'StringList' && param.startsWith('/aws/reference/secretsmanager')) {
-          try {
-            const json = JSON.parse(plainText);
-            return BbPromise.resolve(json);
-          } catch (err) {
-            // return as plain text if value is not JSON
+      .then(
+        response => {
+          const plainText = response.Parameter.Value;
+          const type = response.Parameter.Type;
+          // Only if Secrets Manager. Parameter Store does not support JSON.
+          // We cannot parse StringList types, so don't try
+          if (type !== 'StringList' && param.startsWith('/aws/reference/secretsmanager')) {
+            try {
+              return JSON.parse(plainText);
+            } catch (err) {
+              // return as plain text if value is not JSON
+            }
           }
-        }
-        if (split) {
-          if (type === 'StringList') {
-            return BbPromise.resolve(plainText.split(','));
+          if (split) {
+            if (type === 'StringList') return plainText.split(',');
+
+            logWarning(
+              `Cannot split SSM parameter '${param}' of type '${type}'. Must be 'StringList'.`
+            );
+          }
+          return plainText;
+        },
+        err => {
+          if (!err.providerError || err.providerError.statusCode !== 400) {
+            throw new this.serverless.classes.Error(err.message);
           }
-          logWarning(
-            `Cannot split SSM parameter '${param}' of type '${type}'. Must be 'StringList'.`
-          );
-        }
-        return BbPromise.resolve(plainText);
-      })
-      .catch(err => {
-        if (err.statusCode !== 400) {
-          return BbPromise.reject(new this.serverless.classes.Error(err.message));
         }
-
-        return BbPromise.resolve(undefined);
-      });
+      );
   }
 
   getValueStrToBool(variableString) {


Test patch:
diff --git a/lib/classes/Variables.test.js b/lib/classes/Variables.test.js
index 05800ca09f4..73d7bef1a9a 100644
--- a/lib/classes/Variables.test.js
+++ b/lib/classes/Variables.test.js
@@ -1229,7 +1229,9 @@ module.exports = {
       const awsProvider = new AwsProvider(serverless, options);
       const param = '/some/path/to/invalidparam';
       const property = `\${ssm:${param}}`;
-      const error = Object.assign(new Error(`Parameter ${param} not found.`), { statusCode: 400 });
+      const error = Object.assign(new Error(`Parameter ${param} not found.`), {
+        providerError: { statusCode: 400 },
+      });
       const requestStub = sinon
         .stub(awsProvider, 'request')
         .callsFake(() => BbPromise.reject(error));
@@ -2381,7 +2383,9 @@ module.exports = {
       });
     });
     it('should return undefined if SSM parameter does not exist', () => {
-      const error = Object.assign(new Error(`Parameter ${param} not found.`), { statusCode: 400 });
+      const error = Object.assign(new Error(`Parameter ${param} not found.`), {
+        providerError: { statusCode: 400 },
+      });
       const requestStub = sinon
         .stub(awsProvider, 'request')
         .callsFake(() => BbPromise.reject(error));