Repository: aws/aws-cli

Issue:
cloudformation deploy fails if new parameters are not overridden even if they have default value
After adding new parameters to the template, it has to be deployed with --parameters-override for all the new parameters even if they have default value, otherwise deploy fails with: 

> An error occurred (ValidationError) when calling the CreateChangeSet operation: Invalid input for parameter key Param2. Cannot specify usePreviousValue as true for a parameter key not in the previous template

To reproduce, deploy simple template:
```
AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: A hello world application.
Parameters:
  Param1:
    Default: 123
    Type: String
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs4.3
      CodeUri: ./hello.zip
```

add some parameter to it:

```
...
Parameters:
  Param1:
    Default: 123
    Type: String
  Param2:
    Default: 123
    Type: String
...
```

Try to deploy again, it fails with aforementioned error. 


Discussion:
Thanks for the report, this seems to be related to a previous bugfix: #2573
Its not a regression to previous bugfix, but a new corner case that the code doesn't handle well.

Unfortunately, this is a tough one because I need to pull down the latest template from stack, parse parameters, diff with the new template and merge current-template, new-template and overrides to come up with list of accurate UsePreviousValues. For the majority use-case, this extra call is unnecessary. 

Potential fix would be to use `get-template-summary` call instead of describe-stacks to check presence of stack AND fetch parameter information. 

Until we fix this bug, you can override the parameter explicitly when you add a new parameter with default value. Just once when you first update the stack. Afterwards, UsePreviousValue should work as expected


Resolving patch:
diff --git a/awscli/customizations/cloudformation/deployer.py b/awscli/customizations/cloudformation/deployer.py
--- a/awscli/customizations/cloudformation/deployer.py
+++ b/awscli/customizations/cloudformation/deployer.py
@@ -88,7 +88,6 @@ def create_changeset(self, stack_name, cfn_template,
         # Each changeset will get a unique name based on time
         changeset_name = self.changeset_prefix + str(int(time.time()))
 
-        changeset_type = "UPDATE"
         if not self.has_stack(stack_name):
             changeset_type = "CREATE"
             # When creating a new stack, UsePreviousValue=True is invalid.
@@ -96,6 +95,15 @@ def create_changeset(self, stack_name, cfn_template,
             # or set a Default value in template to successfully create a stack.
             parameter_values = [x for x in parameter_values
                                 if not x.get("UsePreviousValue", False)]
+        else:
+            changeset_type = "UPDATE"
+            # UsePreviousValue not valid if parameter is new
+            summary = self._client.get_template_summary(StackName=stack_name)
+            existing_parameters = [parameter['ParameterKey'] for parameter in \
+                                   summary['Parameters']]
+            parameter_values = [x for x in parameter_values
+                                if not (x.get("UsePreviousValue", False) and \
+                                x["ParameterKey"] not in existing_parameters)]
 
         kwargs = {
             'ChangeSetName': changeset_name,


Test patch:
diff --git a/tests/unit/customizations/cloudformation/test_deployer.py b/tests/unit/customizations/cloudformation/test_deployer.py
--- a/tests/unit/customizations/cloudformation/test_deployer.py
+++ b/tests/unit/customizations/cloudformation/test_deployer.py
@@ -135,10 +135,18 @@ def test_create_changeset_success(self):
 
         # Case 2: Stack exists. We are updating it
         self.deployer.has_stack.return_value = True
+        self.stub_client.add_response("get_template_summary",
+            {"Parameters": [{"ParameterKey": parameter["ParameterKey"]}
+                for parameter in parameters]},
+            {"StackName": stack_name})
         expected_params["ChangeSetType"] = "UPDATE"
         expected_params["Parameters"] = parameters
         self.stub_client.add_response("create_change_set", response,
                                       expected_params)
+        # template has new parameter but should not be included in
+        # expected_params as no previous value
+        parameters = list(parameters) + \
+            [{"ParameterKey": "New", "UsePreviousValue": True}]
         with self.stub_client:
             result = self.deployer.create_changeset(
                     stack_name, template, parameters, capabilities, role_arn,