Repository: cloud-custodian/cloud-custodian

Issue:
aws:asg propagte-tags action errors on asgs without instances
The error message when I try to use the propagate-tags action is the following:

```
[ERROR] 2019-11-19T02:56:38.336Z b1c8af86-d773-412d-b755-638faf7698e8 Error while executing policy
20:56:38 Traceback (most recent call last):
20:56:38 File "/var/task/c7n/policy.py", line 320, in run
20:56:38 results = a.process(resources)
20:56:38 File "/var/task/c7n/resources/asg.py", line 1200, in process
20:56:38 instance_count = sum(list(w.map(self.process_asg, asgs)))
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
20:56:38 yield fs.pop().result()
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 435, in result
20:56:38 return self.__get_result()
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
20:56:38 raise self._exception
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/thread.py", line 57, in run
20:56:38 result = self.fn(*self.args, **self.kwargs)
20:56:38 File "/var/task/c7n/resources/asg.py", line 1221, in process_asg
20:56:38 Tags=[{'Key': k, 'Value': v} for k, v in tag_map.items()])
20:56:38 File "/var/runtime/botocore/client.py", line 357, in _api_call
20:56:38 return self._make_api_call(operation_name, kwargs)
20:56:38 File "/var/runtime/botocore/client.py", line 661, in _make_api_call
20:56:38 raise error_class(parsed_response, operation_name)
20:56:38 botocore.exceptions.ClientError: An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
20:56:38 [ERROR] 2019-11-19T02:56:38.338Z b1c8af86-d773-412d-b755-638faf7698e8 error during policy execution
20:56:38 Traceback (most recent call last):
20:56:38 File "/var/task/c7n/handler.py", line 168, in dispatch_event
20:56:38 p.push(event, context)
20:56:38 File "/var/task/c7n/policy.py", line 991, in push
20:56:38 return mode.run(event, lambda_ctx)
20:56:38 File "/var/task/c7n/policy.py", line 591, in run
20:56:38 return PullMode.run(self)
20:56:38 File "/var/task/c7n/policy.py", line 320, in run
20:56:38 results = a.process(resources)
20:56:38 File "/var/task/c7n/resources/asg.py", line 1200, in process
20:56:38 instance_count = sum(list(w.map(self.process_asg, asgs)))
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
20:56:38 yield fs.pop().result()
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 435, in result
20:56:38 return self.__get_result()
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
20:56:38 raise self._exception
20:56:38 File "/var/lang/lib/python3.7/concurrent/futures/thread.py", line 57, in run
20:56:38 result = self.fn(*self.args, **self.kwargs)
20:56:38 File "/var/task/c7n/resources/asg.py", line 1221, in process_asg
20:56:38 Tags=[{'Key': k, 'Value': v} for k, v in tag_map.items()])
20:56:38 File "/var/runtime/botocore/client.py", line 357, in _api_call
20:56:38 return self._make_api_call(operation_name, kwargs)
20:56:38 File "/var/runtime/botocore/client.py", line 661, in _make_api_call
20:56:38 raise error_class(parsed_response, operation_name)
20:56:38 botocore.exceptions.ClientError: An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
20:56:38 [ERROR] ClientError: An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet Traceback (most recent call last):   File "/var/task/custodian_policy.py", line 4, in run     return handler.dispatch_event(event, context)   File "/var/task/c7n/handler.py", line 168, in dispatch_event     p.push(event, context)   File "/var/task/c7
20:56:38 END RequestId: b1c8af86-d773-412d-b755-638faf7698e8
```


Discussion:
I think this occurs when the CreateTag api is called with either an empty list of resources or one of the tags that you're trying to propagate doesn't exist. 
This shouldn't cause the policy to completely fail.  This needs better error handling if it doesn't find a tag.  Completely failing the policy is causing us issues and is not ideal.  A simple error and then continue would be best.
In my script it verifies that all the tags that it is going to propagate are present and have values so this doesn't make sense why this would bomb out like this.
Ah, ok! Thanks for debugging this! Were you also able to verify that all the ASGs that got filtered do have instances? I agree we should put in some sort of exception handling for those edge cases.
Ahhhh that might be the issue.  We are just creating these ASG to test our new version of tag enforcement and to save on cost just created an ASG with no instances so maybe this needs some error handling around that to just state there are no instances to propagate tags to and then continue with the next action
yeah I was peaking at the code, we should abort if the asg doesn't have instances.

Resolving patch:
diff --git a/c7n/resources/asg.py b/c7n/resources/asg.py
--- a/c7n/resources/asg.py
+++ b/c7n/resources/asg.py
@@ -1215,7 +1215,7 @@ def process_asg(self, asg):
         if self.data.get('trim', False):
             instances = [self.instance_map[i] for i in instance_ids]
             self.prune_instance_tags(client, asg, tag_set, instances)
-        if not self.manager.config.dryrun:
+        if not self.manager.config.dryrun and instances:
             client.create_tags(
                 Resources=instance_ids,
                 Tags=[{'Key': k, 'Value': v} for k, v in tag_map.items()])


Test patch:
diff --git a/tests/data/placebo/test_asg_propagate_tag_no_instances/autoscaling.DescribeAutoScalingGroups_1.json b/tests/data/placebo/test_asg_propagate_tag_no_instances/autoscaling.DescribeAutoScalingGroups_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_asg_propagate_tag_no_instances/autoscaling.DescribeAutoScalingGroups_1.json
@@ -0,0 +1,64 @@
+{
+    "status_code": 200,
+    "data": {
+        "AutoScalingGroups": [
+            {
+                "AutoScalingGroupARN": "arn:aws:autoscaling:us-east-1:123456789012:autoScalingGroup:5fd42b45-825e-45c1-b02a-6a97d9512f30:autoScalingGroupName/c7n.asg.ec2.01",
+                "TargetGroupARNs": [],
+                "SuspendedProcesses": [],
+                "DesiredCapacity": 1,
+                "Tags": [
+                    {
+                        "ResourceType": "auto-scaling-group",
+                        "ResourceId": "CustodianASG",
+                        "PropagateAtLaunch": true,
+                        "Value": "ubuntu",
+                        "Key": "Platform"
+                    }
+                ],
+                "EnabledMetrics": [],
+                "LoadBalancerNames": [],
+                "AutoScalingGroupName": "c7n.asg.ec2.01",
+                "DefaultCooldown": 300,
+                "MinSize": 1,
+                "Instances": [
+                ],
+                "MaxSize": 1,
+                "VPCZoneIdentifier": "subnet-ea6429a3,subnet-2c5dcf01",
+                "HealthCheckGracePeriod": 300,
+                "TerminationPolicies": [
+                    "Default"
+                ],
+                "LaunchConfigurationName": "c7n.asg.01",
+                "CreatedTime": {
+                    "hour": 19,
+                    "__class__": "datetime",
+                    "month": 2,
+                    "second": 3,
+                    "microsecond": 171000,
+                    "year": 2017,
+                    "day": 24,
+                    "minute": 0
+                },
+                "AvailabilityZones": [
+                    "us-east-1a",
+                    "us-east-1b"
+                ],
+                "HealthCheckType": "EC2",
+                "NewInstancesProtectedFromScaleIn": false
+            }
+        ],
+        "ResponseMetadata": {
+            "RetryAttempts": 0,
+            "HTTPStatusCode": 200,
+            "RequestId": "770b2115-fac8-11e6-ac2c-5ff00137fb8b",
+            "HTTPHeaders": {
+                "x-amzn-requestid": "770b2115-fac8-11e6-ac2c-5ff00137fb8b",
+                "vary": "Accept-Encoding",
+                "content-length": "7006",
+                "content-type": "text/xml",
+                "date": "Fri, 24 Feb 2017 19:35:57 GMT"
+            }
+        }
+    }
+}
diff --git a/tests/data/placebo/test_asg_propagate_tag_no_instances/ec2.CreateTags_1.json b/tests/data/placebo/test_asg_propagate_tag_no_instances/ec2.CreateTags_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_asg_propagate_tag_no_instances/ec2.CreateTags_1.json
@@ -0,0 +1,20 @@
+{
+    "status_code": 400,
+    "data": {
+        "Error": {
+            "Code": "MissingParameter",
+            "Message": "The request must contain the parameter resourceIdSet"
+        },
+        "ResponseMetadata": {
+            "RequestId": "f229a23a-def5-4288-8ee0-f754ea63eda2",
+            "HTTPStatusCode": 400,
+            "HTTPHeaders": {
+                "transfer-encoding": "chunked",
+                "date": "Thu, 21 Nov 2019 18:49:02 GMT",
+                "connection": "close",
+                "server": "AmazonEC2"
+            },
+            "RetryAttempts": 0
+        }
+    }
+}
diff --git a/tests/data/placebo/test_asg_propagate_tag_no_instances/ec2.DeleteTags_1.json b/tests/data/placebo/test_asg_propagate_tag_no_instances/ec2.DeleteTags_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_asg_propagate_tag_no_instances/ec2.DeleteTags_1.json
@@ -0,0 +1,9 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {
+            "HTTPStatusCode": 200,
+            "RequestId": "1b312937-e532-443a-99bd-1dab530b4a84"
+        }
+    }
+}
diff --git a/tests/test_asg.py b/tests/test_asg.py
--- a/tests/test_asg.py
+++ b/tests/test_asg.py
@@ -819,6 +819,28 @@ def test_asg_not_propagate_tag_missing(self):
         self.assertEqual(len(resources), 1)
         self.assertEqual(resources[0]["AutoScalingGroupName"], "c7n-asg-np-missing")
 
+    def test_asg_propagate_tag_no_instances(self):
+        factory = self.replay_flight_data("test_asg_propagate_tag_no_instances")
+        p = self.load_policy(
+            {
+                "name": "asg-tag",
+                "resource": "asg",
+                "filters": [{"tag:Platform": "ubuntu"}],
+                "actions": [
+                    {
+                        "type": "propagate-tags",
+                        "trim": True,
+                        "tags": ["CustomerId", "Platform"],
+                    },
+                ],
+            },
+            session_factory=factory,
+        )
+
+        resources = p.run()
+        self.assertEqual(len(resources), 1)
+        self.assertEqual(resources[0]["AutoScalingGroupName"], "c7n.asg.ec2.01")
+
     def test_asg_filter_capacity_delta_match(self):
         factory = self.replay_flight_data("test_asg_filter_capacity_delta_match")
         p = self.load_policy(