Repository: cloud-custodian/cloud-custodian

Issue:
aws - step functions resources need tags pulled
**Describe the bug**
AWS Custodian Policy for StepFunction does not work with config-poll-rule. There is no error message in the Logs, but the policy does not give expected result.
When tested in our AWS account, Config rule must return 19 Non-Compliant and 1 as Compliant. But it says all 20 as Non-Compliant.
Attached the log file and a sample cloudformation template.

**To Reproduce**
Deploy the policy in your account and check your config rule for stepfunction:

**Expected behavior**
On successful execution of the policy, the config rule should flag the StepFunction with missing required tags as Non-compliant resource.


**Background (please complete the following information):**
 - Python Version:  python 3.8.5
 - Custodian Version: 0.9.7
 - Cloud Provider: AWS
 - Policy: [please exclude any account/sensitive information]

---
```
policies:
  - name: d-stepfunction-required-tags
    resource: aws.step-machine
    description: "Type: Config Rule. Compliance: Tag Validation."
    mode:
      type: config-poll-rule
      schedule: TwentyFour_Hours
    filters:
      - and:
          - or:
              - tag:CostCenter: absent
              - tag:RSMs: absent
              - tag:ApplicationIDName: absent
          - tag:__Exception-StepFunctionTags: absent
```
---
**Additional context**
Add any other context about the problem here.
[step_function_logs.txt](https://github.com/cloud-custodian/cloud-custodian/files/5495769/step_function_logs.txt)
[step_test.txt](https://github.com/cloud-custodian/cloud-custodian/files/5495823/step_test.txt)




Discussion:
out of curiosity, could you clarify what value using  `config-poll` mode provides, its seems quite limited in functionality or utility, also your `and` clause is superflous
Hi @kapilt Thanks for getting back. AWS config rule does not support the StepFunction resource. Hence I used the config-poll-rule to create a custom config rule.
The config rule gets created but tags all the resources as Compliant.

But when I test the same policy on the command line using the custodian run -s out test.yaml it seems to give the correct number of non-compliant resources.

about the and yeah, I can remove that. 

please let me know if you need any further details. 
Hi @kapilt, do you have an update on the above bug? Thanks.
> Hi @kapilt Thanks for getting back. AWS config rule does not support the StepFunction resource. Hence I used the config-poll-rule to create a custom config rule.
> The config rule gets created but tags all the resources as Compliant.
> 
> But when I test the same policy on the command line using the custodian run -s out test.yaml it seems to give the correct number of non-compliant resources.
> 
> about the and yeah, I can remove that.
> 
> please let me know if you need any further details.

Note: There is a correction- All resources are marked as 'non-compliant' resources when the config rule gets created.
Here are the logs and policy for only 1 filter.

policy:

```
---
 
policies:
  - name: d-stepfunction-required-tags
    resource: aws.step-machine
    description: "Type: Config Rule. Compliance: Tag Validation."
    mode:
      type: config-poll-rule
      schedule: TwentyFour_Hours
    filters:
      - tag:ApplicationIDName: absent
 
```

Logs: 
[StepFunctionwith1FilterPolicyLogs.docx](https://github.com/cloud-custodian/cloud-custodian/files/5513435/StepFunctionwith1FilterPolicyLogs.docx)

Total resources- 21. Compliant- 1, Non-compliant- 20. However, when the config rule is created all 21 of them are marked non-compliant.
I suspect the root cause is the same as #6204
its unrelated, config-poll-rule doesn't gather data via config because config has no data for these resources, its a service level hack to enable periodic gathering from resources that config doesn't support natively. the issue in 6204 was more to do with translating from config's serialization for a resource it does support to api describe format.
[update] comment moved to code-pipeline issue #6204
nutshell here is just augmenting for tags on step function resources

Resolving patch:
diff --git a/c7n/resources/sfn.py b/c7n/resources/sfn.py
--- a/c7n/resources/sfn.py
+++ b/c7n/resources/sfn.py
@@ -4,7 +4,7 @@
 from c7n.actions import Action
 from c7n.manager import resources
 from c7n.query import QueryResourceManager, TypeInfo
-from c7n.tags import Tag, RemoveTag
+from c7n.tags import Tag, RemoveTag, universal_augment
 from c7n.utils import type_schema, local_session, dumps, chunks
 
 
@@ -26,6 +26,10 @@ class resource_type(TypeInfo):
             "describe_state_machine", "stateMachineArn",
             'stateMachineArn', None)
 
+    def augment(self, resources):
+        resources = super().augment(resources)
+        return universal_augment(self, resources)
+
 
 class InvokeStepFunction(Action):
     """Invoke step function on resources.


Test patch:
diff --git a/tests/data/placebo/test_invoke_sfn/tagging.GetResources_1.json b/tests/data/placebo/test_invoke_sfn/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_invoke_sfn/tagging.GetResources_1.json
@@ -0,0 +1,18 @@
+{
+    "status_code": 200, 
+    "data": {
+        "PaginationToken": "", 
+        "ResourceTagMappingList": [], 
+        "ResponseMetadata": {
+            "RetryAttempts": 0, 
+            "HTTPStatusCode": 200, 
+            "RequestId": "3e252497-7f67-11e7-9836-a96970d9def5", 
+            "HTTPHeaders": {
+                "x-amzn-requestid": "3e252497-7f67-11e7-9836-a96970d9def5", 
+                "date": "Sat, 12 Aug 2017 14:05:05 GMT", 
+                "content-length": "50", 
+                "content-type": "application/x-amz-json-1.1"
+            }
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_invoke_sfn_bulk/tagging.GetResources_1.json b/tests/data/placebo/test_invoke_sfn_bulk/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_invoke_sfn_bulk/tagging.GetResources_1.json
@@ -0,0 +1,18 @@
+{
+    "status_code": 200, 
+    "data": {
+        "PaginationToken": "", 
+        "ResourceTagMappingList": [], 
+        "ResponseMetadata": {
+            "RetryAttempts": 0, 
+            "HTTPStatusCode": 200, 
+            "RequestId": "3e252497-7f67-11e7-9836-a96970d9def5", 
+            "HTTPHeaders": {
+                "x-amzn-requestid": "3e252497-7f67-11e7-9836-a96970d9def5", 
+                "date": "Sat, 12 Aug 2017 14:05:05 GMT", 
+                "content-length": "50", 
+                "content-type": "application/x-amz-json-1.1"
+            }
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sfn_resource/tagging.GetResources_1.json b/tests/data/placebo/test_sfn_resource/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sfn_resource/tagging.GetResources_1.json
@@ -0,0 +1,18 @@
+{
+    "status_code": 200, 
+    "data": {
+        "PaginationToken": "", 
+        "ResourceTagMappingList": [], 
+        "ResponseMetadata": {
+            "RetryAttempts": 0, 
+            "HTTPStatusCode": 200, 
+            "RequestId": "3e252497-7f67-11e7-9836-a96970d9def5", 
+            "HTTPHeaders": {
+                "x-amzn-requestid": "3e252497-7f67-11e7-9836-a96970d9def5", 
+                "date": "Sat, 12 Aug 2017 14:05:05 GMT", 
+                "content-length": "50", 
+                "content-type": "application/x-amz-json-1.1"
+            }
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sfn_tag_resource/tagging.GetResources_1.json b/tests/data/placebo/test_sfn_tag_resource/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sfn_tag_resource/tagging.GetResources_1.json
@@ -0,0 +1,18 @@
+{
+    "status_code": 200, 
+    "data": {
+        "PaginationToken": "", 
+        "ResourceTagMappingList": [], 
+        "ResponseMetadata": {
+            "RetryAttempts": 0, 
+            "HTTPStatusCode": 200, 
+            "RequestId": "3e252497-7f67-11e7-9836-a96970d9def5", 
+            "HTTPHeaders": {
+                "x-amzn-requestid": "3e252497-7f67-11e7-9836-a96970d9def5", 
+                "date": "Sat, 12 Aug 2017 14:05:05 GMT", 
+                "content-length": "50", 
+                "content-type": "application/x-amz-json-1.1"
+            }
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sfn_untag_resource/tagging.GetResources_1.json b/tests/data/placebo/test_sfn_untag_resource/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sfn_untag_resource/tagging.GetResources_1.json
@@ -0,0 +1,18 @@
+{
+    "status_code": 200, 
+    "data": {
+        "PaginationToken": "", 
+        "ResourceTagMappingList": [], 
+        "ResponseMetadata": {
+            "RetryAttempts": 0, 
+            "HTTPStatusCode": 200, 
+            "RequestId": "3e252497-7f67-11e7-9836-a96970d9def5", 
+            "HTTPHeaders": {
+                "x-amzn-requestid": "3e252497-7f67-11e7-9836-a96970d9def5", 
+                "date": "Sat, 12 Aug 2017 14:05:05 GMT", 
+                "content-length": "50", 
+                "content-type": "application/x-amz-json-1.1"
+            }
+        }
+    }
+}
\ No newline at end of file