Repository: cloud-custodian/cloud-custodian

Issue:
datapipeline resource error while using CLI 0.9.13
### Describe the bug

When running a policy to set tags on the AWS data pipeline resource using custodian CLI version 0.9.13 it results in an error.

### What did you expect to happen?

I expected the policy to work successfully and tag datapipeline resources found by the filter.

### Cloud Provider

Amazon Web Services (AWS)

### Cloud Custodian version and dependency information

```shell
Custodian:   0.9.13
Python:      3.8.9 (default, Aug 21 2021, 15:53:23)
             [Clang 13.0.0 (clang-1300.0.29.3)]
Platform:    posix.uname_result(sysname='Darwin', nodename='c02g22zsmd6r', release='20.6.0', version='Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64', machine='x86_64')
Using venv:  True
Docker: False
Installed:

argcomplete==1.12.3
attrs==21.2.0
boto3==1.17.102
botocore==1.20.102
importlib-metadata==4.6.0
jmespath==0.10.0
jsonschema==3.2.0
pyrsistent==0.18.0
python-dateutil==2.8.1
pyyaml==5.4.1
s3transfer==0.4.2
setuptools==49.2.1
six==1.16.0
tabulate==0.8.9
typing-extensions==3.10.0.0
urllib3==1.26.6
zipp==3.4.1
```


### Policy

```shell
policies:
  - name: tags-aws-data-pipeline
    resource: aws.datapipeline
    filters:
      - or:
          - "tag:foo": absent
    actions:
      - type: tag
        tags:
            foo: bar
```
```


### Relevant log/traceback output

```shell
2021-10-01 10:30:21,110: custodian.commands:DEBUG Loaded file aws/aws-datapipeline-tbm-tags.yml. Contains 1 policies
2021-10-01 10:30:21,140: botocore.credentials:INFO Found credentials in environment variables.
2021-10-01 10:30:21,560: custodian.output:DEBUG Storing output with <LogFile file://./aws-data-pipeline-set-policy-tags/custodian-run.log>
2021-10-01 10:30:21,570: custodian.policy:DEBUG Running policy:aws-data-pipeline-set-policy-tags resource:aws.datapipeline region:us-east-1 c7n:0.9.13
2021-10-01 10:30:21,571: botocore.credentials:INFO Found credentials in environment variables.
2021-10-01 10:30:22,353: custodian.output:DEBUG metric:PolicyException Count:1 policy:aws-data-pipeline-set-policy-tags restype:aws.datapipeline
2021-10-01 10:30:22,353: custodian.output:DEBUG metric:ApiCalls Count:2 policy:aws-data-pipeline-set-policy-tags restype:aws.datapipeline
2021-10-01 10:30:22,353: custodian.output:ERROR Error while executing policy
Traceback (most recent call last):
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/policy.py", line 290, in run
    resources = self.policy.resource_manager.resources()
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/query.py", line 527, in resources
    resources = self.filter_resources(resources)
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/manager.py", line 111, in filter_resources
    resources = f.process(resources, event)
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/filters/core.py", line 317, in process
    return self.process_set(resources, event)
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/filters/core.py", line 329, in process_set
    resource_map = {r[rtype_id]: r for r in resources}
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/filters/core.py", line 329, in <dictcomp>
    resource_map = {r[rtype_id]: r for r in resources}
KeyError: 'pipelineId'
2021-10-01 10:30:22,361: custodian.commands:ERROR Error while executing policy aws-data-pipeline-set-policy-tags, continuing
Traceback (most recent call last):
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/commands.py", line 301, in run
    policy()
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/policy.py", line 1205, in __call__
    resources = mode.run()
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/policy.py", line 290, in run
    resources = self.policy.resource_manager.resources()
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/query.py", line 527, in resources
    resources = self.filter_resources(resources)
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/manager.py", line 111, in filter_resources
    resources = f.process(resources, event)
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/filters/core.py", line 317, in process
    return self.process_set(resources, event)
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/filters/core.py", line 329, in process_set
    resource_map = {r[rtype_id]: r for r in resources}
  File "/Users/project/foo/custodian/lib/python3.8/site-packages/c7n/filters/core.py", line 329, in <dictcomp>
    resource_map = {r[rtype_id]: r for r in resources}
KeyError: 'pipelineId'
2021-10-01 10:30:22,363: custodian.commands:ERROR The following policies had errors while executing
 - aws-data-pipeline-set-policy-tags`
```
```


### Extra information or context

_No response_


Discussion:
yeah i've seen this one too recently thanks for the bug report.

Resolving patch:
diff --git a/c7n/resources/datapipeline.py b/c7n/resources/datapipeline.py
--- a/c7n/resources/datapipeline.py
+++ b/c7n/resources/datapipeline.py
@@ -66,6 +66,7 @@ def process_tags(pipe_set):
 
         for pipe_desc in results['pipelineDescriptionList']:
             pipe = pipe_map[pipe_desc['pipelineId']]
+            pipe['pipelineId'] = pipe_desc['pipelineId']
             pipe['Tags'] = [
                 {'Key': t['key'], 'Value': t['value']}
                 for t in pipe_desc['tags']]


Test patch:
diff --git a/tests/test_datapipeline.py b/tests/test_datapipeline.py
--- a/tests/test_datapipeline.py
+++ b/tests/test_datapipeline.py
@@ -61,6 +61,7 @@ def test_reporting(self):
         )
         resources = p.run()
         self.assertEqual(len(resources), 1)
+        assert resources[0]['pipelineId']
         resource = resources[0]
         self.assertEqual(resource["name"], "PipelinesFTW")
         self.assertEqual(resource["Tags"], [{"Key": "foo", "Value": "bar"}])