Repository: cloud-custodian/cloud-custodian

Issue:
aws.secrets-manager:tag action tries to retag secret with existing tags as well
### Describe the bug

I have a policy that checks all AWS Secret Manager secrets for the existence of two particular tags. If the secret is missing either of these tags, it tries to makr (i.e. tag) the secret for later action. However, Custodian fails to apply its marking tag with the following error:

2023-10-18 14:25:20,524 - custodian.policy - INFO - policy:secrets-manager-secret-mark-untagged resource:secrets-manager region:us-west-2 count:2 time:0.03
2023-10-18 14:25:20,525 - custodian.actions - INFO - Tagging 2 resources for notify on 2023/10/19
2023-10-18 14:25:21,519 - custodian.actions - ERROR - Exception with tags: [{'Key': 'c7n-secrets-manager-secret-untagged', 'Value': 'Resource does not meet policy: notify@2023/10/19'}]  An error occurred (InvalidParameterException) when calling the TagResource operation: You can't modify system tags. Tag key: 'aws:cloudformation:stack-name'.

It appears Custodian is trying to retag with all existing tags in addition to the new marking tag. AWS doesn't like you messing with the system tags?

### What did you expect to happen?

I expected Custodian to only add the marking tag rather than try to apply all the existing tags as well.

Is this possible? Is it a bug in Custodian or in the AWS Secrets Manager API?

### Cloud Provider

Amazon Web Services (AWS)

### Cloud Custodian version and dependency information

```shell
Custodian:   0.9.32
Python:      3.8.16 (default, Mar 10 2023, 03:07:48) 
             [GCC 7.3.1 20180712 (Red Hat 7.3.1-15)]
Platform:    posix.uname_result(sysname='Linux', nodename='custodian', release='4.14.320-243.544.amzn2.x86_64', version='#1 SMP Tue Aug 1 21:03:08 UTC 2023', machine='x86_64')
Using venv:  False
Docker: False
Installed: 

argcomplete==3.1.2
attrs==23.1.0
boto3==1.28.58
botocore==1.31.58
docutils==0.18.1
importlib-metadata==5.2.0
importlib-resources==6.1.0
jmespath==1.0.1
jsonschema==4.19.1
jsonschema-specifications==2023.7.1
pkgutil-resolve-name==1.3.10
python-dateutil==2.8.2
pyyaml==6.0.1
referencing==0.30.2
rpds-py==0.10.3
s3transfer==0.7.0
six==1.16.0
tabulate==0.9.0
urllib3==1.26.17
zipp==3.17.0
```


### Policy

```shell
Here is the action bit:

actions:
      - type: mark-for-op
        tag: c7n-secrets-manager-secret-untagged
        op: notify
        days: 1
```


### Relevant log/traceback output

```shell
2023-10-18 14:25:20,524 - custodian.policy - INFO - policy:secrets-manager-secret-mark-untagged resource:secrets-manager region:us-west-2 count:2 time:0.03
2023-10-18 14:25:20,525 - custodian.actions - INFO - Tagging 2 resources for notify on 2023/10/19
2023-10-18 14:25:21,519 - custodian.actions - ERROR - Exception with tags: [{'Key': 'c7n-secrets-manager-secret-untagged', 'Value': 'Resource does not meet policy: notify@2023/10/19'}]  An error occurred (InvalidParameterException) when calling the TagResource operation: You can't modify system tags. Tag key: 'aws:cloudformation:stack-name'.
```


### Extra information or context

_No response_


Discussion:

Resolving patch:
diff --git a/c7n/resources/secretsmanager.py b/c7n/resources/secretsmanager.py
--- a/c7n/resources/secretsmanager.py
+++ b/c7n/resources/secretsmanager.py
@@ -96,7 +96,8 @@ class TagSecretsManagerResource(Tag):
 
     def process_resource_set(self, client, resources, new_tags):
         for r in resources:
-            tags = {t['Key']: t['Value'] for t in r.get('Tags', ())}
+            tags = {t['Key']: t['Value'] for t in r.get('Tags', ())
+                    if not t['Key'].startswith('aws:')}
             for t in new_tags:
                 tags[t['Key']] = t['Value']
             formatted_tags = [{'Key': k, 'Value': v} for k, v in tags.items()]


Test patch: