Repository: cloud-custodian/cloud-custodian

Issue:
 c7n-org and variables
Hi,

I am using this wonderful tool c7n-org. I want to run my policy in multiple accounts.

I saw someone talking about variable like role: arn:aws:iam::**{account_id}**:role/Custodian :

```
policies:
  - name: ec2-daily-backup
    resource: ec2
    mode:
         type: periodic
         schedule: "cron(0 3 * * ? *)"
         **role: arn:aws:iam::{account_id}:role/Custodian**
```

but when I try, I am getting an error

```
c7n-org run -c accounts.yml -s output -u ec2-snapshot.yml --debug 
2018-09-26 11:50:37,492: c7n_org:DEBUG Running policy:ec2-daily-backup account:JCDecaux SA Pub region:eu-west-1
2018-09-26 11:50:43,252: c7n_org:ERROR Exception running policy:ec2-daily-backup account:XXXX region:eu-west-1 error:An error occurred (ValidationException) when calling the CreateFunction operation: 1 validation error detected: Value 'arn:aws:iam::{account_id}:role/Custodian' at 'role' failed to satisfy constraint: Member must satisfy regular expression pattern: arn:(aws[a-zA-Z-]*)?:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@\-_/]+

```

Is there a way to use the same policy for multiple accounts like I tried or should I write a policy by account ?




Discussion:
We have also been having this issue since >0.8.30.0 release but assumed it must have been something we were doing wrong because we did not see anyone else having issues with it. 

Specifically we are only seeing this using when using c7n-org, running an cloudtrail type mode of policy with variable interpolation on the {account_id} in an IAM role. It works fine when running in a single account through c7n, rather than targeting multi accounts with c7n-org.

I know that _policy.expand_variables_ changed quite a bit during #2418 and am wondering if that somehow broke it. I have not had too much time to dig into it but <=0.8.30.0 releases worked fine.

We can get it to constantly fail by having a simple cloudtrail policy such as:
```policies:
  - name: rds-public-terminate-ct
    description: Identify RDS instances that are publicly accessible, notify the owner, and delete the database.
    resource: rds
    filters:
      - PubliclyAccessible: true
    actions:
      - type: delete
        skip-snapshot: true
      - type: notify
        template: emailTemplate.html
        priority_header: 2
        to:
          - resource-owner
        subject: "[cloud-custodian] Publicly accessible RDS instance discovered [{{ account }} {{ region }}]"
        violation_desc: "The following RDS instances were allocated public IP addresses and have been deleted"
        action_desc: RDS databases must not be publicly accessible per <a href="URL">policy</a>
        questions_email: contact@email.com
        questions_slack: general
        transport:
          type: sqs
          queue: MAILER_QUEUE_URL
    mode:
      type: cloudtrail
      events:
        - CreateDBInstance
      role: arn:aws:iam::{account_id}:role/service-role/CustomRole
      tags: {'id': '000000','environment-type': 'PRODUCTION','resource-owner': 'contact@email.com','secondId': '00000'}
```
And running a c7n-org command such as:
c7n-org run -c all-accounts-read.yaml -s s3bucket -u rds.public.terminate-ct.yaml
After a brief initial investigation, it seems like it is the  removal of the "expand_variables" call in the LambdaMode's "provision" method is the cause these effects on c7n-org.  In the 0.8.31.0 release expand_variables was removed from the LambdaMode class and moved to the Policy class, the call to expand variables was also removed from the LambdaMode's provision method.  

**/c7n/policy.py**
![image](https://user-images.githubusercontent.com/43418865/46083095-40a23f80-c166-11e8-87ba-829097721509.png)

This removal seemed to be compensated by placing a call to expand_variables in the  "policy_command" decorator method for the c7n's CLI run command.  While the expand_variables call was added to the c7n CLI run command it was not compensated for in the c7n-org CLI commands.  

**/c7n/command.py**
![image](https://user-images.githubusercontent.com/43418865/46083132-544da600-c166-11e8-8463-143cb0557a57.png)


After the 0.8.31.0 release, when c7n-org is called in run mode, expand_variables is not getting called since the LambdaMode class no longer automatically calls expand_variables inside it's provision method. A code change to the c7n-org CLI  that is equivalent to the addition made to the run command for c7n CLI could be a possible solution.

Adding an expand_variables call to the c7n-org cli.py file on line 468 seemed to fix the behavior we were seeing in CloudTrail mode policies using variable interpolation. 

**/c7n_org/c7n_org/cli.py**
![image](https://user-images.githubusercontent.com/43418865/46083804-c83c7e00-c167-11e8-8715-df3489de22c7.png)

Looking forward to hearing everyone else's feedback.


it's working  for me as well.

Was it removed on purpose ? What am I missing? 
I'm not sure, but I think it was just an oversight in the 0.8.31.0 release.  There was an effort to make variable interpolation consistent globally, so they moved expand_variables to the Policy class instead of just having it in a limited scope.  

Everything works normally when using the regular c7n run command because they added a call to expand_variables in the c7n run command. However, they don't seem to get expanded in the c7n-org run command, leading me to believe it was just overlooked.  

Resolving patch:
diff --git a/tools/c7n_org/c7n_org/cli.py b/tools/c7n_org/c7n_org/cli.py
--- a/tools/c7n_org/c7n_org/cli.py
+++ b/tools/c7n_org/c7n_org/cli.py
@@ -466,6 +466,11 @@ def run_account(account, region, policies_config, output_path,
     st = time.time()
     with environ(**account_tags(account)):
         for p in policies:
+
+            # Variable expansion and non schema validation (not optional)
+            p.expand_variables(p.get_variables())
+            p.validate()
+
             log.debug(
                 "Running policy:%s account:%s region:%s",
                 p.name, account['name'], region)
diff --git a/tools/c7n_org/setup.py b/tools/c7n_org/setup.py
--- a/tools/c7n_org/setup.py
+++ b/tools/c7n_org/setup.py
@@ -21,7 +21,7 @@
 
 setup(
     name="c7n_org",
-    version='0.4.0',
+    version='0.4.1',
     description="Cloud Custodian - Multi Account",
     classifiers=[
         "Topic :: System :: Systems Administration",


Test patch: