Repository: cloud-custodian/cloud-custodian

Issue:
App-ELB Conditions Error - PolicyConditions object has no attribute config
**Describe the bug**
When running a app-elb policy which has a conditions statement to see if the account_id is present in a S3 csv file we are getting the error "PolicyConditions object has no attribute config"

**To Reproduce**
Steps to reproduce the behavior: Running policy produces error

**Expected behavior**
On previous 0.9.1 this policy seemed to work fine.


**Background (please complete the following information):**
 - OS: [e.g. OSX 10.15] Ubuntu v20
 - Python Version: [e.g. python 3.8.1] 3.8.2
 - Custodian Version: [e.g. 0.8.46.1] 0.9.4.0  c7n-org 0.6.3
 - Tool Version: [if applicable] c7n-org 0.6.3
 - Cloud Provider: [e.g. gcp, aws, azure] aws
 - Policy: [please exclude any account/sensitive information]
```

- name: app-elb-tag-now-compliant-poll
  resource: app-elb
  description: |
    Any Application ELBs which have previously been marked as non compliant with tag policies, that are now compliant
    should be unmarked and the customer emailed.
  conditions:
    - type: value
      key: account_id
      op: not-in
      value_from:
         url: s3://mys3bucket/AccountExceptions.csv
         format: csv2dict
  filters:
    - "tag:c7n_tag_compliance_poll": not-null
    - "tag:ResourceContact": present
    - type: value
      key: "tag:ResourceContact"
      op: not-in
      value_from:
         url: s3://mys3bucket/InvalidEmails.csv
         format: csv2dict
    - type: value
      key: "tag:ResourceContact"
      op: regex
      value: (^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$)
    - "tag:BillingCostCenter": present
    - type: value
      key: "tag:BillingCostCenter"
      op: in
      value_from:
         url: s3://mys3bucket/ValidCostCenters.csv
         format: csv2dict
    - "tag:Environment": present
    - type: value
      key: "tag:Environment"
      op: in
      value:
        - tst
        - dev
        - mdl
        - acc
        - prd
        - eat
    - "tag:ResourcePurpose": present
    - type: value
      key: "tag:ResourcePurpose"
      op: regex
      value: ^.{3,}$
    - "tag:Division": present
    - type: value
      key: "tag:Division"
      op: in
      value_from:
         url: s3://mys3bucket/ValidDivisions.csv
         format: csv2dict
    - type: value
      value_type: date
      key: CreatedTime
      op: greater-than
      value: "2020/05/05"
  actions:
    - type: remove-tag
      tags: ["c7n_tag_compliance_poll"]
    - type: notify
      template: default.html
      priority_header: 1
      subject: "Application ELB Tags - Application ELB is now compliant - [custodian {{ account }} - {{ region }}]" 
      violation_desc: |
          The Application ELB which was previously missing or had invalid required tags is now compliant and won't be deleted
      action_desc: "Actions Taken:  Notification Only. The Application ELB is now compliant with tags."
      to:
        - resource-owner
      transport:
        type: sqs
        queue: https://sqs.us-east-1.amazonaws.com/123456789/cloud-custodian-mailer
        region: us-east-1

```
 - Traceback: [if applicable, please exclude sensitive/account information]

```
Traceback (most recent call last):
  File "/root/c7n/lib/python3.8/site-packages/c7n/cli.py", line 362, in main
    command(config)
  File "/root/c7n/lib/python3.8/site-packages/c7n/commands.py", line 136, in _load_policies
    return f(options, list(policies))
  File "/root/c7n/lib/python3.8/site-packages/c7n/commands.py", line 282, in run
    policy()
  File "/root/c7n/lib/python3.8/site-packages/c7n/policy.py", line 1171, in __call__
    elif not self.is_runnable():
  File "/root/c7n/lib/python3.8/site-packages/c7n/policy.py", line 1011, in is_runnable
    return self.conditions.evaluate(event)
  File "/root/c7n/lib/python3.8/site-packages/c7n/policy.py", line 945, in evaluate
    state = all([f.process([policy_vars], event) for f in self.filters])
  File "/root/c7n/lib/python3.8/site-packages/c7n/policy.py", line 945, in <listcomp>
    state = all([f.process([policy_vars], event) for f in self.filters])
  File "/root/c7n/lib/python3.8/site-packages/c7n/filters/core.py", line 530, in process
    return super(ValueFilter, self).process(resources, event)
  File "/root/c7n/lib/python3.8/site-packages/c7n/filters/core.py", line 217, in process
    return list(filter(self, resources))
  File "/root/c7n/lib/python3.8/site-packages/c7n/filters/core.py", line 517, in __call__
    matched = self.match(i)
  File "/root/c7n/lib/python3.8/site-packages/c7n/filters/core.py", line 569, in match
    values = ValuesFrom(self.data['value_from'], self.manager)
  File "/root/c7n/lib/python3.8/site-packages/c7n/resolver.py", line 132, in __init__
    'account_id': manager.config.account_id,
AttributeError: 'PolicyConditions' object has no attribute 'config'
> /root/c7n/lib/python3.8/site-packages/c7n/resolver.py(132)__init__()
-> 'account_id': manager.config.account_id,
(Pdb)

```
 - `custodian version --debug` output 

```
Custodian:   0.9.4
Python:      3.8.2 (default, Apr 27 2020, 15:53:34)
             [GCC 9.3.0]
Platform:    posix.uname_result(sysname='Linux', nodename='ip-10-128-128-28', release='5.4.0-1009-aws', version='#9-Ubuntu SMP Sun Apr 12 19:46:01 UTC 2020', machine='x86_64')
Using venv:  True
Docker: False
Installed:

argcomplete==1.11.1
attrs==19.3.0
boto3==1.14.20
botocore==1.17.20
docutils==0.15.2
importlib-metadata==1.7.0
jmespath==0.10.0
jsonschema==3.2.0
pyrsistent==0.16.0
python-dateutil==2.8.1
pyyaml==5.3.1
s3transfer==0.3.3
setuptools==44.0.0
six==1.15.0
tabulate==0.8.7
urllib3==1.25.9
zipp==3.1.0
```

**Additional context**
Add any other context about the problem here.


Discussion:

Patch:
diff --git a/c7n/policy.py b/c7n/policy.py
index bae8e04cb..faa858ca2 100644
--- a/c7n/policy.py
+++ b/c7n/policy.py
@@ -922,6 +922,15 @@ class PolicyConditions:
         self.policy = policy
         self.data = data
         self.filters = self.data.get('conditions', [])
+        # for value_from usage / we use the conditions class
+        # to mimic a resource manager interface. we can't use
+        # the actual resource manager as we're overriding block
+        # filters which work w/ resource type metadata and our
+        # resource here is effectively the execution variables.
+        self.config = self.policy.options
+        rm = self.policy.resource_manager
+        self._cache = rm._cache
+        self.session_factory = rm.session_factory
         # used by c7n-org to extend evaluation conditions
         self.env_vars = {}
 
diff --git a/c7n/resources/cw.py b/c7n/resources/cw.py
index b0ccf6e14..6c8655a52 100644
--- a/c7n/resources/cw.py
+++ b/c7n/resources/cw.py
@@ -176,6 +176,13 @@ class LogGroup(QueryResourceManager):
         return [r['arn'][:-2] for r in resources]
 
 
+@LogGroup.filter_registry.register('metrics')
+class LogGroupMetrics(MetricsFilter):
+
+    def get_dimensions(self, resource):
+        return [{'Name': 'LogGroupName', 'Value': resource['logGroupName']}]
+
+
 @LogGroup.action_registry.register('retention')
 class Retention(BaseAction):
     """Action to set the retention period (in days) for CloudWatch log groups
diff --git a/docs/source/gcp/gettingstarted.rst b/docs/source/gcp/gettingstarted.rst
index 26563e237..ce7773077 100644
--- a/docs/source/gcp/gettingstarted.rst
+++ b/docs/source/gcp/gettingstarted.rst
@@ -61,7 +61,7 @@ Then run the following command, substituting your username:
 
 .. code-block:: bash
 
-    gcloud auth application-default login <your_user_name>
+    gcloud auth application-default login
 
 Executing the command will open a browser window with prompts to finish configuring
 your credentials. For more information on this command,
@@ -105,7 +105,6 @@ Filename: ``custodian.yml``
           - type: value
             key: name
             value: test
-            op: in
         actions:
           - type: stop
 
@@ -119,7 +118,7 @@ Next, run the following command to execute the policy with Custodian:
 
 .. code-block:: bash
 
-    custodian run --output-dir=. custodian.yml
+    GOOGLE_CLOUD_PROJECT="project-id" custodian run --output-dir=. custodian.yml
 
 If successful, you should see output similar to the following on the command line::
 


Test patch:
diff --git a/tests/data/placebo/test_log_group_metric/logs.DescribeLogGroups_1.json b/tests/data/placebo/test_log_group_metric/logs.DescribeLogGroups_1.json
new file mode 100644
index 000000000..fcbcc6f18
--- /dev/null
+++ b/tests/data/placebo/test_log_group_metric/logs.DescribeLogGroups_1.json
@@ -0,0 +1,22 @@
+{
+    "status_code": 200,
+    "data": {
+        "logGroups": [
+            {
+                "logGroupName": "/aws/lambda/myIOTFunction",
+                "creationTime": 6441605581965,
+                "metricFilterCount": 0,
+                "arn": "arn:aws:logs:us-west-2:644160558196:log-group:/aws/lambda/myIOTFunction:*",
+                "storedBytes": 857324
+            },
+            {
+                "logGroupName": "devlog",
+                "creationTime": 6441605581968,
+                "metricFilterCount": 0,
+                "arn": "arn:aws:logs:us-west-2:644160558196:log-group:devlog:*",
+                "storedBytes": 0
+            }
+        ],
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_log_group_metric/monitoring.GetMetricStatistics_1.json b/tests/data/placebo/test_log_group_metric/monitoring.GetMetricStatistics_1.json
new file mode 100644
index 000000000..5478a41ce
--- /dev/null
+++ b/tests/data/placebo/test_log_group_metric/monitoring.GetMetricStatistics_1.json
@@ -0,0 +1,23 @@
+{
+    "status_code": 200,
+    "data": {
+        "Label": "IncomingBytes",
+        "Datapoints": [
+            {
+                "Timestamp": {
+                    "hour": 11, 
+                    "__class__": "datetime", 
+                    "month": 8, 
+                    "second": 0, 
+                    "microsecond": 0, 
+                    "year": 2016, 
+                    "day": 8, 
+                    "minute": 46
+                }, 
+                "Average": 107.0, 
+                "Unit": "Bytes"
+            }  
+        ],
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_log_group_metric/tagging.GetResources_1.json b/tests/data/placebo/test_log_group_metric/tagging.GetResources_1.json
new file mode 100644
index 000000000..8b704d185
--- /dev/null
+++ b/tests/data/placebo/test_log_group_metric/tagging.GetResources_1.json
@@ -0,0 +1,8 @@
+{
+    "status_code": 200,
+    "data": {
+        "PaginationToken": "",
+        "ResourceTagMappingList": [],
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/test_cwl.py b/tests/test_cwl.py
index ff2525d45..8ba8ec007 100644
--- a/tests/test_cwl.py
+++ b/tests/test_cwl.py
@@ -161,3 +161,20 @@ class LogGroupTest(BaseTest):
         self.assertEqual(
             results[0]['kmsKeyId'],
             'arn:aws:kms:us-west-2:644160558196:key/6f13fc53-8da0-46f2-9c69-c1f9fbf471d7')
+
+    def test_metrics(self):
+        session_factory = self.replay_flight_data('test_log_group_metric')
+        p = self.load_policy(
+            {'name': 'metric-log-group',
+             'resource': 'log-group',
+             'filters': [
+                 {"logGroupName": "/aws/lambda/myIOTFunction"},
+                 {"type": "metrics",
+                  "name": "IncomingBytes",
+                  "value": 1,
+                  "op": "greater-than"}]},
+            config={'region': 'us-west-2'},
+            session_factory=session_factory)
+        resources = p.run()
+        self.assertEqual(len(resources), 1)
+        self.assertIn('c7n.metrics', resources[0])
diff --git a/tests/test_policy.py b/tests/test_policy.py
index 823f55817..d4965f6a0 100644
--- a/tests/test_policy.py
+++ b/tests/test_policy.py
@@ -16,6 +16,7 @@ from datetime import datetime, timedelta
 import json
 import logging
 import mock
+import os
 import shutil
 import tempfile
 
@@ -992,6 +993,24 @@ class TestPolicy(BaseTest):
 
 class PolicyConditionsTest(BaseTest):
 
+    def test_value_from(self):
+        tmp_dir = self.change_cwd()
+        p = self.load_policy({
+            'name': 'fx',
+            'resource': 'aws.ec2',
+            'conditions': [{
+                'type': 'value',
+                'key': 'account_id',
+                'op': 'in',
+                'value_from': {
+                    'url': 'file:///{}/accounts.txt'.format(tmp_dir),
+                    'type': 'txt'}
+            }]
+        })
+        with open(os.path.join(tmp_dir, 'accounts.txt'), 'w') as fh:
+            fh.write(p.ctx.options.account_id)
+        self.assertTrue(p.is_runnable())
+
     def test_env_var_extension(self):
         p = self.load_policy({
             'name': 'profx',