Repository: cloud-custodian/cloud-custodian

Issue:
conditions block for serverless policy with not/event structure results in stack trace
**Describe the bug**
Trying to run a policy with a `conditions` block that includes `type: event` conditions for a `mode: cloudtrail` policy causes stack traces as it tries to check the conditions against no resources and no event before the serverless policy is deployed

**To Reproduce**
Run a policy like the example below.

**Expected behavior**
The `event` conditions should not be evaluated to decide if the serverless policy should be provisioned.  Only after the provisioning has happened and an event triggers the policy.

**Background (please complete the following information):**
 - OS: [e.g. OSX 10.15]
 - Python Version: python 3.7.6
 - Custodian Version: 0.9.1.0
 - Tool Version: [if applicable]
 - Cloud Provider: aws
 - Policy: [please exclude any account/sensitive information]
```yaml
policies:
  - name: run-asg-auto-tag-creator
    resource: asg
    comment: Auto-tag creator if missing.
    mode:
      type: cloudtrail
      events:
        - CreateAutoScalingGroup
      tags: *lambda-tags
      role: *lambda-role
    conditions:
      - not:
          - type: event
            key: "detail.userIdentity.arn"
            op: contains
            value: "CloudCustodian"
      - not:
          - type: event
            key: "detail.userIdentity.arn"
            op: regex-case
            value: '.*(AWS.*ServiceRole.*/|Jenkins.*/|\w{8}(?:-?\w{4}){3}-?\w{12}|LambdaFunction|\/sfr-|\/i-)[^\/]*$'
    filters:
      - "tag:Creator": empty
    actions:
      - type: auto-tag-user
        tag: Creator
```
 - Traceback:
```
$ custodian run -d -v -s out -p run-asg-auto-tag-creator ./auto-tag-creator.yaml
2020-05-06 15:47:07,205: custodian.commands:DEBUG Loaded file ./auto-tag-creator.yaml. Contains 11 policies
2020-05-06 15:47:07,223: custodian.aws:DEBUG using default region:us-west-2 from boto
2020-05-06 15:47:07,778: custodian.commands:ERROR Error while executing policy run-asg-auto-tag-creator, continuing
Traceback (most recent call last):
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/commands.py", line 281, in run
    policy()
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/policy.py", line 1056, in __call__
    resources = PullMode(self).run()
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/policy.py", line 279, in run
    if not self.policy.is_runnable():
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/policy.py", line 906, in is_runnable
    return self.conditions.evaluate(event)
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/policy.py", line 843, in evaluate
    state = all([f.process([policy_vars], event) for f in self.filters])
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/policy.py", line 843, in <listcomp>
    state = all([f.process([policy_vars], event) for f in self.filters])
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/filters/core.py", line 299, in process
    return self.process_set(resources, event)
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/filters/core.py", line 314, in process_set
    resource_map = {r[rtype_id]: r for r in resources}
  File "/Users/tstansell/.local/c7n/lib/python3.7/site-packages/c7n/filters/core.py", line 314, in <dictcomp>
    resource_map = {r[rtype_id]: r for r in resources}
KeyError: 'AutoScalingGroupName'
```
 - `custodian version --debug` output
```
Custodian:   0.9.1.0
Python:      3.7.6 (default, Dec 30 2019, 19:38:28)
             [Clang 11.0.0 (clang-1100.0.33.16)]
Platform:    posix.uname_result(sysname='Darwin', release='19.4.0', version='Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64', machine='x86_64')
Using venv:  True
Docker: False
Installed:
```

**Additional context**
Add any other context about the problem here.



Discussion:
If I remove the `not:` from the `conditions` so it only has this, it works as expected:
```
    conditions:
      - type: event
        key: "detail.userIdentity.arn"
        op: regex-case
        value: '.*(AWS.*ServiceRole.*/|Jenkins.*/|\w{8}(?:-?\w{4}){3}-?\w{12}|LambdaFunction|\/sfr-|\/i-)[^\/]*$'
```
we've got subclasses of and/or/not for conditional evaluation that are intended to handle this, we seem to be picking up the original classes instead of the conditional variants.

Resolving patch:
diff --git a/c7n/filters/core.py b/c7n/filters/core.py
--- a/c7n/filters/core.py
+++ b/c7n/filters/core.py
@@ -35,6 +35,7 @@
 from c7n.registry import PluginRegistry
 from c7n.resolver import ValuesFrom
 from c7n.utils import set_annotation, type_schema, parse_cidr
+from c7n.manager import iter_filters
 
 
 class FilterValidationError(Exception):
@@ -140,11 +141,11 @@ def factory(self, data, manager=None):
         if isinstance(data, dict) and len(data) == 1 and 'type' not in data:
             op = list(data.keys())[0]
             if op == 'or':
-                return Or(data, self, manager)
+                return self['or'](data, self, manager)
             elif op == 'and':
-                return And(data, self, manager)
+                return self['and'](data, self, manager)
             elif op == 'not':
-                return Not(data, self, manager)
+                return self['not'](data, self, manager)
             return ValueFilter(data, manager)
         if isinstance(data, str):
             filter_type = data
@@ -164,6 +165,26 @@ def factory(self, data, manager=None):
                     self.plugin_type, data))
 
 
+def trim_runtime(filters):
+    """Remove runtime filters.
+
+    Some filters can only be effectively evaluated at policy
+    execution, ie. event filters.
+
+    When evaluating conditions for dryrun or provisioning stages we
+    remove them.
+    """
+    def remove_filter(f):
+        block = f.get_block_parent()
+        block.filters.remove(f)
+        if isinstance(block, BooleanGroupFilter) and not len(block):
+            remove_filter(block)
+
+    for f in iter_filters(filters):
+        if isinstance(f, EventFilter):
+            remove_filter(f)
+
+
 # Really should be an abstract base class (abc) or
 # zope.interface
 
@@ -198,16 +219,21 @@ def process(self, resources, event=None):
     def get_block_operator(self):
         """Determine the immediate parent boolean operator for a filter"""
         # Top level operator is `and`
-        block_stack = ['and']
+        block = self.get_block_parent()
+        if block.type in ('and', 'or', 'not'):
+            return block.type
+        return 'and'
+
+    def get_block_parent(self):
+        """Get the block parent for a filter"""
+        block_stack = [self.manager]
         for f in self.manager.iter_filters(block_end=True):
             if f is None:
                 block_stack.pop()
-                continue
-            if f.type in ('and', 'or', 'not'):
-                block_stack.append(f.type)
-            if f == self:
-                break
-        return block_stack[-1]
+            elif f == self:
+                return block_stack[-1]
+            elif f.type in ('and', 'or', 'not'):
+                block_stack.append(f)
 
     def merge_annotation(self, r, annotation_key, values):
         block_op = self.get_block_operator()
@@ -249,6 +275,12 @@ def get_resource_type_id(self):
         resource_type = self.manager.get_model()
         return resource_type.id
 
+    def __len__(self):
+        return len(self.filters)
+
+    def __bool__(self):
+        return True
+
 
 class Or(BooleanGroupFilter):
 
@@ -577,7 +609,7 @@ def match(self, i):
         return False
 
     def process_value_type(self, sentinel, value, resource):
-        if self.vtype == 'normalize':
+        if self.vtype == 'normalize' and isinstance(value, str):
             return sentinel, value.strip().lower()
 
         elif self.vtype == 'expr':
diff --git a/c7n/manager.py b/c7n/manager.py
--- a/c7n/manager.py
+++ b/c7n/manager.py
@@ -28,6 +28,18 @@
 from c7n.utils import dumps
 
 
+def iter_filters(filters, block_end=False):
+    queue = deque(filters)
+    while queue:
+        f = queue.popleft()
+        if f is not None and f.type in ('or', 'and', 'not'):
+            if block_end:
+                queue.appendleft(None)
+            for gf in f.filters:
+                queue.appendleft(gf)
+        yield f
+
+
 class ResourceManager:
     """
     A Cloud Custodian resource
@@ -121,15 +133,7 @@ def get_model(self):
         return self.query.resolve(self.resource_type)
 
     def iter_filters(self, block_end=False):
-        queue = deque(self.filters)
-        while queue:
-            f = queue.popleft()
-            if f and f.type in ('or', 'and', 'not'):
-                if block_end:
-                    queue.appendleft(None)
-                for gf in f.filters:
-                    queue.appendleft(gf)
-            yield f
+        return iter_filters(self.filters, block_end=block_end)
 
     def validate(self):
         """
diff --git a/c7n/policy.py b/c7n/policy.py
--- a/c7n/policy.py
+++ b/c7n/policy.py
@@ -26,6 +26,7 @@
 from c7n.ctx import ExecutionContext
 from c7n.exceptions import PolicyValidationError, ClientError, ResourceLimitExceeded
 from c7n.filters import FilterRegistry, And, Or, Not
+from c7n.manager import iter_filters
 from c7n.output import DEFAULT_NAMESPACE, NullBlobOutput
 from c7n.resources import load_resources
 from c7n.registry import PluginRegistry
@@ -917,8 +918,7 @@ def __init__(self, policy, data):
 
     def validate(self):
         self.filters.extend(self.convert_deprecated())
-        self.filters = self.filter_registry.parse(
-            self.filters, self.policy.resource_manager)
+        self.filters = self.filter_registry.parse(self.filters, self)
 
     def evaluate(self, event=None):
         policy_vars = dict(self.env_vars)
@@ -931,6 +931,7 @@ def evaluate(self, event=None):
             'now': datetime.utcnow().replace(tzinfo=tzutil.tzutc()),
             'policy': self.policy.data
         })
+
         # note for no filters/conditions, this uses all([]) == true property.
         state = all([f.process([policy_vars], event) for f in self.filters])
         if not state:
@@ -938,6 +939,9 @@ def evaluate(self, event=None):
                 'Skipping policy:%s due to execution conditions', self.policy.name)
         return state
 
+    def iter_filters(self, block_end=False):
+        return iter_filters(self.filters, block_end=block_end)
+
     def convert_deprecated(self):
         filters = []
         if 'region' in self.policy.data:
@@ -1141,9 +1145,18 @@ def get_permissions(self):
             permissions.update(a.get_permissions())
         return permissions
 
+    def _trim_runtime_filters(self):
+        from c7n.filters.core import trim_runtime
+        trim_runtime(self.conditions.filters)
+        trim_runtime(self.resource_manager.filters)
+
     def __call__(self):
         """Run policy in default mode"""
         mode = self.get_execution_mode()
+        if (isinstance(mode, ServerlessExecutionMode) or
+                self.options.dryrun):
+            self._trim_runtime_filters()
+
         if self.options.dryrun:
             resources = PullMode(self).run()
         elif not self.is_runnable():


Test patch:
diff --git a/tests/test_manager.py b/tests/test_manager.py
--- a/tests/test_manager.py
+++ b/tests/test_manager.py
@@ -11,8 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+from copy import deepcopy
 from c7n.ctx import ExecutionContext
 from c7n.filters import Filter
+from c7n.filters.core import trim_runtime
 from c7n.resources.ec2 import EC2
 from c7n.tags import Tag
 from .common import BaseTest, instance, Bag
@@ -46,6 +48,33 @@ def test_manager_iter_filters(self):
             [f.type for f in p.resource_manager.iter_filters()],
             ['and', 'listener', 'listener'])
 
+    def test_trim_runtime_filters(self):
+        filter_data = [
+            {'and': [
+                {'not': [{
+                    'type': 'event',
+                    'key': 'xyz',
+                    'value': 'bar'}]},
+                {'key': 'value'}]}
+        ]
+
+        p = self.load_policy({
+            'name': 'xyz',
+            'resource': 'aws.ec2',
+            'mode': {
+                'type': 'config-rule',
+                'role': 'xyz'},
+            'filters': deepcopy(filter_data)})
+        m = p.resource_manager
+        trim_runtime(m.filters)
+        self.assertEqual(
+            [n is not None and n.type or n for n in m.iter_filters(
+                block_end=True)],
+            ['and', 'value', None])
+        # we modify filters array in place on resource manager
+        # but we don't touch the underlying policy data structure
+        self.assertEqual(m.data['filters'], filter_data)
+
     def test_filter_get_block_op(self):
         class F(Filter):
             type = 'xyz'
@@ -60,6 +89,11 @@ class F(Filter):
             ]})
 
         m = p.resource_manager
+        self.assertEqual(
+            [n is not None and n.type or n for n in m.iter_filters(
+                block_end=True)],
+            ['and', 'or', None, None, 'not', None, 'or', None])
+
         f = F({}, m)
         m.filters.append(f)
         self.assertEqual(f.get_block_operator(), 'and')
diff --git a/tests/test_policy.py b/tests/test_policy.py
--- a/tests/test_policy.py
+++ b/tests/test_policy.py
@@ -11,6 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+from copy import deepcopy
 from datetime import datetime, timedelta
 import json
 import logging
@@ -25,7 +26,7 @@
 from c7n.resources import aws, load_available
 from c7n.resources.aws import AWS, fake_session
 from c7n.resources.ec2 import EC2
-from c7n.policy import ConfigPollRuleMode
+from c7n.policy import ConfigPollRuleMode, PullMode
 from c7n.schema import generate, JsonSchemaValidator
 from c7n.utils import dumps
 from c7n.query import ConfigSource, TypeInfo
@@ -1046,6 +1047,45 @@ def test_boolean_not_blocks(self):
                     {'region': 'us-east-1'}]}]})
         self.assertFalse(p.is_runnable())
 
+    def test_dryrun_event_filter(self):
+        pdata = {
+            'name': 'manga',
+            'resource': 'aws.ec2',
+            'mode': {
+                'type': 'config-rule',
+                'role': 'something'
+            },
+            'filters': [{
+                'not': [
+                    {'type': 'event'}
+                ]
+            }]
+        }
+        self.patch(PullMode, 'run', lambda self: [True])
+        p = self.load_policy(
+            deepcopy(pdata), config={'dryrun': True})
+        results = p.run()
+        self.assertEqual(results, [True])
+        self.assertTrue(p.is_runnable())
+        self.assertEqual(pdata, p.data)
+
+    def test_boolean_not_event(self):
+        # event is bound to execution evaluation, when
+        # evaluating conditions for provisioning we
+        # strip any event filters.
+        pdata = {
+            'name': 'manga',
+            'resource': 'aws.ec2',
+            'conditions': [{
+                'or': [
+                    {'not': [
+                        {'type': 'event'}]}]}]}
+        p = self.load_policy(pdata)
+        p._trim_runtime_filters()
+        self.assertTrue(p.is_runnable())
+        self.assertFalse(p.conditions.filters)
+        self.assertEqual(p.data, pdata)
+
 
 class PolicyExecutionModeTest(BaseTest):