Repository: cloud-custodian/cloud-custodian

Issue:
filters - related resource filter requires 'value' when 'match-resource' is True with or filter
This broke validation of a previous policy when this wasn't the case

example
```yaml
policies:
  - name: related-sg-check
    resource: elb
    filters:
      - or:
        - type: security-group
          match-resource: True
          key: "tag:Application"
          op: not-equal
          operator: or
```

working in 0.8.32.0, failing in 0.8.33.0


Discussion:
I'll debug this later today if someone hasn't already gotten to it
fwiw, at the moment your examples fails for missing a name on the policy, and also for having an invalid policy top level attribute `type`, and per below not a great example of exhibiting an issue afaics..

i checked multiple versions and don't think is this related to a custodian version rev that i can see with that policy in isolation, its possible our validation on something trigger as invalid has gotten better. but afaics this isn't a schema policy, its just a filter implemented validation method.


```
(cloud-custodian) kapil@realms-chevaliers:~/projects/cloud-custodian$ custodian version
0.8.31.0
(cloud-custodian) kapil@realms-chevaliers:~/projects/cloud-custodian$ custodian validate -c foo.yml  -v
2019-02-13 19:44:37,448: custodian.cache:DEBUG Disabling cache
2019-02-13 19:44:37,448: custodian.commands:ERROR Configuration invalid: foo.yml
2019-02-13 19:44:37,448: custodian.commands:ERROR Policy: related-sg is invalid: Missing 'value' in value filter {'type': 'security-group', 'match-resource': True, 'key': 
'tag:Application', 'op': 'not-equal', 'operator': 'or'}
```


Looking at the lines raising the error is an even further back down through history

```
f426a60bda c7n/filters/core.py (Kapil Thangavelu 2016-09-05 13:34:45 -0500 371)         if 'value' not in self.data and 'value_from' not in self.data:
8087c58bd0 c7n/filters/core.py (Kapil Thangavelu 2018-05-17 06:00:15 -0400 372)             raise PolicyValidationError(
612b9a7600 janitor/filters.py  (Kapil Thangavelu 2015-09-28 05:42:45 -0700 373)                 "Missing 'value' in value filter %s" % self.data)   
```

also related filter validate's super call has changed in a while either

```
042fcd5644 (Kapil Thangavelu 2016-10-17 15:14:13 -0400  45)         return super(RelatedResourceFilter, self).validate()  
```

the only other possibility is that the actual policy had some or/and blocks nesting this filter, the handling of which was improved i believe between those two versions. but the policy was technically always invalid, so 🤷‍♂️ .. confirmed btw that was the difference, rewriting the example policy to use an `or` block.




Resolving patch:
diff --git a/c7n/filters/core.py b/c7n/filters/core.py
--- a/c7n/filters/core.py
+++ b/c7n/filters/core.py
@@ -369,6 +369,7 @@ class ValueFilter(Filter):
             'op': {'enum': list(OPERATORS.keys())}}}
 
     annotate = True
+    required_keys = set(('value', 'key'))
 
     def __init__(self, data, manager=None):
         super(ValueFilter, self).__init__(data, manager)
@@ -409,10 +410,12 @@ def validate(self):
         if self.data.get('value_type') == 'resource_count':
             return self._validate_resource_count()
 
-        if 'key' not in self.data:
+        if 'key' not in self.data and 'key' in self.required_keys:
             raise PolicyValidationError(
                 "Missing 'key' in value filter %s" % self.data)
-        if 'value' not in self.data and 'value_from' not in self.data:
+        if ('value' not in self.data and
+                'value_from' not in self.data and
+                'value' in self.required_keys):
             raise PolicyValidationError(
                 "Missing 'value' in value filter %s" % self.data)
         if 'op' in self.data:
diff --git a/c7n/filters/vpc.py b/c7n/filters/vpc.py
--- a/c7n/filters/vpc.py
+++ b/c7n/filters/vpc.py
@@ -20,7 +20,15 @@
 from .related import RelatedResourceFilter
 
 
-class SecurityGroupFilter(RelatedResourceFilter):
+class MatchResourceValidator(object):
+
+    def validate(self):
+        if self.data.get('match-resource'):
+            self.required_keys = set('key',)
+        return super(MatchResourceValidator, self).validate()
+
+
+class SecurityGroupFilter(MatchResourceValidator, RelatedResourceFilter):
     """Filter a resource by its associated security groups."""
     schema = type_schema(
         'security-group', rinherit=ValueFilter.schema,
@@ -32,7 +40,7 @@ class SecurityGroupFilter(RelatedResourceFilter):
     AnnotationKey = "matched-security-groups"
 
 
-class SubnetFilter(RelatedResourceFilter):
+class SubnetFilter(MatchResourceValidator, RelatedResourceFilter):
     """Filter a resource by its associated subnets."""
     schema = type_schema(
         'subnet', rinherit=ValueFilter.schema,
@@ -44,7 +52,7 @@ class SubnetFilter(RelatedResourceFilter):
     AnnotationKey = "matched-subnets"
 
 
-class VpcFilter(RelatedResourceFilter):
+class VpcFilter(MatchResourceValidator, RelatedResourceFilter):
     """Filter a resource by its associated vpc."""
     schema = type_schema(
         'vpc', rinherit=ValueFilter.schema,


Test patch:
diff --git a/tests/test_vpc.py b/tests/test_vpc.py
--- a/tests/test_vpc.py
+++ b/tests/test_vpc.py
@@ -1049,6 +1049,22 @@ def test_unused(self):
         resources = p.run()
         self.assertEqual(len(resources), 1)
 
+    def test_match_resource_validator(self):
+
+        try:
+            self.load_policy(
+                {'name': 'related-sg',
+                 'resource': 'elb',
+                 'filters': [
+                     {'type': 'security-group',
+                      'match-resource': True,
+                      'key': "tag:Application",
+                      'op': 'not-equal',
+                      'operator': 'or'}]},
+                validate=True)
+        except PolicyValidationError:
+            self.fail("should pass validation")
+
     @functional
     def test_only_ports(self):
         factory = self.replay_flight_data("test_security_group_only_ports")