Repository: cloud-custodian/cloud-custodian

Issue:
security-group ingress filters do not support both OnlyPorts and Cidr
Here's a sample policy that I've been fighting with for the past week with no success:

```
policies:
  - name: high-risk-security-groups-notify
    resource: security-group
    description: |
      Notifies if any rule from a security group that allows 0.0.0.0/0 
      ingress, not on ports 80 or 443, and notify the user who added the 
      violating rule.
    mode:
        type: cloudtrail
        role: arn:aws:iam::xxx:role/CloudCustodian
        events:
          - source: ec2.amazonaws.com
            event: AuthorizeSecurityGroupIngress
            ids: "requestParameters.groupId"
          - source: ec2.amazonaws.com
            event: AuthorizeSecurityGroupEgress
            ids: "requestParameters.groupId"
          - source: ec2.amazonaws.com
            event: RevokeSecurityGroupEgress
            ids: "requestParameters.groupId"
          - source: ec2.amazonaws.com
            event: RevokeSecurityGroupIngress
            ids: "requestParameters.groupId"
    filters:
        - type: ingress
          OnlyPorts: [80, 443]
          Cidr:
            value: "0.0.0.0/0"
    actions:
      - type: notify
        template: default.html
        priority_header: 1
        subject: "Open Security Group Rule Created-[custodian {{ account }} - {{ region }}]"
        violation_desc: "Security Group(s) Which Had Rules Open To The World:"
        action_desc: |
            "Actions Taken:  The Violating Security Group Rule Violates Our Company's 
            Cloud Policy.  Please Refer To The Cloud FAQ."
        to:
          - event-owner
        transport:
          type: sqs
          queue: https://sqs.us-east-1.amazonaws.com/xxx/cloud-custodian-mailer
          region: us-east-1
```

I've tried using `-and` when defining the filter conditions, but that didn't seem to make a difference.


Discussion:
I setup my sg like this and then ran your filter against the account.
![screen shot 2018-01-17 at 2 22 10 pm](https://user-images.githubusercontent.com/137058/35062325-f15fabbe-fb91-11e7-8e91-200da689d064.png)

Here is my simplified policy:
```yaml
policies:
  - name: github-1943
    resource: security-group
    filters:
      - type: ingress
        OnlyPorts: [80, 443]
        Cidr:
          value: "0.0.0.0/0"
```

That returns the security group.  Is that not what you are expecting or are you not getting the same result?

we were discussing this over gitter before it was filed. i think the crux was that it was applying as an `or` instead of an `and`, looking at the implementation its hardwired for `or`, for most multi attribute filters like this we typically specify a `match-operator` that allows specifying `or|and`
@jhnlsn It's close, but not quite the same.  Using your example if I were to create another rule for TCP/80 or TCP/443 for 0.0.0.0/0, I *should not* get a notification, but I do.  That's where @kapilt's comment comes into play.  I need an `and` condition for the `Cidr` and `OnlyPorts` condition, i.e. the Cidr is 0.0.0.0 AND it is for any port EXCEPT 80 or 443.

The reasoning is that I'm trying to ensure only web ports are open to the world, and no other port should be.
working on this now.. i have to admit, i find the default or behavior incredibly confusing and i'm wondering if we should change it to 'and' by default.

Resolving patch:
diff --git a/c7n/resources/vpc.py b/c7n/resources/vpc.py
--- a/c7n/resources/vpc.py
+++ b/c7n/resources/vpc.py
@@ -773,6 +773,7 @@ class SGPermission(Filter):
         'IpRanges', 'PrefixListIds'))
     filter_attrs = set(('Cidr', 'Ports', 'OnlyPorts', 'SelfReference'))
     attrs = perm_attrs.union(filter_attrs)
+    attrs.add('match-operator')
 
     def validate(self):
         delta = set(self.data.keys()).difference(self.attrs)
@@ -835,10 +836,16 @@ def process_cidrs(self, perm):
 
     def process_self_reference(self, perm, sg_id):
         found = None
+        ref_match = self.data.get('SelfReference')
+        if ref_match is not None:
+            found = False
         if 'UserIdGroupPairs' in perm and 'SelfReference' in self.data:
             self_reference = sg_id in [p['GroupId']
                                        for p in perm['UserIdGroupPairs']]
-            found = self_reference & self.data['SelfReference']
+            if ref_match is False and not self_reference:
+                found = True
+            if ref_match is True and self_reference:
+                found = True
         return found
 
     def expand_permissions(self, permissions):
@@ -866,34 +873,28 @@ def expand_permissions(self, permissions):
                     yield ep
 
     def __call__(self, resource):
-        def _accumulate(f, x):
-            '''
-            Accumulate an intermediate found value into the overall result.
-            '''
-            if x is not None:
-                f = (f is not None and x & f or x)
-            return f
-
         matched = []
         sg_id = resource['GroupId']
+        match_op = self.data.get('match-operator', 'and') == 'and' and all or any
 
         for perm in self.expand_permissions(resource[self.ip_permissions_key]):
-            found = None
-            for f in self.vfilters:
-                if f(perm):
-                    found = True
-                else:
-                    found = False
-                    break
-            if found is None or found:
-                found = _accumulate(found, self.process_ports(perm))
-            if found is None or found:
-                found = _accumulate(found, self.process_cidrs(perm))
-            if found is None or found:
-                found = _accumulate(found, self.process_self_reference(perm, sg_id))
-            if not found:
+            perm_matches = {}
+            for idx, f in enumerate(self.vfilters):
+                perm_matches[idx] = bool(f(perm))
+            perm_matches['ports'] = self.process_ports(perm)
+            perm_matches['cidrs'] = self.process_cidrs(perm)
+            perm_matches['self-refs'] = self.process_self_reference(perm, sg_id)
+
+            perm_match_values = list(filter(
+                lambda x: x is not None, perm_matches.values()))
+
+            # account for one python behavior any([]) == False, all([]) == True
+            if match_op == all and not perm_match_values:
                 continue
-            matched.append(perm)
+
+            match = match_op(perm_match_values)
+            if match:
+                matched.append(perm)
 
         if matched:
             resource['Matched%s' % self.ip_permissions_key] = matched
@@ -909,6 +910,7 @@ class IPPermission(SGPermission):
         # 'additionalProperties': True,
         'properties': {
             'type': {'enum': ['ingress']},
+            'match-operator': {'type': 'string', 'enum': ['or', 'and']},
             'Ports': {'type': 'array', 'items': {'type': 'integer'}},
             'SelfReference': {'type': 'boolean'}
         },
@@ -924,6 +926,7 @@ class IPPermissionEgress(SGPermission):
         # 'additionalProperties': True,
         'properties': {
             'type': {'enum': ['egress']},
+            'match-operator': {'type': 'string', 'enum': ['or', 'and']},
             'SelfReference': {'type': 'boolean'}
         },
         'required': ['type']}


Test patch:
diff --git a/tests/test_vpc.py b/tests/test_vpc.py
--- a/tests/test_vpc.py
+++ b/tests/test_vpc.py
@@ -671,7 +671,7 @@ def test_only_ports(self):
               u'UserIdGroupPairs': []}])
 
     @functional
-    def test_self_reference(self):
+    def test_self_reference_once(self):
         factory = self.replay_flight_data(
             'test_security_group_self_reference')
         client = factory().client('ec2')
@@ -737,12 +737,14 @@ def test_self_reference(self):
             'name': 'sg-find0',
             'resource': 'security-group',
             'filters': [
+                {'GroupName': 'sg1'},
                 {'type': 'ingress',
-                 'SelfReference': False},
-                {'GroupName': 'sg1'}]
+                 'match-operator': 'and',
+                 'Ports': [80],
+                 'SelfReference': False}]
             }, session_factory=factory)
         resources = p.run()
-        self.assertEqual(len(resources), 0)
+        self.assertEqual(len(resources), 1)
 
         p = self.load_policy({
             'name': 'sg-find1',
@@ -1061,6 +1063,7 @@ def test_self_reference_ingress_false_positives(self):
             'filters': [
                 {'type': 'ingress',
                  'Ports': [22],
+                 'match-operator': 'and',
                  'SelfReference': True}
                 ]})
         manager = p.get_resource_manager()
@@ -1072,6 +1075,7 @@ def test_self_reference_ingress_false_positives(self):
             'filters': [
                 {'type': 'ingress',
                  'Ports': [22],
+                 'match-operator': 'and',
                  'SelfReference': False}
                 ]})
         manager = p.get_resource_manager()
@@ -1083,6 +1087,7 @@ def test_self_reference_ingress_false_positives(self):
             'filters': [
                 {'type': 'ingress',
                  'Ports': [22],
+                 'match-operator': 'and',
                  'Cidr': {
                     'value': '0.0.0.0/0',
                     'op': 'eq',
@@ -1135,6 +1140,7 @@ def test_self_reference_ingress_false_positives(self):
             'filters': [
                 {'type': 'ingress',
                  'Ports': [22],
+                 'match-operator': 'and',
                  'Cidr': {
                     'value': '10.42.3.0/24',
                     'op': 'eq',