Repository: ansible/ansible

Issue:
ec2_group exception on CIDR with host bits set
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
ec2_group

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

(but code review suggests that later versions likely also susceptible)

##### CONFIGURATION
n/a

##### OS / ENVIRONMENT
n/a

##### SUMMARY
A playbook using an ```ec2_group``` module with a rule whose ```cidr_ip``` value has any host bits set will execute successfully at most once, but will fail to execute thereafter.

##### STEPS TO REPRODUCE
```yaml
- hosts: localhost
  tasks:
  - name: Test case
    ec2_group:
      description: Test SG
      name: Test SG
      region: us-east-1
      rules: [{proto: all, cidr_ip: '10.0.0.1/8'}]
      rules_egress: [{proto: all, cidr_ip: '0.0.0.0/0'}]
      vpc_id: FIXME
  connection: local
  become: false
  gather_facts: false
```
(replace ```vpc_id```, and maybe ```region```, as applicable)

##### EXPECTED RESULTS
_Preferably_ the rule's validation should fail entirely every time.

Alternatively, the CIDR could be sanitized by Ansible at run-time _before_ it compares the configured rules with the ones currently active in AWS.

##### ACTUAL RESULTS
Initial run executes fine; the rule is created on the security group, however the CIDR is masked so that only the network bits are set (ie 10.0.0.0/8). Thus what shows up in AWS is slightly different than what the user requested, but it works (at least for the first time).

Subsequent runs, however, will all fail with this error:
```An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_y6BGXK/ansible_module_ec2_group.py", line 478, in <module>
    main()
  File "/tmp/ansible_y6BGXK/ansible_module_ec2_group.py", line 373, in main
    group.authorize(rule['proto'], rule['from_port'], rule['to_port'], thisip, grantGroup)
  File "/usr/lib/python2.7/dist-packages/boto/ec2/securitygroup.py", line 203, in authorize
    dry_run=dry_run)
  File "/usr/lib/python2.7/dist-packages/boto/ec2/connection.py", line 3192, in authorize_security_group
    params, verb='POST')
  File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 1223, in get_status
    raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InvalidPermission.Duplicate</Code><Message>the specified rule "peer: 10.0.0.0/8, ALL, ALLOW" already exists</Message></Error></Errors><RequestID>963c0e1b-bcdb-4ab3-815a-89c7ce1b2204</RequestID></Response>

fatal: [localhost]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_name": "ec2_group"
    }, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_y6BGXK/ansible_module_ec2_group.py\", line 478, in <module>\n    main()\n  File \"/tmp/ansible_y6BGXK/ansible_module_ec2_group.py\", line 373, in main\n    group.authorize(rule['proto'], rule['from_port'], rule['to_port'], thisip, grantGroup)\n  File \"/usr/lib/python2.7/dist-packages/boto/ec2/securitygroup.py\", line 203, in authorize\n    dry_run=dry_run)\n  File \"/usr/lib/python2.7/dist-packages/boto/ec2/connection.py\", line 3192, in authorize_security_group\n    params, verb='POST')\n  File \"/usr/lib/python2.7/dist-packages/boto/connection.py\", line 1223, in get_status\n    raise self.ResponseError(response.status, response.reason, body)\nboto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Response><Errors><Error><Code>InvalidPermission.Duplicate</Code><Message>the specified rule \"peer: 10.0.0.0/8, ALL, ALLOW\" already exists</Message></Error></Errors><RequestID>963c0e1b-bcdb-4ab3-815a-89c7ce1b2204</RequestID></Response>\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE"
}
```

ec2_group exception on CIDR with host bits set
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
ec2_group

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

(but code review suggests that later versions likely also susceptible)

##### CONFIGURATION
n/a

##### OS / ENVIRONMENT
n/a

##### SUMMARY
A playbook using an ```ec2_group``` module with a rule whose ```cidr_ip``` value has any host bits set will execute successfully at most once, but will fail to execute thereafter.

##### STEPS TO REPRODUCE
```yaml
- hosts: localhost
  tasks:
  - name: Test case
    ec2_group:
      description: Test SG
      name: Test SG
      region: us-east-1
      rules: [{proto: all, cidr_ip: '10.0.0.1/8'}]
      rules_egress: [{proto: all, cidr_ip: '0.0.0.0/0'}]
      vpc_id: FIXME
  connection: local
  become: false
  gather_facts: false
```
(replace ```vpc_id```, and maybe ```region```, as applicable)

##### EXPECTED RESULTS
_Preferably_ the rule's validation should fail entirely every time.

Alternatively, the CIDR could be sanitized by Ansible at run-time _before_ it compares the configured rules with the ones currently active in AWS.

##### ACTUAL RESULTS
Initial run executes fine; the rule is created on the security group, however the CIDR is masked so that only the network bits are set (ie 10.0.0.0/8). Thus what shows up in AWS is slightly different than what the user requested, but it works (at least for the first time).

Subsequent runs, however, will all fail with this error:
```An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_y6BGXK/ansible_module_ec2_group.py", line 478, in <module>
    main()
  File "/tmp/ansible_y6BGXK/ansible_module_ec2_group.py", line 373, in main
    group.authorize(rule['proto'], rule['from_port'], rule['to_port'], thisip, grantGroup)
  File "/usr/lib/python2.7/dist-packages/boto/ec2/securitygroup.py", line 203, in authorize
    dry_run=dry_run)
  File "/usr/lib/python2.7/dist-packages/boto/ec2/connection.py", line 3192, in authorize_security_group
    params, verb='POST')
  File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 1223, in get_status
    raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InvalidPermission.Duplicate</Code><Message>the specified rule "peer: 10.0.0.0/8, ALL, ALLOW" already exists</Message></Error></Errors><RequestID>963c0e1b-bcdb-4ab3-815a-89c7ce1b2204</RequestID></Response>

fatal: [localhost]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_name": "ec2_group"
    }, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_y6BGXK/ansible_module_ec2_group.py\", line 478, in <module>\n    main()\n  File \"/tmp/ansible_y6BGXK/ansible_module_ec2_group.py\", line 373, in main\n    group.authorize(rule['proto'], rule['from_port'], rule['to_port'], thisip, grantGroup)\n  File \"/usr/lib/python2.7/dist-packages/boto/ec2/securitygroup.py\", line 203, in authorize\n    dry_run=dry_run)\n  File \"/usr/lib/python2.7/dist-packages/boto/ec2/connection.py\", line 3192, in authorize_security_group\n    params, verb='POST')\n  File \"/usr/lib/python2.7/dist-packages/boto/connection.py\", line 1223, in get_status\n    raise self.ResponseError(response.status, response.reason, body)\nboto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Response><Errors><Error><Code>InvalidPermission.Duplicate</Code><Message>the specified rule \"peer: 10.0.0.0/8, ALL, ALLOW\" already exists</Message></Error></Errors><RequestID>963c0e1b-bcdb-4ab3-815a-89c7ce1b2204</RequestID></Response>\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE"
}
```



Discussion:
While this is arguably a case of "garbage in = garbage out", I'm submitting this as a bug anyway because the resulting error users see can be _terribly_ difficult to figure out, especially if your non-canonical CIDR is visually similar to its canonical form.

Eg Consider the two private CIDRs, 172.29.104.192/27 and 172.29.104.199/27. The first is "canonical" in the sense that it satisfies all the masking rules, while the second one isn't -- it has some host bits set (which CIDRs aren't really supposed to have). But if you ignore the host bits, then both values refer to _exactly_ the same range of network addresses. They also happen to _visually_ appear as nearly identical, which can/will cause Ansible admins a lot of stress if they fall into this trap.

Worse (for us), Amazon's APIs auto-magically translate non-canonical CIDRs into canonical ones. That's why the initial run of the playbook works. But thereafter, the module implementation ends up comparing non-identical string values, erroneously considers them as being different, and then erroneously tries to re-create the entry (which still exists). That re-creation attempt is where the exception comes from.
Looks to me like any code fix here would add a dependency, eg on netaddr. I can prepare a patch based on that if desired, or even make one that only acts when netaddr is available. Not really sure what's best there overall.
@s-hertel @ryansb I believe `ec2_group` is an Ansible maintained module.
I think this falls under a user error category, and it seems harmless enough that I'd prefer not to add a dependency to handle it. What we can do is fail whenever the host bits on a network address are set by using the (standard library in Python 3.3+) `ipaddress` module like this:

```
>>> import ipaddress
>>> ipaddress.IPv4Network('1.2.3.0/24')
IPv4Network('1.2.3.0/24')
>>> ipaddress.IPv4Network('1.2.3.4/24')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ryansb/.pyenv/versions/3.6.1/lib/python3.6/ipaddress.py", line 1536, in __init__
    raise ValueError('%s has host bits set' % self)
ValueError: 1.2.3.4/24 has host bits set
1.2.3.4/24 has host bits set
>>>
```

Python 2 users would have to install [ipaddress](https://pypi.python.org/pypi/ipaddress) backport via pypi, but that could be optional. The upside here is that it wouldn't add a required dependency, the downside is an increase in inconsistency between Py2 and 3. We could print a warning, or just document the optional requirement. 

Thoughts @s-hertel @johnboy2 ?
I'm okay with a check being made only when possible, and I'm also okay with it being a warning rather than an outright failure. As to whether to warn or document, I think both are appropriate.
- Documenting covers the worst case when there is no available way to do the check at run-time, but
- Having Ansible warn at runtime is better, if it is possible.

RE "if possible", note that the "netaddr" module is more likely to already be available on existing Ansible/py2 systems, since Ansible's "ipaddr" Jinja filter already depends on it.

So here's what I'm thinking now:
1. Introduce a "normalize_cidr" (?) function, using either ipaddress or netaddr as available (preferring ipaddress), but falling back on "pass-through" if neither are available.
1. Normalize the provided CIDR before sending to AWS, and warn if the normalized version doesn't match the input version.

The major objection in my own mind is doing step 1 within ec2_group, since that seems more like "common code".
cc @adq
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
While this is arguably a case of "garbage in = garbage out", I'm submitting this as a bug anyway because the resulting error users see can be _terribly_ difficult to figure out, especially if your non-canonical CIDR is visually similar to its canonical form.

Eg Consider the two private CIDRs, 172.29.104.192/27 and 172.29.104.199/27. The first is "canonical" in the sense that it satisfies all the masking rules, while the second one isn't -- it has some host bits set (which CIDRs aren't really supposed to have). But if you ignore the host bits, then both values refer to _exactly_ the same range of network addresses. They also happen to _visually_ appear as nearly identical, which can/will cause Ansible admins a lot of stress if they fall into this trap.

Worse (for us), Amazon's APIs auto-magically translate non-canonical CIDRs into canonical ones. That's why the initial run of the playbook works. But thereafter, the module implementation ends up comparing non-identical string values, erroneously considers them as being different, and then erroneously tries to re-create the entry (which still exists). That re-creation attempt is where the exception comes from.
Looks to me like any code fix here would add a dependency, eg on netaddr. I can prepare a patch based on that if desired, or even make one that only acts when netaddr is available. Not really sure what's best there overall.
@s-hertel @ryansb I believe `ec2_group` is an Ansible maintained module.
I think this falls under a user error category, and it seems harmless enough that I'd prefer not to add a dependency to handle it. What we can do is fail whenever the host bits on a network address are set by using the (standard library in Python 3.3+) `ipaddress` module like this:

```
>>> import ipaddress
>>> ipaddress.IPv4Network('1.2.3.0/24')
IPv4Network('1.2.3.0/24')
>>> ipaddress.IPv4Network('1.2.3.4/24')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ryansb/.pyenv/versions/3.6.1/lib/python3.6/ipaddress.py", line 1536, in __init__
    raise ValueError('%s has host bits set' % self)
ValueError: 1.2.3.4/24 has host bits set
1.2.3.4/24 has host bits set
>>>
```

Python 2 users would have to install [ipaddress](https://pypi.python.org/pypi/ipaddress) backport via pypi, but that could be optional. The upside here is that it wouldn't add a required dependency, the downside is an increase in inconsistency between Py2 and 3. We could print a warning, or just document the optional requirement. 

Thoughts @s-hertel @johnboy2 ?
I'm okay with a check being made only when possible, and I'm also okay with it being a warning rather than an outright failure. As to whether to warn or document, I think both are appropriate.
- Documenting covers the worst case when there is no available way to do the check at run-time, but
- Having Ansible warn at runtime is better, if it is possible.

RE "if possible", note that the "netaddr" module is more likely to already be available on existing Ansible/py2 systems, since Ansible's "ipaddr" Jinja filter already depends on it.

So here's what I'm thinking now:
1. Introduce a "normalize_cidr" (?) function, using either ipaddress or netaddr as available (preferring ipaddress), but falling back on "pass-through" if neither are available.
1. Normalize the provided CIDR before sending to AWS, and warn if the normalized version doesn't match the input version.

The major objection in my own mind is doing step 1 within ec2_group, since that seems more like "common code".
cc @adq
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/module_utils/network/common/utils.py b/lib/ansible/module_utils/network/common/utils.py
--- a/lib/ansible/module_utils/network/common/utils.py
+++ b/lib/ansible/module_utils/network/common/utils.py
@@ -31,8 +31,11 @@
 import socket
 
 from itertools import chain
+from struct import pack
+from socket import inet_aton, inet_ntoa
 
 from ansible.module_utils.six import iteritems, string_types
+from ansible.module_utils.six.moves import zip
 from ansible.module_utils.basic import AnsibleFallbackNotFound
 
 try:
@@ -45,6 +48,7 @@
 
 OPERATORS = frozenset(['ge', 'gt', 'eq', 'neq', 'lt', 'le'])
 ALIASES = frozenset([('min', 'ge'), ('max', 'le'), ('exactly', 'eq'), ('neq', 'ne')])
+VALID_MASKS = [2**8 - 2**i for i in range(0, 9)]
 
 
 def to_list(val):
@@ -426,3 +430,106 @@ def contains_vars(self, data):
                 if marker in data:
                     return True
         return False
+
+
+def is_netmask(val):
+    parts = str(val).split('.')
+    if not len(parts) == 4:
+        return False
+    for part in parts:
+        try:
+            if int(part) not in VALID_MASKS:
+                raise ValueError
+        except ValueError:
+            return False
+    return True
+
+
+def is_masklen(val):
+    try:
+        return 0 <= int(val) <= 32
+    except ValueError:
+        return False
+
+
+def to_bits(val):
+    """ converts a netmask to bits """
+    bits = ''
+    for octet in val.split('.'):
+        bits += bin(int(octet))[2:].zfill(8)
+    return str
+
+
+def to_netmask(val):
+    """ converts a masklen to a netmask """
+    if not is_masklen(val):
+        raise ValueError('invalid value for masklen')
+
+    bits = 0
+    for i in range(32 - int(val), 32):
+        bits |= (1 << i)
+
+    return inet_ntoa(pack('>I', bits))
+
+
+def to_masklen(val):
+    """ converts a netmask to a masklen """
+    if not is_netmask(val):
+        raise ValueError('invalid value for netmask: %s' % val)
+
+    bits = list()
+    for x in val.split('.'):
+        octet = bin(int(x)).count('1')
+        bits.append(octet)
+
+    return sum(bits)
+
+
+def to_subnet(addr, mask, dotted_notation=False):
+    """ coverts an addr / mask pair to a subnet in cidr notation """
+    try:
+        if not is_masklen(mask):
+            raise ValueError
+        cidr = int(mask)
+        mask = to_netmask(mask)
+    except ValueError:
+        cidr = to_masklen(mask)
+
+    addr = addr.split('.')
+    mask = mask.split('.')
+
+    network = list()
+    for s_addr, s_mask in zip(addr, mask):
+        network.append(str(int(s_addr) & int(s_mask)))
+
+    if dotted_notation:
+        return '%s %s' % ('.'.join(network), to_netmask(cidr))
+    return '%s/%s' % ('.'.join(network), cidr)
+
+
+def to_ipv6_network(addr):
+    """ IPv6 addresses are eight groupings. The first three groupings (48 bits) comprise the network address. """
+
+    # Split by :: to identify omitted zeros
+    ipv6_prefix = addr.split('::')[0]
+
+    # Get the first three groups, or as many as are found + ::
+    found_groups = []
+    for group in ipv6_prefix.split(':'):
+        found_groups.append(group)
+        if len(found_groups) == 3:
+            break
+    if len(found_groups) < 3:
+        found_groups.append('::')
+
+    # Concatenate network address parts
+    network_addr = ''
+    for group in found_groups:
+        if group != '::':
+            network_addr += str(group)
+        network_addr += str(':')
+
+    # Ensure network address ends with ::
+    if not network_addr.endswith('::'):
+        network_addr += str(':')
+    return network_addr
diff --git a/lib/ansible/modules/cloud/amazon/ec2_group.py b/lib/ansible/modules/cloud/amazon/ec2_group.py
--- a/lib/ansible/modules/cloud/amazon/ec2_group.py
+++ b/lib/ansible/modules/cloud/amazon/ec2_group.py
@@ -289,6 +289,7 @@
 from ansible.module_utils.ec2 import HAS_BOTO3
 from ansible.module_utils.ec2 import boto3_tag_list_to_ansible_dict, ansible_dict_to_boto3_tag_list, compare_aws_tags
 from ansible.module_utils.ec2 import AWSRetry
+from ansible.module_utils.network.common.utils import to_ipv6_network, to_subnet
 import traceback
 
 try:
@@ -521,7 +522,22 @@ def update_rules_description(module, client, rule_type, group_id, ip_permissions
 def authorize_ip(type, changed, client, group, groupRules,
                  ip, ip_permission, module, rule, ethertype):
     # If rule already exists, don't later delete it
-    for thisip in ip:
+    for this_ip in ip:
+
+        split_addr = this_ip.split('/')
+        if len(split_addr) == 2:
+            # this_ip is a IPv4 or IPv6 CIDR that may or may not have host bits set
+            # Get the network bits.
+            try:
+                thisip = to_subnet(split_addr[0], split_addr[1])
+            except ValueError:
+                thisip = to_ipv6_network(split_addr[0]) + "/" + split_addr[1]
+            if thisip != this_ip:
+                module.warn("One of your CIDR addresses ({0}) has host bits set. To get rid of this warning, "
+                            "check the network mask and make sure that only network bits are set: {1}.".format(this_ip, thisip))
+        else:
+            thisip = this_ip
+
         rule_id = make_rule_key(type, rule, group['GroupId'], thisip)
         if rule_id in groupRules:
 


Test patch:
diff --git a/test/integration/targets/ec2_group/tasks/main.yml b/test/integration/targets/ec2_group/tasks/main.yml
--- a/test/integration/targets/ec2_group/tasks/main.yml
+++ b/test/integration/targets/ec2_group/tasks/main.yml
@@ -435,6 +435,112 @@
           - 'result.changed'
           - 'result.group_id.startswith("sg-")'
 
+    # ============================================================
+
+    - name: test adding a rule with a IPv4 CIDR with host bits set (expected changed=true)
+      ec2_group:
+        name: '{{ec2_group_name}}'
+        description: '{{ec2_group_description}}'
+        ec2_region: '{{ec2_region}}'
+        ec2_access_key: '{{ec2_access_key}}'
+        ec2_secret_key: '{{ec2_secret_key}}'
+        security_token: '{{security_token}}'
+        state: present
+        # set purge_rules to false so we don't get a false positive from previously added rules
+        purge_rules: false
+        rules:
+        - proto: "tcp"
+          ports:
+            - 8195
+          cidr_ip: 10.0.0.1/8
+      register: result
+
+    - name: assert state=present (expected changed=true)
+      assert:
+        that:
+          - 'result.changed'
+          - 'result.group_id.startswith("sg-")'
+
+    # ============================================================
+
+    - name: test adding the same rule with a IPv4 CIDR with host bits set (expected changed=false and a warning)
+      ec2_group:
+        name: '{{ec2_group_name}}'
+        description: '{{ec2_group_description}}'
+        ec2_region: '{{ec2_region}}'
+        ec2_access_key: '{{ec2_access_key}}'
+        ec2_secret_key: '{{ec2_secret_key}}'
+        security_token: '{{security_token}}'
+        state: present
+        # set purge_rules to false so we don't get a false positive from previously added rules
+        purge_rules: false
+        rules:
+        - proto: "tcp"
+          ports:
+            - 8195
+          cidr_ip: 10.0.0.1/8
+      register: result
+
+    - name: assert state=present (expected changed=false and a warning)
+      assert:
+        that:
+          # No way to assert for warnings?
+          - 'not result.changed'
+          - 'result.group_id.startswith("sg-")'
+
+    # ============================================================
+
+    - name: test adding a rule with a IPv6 CIDR with host bits set (expected changed=true)
+      ec2_group:
+        name: '{{ec2_group_name}}'
+        description: '{{ec2_group_description}}'
+        ec2_region: '{{ec2_region}}'
+        ec2_access_key: '{{ec2_access_key}}'
+        ec2_secret_key: '{{ec2_secret_key}}'
+        security_token: '{{security_token}}'
+        state: present
+        # set purge_rules to false so we don't get a false positive from previously added rules
+        purge_rules: false
+        rules:
+        - proto: "tcp"
+          ports:
+            - 8196
+          cidr_ipv6: '2001:db00::1/24'
+      register: result
+
+    - name: assert state=present (expected changed=true)
+      assert:
+        that:
+          - 'result.changed'
+          - 'result.group_id.startswith("sg-")'
+
+    # ============================================================
+
+    - name: test adding a rule again with a IPv6 CIDR with host bits set (expected changed=false and a warning)
+      ec2_group:
+        name: '{{ec2_group_name}}'
+        description: '{{ec2_group_description}}'
+        ec2_region: '{{ec2_region}}'
+        ec2_access_key: '{{ec2_access_key}}'
+        ec2_secret_key: '{{ec2_secret_key}}'
+        security_token: '{{security_token}}'
+        state: present
+        # set purge_rules to false so we don't get a false positive from previously added rules
+        purge_rules: false
+        rules:
+        - proto: "tcp"
+          ports:
+            - 8196
+          cidr_ipv6: '2001:db00::1/24'
+      register: result
+
+    - name: assert state=present (expected changed=false and a warning)
+      assert:
+        that:
+          # No way to assert for warnings?
+          - 'not result.changed'
+          - 'result.group_id.startswith("sg-")'
+
     # ============================================================
     - name: test state=absent (expected changed=true)
       ec2_group:
diff --git a/test/units/module_utils/network/common/test_utils.py b/test/units/module_utils/network/common/test_utils.py
--- a/test/units/module_utils/network/common/test_utils.py
+++ b/test/units/module_utils/network/common/test_utils.py
@@ -18,134 +18,189 @@
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
 
 # Make coding more python3-ish
-from __future__ import (absolute_import, division)
+from __future__ import absolute_import, division, print_function
 __metaclass__ = type
 
-from ansible.compat.tests import unittest
+import pytest
 
 from ansible.module_utils.network.common.utils import to_list, sort_list
 from ansible.module_utils.network.common.utils import dict_diff, dict_merge
 from ansible.module_utils.network.common.utils import conditional, Template
+from ansible.module_utils.network.common.utils import to_masklen, to_netmask, to_subnet, to_ipv6_network
+from ansible.module_utils.network.common.utils import is_masklen, is_netmask
 
 
-class TestModuleUtilsNetworkCommon(unittest.TestCase):
-
-    def test_to_list(self):
-        for scalar in ('string', 1, True, False, None):
-            self.assertTrue(isinstance(to_list(scalar), list))
-
-        for container in ([1, 2, 3], {'one': 1}):
-            self.assertTrue(isinstance(to_list(container), list))
-
-        test_list = [1, 2, 3]
-        self.assertNotEqual(id(test_list), id(to_list(test_list)))
-
-    def test_sort(self):
-        data = [3, 1, 2]
-        self.assertEqual([1, 2, 3], sort_list(data))
-
-        string_data = '123'
-        self.assertEqual(string_data, sort_list(string_data))
-
-    def test_dict_diff(self):
-        base = dict(obj2=dict(), b1=True, b2=False, b3=False,
-                    one=1, two=2, three=3, obj1=dict(key1=1, key2=2),
-                    l1=[1, 3], l2=[1, 2, 3], l4=[4],
-                    nested=dict(n1=dict(n2=2)))
-
-        other = dict(b1=True, b2=False, b3=True, b4=True,
-                     one=1, three=4, four=4, obj1=dict(key1=2),
-                     l1=[2, 1], l2=[3, 2, 1], l3=[1],
-                     nested=dict(n1=dict(n2=2, n3=3)))
-
-        result = dict_diff(base, other)
-
-        # string assertions
-        self.assertNotIn('one', result)
-        self.assertNotIn('two', result)
-        self.assertEqual(result['three'], 4)
-        self.assertEqual(result['four'], 4)
-
-        # dict assertions
-        self.assertIn('obj1', result)
-        self.assertIn('key1', result['obj1'])
-        self.assertNotIn('key2', result['obj1'])
-
-        # list assertions
-        self.assertEqual(result['l1'], [2, 1])
-        self.assertNotIn('l2', result)
-        self.assertEqual(result['l3'], [1])
-        self.assertNotIn('l4', result)
-
-        # nested assertions
-        self.assertIn('obj1', result)
-        self.assertEqual(result['obj1']['key1'], 2)
-        self.assertNotIn('key2', result['obj1'])
-
-        # bool assertions
-        self.assertNotIn('b1', result)
-        self.assertNotIn('b2', result)
-        self.assertTrue(result['b3'])
-        self.assertTrue(result['b4'])
-
-    def test_dict_merge(self):
-        base = dict(obj2=dict(), b1=True, b2=False, b3=False,
-                    one=1, two=2, three=3, obj1=dict(key1=1, key2=2),
-                    l1=[1, 3], l2=[1, 2, 3], l4=[4],
-                    nested=dict(n1=dict(n2=2)))
-
-        other = dict(b1=True, b2=False, b3=True, b4=True,
-                     one=1, three=4, four=4, obj1=dict(key1=2),
-                     l1=[2, 1], l2=[3, 2, 1], l3=[1],
-                     nested=dict(n1=dict(n2=2, n3=3)))
-
-        result = dict_merge(base, other)
-
-        # string assertions
-        self.assertIn('one', result)
-        self.assertIn('two', result)
-        self.assertEqual(result['three'], 4)
-        self.assertEqual(result['four'], 4)
-
-        # dict assertions
-        self.assertIn('obj1', result)
-        self.assertIn('key1', result['obj1'])
-        self.assertIn('key2', result['obj1'])
-
-        # list assertions
-        self.assertEqual(result['l1'], [1, 2, 3])
-        self.assertIn('l2', result)
-        self.assertEqual(result['l3'], [1])
-        self.assertIn('l4', result)
-
-        # nested assertions
-        self.assertIn('obj1', result)
-        self.assertEqual(result['obj1']['key1'], 2)
-        self.assertIn('key2', result['obj1'])
-
-        # bool assertions
-        self.assertIn('b1', result)
-        self.assertIn('b2', result)
-        self.assertTrue(result['b3'])
-        self.assertTrue(result['b4'])
-
-    def test_conditional(self):
-        self.assertTrue(conditional(10, 10))
-        self.assertTrue(conditional('10', '10'))
-        self.assertTrue(conditional('foo', 'foo'))
-        self.assertTrue(conditional(True, True))
-        self.assertTrue(conditional(False, False))
-        self.assertTrue(conditional(None, None))
-        self.assertTrue(conditional("ge(1)", 1))
-        self.assertTrue(conditional("gt(1)", 2))
-        self.assertTrue(conditional("le(2)", 2))
-        self.assertTrue(conditional("lt(3)", 2))
-        self.assertTrue(conditional("eq(1)", 1))
-        self.assertTrue(conditional("neq(0)", 1))
-        self.assertTrue(conditional("min(1)", 1))
-        self.assertTrue(conditional("max(1)", 1))
-        self.assertTrue(conditional("exactly(1)", 1))
-
-    def test_template(self):
-        tmpl = Template()
-        self.assertEqual('foo', tmpl('{{ test }}', {'test': 'foo'}))
+def test_to_list():
+    for scalar in ('string', 1, True, False, None):
+        assert isinstance(to_list(scalar), list)
+
+    for container in ([1, 2, 3], {'one': 1}):
+        assert isinstance(to_list(container), list)
+
+    test_list = [1, 2, 3]
+    assert id(test_list) != id(to_list(test_list))
+
+
+def test_sort():
+    data = [3, 1, 2]
+    assert [1, 2, 3] == sort_list(data)
+
+    string_data = '123'
+    assert string_data == sort_list(string_data)
+
+
+def test_dict_diff():
+    base = dict(obj2=dict(), b1=True, b2=False, b3=False,
+                one=1, two=2, three=3, obj1=dict(key1=1, key2=2),
+                l1=[1, 3], l2=[1, 2, 3], l4=[4],
+                nested=dict(n1=dict(n2=2)))
+
+    other = dict(b1=True, b2=False, b3=True, b4=True,
+                 one=1, three=4, four=4, obj1=dict(key1=2),
+                 l1=[2, 1], l2=[3, 2, 1], l3=[1],
+                 nested=dict(n1=dict(n2=2, n3=3)))
+
+    result = dict_diff(base, other)
+
+    # string assertions
+    assert 'one' not in result
+    assert 'two' not in result
+    assert result['three'] == 4
+    assert result['four'] == 4
+
+    # dict assertions
+    assert 'obj1' in result
+    assert 'key1' in result['obj1']
+    assert 'key2' not in result['obj1']
+
+    # list assertions
+    assert result['l1'] == [2, 1]
+    assert 'l2' not in result
+    assert result['l3'] == [1]
+    assert 'l4' not in result
+
+    # nested assertions
+    assert 'obj1' in result
+    assert result['obj1']['key1'] == 2
+    assert 'key2' not in result['obj1']
+
+    # bool assertions
+    assert 'b1' not in result
+    assert 'b2' not in result
+    assert result['b3']
+    assert result['b4']
+
+
+def test_dict_merge():
+    base = dict(obj2=dict(), b1=True, b2=False, b3=False,
+                one=1, two=2, three=3, obj1=dict(key1=1, key2=2),
+                l1=[1, 3], l2=[1, 2, 3], l4=[4],
+                nested=dict(n1=dict(n2=2)))
+
+    other = dict(b1=True, b2=False, b3=True, b4=True,
+                 one=1, three=4, four=4, obj1=dict(key1=2),
+                 l1=[2, 1], l2=[3, 2, 1], l3=[1],
+                 nested=dict(n1=dict(n2=2, n3=3)))
+
+    result = dict_merge(base, other)
+
+    # string assertions
+    assert 'one' in result
+    assert 'two' in result
+    assert result['three'] == 4
+    assert result['four'] == 4
+
+    # dict assertions
+    assert 'obj1' in result
+    assert 'key1' in result['obj1']
+    assert 'key2' in result['obj1']
+
+    # list assertions
+    assert result['l1'] == [1, 2, 3]
+    assert 'l2' in result
+    assert result['l3'] == [1]
+    assert 'l4' in result
+
+    # nested assertions
+    assert 'obj1' in result
+    assert result['obj1']['key1'] == 2
+    assert 'key2' in result['obj1']
+
+    # bool assertions
+    assert 'b1' in result
+    assert 'b2' in result
+    assert result['b3']
+    assert result['b4']
+
+
+def test_conditional():
+    assert conditional(10, 10)
+    assert conditional('10', '10')
+    assert conditional('foo', 'foo')
+    assert conditional(True, True)
+    assert conditional(False, False)
+    assert conditional(None, None)
+    assert conditional("ge(1)", 1)
+    assert conditional("gt(1)", 2)
+    assert conditional("le(2)", 2)
+    assert conditional("lt(3)", 2)
+    assert conditional("eq(1)", 1)
+    assert conditional("neq(0)", 1)
+    assert conditional("min(1)", 1)
+    assert conditional("max(1)", 1)
+    assert conditional("exactly(1)", 1)
+
+
+def test_template():
+    tmpl = Template()
+    assert 'foo' == tmpl('{{ test }}', {'test': 'foo'})
+
+
+def test_to_masklen():
+    assert 24 == to_masklen('255.255.255.0')
+
+
+def test_to_masklen_invalid():
+    with pytest.raises(ValueError):
+        to_masklen('255')
+
+
+def test_to_netmask():
+    assert '255.0.0.0' == to_netmask(8)
+    assert '255.0.0.0' == to_netmask('8')
+
+
+def test_to_netmask_invalid():
+    with pytest.raises(ValueError):
+        to_netmask(128)
+
+
+def test_to_subnet():
+    result = to_subnet('192.168.1.1', 24)
+    assert '192.168.1.0/24' == result
+
+    result = to_subnet('192.168.1.1', 24, dotted_notation=True)
+    assert '192.168.1.0 255.255.255.0' == result
+
+
+def test_to_subnet_invalid():
+    with pytest.raises(ValueError):
+        to_subnet('foo', 'bar')
+
+
+def test_is_masklen():
+    assert is_masklen(32)
+    assert not is_masklen(33)
+    assert not is_masklen('foo')
+
+
+def test_is_netmask():
+    assert is_netmask('255.255.255.255')
+    assert not is_netmask(24)
+    assert not is_netmask('foo')
+
+
+def test_to_ipv6_network():
+    assert '2001:db8::' == to_ipv6_network('2001:db8::')
+    assert '2001:0db8:85a3::' == to_ipv6_network('2001:0db8:85a3:0000:0000:8a2e:0370:7334')
+    assert '2001:0db8:85a3::' == to_ipv6_network('2001:0db8:85a3:0:0:8a2e:0370:7334')