Repository: ansible/ansible

Issue:
Alter Security Groups on EC2 Instances
_From @dbrgn on December 5, 2014 15:47_

##### Issue Type:

Feature Idea
##### Component Name:

ec2 module
##### Ansible Version:

ansible 1.8.1
##### Environment:

N/A
##### Summary:

I did not find a way to change EC2 Security Groups on an existing instance. It is possible to define the groups when creating a new instance with the "ec2" module, but nto to alter, add or remove groups on an existing instance.

Rationale: I create an instance with a limited set of security groups, so that the software cannot connect to the server yet during provisioning. Then I provision the host, and at the end I want to add a security group so that the host can start connecting to the server and process work.
##### Steps To Reproduce:

N/A
##### Expected Results:

Being able to:
- Add security groups to an existing EC2 instance
- Remove security groups from an existing EC2 instance
- Overwrite the security groups on an existing EC2 instance
##### Actual Results:

N/A


_Copied from original issue: ansible/ansible-modules-core#454_
Alter Security Groups on EC2 Instances
_From @dbrgn on December 5, 2014 15:47_

##### Issue Type:

Feature Idea
##### Component Name:

ec2 module
##### Ansible Version:

ansible 1.8.1
##### Environment:

N/A
##### Summary:

I did not find a way to change EC2 Security Groups on an existing instance. It is possible to define the groups when creating a new instance with the "ec2" module, but nto to alter, add or remove groups on an existing instance.

Rationale: I create an instance with a limited set of security groups, so that the software cannot connect to the server yet during provisioning. Then I provision the host, and at the end I want to add a security group so that the host can start connecting to the server and process work.
##### Steps To Reproduce:

N/A
##### Expected Results:

Being able to:
- Add security groups to an existing EC2 instance
- Remove security groups from an existing EC2 instance
- Overwrite the security groups on an existing EC2 instance
##### Actual Results:

N/A


_Copied from original issue: ansible/ansible-modules-core#454_


Discussion:
_From @pguillebert on December 8, 2014 15:2_

See also, for context : https://groups.google.com/forum/#!topic/ansible-project/1LVR-VNetOg

I'm blocked by this issue as well... Has anybody started something on this, or should I start coding ? I have no experience on writing Ansible plugins... Should I start another plugin just for association/dissociation of SG on instance or should modify ec2 (but it's a fairly complex module already) ?

_From @dbrgn on December 8, 2014 15:54_

In my opinion, when passing in an `instance_id` to the `ec2` module, it should update the configuration of that instance according to the parameters. That would go along well with the concept of idempotent plugins.

_From @lamielle on January 16, 2015 15:48_

I just ran into this problem as well.  As @dbrgn points out, not having this certainly breaks the 'idempotent' goals of Ansible.  Updating the config and re-running the playbook should make the state of the world match what is requested.  Re-running again should then not make any changes.  Let me know if there's anything I can do to help out with implementing this request.

_From @lamielle on January 16, 2015 15:51_

I jumped in too soon on this one.  Per the EC2 docs "After you launch an instance in EC2-Classic, you can't change its security groups."  In my case, my instances are in Classic.  Additionally, "After you launch an instance in a VPC, you can change its security groups."  Yet another reason to migrate to VPC.  Reference: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html

_From @dbrgn on January 16, 2015 15:55_

My instances are in a VPC and IIRC it still did not update the security groups.

_From @bahner on March 23, 2015 23:20_

I second this. The module should know how to tell EC2 classic from VPC. This is silly.

_From @bpesics on March 25, 2015 16:42_

bug

_From @klj613 on May 16, 2015 22:20_

just came across this (my instances are in a VPC)

_From @davidneudorfer on May 28, 2015 20:34_

Same issue here. My instances are inside a VPC.

_From @tcr-ableton on June 8, 2015 12:15_

Same issue here. Instances in a VPC too.

_From @SteveMcD on August 8, 2015 0:41_

+1.  Same issue with instances in a VPC

_From @rodrickbrown on August 13, 2015 21:51_

+1 same issue here also inside a VPC 

_From @seanlutner on August 17, 2015 14:43_

+1 also having the issue with instances in a VPC

_From @jamesongithub on August 24, 2015 21:21_

just hit this
ansible v1.9.2
is anyone looking at this?
ugh only P3

_From @peh on August 25, 2015 6:36_

+1

_From @macolo on September 1, 2015 11:58_

+1

_From @cdesales on September 1, 2015 11:58_

+1 same issue

_From @sfontana on September 1, 2015 11:59_

+1

_From @GergelyKalmar on September 2, 2015 7:5_

+1

_From @hyperized on September 16, 2015 11:45_

+1

_From @Constantin07 on September 18, 2015 7:8_

Experiencing the same issue.
+1

_From @kylejohnson on October 1, 2015 4:2_

+1, been dealing with this for a while.

_From @viesti on October 11, 2015 19:32_

+1 This would be neat to have. In the meanwhile, I whipped together a small module to ensure that given groups exist on a node: https://gist.github.com/viesti/786f5d93621985556b78

_From @dverbeek84 on October 25, 2015 19:14_

+1

_From @bdgscotland on November 29, 2015 8:46_

+1

_From @georgeliaw on November 30, 2015 21:9_

+1

_From @Constantin07 on December 9, 2015 17:46_

I'm using AWS CLI equivalent meantime to alter SG association:

aws ec2 modify-instance-attribute --instance-id "{{ instance_id }}" --groups "{{ sg_id }}"

_From @fabfuel on February 9, 2016 10:41_

+1

_From @sdomula on February 15, 2016 12:3_

+1

_From @vre on February 17, 2016 16:9_

Actually you can modify the security groups of existing instance with ec2_eni.. but it bit painful:
1) you need elastic network interface id (eni_id) of the network interface of the instance
2) you need to fish out the security group ids as ec2_eni does not (yet?) process names

```
- ec2_eni:
    eni_id: eni-c063daa6
    state: present
    security_groups: sg-772eae13, sg-961e4df2, sg-991c4ffd
```

One way to avoid the problem altogether is to make sure that every instance has its own security group (or group of instances use same security group) and then modify that security group - your Ansible script should be your documentation anyways.. Life is easier when you assign instance purpose specific a IAM roles and in similar way a security group matching the role.

_From @darrengruber on February 17, 2016 19:48_

+1

_From @jdubs on February 18, 2016 20:34_

+1

_From @CptnKirk on February 20, 2016 4:2_

+1

_From @MichaelDrogalis on March 9, 2016 16:12_

+1

_From @sendai-ono on March 12, 2016 19:25_

> aws ec2 modify-instance-attribute --instance-id "{{ instance_id }}" --groups "{{ sg_id }}"

I'm also using a simple and dirty conditional to respect idempotency

```
local_action:
  module: >
    shell
      aws ec2 modify-instance-attribute --instance-id "{{ ec2_id }}"
      --groups "{{ec2_security_group_ids}}" "{{newgroup.group_id}}"
   when: newgroup.group_id not in ec2_security_group_ids
```

+1

_From @pguillebert on December 8, 2014 15:2_

See also, for context : https://groups.google.com/forum/#!topic/ansible-project/1LVR-VNetOg

I'm blocked by this issue as well... Has anybody started something on this, or should I start coding ? I have no experience on writing Ansible plugins... Should I start another plugin just for association/dissociation of SG on instance or should modify ec2 (but it's a fairly complex module already) ?

_From @dbrgn on December 8, 2014 15:54_

In my opinion, when passing in an `instance_id` to the `ec2` module, it should update the configuration of that instance according to the parameters. That would go along well with the concept of idempotent plugins.

_From @lamielle on January 16, 2015 15:48_

I just ran into this problem as well.  As @dbrgn points out, not having this certainly breaks the 'idempotent' goals of Ansible.  Updating the config and re-running the playbook should make the state of the world match what is requested.  Re-running again should then not make any changes.  Let me know if there's anything I can do to help out with implementing this request.

_From @lamielle on January 16, 2015 15:51_

I jumped in too soon on this one.  Per the EC2 docs "After you launch an instance in EC2-Classic, you can't change its security groups."  In my case, my instances are in Classic.  Additionally, "After you launch an instance in a VPC, you can change its security groups."  Yet another reason to migrate to VPC.  Reference: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html

_From @dbrgn on January 16, 2015 15:55_

My instances are in a VPC and IIRC it still did not update the security groups.

_From @bahner on March 23, 2015 23:20_

I second this. The module should know how to tell EC2 classic from VPC. This is silly.

_From @bpesics on March 25, 2015 16:42_

bug

_From @klj613 on May 16, 2015 22:20_

just came across this (my instances are in a VPC)

_From @davidneudorfer on May 28, 2015 20:34_

Same issue here. My instances are inside a VPC.

_From @tcr-ableton on June 8, 2015 12:15_

Same issue here. Instances in a VPC too.

_From @SteveMcD on August 8, 2015 0:41_

+1.  Same issue with instances in a VPC

_From @rodrickbrown on August 13, 2015 21:51_

+1 same issue here also inside a VPC 

_From @seanlutner on August 17, 2015 14:43_

+1 also having the issue with instances in a VPC

_From @jamesongithub on August 24, 2015 21:21_

just hit this
ansible v1.9.2
is anyone looking at this?
ugh only P3

_From @peh on August 25, 2015 6:36_

+1

_From @macolo on September 1, 2015 11:58_

+1

_From @cdesales on September 1, 2015 11:58_

+1 same issue

_From @sfontana on September 1, 2015 11:59_

+1

_From @GergelyKalmar on September 2, 2015 7:5_

+1

_From @hyperized on September 16, 2015 11:45_

+1

_From @Constantin07 on September 18, 2015 7:8_

Experiencing the same issue.
+1

_From @kylejohnson on October 1, 2015 4:2_

+1, been dealing with this for a while.

_From @viesti on October 11, 2015 19:32_

+1 This would be neat to have. In the meanwhile, I whipped together a small module to ensure that given groups exist on a node: https://gist.github.com/viesti/786f5d93621985556b78

_From @dverbeek84 on October 25, 2015 19:14_

+1

_From @bdgscotland on November 29, 2015 8:46_

+1

_From @georgeliaw on November 30, 2015 21:9_

+1

_From @Constantin07 on December 9, 2015 17:46_

I'm using AWS CLI equivalent meantime to alter SG association:

aws ec2 modify-instance-attribute --instance-id "{{ instance_id }}" --groups "{{ sg_id }}"

_From @fabfuel on February 9, 2016 10:41_

+1

_From @sdomula on February 15, 2016 12:3_

+1

_From @vre on February 17, 2016 16:9_

Actually you can modify the security groups of existing instance with ec2_eni.. but it bit painful:
1) you need elastic network interface id (eni_id) of the network interface of the instance
2) you need to fish out the security group ids as ec2_eni does not (yet?) process names

```
- ec2_eni:
    eni_id: eni-c063daa6
    state: present
    security_groups: sg-772eae13, sg-961e4df2, sg-991c4ffd
```

One way to avoid the problem altogether is to make sure that every instance has its own security group (or group of instances use same security group) and then modify that security group - your Ansible script should be your documentation anyways.. Life is easier when you assign instance purpose specific a IAM roles and in similar way a security group matching the role.

_From @darrengruber on February 17, 2016 19:48_

+1

_From @jdubs on February 18, 2016 20:34_

+1

_From @CptnKirk on February 20, 2016 4:2_

+1

_From @MichaelDrogalis on March 9, 2016 16:12_

+1

_From @sendai-ono on March 12, 2016 19:25_

> aws ec2 modify-instance-attribute --instance-id "{{ instance_id }}" --groups "{{ sg_id }}"

I'm also using a simple and dirty conditional to respect idempotency

```
local_action:
  module: >
    shell
      aws ec2 modify-instance-attribute --instance-id "{{ ec2_id }}"
      --groups "{{ec2_security_group_ids}}" "{{newgroup.group_id}}"
   when: newgroup.group_id not in ec2_security_group_ids
```

+1


Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/ec2.py b/lib/ansible/modules/cloud/amazon/ec2.py
--- a/lib/ansible/modules/cloud/amazon/ec2.py
+++ b/lib/ansible/modules/cloud/amazon/ec2.py
@@ -617,7 +617,8 @@
 
 import time
 from ast import literal_eval
-from ansible.module_utils.six import get_function_code
+from ansible.module_utils.six import get_function_code, string_types
+from ansible.module_utils._text import to_text
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils.ec2 import get_aws_connection_info, ec2_argument_spec, ec2_connect, connect_to_aws
 from distutils.version import LooseVersion
@@ -1392,6 +1393,8 @@ def startstop_instances(module, ec2, instance_ids, state, instance_tags):
     wait_timeout = int(module.params.get('wait_timeout'))
     source_dest_check = module.params.get('source_dest_check')
     termination_protection = module.params.get('termination_protection')
+    group_id = module.params.get('group_id')
+    group_name = module.params.get('group')
     changed = False
     instance_dict_array = []
 
@@ -1438,6 +1441,24 @@ def startstop_instances(module, ec2, instance_ids, state, instance_tags):
                 inst.modify_attribute('disableApiTermination', termination_protection)
                 changed = True
 
+            # Check security groups and if we're using ec2-vpc; ec2-classic security groups may not be modified
+            if inst.vpc_id and group_name:
+                grp_details = ec2.get_all_security_groups(filters={'vpc_id': inst.vpc_id})
+                if isinstance(group_name, string_types):
+                    group_name = [group_name]
+                unmatched = set(group_name) - set(to_text(grp.name) for grp in grp_details)
+                if unmatched:
+                    module.fail_json(msg="The following group names are not valid: %s" % ', '.join(unmatched))
+                group_ids = [to_text(grp.id) for grp in grp_details if to_text(grp.name) in group_name]
+            elif inst.vpc_id and group_id:
+                if isinstance(group_id, string_types):
+                    group_id = [group_id]
+                grp_details = ec2.get_all_security_groups(group_ids=group_id)
+                group_ids = [grp_item.id for grp_item in grp_details]
+            if inst.vpc_id and (group_name or group_id):
+                if set(sg.id for sg in inst.groups) != set(group_ids):
+                    changed = inst.modify_attribute('groupSet', group_ids)
+
             # Check instance state
             if inst.state != state:
                 instance_dict_array.append(get_instance_info(inst))
@@ -1611,7 +1632,7 @@ def main():
             ['network_interfaces', 'group_id'],
             ['network_interfaces', 'private_ip'],
             ['network_interfaces', 'vpc_subnet_id'],
-            ],
+        ],
     )
 
     if not HAS_BOTO:


Test patch:
diff --git a/test/sanity/pep8/legacy-files.txt b/test/sanity/pep8/legacy-files.txt
--- a/test/sanity/pep8/legacy-files.txt
+++ b/test/sanity/pep8/legacy-files.txt
@@ -149,7 +149,6 @@ lib/ansible/modules/cloud/amazon/cloudfront_facts.py
 lib/ansible/modules/cloud/amazon/cloudtrail.py
 lib/ansible/modules/cloud/amazon/cloudwatchevent_rule.py
 lib/ansible/modules/cloud/amazon/dynamodb_table.py
-lib/ansible/modules/cloud/amazon/ec2.py
 lib/ansible/modules/cloud/amazon/ec2_ami.py
 lib/ansible/modules/cloud/amazon/ec2_ami_copy.py
 lib/ansible/modules/cloud/amazon/ec2_ami_find.py