Repository: ansible-collections/amazon.aws

Issue:
EC2 module errors out since it incorrectly includes terminated instances
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
Given this task:
```yaml
    - name: stop ec2 instances
      ec2:
        region: eu-west-1
        state: stopped
        wait: yes
        instance_tags:
          strange-dog-match-tag: "what"
```
If there are existing (recently) terminated ec2 instances matching that tag, Ansible will error out with this message:
`"Unable to change state for instance i-03bb1408d136a9a4b, error: EC2ResponseError: 400 Bad Request\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Response><Errors><Error><Code>IncorrectInstanceState</Code><Message>This instance 'i-03bb1408d136a9a4b' is not in a state from which it can be stopped.</Message></Error></Errors><RequestID>d35eac01-6fa2-4c62-9763-3a5a08651a5c</RequestID></Response>"}`

This seems to happen because Ansible considers all instances matching the `instance_tags`, including terminated instances. The correct behaviour would be to only consider non-terminated instances for all evaluations

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
ec2

##### ANSIBLE VERSION
```paste below
ansible 2.9.12
  config file = None
  configured module search path = ['/home/trond/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.8.2 (default, Jul 16 2020, 14:00:26) [GCC 9.3.0]

```

##### CONFIGURATION
```paste below
N/A
```

##### OS / ENVIRONMENT
Ubuntu 20.04

##### STEPS TO REPRODUCE
See description above

##### EXPECTED RESULTS
Ansible should disregard terminated instances and not error out

##### ACTUAL RESULTS
See description



Discussion:
Files identified in the description:
* [`plugins/modules/ec2.py`](https://github.com/ansible-collections/amazon.aws/blob/main/plugins/modules/ec2.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @jillr @lwade @s-hertel @tgerla @tremble @wimnat
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Hi @trondhindenes  I am working on this issue

Resolving patch:
diff --git a/plugins/modules/ec2.py b/plugins/modules/ec2.py
--- a/plugins/modules/ec2.py
+++ b/plugins/modules/ec2.py
@@ -1413,6 +1413,8 @@ def startstop_instances(module, ec2, instance_ids, state, instance_tags):
         for key, value in instance_tags.items():
             filters["tag:" + key] = value
 
+    filters['instance-state-name'] = ["!terminated", "!shutting-down"]
+
     if module.params.get('id'):
         filters['client-token'] = module.params['id']
     # Check that our instances are not in the state we want to take


Test patch:
diff --git a/tests/integration/targets/ec2/aliases b/tests/integration/targets/ec2/aliases
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/ec2/aliases
@@ -0,0 +1,2 @@
+cloud/aws
+shippable/aws/group4
diff --git a/tests/integration/targets/ec2/defaults/main.yml b/tests/integration/targets/ec2/defaults/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/ec2/defaults/main.yml
@@ -0,0 +1,4 @@
+---
+vpc_cidr: '10.{{ 256 | random(seed=resource_prefix) }}.0.0/16'
+subnet_cidr: '10.{{ 256 | random(seed=resource_prefix) }}.1.0/24'
+ec2_ami_name: 'amzn2-ami-hvm-2.*-x86_64-gp2'
diff --git a/tests/integration/targets/ec2/meta/main.yml b/tests/integration/targets/ec2/meta/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/ec2/meta/main.yml
@@ -0,0 +1,3 @@
+dependencies:
+  - prepare_tests
+  - setup_ec2
diff --git a/tests/integration/targets/ec2/tasks/main.yml b/tests/integration/targets/ec2/tasks/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/ec2/tasks/main.yml
@@ -0,0 +1,196 @@
+---
+- module_defaults:
+    group/aws:
+      aws_region: '{{ aws_region }}'
+      aws_access_key: '{{ aws_access_key | default(omit) }}'
+      aws_secret_key: '{{ aws_secret_key | default(omit) }}'
+      security_token: '{{ security_token | default(omit) }}'
+  collections:
+    - community.aws
+
+  block:
+
+    # SETUP: vpc, ec2 key pair, subnet, security group, ec2 instance
+    - name: list available AZs
+      aws_az_info:
+      register: region_azs
+
+    - name: pick an AZ for testing
+      set_fact:
+        availability_zone: "{{ region_azs.availability_zones[0].zone_name }}"
+
+    - name: create a VPC to work in
+      ec2_vpc_net:
+        cidr_block: '{{ vpc_cidr }}'
+        state: present
+        name: '{{ resource_prefix }}_setup'
+        resource_tags:
+          Name: '{{ resource_prefix }}_setup'
+      register: setup_vpc
+
+    - name: create a key pair to use for creating an ec2 instance
+      ec2_key:
+        name: '{{ resource_prefix }}_setup'
+        state: present
+      register: setup_key
+
+    - name: create a subnet to use for creating an ec2 instance
+      ec2_vpc_subnet:
+        az: '{{ availability_zone }}'
+        tags: '{{ resource_prefix }}_setup'
+        vpc_id: '{{ setup_vpc.vpc.id }}'
+        cidr: '{{ subnet_cidr }}'
+        state: present
+        resource_tags:
+          Name: '{{ resource_prefix }}_setup'
+      register: setup_subnet
+
+    - name: create a security group to use for creating an ec2 instance
+      ec2_group:
+        name: '{{ resource_prefix }}_setup'
+        description: 'created by Ansible integration tests'
+        state: present
+        vpc_id: '{{ setup_vpc.vpc.id }}'
+      register: setup_sg
+
+    - name: Find AMI to use
+      ec2_ami_info:
+        owners: 'amazon'
+        filters:
+          name: '{{ ec2_ami_name }}'
+      register: ec2_amis
+
+    - name: Set fact with latest AMI
+      vars:
+        latest_ami: '{{ ec2_amis.images | sort(attribute="creation_date") | last }}'
+      set_fact:
+        ec2_ami_image: '{{ latest_ami.image_id }}'
+
+    # ============================================================
+
+    - name: test first instance is started
+      ec2:
+        instance_type: t2.micro
+        key_name: '{{ setup_key.key.name }}'
+        state: present
+        image: '{{ ec2_ami_image }}'
+        wait: yes
+        instance_tags:
+          ResourcePrefix: '{{ resource_prefix }}-integration_tests'
+        group_id: '{{ setup_sg.group_id }}'
+        vpc_subnet_id: '{{ setup_subnet.subnet.id }}'
+      register: test_instance_1
+
+    - name: test second instance is started
+      ec2:
+        instance_type: t2.micro
+        key_name: '{{ setup_key.key.name }}'
+        state: present
+        image: '{{ ec2_ami_image }}'
+        wait: yes
+        instance_tags:
+          ResourcePrefix: '{{ resource_prefix }}-another_tag'
+        group_id: '{{ setup_sg.group_id }}'
+        vpc_subnet_id: '{{ setup_subnet.subnet.id }}'
+      register: test_instance_2
+
+    - name: assert instances started
+      assert:
+        that:
+          - "test_instance_1.instances[0].state == 'running'"
+          - "test_instance_2.instances[0].state == 'running'"
+
+    - name: test first instance is terminated
+      ec2:
+        instance_ids: "{{ test_instance_1.instance_ids }}"
+        state: absent
+        wait: yes
+      register: result
+
+    - name: assert instance terminated
+      assert:
+        that:
+          - "result.instances[0].state == 'terminated'"
+
+    - name: test terminated instance is ignored when stopping
+      ec2:
+        instance_tags:
+          ResourcePrefix: '{{ resource_prefix }}-integration_tests'
+        state: stopped
+        wait: yes
+      register: result
+
+    - name: assert resource not changed
+      assert:
+        that:
+          - "result.changed == False"
+
+    - name: test second instance not terminated
+      ec2_instance_info:
+        instance_ids: "{{ test_instance_2.instance_ids }}"
+      register: result
+
+    - name: assert second instance still running
+      assert:
+        that:
+          - (result.instances|length) == 1
+          - "result.instances[0].state.name == 'running'"
+
+    # ========================================================
+
+  always:
+
+    # ============================================================
+
+
+    # TEAR DOWN: ec2 instance, ec2 key pair, security group, vpc
+    - name: Announce teardown start
+      debug:
+        msg: "***** TESTING COMPLETE. COMMENCE TEARDOWN *****"
+
+    - name: get list of test instances
+      ec2_instance_info:
+        filters:
+          "tag:ResourcePrefix": "{{ resource_prefix }}-*"
+      register: test_instances
+
+    - name: delete test instances
+      ec2:
+        instance_ids: "{{ test_instances.instances|map(attribute='instance_id') }}"
+        state: absent
+        wait: yes
+      ignore_errors: yes
+
+    - name: remove setup keypair
+      ec2_key:
+        name: '{{resource_prefix}}_setup'
+        state: absent
+      ignore_errors: yes
+
+    - name: remove setup security group
+      ec2_group:
+        name: '{{ resource_prefix }}_setup'
+        description: 'created by Ansible integration tests'
+        state: absent
+        vpc_id: '{{ setup_vpc.vpc.id }}'
+      ignore_errors: yes
+
+    - name: remove setup subnet
+      ec2_vpc_subnet:
+        az: '{{ ec2_region }}a'
+        tags: '{{resource_prefix}}_setup'
+        vpc_id: '{{ setup_vpc.vpc.id }}'
+        cidr: '{{ subnet_cidr }}'
+        state: absent
+        resource_tags:
+          Name: '{{ resource_prefix }}_setup'
+      ignore_errors: yes
+
+    - name: remove setup VPC
+      ec2_vpc_net:
+        cidr_block: '{{ vpc_cidr }}'
+        state: absent
+        name: '{{ resource_prefix }}_setup'
+        resource_tags:
+          Name: '{{ resource_prefix }}_setup'
+      ignore_errors: yes