Repository: ansible-collections/amazon.aws

Issue:
Composing inventory_hostname manually
### Summary

Here is my current config:
```yaml
---
plugin: amazon.aws.aws_ec2
regions:
  - eu-central-1
hostnames:
  - tag:Name
compose:
  ansible_host: private_ip_address
```

Executing it sets inventory_hostname variable(s) to the "Name" tag value. However, I would like to know whether it's possible to create a completely custom inventory_hostname, as I haven't found a clear way to do it by searching through the docs (as far as I understood it's only allowed to use the values from [here](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options) which doesn't suit my case).

I have an AWS cloud and an Azure cloud, which serves as a mirror of the AWS cloud. The VMs on both clouds are named similarly (e.g. "prometheus"). I would like to refer to these VMs as 'aws-prometheus' and 'azu-prometheus'. The [azure_rm plugin](https://docs.ansible.com/ansible/latest/collections/azure/azcollection/azure_rm_inventory.html#parameter-hostnames) allows creating custom inventory_hostname:

```yaml
hostnames:
  - "'azu-' + name"
```
, where "name" is just a key in the object returned by the API (like 'private_ip_address' for AWS). This way, I can refer to Azure VMs with 'azu-' prefix.

I would like to know whether it's possible to do the same with the aws_ec2 plugin. I've played around with different combinations and values for 'hostnames' but found myself limited with the values allowed by AWS.

This doesn't work because "aws-" is not a recognized value from [here](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options):
```yaml
---
plugin: amazon.aws.aws_ec2
regions:
  - eu-central-1
hostnames:
  - name: "tag:Name"
    prefix: "aws-"
compose:
  ansible_host: private_ip_address
```

Thank you in advance.

### Issue Type

Feature Idea

### Component Name

amazon.aws.aws_ec2

### Additional Information

_No response_

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Thank you, frankly I expected it was me not being able to do this properly rather than this feature missing :)

I had a quick look through the PR, do I understand correctly that a new config would look like this:
```yaml
---
plugin: amazon.aws.aws_ec2
regions:
  - eu-central-1
hostnames:
  - name: "tag:Name"
    prefix: "aws-"
compose:
  ansible_host: private_ip_address
```
or like this:
```yaml
---
plugin: amazon.aws.aws_ec2
regions:
  - eu-central-1
hostnames:
  - name: "tag:Name"
    separator: "-"
    prefix: "aws"
compose:
  ansible_host: private_ip_address
```
E.g. I have a machine on AWS with the tag "Name" of value "prometheus".
Would both of these configs make it possible to refer to this VM as "aws-prometheus" in the playbooks?

Thank you in advance.
> Thank you, frankly I expected it was me not being able to do this properly rather than this feature missing :)
> 
> I had a quick look through the PR, do I understand correctly that a new config would look like this:
> 
> ```yaml
> ---
> plugin: amazon.aws.aws_ec2
> regions:
>   - eu-central-1
> hostnames:
>   - name: "tag:Name"
>     prefix: "aws-"
> compose:
>   ansible_host: private_ip_address
> ```
> 
> or like this:
> 
> ```yaml
> ---
> plugin: amazon.aws.aws_ec2
> regions:
>   - eu-central-1
> hostnames:
>   - name: "tag:Name"
>     separator: "-"
>     prefix: "aws"
> compose:
>   ansible_host: private_ip_address
> ```
> 
> E.g. I have a machine on AWS with the tag "Name" of value "prometheus". Would both of these configs make it possible to refer to this VM as "aws-prometheus" in the playbooks?
> 
> Thank you in advance.

Just double checked and you should be able to use the `hostvars_prefix`! So, that would look like:
```
...
hostvars_prefix: 'aws-'
hostnames:
  - name: "tag:Name"
...
```

Let me know if that works for you! This feature was added in version 3.1.0 

If using `hostvars_prefix` isn't enough (since it only prefixes the names of AWS variables), then to use the linked PR, you would use either:
> ```yaml
> ---
> plugin: amazon.aws.aws_ec2
> regions:
>   - eu-central-1
> hostnames:
>   - name: "tag:Name"
>     prefix: "aws-"
>     separator: ''
> compose:
>   ansible_host: private_ip_address
> ```
> 
> or like this:
> 
> ```yaml
> ---
> plugin: amazon.aws.aws_ec2
> regions:
>   - eu-central-1
> hostnames:
>   - name: "tag:Name"
>     separator: "-"
>     prefix: "aws"
> compose:
>   ansible_host: private_ip_address
> ```

since the default for `separator` is `_`
Thank you for the fast reply.

`hostvars_prefix` is, per documentation, `The prefix for host variables names coming from AWS.`
So, having the following config:
```yaml
---
plugin: amazon.aws.aws_ec2
regions:
  - eu-central-1
hostvars_prefix: "aws-"
hostnames:
  - tag:Name
```
and executing the following command:
`ansible-inventory -i inventory/aws_ec2.yml --list`
returned:
![image](https://user-images.githubusercontent.com/4324562/170028441-9b73656a-22e6-4b50-83b6-27da6a6e3332.png)

While it does what is stated in the documentation, it doesn't alter the hostname itself, which is my use case (and, of course, I'd like to keep the original var names). Thus, eagerly waiting for this PR to get merged :)


Resolving patch:
diff --git a/plugins/inventory/aws_ec2.py b/plugins/inventory/aws_ec2.py
--- a/plugins/inventory/aws_ec2.py
+++ b/plugins/inventory/aws_ec2.py
@@ -39,11 +39,31 @@
         hostnames:
           description:
               - A list in order of precedence for hostname variables.
-              - You can use the options specified in U(http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options).
-              - To use tags as hostnames use the syntax tag:Name=Value to use the hostname Name_Value, or tag:Name to use the value of the Name tag.
           type: list
           elements: dict
           default: []
+          suboptions:
+            name:
+                description:
+                    - Name of the host.
+                    - Can be one of the options specified in U(http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options).
+                    - To use tags as hostnames use the syntax tag:Name=Value to use the hostname Name_Value, or tag:Name to use the value of the Name tag.
+                    - If value provided does not exist in the above options, it will be used as a literal string.
+                type: str
+                required: True
+            prefix:
+                description:
+                    - Prefix to prepend to I(name). Same options as I(name).
+                    - If I(prefix) is specified, final hostname will be I(prefix) +  I(separator) + I(name).
+                type: str
+                default: ''
+                required: False
+            separator:
+                description:
+                    - Value to separate I(prefix) and I(name) when I(prefix) is specified.
+                type: str
+                default: '_'
+                required: False
         filters:
           description:
               - A dictionary of filter value pairs.
@@ -149,6 +169,9 @@
   - name: 'private-ip-address'
     separator: '_'
     prefix: 'tag:Name'
+  - name: 'test_literal' # Using literal values for hostname
+    separator: '-'       # Hostname will be aws-test_literal
+    prefix: 'aws'
 
 # Example using constructed features to create groups and set ansible_host
 plugin: aws_ec2
@@ -387,9 +410,11 @@ def _get_boto_attr_chain(self, filter_name, instance):
             :param instance: instance dict returned by boto3 ec2 describe_instances()
         '''
         allowed_filters = sorted(list(instance_data_filter_to_boto_attr.keys()) + list(instance_meta_filter_to_boto_attr.keys()))
+
+        # If filter not in allow_filters -> use it as a literal string
         if filter_name not in allowed_filters:
-            raise AnsibleError("Invalid filter '%s' provided; filter must be one of %s." % (filter_name,
-                                                                                            allowed_filters))
+            return filter_name
+
         if filter_name in instance_data_filter_to_boto_attr:
             boto_attr_list = instance_data_filter_to_boto_attr[filter_name]
         else:


Test patch:
diff --git a/tests/integration/targets/inventory_aws_ec2/playbooks/test_populating_inventory_with_literal_string.yml b/tests/integration/targets/inventory_aws_ec2/playbooks/test_populating_inventory_with_literal_string.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/inventory_aws_ec2/playbooks/test_populating_inventory_with_literal_string.yml
@@ -0,0 +1,56 @@
+---
+- hosts: 127.0.0.1
+  connection: local
+  gather_facts: no
+  environment: "{{ ansible_test.environment }}"
+  tasks:
+
+    - module_defaults:
+        group/aws:
+          aws_access_key: '{{ aws_access_key }}'
+          aws_secret_key: '{{ aws_secret_key }}'
+          security_token: '{{ security_token | default(omit) }}'
+          region: '{{ aws_region }}'
+      block:
+
+        # Create VPC, subnet, security group, and find image_id to create instance
+        - include_tasks: setup.yml
+
+        # Create new host, refresh inventory
+        - name: create a new host
+          ec2_instance:
+            image_id: '{{ image_id }}'
+            name: '{{ resource_prefix }}'
+            tags:
+              OtherTag: value
+            instance_type: t2.micro
+            security_groups: '{{ sg_id }}'
+            vpc_subnet_id: '{{ subnet_id }}'
+            wait: no
+          register: setup_instance
+
+        - meta: refresh_inventory
+
+        - name: register the current hostname
+          set_fact:
+            expected_hostname: "aws-{{ resource_prefix }}"
+
+        - name: "Ensure we've got a hostvars entry for the new host"
+          assert:
+            that:
+              - expected_hostname in hostvars
+
+      always:
+
+        - name: remove setup ec2 instance
+          ec2_instance:
+            instance_type: t2.micro
+            instance_ids: '{{ setup_instance.instance_ids }}'
+            state: absent
+            name: '{{ resource_prefix }}'
+            security_groups: "{{ sg_id }}"
+            vpc_subnet_id: "{{ subnet_id }}"
+          ignore_errors: yes
+          when: setup_instance is defined
+
+        - include_tasks: tear_down.yml
diff --git a/tests/integration/targets/inventory_aws_ec2/runme.sh b/tests/integration/targets/inventory_aws_ec2/runme.sh
--- a/tests/integration/targets/inventory_aws_ec2/runme.sh
+++ b/tests/integration/targets/inventory_aws_ec2/runme.sh
@@ -38,6 +38,9 @@ ansible-playbook playbooks/create_inventory_config.yml -e "template='inventory_w
 ansible-playbook playbooks/test_populating_inventory_with_constructed.yml "$@"
 ansible-playbook playbooks/create_inventory_config.yml -e "template='inventory_with_concatenation.yml.j2'" "$@"
 ansible-playbook playbooks/test_populating_inventory_with_concatenation.yml "$@"
+ansible-playbook playbooks/create_inventory_config.yml -e "template='inventory_with_literal_string.yml.j2'" "$@"
+ansible-playbook playbooks/test_populating_inventory_with_literal_string.yml "$@"
+
 # generate inventory config with includes_entries_matching and prepare the tests
 ansible-playbook playbooks/create_inventory_config.yml -e "template='inventory_with_include_or_exclude_filters.yml.j2'" "$@"
 ansible-playbook playbooks/test_populating_inventory_with_include_or_exclude_filters.yml "$@"
diff --git a/tests/integration/targets/inventory_aws_ec2/templates/inventory_with_literal_string.yml.j2 b/tests/integration/targets/inventory_aws_ec2/templates/inventory_with_literal_string.yml.j2
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/inventory_aws_ec2/templates/inventory_with_literal_string.yml.j2
@@ -0,0 +1,15 @@
+plugin: amazon.aws.aws_ec2
+aws_access_key_id: '{{ aws_access_key }}'
+aws_secret_access_key: '{{ aws_secret_key }}'
+{% if security_token | default(false) %}
+aws_security_token: '{{ security_token }}'
+{% endif %}
+regions:
+- '{{ aws_region }}'
+filters:
+  tag:Name:
+  - '{{ resource_prefix }}'
+hostnames:
+  - name: 'tag:Name'
+    separator: '-'
+    prefix: 'aws'