Repository: ansible-collections/amazon.aws

Issue:
Exception modifying rule with existing HttpRequestMethodConfig
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
Using community.aws.elb_application_lb we have found that you are not able to modify existing routing rules if the condition field is 'http-request-method'.  It is possible to create/update an ALB and add a rule using http-request-method in the first instance. However any subsequent executions of the same task, or trying to modify the routing rule, result in a KeyError exception. As shown below, community.aws.elb_application_lb is relying on the elbv2 module_util to compare the rule conditions.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
elbv2.py

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.10.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/bradley/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/bradley/.local/lib/python3.8/site-packages/ansible
  executable location = /home/bradley/.local/bin/ansible
  python version = 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
N/A
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
```
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
```

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- hosts: localhost
  tasks:
    - community.aws.elb_application_lb:
        name: test-application-load-balancer
        state: present
        security_groups:
          - sg-group #masked
        subnets:
          - subnet-a #masked
          - subnet-b #masked
          - subnet-c #masked
        listeners:
          - Protocol: HTTP
            Port: 80
            DefaultActions:
              - Type: fixed-response
                FixedResponseConfig:
                  StatusCode: "200"
            Rules:
              - Conditions:
                  - Field: http-request-method
                    HttpRequestMethodConfig:
                      Values:
                        - 'GET'
                Priority: '1'
                Actions:
                  - Type: fixed-response
                    FixedResponseConfig:
                      StatusCode: "200"        
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
It's expected that the above task would in the first instance create a new load balancer, and on subsequent runs execute successfully with no changes.

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes -->
```paste below
The full traceback is:
Traceback (most recent call last):
  File "/home/bradley/.ansible/tmp/ansible-tmp-1604317711.9949253-1774-237985700272502/AnsiballZ_elb_application_lb.py", line 102, in <module>
    _ansiballz_main()
  File "/home/bradley/.ansible/tmp/ansible-tmp-1604317711.9949253-1774-237985700272502/AnsiballZ_elb_application_lb.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/bradley/.ansible/tmp/ansible-tmp-1604317711.9949253-1774-237985700272502/AnsiballZ_elb_application_lb.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.elb_application_lb', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/community/aws/plugins/modules/elb_application_lb.py", line 665, in <module>
  File "/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/community/aws/plugins/modules/elb_application_lb.py", line 659, in main
  File "/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/community/aws/plugins/modules/elb_application_lb.py", line 544, in create_or_update_elb
  File "/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/elbv2.py", line 816, in compare_rules
  File "/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/elbv2.py", line 791, in _compare_rule
  File "/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/elbv2.py", line 727, in _compare_condition
KeyError: 'Values'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/bradley/.ansible/tmp/ansible-tmp-1604317711.9949253-1774-237985700272502/AnsiballZ_elb_application_lb.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/home/bradley/.ansible/tmp/ansible-tmp-1604317711.9949253-1774-237985700272502/AnsiballZ_elb_application_lb.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/bradley/.ansible/tmp/ansible-tmp-1604317711.9949253-1774-237985700272502/AnsiballZ_elb_application_lb.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.elb_application_lb', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/community/aws/plugins/modules/elb_application_lb.py\", line 665, in <module>\n  File \"/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/community/aws/plugins/modules/elb_application_lb.py\", line 659, in main\n  File \"/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/community/aws/plugins/modules/elb_application_lb.py\", line 544, in create_or_update_elb\n  File \"/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/elbv2.py\", line 816, in compare_rules\n  File \"/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/elbv2.py\", line 791, in _compare_rule\n  File \"/tmp/ansible_community.aws.elb_application_lb_payload_mxyup0n8/ansible_community.aws.elb_application_lb_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/elbv2.py\", line 727, in _compare_condition\nKeyError: 'Values'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
```



Discussion:

Resolving patch:
diff --git a/plugins/module_utils/elbv2.py b/plugins/module_utils/elbv2.py
--- a/plugins/module_utils/elbv2.py
+++ b/plugins/module_utils/elbv2.py
@@ -51,6 +51,9 @@ def __init__(self, connection, module):
         else:
             self.elb_attributes = None
 
+        if not self.module.boto3_at_least('1.16.57'):
+            self.module.fail_json(msg="elbv2 requires boto3 >= 1.16.57")
+
     def wait_for_status(self, elb_arn):
         """
         Wait for load balancer to reach 'active' status
@@ -719,11 +722,46 @@ def _compare_condition(self, current_conditions, condition):
         condition_found = False
 
         for current_condition in current_conditions:
-            if current_condition.get('SourceIpConfig'):
+            # host-header: current_condition includes both HostHeaderConfig AND Values while
+            # condition can be defined with either HostHeaderConfig OR Values. Only use
+            # HostHeaderConfig['Values'] comparison if both conditions includes HostHeaderConfig.
+            if current_condition.get('HostHeaderConfig') and condition.get('HostHeaderConfig'):
+                if (current_condition['Field'] == condition['Field'] and
+                        sorted(current_condition['HostHeaderConfig']['Values']) == sorted(condition['HostHeaderConfig']['Values'])):
+                    condition_found = True
+                    break
+            elif current_condition.get('HttpHeaderConfig'):
+                if (current_condition['Field'] == condition['Field'] and
+                        sorted(current_condition['HttpHeaderConfig']['Values']) == sorted(condition['HttpHeaderConfig']['Values']) and
+                        current_condition['HttpHeaderConfig']['HttpHeaderName'] == condition['HttpHeaderConfig']['HttpHeaderName']):
+                    condition_found = True
+                    break
+            elif current_condition.get('HttpRequestMethodConfig'):
+                if (current_condition['Field'] == condition['Field'] and
+                        sorted(current_condition['HttpRequestMethodConfig']['Values']) == sorted(condition['HttpRequestMethodConfig']['Values'])):
+                    condition_found = True
+                    break
+            # path-pattern: current_condition includes both PathPatternConfig AND Values while
+            # condition can be defined with either PathPatternConfig OR Values. Only use
+            # PathPatternConfig['Values'] comparison if both conditions includes PathPatternConfig.
+            elif current_condition.get('PathPatternConfig') and condition.get('PathPatternConfig'):
+                if (current_condition['Field'] == condition['Field'] and
+                        sorted(current_condition['PathPatternConfig']['Values']) == sorted(condition['PathPatternConfig']['Values'])):
+                    condition_found = True
+                    break
+            elif current_condition.get('QueryStringConfig'):
+                # QueryString Values is not sorted as it is the only list of dicts (not strings).
+                if (current_condition['Field'] == condition['Field'] and
+                        current_condition['QueryStringConfig']['Values'] == condition['QueryStringConfig']['Values']):
+                    condition_found = True
+                    break
+            elif current_condition.get('SourceIpConfig'):
                 if (current_condition['Field'] == condition['Field'] and
-                        current_condition['SourceIpConfig']['Values'][0] == condition['SourceIpConfig']['Values'][0]):
+                        sorted(current_condition['SourceIpConfig']['Values']) == sorted(condition['SourceIpConfig']['Values'])):
                     condition_found = True
                     break
+            # Not all fields are required to have Values list nested within a *Config dict
+            # e.g. fields host-header/path-pattern can directly list Values
             elif current_condition['Field'] == condition['Field'] and sorted(current_condition['Values']) == sorted(condition['Values']):
                 condition_found = True
                 break


Test patch: