Repository: ansible/ansible

Issue:
cloudfront_distribution changes forwarded headers order again and again
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report

##### COMPONENT NAME
cloudfront_distribution

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.3.0
  config file = None
  configured module search path = [u'/Users/gennadii_aleksandrov/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/gennadii_aleksandrov/Library/Python/2.7/lib/python/site-packages/ansible
  executable location = /Users/Gennadii_Aleksandrov/Library/Python/2.7/bin/ansible
  python version = 2.7.10 (default, Jul 15 2017, 17:16:57) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
N/A
-->

##### SUMMARY
Ansible always trying to change CloudFront distribution's forwarded headers order.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
1. Create CloudFront distribution with forwarded headers as shown below:
<!--- Paste example playbooks or commands between quotes below -->
```yaml
        default_cache_behavior:
            headers:
              - X-HTTP-Forwarded-For
              - CloudFront-Forwarded-Proto
              - Host
              - Origin
              - Referer
```
2. Run playbook again

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
CloudFront distribution should bot be changed.
##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
CloudFront distribution has been changed:
```
TASK [Create CloudFront distribution] ********************************************************************************************************************************
--- before
+++ after
@@ -3,11 +3,11 @@
         "ForwardedValues": {
             "Headers": {
                 "Items": [
+                    "X-HTTP-Forwarded-For",

                     "CloudFront-Forwarded-Proto",
                     "Host",
                     "Origin",
-                    "Referer",
-                    "X-HTTP-Forwarded-For"
+                    "Referer"
                 ]
             }
         }

changed: [localhost]
```
<!--- Paste verbatim command output between quotes below -->




Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/cloudfront_distribution.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/cloudfront_distribution.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @willthames @wilvk
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/cloudfront_distribution.py b/lib/ansible/modules/cloud/amazon/cloudfront_distribution.py
--- a/lib/ansible/modules/cloud/amazon/cloudfront_distribution.py
+++ b/lib/ansible/modules/cloud/amazon/cloudfront_distribution.py
@@ -1405,10 +1405,10 @@ def validate_origins(self, client, config, origins, default_origin_domain_name,
                 all_origins[origin['domain_name']] = origin
                 new_domains.append(origin['domain_name'])
             if purge_origins:
-                for domain in all_origins:
+                for domain in list(all_origins.keys()):
                     if domain not in new_domains:
                         del(all_origins[domain])
-            return ansible_list_to_cloudfront_list(all_origins.values())
+            return ansible_list_to_cloudfront_list(list(all_origins.values()))
         except Exception as e:
             self.module.fail_json_aws(e, msg="Error validating distribution origins")
 
@@ -1488,7 +1488,7 @@ def validate_cache_behaviors(self, config, cache_behaviors, valid_origins, purge
             if purge_cache_behaviors:
                 for target_origin_id in set(all_cache_behaviors.keys()) - set([cb['path_pattern'] for cb in cache_behaviors]):
                     del(all_cache_behaviors[target_origin_id])
-            return ansible_list_to_cloudfront_list(all_cache_behaviors.values())
+            return ansible_list_to_cloudfront_list(list(all_cache_behaviors.values()))
         except Exception as e:
             self.module.fail_json_aws(e, msg="Error validating distribution cache behaviors")
 
@@ -1540,6 +1540,8 @@ def validate_forwarded_values(self, config, forwarded_values, cache_behavior):
                 forwarded_values = dict()
             existing_config = config.get('forwarded_values', {})
             headers = forwarded_values.get('headers', existing_config.get('headers', {}).get('items'))
+            if headers:
+                headers.sort()
             forwarded_values['headers'] = ansible_list_to_cloudfront_list(headers)
             if 'cookies' not in forwarded_values:
                 forward = existing_config.get('cookies', {}).get('forward', self.__default_cache_behavior_forwarded_values_forward_cookies)


Test patch:
diff --git a/test/integration/targets/cloudfront_distribution/defaults/main.yml b/test/integration/targets/cloudfront_distribution/defaults/main.yml
--- a/test/integration/targets/cloudfront_distribution/defaults/main.yml
+++ b/test/integration/targets/cloudfront_distribution/defaults/main.yml
@@ -1,12 +1,16 @@
-cloudfront_hostname: "{{ resource_prefix | lower }}01"
+cloudfront_hostname: "{{ resource_prefix }}01"
 # Use a domain that has a wildcard DNS
-cloudfront_alias: "{{ cloudfront_hostname | lower }}.github.io"
+cloudfront_alias: "{{ cloudfront_hostname }}.github.io"
 
 cloudfront_test_cache_behaviors:
   - path_pattern: /test/path
     forwarded_values:
       headers:
         - Host
+        - X-HTTP-Forwarded-For
+        - CloudFront-Forwarded-Proto
+        - Origin
+        - Referer
     allowed_methods:
       items:
         - GET