Repository: ansible/ansible

Issue:
Unable to pass in the "headers:" parameter into win_get_url (unlike get_url)
##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
win_get_url module

##### ANSIBLE VERSION
ansible 2.1.0.0

##### CONFIGURATION

##### OS / ENVIRONMENT
N/A
##### SUMMARY
The "win_get_url" is, in theory, the Windows equivalent of the standard "get_url" module (which is normally run on a Linux node).  However, in practice, there are certain limitations that the win_get_url module has versus the get_url module.  For me, the most striking difference is the lack of a "headers:" parameter.  This is something that we need, as we need some way of passing in an API token in order to download a file.   While the "win_uri" module DOES allow for a "headers" parameter, it is also limited by not having a "dest" parameter (which I logged as a separate issue, 19575: https://github.com/ansible/ansible/issues/19575).

##### STEPS TO REPRODUCE
Compare the acceptable options here:
http://docs.ansible.com/ansible/get_url_module.html

versus here:

http://docs.ansible.com/ansible/win_get_url_module.html

<!--- Paste example playbooks or commands between quotes below -->
tasks:
    - name: download FILE
      win_get_url:
         force: yes
         dest: C:\Temp\
         url: <URL>
         headers:
            <API_TOKEN>: <API_TOKEN_VALUE>

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
This should be a valid use of the win_get_url module, and the file should be downloaded to the specified directory after passing in the correct API token credentials.

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

I get a 404 error message, presumably because this extra unknown option is messing with the web request.

<!--- Paste verbatim command output between quotes below -->
```
Exception calling \"DownloadFile\" with \"2\" argument(s): \"The remote server returned an error: (404) Not Found.\""}
```



Discussion:
cc @jhawkesworth
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
This is a nice idea 

as a workaround you could of course call get_url on the controller (delegate_to: localhost) and then copy the downloaded file across with win_copy.

needs_contributor
@jhawkesworth just tried your workaround using delegate_to and it works.
Thanks for the tip!

Resolving patch:
diff --git a/lib/ansible/modules/windows/win_get_url.py b/lib/ansible/modules/windows/win_get_url.py
--- a/lib/ansible/modules/windows/win_get_url.py
+++ b/lib/ansible/modules/windows/win_get_url.py
@@ -31,75 +31,84 @@
 version_added: "1.7"
 short_description: Fetches a file from a given URL
 description:
- - Fetches a file from a URL and saves to locally
- - For non-Windows targets, use the M(get_url) module instead.
+- Fetches a file from a URL and saves it locally.
+- For non-Windows targets, use the M(get_url) module instead.
 author:
-    - "Paul Durivage (@angstwad)"
-    - "Takeshi Kuramochi (tksarah)"
+- Paul Durivage (@angstwad)
+- Takeshi Kuramochi (tksarah)
 options:
   url:
     description:
-      - The full URL of a file to download
-    required: true
-    default: null
+    - The full URL of a file to download.
+    required: yes
   dest:
     description:
-      - The absolute path of the location to save the file at the URL. Be sure
-        to include a filename and extension as appropriate.
-    required: true
-    default: null
+    - The location to save the file at the URL.
+    - Be sure to include a filename and extension as appropriate.
+    required: yes
   force:
     description:
-      - If C(yes), will always download the file. If C(no), will only
-        download the file if it does not exist or the remote file has been
-        modified more recently than the local file. This works by sending
-        an http HEAD request to retrieve last modified time of the requested
-        resource, so for this to work, the remote web server must support
-        HEAD requests.
+    - If C(yes), will always download the file. If C(no), will only
+      download the file if it does not exist or the remote file has been
+      modified more recently than the local file.
+    - This works by sending an http HEAD request to retrieve last modified
+      time of the requested resource, so for this to work, the remote web
+      server must support HEAD requests.
+    type: bool
+    default: 'yes'
     version_added: "2.0"
-    required: false
-    choices: [ "yes", "no" ]
-    default: yes
-  username:
+  headers:
     description:
-      - Basic authentication username
-    required: false
-    default: null
-  password:
+    - Add custom HTTP headers to a request (as a dictionary).
+    version_added: '2.4'
+  url_username:
+    description:
+    - Basic authentication username.
+    aliases: [ username ]
+  url_password:
     description:
-      - Basic authentication password
-    required: false
-    default: null
+    - Basic authentication password.
+    aliases: [ password ]
   skip_certificate_validation:
     description:
     - This option is deprecated since v2.4, please use C(validate_certs) instead.
     - If C(yes), SSL certificates will not be validated. This should only be used
       on personally controlled sites using self-signed certificates.
-    default: 'no'
     type: bool
+    default: 'no'
   validate_certs:
     description:
     - If C(no), SSL certificates will not be validated. This should only be used
       on personally controlled sites using self-signed certificates.
     - If C(skip_certificate_validation) was set, it overrides this option.
-    default: 'yes'
     type: bool
+    default: 'yes'
     version_added: '2.4'
   proxy_url:
     description:
-      - The full URL of the proxy server to download through.
+    - The full URL of the proxy server to download through.
     version_added: "2.0"
-    required: false
   proxy_username:
     description:
-      - Proxy authentication username
+    - Proxy authentication username.
     version_added: "2.0"
-    required: false
   proxy_password:
     description:
-      - Proxy authentication password
+    - Proxy authentication password.
     version_added: "2.0"
-    required: false
+  use_proxy:
+    description:
+    - If C(no), it will not use a proxy, even if one is defined in an environment
+      variable on the target hosts.
+    type: bool
+    default: 'yes'
+    version_added: '2.4'
+# TODO: Once we have implemented DownloadFile() using $WebRequest, enable timeout again
+#  timeout:
+#    description:
+#    - Timeout in seconds for URL request.
+#    default: 10
+#    version_added : '2.5'
 notes:
  - For non-Windows targets, use the M(get_url) module instead.
 '''
@@ -126,14 +135,24 @@
 '''
 
 RETURN = r'''
+dest:
+    description: destination file/path
+    returned: always
+    type: string
+    sample: C:\Users\RandomUser\earthrise.jpg
 url:
     description: requested url
     returned: always
     type: string
     sample: http://www.example.com/earthrise.jpg
-dest:
-    description: destination file/path
+msg:
+    description: Error message, or HTTP status message from web-server
     returned: always
     type: string
-    sample: C:\Users\RandomUser\earthrise.jpg
+    sample: OK
+status_code:
+    description: HTTP status code
+    returned: always
+    type: int
+    sample: 200
 '''


Test patch:
diff --git a/test/integration/targets/win_get_url/defaults/main.yml b/test/integration/targets/win_get_url/defaults/main.yml
--- a/test/integration/targets/win_get_url/defaults/main.yml
+++ b/test/integration/targets/win_get_url/defaults/main.yml
@@ -1,8 +1,8 @@
 ---
 
 test_win_get_url_host: www.redhat.com
-test_win_get_url_link: "https://{{test_win_get_url_host}}"
+test_win_get_url_link: "https://{{ test_win_get_url_host }}"
 test_win_get_url_invalid_link: https://www.redhat.com/skynet_module.html
 test_win_get_url_invalid_path: 'Q:\Filez\Cyberdyne.html'
 test_win_get_url_invalid_path_dir: 'Q:\Filez\'
-test_win_get_url_path: '{{ test_win_get_url_dir_path }}\docs_index.html'
+test_win_get_url_path: '%TEMP%\docs_index.html'
diff --git a/test/integration/targets/win_get_url/tasks/main.yml b/test/integration/targets/win_get_url/tasks/main.yml
--- a/test/integration/targets/win_get_url/tasks/main.yml
+++ b/test/integration/targets/win_get_url/tasks/main.yml
@@ -16,120 +16,128 @@
 # You should have received a copy of the GNU General Public License
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
 
-- name: get tempdir path
-  raw: $env:TEMP
-  register: tempdir
+- setup:
 
-- name: set output path dynamically
-  set_fact:
-    test_win_get_url_dir_path: "{{ tempdir.stdout_lines[0] }}"
-
-- name: remove test file if it exists
-  raw: >
-    PowerShell -Command Remove-Item "{{test_win_get_url_path}}" -Force
-  ignore_errors: true
+- name: Remove test file if it exists
+  win_file:
+    path: '{{ test_win_get_url_path }}'
+    state: absent
 
-- name: test win_get_url module
+- name: Test win_get_url module
   win_get_url:
-    url: "{{test_win_get_url_link}}"
-    dest: "{{test_win_get_url_path}}"
+    url: '{{ test_win_get_url_link }}'
+    dest: '{{ test_win_get_url_path }}'
   register: win_get_url_result
 
-- name: check that url was downloaded
+- name: Check that url was downloaded
   assert:
     that:
-      - "not win_get_url_result|failed"
-      - "win_get_url_result|changed"
-      - "win_get_url_result.win_get_url.url"
-      - "win_get_url_result.win_get_url.dest"
+      - not win_get_url_result|failed
+      - win_get_url_result|changed
+      - win_get_url_result.url
+      - win_get_url_result.dest
 
-- name: test win_get_url module again (force should be yes by default)
+- name: Test win_get_url module again (force should be yes by default)
   win_get_url:
-    url: "{{test_win_get_url_link}}"
-    dest: "{{test_win_get_url_path}}"
+    url: '{{ test_win_get_url_link }}'
+    dest: '{{ test_win_get_url_path }}'
   register: win_get_url_result_again
 
-- name: check that url was downloaded again
+- name: Check that url was downloaded again
   assert:
     that:
-      - "not win_get_url_result_again|failed"
-      - "win_get_url_result_again|changed"
+      - not win_get_url_result_again|failed
+      - win_get_url_result_again|changed
 
-- name: test win_get_url module again with force=no
+- name: Test win_get_url module again with force=no
   win_get_url:
-    url: "{{test_win_get_url_link}}"
-    dest: "{{test_win_get_url_path}}"
+    url: '{{ test_win_get_url_link }}'
+    dest: '{{ test_win_get_url_path }}'
     force: no
   register: win_get_url_result_noforce
 
-- name: check that url was not downloaded again
+- name: Check that url was not downloaded again
   assert:
     that:
-      - "not win_get_url_result_noforce|failed"
-      - "not win_get_url_result_noforce|changed"
+      - not win_get_url_result_noforce|failed
+      - not win_get_url_result_noforce|changed
 
-- name: test win_get_url module with url that returns a 404
+- name: Test win_get_url module with url that returns a 404
   win_get_url:
-    url: "{{test_win_get_url_invalid_link}}"
-    dest: "{{test_win_get_url_path}}"
+    url: '{{ test_win_get_url_invalid_link }}'
+    dest: '{{ test_win_get_url_path }}'
   register: win_get_url_result_invalid_link
   ignore_errors: true
 
-- name: check that the download failed for an invalid url
+- name: Check that the download failed for an invalid url
   assert:
     that:
-      - "win_get_url_result_invalid_link|failed"
+      - win_get_url_result_invalid_link|failed
+      - win_get_url_result_invalid_link.status_code == 404
 
-- name: test win_get_url module with an invalid path
+- name: Test win_get_url module with an invalid path
   win_get_url:
-    url: "{{test_win_get_url_link}}"
-    dest: "{{test_win_get_url_invalid_path}}"
+    url: '{{ test_win_get_url_link }}'
+    dest: '{{ test_win_get_url_invalid_path }}'
   register: win_get_url_result_invalid_path
   ignore_errors: true
 
-- name: check that the download failed for an invalid path
+- name: Check that the download failed for an invalid path
   assert:
     that:
-      - "win_get_url_result_invalid_path|failed"
+      - win_get_url_result_invalid_path|failed
 
-- name: test win_get_url module with a valid path that is a directory
+- name: Test win_get_url module with a valid path that is a directory
   win_get_url:
-    url: "{{test_win_get_url_link}}"
-    dest: "{{test_win_get_url_dir_path}}"
+    url: '{{ test_win_get_url_link }}'
+    dest: '%TEMP%'
   register: win_get_url_result_dir_path
   ignore_errors: true
 
-- name: check that the download did NOT fail, even though dest was directory
+- name: Check that the download did NOT fail, even though dest was directory
   assert:
     that:
-      - "win_get_url_result_dir_path|changed"
+      - win_get_url_result_dir_path|changed
 
-- name: test win_get_url with a valid url path and a dest that is a directory (from 2.4 should use url path as filename)
+- name: Test win_get_url with a valid url path and a dest that is a directory (from 2.4 should use url path as filename)
   win_get_url:
-    url: "{{test_win_get_url_link}}"
-    dest: "{{test_win_get_url_dir_path}}"
+    url: '{{ test_win_get_url_link }}'
+    dest: '%TEMP%'
   register: win_get_url_result_dir_path_urlpath
   ignore_errors: true
 
-- name: set expected destination path fact
+- name: Set expected destination path fact
   set_fact:
-    expected_dest_path: '{{test_win_get_url_dir_path}}\{{test_win_get_url_host}}'
+    expected_dest_path: '{{ ansible_env.TEMP }}\{{ test_win_get_url_host }}'
 
-- name: check that the download succeeded (changed) and dest is as expected
+- name: Check that the download succeeded (changed) and dest is as expected
   assert:
     that:
-      - "win_get_url_result_dir_path_urlpath|changed"
-      - "win_get_url_result_dir_path_urlpath.win_get_url.actual_dest==expected_dest_path"
-
-#- name: since 2.4 check you get a helpful message if the parent folder of the dest doesnt exist
-#  win_get_url:
-#    url: "{{test_win_get_url_link}}"
-#    dest: "{{test_win_get_url_invalid_path_dir}}"
-#  register: win_get_url_result_invalid_dest
-#  ignore_errors: true
-#
-#- name: check if dest parent dir does not exist, module fails and you get a specific error message
-#  assert:
-#    that:
-#      - "win_get_url_result_invalid_dest|failed"
-#      - "win_get_url_result_invalid_dest.msg is search('does not exist')"
+      - win_get_url_result_dir_path_urlpath|changed
+      - win_get_url_result_dir_path_urlpath.dest == expected_dest_path
+
+- name: Check you get a helpful message if the parent folder of the dest doesn't exist
+  win_get_url:
+    url: '{{ test_win_get_url_link }}'
+    dest: 'Q:\Filez\'
+  register: win_get_url_result_invalid_dest
+  ignore_errors: true
+
+- name: Check if dest parent dir does not exist, module fails and you get a specific error message
+  assert:
+    that:
+      - win_get_url_result_invalid_dest|failed
+      - win_get_url_result_invalid_dest.msg is search('invalid path')
+
+- name: Check you get a helpful message if the parent folder of the dest doesn't exist
+  win_get_url:
+    url: '{{ test_win_get_url_link }}'
+    dest: 'C:\Filez\'
+  register: win_get_url_result_invalid_dest2
+  ignore_errors: true
+
+- name: Check if dest parent dir does not exist, module fails and you get a specific error message
+  assert:
+    that:
+      - win_get_url_result_invalid_dest2|failed
+      - win_get_url_result_invalid_dest2.msg is search('does not exist')