Repository: ansible/ansible

Issue:
get_url should check for URL existence in check mode
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Feature Idea


##### COMPONENT NAME
get_url plugin/library

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.2.0.0
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->
N/A

##### SUMMARY
<!--- Explain the problem briefly -->
Check mode for get_url should make sure the URL to fetch exists

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml

```
Ran get_url using variables to construct a URL that did not exist for some hosts

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Expected check mode to fail on nonexistent URL

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
Check mode is skipped
<!--- Paste verbatim command output between quotes below -->
```

```



Discussion:

Resolving patch:
diff --git a/lib/ansible/modules/net_tools/basics/get_url.py b/lib/ansible/modules/net_tools/basics/get_url.py
--- a/lib/ansible/modules/net_tools/basics/get_url.py
+++ b/lib/ansible/modules/net_tools/basics/get_url.py
@@ -40,6 +40,8 @@
        or by using the use_proxy option.
      - HTTP redirects can redirect from HTTP to HTTPS so you should be sure that
        your proxy environment for both protocols is correct.
+     - From Ansible 2.4 when run with C(--check), it will do a HEAD request to validate the URL but
+       will not download the entire file or verify it against hashes.
 version_added: "0.6"
 options:
   url:
@@ -241,8 +243,12 @@ def url_get(module, url, dest, use_proxy, last_mod_time, force, timeout=10, head
 
     Return (tempfile, info about the request)
     """
+    if module.check_mode:
+        method='HEAD'
+    else:
+        method='GET'
 
-    rsp, info = fetch_url(module, url, use_proxy=use_proxy, force=force, last_mod_time=last_mod_time, timeout=timeout, headers=headers)
+    rsp, info = fetch_url(module, url, use_proxy=use_proxy, force=force, last_mod_time=last_mod_time, timeout=timeout, headers=headers, method=method)
 
     if info['status'] == 304:
         module.exit_json(url=url, dest=dest, changed=False, msg=info.get('msg', ''))
@@ -318,7 +324,8 @@ def main():
     module = AnsibleModule(
         # not checking because of daisy chain to file module
         argument_spec = argument_spec,
-        add_file_common_args=True
+        add_file_common_args=True,
+        supports_check_mode=True,
     )
 
     url  = module.params['url']
@@ -412,6 +419,12 @@ def main():
     checksum_src   = None
     checksum_dest  = None
 
+    # If the remote URL exists, we're done with check mode
+    if module.check_mode:
+        os.remove(tmpsrc)
+        res_args = dict( url = url, dest = dest, src = tmpsrc, changed = True, msg = info.get('msg', ''))
+        module.exit_json(**res_args)
+
     # raise an error if there is no tmpsrc file
     if not os.path.exists(tmpsrc):
         os.remove(tmpsrc)


Test patch:
diff --git a/test/integration/targets/get_url/tasks/main.yml b/test/integration/targets/get_url/tasks/main.yml
--- a/test/integration/targets/get_url/tasks/main.yml
+++ b/test/integration/targets/get_url/tasks/main.yml
@@ -65,6 +65,28 @@
     that:
         - result.failed
 
+- name: test HTTP HEAD request for file in check mode
+  get_url: url="http://{{ httpbin_host }}/get" dest={{ output_dir }}/get_url_check.txt force=yes
+  check_mode: True
+  register: result
+
+- name: assert that the HEAD request was successful in check mode
+  assert:
+    that:
+    - result.changed
+    - '"OK" in result.msg'
+
+- name: test HTTP HEAD for nonexistent URL in check mode
+  get_url: url="http://{{ httpbin_host }}/DOESNOTEXIST" dest={{ output_dir }}/shouldnotexist.html force=yes
+  check_mode: True
+  register: result
+  ignore_errors: True
+
+- name: assert that HEAD request for nonexistent URL failed
+  assert:
+    that:
+    - result.failed
+
 - name: test https fetch
   get_url: url="https://{{ httpbin_host }}/get" dest={{output_dir}}/get_url.txt force=yes
   register: result