Repository: ansible/ansible

Issue:
win_command,win_shell 'removes'/'creates' argument and win_stat does not detect certain System files
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
 - Bug Report


##### COMPONENT NAME
win_command
win_shell
win_stat
+ others?

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.3.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.5 (default, Nov  6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]

```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
Managing from: CentOS
Managed systems: Windows

##### SUMMARY
When using the 'removes' or 'creates' argument with win_command or win_shell, some system files such as pagefile.sys, swapfile.sys or hiberfil.sys are not recognized.
Win_stat also omits these files and reports them as non-existing.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
Example step:

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: disable hibernation (remove hiberfil.sys)
  win_command: powercfg.exe -h off
  args:
    removes: '{{ ansible_env.SystemDrive }}\hiberfil.sys'
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Step should run once and the command removes the hiberfil.sys file (hence it should be skipped the next time).

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
Step is never run and skipped with "skipped, since C:\\hiberfil.sys does not exist" in spite the file does exist on the target system.

<!--- Paste verbatim command output between quotes below -->
```
skipping: [testsystem] => {
    "changed": false, 
    "cmd": "powercfg.exe -h off", 
    "msg": "skipped, since C:\\hiberfil.sys does not exist", 
    "rc": 0, 
    "skipped": true
}
```

##### DETAILS
Problem is Powershell 'Test-Path' command, which does omit certain System files - http://www.powertheshell.com/test-path-ignores-hidden-files/ describes the issue and gives a possible fix: eg. in ansible/lib/ansible/modules/windows/win_command.ps1 change Test-Path from
```
If($removes -and -not $(Test-Path -Path $removes)) {
```
to something like
```
if ($removes -and -not ((Test-Path (Split-Path -Path $removes)) -And ([System.IO.Directory]::EnumerateFiles((Split-Path -Path $removes)) -contains $removes))){
```

It seems that not many windows files are affected, but it is annoying when it occurs and it took quite some for me to figure this out. Since win_stat also relies on 'Test-Path', the only workaround I see for my active issue are some raw commands get check the existence of the file.


Discussion:
cc @dagwieers @jborean93 @jhawkesworth
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Arghh that's really annoying, there are quite a few modules that use `Test-Path` but not all of them are really affected by the system files.

Thanks for raising the issue and bringing it to our attention.

Resolving patch:
diff --git a/lib/ansible/modules/windows/win_stat.py b/lib/ansible/modules/windows/win_stat.py
--- a/lib/ansible/modules/windows/win_stat.py
+++ b/lib/ansible/modules/windows/win_stat.py
@@ -44,6 +44,7 @@
               use specified algorithm.
             - This option is deprecated in Ansible 2.3 and is replaced with
               C(checksum_algorithm=md5).
+            - This option will be removed in Ansible 2.7
         required: no
         default: True
     get_checksum:
@@ -199,7 +200,7 @@
             type: string
             sample: C:\temp
         md5:
-            description: The MD5 checksum of a file (Between Ansible 1.9 and 2.2 this was returned as a SHA1 hash)
+            description: The MD5 checksum of a file (Between Ansible 1.9 and 2.2 this was returned as a SHA1 hash), will be removed in 2.7
             returned: success, path exist, path is a file, get_md5 == True, md5 is supported
             type: string
             sample: 09cb79e8fc7453c84a07f644e441fd81623b7f98


Test patch:
diff --git a/test/integration/targets/win_command/tasks/main.yml b/test/integration/targets/win_command/tasks/main.yml
--- a/test/integration/targets/win_command/tasks/main.yml
+++ b/test/integration/targets/win_command/tasks/main.yml
@@ -77,6 +77,18 @@
     - cmdout|skipped
     - cmdout.msg is search('exists')
 
+- name: test creates with hidden system file, should skip
+  win_command: echo no
+  args:
+    creates: C:\pagefile.sys
+  register: cmdout
+
+- name: validate result
+  assert:
+    that:
+    - cmdout|skipped
+    - cmdout.msg is search('exists')
+
 - name: ensure testfile is still present
   win_stat:
     path: c:\testfile.txt
diff --git a/test/integration/targets/win_module_utils/library/file_util_test.ps1 b/test/integration/targets/win_module_utils/library/file_util_test.ps1
new file mode 100644
--- /dev/null
+++ b/test/integration/targets/win_module_utils/library/file_util_test.ps1
@@ -0,0 +1,59 @@
+#!powershell
+
+#Requires -Module Ansible.ModuleUtils.Legacy
+#Requires -Module Ansible.ModuleUtils.FileUtil
+
+Function Assert-Equals($actual, $expected) {
+    if ($actual -cne $expected) {
+        Fail-Json @{} "actual != expected`nActual: $actual`nExpected: $expected"
+    }
+}
+
+# Test-FilePath Hidden system file
+$actual = Test-FilePath -path C:\pagefile.sys
+Assert-Equals -actual $actual -expected $true
+
+# Test-FilePath File that doesn't exist
+$actual = Test-FilePath -path C:\fakefile
+Assert-Equals -actual $actual -expected $false
+
+# Test-FilePath Normal directory
+$actual = Test-FilePath -path C:\Windows
+Assert-Equals -actual $actual -expected $true
+
+# Test-FilePath Normal file
+$actual = Test-FilePath -path C:\Windows\System32\kernel32.dll
+
+# Test-FilePath fails with wildcard
+try {
+    Test-FilePath -Path C:\Windows\*.exe
+    Fail-Json @{} "exception was not thrown with wildcard search for Test-FilePath"
+} catch {
+    Assert-Equals -actual $_.Exception.Message -expected "found multiple files at path 'C:\Windows\*.exe', make sure no wildcards are set in the path"
+}
+
+# Get-FileItem file
+$actual = Get-FileItem -path C:\pagefile.sys
+Assert-Equals -actual $actual.FullName -expected C:\pagefile.sys
+Assert-Equals -actual $actual.PSIsContainer -expected $false
+Assert-Equals -actual $actual.Exists -expected $true
+
+# Get-FileItem directory
+$actual = Get-FileItem -path C:\Windows
+Assert-Equals -actual $actual.FullName -expected C:\Windows
+Assert-Equals -actual $actual.PSIsContainer -expected $true
+Assert-Equals -actual $actual.Exists -expected $true
+
+# Get-FileItem doesn't exists
+$actual = Get-FileItem -path C:\fakefile
+Assert-Equals -actual $actual -expected $null
+
+# Get-FileItem fails with wildcard
+try {
+    Get-FileItem -Path C:\Windows\*.exe
+    Fail-Json @{} "exception was not thrown with wildcard search for Get-FileItem"
+} catch {
+    Assert-Equals -actual $_.Exception.Message -expected "found multiple files at path 'C:\Windows\*.exe', make sure no wildcards are set in the path"
+}
+
+Exit-Json @{ data = 'success' }
diff --git a/test/integration/targets/win_module_utils/tasks/main.yml b/test/integration/targets/win_module_utils/tasks/main.yml
--- a/test/integration/targets/win_module_utils/tasks/main.yml
+++ b/test/integration/targets/win_module_utils/tasks/main.yml
@@ -80,3 +80,11 @@
   win_file:
     path: C:\ansible testing
     state: absent
+
+- name: call module with FileUtil tests
+  file_util_test:
+  register: file_util_test
+
+- assert:
+    that:
+    - file_util_test.data == 'success'
diff --git a/test/integration/targets/win_shell/tasks/main.yml b/test/integration/targets/win_shell/tasks/main.yml
--- a/test/integration/targets/win_shell/tasks/main.yml
+++ b/test/integration/targets/win_shell/tasks/main.yml
@@ -105,6 +105,18 @@
     - shellout|skipped
     - shellout.msg is search('exists')
 
+- name: test creates with hidden system file, should skip
+  win_shell: echo test
+  args:
+    creates: C:\pagefile.sys
+  register: shellout
+
+- name: validate result
+  assert:
+    that:
+    - shellout|skipped
+    - shellout.msg is search('exists')
+
 - name: ensure testfile is still present
   win_stat:
     path: c:\testfile.txt
diff --git a/test/integration/targets/win_stat/tasks/main.yml b/test/integration/targets/win_stat/tasks/main.yml
--- a/test/integration/targets/win_stat/tasks/main.yml
+++ b/test/integration/targets/win_stat/tasks/main.yml
@@ -295,6 +295,9 @@
     - stat_readonly.stat.size == 3
 
 # Requires more work once modular powershell utils are in
+- name: weird issue, need to access the file in anyway to get the correct date stats
+  win_command: powershell.exe Test-Path {{win_output_dir}}\win_stat\nested\hard-link.ps1
+
 - name: test win_stat on hard link file
   win_stat:
     path: "{{win_output_dir}}\\win_stat\\nested\\hard-link.ps1"
@@ -541,3 +544,13 @@
   win_file:
     path: "{{win_output_dir}}\\win_stat"
     state: absent
+
+- name: get stat of pagefile
+  win_stat:
+    path: C:\pagefile.sys
+  register: pagefile_stat
+
+- name: assert get stat of pagefile
+  assert:
+    that:
+    - pagefile_stat.stat.exists == True
diff --git a/test/integration/targets/windows-paths/tasks/main.yml b/test/integration/targets/windows-paths/tasks/main.yml
--- a/test/integration/targets/windows-paths/tasks/main.yml
+++ b/test/integration/targets/windows-paths/tasks/main.yml
@@ -82,7 +82,7 @@
     - trailing_result|success
     - trailing_result.stat.attributes == 'Directory'
     - trailing_result.stat.exists == true
-    - trailing_result.stat.path == trailing
+    - trailing_result.stat.path == no_quotes_single  # path is without the trailing \
 
 - name: Set variables in key=value syntax
   set_fact:
@@ -178,7 +178,7 @@
     - trailing_result|success
     - trailing_result.stat.attributes == 'Directory'
     - trailing_result.stat.exists == true
-    - trailing_result.stat.path == trailing
+    - trailing_result.stat.path == no_quotes_single  # path is without the trailing \
 
 - name: Test tab path {{ tab }}
   win_stat: