Repository: ansible/ansible

Issue:
Powershell scripts that use Start-Job fail on Ansible 2.0+
<!--- Verify first that your issue/request is not already reported in GitHub -->
##### ISSUE TYPE

<!--- Pick one below and delete the rest: -->
- Bug Report

[dacpac.txt](https://github.com/ansible/ansible/files/253976/dacpac.txt)
##### ANSIBLE VERSION

```
<!--- Paste verbatim output from “ansible --version” between quotes -->
ansible 2.0.1.0
```
##### CONFIGURATION

<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
##### OS / ENVIRONMENT

<!---
centos 7.2
-->
##### SUMMARY

<!--- Explain the problem briefly -->

task that work in ansible 1.9.4 not working in 2.0  
(TASK [dbonline : Run dacpac on Infra_Telephony, JajahCore, jajah, jajahccsc, jajahbilling, JajahSupportTool DBs] ***)

failed with error below 
failed: [rndv-ci1-con02] => (item=default) => {"failed": true, "invocation": {"module_name": "dacpac"}, "item": "default", "msg": "*****_Action=Publish DacPackPackage=E:\temp\workarea\rndv-ci1-con02\dbonline-10.18.0-SNAPSHOT\dbonline\schemas\Jajah.dacpac,E:\temp\workarea\rndv-ci1-con02\dbonline-10.18.0-SNAPSHOT\dbonline\schemas\JajahCore.dacpac,E:\temp\workarea\rndv-ci1-con02\dbonline-10.18.0-SNAPSHOT\dbonline\schemas\Infra_Telephony.dacpac,E:\temp\workarea\rndv-ci1-con02\dbonline-10.18.0-SNAPSHOT\dbonline\schemas\JajahCcsc.dacpac,E:\temp\workarea\rndv-ci1-con02\dbonline-10.18.0-SNAPSHOT\dbonline\schemas\jajahbilling.dacpac,E:\temp\workarea\rndv-ci1-con02\dbonline-10.18.0-SNAPSHOT\dbonline\schemas\JajahSupportTool.dacpac SqlServer=rndv-ci1-con02 DBInstancePort=1500**_*****_**Job has failed: The background process reported an error with the following message: Cannot process an element with node type \"Text\". Only Element and EndElement node types are supported.   at System.Management.Automation.Remoting.OutOfProcessUtils.ProcessData(String data, DataProcessingDelegates callbacks). [localhost] The background process reported an error with the following message: Cannot process an element with node type \"Text\". Only Element and EndElement node types are supported.   at System.Management.Automation.Remoting.OutOfProcessUtils.ProcessData(String data, DataProcessingDelegates callbacks). ScriptHalted ScriptHalted ScriptHalted ScriptHalted ScriptHalted dacpak failed with exitcode  *__"}_*
### STEPS TO REPRODUCE

<!---
running the command 
ansible-playbook dbonline.yml -i gated_env/rndv-ci1/hosts -e "inp_deploy_version=10.18.0-SNAPSHOT" -vvvvv

-->

```
<!--- Paste example playbooks or commands between quotes -->
```

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

<!--- What did you expect to happen when running the steps above? -->
##### ACTUAL RESULTS

<!--- What actually happened? If possible run with high verbosity (-vvvv) -->

```
<!--- Paste verbatim command output between quotes -->
```



Discussion:
Ansible 2.0 enforces powershell strict mode for windows modules, this is the most likely cause of your failures.  See https://technet.microsoft.com/en-us/library/hh849692.aspx?f=255&MSPPError=-2147217396

Once you have modified your code for compliance with strict mode, if you still having problems, please ask on the ansible-project google group or #ansible irc channel.

Closing this so we can keep the github issue tracker for ansible defects.

thanks
i forgot to tell that i set in all relevant places (powershell.ps1
,powershell.oy) , winrm.py - Set-StrictMode -Off  and it still happens.

Best regards
zahi

On Sun, May 8, 2016 at 7:44 PM, jhawkesworth notifications@github.com
wrote:

> Ansible 2.0 enforces powershell strict mode for windows modules, this is
> the most likely cause of your failures. See
> https://technet.microsoft.com/en-us/library/hh849692.aspx?f=255&MSPPError=-2147217396
> 
> Once you have modified your code for compliance with strict mode, if you
> still having problems, please ask on the ansible-project google group or
> #ansible irc channel.
> 
> Closing this so we can keep the github issue tracker for ansible defects.
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly or view it on GitHub
> https://github.com/ansible/ansible/issues/15770#issuecomment-217731808

hi,
i also saw the in your modules (win_get_url.ps1 ,win_acl.ps1
 ,win_environment.ps1 ,win_iis_webbinding.ps1
,win_scheduled_task.ps1,win_unzip.ps1) you set it to off so why we can't do
it to our code?

thanks
zahi

On Sun, May 8, 2016 at 7:44 PM, jhawkesworth notifications@github.com
wrote:

> Ansible 2.0 enforces powershell strict mode for windows modules, this is
> the most likely cause of your failures. See
> https://technet.microsoft.com/en-us/library/hh849692.aspx?f=255&MSPPError=-2147217396
> 
> Once you have modified your code for compliance with strict mode, if you
> still having problems, please ask on the ansible-project google group or
> #ansible irc channel.
> 
> Closing this so we can keep the github issue tracker for ansible defects.
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly or view it on GitHub
> https://github.com/ansible/ansible/issues/15770#issuecomment-217731808

You can turn of strict mode in your modules.  But since ansible 2.0 expects
that modules will comply with strict mode, it is most likely the cause of
why your module is not working after upgrading from 1.9.4.

On 9 May 2016 at 09:39, ZahiCahana notifications@github.com wrote:

> hi,
> i also saw the in your modules (win_get_url.ps1 ,win_acl.ps1
> ,win_environment.ps1 ,win_iis_webbinding.ps1
> ,win_scheduled_task.ps1,win_unzip.ps1) you set it to off so why we can't do
> it to our code?
> 
> thanks
> 
> zahi
> 
> On Sun, May 8, 2016 at 7:44 PM, jhawkesworth notifications@github.com
> wrote:
> 
> > Ansible 2.0 enforces powershell strict mode for windows modules, this is
> > the most likely cause of your failures. See
> > 
> > https://technet.microsoft.com/en-us/library/hh849692.aspx?f=255&MSPPError=-2147217396
> > 
> > Once you have modified your code for compliance with strict mode, if you
> > still having problems, please ask on the ansible-project google group or
> > #ansible irc channel.
> > 
> > Closing this so we can keep the github issue tracker for ansible defects.
> > 
> > —
> > You are receiving this because you authored the thread.
> > Reply to this email directly or view it on GitHub
> > https://github.com/ansible/ansible/issues/15770#issuecomment-217731808
> 
> —
> You are receiving this because you modified the open/close state.
> Reply to this email directly or view it on GitHub
> https://github.com/ansible/ansible/issues/15770#issuecomment-217807329

We've had other reports of this problem as well- I narrowed it down to a change that occurred with 2.0 where we're setting the WinRM codepage to UTF-8 (65001) in 2.x so we can properly handle any non-ASCII input, where in 1.x it was hardcoded to Western European (437). It appears that Powershell's background job API is busted under the UTF-8 codepage. You can reproduce this outside of Ansible by opening cmd.exe and running the following:

```
chcp 65001
powershell
(run your script here)
```

We're going to look into options here, but it appears to be a Microsoft bug, and a lot more things are broken for international text by having a non-UTF-8 codepage (thus we're probably not going to switch back to the pre-2.0 behavior), so I'm not really optimistic at this point.

I've created a Uservoice issue for this, feel free to upvote at: https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/14915283-job-cmdlets-fail-with-utf-8-codepage

Someone's already posted a workaround on the Uservoice page that switches the InputEncoding to not expect a BOM; that fixes the standalone case, I just need to try plugging that into our shell setup, but hopefully should work great!


Resolving patch:
diff --git a/lib/ansible/plugins/shell/powershell.py b/lib/ansible/plugins/shell/powershell.py
--- a/lib/ansible/plugins/shell/powershell.py
+++ b/lib/ansible/plugins/shell/powershell.py
@@ -124,6 +124,9 @@
         $ps.AddCommand("Out-Null") | Out-Null
     }
 
+    # force input encoding to preamble-free UTF8 so PS sub-processes (eg, Start-Job) don't blow up
+    $ps.AddStatement().AddScript("[Console]::InputEncoding = New-Object Text.UTF8Encoding `$false") | Out-Null
+
     $ps.AddStatement().AddScript($entrypoint) | Out-Null
 
     $output = $ps.Invoke()


Test patch:
diff --git a/test/integration/targets/win_raw/tasks/main.yml b/test/integration/targets/win_raw/tasks/main.yml
--- a/test/integration/targets/win_raw/tasks/main.yml
+++ b/test/integration/targets/win_raw/tasks/main.yml
@@ -115,3 +115,13 @@
     that:
       - "not raw_with_items_result|failed"
       - "raw_with_items_result.results|length == 32"
+
+- name: test raw with job to ensure that preamble-free InputEncoding is working
+  raw: Start-Job { echo yo } | Receive-Job -Wait
+  register: raw_job_result
+
+- name: check raw with job result
+  assert:
+    that:
+    - raw_job_result | succeeded
+    - raw_job_result.stdout_lines[0] == 'yo'
diff --git a/test/integration/targets/win_shell/tasks/main.yml b/test/integration/targets/win_shell/tasks/main.yml
--- a/test/integration/targets/win_shell/tasks/main.yml
+++ b/test/integration/targets/win_shell/tasks/main.yml
@@ -165,6 +165,16 @@
     - shellout.stdout_lines == ["line1 ", "line2"]
     - shellout.stderr == ""
 
+- name: test with job to ensure that preamble-free InputEncoding is working
+  win_shell: Start-Job { echo yo } | Receive-Job -Wait
+  register: shellout
+
+- name: check job result
+  assert:
+    that:
+    - shellout | succeeded
+    - shellout.stdout_lines[0] == 'yo'
+
 - name: interleave large writes between stdout/stderr (check for buffer consumption deadlock)
   win_shell: $ba = New-Object byte[] 4096; (New-Object System.Random 32).NextBytes($ba); $text = [Convert]::ToBase64String($ba); Write-Output startout; Write-Error starterror; Write-Error $text; Write-Output $text; Write-Error $text; Write-Output $text; Write-Error $text; Write-Output $text; Write-Output doneout Write-Error doneerror
   register: shellout