Repository: ansible/ansible

Issue:
Broken checksumming with tcsh as login shell
It looks like commit 8150b14d8d4a1d49a80ff9b4ace660ba4d03141e broke dry-run execution for template module, as backslash escaping is unsupported feature in csh/tcsh shell family.



Discussion:
This is also broken when using Pypy as the remove python - tested with 1.8.4 works fine if you downgrade to 1.8.2

@pixie79 Please open a separate ticket for your problem.  It's different than the csh one.  Also note that I was unable to reproduce the pypy issue here.  So we'll need a lot more information about how you have things setup.  Where is pypy located (You said in the user's home dir on IRC).  Is that path on the user's PATH?  Are you using ansible_python_interpreter?  and anything else you can think of for us to reproduce.

@lonerr can you give more information about what behaviour you're seeing and how to reproduce it?  I see that --check is reporting that the file will be changed when it won't be but I'm not sure if that's the sole problem you're seeing or not.

@abadger, yes I see the same behaviour. Running ansible-playbook with '--check --diff' options shows the changes (yellow coloured output) without the actual diff lines.


Resolving patch:
diff --git a/lib/ansible/runner/shell_plugins/csh.py b/lib/ansible/runner/shell_plugins/csh.py
--- a/lib/ansible/runner/shell_plugins/csh.py
+++ b/lib/ansible/runner/shell_plugins/csh.py
@@ -19,5 +19,8 @@
 
 class ShellModule(ShModule):
 
+    # How to end lines in a python script one-liner
+    _SHELL_EMBEDDED_PY_EOL = '\\\n'
+
     def env_prefix(self, **kwargs):
         return 'env %s' % super(ShellModule, self).env_prefix(**kwargs)
diff --git a/lib/ansible/runner/shell_plugins/sh.py b/lib/ansible/runner/shell_plugins/sh.py
--- a/lib/ansible/runner/shell_plugins/sh.py
+++ b/lib/ansible/runner/shell_plugins/sh.py
@@ -24,6 +24,9 @@
 
 class ShellModule(object):
 
+    # How to end lines in a python script one-liner
+    _SHELL_EMBEDDED_PY_EOL = '\n'
+
     def env_prefix(self, **kwargs):
         '''Build command prefix with environment variables.'''
         env = dict(
@@ -103,14 +106,19 @@ def checksum(self, path, python_interp):
         # 2: No read permissions on the file
         # 3: File is a directory
         # 4: No python interpreter
-        test = "rc=flag; [ -r \'%(p)s\' ] || rc=2; [ -f \'%(p)s\' ] || rc=1; [ -d \'%(p)s\' ] && rc=3; %(i)s -V 2>/dev/null || rc=4; [ x\"$rc\" != \"xflag\" ] && echo \"${rc}\"\'  %(p)s\' && exit 0" % dict(p=path, i=python_interp)
+
+        # Quoting gets complex here.  We're writing a python string that's
+        # used by a variety of shells on the remote host to invoke a python
+        # "one-liner".
+        shell_escaped_path = pipes.quote(path)
+        test = "rc=flag; [ -r %(p)s ] || rc=2; [ -f %(p)s ] || rc=1; [ -d %(p)s ] && rc=3; %(i)s -V 2>/dev/null || rc=4; [ x\"$rc\" != \"xflag\" ] && echo \"${rc}  \"%(p)s && exit 0" % dict(p=shell_escaped_path, i=python_interp)
         csums = [
-            "(%s -c 'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();\nafile = open(\"%s\", \"rb\")\nbuf = afile.read(BLOCKSIZE)\nwhile len(buf) > 0:\n\thasher.update(buf)\n\tbuf = afile.read(BLOCKSIZE)\nafile.close()\nprint(hasher.hexdigest())' 2>/dev/null)" % (python_interp, path),      # Python > 2.4 (including python3)
-            "(%s -c 'import sha; BLOCKSIZE = 65536; hasher = sha.sha();\nafile = open(\"%s\", \"rb\")\nbuf = afile.read(BLOCKSIZE)\nwhile len(buf) > 0:\n\thasher.update(buf)\n\tbuf = afile.read(BLOCKSIZE)\nafile.close()\nprint(hasher.hexdigest())' 2>/dev/null)" % (python_interp, path),      # Python == 2.4
+            "({0} -c 'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();{2}afile = open(\"'{1}'\", \"rb\"){2}buf = afile.read(BLOCKSIZE){2}while len(buf) > 0:{2}\thasher.update(buf){2}\tbuf = afile.read(BLOCKSIZE){2}afile.close(){2}print(hasher.hexdigest())' 2>/dev/null)".format(python_interp, shell_escaped_path, self._SHELL_EMBEDDED_PY_EOL),      # Python > 2.4 (including python3)
+            "({0} -c 'import sha; BLOCKSIZE = 65536; hasher = sha.sha();{2}afile = open(\"'{1}'\", \"rb\"){2}buf = afile.read(BLOCKSIZE){2}while len(buf) > 0:{2}\thasher.update(buf){2}\tbuf = afile.read(BLOCKSIZE){2}afile.close(){2}print(hasher.hexdigest())' 2>/dev/null)".format(python_interp, shell_escaped_path, self._SHELL_EMBEDDED_PY_EOL),      # Python == 2.4
         ]
 
         cmd = " || ".join(csums)
-        cmd = "%s; %s || (echo \'0  %s\')" % (test, cmd, path)
+        cmd = "%s; %s || (echo \'0  \'%s)" % (test, cmd, shell_escaped_path)
         return cmd
 
     def build_module_command(self, env_string, shebang, cmd, rm_tmp=None):


Test patch:
diff --git a/test/integration/check_mode.yml b/test/integration/check_mode.yml
--- a/test/integration/check_mode.yml
+++ b/test/integration/check_mode.yml
@@ -1,3 +1,4 @@
 - hosts: testhost
   roles:
   - { role: test_always_run, tags: test_always_run }
+  - { role: test_check_mode, tags: test_check_mode }
diff --git a/test/integration/roles/prepare_tests/tasks/main.yml b/test/integration/roles/prepare_tests/tasks/main.yml
--- a/test/integration/roles/prepare_tests/tasks/main.yml
+++ b/test/integration/roles/prepare_tests/tasks/main.yml
@@ -19,11 +19,13 @@
 
 - name: clean out the test directory
   file: name={{output_dir|mandatory}} state=absent
+  always_run: True
   tags:
     - prepare
 
 - name: create the test directory
   file: name={{output_dir}} state=directory
+  always_run: True
   tags:
     - prepare
 
diff --git a/test/integration/roles/test_check_mode/files/foo.txt b/test/integration/roles/test_check_mode/files/foo.txt
new file mode 100644
--- /dev/null
+++ b/test/integration/roles/test_check_mode/files/foo.txt
@@ -0,0 +1 @@
+templated_var_loaded
diff --git a/test/integration/roles/test_check_mode/meta/main.yml b/test/integration/roles/test_check_mode/meta/main.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/roles/test_check_mode/meta/main.yml
@@ -0,0 +1,3 @@
+dependencies: 
+  - prepare_tests
+
diff --git a/test/integration/roles/test_check_mode/tasks/main.yml b/test/integration/roles/test_check_mode/tasks/main.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/roles/test_check_mode/tasks/main.yml
@@ -0,0 +1,39 @@
+# test code for the template module
+# (c) 2014, Michael DeHaan <michael.dehaan@gmail.com>
+
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+- name: fill in a basic template in check mode
+  template: src=foo.j2 dest={{output_dir}}/foo.templated mode=0644
+  register: template_result
+
+- name: verify that the file was marked as changed in check mode
+  assert: 
+    that: 
+      - "template_result.changed == true"
+
+- name: Actually create the file
+  template: src=foo.j2 dest={{output_dir}}/foo.templated2 mode=0644
+  always_run: True
+
+- name: fill in a basic template in check mode
+  template: src=foo.j2 dest={{output_dir}}/foo.templated2 mode=0644
+  register: template_result2
+
+- name: verify that the file was marked as not changed in check mode
+  assert: 
+    that: 
+      - "template_result2.changed == false"
diff --git a/test/integration/roles/test_check_mode/templates/foo.j2 b/test/integration/roles/test_check_mode/templates/foo.j2
new file mode 100644
--- /dev/null
+++ b/test/integration/roles/test_check_mode/templates/foo.j2
@@ -0,0 +1 @@
+{{ templated_var }}
diff --git a/test/integration/roles/test_check_mode/vars/main.yml b/test/integration/roles/test_check_mode/vars/main.yml
new file mode 100644
--- /dev/null
+++ b/test/integration/roles/test_check_mode/vars/main.yml
@@ -0,0 +1 @@
+templated_var: templated_var_loaded
diff --git a/test/integration/roles/test_template/tasks/main.yml b/test/integration/roles/test_template/tasks/main.yml
--- a/test/integration/roles/test_template/tasks/main.yml
+++ b/test/integration/roles/test_template/tasks/main.yml
@@ -138,3 +138,39 @@
     that:
       - "template_result.changed == False"
 
+# Test strange filenames
+
+- name: Create a temp dir for filename tests
+  file:
+    state: directory
+    dest: '{{ output_dir }}/filename-tests'
+
+- name: create a file with an unusual filename
+  template:
+    src: foo.j2
+    dest: "{{ output_dir }}/filename-tests/foo t'e~m\\plated"
+  register: template_result
+
+- assert:
+    that:
+      - "template_result.changed == True"
+
+- name: check that the unusual filename was created
+  command: "ls {{ output_dir }}/filename-tests/"
+  register: unusual_results
+
+- assert:
+    that:
+      - "\"foo t'e~m\\plated\" in unusual_results.stdout_lines"
+      - "{{unusual_results.stdout_lines| length}} == 1"
+
+- name: check that the unusual filename can be checked for changes
+  template:
+    src: foo.j2
+    dest: "{{ output_dir }}/filename-tests/foo t'e~m\\plated"
+  register: template_result
+
+- assert:
+    that:
+      - "template_result.changed == False"
+