Repository: ansible/ansible

Issue:
add a regex Jinja2 filter
Folks would like to do

```
foo | search(some_regex)
```

in expressions from Jinja2 and re.search does not work.

Need to test if foo.search is already avail in regular templates and then it may just be a matter of importing in playbooks to enable foo.search



Discussion:
Jinja2 does not have any regex filters.  I have written `match` and `search` filters for use in my playbooks.  However for my use case, I am returning bool().  What are others looking for?  Would you prefer match groups or are sticking with booleans sufficient?

When statements want a True/False value, so I think that would cover nearly all of the cases, want to submit a pull request for "core.py" ?

Absolutely.  I am putting some final touches on the code and adding test cases.

Pull request has been added: https://github.com/ansible/ansible/pull/4288

``` shell
$ nosetests test/
.........................................................................S..............................................
----------------------------------------------------------------------
Ran 120 tests in 28.143s

OK (SKIP=1)
```

``` shell
$ nosetests -v test/TestFilters.py
test_bool_no (TestFilters.TestFilters) ... ok
test_bool_none (TestFilters.TestFilters) ... ok
test_bool_true (TestFilters.TestFilters) ... ok
test_bool_yes (TestFilters.TestFilters) ... ok
test_match_case_insensitive (TestFilters.TestFilters) ... ok
test_match_case_sensitive (TestFilters.TestFilters) ... ok
test_match_no_match (TestFilters.TestFilters) ... ok
test_quotes (TestFilters.TestFilters) ... ok
test_regex (TestFilters.TestFilters) ... ok
test_search_case_insensitive (TestFilters.TestFilters) ... ok
test_search_case_sensitive (TestFilters.TestFilters) ... ok

----------------------------------------------------------------------
Ran 11 tests in 0.005s

OK
```

closing as we have the PR open now, thanks @sivel!

I could not find documentation on this on Ansible doc so I ended up looking for a regex filter on Jinja2 side... and ended up here!

Could we add that in the doc? I can submit a patch if you tell me where you want to see this appear plus anything that can help me with style (like if you see it as a new section, what kind of info, ...).

The regex filter docs can be found in the section at the following URL:

http://docs.ansible.com/playbooks_variables.html#other-useful-filters

Wow, I need better glasses... Thanks and sorry for the noise!

Just a quick update for those that land here, the docs for these filters are now here:
http://docs.ansible.com/playbooks_filters.html#other-useful-filters


Resolving patch:
diff --git a/lib/ansible/plugins/action/__init__.py b/lib/ansible/plugins/action/__init__.py
--- a/lib/ansible/plugins/action/__init__.py
+++ b/lib/ansible/plugins/action/__init__.py
@@ -428,7 +428,7 @@ def _execute_remote_stat(self, path, all_vars, follow, tmp=None):
 
         return mystat['stat']
 
-    def _remote_checksum(self, path, all_vars):
+    def _remote_checksum(self, path, all_vars, follow=False):
         '''
         Produces a remote checksum given a path,
         Returns a number 0-4 for specific errors instead of checksum, also ensures it is different
@@ -440,7 +440,7 @@ def _remote_checksum(self, path, all_vars):
         '''
         x = "0" # unknown error has occured
         try:
-            remote_stat = self._execute_remote_stat(path, all_vars, follow=False)
+            remote_stat = self._execute_remote_stat(path, all_vars, follow=follow)
             if remote_stat['exists'] and remote_stat['isdir']:
                 x = "3" # its a directory not a file
             else:
diff --git a/lib/ansible/plugins/action/unarchive.py b/lib/ansible/plugins/action/unarchive.py
--- a/lib/ansible/plugins/action/unarchive.py
+++ b/lib/ansible/plugins/action/unarchive.py
@@ -71,7 +71,7 @@ def run(self, tmp=None, task_vars=None):
             else:
                 source = self._loader.path_dwim_relative(self._loader.get_basedir(), 'files', source)
 
-        remote_checksum = self._remote_checksum(dest, all_vars=task_vars)
+        remote_checksum = self._remote_checksum(dest, all_vars=task_vars, follow=True)
         if remote_checksum == '4':
             result['failed'] = True
             result['msg'] = "python isn't present on the system.  Unable to compute checksum"
diff --git "a/test/integration/roles/test_unarchive/files/test-unarchive-nonascii-a\314\200a\314\202\303\246c\314\247e\314\201e\314\200i\314\210i\314\202o\314\202\305\223.tar.gz" "b/test/integration/roles/test_unarchive/files/test-unarchive-nonascii-a\314\200a\314\202\303\246c\314\247e\314\201e\314\200i\314\210i\314\202o\314\202\305\223.tar.gz"
new file mode 100644
Binary files /dev/null and "b/test/integration/roles/test_unarchive/files/test-unarchive-nonascii-a\314\200a\314\202\303\246c\314\247e\314\201e\314\200i\314\210i\314\202o\314\202\305\223.tar.gz" differ


Test patch:
diff --git a/test/integration/roles/test_unarchive/tasks/main.yml b/test/integration/roles/test_unarchive/tasks/main.yml
--- a/test/integration/roles/test_unarchive/tasks/main.yml
+++ b/test/integration/roles/test_unarchive/tasks/main.yml
@@ -21,9 +21,9 @@
   yum: name=zip state=latest
   when: ansible_pkg_mgr  ==  'yum'
 
-- name: Ensure zip is present to create test archive (dnf)
-  dnf: name=zip state=latest
-  when: ansible_pkg_mgr  ==  'dnf'
+  #- name: Ensure zip is present to create test archive (dnf)
+  #  dnf: name=zip state=latest
+  #  when: ansible_pkg_mgr  ==  'dnf'
 
 - name: Ensure zip is present to create test archive (apt)
   apt: name=zip state=latest
@@ -287,6 +287,35 @@
 - name: remove quotable chars test
   file: path="{{ output_dir }}/test-quotes~root" state=absent
 
+- name: create our unarchive destination
+  file:
+    path: "{{ output_dir }}/test-unarchive-nonascii-àâæçéèïîôœ-tar-gz"
+    state: directory
+
+- name: test that unarchive works with an archive that contains non-ascii filenames
+  unarchive:
+    # Both the filename of the tarball and the filename inside the tarball have
+    # nonascii chars
+    src: "test-unarchive-nonascii-àâæçéèïîôœ.tar.gz"
+    dest: "{{ output_dir }}/test-unarchive-nonascii-àâæçéèïîôœ-tar-gz"
+    mode: "u+rwX,go+rX"
+    copy: yes
+  register: nonascii_result0
+
+- name: Check that file is really there
+  stat:
+    path: "{{ output_dir | expanduser }}/test-unarchive-nonascii-àâæçéèïîôœ-tar-gz/storage/àâæçéèïîôœ(copy)!@#$%^&-().jpg"
+  register: nonascii_stat0
+
+- name: Assert that nonascii tests succeeded
+  assert:
+    that:
+      - "nonascii_result0.changed == true"
+      - "nonascii_stat0.stat.exists == true"
+
+- name: remove nonascii test
+  file: path="{{ output_dir }}/test-unarchive-nonascii-àâæçéèïîôœ-tar-gz" state=absent
+
 # Test that unarchiving is performed if files are missing
 # https://github.com/ansible/ansible-modules-core/issues/1064
 - name: create our unarchive destination
@@ -330,3 +359,69 @@
 
 - name: remove our tar.gz unarchive destination
   file: path={{ output_dir }}/test-unarchive-tar-gz state=absent
+
+#
+# Symlink tests
+#
+
+- name: Create a destination dir
+  file:
+    path: "{{ output_dir }}/test-unarchive-tar-gz"
+    state: directory
+
+- name: Create a symlink to the detination dir
+  file:
+    path: "{{ output_dir }}/link-to-unarchive-dir"
+    src: "{{ output_dir }}/test-unarchive-tar-gz"
+    state: "link"
+
+- name: test that unarchive works when dest is a symlink to a dir
+  unarchive:
+    src: "{{ output_dir }}/test-unarchive.tar.gz"
+    dest: "{{ output_dir | expanduser }}/link-to-unarchive-dir"
+    mode: "u+rwX,go+rX"
+    copy: no
+  register: unarchive_11
+
+- name: Check that file is really there
+  stat:
+    path: "{{ output_dir | expanduser }}/test-unarchive-tar-gz/foo-unarchive.txt"
+  register: unarchive11_stat0
+
+- name: Assert that unarchive when dest is a symlink to a dir worked
+  assert:
+    that:
+      - "unarchive_11.changed == true"
+      - "unarchive11_stat0.stat.exists == true"
+
+- name: remove our tar.gz unarchive destination
+  file: path={{ output_dir }}/test-unarchive-tar-gz state=absent
+
+
+- name: Create a file
+  file:
+    path: "{{ output_dir }}/test-unarchive-tar-gz"
+    state: touch
+
+- name: Create a symlink to the file
+  file:
+    path: "{{ output_dir }}/link-to-unarchive-file"
+    src: "{{ output_dir }}/test-unarchive-tar-gz"
+    state: "link"
+
+- name: test that unarchive fails when dest is a link to a file
+  unarchive:
+    src: "{{ output_dir }}/test-unarchive.tar.gz"
+    dest: "{{ output_dir | expanduser }}/link-to-unarchive-file"
+    mode: "u+rwX,go+rX"
+    copy: no
+  ignore_errors: True
+  register: unarchive_12
+
+- name: Assert that unarchive when dest is a file failed
+  assert:
+    that:
+      - "unarchive_12.failed == true"
+
+- name: remove our tar.gz unarchive destination
+  file: path={{ output_dir }}/test-unarchive-tar-gz state=absent