Repository: ansible/ansible-modules-core

Issue:
unarchive in 2.2 fails to unarchive a zip file
##### ISSUE TYPE
- Bug Report
##### COMPONENT NAME

unarchive

```
ansible 2.2.0 (devel 6f678e36d6) last updated 2016/08/23 09:52:36 (GMT +300)
  lib/ansible/modules/core: (devel 91a839f1e3) last updated 2016/08/18 20:46:51 (GMT +300)
  lib/ansible/modules/extras: (detached HEAD 1aeb9f8a8c) last updated 2016/08/23 09:03:03 (GMT +300)
  config file = /home/mg/src/deployments/provisioning/ansible.cfg
  configured module search path = Default w/o overrides
```
##### CONFIGURATION

nothing pertinent
##### OS / ENVIRONMENT

Ubuntu 16.04 controller, Ubuntu 14.04 target
##### SUMMARY

The unarchive module in current git devel fails to extract a zip file.  This is a regression: the same playbook works fine with Ansible 2.1.1.0.

The playbook is

```
# variables:
# logicaldoc_version: "7.3"
# logicaldoc_installer_url: "http://netcologne.dl.sourceforge.net/project/logicaldoc/distribution/LogicalDOC%20CE%207.3/logicaldoc-community-installer-7.3.zip"
# logicaldoc_installer_sha256sum: "c0379767b80b225468c068615fc21573405aac9687c3fb5526c71373c827eb3b"
# logicaldoc_installer_jar: "logicaldoc-community-installer-7.3/logicaldoc-installer.jar"

- name: download logicaldoc installer
  get_url: dest=/tmp/logicaldoc-installer-{{ logicaldoc_version }}.zip url={{ logicaldoc_installer_url }} sha256sum={{ logicaldoc_installer_sha256sum }}

- name: create a directory for the installer
  file: dest=/tmp/logicaldoc-installer-{{ logicaldoc_version }} owner=logicaldoc state=directory

- name: extract logicaldoc installer
  unarchive: dest=/tmp/logicaldoc-installer-{{ logicaldoc_version }} src=/tmp/logicaldoc-installer-{{ logicaldoc_version }}.zip copy=no creates=/tmp/logicaldoc-installer-{{ logicaldoc_version }}/{{ logicaldoc_installer_jar }} owner=logicaldoc
```

The error given is

```
fatal: [trusty]: FAILED! => {"changed": true, "dest": "/tmp/logicaldoc-installer-7.3", "extract_results": {"cmd": "/usr/bin/unzip -o \"/tmp/logicaldoc-installer-7.3.zip\" \"logicaldoc-community-installer-7.3/\" -d \"/tmp/logicaldoc-installer-7.3\"", "err": "", "out": "Archive:  /tmp/logicaldoc-installer-7.3.zip\n", "rc": 0}, "failed": true, "gid": 0, "group": "root", "handler": "ZipArchive", "mode": "0755", "msg": "Unexpected error when accessing exploded file: [Errno 2] No such file or directory: '/tmp/logicaldoc-installer-7.3/logicaldoc-community-installer-7.3/CHANGELOG.txt'", "owner": "logicaldoc", "size": 4096, "src": "/tmp/logicaldoc-installer-7.3.zip", "stat": {"exists": false}, "state": "directory", "uid": 995}
```

The directory structure after the failure looks like this:

```
root@vagrant-ubuntu-trusty-64:/tmp# tree /tmp/logicaldoc-installer-7.3
/tmp/logicaldoc-installer-7.3
└── logicaldoc-community-installer-7.3
    └── INSTALL.txt
```

i.e. one of the files is successfully extracted, but none of the rest.
##### STEPS TO REPRODUCE

<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

Run a playbook like mine -- the logicaldoc installer zip is publically accessible at the URL given.

<!--- Paste example playbooks or commands between quotes below -->

``` yml
# test.yml

---
- hosts: localhost
  gather_facts: no
  tasks:
    - get_url: dest=/tmp/logicaldoc-installer-7.3.zip url=http://netcologne.dl.sourceforge.net/project/logicaldoc/distribution/LogicalDOC%20CE%207.3/logicaldoc-community-installer-7.3.zip
    - file: dest=/tmp/logicaldoc-installer-7.3 state=directory
    - unarchive: dest=/tmp/logicaldoc-installer-7.3 src=/tmp/logicaldoc-installer-7.3.zip copy=no
    - stat: path=/tmp/logicaldoc-installer-7.3/logicaldoc-community-installer-7.3/logicaldoc-installer.jar
      register: result
    - assert: that=result.stat.exists
```

`ansible-playbook -i localhost, -c local test.yml`

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

<!--- What did you expect to happen when running the steps above? -->

No error
##### ACTUAL RESULTS

<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->

```
...
fatal: [trusty]: FAILED! => {
    "changed": true, 
    "dest": "/tmp/logicaldoc-installer-7.3", 
    "diff": {
        "prepared": ".d....o..?? logicaldoc-community-installer-7.3/\n"
    }, 
    "extract_results": {
        "cmd": "/usr/bin/unzip -o \"/tmp/logicaldoc-installer-7.3.zip\" \"logicaldoc-community-installer-7.3/\" -d \"/tmp/logicaldoc-installer-7.3\"", 
        "err": "", 
        "out": "Archive:  /tmp/logicaldoc-installer-7.3.zip\n", 
        "rc": 0
    }, 
    "failed": true, 
    "gid": 0, 
    "group": "root", 
    "handler": "ZipArchive", 
    "invocation": {
        "module_args": {
            "backup": null, 
            "content": null, 
            "copy": false, 
            "creates": "/tmp/logicaldoc-installer-7.3/logicaldoc-community-installer-7.3/logicaldoc-installer.jar", 
            "delimiter": null, 
            "dest": "/tmp/logicaldoc-installer-7.3", 
            "directory_mode": null, 
            "exclude": [], 
            "extra_opts": [], 
            "follow": false, 
            "force": null, 
            "group": null, 
            "keep_newer": false, 
            "list_files": false, 
            "mode": null, 
            "original_basename": "logicaldoc-installer-7.3.zip", 
            "owner": "logicaldoc", 
            "regexp": null, 
            "remote_src": false, 
            "selevel": null, 
            "serole": null, 
            "setype": null, 
            "seuser": null, 
            "src": "/tmp/logicaldoc-installer-7.3.zip", 
            "unsafe_writes": null, 
            "validate_certs": true
        }
    }, 
    "mode": "0755", 
    "msg": "Unexpected error when accessing exploded file: [Errno 2] No such file or directory: '/tmp/logicaldoc-installer-7.3/logicaldoc-community-installer-7.3/CHANGELOG.txt'", 
    "owner": "logicaldoc", 
    "size": 4096, 
    "src": "/tmp/logicaldoc-installer-7.3.zip", 
    "stat": {
        "exists": false
    }, 
    "state": "directory", 
    "uid": 995
}
```



Discussion:
<!-- @mgedmin: Thanks for filing this bug report! -->

@dagwieers, @pileofrogs, ping. This issue is waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_bug_new --->

Probably best to enable the debug-lines inside the module to understand what is going on, or provide us with a simple reproducer. needs_info

From the output is appears that only the directory is taken into account to be unarchived, so I assume the ZIP file is somehow unorthodox to what the implementation expects. Since we rely on the unzip command and scraping the output, it is not impossible that the output simply doesn't match what we expect from unzip. (Yes, this is all horrible and we need a pure-python implementation rather than the code we have now).

bot_broken

Once again, ansibot assumes that the **needs_info** can be removed when the requester for more information adds additional information. When **needs_info** is set, ansibot should only consider comments from the original reporter (or maybe from outsiders) as an indication that information is provided IMO.

(Or maybe it should only act on **!needs_info** to remove the label ?)

needs_info

The zip file is here: http://netcologne.dl.sourceforge.net/project/logicaldoc/distribution/LogicalDOC%20CE%207.3/logicaldoc-community-installer-7.3.zip

`unzip -l` shows its contents as

```
unzip -l logicaldoc-community-installer-7.3.zip 
Archive:  logicaldoc-community-installer-7.3.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2015-08-05 12:05   logicaldoc-community-installer-7.3/
    28212  2015-08-05 11:03   logicaldoc-community-installer-7.3/CHANGELOG.txt
     1444  2015-08-05 10:24   logicaldoc-community-installer-7.3/INSTALL.txt
      892  2015-08-05 10:55   logicaldoc-community-installer-7.3/LICENSE.txt
137802438  2015-08-05 12:00   logicaldoc-community-installer-7.3/logicaldoc-installer.jar
    48008  2015-08-05 11:28   logicaldoc-community-installer-7.3/setup.exe
---------                     -------
137880994                     6 files
```

I've updated the original description with an example playbook that reproduces this.   It's also available at: https://github.com/mgedmin/ansible-bug-modules-core-4554 (clone and run `make`).

Here's the output of `unzip -ZT -s /tmp/logicaldoc-installer-7.3.zip`:

```
Archive:  /tmp/logicaldoc-installer-7.3.zip
Zip file size: 137312095 bytes, number of entries: 6
drwx---     6.3 fat        0 bx stor 20150805.120506 logicaldoc-community-installer-7.3/
-rw-a--     6.3 fat    28212 bx defN 20150805.110340 logicaldoc-community-installer-7.3/CHANGELOG.txt
-rw----     6.3 fat     1444 bx defN 20150805.102414 logicaldoc-community-installer-7.3/INSTALL.txt
-rw-a--     6.3 fat      892 bx defN 20150805.105500 logicaldoc-community-installer-7.3/LICENSE.txt
-rw-a--     6.3 fat 137802438 bx defN 20150805.120000 logicaldoc-community-installer-7.3/logicaldoc-installer.jar
-rwxa--     6.3 fat    48008 bx defN 20150805.112856 logicaldoc-community-installer-7.3/setup.exe
6 files, 137880994 bytes uncompressed, 137310845 bytes compressed:  0.4%
```

Note all those `a` permission bits?  That's probably the MS-DOS "archive" bit.  AFAICT unarchive.py skips these entries because of [this check in ZipArchive.is_unarchived](https://github.com/ansible/ansible-modules-core/blob/ecca37a959df39794b2421cfcddbb5344c88a31c/files/unarchive.py#L342-L343):

``` python
            if pcs[0][0] not in 'dl-?' or not frozenset(pcs[0][1:]).issubset('rwxst-'):
                continue
```

Yes, that's it.


Resolving patch:
diff --git a/files/unarchive.py b/files/unarchive.py
--- a/files/unarchive.py
+++ b/files/unarchive.py
@@ -338,8 +338,13 @@ def is_unarchived(self):
             # Check first and seventh field in order to skip header/footer
             if len(pcs[0]) != 7 and len(pcs[0]) != 10: continue
             if len(pcs[6]) != 15: continue
-
-            if pcs[0][0] not in 'dl-?' or not frozenset(pcs[0][1:]).issubset('rwxst-'):
+ 
+            # Possible entries:
+            #   -rw-rws---  1.9 unx    2802 t- defX 11-Aug-91 13:48 perms.2660
+            #   -rw-a--     1.0 hpf    5358 Tl i4:3  4-Dec-91 11:33 longfilename.hpfs
+            #   -r--ahs     1.1 fat    4096 b- i4:2 14-Jul-91 12:58 EA DATA. SF
+            #   --w-------  1.0 mac   17357 bx i8:2  4-May-92 04:02 unzip.macr
+            if pcs[0][0] not in 'dl-?' or not frozenset(pcs[0][1:]).issubset('rwxstah-'):
                 continue
 
             ztype = pcs[0][0]


Test patch: