Repository: ansible-collections/community.general

Issue:
Archive 'remove' fails in AWX/Tower templates in 3.3.0, 3.2.0 works
### Summary

Using archive method "Remove" works with Ansible 4.2.0 (with community.general 3.3.0) from anywhere but the UI of AWX/Tower. If I ssh into Tower and run the playbook from the CLI, then it works.

The TAR is ok and the source files are deleted correctly, so I don't get why it fails at the "remove" step.

If it set Tower to use Ansible 4.1.0 (with community.general 3.2.0) then it works.

Any idea why this is happening and if it is a problem of community.general or something else? Thanks for your help

### Issue Type

Bug Report

### Component Name

archive + remove: yes

### Ansible Version

```console (paste below)
$ ansible --version
ansible [core 2.11.2] 
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/my-tower-venv/ansible-4.2.0/lib/python3.8/site-packages/ansible
  ansible collection location = /var/lib/awx/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/my-tower-venv/ansible-4.2.0/bin/ansible
  python version = 3.8.3 (default, Aug 18 2020, 08:56:04) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
  jinja version = 3.0.1
  libyaml = True
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed
ANSIBLE_PIPELINING(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = True
CALLBACKS_ENABLED(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = ['ansible.posix.timer', 'ansible.posix.profile_roles']                                         
COLLECTIONS_PATHS(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = ['/var/lib/awx/ansible_mysql_live_indermue/collections', '/var/lib/awx/python-venv/ansible-4.2.0/lib/python3.8/site-packages/ansible_collections', '/opt/my-tower-venv/ansible-4.2.0/lib/python3.8/site-packages/ansible_collections', '/var/lib/awx/.ansible/collections/ansible_collections']
DEFAULT_FORKS(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = 8
DEFAULT_HOST_LIST(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = ['/var/lib/awx/ansible_mysql_live_indermue/inventories']                                       
DEFAULT_ROLES_PATH(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = ['/var/lib/awx/ansible_mysql_live_indermue/roles', '/var/lib/awx/.ansible/roles']             
DEFAULT_STDOUT_CALLBACK(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = yaml
DEFAULT_VAULT_PASSWORD_FILE(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = /var/lib/awx/.ansible-vault-mysql                                                    
INTERPRETER_PYTHON(/var/lib/awx/ansible_mysql_live_indermue/ansible.cfg) = auto_silent     
```


### OS / Environment

RHEL 8

### Steps to Reproduce

Play the bellow task in a template in Ansible Tower 3.8.3 using a virtualenv created as follow:

```bash
sudo yum install python3.8 python3-virtualenv
mkdir -p ~/python-venv/ansible-4.2.0
virtualenv --python=python3.8 ~/python-venv/ansible-4.2.0
source ~/python-venv/ansible-4.2.0/bin/activate
python -m  pip install --upgrade pip
pip install \
  ansible==4.2.0 \
  ansible-lint \
  antsibull-changelog \
  molecule \
  molecule-podman \
  yamllint \
  selinux \
  psutil \
  argcomplete \
  boto3
```

<!--- Paste example playbooks or commands between quotes below -->
```yaml
    - name: TAR the datadir
      community.general.archive:
        path: "/u02/mysqldata/mysqld4/"
        dest: "/backup/datadir.tar"
        format: tar
        owner: mysql
        group: mysql
        mode: '0644'
        remove: true
```


### Expected Results

```console
{
    "archived": [
        "/u02/mysqldata/mysqld4/aria_log_control",
        "/u02/mysqldata/mysqld4/aria_log.00000001",
        "/u02/mysqldata/mysqld4/ibdata1",
        [...]
    ],
    "dest": "/backup/datadir.tar",
    "changed": true,
    "state": "file",
    "arcroot": "/u02/mysqldata/",
    "missing": [],
    "expanded_paths": [
        "/u02/mysqldata/mysqld4"
    ],
    "expanded_exclude_paths": [],
    "uid": 321,
    "gid": 321,
    "owner": "mysql",
    "group": "mysql",
    "mode": "0644",
    "secontext": "unconfined_u:object_r:default_t:s0",
    "size": 2297241600,
    "invocation": {
        "module_args": {
            "path": [
                "/u02/mysqldata/mysqld4"
            ],
            "dest": "/backup/datadir.tar",
            "format": "tar",
            "owner": "mysql",
            "group": "mysql",
            "mode": "0644",
            "remove": true,
            "force_archive": false,
            "unsafe_writes": false,
            "exclude_path": null,
            "exclusion_patterns": null,
            "seuser": null,
            "serole": null,
            "selevel": null,
            "setype": null,
            "attributes": null
        }
    },
    "_ansible_no_log": false
}
```


### Actual Results

```console
{
    "dest": "/backup/datadir.tar",
    "files": [
        "/u02/mysqldata/mysqld4/mysql/innodb_table_stats.frm",
        "/u02/mysqldata/mysqld4/mysql/servers.MAD",
        "/u02/mysqldata/mysqld4/mysql/time_zone.frm",
        [...]
    ],
    "msg": "Error deleting some source files: ",
    "uid": 321,
    "gid": 321,
    "owner": "mysql",
    "group": "mysql",
    "mode": "0664",
    "state": "file",
    "secontext": "unconfined_u:object_r:default_t:s0",
    "size": 2297241600,
    "invocation": {
        "module_args": {
            "path": [
                "/u02/mysqldata/mysqld4/"
            ],
            "dest": "/backup/datadir.tar",
            "format": "tar",
            "owner": "mysql",
            "group": "mysql",
            "mode": "0644",
            "remove": true,
            "exclude_path": [],
            "force_archive": false,
            "unsafe_writes": false,
            "exclusion_patterns": null,
            "seuser": null,
            "serole": null,
            "selevel": null,
            "setype": null,
            "attributes": null
        }
    },
    "_ansible_no_log": false,
    "changed": false
}
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
If this only happens with Tower, you should file a bug with Tower, resp. contact Tower support. They might be able to help.
Ok then, I'll open a support request at Tower support.
@felixfontein, this is an issue introduced with `3.3.0` which I have reproduced.
It's a one line fix however so I will push shortly with an accompanying test case to prevent regression.

Resolving patch:
diff --git a/plugins/modules/files/archive.py b/plugins/modules/files/archive.py
--- a/plugins/modules/files/archive.py
+++ b/plugins/modules/files/archive.py
@@ -399,13 +399,14 @@ def has_unfound_targets(self):
 
     def remove_targets(self):
         for path in self.successes:
-            try:
-                if os.path.isdir(path):
-                    shutil.rmtree(path)
-                else:
-                    os.remove(path)
-            except OSError:
-                self.errors.append(_to_native(path))
+            if os.path.exists(path):
+                try:
+                    if os.path.isdir(path):
+                        shutil.rmtree(path)
+                    else:
+                        os.remove(path)
+                except OSError:
+                    self.errors.append(_to_native(path))
         for path in self.paths:
             try:
                 if os.path.isdir(path):


Test patch:
diff --git a/tests/integration/targets/archive/tasks/remove.yml b/tests/integration/targets/archive/tasks/remove.yml
--- a/tests/integration/targets/archive/tasks/remove.yml
+++ b/tests/integration/targets/archive/tasks/remove.yml
@@ -148,7 +148,39 @@
 - name: verify that excluded sub file is still present
   file: path={{ output_dir }}/tmpdir/sub/subfile.txt state=file
 
-- name: remove temporary directory
+- name: prep our files in tmpdir again
+  copy: src={{ item }} dest={{ output_dir }}/tmpdir/{{ item }}
+  with_items:
+    - foo.txt
+    - bar.txt
+    - empty.txt
+    - sub
+    - sub/subfile.txt
+
+- name: archive using gz and remove src directory
+  archive:
+    path:
+      - "{{ output_dir }}/tmpdir/"
+    dest: "{{ output_dir }}/archive_remove_05.gz"
+    format: gz
+    remove: yes
+    exclude_path: "{{ output_dir }}/tmpdir/sub/subfile.txt"
+  register: archive_remove_result_05
+
+- name: verify that the files archived
+  file: path={{ output_dir }}/archive_remove_05.gz state=file
+
+- name: Verify source files were removed
   file:
     path: "{{ output_dir }}/tmpdir"
     state: absent
+  register: archive_source_file_removal_05
+
+- name: Verify that task status is success
+  assert:
+    that:
+      - archive_remove_result_05 is success
+      - archive_source_file_removal_05 is not changed
+
+- name: remove our gz
+  file: path="{{ output_dir }}/archive_remove_05.gz" state=absent