Repository: ansible/ansible

Issue:
Out of disk space condition gives error: 'temp_path' is not defined
##### ISSUE TYPE
- Bug Report
##### COMPONENT NAME

core assembly
##### ANSIBLE VERSION

```
ansible 2.1.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```
##### CONFIGURATION

Using ANSIBLE_KEEP_REMOTE_FILES=1:
fatal: [tw0]: UNREACHABLE! => {"changed": false, "msg": "mkdir: cannot create directory '/home/x/.ansible': No space left on device\n", "unreachable": true}
##### OS / ENVIRONMENT

Ubuntu 16.04
##### SUMMARY

When running ansible on a machine which has a 100% full disk the error is not indicative of the problem.
##### STEPS TO REPRODUCE

```
x@x:~/work-fl/cm-ansible$ ansible -i inventories/ota tw0 -m setup
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: NameError: name 'temp_path' is not defined
tw0 | FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 123, in <module>\nNameError: name 'temp_path' is not defined\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "parsed": false
}
x@x:~/work-fl/cm-ansible$ ANSIBLE_KEEP_REMOTE_FILES=1 ansible -i inventories/ota tw0 -m setup 
tw0 | UNREACHABLE! => {
    "changed": false, 
    "msg": "mkdir: cannot create directory '/home/x/.ansible': No space left on device\n", 
    "unreachable": true
}
x@x:~/work-fl/cm-ansible$ 

```

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

I expect an error saying "No space left on device"
##### ACTUAL RESULTS

```
NameError: name 'temp_path' is not defined
```



Discussion:
Might be an easy fix to the ansiballz wrapper, but surprised that mkdtemp() isn't giving us the "out of space" error (and in fact possibly is failing silently?)...

cc @abadger 

I'm not sure if this is related, but I'm seeing similar errors when using 'become' on the git module. It looks like it's not even checking arguments before barfing. I got my test case down to this:

`ansible my_server -m git -a "" -b --become-user=some_user -vvv`

(obviously replace my_server and some_user with something valid). I get the following output:

```
Using /opt/sysadmin/ansible/ansible.cfg as config file
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/source_control/git.py
<my_host> ESTABLISH SSH CONNECTION FOR USER: None
<my_host> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r my_host '/bin/sh -c '"'"'sudo -H -S -n -u some_user /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-doquzrgecuimuufnucirjrcblyxtxgkf; /usr/bin/python'"'"'"'"'"'"'"'"' && sleep 0'"'"''
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "<stdin>", line 133, in <module>
NameError: name 'temp_path' is not defined
my_host | FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_name": "git"
    },
    "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 133, in <module>\nNameError: name 'temp_path' is not defined\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE"
}
```

If I remove the 'become' then it works fine.

Getting Ansible to leave temporary files and some find/grep lead me to the /usr/lib/python2.7/site-packages/ansible/executor/module_common.py which contains the bit of code that's really got the problem. I found that if I initialised temp_path with "" I got an "IOError: No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/root']" error, which lead me to look at my /tmp and /var/tmp directories - neither of which were group/world writable. Fixed that, then all was well with Ansible once again.

I'd say 'real' errors are being swallowed somewhere, which cost me a couple of hours of head-scratching. It's doing the right thing, just not telling you what that is ;-)

Using ansible-2.2.0.0-4.el7.noarch.
I'm having a similar problem on Ansible 2.3 as well, but I'm getting two different errors depending on the task. I have two playbooks, one with a simple sync:

    ---
    - name: sync the application directory
      hosts: "{{ hostgroup }}"
      gather_facts: yes
      become: yes
      become_user: root
      remote_user: "{{ remote_user }}"
      tasks:
        - name: synchronize the working directory
          synchronize:
            src: ../../src/
            dest: "/opt/{{ application_name }}"
            rsync_opts: -F
      vars_files:
        - global_vars.yml

When run with `vvv`, this one gives this output:

    Using /home/berislav/Development/myapp/ansible.cfg as config file

    PLAYBOOK: test_sync.yml **************************************************************************************************************************************************************************************************************************
    1 plays in operations/ansible/test_sync.yml

    PLAY [sync the application directory] ************************************************************************************************************************************************************************************************************

    TASK [Gathering Facts] ***************************************************************************************************************************************************************************************************************************
    Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/system/setup.py
    <54.171.119.161> ESTABLISH SSH CONNECTION FOR USER: ubuntu
    <54.171.119.161> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/berislav/.ssh/mykey.pem"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ubuntu -o ConnectTimeout=10 -o ControlPath=/home/berislav/.ansible/cp/86838d2333 54.171.119.161 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-dlbcigtyfiwhwomtorrzqumscyfrmmum; /usr/bin/python'"'"'"'"'"'"'"'"' && sleep 0'"'"''
    <54.171.119.161> (1, '', 'Traceback (most recent call last):\n  File "<stdin>", line 136, in <module>\nNameError: name \'temp_path\' is not defined\n')
    The full traceback is:
    Traceback (most recent call last):
      File "<stdin>", line 136, in <module>
    NameError: name 'temp_path' is not defined

    fatal: [54.171.119.161]: FAILED! => {
        "changed": false, 
        "failed": true, 
        "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 136, in <module>\nNameError: name 'temp_path' is not defined\n", 
        "module_stdout": "", 
        "msg": "MODULE FAILURE", 
        "rc": 1
    }
      to retry, use: --limit @/home/berislav/Development/myapp/operations/ansible/test_sync.retry

    PLAY RECAP ***************************************************************************************************************************************************************************************************************************************
    54.171.119.161             : ok=0    changed=0    unreachable=0    failed=1   

Then there is the other playbook, simply running `raw` to install Python 2 on Xenial before the playbook is actually executed:

    ---
    - hosts: "{{ hostgroup }}"
      gather_facts: no
      become: yes
      become_user: root
      remote_user: ubuntu
      tasks:
        - name: install python2
          raw: sudo apt-get -y install python-simplejson
      vars_files:
          - global_vars.yml

This one gives this output, on the same inventory:

    Using /home/berislav/Development/myapp/ansible.cfg as config file

    PLAYBOOK: test_raw.yml ***************************************************************************************************************************************************************************************************************************
    1 plays in operations/ansible/test_raw.yml

    PLAY [tag_App_myapp:&tag_Stage_dev:&tag_Version_1_0_0b] ***************************************************************************************************************************************************************************************
    META: ran handlers

    TASK [install python2] ***************************************************************************************************************************************************************************************************************************
    task path: /home/berislav/Development/myapp/operations/ansible/test_raw.yml:8
    <54.171.119.161> ESTABLISH SSH CONNECTION FOR USER: ubuntu
    <54.171.119.161> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/berislav/.ssh/myapp.pem"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ubuntu -o ConnectTimeout=10 -o ControlPath=/home/berislav/.ansible/cp/86838d2333 -tt 54.171.119.161 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-fwwbahzcnmnyzclwqcjryriinozvukwg; sudo apt-get -y install python-simplejson'"'"''
    <54.171.119.161> (100, '\rReading package lists... 0%\r\rReading package lists... 0%\r\rReading package lists... 16%\r\rReading package lists... Error!\r\r\nE: Write error - write (28: No space left on device)\r\nE: IO Error saving source cache\r\nE: The package lists or status file could not be parsed or opened.\r\n', 'Shared connection to 54.171.119.161 closed.\r\n')
    fatal: [54.171.119.161]: FAILED! => {
        "changed": true, 
        "failed": true, 
        "rc": 100, 
        "stderr": "Shared connection to 54.171.119.161 closed.\r\n", 
        "stdout": "\rReading package lists... 0%\r\rReading package lists... 0%\r\rReading package lists... 16%\r\rReading package lists... Error!\r\r\nE: Write error - write (28: No space left on device)\r\nE: IO Error saving source cache\r\nE: The package lists or status file could not be parsed or opened.\r\n", 
        "stdout_lines": [
            "", 
            "Reading package lists... 0%", 
            "", 
            "Reading package lists... 0%", 
            "", 
            "Reading package lists... 16%", 
            "", 
            "Reading package lists... Error!", 
            "", 
            "E: Write error - write (28: No space left on device)", 
            "E: IO Error saving source cache", 
            "E: The package lists or status file could not be parsed or opened."
        ]
    }
      to retry, use: --limit @/home/berislav/Development/myapp/operations/ansible/test_raw.retry

    PLAY RECAP ***************************************************************************************************************************************************************************************************************************************
    54.171.119.161             : ok=0    changed=0    unreachable=0    failed=1   

I can see differences in details like task path, but I'm not familiar with ins and out of Ansible; maybe this will help find the cause.
+1 
I got the same in here ! any update ? 
+1 "the error is not indicative of the problem" 
Had the same error for a RPi in the wild that had a full drive, took me a while to get to 'No space left on device'  from 'temp_path' is not defined.

Resolving patch:
diff --git a/lib/ansible/executor/module_common.py b/lib/ansible/executor/module_common.py
--- a/lib/ansible/executor/module_common.py
+++ b/lib/ansible/executor/module_common.py
@@ -345,7 +345,7 @@ def debug(command, zipped_mod, json_params):
     finally:
         try:
             shutil.rmtree(temp_path)
-        except OSError:
+        except (NameError, OSError):
             # tempdir creation probably failed
             pass
     sys.exit(exitcode)


Test patch: