autotest/autotest — Issue #134: Temporary files are not deleted for aborted jobs
Author: kparal
Created: 2012-01-10 08:44:07+00:00
State: open

Lately a lot of our clients started to fail verification because of insufficient disk space. I studied the issue and found out that most of the disk space is occupied by ~autotest/tmp. It looks like this:

```
# du -csh ~autotest/tmp/*
4.0K	/usr/share/autotest/tmp/_autotmp_27K9zyharness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_2FU0Qgharness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_Aez56charness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_aNUKsRharness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_AwDEuZharness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_GctnoNharness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_GPL4Q3harness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_hIrBY3harness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_K7gT_Sharness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_mEatF_harness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_RcQyVaharness-fifo
4.0K	/usr/share/autotest/tmp/_autotmp_yVpIqeharness-fifo
60M	/usr/share/autotest/tmp/tmp2ZCagZ_depcheck
58M	/usr/share/autotest/tmp/tmp5ddUNk_depcheck
1.4G	/usr/share/autotest/tmp/tmp8IAWM1_rpmlint
458M	/usr/share/autotest/tmp/tmpaV_ikP_depcheck
384M	/usr/share/autotest/tmp/tmpAw8P1T_depcheck
463M	/usr/share/autotest/tmp/tmpCaoGCS_depcheck
464M	/usr/share/autotest/tmp/tmpfvMNjz_depcheck
363M	/usr/share/autotest/tmp/tmpJgOUn7_depcheck
240M	/usr/share/autotest/tmp/tmpJIM4cV_depcheck
351M	/usr/share/autotest/tmp/tmpjkRcd1_depcheck
2.3M	/usr/share/autotest/tmp/tmpkWn4Ew_depcheck
573M	/usr/share/autotest/tmp/tmpm0Ffn1_depcheck
365M	/usr/share/autotest/tmp/tmpmE4KwM_depcheck
353M	/usr/share/autotest/tmp/tmpnURkM9_depcheck
4.0K	/usr/share/autotest/tmp/tmpo_aCks_depcheck
506M	/usr/share/autotest/tmp/tmpPfnwQ2_rpmguard
364M	/usr/share/autotest/tmp/tmpsQt6h9_depcheck
377M	/usr/share/autotest/tmp/tmpUKnulv_depcheck
419M	/usr/share/autotest/tmp/tmpvp5Yli_depcheck
1.4G	/usr/share/autotest/tmp/tmpy5Cb4T_rpmlint
96M	/usr/share/autotest/tmp/tmpYuzZvx_depcheck
8.5G	total
```

I grepped the logs for those temporary directories and found out that all these tests were aborted (that is connected to issue #117). It means that if autotest aborts a running job (using a watchdog or maybe even after manual user request), it doesn't delete its temporary files. That is a problem, because if it happens often, it continuously eats out the available disk space and once in a while I have to manually log in to all our test clients and prune the ~autotest/tmp directory.

Autotest 0-13.1