Clean up tmp dir after applying job

Signed-off-by: Damien Le Berrigaud <damien@pivotallabs.com>