cloudnull/turbolift — pull request #21 by VeggieMeat (2013-09-07 07:59:07+00:00)
Title: check if a file exists before adding it to the archive

If you're adding a lot of files to an archive and a file is removed between the time that it is added to the filelist and the time that it is actually added to the archive, the script dies and complains that the file did not exist.

My particular use case is archiving a Jenkins /jobs/ directory (maintaining history in the backup) that included jobs that do their own cleanup such as removing history older than X days or X runs.

At nearly 100,000 files, it caused a slight bit of a race. Of course, Jenkins won every time.