wal-e/wal-e — pull request #33 by fdr (2013-09-11 23:44:18+00:00)
Title: While transferring WAL ensure concurrent greenlets can clean up

There have been some reports that WAL-E can leave around temporary
files pathologically often.

This is because when an error is propagated to the main greenlet,
there's a good chance that it will exit the process before allowing
the other concurrent greenlets to clean up: instead, those greenlets
are abruptly terminated, and in doing with temporary files are leaked.

This bug probably exists elsewhere in WAL-E (e.g. base backups), but
the introduction of parallel WAL upload has rendered its effect
uniquely noticeable to date, probably because so many of
start-up-shut-down cycles of WAL-E are performed in a short period of
time, and each such cycle is an opportunity to leak these files given
an error condition.