Remove old backup only if there is one.