make sure the archive loop gets completed even if the rate limit headers are missing