Fix race condition in backoff caused by attribute access.

I have zero reasons that this fixes things, but it seems to make the test not flaky any more :/