Fix race condition in retry_all, fixes #1851
Note about assignment rights