I do wonder what we should do here? If the removal from the collection would fail (unlikely but always possible), should it fail aka rethrow the exception or just log the exception and continue? We need to keep in mind that the exception may also occur on the finalizer thread. Either way, if it would fail, then there is no retry afterwards (except if the first failure happens from the Dispose, then we still have the finalizer as a backup. So maybe rethrow conditionally (only when disposing == true) ?