Fix concurrency issue in lifetime manager.

Change-Id: I9fb31e1dc7e5f67d12c33bc874d3caf8e9177863