Fix concurrency issue in lifetime manager.

Change-Id: Id9ee405b98797f6ef8ab30717712995b06363a98