Fix concurrency issue in lifetime manager.

Change-Id: I03f561a4c4212c4890e9a8470b43462de8dfc3d8