Fix concurrency issue in lifetime manager.

Change-Id: I826eb145059ba563ded170bd80f577e879fd73a5