Fix concurrency issue in lifetime manager.

Change-Id: I442cd230957a10f112f1d2ee5bb7b6d731cb1456