Fix concurrency issue in lifetime manager.

Change-Id: Ib30736ac42c5132bda707138d6b9f484ffa50b6e