Fix concurrency issue in lifetime manager.

Change-Id: Ica774301805746df79958835c981bb1127502677