More efficient way to check & insert listener into ptr_map.

Change-Id: I09ec8ed8f05621b72984a34f47db50d20ebd6c06
(cherry picked from commit 42c8178ce73756f85fa41473bb16f667f7f564e9)