Revert "Guard access to external ref manager instance."

This reverts commit 7cf9ea71ad1364d00eaca95b309cc6c0f35cf6cb.
Remove mutexes from external ref manager.

Change-Id: I4857bb3a1804d4cd53c3e25a7586bd68ab95a202