Some tweaks and reduce contention in  HandlerManager