Store shared pointers in map instead of naked pointers.