Replace the object map with a weakmap.