Actually, the problem is you need a ```ConcurrentIdentityHashMap```.  You can achieve this by wrapping the objects you add into the map with a thin wrapper that uses identity equality instead of equals.