I wonder if using ConcurrentHashMap would be more suited here, so you can drop the synchronized accessors