After some discussions with Christoph we decided to use Collections.synchronizedMap() in this case, although it is not a beautiful solution.