Don't cache equivalent converters in StructureModifier. 

This should hopefully fix a memory leak discovered by Stevenpcc 
(thanks).