Avoid expensive for..in loops involving CMaps

This is much the same as #4804, which was abandoned.