How much performance does this cost when the sets do not share a common heritage?  If it's non-negligible, it might be good to think more deeply about the relative frequency of encountering sets with common heritage and sets without.