Sort: Make `ignore_unmapped` work for cross-index queries.

Close #2255
can we maybe share these on an index level, instead of doing the whole build process over and over? just a set of static (index level) common types?