Surely one of these should be the other way round ?  Given a naive guess at what GenericItemLess does, I see <code>
min = min < rp ? rp : min;
max = max < rp ? rp : max;
</code> in which the first should surely be <code> min < rp ? min : rp; </code> except that I'd far sooner phrase this as <code>
if (!GenericItemLess(sort_type, (void *) min, (void *) rp)) { min = rp; }
if (GenericItemLess(sort_type, (void *)max, (void *) rp)) { max = rp; }
</code> so as to not do a vacuous min = min or max = max assignment when not setting to rp.