Title: Nonparametric, Contextual Preference Estimation and  Assortment Optimization
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: optimal assortment, model evaluation, decision model, nonparametric statistics

Abstract: The growth of AI- and ML-based decision tools provides an array of decision- support agents that can be implemented into the user’s decision-making process. However few tools exist for contextual evaluation for the alignment of decision agents with clinicians' workflows. Consequently, few methods exist to test properties of a contextually-optimal set of aligned agents to target for adoption. Contextual evaluation of decision models is particularly important in settings with few or no gold-standard decisions or preference alignment, such as clinical decision-making. Our work adopts the multinomial logit choice (MNL) model as a framework for evaluating agent-alignment and identifying an optimal agent-set. We assume the observation of selections among a set of agents according to a contextual MNL model, characterized by context-dependent preference parameters. We study a weighted, regularized local likelihood maximization estimator, providing a uniform convergence rate over a compact context space. Additionally, when agent-specific utility parameters or functions are known, we provide results for the identification of a utility-optimal assortment of agents. In this setting, we provide results to construct valid confidence bands on inferential objects of interest and the ability to perform asymptotically valid tests on the composition of this optimal assortment.

TL;DR: We provide methods for the contextual evaluation of decision-aid agents and the identification of a contextually-optimal set of agents, based on multiway comparisons data.