Learning in Multi-Stage Decentralized Matching Markets

1 Introduction . Two-sided matching markets have played an important role in microeconomics for several decades [ 34 ] . Matching markets are used to allocate indivisible “ goods ” to multiple decision-making agents based on mutual compatibility as assessed via sets of preferences . Such a market does not clear through prices . For example , a student applicant can not simply demand the college she prefers but must also be chosen by the college . Matching markets are often organized in a decentralized way . Each agent makes their decision independently of others ’ decisions , and each agent can have multiple stages of interactions with the other side of the market . College admissions with waiting lists and academic job markets are notable examples . We refer to such markets as multi-stage decentralized matching markets . Uncertain preference is ubiquitous in multi-stage decentralized matching markets . For instance , colleges competing for students lack information on students ’ preferences . An admitted student may receive offers from other colleges . She needs to accept one or reject all offers within a short period during each stage of early , regular , and waiting-list admissions [ 5 ] . This admission process provides little opportunity for colleges to learn students ’ preferences , which are uncertain due to competition among colleges and variability in the relative popularity of colleges over time . Such uncertain preferences pose a challenge for colleges in their attempt to formulate an optimal admission strategy . Consequently , colleges may end up enrolling too many or too few students relative to their capacity or having enrolled students overly far from the attainable optimum in quality . This paper addresses the following two research questions : ( i ) Given the uncertain preferences on one side of the market ( e.g. , students ) , how can agents ( e.g. , colleges ) learn an optimal strategy that maximizes expected payoffs based on historical data ? ( ii ) What are the fundamental implications of multi-stage decentralized matching on the welfare and fairness for both sides of the market ? We study these two questions using nonparametric statistical methodology and variational analysis . We propose a new algorithm for maximizing agents ’ expected payoffs that is based on learning 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . stage-wise optimal strategies and calibrating state parameters based on historical data . In particular , our algorithm balances the opportunity cost and the penalty for exceeding the quota for calibration . Based on the calibrated state , the algorithm efficiently learns an optimal strategy using statistical machine learning methods . The statistical model not only provides a foundation for the algorithm but it also provides an analytical framework for understanding the implications of the approach for welfare and fairness . We show that agents will favor arms with realistic and stable opportunities for matching instead of only targeting the top-ranked arms . Moreover , we show that agents are better off with multi-stage decentralized matching as compared to single-stage decentralized matching . Adopting literature from the bandit literature , our model has a set of agents , each with limited capacity , and a set of arms . Each agent values two attributes of an arm : a “ score '' that is common to all agents and a “ fit '' that is agent-specific and independent across agents . Agents rank arms according to their scores and fits . An agent ’ s strategy consists of how many and which arms to pull at each stage . On the other hand , there is no restriction on the preferences of arms . The model allows uncertainty in the preferences , which is incorporated into the arms ’ stage-wise acceptance probabilities . The acceptance probability depends on the unknown state of the world and the competition of agents at each stage . We consider a simple timeline for multi-stage markets . At each stage , agents simultaneously pull sets of arms . Each arm accepts at most one of the agents that pulled it . The arms have to make irreversible decisions at each stage without knowing which other agents might select them in later stages . Our contributions There are two main contributions in this paper , which correspond to the two questions above . Our first contribution is to propose a new algorithm that maximizes the agent ’ s expected payoff in multi-stage decentralized matching markets . The algorithm sequentially learns the optimal strategy at each stage and is built upon notions of lower uncertainty bound ( LUB ) and calibrated decentralized matching ( CDM ) . The key idea is to calibrate the state parameter in a data-driven approach and take the opportunity cost and penalty for exceeding the quota into account . The calibration can be performed under both average-case and worst-case metrics , depending on whether we are maximizing the averaged or minimal expected payoff with respect to the uncertain state . Given the calibrated state , the algorithm efficiently learns the optimal strategy using historical data via statistical machine learning methods . The second contribution is providing an analytical framework for understanding the welfare and fairness implications . We show that agents favor arms with low uncertainty in levels of acceptance , suggesting that agents prefer arms with a realistic and stable chance for matching instead of only targeting the top-ranked arms . Such strategic behavior improves the agent ’ s expected payoff since otherwise , by the time that arms have rejected that agent , the next-best arms that the agent has in mind may already have accepted other agents . However , the strategic behavior leads to unfair outcomes for arms because some arms are not pulled by their favorite agents even though these agents pull arms ranked below them . We prove that agents are better off in multi-stage decentralized matching markets compared to single-stage decentralized matching markets . Related work This paper is related to three strands of literature . The first line is on matching markets . Most theoretical work on matching markets traces back to [ 21 ] that formulated a model of two-sided matching without side payments , and [ 37 ] that formulated a model of two-sided matching with side payments . The model in [ 37 ] is also related to the maximum weighted bipartite matching and its to stochastic and online generalizations [ 27 ] . Our goal is to design algorithms for maximizing the agent ’ s welfare under the model of [ 21 ] , given the uncertain preferences of arms . This is different from the goal of finding a matching with the largest size in maximum matching literature [ 37 , 27 ] . The second strand of literature is on the decentralized interactions in matching markets [ 15–17 , 30 , 35 ] and search literature [ 28 , 31 ] . Our paper contributes to this strand of literature via its analysis of multi-stage markets that allow uncertain preferences . We also study the economic implications for strategic behaviors in multi-stage decentralized markets . The third related body of literature is on algorithmic studies of college admissions . The celebrated work in [ 21 ] introduced the deferred acceptance algorithm implemented under central clearinghouses . Recent works have been focused on equilibrium admissions , students ’ efforts , and students ’ information acquisition costs in forming preferences ; see , [ 7 , 11–13 , 18 , 20 , 22 , 24 ] . In contrast , we emphasize students ’ multidimensional abilities and multiple colleges competing for students . The students ’ preferences are uncertain due to the competition among colleges and variability in the relative popularity of colleges over time . We develop a statistical model for learning the optimal strategies using historical data . 2 Problem Formulation . Multi-stage decentralized matching markets Let P = { P1 , P2 , . . . , Pm } be a set of m agents . Let A = { A1 , A2 , . . . , An } the a set of n arms . Here P and A are the sets of participants on the two sides of the matching market . Each agent Pi has a quota qi 1 . We assume that q1+q2+· · ·+qm n. There are total of K 1 stages of the matching process . At each stage , an agent who has not used up its quota can pull available arms in the market . When multiple agents select the same arm , only one agent can successfully pull the arm according to the arm ’ s preference . We denote [ m ] ⌘ { 1 , . . . , m } , [ n ] ⌘ { 1 , . . . , n } , and [ K ] ⌘ { 1 , . . . , K } . Decentralized matching markets require participants to make their decisions independently of others ’ decisions [ 33 , 35 ] . Notable examples of such markets include college admissions in the United States , Korea , and Japan , where P and A represent the sets of colleges and students , respectively [ 5 , 6 ] . Our goal is to learn the agent ’ s optimal strategy for maximizing the expected payoff . A strategy consists of deciding how many and which arms to pull at each stage . Agent ’ s decision-making in decentralized markets faces incomplete information about other agents ’ decisions and arms ’ preferences . Participants ’ preferences The agents ’ preferences are based on the arms ’ latent utilities . Consider the following latent utility model : Ui ( Aj ) = vj + eij , 8i 2 [ m ] , j 2 [ n ] , ( 1 ) where vj 2 [ 0 , 1 ] is arm Aj ’ s systematic score considered by all agents , and eij 2 [ 0 , 1 ] is an agent-specific idiosyncratic fit considered only by agent Pi , i 2 [ m ] . A utility model with a similar separable structure has been widely used in the matching market literature [ 4 , 13 , 15 ] . The arms ’ preferences have no restrictions and can involve uncertainty . From an agent ’ s perspective , arms accept offers with probabilities dependent on opponents ’ strategies and arms ’ preferences . Let the parameter si , k 2 [ 0 , 1 ] be the state of the world [ 36 ] for agent Pi , such that the probability that an arm Aj accepts Pi at stage k is ⇡i , k ( si , k , vj ) , 8i 2 [ m ] , j 2 [ n ] , k 2 [ K ] . Since agents compete for arms with a higher score , the acceptance probability ⇡i , k ( si , k , vj ) models the agents ’ competition through the dependence on the score vj . Moreover , ⇡i , k ( si , k , vj ) incorporates the arm ’ s uncertain preference into the state si , k . It is known that there exists a valid probability mass function ⇡i , k ( si , k , vj ) [ 15 ] . We assume that ⇡i , k ( si , k , vj ) is strictly increasing and continuous in si , k . Thus , a larger value of the state si , k corresponds to the case that agent Pi is more popular . In practice , the true state is unknown a priori to Pi and needs to be estimated from data . For instance , the yield in college admissions is defined as the rate at which a college ’ s admitted students accept the offers . However , the yield is unknown a priori to the college in the current year [ 13 ] . Colleges can only estimate the distribution of the yield from historical data . In this paper , we study a nonparametric model of ⇡i , k ( · , · ) by assuming it belongs to a reproducing kernel Hilbert space ( RKHS ) [ 3 , 39 ] . Later , in Section 3.2 , we propose an algorithm for calibrating si , k and efficiently estimating ⇡i , k ( · , · ) using historical data . Given the latent utility Ui ( Aj ) and the acceptance probability ⇡i , k ( si , k , vj ) , agent Pi ’ s expected utility of pulling arm Aj at stage k is ( vj + eij ) ⇡i , k ( si , k , vj ) . Timeline of the matching First , Nature draws a state such that arms ’ preferences are realized . Denote by s⇤i , k the true state for agent Pi at stage k. Next , arms display their interests to all agents . For example , students apply to colleges in a given period . Under the assumption that students incur negligible application costs , submitting applications to all colleges is the dominant strategy as students lack information on how colleges evaluate their academic ability or personal essays [ 6 , 13 ] . Next , at each stage k 2 [ K ] , agents simultaneously pull available arms that have not previously rejected them . Each arm either accepts one of the agents that pulled it ( if any ) or rejects all . An arm exits the market once it accepts an agent , and agents are allowed to exit the market at any time . The arms act simultaneously at each stage . They can not “ hold '' offers for accepting or rejecting at a later stage . Hence , agents make “ exploding '' offers , and arms have to make irreversible decisions without knowing what other offers are coming in later stages . Finally , this multi-stage matching process ends when all agents have exited or when a pre-specified number of stages has been reached . If there remain arms in the market when the matching has terminated , these arms are unmatched . Agent ’ s expected payoff An agent ’ s goal is to maximize the expected payoff , which consists of two parts : the expected utilities and the penalty for exceeding the quota . Let Ak be the set of arms that are available in the market at stage k 2 [ K ] . Suppose that agent Pi pulls arms from the set Bi , k ✓ { Ak \ [ lk 1Bi , l } at stage k , where A \ B denotes that set A minus set B . Let Ci , k ✓ Bi , k be the set of arms that accept Pi at stage k. Then Ci , k is unknown until stage k+1 , where k K 1 , and Ci , K is unknown until the end of the matching process . Then Pi ’ s expected payoff at stage k 2 [ K ] is lower bounded by Ui , k [ Bi , k ] ⌘ X j2Bi , k ( vj + eij ) ⇡i , k ( s ⇤ i , k , vj ) i max { Ni , k ( Bi , k ) qi , 0 } . ( 2 ) Here Ni , k ( Bi , k ) ⌘ P j2Bi , k ⇡i , k ( s⇤i , k , vj ) + card ( [ lk 1Ci , l ) , and s ⇤ i , k is the true state for agent Pi at stage k. We assume that the marginal penalty i satisfies i > maxj2A { vj + eij } , which implies that the penalty is greater than arm ’ s latent utility . Since our model involves unknown strategies of the opponents and uncertain arms ’ preferences , we consider the optimal expected payoff in ( 2 ) instead of the optimal realized payoff . Similar expected payoff have been studied in [ 13 , 15 ] .

Summary: The paper studies learning in multi-stage decentralized markets. Motivated by applications such as college admissions and the academic job market, the paper studies the classic matching problem under the assumptions that agents do not know their preferences and matching decisions can take place in multiple stages. As in the classic matching problem, a constraint is that colleges and applicants have quotas (e.g., students cannot accept more than one offer).  The paper models this problem as a multi-armed bandit problem with K>1 stages and quota constraints. Colleges are the agents and students are the arms. The arms’ preferences have no restrictions and are modeled through a probability function. Each agent values both a common quality aspect (the “score”) and a “fit” which is agent-specific. In this model, the paper proposes an algorithm that is based on the notions of lower uncertainty bound and calibrated decentralized matching; this algorithm learns the optimal strategy at each stage. Further, the paper investigates the economic implications of the model. For example, they show that strategic behavior may lead to unfairness (measured in terms of no justified envy) and that agents are better off with multi-stage compared to single-stage matching. Finally, they include simulations using data from college admissions.