Learning in Multi-Stage Decentralized Matching Markets

1 Introduction . Two-sided matching markets have played an important role in microeconomics for several decades [ 34 ] . Matching markets are used to allocate indivisible “ goods ” to multiple decision-making agents based on mutual compatibility as assessed via sets of preferences . Such a market does not clear through prices . For example , a student applicant can not simply demand the college she prefers but must also be chosen by the college . Matching markets are often organized in a decentralized way . Each agent makes their decision independently of others ’ decisions , and each agent can have multiple stages of interactions with the other side of the market . College admissions with waiting lists and academic job markets are notable examples . We refer to such markets as multi-stage decentralized matching markets . Uncertain preference is ubiquitous in multi-stage decentralized matching markets . For instance , colleges competing for students lack information on students ’ preferences . An admitted student may receive offers from other colleges . She needs to accept one or reject all offers within a short period during each stage of early , regular , and waiting-list admissions [ 5 ] . This admission process provides little opportunity for colleges to learn students ’ preferences , which are uncertain due to competition among colleges and variability in the relative popularity of colleges over time . Such uncertain preferences pose a challenge for colleges in their attempt to formulate an optimal admission strategy . Consequently , colleges may end up enrolling too many or too few students relative to their capacity or having enrolled students overly far from the attainable optimum in quality . This paper addresses the following two research questions : ( i ) Given the uncertain preferences on one side of the market ( e.g. , students ) , how can agents ( e.g. , colleges ) learn an optimal strategy that maximizes expected payoffs based on historical data ? ( ii ) What are the fundamental implications of multi-stage decentralized matching on the welfare and fairness for both sides of the market ? We study these two questions using nonparametric statistical methodology and variational analysis . We propose a new algorithm for maximizing agents ’ expected payoffs that is based on learning 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . stage-wise optimal strategies and calibrating state parameters based on historical data . In particular , our algorithm balances the opportunity cost and the penalty for exceeding the quota for calibration . Based on the calibrated state , the algorithm efficiently learns an optimal strategy using statistical machine learning methods . The statistical model not only provides a foundation for the algorithm but it also provides an analytical framework for understanding the implications of the approach for welfare and fairness . We show that agents will favor arms with realistic and stable opportunities for matching instead of only targeting the top-ranked arms . Moreover , we show that agents are better off with multi-stage decentralized matching as compared to single-stage decentralized matching . Adopting literature from the bandit literature , our model has a set of agents , each with limited capacity , and a set of arms . Each agent values two attributes of an arm : a “ score '' that is common to all agents and a “ fit '' that is agent-specific and independent across agents . Agents rank arms according to their scores and fits . An agent ’ s strategy consists of how many and which arms to pull at each stage . On the other hand , there is no restriction on the preferences of arms . The model allows uncertainty in the preferences , which is incorporated into the arms ’ stage-wise acceptance probabilities . The acceptance probability depends on the unknown state of the world and the competition of agents at each stage . We consider a simple timeline for multi-stage markets . At each stage , agents simultaneously pull sets of arms . Each arm accepts at most one of the agents that pulled it . The arms have to make irreversible decisions at each stage without knowing which other agents might select them in later stages . Our contributions There are two main contributions in this paper , which correspond to the two questions above . Our first contribution is to propose a new algorithm that maximizes the agent ’ s expected payoff in multi-stage decentralized matching markets . The algorithm sequentially learns the optimal strategy at each stage and is built upon notions of lower uncertainty bound ( LUB ) and calibrated decentralized matching ( CDM ) . The key idea is to calibrate the state parameter in a data-driven approach and take the opportunity cost and penalty for exceeding the quota into account . The calibration can be performed under both average-case and worst-case metrics , depending on whether we are maximizing the averaged or minimal expected payoff with respect to the uncertain state . Given the calibrated state , the algorithm efficiently learns the optimal strategy using historical data via statistical machine learning methods . The second contribution is providing an analytical framework for understanding the welfare and fairness implications . We show that agents favor arms with low uncertainty in levels of acceptance , suggesting that agents prefer arms with a realistic and stable chance for matching instead of only targeting the top-ranked arms . Such strategic behavior improves the agent ’ s expected payoff since otherwise , by the time that arms have rejected that agent , the next-best arms that the agent has in mind may already have accepted other agents . However , the strategic behavior leads to unfair outcomes for arms because some arms are not pulled by their favorite agents even though these agents pull arms ranked below them . We prove that agents are better off in multi-stage decentralized matching markets compared to single-stage decentralized matching markets . Related work This paper is related to three strands of literature . The first line is on matching markets . Most theoretical work on matching markets traces back to [ 21 ] that formulated a model of two-sided matching without side payments , and [ 37 ] that formulated a model of two-sided matching with side payments . The model in [ 37 ] is also related to the maximum weighted bipartite matching and its to stochastic and online generalizations [ 27 ] . Our goal is to design algorithms for maximizing the agent ’ s welfare under the model of [ 21 ] , given the uncertain preferences of arms . This is different from the goal of finding a matching with the largest size in maximum matching literature [ 37 , 27 ] . The second strand of literature is on the decentralized interactions in matching markets [ 15–17 , 30 , 35 ] and search literature [ 28 , 31 ] . Our paper contributes to this strand of literature via its analysis of multi-stage markets that allow uncertain preferences . We also study the economic implications for strategic behaviors in multi-stage decentralized markets . The third related body of literature is on algorithmic studies of college admissions . The celebrated work in [ 21 ] introduced the deferred acceptance algorithm implemented under central clearinghouses . Recent works have been focused on equilibrium admissions , students ’ efforts , and students ’ information acquisition costs in forming preferences ; see , [ 7 , 11–13 , 18 , 20 , 22 , 24 ] . In contrast , we emphasize students ’ multidimensional abilities and multiple colleges competing for students . The students ’ preferences are uncertain due to the competition among colleges and variability in the relative popularity of colleges over time . We develop a statistical model for learning the optimal strategies using historical data . 2 Problem Formulation . Multi-stage decentralized matching markets Let P = { P1 , P2 , . . . , Pm } be a set of m agents . Let A = { A1 , A2 , . . . , An } the a set of n arms . Here P and A are the sets of participants on the two sides of the matching market . Each agent Pi has a quota qi 1 . We assume that q1+q2+· · ·+qm n. There are total of K 1 stages of the matching process . At each stage , an agent who has not used up its quota can pull available arms in the market . When multiple agents select the same arm , only one agent can successfully pull the arm according to the arm ’ s preference . We denote [ m ] ⌘ { 1 , . . . , m } , [ n ] ⌘ { 1 , . . . , n } , and [ K ] ⌘ { 1 , . . . , K } . Decentralized matching markets require participants to make their decisions independently of others ’ decisions [ 33 , 35 ] . Notable examples of such markets include college admissions in the United States , Korea , and Japan , where P and A represent the sets of colleges and students , respectively [ 5 , 6 ] . Our goal is to learn the agent ’ s optimal strategy for maximizing the expected payoff . A strategy consists of deciding how many and which arms to pull at each stage . Agent ’ s decision-making in decentralized markets faces incomplete information about other agents ’ decisions and arms ’ preferences . Participants ’ preferences The agents ’ preferences are based on the arms ’ latent utilities . Consider the following latent utility model : Ui ( Aj ) = vj + eij , 8i 2 [ m ] , j 2 [ n ] , ( 1 ) where vj 2 [ 0 , 1 ] is arm Aj ’ s systematic score considered by all agents , and eij 2 [ 0 , 1 ] is an agent-specific idiosyncratic fit considered only by agent Pi , i 2 [ m ] . A utility model with a similar separable structure has been widely used in the matching market literature [ 4 , 13 , 15 ] . The arms ’ preferences have no restrictions and can involve uncertainty . From an agent ’ s perspective , arms accept offers with probabilities dependent on opponents ’ strategies and arms ’ preferences . Let the parameter si , k 2 [ 0 , 1 ] be the state of the world [ 36 ] for agent Pi , such that the probability that an arm Aj accepts Pi at stage k is ⇡i , k ( si , k , vj ) , 8i 2 [ m ] , j 2 [ n ] , k 2 [ K ] . Since agents compete for arms with a higher score , the acceptance probability ⇡i , k ( si , k , vj ) models the agents ’ competition through the dependence on the score vj . Moreover , ⇡i , k ( si , k , vj ) incorporates the arm ’ s uncertain preference into the state si , k . It is known that there exists a valid probability mass function ⇡i , k ( si , k , vj ) [ 15 ] . We assume that ⇡i , k ( si , k , vj ) is strictly increasing and continuous in si , k . Thus , a larger value of the state si , k corresponds to the case that agent Pi is more popular . In practice , the true state is unknown a priori to Pi and needs to be estimated from data . For instance , the yield in college admissions is defined as the rate at which a college ’ s admitted students accept the offers . However , the yield is unknown a priori to the college in the current year [ 13 ] . Colleges can only estimate the distribution of the yield from historical data . In this paper , we study a nonparametric model of ⇡i , k ( · , · ) by assuming it belongs to a reproducing kernel Hilbert space ( RKHS ) [ 3 , 39 ] . Later , in Section 3.2 , we propose an algorithm for calibrating si , k and efficiently estimating ⇡i , k ( · , · ) using historical data . Given the latent utility Ui ( Aj ) and the acceptance probability ⇡i , k ( si , k , vj ) , agent Pi ’ s expected utility of pulling arm Aj at stage k is ( vj + eij ) ⇡i , k ( si , k , vj ) . Timeline of the matching First , Nature draws a state such that arms ’ preferences are realized . Denote by s⇤i , k the true state for agent Pi at stage k. Next , arms display their interests to all agents . For example , students apply to colleges in a given period . Under the assumption that students incur negligible application costs , submitting applications to all colleges is the dominant strategy as students lack information on how colleges evaluate their academic ability or personal essays [ 6 , 13 ] . Next , at each stage k 2 [ K ] , agents simultaneously pull available arms that have not previously rejected them . Each arm either accepts one of the agents that pulled it ( if any ) or rejects all . An arm exits the market once it accepts an agent , and agents are allowed to exit the market at any time . The arms act simultaneously at each stage . They can not “ hold '' offers for accepting or rejecting at a later stage . Hence , agents make “ exploding '' offers , and arms have to make irreversible decisions without knowing what other offers are coming in later stages . Finally , this multi-stage matching process ends when all agents have exited or when a pre-specified number of stages has been reached . If there remain arms in the market when the matching has terminated , these arms are unmatched . Agent ’ s expected payoff An agent ’ s goal is to maximize the expected payoff , which consists of two parts : the expected utilities and the penalty for exceeding the quota . Let Ak be the set of arms that are available in the market at stage k 2 [ K ] . Suppose that agent Pi pulls arms from the set Bi , k ✓ { Ak \ [ lk 1Bi , l } at stage k , where A \ B denotes that set A minus set B . Let Ci , k ✓ Bi , k be the set of arms that accept Pi at stage k. Then Ci , k is unknown until stage k+1 , where k K 1 , and Ci , K is unknown until the end of the matching process . Then Pi ’ s expected payoff at stage k 2 [ K ] is lower bounded by Ui , k [ Bi , k ] ⌘ X j2Bi , k ( vj + eij ) ⇡i , k ( s ⇤ i , k , vj ) i max { Ni , k ( Bi , k ) qi , 0 } . ( 2 ) Here Ni , k ( Bi , k ) ⌘ P j2Bi , k ⇡i , k ( s⇤i , k , vj ) + card ( [ lk 1Ci , l ) , and s ⇤ i , k is the true state for agent Pi at stage k. We assume that the marginal penalty i satisfies i > maxj2A { vj + eij } , which implies that the penalty is greater than arm ’ s latent utility . Since our model involves unknown strategies of the opponents and uncertain arms ’ preferences , we consider the optimal expected payoff in ( 2 ) instead of the optimal realized payoff . Similar expected payoff have been studied in [ 13 , 15 ] .

Summary: The authors study a multi sided matching market under a statistical model, with the goal of finding the optimal strategies. The model uses agents and arms, but due to its applications the intuition is easier to capture assuming the agents are universities and the arms students. Specifically, there are $n$ agents and $m$ arms, which are pulled over $K$ rounds. Each agent has a quota $q_i \ge 1$ of arms they can use, and the sum of all quotas is at most $n$. The market is two-sided because the arms (students) have preferences over the agents (universities) (and these preferences can be fuzzy) and the agents (universities) have preferences over the arms (students): each arm $j$ has an intrinsic value $v_j$ and an extra value $e_{ij}$, specific to agent $i$.  At every step, each agent can pull any number of arms. However, each arm can only respond to at most one agent, or reject all of them. The arms don’t have such explicit preferences: their interaction with the agents is captures by the probability $\pi_{i,k}(s_{i,k}, v_j)$ that arm $i$ will accept agent $j$ at time $k$, where $s_{i,k}$ is the state at time $k$. This probability is generally not known but is inferred from data. Overall, agent $i$’s utility is the sum of $(v_j + e_{ij}) \cdot \pi_{i,k}(s_{i,k}, v_j) $ over every turn $k$ and all arms they tried using, minus a penalty if the quota $q_i$ is exceeded. Clearly, this induces a game between the agents, which have to decide how to use their quotas to select arms that are good enough, but not so good that they might be snatched by other agents.  The authors provide a variety of results:  1. Three algorithms (one of which is their main result) on the side of learning the optimal strategy from the perspective of agent $i$. 2. Fairness and Welfare trade-offs. 3. Experiments.   In particular, the first algorithm yields an optimal strategy for maximising the welfare of agent $i$ by minimising their *variational loss*, which is (roughly described) a modification of their utility with an added factor depending on $\max_{s_{i,k}} \pi_{i,k}(s_{i,k}, v_j)  - \min_{s_{i,k}} \pi_{i,k}(s_{i,k}, v_j)$ (the ‘variance’) and weighted by $\eta_{i, k}$, which reflects that subsequent rounds are typically less valuable than the first ones. Intuitively this makes sense: minimising the variance would maximise the way the quota is used. This strategy is NP-hard to solve for though, which leads to the development of a *greedy* approximation: rather than immediately solving for variance, the arms are chosen greedily based on their expected utility per unit of acceptance probability. The second algorithm (and arguably main result of the paper) uses historic data to estimate the $\pi_{i,k}(s_{i,k}, v_j)$’s. The actual algorithm has many details, but I hope is fairly represent an abstraction as follows: 1. Use an off the shelf estimator for the $\pi_{i,k}(s_{i,k}, v_j)$. 2. Modify them by subtracting a weighted variational term (as in the first algorithm), or set to 1 if the arm has not been used. 3. Balance exploration and exploitation of the arms used.  The fairness results revolve around *justified envy*: in a nutshell, an arm would have justified envy if it is pulled by agent $i’$, but agent $i$ (which the arm prefers) is trying a different arm on the same turn. This would reflect the real world situation where highly skilled individual might miss an offer because a potential employer might think they have a high chance of rejecting it. The authors show that higher uncertainly results in higher probability of an arm being envious. Moreover, the higher the $\eta_{i,k}$ terms are, the higher the number of envious arms. Finally, they compare with the single-stage matching markets (which do not produce envy in their optimal outcome) and show that multi-stage markets provide increased welfare, at the expense of fairness.  The authors also have experiments on synthetic data that support their learning procedure, comparing to a simpler strategy, where every agent selects their favourite arms within their remaining quota, at every step. As expected, both strategies perform well for popular agents, but have very different performance for less desirable ones. Again, this reflects that minimising variance (which is easier for popular agents: imagine a top university offering positions to students, knowing most would accept) is the key, which is further supported by extrapolating from real data sets of admissions and university rankings.