Learning Equilibria in Matching Markets from Bandit Feedback

1 Introduction . Data-driven marketplaces face the simultaneous challenges of learning agent preferences and aligning market outcomes with the incentives induced by these preferences . Consider , for instance , online platforms that match two sides of a market to each other ( e.g. , Lyft , TaskRabbit , and Airbnb ) . On these platforms , customers are matched to service providers and pay for the service they receive . If agents on either side are not offered desirable matches at fair prices , they would have an incentive to leave the platform and switch to a competing platform . Agent preferences , however , are often unknown to the platform and must be learned . When faced with uncertainty about agent preferences ( and thus incentives ) , when can a marketplace efficiently explore and learn market outcomes that align with agent incentives ? We center our investigation around a model called matching with transferable utilities , proposed by Shapley and Shubik [ 32 ] . In this model , there is a two-sided market of customers and service providers . Each customer has a utility that they derive from being matched to a given provider and vice versa . The platform selects a matching between the two sides and assigns a monetary transfer between each pair of matched agents . Transfers are a salient feature of most real-world matching †Equal contribution . 1A full version of this paper ( referenced as [ 18 ] ) is available at https : //arxiv.org/abs/2108.08843 . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . markets : riders pay drivers on Lyft , clients pay freelancers on TaskRabbit , and guests pay hosts on Airbnb . An agent ’ s net utility is their value for being matched to their partner plus the value of their transfer ( either of which can be negative in the cases of costs and payments ) . In matching markets , the notion of stability captures alignment of a market outcome with agent incentives . Informally , a market outcome is stable if no pair of agents would rather match with each other than abide by the market outcome , and stable matchings can be computed when preferences are fully known . However , in the context of large-scale matching platforms , the assumption that preferences are known breaks down . Platforms usually can not have users report their complete preference profiles . Moreover , users may not even be aware of what their own preferences are . For example , a freelancer may not exactly know what types of projects they prefer until actually trying out specific ones . In reality , a data-driven platform is more likely to learn information about preferences from repeated feedback over time . Two questions now emerge : In such marketplaces , how can stable matchings be learned ? And what underlying structural assumptions are necessary for efficient learning to be possible ? Toward answering these questions , we propose and investigate a model for learning stable matchings from noisy feedback . We model the platform ’ s learning problem using stochastic multi-armed bandits , which lets us leverage the extensive body of techniques in the bandit literature to analyze the data efficiency of learning . Specifically , our main contributions are : ( i ) We develop an incentiveaware learning objective—Subset Instability—that captures the distance of a market outcome from equilibrium . ( ii ) Using Subset Instability as a measure of regret , we show that any “ UCB-based ” algorithm can be adapted to this incentive-aware setting . ( iii ) We instantiate this idea for two families of preference structures to design efficient algorithms for incentive-aware learning , helping elucidate how preference structure affects the complexity of learning stable matchings . Designing the learning objective . Since mistakes are inevitable while exploring and learning , achieving exact stability at every time step is an unattainable goal . To address this issue , we lean on approximation , focusing on learning market outcomes that are approximately stable . Thus , we need a metric that captures the distance of a market outcome from equilibrium . We introduce a notion for approximate stability that we call Subset Instability . Specifically , we define the Subset Instability of a market outcome to be the maximum difference , over all subsets S of agents , between the total utility of the maximum weight matching on S and the total utility of S under the market outcome.We show Subset Instability can be interpreted as how much the platform would have to subsidize participants to keep them on the platform and make the resulting matching stable . We also show that Subset Instability is the maximum gain in utility that a coalition of agents could have derived from an alternate matching such that no agent in the coalition is worse off . Subset Instability also satisfies the following properties , which make it suitable for learning : ( i ) Subset Instability is equal to 0 if and only if the market outcome is ( exactly ) stable ; ( ii ) Subset Instability is robust to small perturbations to the utility functions of individual agents , which is essential for learning with noisy feedback ; ( iii ) Subset Instability upper bounds the utility difference of a market outcome from the socially optimal market outcome . Designing algorithms for learning a stable matching . Using Subset Instability , we investigate the problem of learning a stable market outcome from noisy user feedback in a stochastic bandit model . In each round , the platform selects a market outcome ( i.e. , a matching along with transfers ) , with the goal of minimizing cumulative instability . We develop a general approach for designing bandit algorithms within our framework . Our approach is based on a primal-dual formulation of matching with transfers [ 32 ] We find that “ optimism in the face of uncertainty , ” the principle underlying many UCB-style bandit algorithms [ 3 , 22 ] , can be adapted to this primal-dual setting . The resulting algorithm is simple : maintain upper confidence bounds on the agent utilities and compute , in each round , an optimal primal-dual pair in terms of these upper confidence bounds . Following this approach , we show regret bounds for two sets of structural assumptions on agent preferences , which we now state . Theorem 1.1 ( Unstructured Preferences , Informal ) . There exists a UCB-style algorithm that incurs Õ ( N √ nT ) regret according to Subset Instability after T rounds , where N is the number of agents on the platform and n is the number of agents that arrive in any round . ( In fact , this bound is optimal up to logarithmic factors . ) Theorem 1.2 ( Separable Linear Preferences , Informal ) . Suppose each agent a gets utility 〈φ ( a ) , ca′〉 from matching with a′ , where φ ( a ) ∈ Rd is unknown and ca′ ∈ Rd is known . Then there exists a UCB-style algorithm that incurs Õ ( d √ N √ nT ) regret according to Subset Instability after T rounds , for N the number of agents on the platform and n the maximum number of agents in any round . These results highlight the role of preference structure on the complexity of learning a stable matching . To see this concretely , consider the case where all agents show up each round , so n = N . In this case , our regret bound for unstructured preferences is superlinear in N ; and this dependence on N is necessary due to a lower bound ( see Lemma 5.4 ) . On the other hand , our regret bound is linear in N for separable linear preferences . This means that in large markets , a centralized platform can efficiently learn a stable matching with this preference structure assumption . Extensions . In the full version [ 18 ] , we provide several extensions of our framework and results to additional classes of preference structures , alternative forms of regret bounds , and richer platform objectives , and other models of matching markets . In Section 6 , we briefly discuss two of these extensions—instance-dependent regret bounds and matching markets with non-transferable utilities . 1.1 Related work . In the machine learning literature , starting with Das and Kamenica [ 11 ] and Liu et al . [ 24 ] , several works [ 11 , 24 , 30 , 25 , 7 , 6 ] study learning stable matchings from bandit feedback in the Gale-Shapley stable marriage model [ 15 ] . A major difference between this setting and ours is the absence of monetary transfers between agents . These works focus on the utility difference rather than the instability measure that we consider . Cen and Shah [ 7 ] extend this bandits model to incorporate fixed , predetermined cost/transfer rules . However , they do not allow the platform to set arbitrary transfers between agents . Moreover , they also consider a weaker notion of stability that does not consider agents negotiating arbitrary transfers : defecting agents must set their transfers according to a fixed , predetermined structure . In contrast , we follow the classical definition of stability [ 32 ] . Outside of machine learning , several papers also consider the complexity of finding stable matchings in other feedback and cost models , e.g. , communication complexity [ 16 , 2 , 33 ] and query complexity [ 12 , 2 ] . Of these works , Shi [ 33 ] , which studies the communication complexity of finding approximately stable matchings with transferable utilities , is most similar to ours . Shi assumes agents know their preferences and focuses on the communication bottleneck , whereas we study the costs associated with learning preferences . Moreover , the approximate stability notion in Shi [ 33 ] is the maximum unhappiness of any pair of agents , whereas Subset Instability measures the maximum unhappiness over any subset of agents . For learning stable matchings , Subset Instability has the advantages of being more fine-grained and having a primal view that motivates a clean UCB-based algorithm . Our notion of instability connects to historical works in coalitional game theory : related are the concepts of the strong-ε core of Shapley and Shubik [ 31 ] and the indirect function of Martínez-Legaz [ 27 ] , although each was introduced in a very different context than ours . Nonetheless , they reinforce the fact that our instability notion is a very natural one to consider . Multi-armed bandits have also been applied to learning in other economic contexts . For example , learning a socially optimal matching ( without learning transfers ) is a standard application of combinatorial bandits [ 8 , 14 , 9 , 10 , 21 ] . Other applications at the interface of bandit methodology and economics include dynamic pricing [ 28 , 20 , 5 ] , incentivizing exploration [ 13 , 26 ] , learning under competition [ 1 ] , and learning in matching markets without incentives [ 19 ] . Finally , primal-dual methods have also featured elsewhere in the bandits literature ( e.g. , [ 17 , 34 , 23 ] ) . 2 Preliminaries . The foundation of our framework is the matching with transfers model of Shapley and Shubik [ 32 ] . In this section , we introduce this model along with the concept of stable matching . 2.1 Matching with transferable utilities . Consider a two-sided market that consists of a finite set I of customers on one side and a finite set J of providers on the other . Let A : = I ∪ J be the set of all agents . A matching X ⊆ I × J is a set of pairs ( i , j ) that are pairwise disjoint , representing the pairs of agents that are matched . Let XA denote the set of all matchings on A . For notational convenience , we define for each matching X ∈XA an equivalent functional representation µX : A → A , where µX ( i ) = j and µX ( j ) = i for all matched pairs ( i , j ) ∈ X , and µX ( a ) = a if a ∈ A is unmatched . When a pair of agents ( i , j ) ∈ I × J matches , each experiences a utility gain . We denote these utilities by a global utility function u : A × A → R , where u ( a , a′ ) denotes the utility that agent a gains from being matched to agent a′ . ( If a and a′ are on the same side of the market , we take u ( a , a′ ) to be 0 by default . ) We allow these utilities to be negative , if if matching results in a net cost ( e.g. , if an agent is providing a service ) . We assume each agent a ∈ A receives zero utility if unmatched , i.e. , u ( a , a ) = 0 . When we wish to emphasize the role of an individual agent ’ s utility function , we will use the equivalent notation ua ( a′ ) : = u ( a , a′ ) . A market outcome consists of a matching X ∈XA along with a vector τ ∈ RA of transfers , where τa is the amount of money transferred from the platform to agent a for each a ∈ A . These monetary transfers are a salient feature of most real-world matching markets : riders pay drivers on Lyft , clients pay freelancers on TaskRabbit , and guests pay hosts on Airbnb . Shapley and Shubik [ 32 ] capture this aspect of matching markets by augmenting the classical two-sided matching model with transfers of utility between agents . Transfers are typically required to be zero-sum , meaning that τi + τj = 0 for all matched pairs ( i , j ) ∈ X and τa = 0 if a is unmatched . Here , X represents how agents are matched and τa represents the transfer that agent a receives ( or pays ) . The net utility that an agent a derives from a matching with transfers ( X , τ ) is therefore u ( a , µX ( a ) ) + τa . Stable matchings . In matching theory , stability captures when a market outcome aligns with individual agents ’ preferences . Roughly speaking , a market outcome ( X , τ ) is stable if : ( i ) no individual agent a would rather be unmatched , and ( ii ) no pair of agents ( i , j ) can agree on a transfer such that both would rather match with each other than abide by ( X , τ ) . Formally : Definition 2.1 . A market outcome ( X , τ ) is stable if : ( i ) it is individually rational , i.e. , ua ( µX ( a ) ) + τa ≥ 0 ( 1 ) for all agents a ∈ A , and ( ii ) it has no blocking pairs , i.e. , ( ui ( µX ( i ) ) + τi ) + ( uj ( µX ( j ) ) + τj ) ≥ ui ( j ) + uj ( i ) ( 2 ) for all pairs of agents ( i , j ) ∈ I × J .2 A fundamental property of the matching with transfers model is that if ( X , τ ) is stable , then X is a maximum weight matching , i.e. , X maximizes ∑ a∈A ua ( µX ( a ) ) over all matchings X ∈XA [ 32 ] . The same work shows that stable market outcomes coincide with Walrasian equilibria . To make the matching with transfers model concrete , we use the simple market depicted in the center panel of Figure 1 as a running example throughout the paper . This market consists of a customer Charlene and two providers Percy and Quinn , which we denote by I = { C } and J = { P , Q } . If the agents ’ utilities are as given in Figure 1 , then Charlene would prefer Quinn , but Quinn ’ s cost of providing the service is much higher . Thus , matching Charlene and Percy is necessary for a stable outcome . This matching is stable for any transfer from Charlene to Percy in the interval [ 5 , 7 ] .

Summary: The paper considers two-sided platforms who seek to match users on   both sides under uncertainty about user preferences. The focus is on   the transferable utility setting, where the platform, in addition to   orchestrating a matching, imposes transfers between each pair (as   well as from the platform to the pair). After each matching, the   platform receives noisy (bandit) feedback regarding the utilities   received by each member in a matched pair. The goal of the platform   is to learn a stable market outcome efficiently. To capture the   (in)efficiency of learning, the authors introduce a measure of   instability of any matching, namely subsidy instability, which is   equal to the smallest total (non-negative) transfer the platform   needs to make to the users so that matching is stable. The regret of   any learning algorithm is then defined as the total subsidy   instability accumulated over a time horizon.    The authors first show that the subsidy instability is indeed an   appropriate notion to capture learning efficiency, by providing   natural economic interpretations as well as exhibiting a number of   smoothness properties and bounds. The authors then propose a natural   UCB-type algorithm, where each agent reports upper confidence bounds   on their preferences and the platform computes the optimal stable   matching and transfers under these reported preferences. The authors   show that this algorithm achieves an expected regret that is of   order $O(\sqrt{T\log T})$, and furthermore provide a matching lower   bound. (The bounds also establish regret dependence on other model   parameters.)