Bandit Learning in Many-to-one Matching Markets with Uniqueness Conditions
Anonymous Author(s)
AffiliationAddress email
Abstract
An emerging line of research is dedicated to the problem of one-to-one matchingmarkets with bandits, where the preference of one side is unknown and thus weneed to match while learning the preference through multiple rounds of interaction.
However, in many real-world applications such as online recruitment platform forshort-term workers, one side of the market can select more than one participant fromthe other side, which motivates the study of the many-to-one matching problem.
Moreover, the existence of a unique stable matching is crucial to the competitiveequilibrium of the market.
In this paper, we first introduce a more general new ˜ αcondition to guarantee the uniqueness of stable matching in many-to-one matchingproblems, which generalizes some established uniqueness conditions such as SPCand Serial Dictatorship , and recovers the known α -condition if the problem isreduced to one-to-one matching.
Under this new condition, we design an MOUCB-D4 algorithm with O (cid:16) NK log( T ) ∆(cid:17) regret bound, where T is the time horizon,
N is the number of agents, K is the number of arms, and ∆ is the minimumreward gap.
Extensive experiments show that our algorithm achieves uniform goodperformances under different uniqueness conditions.
Introduction
The rise of platforms for the online matching market has led to an emergence of opportunities forcompanies to participate in personalized decision-making [14, 18].
Companies (like Thumbtackand Taskrabbit and Upwork platforms) use online platforms to address short-term needs or seasonalspikes in production demands, accommodate workers who are voluntarily looking for more flexiblework arrangements or probation period before permanent employment.
The supply and demandsides in two-sided markets make policies on the basis of their diversified needs, which is abstractedas a matching market with agent side and arm side and each side has a preference profile over theopposite side.
They choose from the other side according to preference and perform a matching.
Thestability of the matching result is a key property of the market [31, 1, 26].
The preferences in the online labor market may be unknown to one side in advance, thus matchingwhile learning the preferences is necessary.
The multi-armed bandit (MAB) [35, 13, 4] is an importanttool for N independent agents in matching market simultaneously selecting arms adaptively fromreceived rewards at each round.
The idea of applying MAB to one-to-one matching problems,introduced by [21], assumes that there is a central platform to make decisions for all agents.
Followingthis, other works [22, 33, 7] consider a more general decentralized setting where there is no centralplatform to arrange matchings, and our work is also based on this setting.
However, it is not enough to just study the one-to-one setting.
Take online short-term workeremployment as an example, it is an online platform design with an iterative matching, where employers have numerous similar short-term tasks or internships to be recruited.
Workers can onlychoose one task according to the company’s needs at a time while one company can accept morethan one employee.
Each company makes a fixed ranking for candidates according to its ownrequirements but workers have no knowledge of companies’ preferences.
The reward for workersis a comprehensive consideration of salary and job environment.
Since tasks are short-term, eachcandidate can try many times in different companies to choose the most suitable job.
We abstractcompanies as arms and workers as agents.
Each arm has a capacity q which is the maximum numberof agents this arm can accommodate.
When an arm faces multiple choices, it accepts its most qpreferred agents.
Agents thus compete for arms and may receive zero reward if losing the conflict.
Itis worth mentioning that arms with capacity q in the many-to-one matching can not just be replacedby q independent individuals with the same preference since there would be implicit competitionamong different replicates of this arm, not equal treatment.
In addition, when multiple agents selectone arm at a time, there may be no collision, which will hinder the communication among differentagents under the decentralized assumption.
They cannot distinguish who is more preferred by thisarm in one round as it can accept more than one agent while this can be done in one-to-one case.
Communication here lets each agent learn more about the preferences of arms and other agents, so asto formulate better policies to reduce collisions and learn fast about their stable results.
This work focuses on a many-to-one market under uniqueness conditions.
Previous work [10, 15]emphasize the importance of constructing a unique stable matching for the equilibrium of matchingproblems and some existing uniqueness conditions are studied in many-to-one matching, such as
Sequential Preference Condition (SPC) and Acyclicity [25, 2].
Our work is motivated by [7], but theunique one-to-one mapping between arms and agents in their study which gives a surrogate thresholdfor arm elimination does not work in the many-to-one setting.
And the uniqueness conditions inmany-to-one matching are not well-studied, which also brings a challenge to identify and leveragethe relationship between the resulting stable matching and preferences of two sides in the designof bandit algorithms.
We propose an ˜ α -condition that can guarantee a unique stable matching andrecover α -condition [19] if reduced to the one-to-one setting.
We establish the relationships betweenour new ˜ α -condition and existing uniqueness conditions in many-to-one setting.
In this paper, we study the bandit algorithm for a decentralized many-to-one matching marketwith uniqueness conditions.
Under our newly introduced ˜ α -condition , we design an MO-UCB-D4algorithm with arm elimination and the regret can be upper bounded by O (cid:16) NK log( T ) ∆(cid:17) , where Nis the number of agents, K is the number of arms, and ∆ is the minimum reward gap.
Finally,we conduct a series of experiments to simulate our algorithm under various conditions of Serialdictatorship , SPC and ˜
α -condition to study the stability and regret of the algorithm.
Related Work
The study of matching markets has a long history in economics and operationresearch [8, 6, 31] with real applications like school enrollment, labor employment, hospital resourceallocation, and so on [1, 23, 30, 17].
A salient feature of market matching is making decisions forcompeting players on both sides [35, 12].
MAB is an important tool to study matching problems underuncertainty to obtain a maximum reward, and upper confidence bound algorithm (UCB) [4] is a typicalalgorithm, which sets a confidence interval to represent uncertainty.
Matching market with MAB isstudied in both centralized and decentralized setting [21, 22].
Following these, Abishek Sankararamanet al.
[33] propose a phased UCB algorithm under a uniqueness condition, Serial Dictatorship , tomanage collisions.
They solve the problem of the decentralized market without knowing arm-gapsor time horizon, and reduce the probability of linear regret through non-monotonic arm elimination.
The introduction of the uniqueness condition plays an important role in the equilibrium of matchingresults [15, 7].
Under a stronger and robust condition, Uniqueness Consistency [19], Soumya Basuet.al [7] apply MAB to online matching and obtain robust results that the subset of stable matchingsbeing separated from the system does not affect other stable matchings.
We discuss many-to-one problems such as online short-term employment and MOOC [14, 24, 18] asthe one-to-one setting has limitations in practice.
Somouaoga Bonkoungo [9] runs a student-proposingdeferred acceptance algorithm (DA) [12] to study decentralized college admission.
Ahmet Altinok[3] considers dynamic matching in many-to-one that can be solved as if it is static many-to-one ordynamic one-to-one under certain assumptions.
As the existence and uniqueness of competitiveequilibrium and core are important to allocations, the unique stable results need to be considered [26].
Similar to conditions for unique stable matching in one-to-one, some uniqueness conditions of stableresults in the many-to-one setting also are studied [16, 27, 15, 2, 26].
Setting
This paper considers a many-to-one matching market M =
( K , J , P ) , where K = [ K ] , J =
[ N ]are a finite arm set and a finite agent set, respectively.
And each arm k has a capacity q k ≥ 1 .
Toguarantee that no agents will be unmatched, we focus on the market with N ≤ (cid:80)
Ki =
1 q i .
P is thefixed preference order of agents and arms, which is ranked by the mean reward.
We assume that armpreferences for agents are unknown and needed to be learned.
If agent j prefers arm k over k ′ , whichalso means that µ j,k > µ j,k ′ , we denote by k ≻ j k ′ .
And the preference is strict that µ j,k ̸ = µ j,k ′ ifk ̸ = k ′ .
Similarly, each arm k has a fixed and known preference ≻
k over all agents, and specially,j ≻ k j ′ means that arm k prefers agent j over j ′ .
Throughout, we focus on the market where allagent-arm pairs are mutually acceptable , that is, j ≻ k ∅ and k ≻
j ∅ for all k ∈ [ K ] and j ∈ [ N ] .
Let mapping m be the matching result.
m t ( j ) is the matched arm for agent j at time t , and γ t ( k ) isthe agents set matched with arm k 1 .
Every time agent j selects an arm I t ( j ) , and we use M t ( j ) todenote whether j is successfully matched with its selected arm.
M t ( j ) =
if agent j is matched with
I t ( j ) , and M t ( j ) = 0 , otherwise.
If multiple agents select arm k at the same time, only top q k agentscan successfully match.
The agent j matched with arm k can observe the reward X j,m t ( j ) ( t ) , wherethe random reward X j,k ( t ) ∈ [0 , 1] is independently drawn from a fixed distribution with mean µ j,k .
While the unmatched ones have collisions and receive zero reward.
Generally, the reward obtained byagent j is X j,I t ( j ) ( t ) M t ( j ) .
An agent j and an arm k form a blocking pair for a matching m if they are not matched but prefereach other over their assignments, i.e. k ≻ j m ( j ) and ∃ j ′ ∈ γ ( k ) , j ≻ k j ′ .
We say a matchingsatisfies individually rationality (IR), if a j ≻ p i ∅ and p i ≻ a j ∅ for all i ∈
[ N ] and j ∈ [ K ] , that is,every worker prefers to find a job rather than do nothing, and every company also wants to recruitworkers rather than not recruit anyone.
Under the IR condition, a matching in the many-to-one settingis stable if there does not exist a blocking pair [32, 34].
This paper considers the matching markets under the uniqueness condition.
Thus the overall goal isto find the unique stable matching between the agent side and arm side through iterations.
Let m ∗ ( j )be the stable matched arm for agent j under the stable matching m ∗ .
The reward obtained by agent jis compared against the reward received by matching with m ∗ ( j ) at each time.
We aim to minimizethe expected stable regret for agent j over time horizon T , which is defined as
[Equation]
Algorithm
In this section, we introduce our MO-UCB-D4 Algorithm (Many-to-one UCB with Decentralized
Dominated arms Deletion and Local Deletion Algorithm)
(Algorithm 1) for the decentralized manyto-one market, where there is no platform to arrange actions for agents, which leads to conflictsamong agents.
The MO-UCB-D4 algorithm for each agent j first takes agent set J and arm set K asinput and chooses a parameter θ ∈ (0 , 1 /K )
(discussed in Section C).
It sets multiple phases, andeach phase i mainly includes regret minimization block (line 6 - 12) and communication block (line13 - 16) with duration 2 i − 1 , i = 1 , 2 , · · · .
For each agent j in phase i , the algorithm adds arm deletion to reduce potential conflicts, whichmainly contains global deletion and local deletion.
The former eliminates the arms most preferredby agents who rank higher than agent j and obtain active set
Ch j
[ i ] (line 4), and the latter deletesthe arms that still have many conflicts with agent j after global deletion (line 6).
We set a collisioncounter C j,k
[ i ] to record the number of collisions for agent
j pulling arm k .
In regret minimization block of phase i , we use L j [ i ] =
{ k : C j,k
[ i ] ≥ ⌈ θ 2 i ⌉} to represent thearms that collide more times than a threshold ⌈ θ 2 i ⌉ when matching with agent j .
Arms in L j [ i ] arefirst locally deleted to reduce potential collisions for agent j (line 6).
After that, agent j selects anoptimal action I t ( j ) from remaining arms in Ch j
[ i ] \ L j [ i ] in phase i according to UCB index, which iscomputed by ˆ µ j,k ( t − 1)+ (cid:113) 2 α log( t ) N j,k ( t − 1) (line 7), where N j,k ( t − 1) is the number that agent j and arm
Input: θ ∈ (0 , 1 /K ) , α > 1 .
1: Set global dominated set G j [0] =
ϕ 2: for phase i = 1 , 2 , ... do 3:
Reset the collision set C j,k
[ i ] = 0 , ∀ k ∈
[ K ] ; 4:
Reset active arms set Ch j
[ i ] =
[ K ] \ G j
[ i − 1] ; 5: if t <
i + NK ( i − 1) then 6:
Local deletion L j
[ i ] =
{ k : C jk [ i ] ≥ ⌈ θ 2 i ⌉} ; 7:
Play arm I t ( j ) ∈ arg max k ∈
Ch j [ i ] \ L j [ i ] (cid:16) ˆ µ j,k ( t − 1)
+ (cid:113) 2 α log( t ) N j,k ( t − 1) (cid:17) ; 8: if k = I t ( j ) is successfully matched with agent j , i.e. m t ( j ) =
k then 9:
Update estimate ˆ µ j,k ( t ) and matching count N j,k ( t ) ; 10: else 11:
C j,k
[ i ] =
C j,k
[ i ] + 1 ; 12: end if 13: else if t =
i + NK ( i − 1) then 14:
O j
[ i ] ← most matched arm in phase i ; 15:
G j [ i ] ← COMMUNICATION ( i, O j [ i ]) ; 16: end if 17: end for k have been matched at time t − 1 .
If the selected arm is successfully matched with agent j , then thealgorithm updates estimated reward ˆ µ j,k ( t ) =
1 N j,k ( t ) (cid:80) ts =1 1 { I s ( j ) =
k and M s ( j ) = 1 } X j,k ( t )and N j,k ( t ) (line 9).
Otherwise, the collision happens (line 11) and j receives zero reward.
Theregret minimization block identifies the most played arm O j
[ i ] for agent j in each phase i , which isestimated as the best arm for j , thus making optimal policy to minimize expected regret.
Algorithm
In the communication block (Algorithm 2), there are N sub-blocks, each with duration K .
In theℓ − th sub-block, only agent ℓ pulls arm 1 , arm 2 , · · · , arm K in round-robin while the other agentsselect their most preferred arms estimated as the most played ones (line 4).
This block aims to detectglobally dominated arms for agent j : G j [ i ] ⊂ {O j ′
[ i ] : j ′ ≻
O j ′
[ i ] j } .
Under stable matching m ∗ , theglobally dominated arms set for agent j is denoted as G ∗ j .
After the communication block in phasei , each agent j updates its active arms set Ch j [ i + 1] for phase i + 1 , by globally deleting arms set
G j [ i ] , and enters into the next phase (line 4 in Algorithm 1).
Hence, multi-phases setting can guarantee that the active set in different phases has no inclusionrelationship so that if an agent deletes an arm in a certain phase, this arm can still be selected in thelater rounds.
This ensures that each agent will not permanently eliminate its stable matched arm, andwhen the agent mistakenly deletes an arm, it will not lead to linear regret.
Results
4.1 Uniqueness Conditions
4.1. ˜ α -condition
Constructing a unique stable matching plays an important role in market equilibrium [15] and someuniqueness conditions have been studied for many-to-one market [25, 2, 27].
In this section, wepropose a new ˜ α -condition to guarantee a unique stable result.
First, we introduce uniquenessconsistency (Unqc) [19], which guarantees robust and stable results in MO-UCB-D4 algorithm witharm deletion.
Definition 1.
A preference profile satisfies uniqueness consistency if and only if(i) there exists a unique stable matching m ∗ ;(ii) for any subset of arms or agents, the restriction of the preference profile on this subset with theirstable-matched pair has a unique stable matching.
It guarantees that even if an arbitrary subset of agents are deleted out of the system with theirrespective stable matched arms, there still exists a unique stable matching among the remainingagents and arms.
This condition allows any algorithm to identify at least one stable pair in a uniquestable matching system and guides the system to a global unique stable matching in an iterativemanner.
To obtain robust stable matching results in our setting, we propose a new ˜ α -condition in themany-to-one market to guarantee a unique stable matching with consistency, which is a sufficient andnecessary condition for Unqc (proved in Appendix B).
We considers a finite set of arms [ K ] = { 1 , 2 , · · · , K } and a finite set of agents [ N ] = { 1 , 2 , · · · , N }with preference profile P .
Assume that [ N ] r
=
{ A 1 , A 2 , · · · , A N } is a permutation of { 1 , 2 , · · · , N }and [ K ] r = { c 1 , c 2 , · · · , c K } is a permutation of { 1 , 2 , · · · , K } .
Denote [ N ] , [ K ] as the left order and[ N ] r , [ K ] r as the right order.
The k -th arm in the right order set [ K ] r has the index
c k in the leftorder set
[ K ] and the j -th agent in the right order set [ N ] r has the index
A j in the left order set [ N ] .
Considering arm capacity, we denote γ ∗ ( c k ) (right order) as the stable matched agents set for arm c k .
Definition 2.
A many-to-one matching market satisfies the ˜
α -condition
if,
(i)
The left order of agents and arms satisfies ∀ j ∈ [ N ] , ∀ k > j, k ∈
[ K ] , µ
[Equation]
,m ∗ ( j ) > µ j,k and m ∗ ( j ) is agent j ’s stable matched arm;
(ii)
The right order of agents and arms satisfies
[Equation]
where the set γ ∗ ( c k ) is more preferred than A (cid:80) k ′− 1 i =
1 q ci +1 means that the least preferred agent inγ ∗ ( c k ) for c k is better than A (cid:80) k ′− 1 i =
1 q ci +1 for c k .
Similar to α -condition in marriage problem [19], the ˜ α -condition detection contains two steps: Firstrun the Gale–Shapley algorithm in many-to-one setting [12] and find a stable matching m ∗ .
Nextfind two orders of arms and agents by sequentially eliminating agents and their the most preferredmatching pairs.
If the algorithm fails to find two orders, then the ˜ α -condition is violated.
Thiscondition is more general than existing uniqueness conditions like SPC [27] and can recover theknown α -condition in one-to-one matching market [19].
The relationship between the existinguniqueness conditions and our proposed conditions will be analyzed in detail later in Section 4.1.2.
Under our ˜ α -condition , the left order and the right order satisfy the following rule.
The left ordergives rankings according to agents’ preferences.
The first agent in the left order set [ N ] prefers arm1 in [ K ] most and has it as the stable matched arm.
Similar properties for the agent 2 to q 1 sincearm 1 has q 1 capacity.
Then the ( q 1 + 1) -th agent in the left order set [ N ] has arm 2 in [ K ] as herstable matched arm and prefers arm 2 most except arm 1 .
The remaining agents follow similarly.
Similarly, the right order gives rankings according to arms’ preferences.
The first arm 1 in theright order set [ K ] r most prefers first q c 1 agents in the right order set [ N ] r and takes them as its stable matched agents.
The remaining arms follow similarly.
The main idea from one-to-one tomany-to-one analysis is to replace individuals with sets.
In general, under ˜ α -condition , the left ordersatisfies that when arm 1 to arm k − 1 are removed, agents (cid:0) (cid:80) k − 1 i =1 q i + 1 (cid:1) to (cid:0) (cid:80) ki
=1 q i (cid:1) preferk most, and the right order means that when A 1 to agents A (cid:80) k − 1 i =
1 q
i are removed, arm k prefersagents
A k =
{ A (cid:80)
k
− 1 i =1 q ci +1 , A (cid:80)
k − 1 i =1 q ci +2 , · · · , A (cid:80)
ki =
1 q ci } , where A k is the agent set that are mostq k preferred by arm k among those who have not been matched by arm 1 , 2 , · · · , k − 1 .
Te nexttheorem give a summary.
Theorem 1.
If a market M = ( K , J , P ) satisfies ˜ α -condition, then m ∗ ( (cid:80) j − 1 i =1 q i + 1)
=m ∗ ( (cid:80) j − 1 i =
1 q i + 2) = · · · = m ∗ ( (cid:80) ji =1 q i ) =
j (the left order), γ ∗ ( c k ) =
A k and m ∗ ( A j ) =
c j (theright order) under stable matching.
Under ˜ α -condition, the stable matched arm may not be the most preferred one for each agent j ,j ∈ [ N ] , thus (i) we do not have m ∗ ( j ) to be dominated only by the agent 1 to agent j − 1 , i.e. theremay exist
j ′ >
j , s.t.
j ′ ≻ m ∗ ( j ) j ; (ii) the left order may not be identical to the right order, wedefine a mapping lr to match the index of an agent in the left order with the index in the right order,
i.e. A lr ( j ) =
j .
From Theorem 1, the stable matched set for arm k is its first q k preferred agentsγ ∗ ( c k )
=
A k .
We define lr as lr ( i ) =
max { j : A j ∈ γ ∗ ( m ∗ ( i )) , j ∈ [ N ] } , that is, in the rightorder, the mapping for arm k ∈ [ K ] is the least preferred one among its most q k preferred agents.
Note that this mapping is not an injective, i.e. ∃ j, j ′ , s.t. agent
j =
A lr ( j ) =
A lr ( j ′ ) .
An intuitiverepresentation can be seen in Figure 4 in Appendix A.1.
4.1.2 Unique Stable Conditions in Many-to-one Matching
Uniqueness consistency (Unqc) leads the stable matching to a robust one which is a desirable propertyin large dynamic markets with constant individual departure [7].
A precondition of Unqc is to ensureglobal unique stability, hence finding uniqueness conditions is essential.
The existing unique stable conditions are well established in one-to-one setting (analysis can befound in Appendix B), and in this section, we focus on uniqueness conditions in many-to-one market,such as SPC , [27], Aligned Preference , Serial Dictatorship Top-top match and Acyclicity [25, 2, 27](Definition 9, 7, 8, 10 in Appendix B.2). Takashi Akahoshi [2] proposes a necessary and sufficientcondition for uniqueness of stable matching in many-to-one matching where unacceptable agentsand arms may exist on both sides. We denote their condition as Acyclicity ∗ . Under our setting, bothtwo sides are acceptable, and we first give the proof of that Acyclicity ∗ is a necessary and sufficientcondition for uniqueness in this setting (see Section B.2.4 in Appendix B).
We then give relationshipsbetween our newly ˜ α -condition and other existing uniqueness conditions, intuitively expressed in
Figure 1, and we give proof for this section in Appendix B.2.
Lemma 1.
In a many-to-one matching market M =
( K , J , P ) , both Serial Dictatorship and Aligned
Preference can produce a unique stable matching and they are equivalent.
Theorem 2.
In a many-to-one matching market M =
( K , J , P ) , our ˜ α -condition satisfies:
(i) SPC is a sufficient condition
to
˜ α -condition;
(ii) ˜ α -condition is a necessary and sufficient condition to Unqc;
(iii) ˜ α -condition is a sufficient condition to Acyclicity ∗ .
4.2 Theoretical Results of Regret
We then provide theoretical results of MO-UCB-D4 algorithm under our ˜ α -condition.
Recall that G ∗ jis the globally dominated arms for agent j under stable matching m ∗ .
For each arm k / ∈ G ∗ j , we givethe definition of the blocking agents for arm k and agent j : B jk = { j ′ :
j ′ ≻
k j, k / ∈ G ∗ j } , whichcontains agents more preferred by arm k than j .
The hidden arms for agent j is H j
= { k : k / ∈
G ∗ j } ∩ { k : B jk ̸ = ∅} .
The reward gap for agent j and arm k is defined as ∆ jk = | µ j,m ∗ ( j ) − µ j,k |and the minimum reward gap across all arms and agents is ∆ =
min j ∈ [ N ] { min k ∈ [ K ] ∆ j,k } .
We 243 assume that the reward is different for each agent, thus ∆ j,k > 0 for every agent j and arm k .
Theorem 3.
(Regret upper bound)
Let J max ( j )
=
max { j + 1 , { j ′ :
∃ k ∈ H j , j ′ ∈ B jk }} be themax blocking agent for agent j and f ˜ α ( j )
=
j + lr max ( j ) is a fixed factor depends on both the left order and the right order for agent j .
Following MO-UCB-D4 algorithm with horizon T , the expectedregret of a stable matching under ˜ α -condition (Definition 2) for agent j ∈
[ N ] is upper bounded by
E [ R j ( T )] ≤ (cid:88)
[Equation]
where i ∗ = max { 8 , i 1 , i 2 } (then i ∗ ≤ 8 and i 1 , i 2 are defined in equation (3) ), and lr max ( j ) =
max { lr ( j ′ ) : 1 ≤ j ′ ≤ j } , is the maximum right order mapping for agent j ′ who ranks higher thanj .
From Theorem 3, the scale of the regret upper bound under ˜ α -condition is O (cid:16) NK log( T ) ∆(cid:17) and theproof is in Section 3.
Proof Sketch of Theorem 3 .
Under ˜ α -condition , we only need to discuss the regret of the uniqueresult.
We construct a good phase (in Appendix A.2) and denote that the time point of agent jreaching its good phase by τ j .
After τ j , agent j could identify its best arm and matches with hisstable pair.
Thus, from phase τ j on-wards, agent j + 1 will find the set of globally dominated arms
G ∗ j +1 and will eliminate arm m ∗ ( j ) if m ∗ ( j ) brings collisions in communication block accordingto Algorithm 1.
Global deletion here follows the left order.
Then when agent j enters into regretminimization block next phase, the times it plays a sub-optimal arm is small which leads to a smalltotal number of collisions experienced by agent j + 1 .
Then the process of each agent after goodphase is divided into two stages: before τ j and after τ j .
After τ j , according to the causes of regret, itis divided into four blocks: collision, local deletion, communication, and sub-optimal play.
Phasesbefore τ j can be bounded by induction.
The regret decomposition is bound by the following.
Lemma 2.
(Regret Decomposition)
For a stable matching under ˜ α -condition, the upper bound ofregret for the agent j ∈ [ N ] under our algorithm can be decomposed by:
[Equation]
where F αj , V αj are the time points when agent j enters into ˜ α -Good phase and ˜ α -Low Collisionphase respectively, mentioned as "good phase" above, are defined in Appendix A.2.
Difficulties and Solutions
While putting forward our ˜ α -condition in the many-to-one setting, many new problems need to betaken into account.
From one-to-one setting to many-to-one setting First, although we assume that arm preference isover individuals rather than combination of agents, the agents matched by one arm are not independent.
Specially, arms with capacity q can not just be replaced by q independent individuals with the samepreference.
Since there would be implicit competition among different replicates of this arm, and itcan reject the previously accepted agents when it faces a more preferred agent.
Secondly, collisionsamong agents is one of main causes of regret in decentralized setting, while capacity will hinder thecollision-reducing process.
In communication block, when two agents select one arm at a time, asan arm can accept more than one agent, these two cannot distinguish who is more preferred by thisarm, while it can be done in one-to-one markets.
Thus it is more difficult to identify arm preferencesfor each agent.
The lr in [7] is a one-to-one mapping that corresponds the agent index in the leftorder and the agent index in the right order, which is related to regret bound (Theorem 3 in [7] and Theorem 3 in our work).
While it does not hold in our setting.
To give a descriptive range of matchedresult for each arm under ˜ α -condition, we need to define a new mapping.
In order to solve these problems, we explain as follows:
First, since capacity influence the communication among agents, we add communication block and introduce an arm set G ∗ j , which willbe deleted before each phase to reduce collisions, where G ∗ j contains arms that will block agent jglobally under stable matching m ∗ .
Second, the idea from one-to-one to many-to-one is a transitionfrom individual to set.
It is natural to split sets into individuals or design a bridge to correspond setsto individuals.
We construct a new mapping lr (Figure 4 in Appendix A) from agent j in the left orderto agents in the right order under ˜ α -condition.
lr maps each arm k to the least preferred one of itsstable matched agents in the right order, thus giving a matching between individuals and individualsand constructing the range of the stable matched agents set (Theorem 1).
Except lr , capacity alsoinfluences regret mainly in communication block, as mentioned in the first paragraph.
From α -condition to ˜ α -condition
To extend α -condition to the many-to-one setting, it needsto define preferences among sets.
However, there might be exponential number of sets due to thecombinatorial structure and simply constraining preferences over all possible sets will lead to highcomplexity.
Motivated by α -condition which characterizes properties of matched pairs in one-to-onesetting, we come up with a possible constraint by regarding the arm and its least preferred agent in thematched set as the matched pair and define preferences according to this grouping.
It turns out thatwe only need to define the preferences of arms over disjoint sets of agents to complete the extensionas α -condition is defined under the stable matching, which can also fit the regret analysis well.
As asummary, there might be other possible ways to extend the α -condition but we present a successfultrial to not only give a good extension with similar inclusion relationships but also guarantee goodregret bound.
Experiments
In this section, we verify the experimental results of our MO-UCB-D4 algorithm (Algorithm 1) fordecentralized many-to-one matching markets.
For all experiments, the rankings of all agents andarms are sampled uniformly.
We set the reward value towards the least preferred arm to be 1 /Nand the most preferred one as 1 for each agent, then the reward gap between any adjacently rankedarms is ∆ = 1 /N .
The reward for agent j matches with arm k at time t X j,k ( t ) is sampled from
Ber( µ j,k ) .
The capacity is equally set as q = N/K .
We investigate how the cumulative regret andcumulative market unstability depend on the size of the market and the number of arms under threedifferent unique stability conditions: Serial Dictatorship , SPC , ˜ α -condition .
The former cumulativeregret is the total mean reward gap between the stable matching result and the simulated result, andthe latter cumulative unstability is defined as the number of unstable matchings in round t .
In ourexperiments, all results are averaged over 10 independent runs, hence the error bars are calculated asstandard deviations divided by √ 10 .
Varying the market size To test effects on two indicators, cumulative regret and cumulativeunstability, we first varying N with fixed K with market size of N ∈ { 10 , 20 , 30 , 40 } agents and K = 5 arms.
The number of rounds is set to be 100 , 000 .
The cumulative regret in Figure2(a)(c)(e) show an increasing trend with convergence as the number of agents increases under thesethree conditions.
When the number of agents increases, there is a high probability of collisionsamong different agents, resulting in the increase of cumulative regret.
Similar results for cumulativeunstability are shown in Figure 2(b)(d)(f).
When N is larger, the number of unstable pairs becomesmore.
With the increase of the number of rounds, both two indicators increase first and then tend tobe stable.
The jumping points are caused by multi-phases setting of MO-UCB-D4 algorithm.
[Figure]
[Table] [Table] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
10K 12K
E x p e c t e d C u m u l a t
i v e R e g r e t (e) -condition
K=20K=10K=5K= 0 50K 100K 150K 200K 250K 300K 350K 400K Time25K 50K 75K 100K 125K 150K 175K 200K
E x p e c t e d C u m u l a t
i v e U n s t a b ili t y (f) -condition
K=20K=10K=5K=
Varying arm capacity The number of arms K is chosen by K ∈ { 2 , 5 , 10 , 20 } , with N = 20 andq =
N/K .
The number of rounds we set is 400 , 000 .
With the increase of K , both the cumulativeregret in Figure 3(a)(c)(e) and the cumulative unstability in Figure 3(b)(d)(f) increase monotonously.
When K increases, the capacity q k for each arm k decreases, and then the number of collisionswill increase, which leads to an increase of cumulative regret.
And it also leads to more unstablepairs, which needs more communication blocks to converge to a stable matching.
Under these threeconditions, the performances of the algorithm are similar.
Conclusion
We are the first to study the bandit algorithm for the many-to-one matching market under the uniquestable matching.
This work focuses on a decentralized market.
A new ˜ α -condition is proposedto guarantee a unique stable outcome in many-to-one market, which is more general than existinguniqueness conditions like SPC , Serial Dictatorship and could recover the usual α -condition inone-to-one setting.
We propose a phase-based algorithm of MO-UCB-D4 with arm-elimination,which obtains O (cid:16) NK log( T ) ∆(cid:17) stable regret under ˜ α -condition .
By carefully defining a mapping fromarms to the least preferred agent in its stable matched set, we could effectively correspond arms andagents by individual-to-individual.
A series of experiments under two environments of varying themarket size and varying arm capacity are conducted.
The results show that our algorithm performswell under Serial Dictatorship , SPC and ˜
α -condition respectively.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] Please see Abstract and Section 1.(b)
Did you describe the limitations of your work?
[Yes] Please see Section C.4.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
This
work mainly focuses on the online learning theory, which does not have any potentialnegative societal impacts.(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Please seeSection 2.
(b) Did you include complete proofs of all theoretical results?
[Yes] Please see Appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] Please seesupplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] Please see Section 6 and supplemental material.
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] Please see Section 6.
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]