A conditional match rate anomaly and an order change pressure in
residency matching programs∗
Munetomo Ando†
Nihon University

Minoru Kitahara‡
Osaka Metropolitan University
April 28, 2022

Abstract
In the medical residency matching markets in the U.S. and Japan, we observe that an applicant’s probability of being matched to the ﬁrst listed program is disproportionately higher than that to the second listed
conditional on that the applicant is rejected by the ﬁrst, while the conditional probabilities are relatively
stable after that. In addition, several experts have pointed out that participating programs sometimes exertpressure on applicants to inﬂuence the creation of their lists.
In this paper, we ﬁrst show that the latter can explain the former, considering a specialty of ﬁrst listing.
Then, we use the data to ﬁnd both substantial pressure and welfare loss, assuming a simple reduced-form
acceptance and forcing process. In addition, we consider a probabilistic permutation of the submitted list to
prevent pressure-induced list reordering and ﬁnd in the data that the beneﬁts of the intervention outweighthe associated efﬁciency losses.
JEL Classiﬁcation numbers: C78, D47
Keywords: Random permutation, ranking change pressure, residency matching.

1 Introduction
In many countries, when a medical student passes the Medical Licensing Examination and becomes a doctor,
he or she needs to attend a residency program. A centralized matching method is currently used to determine
the training program for medical students.1 In the U.S., the National Resident Marching Program (NRMP) isthe central clearinghouse, and in Japan, Japan Residency Matching Program (JRMP) plays the role.
Each applicant decides his or her preference for the residency programs for which he or she has completed
the necessary procedures and then submits the rank order list to the clearinghouse.2 Each residency program
also submits its preference list and the maximum number of residents it will accept. After that, the clearinghouse determines the matching outcome according to an algorithm and reports the results for the participants.
The clearinghouses in the U.S. and Japan use a modiﬁed version of the deferred acceptance algorithm(DAA) as the matching procedure. The original DAA provides a stable matching, and the students-proposing
DAA is strategy-proof for students in the sense that truth-telling is a weakly dominant strategy. In contrast,
residency programs generally do not satisfy strategy-proofness, but truthful reporting has been shown to beapproximately optimal in sufﬁciently large markets (Kojima and Pathak 2009).
∗ This work was supported by JSPS KAKENHI Grant Numbers JP17K03776, JP18K01513 and JP19K01542.
† E-mail: ando.munetomo@nihon-u.ac.jp.
‡ E-mail: kitahara@econ.osaka-cu.ac.jp.

1 Roth (2003) provides a brief overview of the history of residency matching.
2 The average number of programs on an applicant’s list is 9.83 in the U.S. and 3.25 in Japan. The former is a weighted
average of 11.22 (matched) and 4.21 (unmatched). Note that these ﬁgures are for 2019 (see https://www.nrmp.org/wpcontent/uploads/2021/08/Impact-of-Length-of-ROL-on-Match-Results-2021.pdf and https://www.jrmp2.jp/toukei/2019/2019toukei5.pdf).

1

Then, do applicants actually make the list following to their own preferences? Rees-Jones (2018), basedon the data from 579 respondents from 23 U.S. medical schools in 2012, reported that 83% of respondents
made their lists according to their preferences, but 16% submitted something different from their preferences
in a way that was not just a mistake.3
So what could be the reason for the difference between the applicant’s preference and the submitted list?
One possible explanation is that the residency program is exerting pressure on the applicant to change the list.
Of course, the NRMP has a rule against putting pressure on applicants. The match communication code of
conduct requires the director of the program to commit to the ﬁve clauses, including the following:• Respecting an applicant’s right to privacy and conﬁdentiality
Program directors and other interviewers may freely express their interest in a candidate, but they shallnot ask an applicant to disclose the names, specialties, geographic location, or other identifying information about programs to which the applicant has or may apply.
• Discouraging unnecessary post-interview communication
Program directors shall not solicit or require post-interview communication from applicants, nor shallprogram directors engage in post-interview communication that is disingenuous for the purpose of inﬂuencing applicants’ ranking preferences.4
However, some programs do not comply with this code of conduct, and applicants are pressured to change
their rank order lists. For example, Yarris et al. (2010) reported post-interview communication in emergency
medicine in the U.S. based on a questionnaire survey. They found that 89% of respondents were contacted
by the programs after the interview but before the list was submitted. This practice may violate the rulesthat prohibit unnecessary communication after the interview.5 Moreover, Grimm et al. (2016) reported thatsome residents changed their rankings on the submission list because they felt pressured in post-interview
communication.
In Japan, due to the nature of the matching algorithm, residency programs should not ask applicants about
their order of preference, although the JRMP does not have such an explicit prohibition. In practice, however,a survey of students who participated in the 2018 residency match included the question “Have you ever beenasked about your preference ranking during an interview?” To this, 62% of respondents answered “yes.”6
So how exactly do residency programs put pressure on applicants? These are guesses, but for example, aprogram might ask an applicant, “We want you to put our program ﬁrst on your list,” or ask, “Is our program
your ﬁrst choice?”
In fact, if the program were to put pressure on applicants, asking them to list the program in the ﬁrst
place would be efﬁcient. If the applicant actually lists the program as the ﬁrst place, and the program lists theapplicant at the top of the list, then the match between the two is guaranteed.7 Another advantage is that thepromise to write the program ﬁrst on the list is easy to keep. This is because if an applicant promises to write
the program in the ﬁrst place, but the matching does not succeed, it will surely be discovered that the applicant
has broken the promise.8
We want to quantify how much of the change in the ranking of the list due to pressure exists. To do this,
we ﬁrst identify whether applicants have a higher probability of matching with the program listed ﬁrst on thelist than with any other program.
Figures 1 and 2 show a conditional match rate anomaly. Figures 1 and 2 tell us that an applicant’s match
probability with the program listed ﬁrst on the list is consistently high. In the U.S., an applicant has approximately a 50% probability of being matched with the ﬁrst program on the list. In Japan, the probability is
3 See Rees-Jones and Skowronek (2018) for an experiment in preference misrepresentation.
4 See https://www.nrmp.org/communication-code-of-conduct/
5 See Sbicca et al. (2012) for a detailed explanation of the code violations.
6 See https://jrmp2.s3-ap-northeast-1.amazonaws.com/question/2018gakusei.pdf
7 Grimm et al. (2016) reported that “[i]n our survey, only 5.2% (14 of 268) of program directors reported that they always or usually
move applicants up their rank order lists after the applicant promises to rank their program No. 1.”
8 Interestingly, in Grimm et al. (2016)’s survey, “52.6% (141 of 268) of program directors reported that at least once a year 1 ormore applicants falsely claim they are ranking their program No. 1.”

2

approximately 75%.9 In contrast, the conditional probability that an applicant will be matched with the second
through fourth programs on the list is relatively low and stable. These probabilities are conditional on being rejected by all programs listed higher.10 This data is consistent with the story that some programs are pressuring
applicants to change their lists.

Figure 1: The U.S.

Figure 2: Japan

If the programs pressure applicants and the applicants actually change their lists, does this result in a lossof efﬁciency? The match may change if an applicant submits a list that differs from the true preference orderdue to pressure. For example, if an applicant had submitted a list based on the true preference order, he or she
would have been matched with the second program, but the actual match would have been the fourth program.
We refer to this as the number of the rank losses caused by the forced list change to measure the welfare loss,
which is two.
Of course, a high probability of a forced list change does not necessarily imply a large welfare loss. In
particular, if all acceptant programs put pressure to list themselves ﬁrst, then, as long as the applicant chooses
the best one among them as the ﬁrst, the match outcome does not change (Remark 1). Therefore, it is necessary
to empirically show what percentage of applicants are actually under pressure and how much welfare loss
occurs. Then, the U.S. and Japanese data imply substantial forces and welfare loss by assuming a simple
reduced-form acceptance and forcing process (Section 3).11
How can such losses be reduced? In the ﬁrst place, an applicant changes his or her list according to pressure
9 Figures 1 and 2 show the probabilities in the past 10 years through 2019. In these ﬁgures, we have not used data beyond 2020

because COVID-19 may have changed the way programs interview applicants in the residency matching. Since the current matchingsystem in Japan started in 2003, Figure 2 uses data from 2003 (rather than 2010).
10 Both in Japan and the U.S., how many applicants are matched to their which rank programs are available. Moreover, in Japan,
how many applicants are unmatched with which length of their lists are also available. Then, we can calculate the acceptance rate of aprogram with a list rank conditional on that the applicant has been rejected by all higher rank programs as
the kth rate =#matched to the kth#applicants − ∑k′ <k (#unmatch with the list length of k′ + #matched to the k′ th)for Japan, and approximately for the U.S. as
the kth rate ≈#matched to the kth,#applicants − ∑k′ <k #matched to the k′ th

as long as the number of unmatch applicants with their list lengths shorter than k is so small. Note that for the match probability
with the second and subsequent programs on the list, it is necessary to take into account that some applicants may not have enoughprograms on the list to calculate the probability correctly. Therefore, when we derived the probabilities for Japan, we properly dealt
with this issue in our calculations. However, we did not take into account it when calculating the probabilities for the U.S. due to data
limitations.
11 For simplicity, we assume the following reduced-form process. The applicants are ex-ante homogeneous. The programs simultaneously decide whether the applicants are acceptable with an exogenous acceptance probability, a. We also assume that each acceptant
program exerts pressure on the applicant with an exogenous probability, e. The resident who receives effective pressure from at least
one program will choose the best program and place it ﬁrst on the list.

3because if he or she does not keep his or her promise and is not matched with the program, he or she will beidentiﬁed as a betrayer. And suppose it becomes clear that he or she has broken a promise. In that case, theapplicant may be disadvantaged as an untrustworthy person in his or her subsequent activities in the medical
community.
A possible solution is an introducing a random permutation of the applicants’ submitted lists before runningthe matching algorithm. Speciﬁcally, we consider a random exchange of the ﬁrst and the second with a certain
probability, ε (Section 2.1.5). Under such interventions, even if the pressured program could not match theapplicant, it would not be possible to judge with 100% certainty whether the applicant had betrayed the program
or not. Moreover, with a sufﬁciently large exchange probability, the certainty of the applicant’s betrayal falls
below a required level (Remark 2). Consequently, the applicant can submit the list based on the true preferences
order on the programs, and the welfare loss caused by the pressure disappears.
On the other hand, the random exchange itself causes an associated welfare loss, which makes determining
whether this intervention is proﬁtable or not an empirical issue. Again with our simpliﬁed model, we show thatthe beneﬁts of this proposed intervention outweigh the associated losses with data from the U.S. and Japan. In
fact, in Japan, we show that expected rank loss can be reduced to about 1/4 when the signiﬁcance level is set
to 15% (again in Section 3).
The rest of the paper is organized as follows: Section 2 describes the model. Section 3 describes the resultsof empirical evaluations. Section 4 concludes.

2

Model

We consider a representative doctor to be matched with at most one residency program in I. Here I = {1, . . . , L}with L ≥ 2 is the set of programs that the doctor has applied to and found to be acceptable, where program k is
the kth most preferred one for the doctor. Thus, we implicitly assume that there is no tie in ranking.

2.1

Process

2.1.1 Matching
Given the doctor’s list of programs l = (l1 , . . . , lL ) ∈ {(l1′ , . . . , lL′ ) ∈ I L |lk ̸= lk′ for k′ ̸= k}, where program lk is
the program listed kth in the list, and the set of programs A ⊂ I which would accept the doctor, the doctor ismatched with program µ (ll , A) = lk if (i) lk ∈ A and (ii) lk′ ∈/ A for all k′ < k, and is unmatched if A = 0.
/Note that the elements of set A generally depend on other doctors’ lists. However, we ignore the possibility
in our model. In this sense, our model is just a simpliﬁed reduced-form one.
We also note that the doctor’s list l is not necessarily equal to his or her preferences, I = {1, . . . , L}. Thisis because, as we will see below, the doctor may change the submitting list if he or she feels pressured by the
programs. Moreover, as we will study later, the list l may further be different from the submitted list, becauseinterventions implemented by the clearinghouse may result in a random permutation of the list.12
2.1.2

Order change pressure

Given the set of programs which the doctor is under (effective) pressure to list ﬁrst, F ⊂ {1, . . . , L}, we let thedoctor’s submitted list under the pressure structure F, λ (F) = (λ1 (F), . . . , λL (F)), be such that (i) λ1 (F) = i if
i ∈ F and i < i′ for all i′ ∈ F \ {i}, and (ii) λ (F) = (1, . . . , L) if F = 0.
/ That is, the doctor chooses to list ﬁrst
the most preferred program among pressuring ones, and submits just the true preference if there is no pressure.
Note that our assumption on the doctor’s choice among pressuring programs is most optimistic about thewelfare loss caused by the list change, since in general a less preferable pressuring program may be chosen
to be listed ﬁrst. In this sense, our welfare loss estimate derived later is just the lower bound, so that it may
underestimate the value of removing such pressures.
12 In summary, there are three kinds of orders for a particular applicant: the order by true preference (Section 2.1.1), the order inthe list submitted to the clearinghouse (Section 2.1.2), and the order in the list used by the matching algorithm after probabilistic
permutation (Section 2.1.5).

4

2.1.3 Pressure and acceptanceLet e ∈ (0, 1) be an independent and an identical probability that each program that is going to accept thedoctor will exert effective pressure on that doctor. Also, the probability that a program that does not accept thedoctor will exert pressure is zero. In summary, we denote F̃ e (A) for the pressure structure, that is i ∈ F̃ e (A)with probability e ∈ (0, 1) if i ∈ A, and 0 if i ∈/ A, independently.
Note that, by letting the probability be zero if the program i is not the element of the set A, we ignore thepossibility that a program uncertain of the acceptance may succeed in put an effective pressure on the doctor.13
By the identicalness and independence, we also ignore that the decisions of the doctor and a program regarding
the pressure in general depend on how the doctor evaluates the program and whether other programs also put
pressure.
2.1.4 Acceptance
We assume that the probability that a residency program will accept the doctor is a ∈ (0, 1), and the acceptancedecisions of individual programs are independent. We denote as Ãa the set of programs that accept the doctor,
that is, i ∈ Ãa with probability a ∈ (0, 1), independently.
By the identicalness, we ignore the possibility of correlation between the preference of a doctor and thepriority for each program, such as that caused by the ﬁtness of the doctor for each program.14 By the independence, we also ignore the possibility of some doctor generally being prioritized over another doctor, such asthat caused by the former having higher ability than the latter.
Then, by letting λ̃ a,e = (λ̃1a,e , · · · , λ̃La,e ) ≡ λ (F̃ e (Ãa )) for notational simplicity, the event that a program i is
the kth in the submitted list is represented by i = λ̃ka,e .
2.1.5 Random exchange
We will also consider adding a random permutation of the doctor’s submitted list l = (l1 , l2 , l3 , . . . , lL ) before
running the matching algorithm. In particular, for ε ∈ [0, 1/2], let π̃ ε (ll ) = (l2 , l1 , l3 , . . . , lL ) with probability εand l with probability 1 − ε .15 That is, we will consider a random exchange of the ﬁrst and the second, with theprobability of ε > 0,16 and then, given a submitted list l and an acceptance structure A, the resulting matching
becomes µ̃ ε (ll , A) ≡ µ (π̃ ε (ll ), A). Note that if ε = 0, then they are the original list and procedure, π̃ 0 (ll ) = l andhence µ̃ 0 (ll , A) = µ (ll , A).

2.2 Key statistics and remarks
2.2.1 Conditional match rates
Let Pka,e denote the kth conditional match rate, the probability to be matched with the kth program
conditionalon that the doctor has already been rejected by all programs listed higher, that is, P1a,e ≡ Pr λ̃1a,e ∈ Ãa , and

13 Taking such a possibility into account may be important, given that it is pointed out (see, e.g., Doolittle (2017)) that an applicant
may misunderstand a message from a program which itself has no intention to put such a pressure. We can introduce and quantify thispossibility by instead letting i ∈ F̃ e (A) with probability e ∈ (0, e] if i ∈/ A. Then, the possibility of misunderstanding is captured by the
decline in the conditional match rates from the second to the third, since it becomes possible that even after the rejection by the ﬁrst,
there remains a program which both puts pressure on and would accept the doctor, which cannot be if e = 0, as discussed below.
14 In fact, by assuming an extreme correlation distribution, we can reproduce the conditional match rate pattern without resortingto the pressure process, which may actually be plausible to explain just one observation. However, the pattern is found both in theJapanese and the U.S. markets and also across time, with so different values that considering that the parameters of the extreme
distribution happens to be adjusted appropriately for each case may be unreasonable.
15 This paper examines a simple method of changing the doctor’s submitted list. Naturally, there can be other effective ways tochange the list. It is also important to consider the measures that the programs can take against such intervention methods. For
example, a program could not only ask a doctor to write it ﬁrst on the list, but also request that the second and lower places on the listbe left blank.
16 For the cases where the exchange probability e is greater than 1/2, the doctor can deal with it by replacing the ﬁrst and the second
positions on his or her list in advance. Therefore, it is sufﬁcient to consider cases where the probability is less than 1/2.

5

for all k ∈ {2, . . . , L},a′Pka,e ≡ Pr λ̃ka,e ∈ Ãa λ̃ka,e
∈
/
Ã
for
all
k
<
k
.
′

Then, we show that the conditional match rate ﬁrst drops and then becomes constant, as observed in theJapanese and the U.S. markets.
Proposition. P1a,e > P2a,e = · · · = PLa,e .
1−a1−aProof. We show that P1a,e = 1 − (1 − ae)L 1−ae
, which is larger than 1 − (1 − a × 0) 1−a×1= a, and for all

a,e
a−ae
a−a×0
a,e
k ∈ {2, . . . , L}, Pk = 1−ae , which is smaller than 1−a×0 = a. For notational simplicity, let F̃ ≡ F̃ e Ãa .

e (A)|i ∈For the former, note that λ̃1a,e ∈ Ãa if F̃ a,e ̸= 0,/ since by
Pr
i
∈
F̃
/
A
= 0, i ∈ F̃ e (A) implies i ∈ A,

and that λ̃ a,e = (1, . . . , L) if F̃ a,e = 0.
/ Thus, P1a,e = 1 − Pr λ̃1a,e ∈/ Ãa is equal to

1 − Pr(F̃ a,e = 0)
/ Pr(1 ∈/ Ãa |F̃ a,e = 0)
/ = 1 − Pr(F̃ a,e = 0)
/ Pr(1 ∈/ Ãa |1 ∈/ F̃ a,e ),where we use the conditional independence for the last equality.

Pr(λ̃ka,e/ Ãa for all k′ <k but λ̃ka,e ∈Ãa )′ ∈For the latter, by Pr λ̃1a,e ∈/ Ãa F̃ a,e ̸= 0/ = 0, Pka,e =is equal toPr(λ̃ka,e/ Ãa for all k′ <k)′ ∈Pr λ̃ka,e/ Ãa for all k′ < k but λ̃ka,e ∈ Ãa F̃ a,e = 0/
′ ∈


,
a for all k′ < k F̃ a,e = 0Pr λ̃ka,e
∈
/
Ã
/
′
/
which is equal to, since λ̃ a,e = (1, . . . , L) if F̃ a,e = 0,Pr k′ ∈/ Ãa for all k′ < k but k ∈ Ãa F̃ a,e = 0/Pr k′ ∈/ Ãa for all k′ < k F̃ a,e = 0/
/ Ãa for all k′ < k but k ∈ Ãa
Pr k′ ∈/ Ãa for all k′ < k but k ∈ Ãa Pr F̃ a,e = 0/ k′ ∈=Pr k′ ∈/ Ãa for all k′ < k Pr F̃ a,e = 0/ k′ ∈/ Ãa for all k′ < kPr k ∈ Ãa Pr k ∈/ F̃ a,e k ∈ Ãa,=
Pr k ∈/ F̃ a,e2

where we use the conditional independence for the last equality.

That is, since programs which put the pressures are also those which would accept the doctor, and thedoctor lists ﬁrst a program among who puts the pressure if there is at least one such a program, the factthat the program listed ﬁrst rejects the doctor implies that all remaining programs are not those which both
put the effective pressures and would accept the doctor, and that the submitted list is unchanged from theoriginal one,17 which uniformly reduce no less than the second conditional match rates, from Pr(i ∈ A) to
Pr(i ∈ A|i ∈/ F̃ e (A)). Moreover, the cases where the most preferred program would not accept the doctor but
there is one which puts the pressure are added to the ﬁrst match cases, increasing the probability from Pr(i ∈ A)
to 1 − Pr F̃ a,e = 0/ Pr 1 ∈/ Ãa |1 ∈/ F̃ a,e .
Note that, given L, we can recover a and e from the values of P1a,e and P2a,e . In particular, since 1 − P2a,e =1/L 

1−P1a,e1−Pa,e 1/La−ae
1−a
1 − 1−ae = 1−ae , 1 − ae == 1−P1a,e, and hence
1−a
1−ae

2


a = 1 − (1 − ae) (1 − P2a,e ) = 1 −1 − P1a,e
1 − P2a,e1/L(1 − P2a,e ) ,(1)

17 The critical assumption is that programs who would not accept the doctor cannot put the effective pressure. See also footnote 13.

6

by which a,e 1/L
1−P1
1−
1 − (1 − ae)
1−P2a,ee==.
 a,e 1/L
a1−P
1 − 1−P1a,e(1 − P2a,e )(2)

2

We will use them to empirically evaluate the Japanese and the U.S. markets in Section 3.
2.2.2 Rank loss
We discuss the rank loss caused by the pressure. The rank loss refers to how the actual matching deviates fromthat under the true preferences.
Let RLa,e denote the average rank loss caused by the forced list change, that is,h iRLa,e ≡ E µ λ̃ a,e , Ãa − µ (1, . . . , L), Ãa ,where we let µ λ̃ a,e , Ãa − µ (1, . . . , L), Ãa = 0 if Ãa = 0.
/ 18
We use this as a measure of the welfare loss. Note that then, many points regarding welfare concern are
ignored. For example, the importance of one rank gain may be different depending on rank levels, which may
be incorporated by introducing some non-linear function to evaluate each rank as u(i). Moreover, the existenceof pressure itself may be stressful (see, e.g., Doolittle (2017)), even if the resulting match is unchanged, forwhich some evaluation of F̃ a,e itself, such as −c|F̃ a,e | with c > 0 being the constant stress cost, is necessary.
It is remarkable that while the size of the ﬁrst drop increases as the pressure probability (e) increases, theaverage rank loss, which ﬁrst rises from zero, vanishes as it becomes almost sure.
Remark 1. While P1a,e − P2a,e is increasing in e, RLa,e > lime↓0 RLa,e = lime↑1 RLa,e = 0.
1−aa−aeProof. The former part simply follows from that P1a,e = 1 − (1 − ae)L 1−ae
is increasing, and P2a,e = 1−aeisdecreasing in e.
For the latter, note that RLa,e is equal to

 
 


′
′
′
′
′
′
=
i
Pr
min
i
>
i
=
i
E
min
i
−
i
min
i
>
min
i
=
i
,
Pr
min
i
min
i
∑

L−1
i=1

i′ ∈Ãai′ ∈F̃ a,ei′ ∈F̃ a,ei′ ∈Ãai′ ∈F̃ a,ei′ ∈Ãathe second term of which is equal to, by the conditional independence,!

Pr i ∈/ F̃ a,e i ∈ Ãa/ F̃ a,e
1 − ∏ Pr i′ ∈= (1 − e)(1 − (1 − ae)L−i ),i′ >iwhich converges, by i < L, to 0 as e approaches either 0 or 1.

2

That is, given that the pressure probability conditional on the acceptance is almost 1, no pressure from a
program implies almost surely that the program will not accept the doctor anyway, and hence lowering its rank
in the list does not matter. Moreover, as long as the doctor chooses the best among the pressured programs as
the ﬁrst listed, the match outcome does not change.
Thus, a large drop in the conditional match rate as observed in the Japanese and the U.S. markets could
be compatible with a negligible rank loss. Therefore, it is still an empirical issue whether the pressure so
substantial as to cause such a large observational gap also causes a non-negligible welfare loss, which we will
evaluate in the empirical part.

18 Note that µ
λ (F̃ e (A), A = 0/ if and only if µ ((1, . . . , L), A) = 0,
/ which is equivalent to A = 0.
/7

2.2.3 Type I errors
In the following, we examine the possibility that a program that pressures a doctor to write itself ﬁrst on the
doctor’s submitted list may be able to determine whether the doctor actually did write itself ﬁrst.
εLet Qa,e,
i→k denote the probability of program i not to be matched with the doctor conditional on that the
program would accept the doctor and the doctor follows the pressure to list the program kth, for which wea,efurther specify that the orders among others would remain the same in the changed list. That is, let λ̃i→k
be

a,e
a,e
a,e
a,e
a,e
a,e
′
′′
′′′
such that λ̃i→k,k = i and for all k , k ∈ k ∈ {1, . . . , L} λk′′′ ̸= i , λ̃i→k,k′ > λ̃i→k,k′′ if and only if λ̃k′ > λ̃k′′ .
Then,


 
ε
a,e
ε
a
a,e
a
Qa,e,
µ̃
λ̃
,
Ã
=
̸
i
i
∈
F̃
∩
Ã
.
=
Pr
i→k
i→k
Note that Qa,e,0
i→k is the probability under the original procedure, with no addition of permutation.
We consider that with a signiﬁcance level α > 0, conditional on that the program is not matched with thedoctor, the hypothesis that the doctor follows the pressure to list kth a program that would accept the doctorε
is rejected if Qa,e,i→k ≤ α , and this rejection possibility makes the doctor follow the pressure. Then, we see thespecialty of ﬁrst listing under the original procedure (ε = 0), that the hypothesis rejection is possible with any
small signiﬁcance level, or the zero type I error, while the necessary signiﬁcance level is bounded away fromzero for all other ranks.
a,e,0
a,e
Remark 2. Qa,e,0≡ 1 − (1 − a)(1 − ae)L−2 .
i→1 = 0 ≤ α for any α . Moreover, for all k ≥ 2, Qi→k ≥ Qa,ea,eProof. By µ̃ 0 = µ , λ̃i→1,1= i and i ∈ Ãa imply µ̃ 0 (λ̃i→1, Ãa ) = i, which proves the former. Moreover, for thelatter, for k ≥ 2, Qa,e,0
i→k is equal to
1 − Pr Ãa ∩ {1, . . . , k − 1i≥k } \ {i} = 0/ Pr F̃ a,e ∩ {k + 1i<k , . . . , L} \ {i} = 0/ Ãa ∩ {1, . . . , k − 1i≥k } \ {i} = 0/
= 1 − (1 − a)k−1 (1 − ae)L−k ,(3)where we use the conditional independence for the last equality.19

2

That is, since the program would accept the doctor, as long as the doctor truly listed the program ﬁrst, theprogram would be certainly matched with the doctor. On the other hand, for other ranks, even if the doctor
follows the listing, the unmatch may occur because the doctor may match another program listed higher, theprobability of which is enhanced by the existence of ﬁrst listing pressure which makes programs listed higher
more probable to accept the doctor. Of course, it may be negligible if the lower bound itself is positive but sosmall as standard signiﬁcance levels, such as 10%, which will be checked after the estimation of a and e.
Then, we consider introducing a random exchange of the ﬁrst and second in the submitted list with aprobability small but sufﬁcient to negate the testing. Given the following monotonicities, for the complete
negation (in the sense of any k and e), it is necessary and sufﬁcient letting the exchange probability ε so largethat the type I error for the ﬁrst listing given that other programs do not exercise any pressuring is larger than
a,e,ε
a,0,ε
the signiﬁcance level α , Qi→1
≡ lime→0 Qi→1
> α.
a,0,ε
a,e,ε
ε
≥ Qi→1
Remark 3. Qi→kirrespective of e and k. Moreover, Qa,0,i→1 is increasing in ε .

Proof. For the former, since (π̃1ε (ll ), π̃2ε (ll )) = (l1 , l2 ) with probability ε and (l2 , l1 ) with probability 1 − ε ,
a,e,0
a,e,ε
a,e,0
a,e,0
a,e,ε
ε
ε
+ (1 − ε )Qa,e,0
Qi→1= ε Qi→2
i→1 and Qi→2 = ε Qi→1 + (1 − ε )Qi→2 . Moreover, since {π̃1 (ll ), . . . , π̃k−1 (ll )} =
a,e,0
a,e,ε
a,e,0
a,e,0
a,e,0
{l1 , . . . , lk−1 } for k > 2, Qi→k = Qi→k for k > 2. Then, since by (3), Qi→k ≥ Qi→2 ≥ Qi→1 for k > 2, by
ε
a,e,ε
a,0,ε
a,e,ε
ε ≤ 1/2, Qa,e,i→k ≥ Qi→1 for k ≥ 2. Moreover, again by (3), Qi→1 ≥ Qi→1 .
εFor the latter, by (3), Qa,0,2i→1 = 1 − ε (1 − a) − (1 − ε ) = aε .

19 Here 1

x is the indicator function that is eqaul to 1 if x holds; 0 otherwise.

8

At the same time, since the exchange itself causes a welfare loss, even if the negation is achieved, theassociated loss by the permutation intervention,
PRLa,ε ≡ E µ̃ ε (1, . . . , L), Ãa − µ (1, . . . , L), Ãa ,may also be substantial.
Remark 4. PRLa,ε is also increasing in ε .
Proof. Note that PRLa,ε = (2 − 1) Pr {1, 2} ⊂ Ãa Pr (π̃1ε ((1, . . . , L)) = 2) = a2 ε .

2Moreover, while the loss to yield a given type I error is smaller with a smaller acceptance rate, a smaller
rate also more restricts the possibility of the negation.
εa,ε with ε solving
Remark 5. Qa,0,i→1 = α for some ε if and only if α ≤ a/2, which is increasing in a, while PRLa,0,εQi→1 = α is also increasing in a.
εa,ε = a2 ε .
Proof. For the former, recall Qa,0,i→1 = aε , which also implies the latter by PRL

2Thus, whether in the ﬁrst place, a sufﬁciently large type error can be achieved, and if so, with so allowableassociated loss also remains the empirical issue.

3

Empirical evaluation

For each market, we set (P1 , P2 , L),20 derive (a, e) from (1) and (2) with them, and calculate key statistics asathe following table, where RLrand
denotes the rank loss if the matched program is randomly chosen from thosewhich would accept the doctor, L + 1

 a

a
a
a
RLrand = 1 − Pr Ã = 0/
− E µ (1, . . . , L), Ã Ã ̸= 0/ ,2
and PRLαa denotes the rank loss in order to attain the type I error α , i.e., with aε = α ,PRLαa = PRLa,α /a .
Market
Japan
U.S.

P1
0.7
0.5

P2
0.5
0.3

L
4
10

a
0.55990.3232e
0.21410.1024RL
0.34250.7740RLrand0.83732.5588Q
0.6591
0.4829PRL0.150.08400.0485QL
0.06850.0704Thus, the rank loss reaches into more than 30% (0.7740/0.2558 ≈ 0.302) of what the efﬁcient assignment
adds to the random assignment. Moreover, we can achieve the type I error of 15% (0.3232/2 ≈ 0.162), withthe associated loss less than a quarter (0.0840/0.3425 ≈ 0.245) of the improvement by the negation.
For the rank loss evaluation, consider that the match value of each program for the doctor is independently
drawn from an identical continuous distribution, and the preference order is determined according to thesevalues. Then, as is easily calculated, one rank difference is translated into 1/(L + 1) quantile difference, thatis, the expected quantile loss is
RLa,e
QLa,e =,
L+1
which for each market is also reported in the table, where we ﬁnd that once evaluated in quantile, the estimatedlosses are very similar. Thus, on average, around 7 percent point quantile is lost in both markets.
We also plot how the estimated quantile loss would be if P2 took different values, as follows. Somewhat
interestingly, the actual P2 attains almost highest value in both markets.
20 There is a much larger room for the choice of L since it is very heterogeneous in the data, while we also ﬁnd that the results are

fairly robust to its reasonable variation.

9

QL (vertical) by P2 (horizontal) for Japan (left) and U.S. (right)
Finally, it may also be interesting that the estimated pressure probability is much smaller (0.1024 ≪ 0.2141)
in the U.S. market, where the matching authority explicitly prohibits such conduct.

4 Concluding remarks
In this paper, we studied a conditional match rate anomaly and the reason why this occurs. First, we showedthat such an anomaly could be explained by the possibility that a program planning to accept an applicant
forces the applicant to list it ﬁrst. Then, under a simple reduced-form acceptance and forcing process, wedemonstrate that the U.S. and Japanese data imply substantial forces and welfare loss.
Finally, we have suggested one idea that could alleviate this problem, that is introducing a random permutation of the applicants’ submitted list before running the matching algorithm and showed that the introduction
pays if the intervention makes the pressure ineffective.
This paper showed that the probability of a program exerting pressure on an applicant is approximately
twice as high in Japan (0.2141) as in the U.S. (0.1024). This is an interesting difference, which may be due tothe weak (or non-existent) direct regulation by the communication code of conduct in Japan.
The method of estimating the pressure probabilities described in this paper may be used for the matchingmarkets other than the residency matching. In recent years, the matching algorithms have been utilized inmany environments. For example, Google uses algorithms in determining worker assignments.21 When thematching algorithm is used within a company in this way, the players may try to inﬂuence the matching resultsby putting pressure on each other. We need to do further research on the inefﬁciencies created by pressure andhow to solve them.

References[1] Doolittle, B.D. (2017) “Communication after the residency interview: A call for professionalism,” NEJM
Knowledge+, (https://knowledgeplus.nejm.org/blog/communication-after-residency-interview/).
[2] Grimm, L. J., Avery, C. S., and Maxﬁeld, C. M. (2016) “Residency postinterview communications: Moreharm than good?” Journal of Graduate Medical Education, 8(1): 7–9.
[3] Kojima, F., and Pathak, P. A. (2009) “Incentives and stability in large two-sided matching markets.”
American Economic Review, 99 (3): 608–27.
[4] Rees-Jones, A. (2018) “Suboptimal behavior in strategy-proof mechanisms: Evidence from the residency
match,” Games and Economic Behavior, 108, 317–330.
[5] Rees-Jones, A. and Skowronek, A. (2018) “An experimental investigation of preference misrepresentationin the residency match,” Proceedings of the National Academy of Sciences, 115 (45): 11471–11476.
21 See for detail “Google’s algorithm-powered internal job marketplace” (https://rework.withgoogle.com/blog/googles-algorithm-

powered-internal-job-marketplace/).

10[6] Roth A.E. (2003) “The origins, history, and design of the resident match,” JAMA. 289(7): 909-–912.
doi:10.1001/jama.289.7.909 (https://jamanetwork.com/journals/jama/fullarticle/195998).
[7] Sbicca, J. A., Gordon, K., and Takahashi, S. (2012) “National resident matching program violations,”
AMA Journal of Ethics, 14(12): 932–936.
[8] Yarris, L.M., Delorio, N.M., and Gaines, S.S. (2010) “Emergency medicine residency applicants’ perceptions about being contacted after interview day,” Western Journal of Emergency Medicine, 11(5):474–478.

11