Recruitment Strategies That Take a Chance
Anonymous Author(s)
AfﬁliationAddress email
Abstract
In academic recruitment settings, including faculty hiring and PhD admissions,committees aim to maximize the overall quality of recruited candidates, but there isuncertainty about whether a candidate would accept an offer if given one.
Previouswork has considered algorithms that make offers sequentially and are subject to ahard budget constraint.
We argue that these modeling choices may be inconsistentwith the practice of academic recruitment.
Instead, we restrict ourselves to a singlebatch of offers, and we treat the target number of positions as a soft constraint, sowe risk overshooting or undershooting the target.
Speciﬁcally, our objective is toselect a subset of candidates that maximizes the overall expected value associatedwith candidates who accept, minus an expected penalty for deviating from the target.
We ﬁrst analyze the guarantees provided by natural greedy heuristics, showingtheir desirable properties despite the simplicity.
Depending on the structure ofthe penalty function, we further develop algorithms that provide fully polynomialtime approximation schemes and constant-factor approximations to this objective.
Empirical evaluation of our algorithms corroborates these theoretical results.
Introduction
Anyone who has served on a faculty hiring committee or a PhD admissions committee knows that asuccessful outcome requires resolving the tension between two competing goals.
On the one hand,some candidates are (perceived to be) better qualiﬁed than others, and the aim is to recruit the bestcandidates.
On the other hand, there is a given number of positions to be ﬁlled, and while there istypically some ﬂexibility, there is a real cost to recruiting too many or too few people.
The tensionarises because the stronger a candidate is, the more likely they are to receive multiple attractive offersand the less likely they are to accept any particular offer.
In order to manage uncertainty, a goodstrategy may involve a mix of offers to stellar candidates and “safer” candidates.
To formalize this problem we assume that we have access to two numbers for each candidate i :their value x i and their probability p i of accepting an offer.
We acknowledge that in current practice,these numbers are not explicitly estimated.
However, committees typically rank or assign numericalscores to candidates based on their strength or ﬁt, and savvy committees roughly estimate recruitmentchances by classifying candidates as, say, “high yield,” “low yield” or “extremely low yield”, forexample, by past experience or assistive computational tools [12, 1].
Therefore, we believe that thegap between current practice and explicit value and probability estimates is not large.
Our approach builds on the work of Purohit et al.
[11], who framed hiring under uncertainty as anoptimization problem.
In their basic model, there are n candidates (each associated with a valueand probability), k positions, and t time steps.
In each time step the algorithm may make an offerto a single candidate and receive a response, that is, at most t sequential offers can be made, andthe budget of k cannot be exceeded.
The goal is to maximize the expected value of candidates whoaccept offers.
Purohit et al.
also consider the setting where the algorithm may make parallel offers in each round.
For both problems, they develop polynomial-time, constant-factor approximationalgorithms (with approximations ratios of 2 and 8, respectively).
This problem formulation captures key aspects of academic recruitment, but, in our view, it does havetwo shortcomings.
First, in a sense it is overcomplicated, as computational hardness stems from theassumption that offers are made sequentially:
Even with parallel offers, if there is a single time step( t = 1) then it is optimal to make offers to the k candidates with largest p i · x i .
But, in our experienceof faculty hiring and PhD admissions in several universities, offers are typically made in one batch.
Indeed, delayed offers (in the case of faculty hiring) and waitlists (in the case of PhD admissions)are usually avoided as they negatively impact yield.
Waitlists also increase the mental stress of thestudents that are placed on these waitlists.
The second, and more crucial, shortcoming is that Purohit et al.
[11] consider the constraint of hiringk candidates as hard.
Again, this is inconsistent with our experience: Offers are made so that theexpected yield roughly matches a desired target, but some faculty hiring or PhD admission cycles are“too successful,” in the sense that the number of candidates who accept their offers is much largerthan expected.
This has a real cost: in the case of PhD students, it creates difﬁculties in ﬁndingfunding and advisors, and in the case of faculty hiring, it may precipitate a shortage of resources withlong-term impacts on future hiring and even tenure.
For example, in one of our institutions, a facultyhiring cycle with yield that was much higher than expected led to the cancellation of the subsequentyear’s search.
Let us, therefore, reformulate the problem of hiring under uncertainty in a way that avoids both issues,that is, offers are made in a single batch and the target number of positions k is treated as a softconstraint.
Speciﬁcally, a penalty is incurred for deviating from the target number of positions; weconsider several different options for the penalty function.
The optimization problem, therefore, isthis:
Select a subset S of candidates that maximizes overall expected
reward (cid:80) i ∈
S p i · x iminus expected penalty for deviating from the target number of positions.
Enumerating all possible subsets S may be practicable in the case of faculty hiring in small departments.
However, a brute-force approach will not work for this purpose in larger departments, or at thescale of PhD admissions even in smaller programs, which motivates our search for good algorithms.
Our results.
We ﬁrst consider a simpliﬁed case where the goal is to solely minimize the penalty termof our objective (irrespective of the rewards), and show that the greedy algorithm selecting candidatesin the decreasing order of probabilities is optimal (Section 3.1).
The full objective is considerably more complex, and we analyze it under two natural penalty functions.
When the penalty function is the mean squared error from the target, we show that the optimizationproblem is weakly NP-hard and provide a fully polynomial-time approximation scheme (FPTAS).
When the penalty is linear in the extent to which the target is exceeded (that is, a linear penalty isincurred for overshooting, not undershooting), we show that two greedy heuristics — picking in thedecreasing order of the value x i and the expected value p i x i — provide approximations to the optimalsolution that are polynomial in the minimum probability p min .
We then present a constant-factorapproximation algorithm that runs in polynomial time for a ﬁxed p min , thereby improving upon thegreedy heuristics.
Finally, we carry out experiments on synthetic data (Section 4), focusing on L +1 loss.
We observethat the two greedy heuristics perform reasonably well, especially if the values and probabilitiesare positively correlated.
At the same time, compared to the greedy heuristics, the constant-factorapproximation algorithm adapts to the speciﬁc instances better, especially when the correlation isnegative.
These numerical experiments corroborate our theoretical results that the greedy heuristicsprovide reasonable guarantees, justifying their use in practice for both simplicity and good performance.
However, in some regimes the additional ﬂexibility of the constant-factor approximationalgorithm may be worth the complexity overhead.
Related work.
In practice, the challenge of uncertainty in admissions is mitigated by practices suchas admitting students in multiple rounds, using a waitlist, and using a rolling process [9].
Thereare also assistive computational tools that predict student yield rate with machine learning [12, 1].
There are a few theoretical formulations that model and address the uncertainty in such problems.
Purohit et al.
[11] considers an online setting where the acceptance of each candidate is revealedimmediately if the candidate is given an offer at that timestep, and analyzes the optimal ordering togive the candidates offers to subject to a hard constraint on the total number of acceptances.
Gangulyet al.
[7] considers a setting with multiple rounds where the yield rate in each round is either q L or q H(with q L < q H ).
To model the negative correlation between the student quality and the probability ofacceptance, the authors assume the probability of the yield rate being q H is linear in the number ofstudents admitted.
They subsequently derive a decision tree that computes the number of admissionsto make in each round.
For one batch selection, Zhang and Pippins [13] analyze the optimal number ofapplicants to admit using tools from yield management, assuming that each applicant has the identicalvalue and probability.
A series of work models the admissions problem as a decentralized matchingmarket [4, 5], where the uncertainty in acceptance is modeled by the students’ stochastic preferencesover multiple schools.
An objective combining the utility and the accepted size is considered, but thepenalty is on the expected size, not counting the variance involved as opposed to ours.
Our proposed formulation is closely related to the knapsack problem and its myriad variants.
In thestochastic knapsack problem, each item has a deterministic value and an independent stochastic size;the actual size of an item is revealed only after it is selected [8, 6, 2].
For the “one-shot” version ofthis problem, the aim is to choose a subset maximizing the expected value of the realized items, suchthat the probability that these realizations violate the knapsack constraint is below some threshold.
Incontrast, our “items” have equal size, but we pay a penalty which is some function of our realizeddistance from our knapsack target size, exchanging the constraint for a mixed-sign objective.
In thisspirit, the one-sided loss functions we consider are similar to objectives which arise in the penaltymethod for solving constrained optimization problems [10], though our setting is stochastic and wedo not aim to ultimately satisfy a hard constraint.
Problem Formulation
We begin with some n items (corresponding to candidates) with associated values x 1 , . . .
, x n ∈ R .
Ifwe select an item i ∈ [ n ] , there is a probability p i ∈ [0 , 1] that we receive this item (the candidateaccepts the offer).
We write these values and probabilities as vectors x ∈ R n and p i ∈ [0 , 1] n .
Let
Z i ∈ { 0 , 1 } be the indicator variable that we receive item i if it is selected, sampled as Z i ∼ Ber ( p i ) .
We assume that the events that we receive each individual item are independent.
Let S Z ⊆ S denotethe random realization of received items, that is, S Z :=
{ i ∈ S : Z i = 1 } .
Our goal is to select asubset S ⊆ [ n ] .
First, we consider the reward for a subset S as the expected total value obtained:
[Equation]
At the same time, let M ∈ N + denote a target size that we want to achieve.
We want to control thevariance on the deviation between the realized size of S Z , which is | S Z | =
(cid:80) i ∈
S Z i , and the targetsize M .
We consider this penalty as
[Equation]
where ρ : N × N + → R ≥ 0 is a loss function to be speciﬁed later.
Combining the two parts, we deﬁnethe overall objective as
[Equation]
where λ ∈ R + is a hyperparameter that governs the importance of the penalty relative to the reward.
Our goal is to ﬁnd the subset that maximizes the overall expected utility:
[Equation]
We consider a range of choices of loss function ρ .
It is worth noting that since U ( S ) is a mixed-signobjective, the optimal value of U ( S ∗ ) may be negative depending on the instance and choice of ρ .
Given a target size M , it is natural for ρ to be a convex function minimized at M , which penalizes anydeviation from M , or alternatively a monotone convex function that is nonzero above M , which canbe seen as penalizing violation of the budget constraint.
We focus in particular on one- and two-sidedlinear and quadratic losses, which are formally introduced in Section 3.2 below.
Theoretical Results
To begin we note that if we only consider the reward term and set the penalty term as V ( S ) ≡ 0 , thenthe solution is to trivially select all items.
In what follows, we ﬁrst discuss the other extremal case,taking R ( S ) ≡ 0 and considering the penalty V in isolation.
These may be viewed as the cases whenλ = 0 and λ → ∞ .
We then turn to the general objective and consider both terms jointly.
3.1 Warm-Up: Penalty Only
To gain intuition for this problem, we ﬁrst consider the simpliﬁed case where the goal is only tominimize the penalty term.
Note that in this case our objective is strictly non-positive.
When thereare multiple solutions that minimize the penalty term, we deﬁne the set of optimal solutions as theones with the maximal reward.
Require: x ∈ R n , p ∈ [0 , 1] n 1: S ← ∅ .
2: Sort i ∈ [ n ] lexicographically in decreasing order according to ( p i , x i ) 3: for i = 1 , 2 , . .
. , n do 4: if U ( S ∪ { i } ) ≥ U ( S ) then 5:
S ← S ∪ { i } 6: else 7: break 8: return S
We consider P G REEDY , the greedy algorithm with respect to p i (Algorithm 1).
In words, P G REEDYselects items in their decreasing order of probabilities, and if there are multiple items with the sameprobability it adopts the tie-breaking strategy that selects items in the decreasing order of their valuex i .
The algorithm keeps selecting the next item deﬁned by this order, and terminates when adding thenext item would decrease the objective.
As usual this greedy algorithm is efﬁcient, since the stoppingcriterion can be checked in polynomial time given access to ρ .
Note that the item values x are onlyused for tie-breaking.
Surprisingly, P G REEDY is optimal for minimizing V ( S ) in isolation.
Proposition 1.
Let M ∈ N + be any target size.
For any convex loss ρ ( · , M ) , P G REEDY (Algorithm 1)yields an optimal solution to minimizing the penalty min S ⊆ [ n ] V ( S ) .
The proof of this proposition is provided in ?? .
Note that this result is not obvious, as one mayexpect that as the sum of probabilities of all selected items so far gets closer to the target size, it maybe better to select an item with lower probability than an item with higher probability to “ﬁll thegap.”
This is not true.
Intuitively, it is because the realization of each Z i is binary, so the outcome ofadding another item i into the selection is either we add this item (with probability p i ) or not (withprobability 1 − p i ).
If adding this item gives lower penalty, then we desire to add the item with thelargest probability possible.
3.2 The General Objective
We now turn to the general objective.
At the outset it bears noting that U ( S ) is submodular in Swhenever ρ is convex, as is the case for the loss functions we consider.
Remark 1.
Suppose ρ ( · , M ) is convex, then U ( S ) is submodular in S .
The submodularity is used for analyzing ρ =
L +1 (in Section 3.2.2 and the proofs in ?? ).
Unfortunatelythe existing body of work on (non-monotone) submodular maximization cannot be deployed to obtaina general-purpose approximation to U ( S ) , since U is mixed-sign and may be negative even atoptimality, and applying an afﬁne transformation in order to engineer nonnegativity in generaldestroys any approximation guarantees.
We focus on a few natural choices of the loss function.
First, we consider ρ given by linear andquadratic losses, which we denote by L 1 and L 2 respectively.
These yield penalty terms V ( S ) whichare equal to the mean average error (MAE) and mean squared error (MSE) for the realized size of the subset S Z .
We also consider the corresponding one-sided losses, which are
L +1 ( | S Z | , M ) : = (cid:26) | S Z | − M if | S Z | ≥ M otherwise , and
L +2 ( | S Z | , M ) :
= L +1 ( | S Z | , M ) 2 .
All of these losses considered penalize the case where the realized size is greater than the target size.
In applications such as admissions and hiring, there is a limited, pre-speciﬁed amount of resourcesallocated to the newly admitted or hired people.
Hence, having more people than the target size is notdesired.
At the same time, the two-side losses give explicit preference that the realized size shouldalso not be smaller than the target size.
This explicit penalty for undershooting could stem from a hitto morale (an unsuccessful recruitment cycle really is demoralizing) or insufﬁcient manpower forrequired tasks such as teaching certain courses.
The one-side loss functions also implicitly capturethis consideration, as there is an implicit opportunity cost described by the reward term when fewercandidates accept.
3.2.1 An FPTAS for L 2 Loss
Recall that we have the solutions in both extremal cases (when only considering the reward term orthe penalty term), so one might hope that some interpolation between them could solve the generalcase.
However, the general case is more complicated.
In particular X G REEDY and XP G REEDY aregreedy algorithms analogous to Algorithm 1 which select i ∈ [ n ] in decreasing order of their valuex i (with ties broken in the decreasing order of p i ) and expected reward x i p i , respectively.
Despitethese heuristics being natural and intuitive, in our setting they perform arbitrarily poorly even for the
MSE-reminiscent loss L 2 .
To see this, consider the limiting case above for which P G REEDY is optimal.
When λ becomes largerelative to the values x i , the variance that i with smaller probabilities p i confer to the loss term willbecome more important relative to their expected reward, and X G REEDY and XP G REEDY can bemade to choose items with smaller p i than they should.
At the same time, P G REEDY may be inducedto incur only arbitrarily small positive value, even when large positive value is attainable.
For a moreformal demonstration, see ?? .
Despite the failure of heuristic approaches, when the chosen loss function is ρ =
L 2 , our problem canin fact be approximated up to a negligible error.
For this loss, our full objective (1) may be written as
[Equation]
Letting b i ∈ { 0 , 1 } be the binary decision variable for whether i ∈ S , that is, b i :
= 1 { i ∈ S } , the optimization problem then becomes
[Equation]
Expanding Eq.
(3) yields a collection of terms which are constant, linear, and quadratic in the s i , andso the objective can be reformulated as an unconstrained binary quadratic program (UBQP).
Although
UBQP is strongly NP-hard, Çela et al.
[3] present a pseudo-polynomial algorithm for UBQP whenthe coefﬁcient matrix for the quadratic form in the objective has constant rank.
By proving that theobjective (2) is sufﬁciently insensitive to small changes in our problem parameters, we may leveragethis pseudo-polynomial time algorithm to derive a FPTAS for approximating the optimal objectivevalue for our problem.
A standard search-to-decision reduction then yields the following result.
Theorem 1.
For ρ =
L 2 , there exists an algorithm that identiﬁes some S ⊆ [ n ] satisfying
U ( S ) ≥ U ( S ∗ ) − (cid:15) in time poly (1 /(cid:15), n, λ, x max ) .
Here x max is the maximum value
x i over all i ∈ [ n ] .
On the other hand, by a reduction fromequipartition we have the following hardness:
Theorem 2.
For ρ =
L 2 , optimizing U ( S ) is weakly NP-hard.
The proofs are provided in ?? .
The hardness landscape of our problem when ρ =
L 2 is thereforesimilar to that of the knapsack problem, which is heartening since the knapsack problem is relativelytractable in practice.
By contrast, the one-sided loss L +2 is not so well-behaved.
In this case the objective does not admit aquadratic factorization in terms of decision variables, and although the objective is nonnegative, it isdifﬁcult to analyze the performance of the greedy algorithm or contend with the nonlinearity of theloss function in a principled way.
We instead turn to L +1 loss, where surprisingly these obstacles canbe overcome.
3.2.2 Approximations for L +1 Loss
The loss ρ =
L +1 enables the possibility of a multiplicative approximation because the optimumvalue U ( S ∗ ) of the mixed-sign objective is non-negative.
More generally, any ρ with ρ (0 , M ) = 0has a non-negative optimal value, since in this case U ( ∅ ) = 0 ; for ρ
= L +1 any singleton item i withpositive x i and p i has strictly positive objective, since M ≥ 1 implies that L +1 (1 , M )
=
and so
U ( { i } ) =
p i x i .
The L +1 loss also appears likely amenable to a greedy algorithmic approach, since early items incurno penalty and the marginal penalty of adding a later item i is simply proportional to the probabilitythat the current solution exceeds the target M .
However, we ﬁnd that natural greedy algorithms,while efﬁcient, perform arbitrarily badly compared to the optimal solution in the worst case.
Recallthat one approach, X G REEDY , is to choose items in order by value.
But if M = 1 , items are of type( x a , p a ) =
(1 , p ) or ( x b , p b ) =
(1 , 1) and λ is large, then X G REEDY is only a p -approximation to
U ( S ) .
Another greedy algorithm mentioned earlier is XP G REEDY , which chooses items i in the order of theirexpected value x i p i .
Note that this is the marginal contribution of i to the reward term of the objective.
But the instance given by M = 1 , λ = 1 /p , and types ( x a , p a ) =
(1 /p, p ) and ( x b , p b ) =
(1 , 1)shows that XP G REEDY is also at best a p -approximation; for details see ?? .
These upper bounds on the performance of X G REEDY and XP G REEDY are intriguing in that theyboth scale like p , where p is a probability in the instance which becomes small.
It suggests thatthe minimum probability p min is a natural parameter for measuring the complexity of an instancewith respect to ρ =
L +1 loss.
Surprisingly, the performance of these greedy algorithms can belower-bounded in terms of p min as well, under an additional assumption about the value of items.
Theorem 3.
If ρ = L [-+
-]+1 and there is some c for which 0 ≤ x i ≤ (1 − c ) λ , then XP G REEDY isa Θ( p min ) approximation to U ( S ∗ ) .
Furthermore, X G REEDY is between an O ( p min ) and Ω( p 2 min )approximation to U ( S ∗ ) in this setting.
The proof of this theorem is provided in ?? .
This is in notable contrast to when ρ =
L 2 , wheremultiplicative guarantees are ill-deﬁned and X G REEDY and XP G REEDY may incur arbitrarily largeadditive loss.
Theorem 3 also raises a natural question: is this 1 /p min threshold tight, or is it possibleto efﬁciently attain better approximations to U ( S ∗ ) which do not depend on p min ?
We address this byintroducing O NE S IDED L +1 , presented in Algorithm 2, which attains a constant-factor approximationto the optimal solution.
In Algorithm 2, for any vector v ∈ R n and set S ⊆ [ n ] , we use v | S todenote the | S | -dimensional vector by restricting v to the coordinates indexed by S .
Its runtime isparameterized by the minimum probability p min and the ratio of the maximum value x max to thepenalty parameter λ ; in particular for ﬁxed p min and λ/x max it is polynomial in n .
At a high level, this algorithm proceeds ﬁrst by dividing the items into three groups according totheir values x i , and considering each group in turn.
Since U is submodular (Remark 1), the optimalsolution within at least one of these groups will be constant-competitive with U ( S ∗ ) .
We obtain aconstant-factor approximation for each group in a different way.
For the items with low values bounded away from λ , L OW V ALUE L +1 ( ?? in ?? ) checks all smallsubsets, which succeeds if the optimal subset for this group is small.
It also computes roundedprobabilities and values ( q i , y i ) for each item in this group, and then efﬁciently computes the optimalsolution according to this rounded instance.
If the optimal subset is large, we then prove that thissearch over rounded solutions necessarily identiﬁes a subset with objective value comparable to thatof the optimal subset.
This is the technical crux of proving that O NE S IDED L 1 is a constant-factorapproximation.
For the items with values just below λ , M EDIUM V ALUE L +1 ( ?? in ?? ) returns the optimal subset ifthe group is small.
If the group is large, it tries to choose a subset such that the expected numberof realizations is about M ; if there are not enough it chooses a subset with approximately half the expected realizations of the group overall.
Finally for the group of items with values above λ , it iseasy to see that choosing the entire group is optimal.
The pseudocode and related proofs for thesealgorithms appear in ?? .
The following result provides a theoretical guarantee for Algorithm 2.
Algorithm
Require: x ∈ R n , p ∈ [0 , 1] n , M ∈ N + , λ ≥
Ensure: S ⊆
[ n ] for which U ( S ) ≥ c · U ( S ∗ ) for universal constant c 1: N L ← { i ∈ [ n ]: x i ≤ (1 − p min 4 ) · λ } 2: N M ← { i ∈ [ n ]: (1 − p min 4 ) · λ < x i < λ } 3: N H ← { i ∈ [ n ]: x i ≥ λ } 4: S L ← L OW V ALUE L +1 ( x | N L , p | N L , λ, M ) 5: S M ← M EDIUM V ALUE L +
[Table]
x | N M , p | N M , λ,
: S
← N
Theorem 4 (Constant-factor approximation for L +1 ) .
Algorithm 2 is a constant-factor approximationto U ( S ∗ ) which runs in time n O (cid:16) 1 /p 2 min log
[Table]
Intuitively, the reason Algorithm 2 divides the items into cases depending on their values is to handlethe case when the reward portion of U ( S ∗ ) is almost equal to the penalty portion.
This presentsan impediment to the efﬁcient strategy of solving a rounded version of the instance, since in thiscase the magnitude and even sign of U ( S ∗ ) is potentially quite sensitive to changes in the p i
and x i .
By restricting attention to the low-value case, we may prove that the expected number of realizeditems in S ∗ is not much more than the threshold M ; this then allows us to argue that there exist goodrounded solutions.
We conclude our theoretical results with a surprising equivalence between one- and two-sided linearlosses L +1 and L 1 .
In what follows, let U L 1 and U L +1 denote the objective with ρ =
L 1 and ρ =
L +1 ,respectively.
Consider the afﬁne map between problem instances ψ : ( x, p, λ, M ) (cid:55)→ ( x (cid:48) , p (cid:48) , λ (cid:48) , M (cid:48) )which takes x (cid:48) i :
= x i + λ and λ (cid:48) :
= 2 λ , and which leaves M and p unchanged.
If S (cid:48) denotes S ⊆ [ n ]in the instance transformed by ψ , then we have the following:
Theorem 5 (Equivalence between L 1 and L +1 ) .
For all instances ( x, p, λ, M ) and for all S ⊆ [ n ] ,
[Equation]
In particular, since λ and M do not depend on S , this implies that S maximizes U L 1 on ( x, p, λ, M )if and only if it maximizes U L +1 on ( x (cid:48) , p (cid:48) , λ (cid:48) , M (cid:48) ) .
Although Theorem 5 establishes a correspondence between the solutions to our problem for ρ =
L 1and ρ =
L +1 , it is not approximation preserving and so does not convert O NE S IDED L 1 into anapproximation algorithm for the two-sided setting.
Indeed, as noted above, the optimal value whenρ =
L 1 may be strictly negative.
Numerical Experiments
Having established worst-case theoretical guarantees, we wish to test how well our algorithmsperform empirically.
We focus on L +1 loss because our result for L 2 is an FPTAS, so we know itsperformance is arbitrarily close to optimal.
All error bars shown in the plots represent standard errorof the mean.
4.1 Experimental Setting
In constructing instances we follow the approach of Purohit et al.
[11] in their use of beta distributionsto orchestrate correlation between x i and p i .
We therefore ﬁrst draw x i ∼ Unif [0 , 1] , and thenproduce three types of correlation as follows:
• Negative correlation: p i ∼ p min + (1 − p min ) ·
Beta (10(1 − x i ) , 10 x i ) .
5 target
O b j ec t i v e pos
LowValueL1+ xGreedy xpGreedy    5 target 0 .1 .1 .2 .2 .3 .3 .
O b j ec t i v e no
LowValueL1+ xGreedy xpGreedy    5 target 0 .1 .1 .2 .
O b j ec t i v e neg
LowValueL1+ xGreedy xpGreedy
• Positive correlation: p i ∼ p min + (1 − p min ) ·
Beta (10 x i 10(1 − x i )) .•
No correlation : p i ∼ Unif [ p min , 1] .
This construction differs from the sampling paradigm of Purohit et al.
[11] only in that we renormalize the probabilities { p i } , so that they are bounded in [ p min , 1] .
We consider n = 50 andp min = 0 .
01 throughout, and explore the greedy heuristics X G REEDY and XP G REEDY , as well asthe constant-factor approximation algorithm O
NE S IDED L +1 (Algorithm 2), for a range of M and λ .
The lower bound on p min ensures that the performance of the greedy heuristics and runtime of ouralgorithm are reasonable; a value of 0 .
01 is very realistic because in practice, if a candidate takes thetime and effort to apply, it is reasonable to assume that they at least have some non-trivial probabilityto accept if they were given an offer.
We also focus on the regime where x i < λ , which is assumedby Theorem 3 and handled in Algorithm 2 by the subroutine L OW V ALUE L +1 .
We believe that this isthe main regime of practical interest: candidates with x i ≥ λ are so exceptional that (under L +1 loss)we would want them to join regardless of how many candidates have already accepted offers, andsuch candidates are very rare indeed.
Note that our theoretical guarantees (Theorem 4) necessitate that all candidate solutions up tosize τ = poly (1 /p min ) are checked by brute force.
In this implementation of L OW V ALUE L +1 wetake τ = 0 and isolate its search over rounded solutions.
As we consider small target sizes M ,this prevents L OW V ALUE L +1 from outperforming the greedy algorithms simply by virtue of bruteforcing over all relevant solutions.
This only hinders the performance of L OW V ALUE L +1 .
In fact, L OW V ALUE L +1 only considers rounded solutions S satisfying (cid:80) i ∈
S x i p i ≤ 2 M .
So long as x i < λ ,such a stopping condition may be deployed without loss of generality.
For details see ?? . We believethis offers a favorable tradeoff between runtime and accuracy, and again serves as a lower bound to
L OW V ALUE L + 1 as it is written.
4.2 Experimental Results
The objective values that our algorithms of interest attain for these distributions are shown in Figure 1.
Note that positive correlation leads X G REEDY and XP G REEDY to pursue very similar (and optimal)strategies, as expected.
This is intuitively the easier setting, and here L OW V ALUE L +1 performs on parwith the greedy heuristics.
In the no-correlation and negative-correlation settings, there are regimeswhere one of the two greedy heuristics performs better than the other one, whereas L OW V ALUE L +1appears to perform as well as the better of two depending on the regimes, showing its better adaptivityacross these instances in practice as well as in theory.
Negative correlation between x i and p i is of particular interest to us.
It is also the regime that is themost relevant for the setting of faculty hiring and PhD admissions, and in fact hiring and admissionsmore broadly.
In Figure 1, we also observe that negative correlation is the setting that displays themost heterogeneity in algorithm behavior.
We therefore turn to this negative-correlation setting andexplore the effect of increasing the penalty regularizer λ in Figure 2.
In general, L OW V ALUE L +1appears comparable to the better of the two greedy approaches across the values of M and λ that
5 target 0 .1 .1 .2 .2 .
O b j ec t i v e lambda=1.
LowValueL1+ xGreedy xpGreedy    5 target 0 .1 .1 .2 .
O b j ec t i v e lambda=
LowValueL1+ xGreedy xpGreedy    5 target 0 .0 .1 .1 .2 .
O b j ec t i v e lambda=
LowValueL1+ xGreedy xpGreedy we examine, though there is a small gap between the objectives achieved by L OW V ALUE L +1 and X G REEDY when λ = 1 . 5 .
This is also good news for X G REEDY and XP G REEDY , because it suggests that the two of themtogether remain competitive across a wide range of instances.
To the extent that L OW V ALUE L +1falls short of U ( S ) , it is due to systematically rounding the probabilities p i up by a constant factorwhen computing the prospective utility of solutions.
Because its rounding preserves the rewardterm, such a systematic overestimate in the p i leads it to overestimate the penalty term of any setunder consideration.
The presence of a large number of ( p i , x i ) with individually balanced butcollectively large impact on the objective could therefore explain the extent to which L OW V ALUE L +1lags behind X G REEDY in Figure 2; the latter chooses many such individuals while the former judgestheir inﬂuence on the penalty to be too large.
However, this can be mitigated by choosing smallermultiplicative bucket sizes for L OW V ALUE L +1 , which is particularly effective in the case where thep i of an instance fall in a small number of clusters or exhibit other structure.
Discussion
One of the takeaways from our theoretical and empirical results is that the greedy algorithm X G REEDY ,which makes offers to a subset of candidates with the highest values, is practicable for L +1 loss.
This is intriguing because the algorithm is quite similar to how faculty hiring and PhD admissionscommittees typically think: they want to make offers to the best candidates.
The difference is that
X G REEDY carefully selects the number of offers to be made, in a way that (greedily) maximizesthe objective.
Since X G REEDY amounts to a relatively small tweak to current practice, we believecommittees would ﬁnd the algorithm to be especially palatable.
An issue our results do not address is which penalty function best matches the needs of a speciﬁcrecruitment process.
For example, is there a rigorous way to argue that a particular choice of penaltyfunction is more broadly applicable than another?
That said, the choice between one-sided and twosided penalty is rather intuitive, depending on the application.
And our results provide computationalarguments in favor of L 2 when two-sided penalty is desired, and L +1 for one-sided penalty.
From an ethical viewpoint, a potential concern is that our proposal may ultimately have unintendednegative consequences.
For example, if many faculty hiring committees adopted our optimizationbased approach, might faculty candidates have fewer opportunities?
We believe, however, that theopposite is true.
Currently the academic job market is strikingly inefﬁcient, as committees convergeon a few candidates who are inundated with interviews and offers, while comparably strong candidatesare left with nothing.
If our approach is adopted widely, it is likely to widen the pool of candidateswho receive appealing offers.
Granted, a centralized matching market (in the style of the National
Resident Matching Program) may be an even better solution, but creating such a market requires ahuge — and often impractical — degree of coordination; by contrast, our approach can be adoptedindependently by institutions and even by individual departments or committees.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]