Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Authors: Lee Cohen
Connie Hong
Jack Hsieh
Judy Hanwen Shen
Venue: ICML 2025 poster (2025)
Keywords: fairness
strategic classification
hiring
societal impacts
trustworthy machine learning
Primary area: social_aspects->fairness
TL;DR: A new model for LLM strategic manipulations grounded in real world hiring systems

Abstract: In an era of increasingly capable foundation models, job seekers are turning to generative AI tools to enhance their application materials. However, unequal access to and knowledge about generative AI tools can harm both employers and candidates by reducing the accuracy of hiring decisions and giving some candidates an unfair advantage. To address these challenges, we introduce a new variant of the strategic classification framework tailored to manipulations performed using large language models, accommodating varying levels of manipulations and stochastic outcomes. We propose a "two-ticket" scheme, where the hiring algorithm applies an additional manipulation to each submitted resume and considers this manipulated version together with the original submitted resume. We establish theoretical guarantees for this scheme, showing improvements for both the fairness and accuracy of hiring decisions when the true positive rate is maximized subject to a no false positives constraint. We further generalize this approach to an $n$-ticket scheme and prove that hiring outcomes converge to a fixed, group-independent decision, eliminating disparities arising from differential LLM access. Finally, we empirically validate our framework and the performance of our two-ticket scheme on real resumes using an open-source resume screening tool.

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Lee Cohen 1
Connie Hong"
Jack Hsieh 1
Judy Hanwen Shen 1
Abstract
journalists (Lytton, 2024), and policy makers (City of New York, 2021). Al tools that exhibit undue biases and un- explainable behavior present major barrier to achieving accountability in these algorithmic hiring schemes (Dastin, 2018). Although algorithmic hiring tools are designed with the goal of hiring the best candidates, these tools may not be robust to candidates manipulating their application ma- terials. For example, Applicant Tracking Systems (ATS) are widely used to screen qualified candidates in the first stage of the hiring process: however, there exist many tuto- rials for manipulating resumes for this system (Stahl, 2022). This problem has been studied through the lens strategic classification where individuals can manipulate their inputs (e.g., a job application) influence the decision made by classifier (e.g., hiring algorithm) (Hardt et al., 2016a; Kleinberg & Raghavan, 2020; Levanon & Rosenfeld, 2021). The goal of the hiring side is to design strategy- -proof se- lection algorithm, whereas the goal of the applicants is to maximize their utility: the difference between the benefit of receiving a positive prediction and the cost of manipulation. The applicant's "best response" is then the optimal way an individual should manipulate their input - given knowl- edge of the classifier's behavior in order to maximize their utility. The challenge lies in designing classifiers that are robust to such manipulations while maintaining fairness and accuracy.
In an era of increasingly capable foundation mod- els, job seekers are turning to generative Al tools to enhance their application materials. However, unequal access to and knowledge about genera- tive Al tools can harm both employers and candi- dates by reducing the accuracy of hiring decisions and giving some candidates an unfair advantage. To address these challenges, we introduce a new variant of the strategic classification framework tailored to manipulations performed using large language models, accommodating varying levels of manipulations and stochastic outcomes. We propose a "two-ticket" scheme, where the hiring algorithm applies an additional manipulation to each submitted resume and considers this manipu- lated version together with the original submitted resume. We establish theoretical guarantees for this scheme, showing improvements for both the fairness and accuracy of hiring decisions when the true positive rate is maximized subject to a no false positives constraint. We further generalize this approach to an n-ticket scheme and prove that hiring outcomes converge to a fixed, group- independent decision, eliminating disparities aris- ing from differential L.MM access. Finally, we empirically validate framework and the perfor- mance of our two-ticket scheme on real resumes using an open-sourc resume screening tool.
With the recent proliferation of generative AI services that are now widely used by job seekers (Chamorro- -remmrice 2024), new variable has been introduced to the algorithm hiring cycle and strategic classification. Writing editing resume using generative AI has become accessible and widespread. In a recent survey, 57% of respondents admitted to using AI to create their resume (Business Wire, 2025). Since candidates have no knowledge of how employers make hiring decisions, the optimal strategic classification response becomes straightforward: candidates edit their resumes using their preferred AI tool, opting for premium version if they recognize its advantages and can afford it. As a result, those accessing better models may gain an unfair advantage in the selection stage of automatic hiring a algorithms. Thus, companies may be filtering for candidates who used the best LLMs rather than candidates who are the most qualified.
1. Introduction
Hiring decisions can profoundly impact an individual's 's pro- fessional path and long-term success. As algorithmic tools are increasingly deployed to recommend or make these decisions, they have rightfully come under scrutiny from economists (Hu& & Chen, 2018: van den Brock al., 2020),
'qqual contribution 'Stanford University, Palo Alto, CA. USA. Correspondence to: Judy Hanwen Shen <jhshen eddm
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
The interactions between hiring algorithms and application- enhancing generative Al tools create unique setting to examine fairesss and strategic behavior. Since manipulation in this setting is low-effort, many candidates will choose to manipulate their resumes, even this does not guarantee a positive outcome. This contrasts with the classic model, where individuals manipulate their input only when posi- tive outcome is achieved. Moreover, strategic classification in the era of LLMs introduces two key challenges: (1) unlike prior group-based fairness settings, the hirer cannot directly determine whether an application has been manipulated or which LLM was used, and (2) unlike the classic strategic classification setting, manipulations are stochastic, as LLM outputs are inherently non-deterministic. Motivated by this complex yet realistic interaction between strategically gener- ated application materials and algorithmic hiring algorithms, work presents first step into modeling and analyzing al algorithmic hiring ecosystems in the era of generative Al; our contributions are as follows:
Scores Applicant Mariputations (DoorDash Product Manager)
Monipulation
Claudo 3.5 Sonnett
App kook Manipusaton Model
DeepBook- 67B
DeepBack-VB
OPT-35-Tubee
QPT4wmm
GPT4
Unmmdd Ii--iattnc--urte
80
85
Resume Score
Qualteed
Unqualited
Quuvltted pS0
Figure 1. Resume score distribution of 50 qualified (matching oc- cupation) and 50 unqualified (different occupation) resumes before and after LLM manipulations for DoorDash Product Manager job posting. Models such GPT-4o, Claude-3.5-Somen and DeepSeek- V3 generate distribution of unqualified resumes that is indistinguishable from qualified resumes.
We show that some models enhance resumne relevance scores more than other models, and that the benefits of repeated LLM manipulations stagnate (Section 3).
We translate the empirical behavior of LLMs used for job applications into a realistic model for strategic clas- sification (Section 4).
impact of manipulative tactics on classification problems. We draw on several later works that provide a modified strategic classification game that models disparities in ma- nipulation abilities (Hu et al., 2019b; Milli et al., 2019; Chen al., 2020; Diana et al., 2024). Furthermore, we use techniques from prior work (Braverman & Garg, 2020) to describe "random" classifiers in light of stochastic strategic manipulations. Similarly to several previous works (Ghalme et al., 2021; Cohen et al., 2024), we assume that the de- ployed classifier is unknown to the candidates. Finally, previous work (Somerstep et al., 2024) has explored the use of casual strategic classification to understand labor mar- ket dynamies: our work extends this by highlighting the effects of strategic behavior employer and labor force outcomes."
We prove that under existing hiring schemes, dispar- ities in access to LLMs lead to disparities in hiring outcomes, even under stochastic manipulations (Sec- tion 5) and unknown deployed model.
We introduce two-ticket scheme where the hiring algorithm applies an additional LLM manipulation to each submitted resume and considers this manip- ulated version together with the original submitted re- sume. We prove that this scheme improves disparities among candidates and accuracy for employers. We also generalize the two-ticket scheme to an n-ticket scheme, proving that the n-ticket scheme eliminates group-dependent disparities as oo, with outcomes converging exponentially to a fixed, group sooup-nndependent decision (Section 6).
3. Empirical Motivation: Stochastic Resume Manipulation using LMMs
We validate our theoretical model and results through a case study using real resumes and an open-source resume scoring algorithm (Section 7), demonstrating that two-ticket scheme enhances both faimnsss and accuracy in practice.
Since prior works in strategic classification focus on deter- ministic manipulations, we empirically motivate our theo- retical model of stochastic LLM manipulations. We prompt a variety of models to improve technology sector resumes (Drushchak Romanyshyn, 2024).2 We used a general prompt to simulate job applicant who aiming ap-
2. Related Work
Full related work including fairness in algorithmic hiring and the risks of generative models can he found in Appendix A. "See Appendix D for prompt details and prompt analysis periments to reduce hallucinations.
Strategic Classification Hardt et al. (2016a) introduce strategic classification as Stackelberg game to address the
2

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
ply to multiple jobs with the same enhanced resume (e-g., via recruitment agency). The resumes were then evalu- ated against the target job descriptions through open source software that scores resumes against designated job de- scription to produce a relevance score. This type of simple scoring model, as a first filter for resumes, is widespread, with 98.4% of Fortune 500 companies using them within ap- plicant tracking systems (Pureell, 2024). We identified three key behaviors of LLM manipulations: (1) LLM manipula- tions stochastically enhance resume scores (Figure 4a), The effectiveness of LLM manipulations varies by model: newer, premium LLMs improve resume scores more (Fig- ure 1), (3) Improvements from manipulations stagnate with repetition: applying the same LLM repeatedly to the same resume results in diminishing changes (Figure 4b).
according to a conventional resume structure. However, a second round of manipulations did not deviate substantially from the first. This observation is also reflected by the sim- ilarity of the resume score distributions of the once and manipulaae resumes (Figure 4b).
Together, these three key observations regarding LLM ma- nipulations - the potential for resume improvement, the differences in results among various LLM models, and the stagnation in changes from multiple iterations motivate our proposed model strategic manipulation in Section 4.
4. Model
We represent each candidate as a triplet (1r, 9. V). where x E g represents the candidate's original (unmanipulated) resume features; g E {P,, U} denotes the group membership. with P indicating the privileged group and U indicating the unprivileged group; and y E (0, 1} represents the true label, with 0 indicating an unqualified candidate and 1 indicating a qualified candidate. It is important to note that we do not require that æ fully determines y.
Figure illustrates that using simple job-agnostic prompt with an input resume significantly improves the scores com- puted by a resume screening system. A second observation that motivates our study of disparities is the differential out- comes resulting from applying different LLMs to a candi- date's resume. Figure 3 shows the post-manipulation resume scores across a broad set of models. Using the dotted lines as reference for the median score of the original resumes of the qualified group, it is evident that applying different LLMs has different effects on the outcome relevance score. Distinguishing qualified and unqualified candidates is al- ready a difficult task, but candidate manipulation makes harder. Some models, particularly the higher cost-to- access models (e.g., CLAUDE-3. 3.5-E--.5--ONNNE GPT-4o) im- proved the resume scores of the unqualified resumes so that they were indistinguishable or better than the qualified re- sumes without LLM manipulations, while cheaper or free-to- access models (c.g.. GPT-3.5- TURBO, MIXTRAL-8X" B- INSTRUCT) did not significantly improve scores on average of the unmanipulated resumes regardless of qualification." 3 By qualitatively inspecting the manipulated resumes, we found that LLMs yielding larger score improvements (c.g., GPT-4o) better adhered to the traditional elements of resume while less effective models simply reorganized the input resume. For example, GPT-40 also added additional elements such as a resume summary and dedicated sections for educational history.4 We also found that all of the newer, premium models (particularly CLAUDE-3.5- S.NNNN) in- creased the average resume similarity (embedding distance) (Figure 8).
Our model accommodates any combination of d₃ funda- mental and d2 style features in the feature space (i.e., d di + d2). Style features (21, vdd, refer to attributes about resume's presentation such as writing quality, vocabulary, and grammar, whereas fundamental features (cI, C...,.....CA refer to technical attributes such as programming skills, years of experience, or educational background.
Overall, we express each candidate's 's resume features an d-dimensional feature vector in I::
[21; 22 ................................ Idy C12 C2 ..., Cazl.
We model the candidate population as a joint distribution D over feature vectors, group memberships, and true labels. We define the random variable triplet (X, G, Y) ~ D with X E Rd. G E (P,U), and Y E {0, 1). Moreover, we assume that both groups have identical distributions over resume feature vectors, and that the true label is independent of group membership - that is, we assume that X and G are independent and that Y and G are conditionally indepen- dent given X. For our model to be appropriate, each group comprises a nen-nggligible proportion of the population: that is, P-G P), P(G U) > O.
Finally, we also observed that repeated manipulations did not significantly alter resumes. The first round of modi- fications typically standardized language and formatting
4.1. LLM Manipulation
We assume some candidates are manipulating using LLMs (Verma & Renjarla, 2024; Stahl, 2022). In what follows, we formalize our model for LLM manipulation of
Model pricing rapidly changes for consumer platforms. Fur- thermore, not all models are available on consumer platforms. We include a cost analysis for API access to simulate third- party career services with tiered offerings in Table 7.
$If all features are fundamental features (d, = 0), then the scenario reduces to traditional non-strategic classification.
'Representative examples can be found in Appendix C.4.
3

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
to a fixed random distribution. We remark that L could rep- resent a single use multiple uses of an LLM to improve resume.
69
X2 (eg. Years of Experionce)
Decisiin Boundary
6.-
4.1.1. 1. HIRING SCHEMES
In our hiring scheme, we define the Hirer who is making the hiring decisions and the Candidate who is applying for the job. Our work focuses on job positions receiving large volumes of applications: for this reason, we assume that the Hirer screens each candidate's 's resume by assigning each real-valued score. More specifically, our model assumes that the Hirer uses some fixed scorer to evaluate the candidates resumes (in our experiments, we use Resume Matcher as the scorer). We represent this scorer as function 8 : IR + R. We make no assumptions about other than that it is monotonically non-decreasing.
X
[.(x)
63
62
11 (eg. Resume Readabilty)
01
02
67
Figure 2. Visualization of an LLM Manipulation over = with one style feature (ec) ) and one fundamental feature (u2).. Modifying 2i may move the candidate into the acceptance region (in blue).
Note that we assume that the Hirer has additional con- trol over s. In practice, employers have little control over the scorers purchased from applicant screening software providers at the candidate screening stage. They can, at best, choose the best (e.g.. most accurate) scorer available and tweak it accordingly. Under our model, as the Hirer does not have the resources to manually filter resumes, the resume score is how the Hirer decides which candidates move to the next stage of hiring. This reflects the widespread usage of applicant tracking systems by employers.
resumes.
Definition 4.1 (Mathematical formulation of Strategic LLM Manipulation). An LLM manipulation is random function L R + R characterized by a series of (not necessarily independent) real-valued random variables X1. Xd, When called upon a feature vector æ = [21,.... Id, Cl...,,Cddll Cdyl.
1. L replaces each x; with a value drawn from X for ssisan d1.
We assume that each Candidate group has access to its own LLM, Lg. Likewise, the Hirer also has access to their own L.MSs that are separate from the Candidates'.
2. L preserves the value of ej for 1 S S d2.
That is,
4.2. Traditional Hiring with LLM Manipulations
T([xI...., C1...,, cd,l) = (x,...;Xaa C1,... Cdg]-
We now introduce our strategic LLM classification game for traditional hiring with Candidate LLM manipulation. candidate can use the LLM available to their group (Lg) to manipulate their resume the candidate then chooses which of these two resumes (original or manipulated) to submit to the Hirer.
Our formulation of LLM manipulations is based on our ob- servations that LLMs can standardize style features such as writing quality, vocabulary, and organization - to the point that the original values are irrelevant and are redistributed according to a distribution dependently only on the LLM. On the other hand, candidates would like LLMs to preserve their fundamental features. Changes their fundamental features may be extremely costly to the candidate, hir- ers may decide to blacklist or dismiss dishonest candidates. The prompts for our experiments also work to elicit this out- come: to minimize the chances of hallucination, our prompt states explicitly that is "imperative that the new resume do not add any facts that are not in the original resume". We manually inspected some sampled outputs to confirm that generated outputs were free of hallucinations, though not all outputs were inspected. This perspective is informed by our empirical observations detailed in Section 3. Our exper- iments first indicated that LLM manipulations effectively overwrite writing style attributes. This is captured our model, where the style features are redistributed according
The Hirer determines a threshold t E R and accepts can- didates with scores equal to or greater than the threshold. Namely, the Hirer decision regarding a candidate with a submitted resume x' is (x sume that there are many candidates, in which case mini- mizing false positives is natural hiring objective, as the cost of interviewing or hiring unqualified candidates out- weighs the cost of overlooking qualified ones. False positive has been studied in the context of fairness (c.g., Cohen et al. (2020), Blum al. (2022)), and strategic classification (e.g., Ahmadi al. (2022), Shao al. (2023)). In this vein, we introduce the No False Positives Objective.
[s(x') 2 r]. We as-
Definition 4.2 (No False Positives). The No False Posi- tives Objective is achieved when the Hirer maximizes true
4

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
positive rate (TPR) subject to false positives. The opti- mization problem is:
application. This is in contrast prior work in strategic classification, where manipulations, such getting multiple credit cards, require time and effort (Hardt et al., 2016a). Our model does however separate Lp and Lu: this is equiv- alent to how privileged groups in prior works are given larger budgets when costs are incurred (Milli et al., 2019).
maxmize, subject to FPR(T) =
TPR(T)
where
The Hirer does not know and may not infer whether a resume has been manipulated or from which group a resume comes. Rather, the Hirer must use the same scoring scheme and threshold for all candidates.
TPR(T) = P()()) = and
FPR(T) = ((f-((() = = 0).
We let 7" denote the minimum threshold in the solution set.
Our work specifically focuses on classifiers that optimize true positive rates: this approach will specifically inform our further study of disparities between groups in Section 5. We aim to satisfy a specific case of Equalized Odds (Hardt et al., 2016b) when the false positive rate fixed zero, which is special case of equalized false positive rates across groups.
5. Disparities in Traditional Hiring with Unequal Candidate LLM Manipulation
In this section, we show that, under a traditional hiring scheme, disparities in LLM qualities between candidate groups can lead to disparities in hiring outcomes. We begin by defining a useful metric for disparity hiring outcomes. Since we assume that groups P and U have the same unma- nipulated feature vector distribution, we define the resume outcome disparity as follows.
Definition 4.3 (TRADITIONAL Hiring Scheme under LLM Manipulation). The Hirer and the Candidate play the fol- lowing Stackelberg game.
1. The Hirer commits to a scorer s and a threshold r E R. both unknown to candidates.
Definition 5.1. Given a resume feature vector æ E Rdd the resume outcome disparity A is defined as
2. Each candidate (æ,, g. y) chooses to submit either their original resume x' = æ or their LLM-mmnipulated resume x' = -g (zx).
A(x) P (f+ (x(p) = 1) P (f+(z()) 1), Le
L.
where x'1 argmax (., s(z) for g (P.U).
3. The Hirer accepts candidates according to the threshold classifier (x') = 1 [s(x') 2 r].
Observe that if the original unmanipulated resume is already accepted (that is, f. (x) = 1). then A(x) = O.
Each player has the following payoffs:
To capture the differences in the output quality of different LLMs, we use the notion of multivariate stochastic domi- nance (Levhari et al., 1975).
1. The Candidate payoff is whether they are accepted:
1 [x'' = 1].
Definition 5.2 (Leeharri et al., 1975)). Let Z1, Z2 be ran- dom variables over Rd. For any E Rd let Fa(()) = P(Zk S a), where I denotes component-wise order. We say that Z1 stochastically dominates Zz if for any open lower set C RRd
2. The Hirer's payoff' is defined according to the No False Positives Objective (Definition 4.2).
Remark. Unlike classic strategic classification, our game does not directly assume that the Candidate has perfect knowledge about fr. hiring schemes are often opaque. However, we assume that candidates know which of the two versions (unmanipulated and manipulated) of their resume will score higher. Additionally, since writing a prompt in an I.M is very easy, we assume it has negligible cost and that each candidate will use the more advanced LLM if they have access to it (i.e., a candidate from the privileged group will not use Lv). A best-responding candidate in group g will therefore submit
ddz. dF S J dF2.
s
This is a generalization of (first-order) univariate stochastic dominance to multivariate distributions. Intuitively. stochas- tic dominance requires that the generalized CDF of Z1 must always be "less" than the generalized CDF of Zz. Stochastic dominance induces a partial order over multivariate random variables. Furthermore, we use the following key property about stochastic dominance.
J = argmax s(2).
=E (=)))))()))
Lemma 5.1 ((Levaari et al., 1975)). Z1 stochastically dom- inates Zz if and only if for every non non-decreasing function u,
In our model, candidates do not incur costs for prompting their L.M for the manipulation or for selecting the better
B(u(Zi)] 2 Elu((2)).
5

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
We use this definition define ordering over LLM quality.
6. Combating LLM Disparities: Two- Ticket Scheme
Definition 5.3. Let L1, L2 be LLM manipulations, We say that I, dominates L2 (I1 = L2) if for all E R (I()) stochastically dominates (z)).
To counteract the disparity in hiring outcomes due to un- equal LLM access, we propose modified hiring scheme where the Hirer performs their own round of LLM manip- ulation over the possibly manipulated applications. Our motivating experiments (Section 3) show that running a re- sume through nigh-quality LLM twice changes the resume much less the second than the first. Consequently, we show that bestowing both groups with the benefit of round of high-quality LLM manipulation can help level the playing field.
Informally, an LLM L1 may be considered "better" than L2 if stochastically dominates L2 on each input, indicating that L1 has a greater likelihood of feature improvement than L2. Note that this only implies that Lı tends to produce better output than L2; L2 may produce a better output than L on certain realizations of their stochastic outputs.
Remark. To simulate the absence of access to LLM in our strategic classification game, it is helpful to artificially define a "null LLM" (Lz) that is dominated by all other LLMs. We might informally conceptualize Lg as having random variables X= -co for 1 S S d1. This will allow us later to conceptualize traditional hiring special case of our proposed Hirer deploys the null LLM.
6.1. Two-Ticket Scheme
We present the modified strategic classification game un- der the TWO-TICKET scheme. This scheme is identical to TRADITIONAL hiring except that the Hirer now uses their own LLM (L#) to manipulate each submitted resume. The Hirer then scores the best of these two versions to determine whether to accept each candidate.
hiring scheme in which the
We now show that, under this definition, using a better LLM on the same resume leads to a better hiring outcome.
Definition 6.1 (TWO-TICKET Hiring Scheme under LLM Manipulation).
Theorem 1. Suppose Lp = Ly. Then for all x E
Rdd A (x) 2
1. The Hirer commits to a scorer and a threshold T E R, both unknown to candidates and some LLM L
Sketch). Since Prr,, (f= =) Rr, (xrr))) and fr is non-decreasing, we can apply Lemma 5.1 show that (f = 1) 2 PLu (f (x')) 1).
2. Each candidate (ær, 9.9)) chooses to submit either their original resume x' = or their LLM manipulated resume x' = Lg(x).
This disparity in resume outcomes naturally leads dis- parity in group outcomes. Under the No False Positives Objective, it is natural to measure group outcomes by com- paring groups' true positive rates. We denote the TPR over a group g as
3.
The Hirer chooses to consider the higher scoring re- sume among the submitted resume
x" = x' and the
submission x"
L (z').
4. The Hirer accepts candidates according to the threshold classifier f=(")) = 1 [s(x") T].
TPRg IP 1
=
To address fairness, we define the disparity between the TPRs of two groups. This fairness notion has been stud- ied previously in the context of strategic classification (e.g., (Keswani & Celis, 2023)).
Each player then has the following payoffs:
1. The candidate payoff is the probability that they are accepted: P... (f-("") = 1).
Definition 5.4. The TPR disparity ATrR is defined
2. The Hirer's payoff is defined according the No False Positives Objective (Definition 4.2).
ATPR = TPRp - TPRg.
"Text in blue distinguishes our Two-TICKET hiring scheme from the TRADITIONAL hiring scheme.
Having defined the TPR disparity, we show that qualified candidates from the privileged group have a higher (or equal) probability of being accepted compared to qualified candi- dates from the unprivileged group.
In practice, the Hirer scores both the submitted resume and the Hirer LLM-manipulated resume, and accepts the candi- date if one of the scores passes the threshold. This is where our name TTwo-Ticket Hiring" comes from: each candidate
Corollary 1. Suppose Lp 2 Ly. Then, ATPR 2 O.
(Proof Sketch). This follows from applying Theorem 1 over candidates with Y
6

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
is essentially given two avenues to acceptance. 6 This is equivalent to the above definition: our chosen presentation emphasizes the symmetry of the Hirer's LLM manipulation and the Candidate's s LLM manipulation.
Before giving our main results on outcome disparity, we reformulate the probability of acceptance under the Two- TICKET hiring scheme.
Lemma 6.2. For Hirer LLM Lu and threshold T, the prob- ability that candidate (a.g. g. y) is accepted is
While LLM manipulations generally improve resume qual- ity, there is a chance that they can decrease a candidate's s score (Section 3). To ensure that candidates are not unfairly harmed by LLM manipulations, we safeguard against this possibility by requiring the Hirer to evaluate the maximum of the candidate's 's submitted and its Hiver-manipulated ver- sion of each resume.
IP (fr (x%)) = =1 Lg
[s(x) < 7| E. (s(L)(())
<>) IP (s(LH(0)) <T). Lu
(Proof Sketch). This follows from Definition 6.1, using that Lu (x()) and a'g are conditionally independent given x.
The TRADITIONAL hiring game (Definition 4.3) can be con- sidered a special case of the (Definition 6.1), where Lu is the null LLM discussed in Section 5; in which case (x(x)) is always inferior to, and discarded in favor of, the original submitted re- sume x'. Thus, we compare the behavior of two different TWO-TICKET games: the game under TRADITIONAL hir- ing (Definition 4.3) and the game under TWO-TICKET hir- ing schemes (Definition 6.1). which now differ only in the Hirer's choice of L.LM and threshold in our formalization.
hiring game
Using Lemma 6.2, we derive our main result showing the improvement in resume outcome disparity by shifting from TRADITIONAL to TWO-TICKET scheme.
Theorem 2. Let = 7=(2), Lp - Lu, and 1 = Ln
Then for all x E Rd, A (2) (z) S A (1) (z).
Remark. Lemma 6.1 provides a simple and sufficient but not necessary condition that (1) = (2) under the No False Positives Objective. In fact, Theorem applies un- der any Hirer objective so long the optimal deployed threshold is the same for Hiring Scheme and 2.
6.2. Guaranteed Two-Ticket Improvements
We now prove that under natural conditions, a TWO-TICKES hiring scheme can decrease the resume outcome disparity between the two groups, leading to improvement in accuracy and fairness.
Under the No False Positives Objective, the decrease in resume outcome disparity immediately implies decrease in group outcome disparity and an increase in accuracy (or equivalently, true positive rate under the No False Positives Objective) for both groups.
For k E (1,2}. we define Hiring Scheme & to be the Two-TICKET scheme using Hirer LLM L and scheme- dependent threshold r(k), resulting in deployed classifier f(A). We let (a) (z) and A IPR denote the resulting re- sume and group outcome disparity respectively for Hiring Scheme k. To compare the TRADITIONAL hiring scheme with the TWO-TICKET hiring scheme, we denote the TRA- DITIONAL hiring scheme as k = 1 with [ = Lg. With this definition, that the following results still apply if Hiring Scheme 1 is TWO-TICKET hiring scheme with a non-null Hirer LLM.
Corollary 2. Let TPR( denote the true positive rate over
group g under Hiring Scheme k. Let (1)) (2) Lp t LU, and L > D Then
1. A (2)
APPR
< ATPR
2. TPR(2 9
> T for g E (P,U).
guaranteed that T.H [ Note
3. TPR(2) (2) @))())
Our results apply when the same threshold can be used to achieve the No False Positives Objective across both schemes. We show that when the Hirer chooses LLM that are stochastically dominated by the privileged group's LLM, then this choice is sufficient (though not necessary) to ensure that the optimal threshold remains the same across hiring schemes.
Since the threshold 7' already prevents false positives (Defi- nition 4.2), (3) also implies that accuracy does not decrease.
6.3. The n- n-Ticket Scheme and Group Dependence Bias Mitigation
While the it may not be sufficient since the privileged group still has the advantage of a "better" first ticket, increaseing their acceptance probability. We therefore propose generalizing the idea to an n- n-Ticket Hiring Scheme. Let L represent the application of the two-ticket scheme n E NN times using I.I. LH (the n-ticket scheme). That is, for any x E R...
scheme helps mitigate disparities,
Lemma 6.1. Lp 2 1 L then (1) = (2))
'Traditional hiring can be analogously thought of as "One- Ticket" hiring, as the candidate's submitted resume is their only avenue to acceptance.
7

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Step 3 in Definition 6.1 is repeated n times, each time after the first with æ" the submitted resume. We will show that by applying the n-ticket scheme, the outcome becomes independent of group membership.
Corollary 3. If L ł Lp = Ly, then for every unmodified
resume E IRd, there exists hin (0) that depends on LH and æ such that for any n 2 2, (Lo()))))))) + (L - O(k)). Hence,
We start by defining a contraction operator and stating Ba- nach's Fixed Point Theorem, which will be useful in the proof of the main theorem in this section.
1. A (n)
TPR - O(K)). where k max, hig.
2. TPR (m)
Σ T (2-1) for g E (P.U) and n > 1.
Definition 6.2. Let (Z, d) be a metric space. A function T: 2 + Z is a contraction operator if for some &t E 1),
9
9
3. TPR( (n) > TPR (--1) for every n > 1.
d(T(z),T(z))) S 2(2,22) for all 2,5' e E Z.
Since the threshold 7" already prevents false positives (Defi- nition 4.2), (3) also implies that accuracy does not decrease.
Theorem 3 (Banach's Fixed Point (Banach, 1922)). Let
T: : Z Z be contraction operator. Then,
7. Empirical Validation: Resume Selection in the Technology Sector
The equation T(z) = 2 has a unique solution 2" E Z.
For any 2u E Z. lim (20) = 2-. Furthermore,
In this section, we empirically validate our theoretical re- sults by closely simulating a hiring scenario in which an employer has two positions to fill. We examined 520 re- sumes from the Djjini dataset (Drushchak & Romanyshyn, 2024), which includes resumes from the technology sec- tor. Our sample consisted of equal parts UI/UX designers and project managers (PM). To replicate real-world ap- plicant tracking system, we used an open-source resume scorer, Resume Matcher, to assign a relevance score (e.g., 0-100) for all resumes against a PM job description and UX job description. We note that Resume-Matchie as- signs its scores based word-similarity metries between the inputted resumes and target job descriptions.
n n0000
(2n) - 21 < Okk)), where k is the contraction coefficient.
Before we state the main theorem, we note that the n-ticket scheme is defined in such way that, once an applicant has at least one (possibly LLM-manipulated) resume that receives a score above the threshold r. then they are guaranteed to be accepted: this arises from the fact that the Hirer considers the best-scoring of the applicant's n "tickets". In the main theorem, show that for an infinite number of tickets, the hiring outcome becomes independent of group membership.
Theorem 4. Let be the threshold used by the Hirer in the n-ticket scheme. If LH - Lp ž LU, applying the n-ticket scheme and taking the limit as - 20, then any applicant
We examine the GPT family of OpenAl models due to its widespread use." 8 WW randomly assigned half of the resumes to group P (privileged) and the remaining half to group U (unprivileged). Only the candidates assigned to group could manipulate their resumes with the same model the employer (GPT-40). The resumes in group U could only access GPT-3.5-TURBO, GPT-4O-MINI to edit the original resumes based on resume improve- ment prompt. 9 Our theoretical results assume candidates to be best-responding, hence in our experiments, the candi- dates would submit the higher scoring resume between their choice of their LLM manipulated and original resume.
E D is guaranteed to receive a group-independerr out- come, o = o(x, I.I1) E (0. 1). Furthermore, there exists k E [0, 1). dependent on Lø and æ, such that
no LLM at all
[P(f
1)
of ol = O(K))
To prove the theorem, we show that providing an additional ticket for an applicant x is contraction operator on [0, I], independent of group membership, and show the existence of a Banach fixed point (See Appendix F for full proof).
The Hirer learns threshold that maximizes the true positive rate while minimizing the false positive rate (this objective
The theorem implies that by using the n-ticket scheme with an LLM as least as strong as the privileged group. the Hirer can significantly reduce any group-dependency bias in the hiring scheme, and the probability of not receiving the right outcome for the applicant drops exponentially in the number of tickets. As a corollary, the probability of disparity in outcomes of candidates with the same feature vector but different groups and the TPR disparity drop exponentially with n.
'This open-source source resume scorer is designed to mimic appli- cant tracking systems that many hiring companies use for ranking applicant relevance (Purcell, 2024). Job applicants can use these ATS tools improve their resume relevance. To the hest of our knowledge, ResumeMatcher is the only open-source ATS tool available (https: /apps://rruummmee che... fyil).
"ChatGPT is reported to have 2.4 billion monthly visits in March 2024, 10 times the next most popular platform (Zhu, 2024). "Appendix D includes prompts, model versions, and costs.
8

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
PM Role
UX Designer Role
Condition
Method
TPR
TPR Disparity
TPR
TPR Disparity
U: No LLMs, P:GPT-40
Traditional
0.11 + 0.004
0.10 + 0.005
0.22 0.008
0.27 + 0.006 0.01 + 0.007
Two-Ticket
0.14 + 0.005
0.05 + 0.005
0.38 + 0.008
U: P:GPT-40
Traditional
0.09 + 0.004
0.09 1 0.005
0.26 + 0.010
0.15 + 0.008
Two-Ticket
0.11 + 0.004
0.08 + 0.005
0.30 = 0.010
0.08 + 0.007
U: GPT-40-MINI, P:GPT-444 P:GPT-40
Traditional
0.12 + 0.004
0.04 + 0.005
0.33 + 0.010
0.00 + 0.007
Two-Ticket
0.13 + 0.007
0.03 + 0.010
0.36 + 0.010
0.01 + 0.008
Table 1. Resume screening results where Groups P and U have access various models of GPT family models for PM and Design Job description respectively. Results are presented with 95% CIs computed over 500 train-test splits.
approximates the objective of no false positives in Defini- tion 4.2). For the TRADITIONAL hiring scheme, the Hirer directly uses the "submittedd" resumes from the two groups For the TWO-TICKET scheme, the Hirer also manipulates each submitted resume with the employer model (GPT-40). The Hirer then acts on the higher-scoring resume between the submitted and Hirer- manipulated versions of the resume. In both schemes, the Hirer has no knowledge about which individuals belong to which group; thus, membership-bee faimess interventions cannot be applied to our setting.
with LLMs.
In our model, the Hirer does not know a priori whether a candidate has manipulated their resume. Our theoretical results imply that using the TWO-TICKET scheme, both the TPR and the TPR disparity are improved even without this knowledge. Specifically, our theoretical results suggest that this improvement is greatest when the Hirer deploys an LLM that is as strong as possible, while being weaker than the candidate's 's strongest LLM. Therefore, careful thought and evaluation must be used when applying our TWO-TICKET scheme in practice. Although our findings focus on manipu- lations that preserve the distinguishability between negative and positive labels within our candidate screening task, fu- ture work should investigate the full spectrum of choice of LLM. This is particularly impactful when companies may introduce increasingly premium LLM services.
Table shows the empirical verification of our theoretical results, comparing the performance of the TRADITIONAL. hiring scheme and the TWO-TICKET hiring scheme vali- dated for both job descriptions over 520 resumes each. True positive rates were improved and TPR disparities were re- duced by the TWO-TICKET scheme. The improvement was strongest when group U used a weaker modification (i.e., GPT-3,5-TURBO or no LLM). When group U used a similar level LLM (e.g.. EPT-40-MINI) there was no improvement in the TPR disparity. Our results demonstrate that our proposed scheme helps better discern qualified candidates from candidates using stronger LLM manipulations !I
We assume that companies use LLM APIs such that queries are not stored by the company, to preserve candidate's 's data privacy. We do not anticipate the resume manipulation itself to be any less private than the companies storing a candi- date's resume. A differentially private threshold function can be used to ensure differential privacy (Bun et al., 2024).
8. Discussion
As our work focused on simple prompts to capture low- effort (zero-cost) manipulation. future work should address the variable behavior of LLMs that can arise from using dif- ferent or more prescriptive prompts. Although we provide a theoretical guarantee for improvements in our TWO-TICKET scheme, relaxing the condition that the optimal threshold does not change could help establish stronger guarantees. Our experiments use the only open source ATS system avail- able; future audits of actual hiring systems should test black- box and comman-in-the-oee systems.
Our work a first step toward understanding and designing better selection algorithms under stochastic LLM manip- ulations. Similarly to prior work (Hu et al., 2019a), we show that members of the privileged group more easily admitted or hired. Here, privilege includes both access to more advanced LLMS and knowledge of the performance of different LLMs. While we focus on the setting of hiring, our TWO-TICKET scheme can be generalized to many other scenarios in which candidates can manipulate their materials
At the time of our submission, GPT-40-MINI is offered for free by OpenAl. However, hefore GPT-40-MINI was released (after GPT-40 was released), GPT-3.5-TURBA was the free model offered. In the future, LLM providers may offer new versions of models where the paid version much better than the free version.
Impact Statement
This paper is a theoretical work that examines the effect of an increasing number of job applications being produced by generative AI. Notably, we do not advocate for hiring sys- tems using AI but study the problem of candidates using AI to modify their resumes. Rather, we developed this project
"ur code and data is available at https: //gi thub. com/ heyy juces / 1mm hi hiring- ecosystem
9

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
address the potential downstream impacts of generative
functions, 2024, URL htL.ttsss///axii.orrg
AI.
Nonetheless, the subsequent hiring decisions made by hu- mans may introduce further disparities that may affect em- ployee satisfaction in the long run. In future work, we plan to do an empirical study of the impact LLM-aided appli- cation materials across variety of industries in consultation with economists.
Business Wire. Designing your dream job: Canva study reveals ai and visuals are trans- forming the job hunt, January 2025.
URL
home hotm//20220111566600///aeesappgp
AI ar-and-iisslla -reeelss--r--eee are.aaas--ae--rraaffooommmmi Job-Hunt.
At the moment, we also do not foresee negative conse- quences to our analysis at this time. Our work by means is comprehensive in studying the allocation of opportuni- ties in the era of generative AI. We hope that future works continue to examine this area of sociotechnical safety.
Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tramèr, F,, and Zhang. C. Quantifying memorization across neural language models. In The Eleventh International Confer- ence on Learning Representations, ICLR, 2023.
Acknowledgments
Chamorro-Premann T. An in-class AI exercise to help your students get hired, Apr 2024. URL http://bss.haaaaa..eeddd////iaaa mitudh/ann--n-cclassa--eereessse help-yyou--studeets--ttt-t
The authors thank Omer Reingold for his helpful discussion. The authors are supported by the Simons Foundation Col- laboration on the Theory of Algorithmic Fairness, the Sloan Foundation Grant 139011 and the Simons Foundation investigators award 689988. Connie and Jack are grateful participants of the CURIS program that facilitated matching undergraduates to this project.
Chen, Y. Jang. and Liu, Y. Strategic recourse in linear
classification. arXiv preprint arXiv:2011.00355 236, 2020.
City of New York. Local law 144, 2021.
References
https: http://ww..cccco....ooooooo nyc. www.ggg///iiiee wttp:///s////eeeeeeeeeeeeeeeeees /ca aaca///ut/////
Ahmadi, S., Beyhaghi, H., Blum, A., and Naggita, K. On classification of strategic agents who can both game and improve. In Celis, L. E. (ed.), 3rd Symposium on Founda- tions of Responsible Computing, FORC 2022, June 6-8, 2022, Cambridge, MA, USA, 2022.
tools, cools.page.
Cohen, L., Lipton, Z. C.,, and Mansour, Y. Efficient candi- date screening under multiple tests and implications for fairness. In 1st Symposium on Foundations of Responsible Computing (FORC 2020), 2020.
Banach, S. Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fundamenta Mathematicae, 1922.
Cohen, L., Malvajerdi, S. S., Stangl, K., Vakilian, A., and Ziani, Bayesian strategic classification. In The Thirty- eighth Annual Conference on Neural Information Pro- cessing Systems, 2024. URL https: //pp:://////eeiiwww/ net
Bertrand, M. and Mullainathan, S.
Are Emily and Greg more employable than Lakisha and Jamal? a field experi- ment on labor market discrimination. American economic review, 2004.
Dastin, J. Insight Amazon scraps secret Al recruit- ing tool that showed bias against women, 2018. URL wo wottl//aasgggt--mmmmooo-caaaaeee a ai-recruttinng--oo---ttt----eeee ueadatt--omme---ddsscLLLL-L-AA Accsss September 25, 2024.
Blum, A.. Stangl, K., and Vakilian, A. Multi stage screening: Enforcing fairness and maximizing efficiency in pre- existing pipeline. In Proceedings of the 2022 ACM Con- ference on Fairness, Accountubility, and Transparency, FAceT '22. Association for Computing Machinery, 2022.
Accessed on
Diana, F., Sharifi-Malvajeddi, S., and Vakilian, A. Minimax group faimess in strategic classification. arXiv preprint arXiv:2410.02513, 2024.
Braverman, M. and Garg, S. The role of randomness and noise in strategic classification. In 1st Symposium on Foundations of Responsible Computing, FORC 2020, 2020.
DoorDash Inc. n.d.
Project manager (multiple levels),
URL https: http:::://caeeeraaaodaaaa
Bun. M., Nissim, K., Stemmer, U., and Vadhan, S. Dif- ferentially private release and learning of threshold
levels/5523275/
Accessed: 2024-10-09.
10

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Drushchak, N. and Romanyshyn, M. Introducing the Djinni recruitment dataset: A corpus of anonymized CVs and job postings. In Romanyshyn, M., Romanyshyn, N., Hlybovets, A.. and Ignatenko, O. (eds.), Proceedings of the Third Ukrainian Natural Language Processing Workshop (UNLP) LREC-COLING 2024, 2024.
Levanon, and Rosenfeld, N. Strategic classification made practical. In International Conference Machine Learn- ing, pp. 6243-6253, PMLR, 2021.
Levhari, D., Paroush, J., and Peleg, B. Efficiency analysis for multivariate distributions. The Review of Economic Studies, 1975.
Ghalme, G., Nair, V., Eilat, L,, Talaam-Cohess I., and Rosen- feld, N. Strategic classification the dark. Proceed- ings of the 38th International Conference on Machine Learning, ICML,, 2021.
Lin, Y.-T., Hung, T.W.., and Huang, L. T.-L. Engineering equity: How Al can help reduce the harm of implicit bias. Philosophy Technology, 2021.
Google. URL /ttps::/www..ooo/ coxrerrsapppllicccton/////oe
Staff ux designer @ google, n.d. cttp://e..co//ooooo/o about
Lytton, C. AI hiring tools may be filtering out the best job applicants. BBC, 2024. URL https::/wwwe bbc. ccn/workkl 12200244aii-recruutiingg wtw.taare-baas-dissiiiiieeeeeeee
129448852775150277 staff-tx-ff--e-ee------
Accessed: 2024-10-09.
Hardt, M., Megiddo, N., Papadimitriou, C. H., and Wootters, M. Strategic classification. In Proceedings of the 2016 ACM Conference Innovations in Theoretical Computer Science (ITCS), 2016a.
Milli, S., Miller, J., Dragan, A. D., and Hardt, M. The social cost of strategic classification. In Proceedings of the Con- ference on Fairness, Accountability, and Transparency, pp. 230-2330022 2019.
Hardt, M., Price, E.,, and Srebro, N. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016b.
Purcell, K. (ats) usage report: Key shifts and strate- gies for job seekers, November 2024. URL https://wwes htwp:::.oobccr. jobscan. wttbblog/oootuuaaaee/eeeeeeoeeae
2024 applicant tracking system
Hu, L. and Chen, Y. A short-term intervention for long-tent fairness in the labor market. In Proceedings of the 2018 World Wide Web Conference, pp. 1389-1398, 2018.
Raghavan, M.. Barocas, S., Kleinberg. J., and Levy, K.
Hu, L., Immorlica, N., and Vaughan, J. W. The disparate effects of strategic manipulation. FAT '19, 2019a.
Mitigating bias in algorithmic hiring: Evaluating claims and practices. In Proceedings the 2020 conference on fairness, accountability, and transparency, pp. 469-481, 2020.
Hu, L.., Immorlica, N., and Vaughan, J. W. The disparate ef- fects of strategic manipulation. In Proceedings of the Con- ference on Fairness, Accountability, and Transparency, FAT '19, 2019b.
Shao, H., Blum, A., and Montasser, O. Strategic classi- fication under unknown personalized manipulation. In Teirty-seventh Conference on Neural Information Pro- cessing Systems, 2023. URL htttp:://ppeereeeeeeee net/forum?ide ftt.foocuumm.d6....cxx.....
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng. W., Feng, X., Qin, B., et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.052.22 2023.
6cJKcIxPck.
Somerstep, S., Sun, Y., and Ritov, Y. Learning in reverse causal strategic environments with ramifications on two sided markets, 2024. URL https://azxiv..rg/ abs/ 2tt./240..12244
Hunkenschroer, A. L. and Luetge, C. Ethics of Al-enabled recruiting and selection: review and research agenda. Journal of Business Ethics, 178, 2022.
Stahl, A. 5 resume hacks pass ATS, 2022. URL https://www. forbes com/ site//tee/////// ashleystahl12202212222 resume-
Keswani, V. and Celis, L. E. Addressing strategic manipula- tion disparities in fair classification. EAAMO, 2023.
Kleinberg, J. and Raghavan, M. How do classifiers induce agents to invest effort strategically? ACM Transactions on Economics and Computation (TEAC), 2020.
Accessed on Septem-
ber 25, 2024.
van den Brock, E., Sergeeva, A., and Huysman, M. Hiring algorithms: An ethnography of faimess in practice. In 40th international conference on information systems, ICIS 2019. Association for Information Systems, 2020.
Kline, P., Rose, E. K., and Walters, C. R. Systemic dis- crimination among large US employers. The Quarterly Journal of Economics, 2022.

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Verma, P. and Renjarla, R. Can ChatGPT write college ad- mission essay? We tested it Washington Post, Jan 2024. URL https://wwer www.wwwttttoooootttttttt.o.oo com/
Zhu, K. Ranked: The most popular generative AI
tools in 2024, 10 2024. URL https://www.
visualcapitaliste con/rankee-tthtttott
12

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
PM Role
UX Designer Role
Condition
Method
Accuracy
Accuracy
U:No LLMs, P:GPT-40
Traditional
0.548 + 0.003
0.633 + 0.005
Two-Ticket
0.563 + 0.004
0.689 + 0.005
P:GPTT-4
Traditional
0.543 + 0.003
0.629 + 0.006
Two-Ticket
0.551 + 0.003
0.650 + 0.006
U:GPT-40-MINI, P:GPTT-0
Traditional
0.554 + 0.003
0.662 + 0.005
Two-Ticket
0.561 + 0.003
0.677 + 0.005
Table Accuracy results for experiments conducted Table see improvements accuracy 95% confidence.
A. Full Related Work
Fairness in Algorithmic Hiring Audits of hiring systems have consistently found discrimination in outcomes based on race, gender, and age (Bertrand & Mullainathan, 2004; Kline al., 2022). Raghavan al. (2020) study the screening stage of the hiring algorithms and connect legal perspectives with algorithmic approaches mitigate the disparate impact. Mitigating biases has also been described as crucial to the "ethicality of the Al tool design" (Hunkenschroer & Luetge, 2022). In terms of proposed solutions, (Lin et al., 2021) suggest "augmentation basedd interventions where Al-assisted decisions can best achieve equitable outcomes. A key assumption of prior work is access to (explicit or inferred) group membership. In our work, the hiring side has no knowledge of the group membership of candidates, yet we can mitigate bias nevertheless.
Strategic Classification Hardt al. (2016a) introduce strategic classification
Stackelberg game address the impact of manipulative tacties classification problems. We draw several later works that provided modified strategic classification game that models disparities in manipulation abilities (Hu et al., 2019b; Milli et al., 2019; Chen et al., 2020; Diana et al., 2024). Furthermore, we use techniques from Braverman & Garg (2020) to describe "random" classifiers in light of stochastic strategic manipulations. Similarly to several previous works (Ghalme et al., 2021; Cohen et al., 2024), we assume that the deployed classifier is unknown to the candidates. Finally, previous work of (Somerstep et al., 2024) has explored the use of casual strategic classification to also understand labor market dynamics: our work extends this by highlighting the effects of strategic behavior on employer and labor force outcomes.
Behavior and Risks of Generative Models Guidance counselors and career coaches alike now recommend using generative AI tools to help with application materials (Verma & Renjarla, 2024; Shamorro-Premuzeee 2024). However, recent research has highlighted significant risks. For example, LLMs have been shown to hallucinate, which may mislead employers (Huang et al., 2023), or memorize text, which can result in unintended plagiarism (Carlini et al., 2023). As unintended plagiarism difficult for job applicants to detect using these tools, the benefits of applying LLMs to application materials may be stochastic.
B. Additional Results
Figure 3 illustrates that using simple job-agnostic prompt with an input resume significantly improves the scores computed by a resume screening system. Qualitatively, we observed a drastic improvement in writing quality (examples available in Appendix C.4); the L.LMs were able to transform resumes mostly containing bullet points about the candidate's 's interests or skills into more effective, reworded resumes delineating prior roles. However, scores did not improve monotonically across resumes; some resume scores decreased after applying LLM manipulation (Figure 4a).
B.I. Accuracy
In addition to observed improvements in TPR and TPR Disparity values from our experiments in Section 6.3, our experiments also yielded improvements in accuracy (Table 2).
B.2. Thresholds
The main results of Section 7 highlight the validity of our theoretical findings: namely, we show that a TWO-TICKET scheme can both improve a classifier's 's true positive rate (TPR) and reduce disparities in TPRs between privileged and
13

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Resume Scores Under Applicent Manipulations (DoorDash Product Manager)
Resume Scores Under Manipultions (Googic UX Designer)
No Manipulation
Maripulation
Cauce-33 S-Sonnet
Claude 3.5 S--nonet
Applicant Moripulation Model
Applica nt Moripulettin Model
DeepSeok-67B
DespSeok-67B
DeepSeek-VS
SeekVV
GPT-3.5-Turbo
GPr-3.5-Turbo
GPT4omiii
GPT4o-mir
Gamas-70B-instruct Turbo
1amm3-708-irttuc--iie
Mostral-Bx7B-innsttue
Mixtral-Bx Bx--instuut
70
75
80
85
70
75
85
90
96
Score
Score
Qualified ledi
Unqualfied
Qualified p50
Qusifed
Unqualfied
Qualified p50
(a) DoorDash Product Manager Job Posting
(b) Google UX Designer Job Posting
Figure 3. Resume score distribution of 50 qualified (matching occupation) and 50 unqualified (different occupation) resumes before and
after LLM manipulations for two job descriptions. Models such GPT-40 and CLAUDE-3.5-SONNE and DEEPSEEK-V -3 generate distribution of unqualified resumes that is indistinguishable from qualified resumes without manipulation for the Product Manager job and significantly enhance the scores of the qualified resumes for the UX Designer position.
unprivileged groups when group membership is unknown to the classifier. In this section, we present further empirical findings to support our modeling assumptions. In particular, we validate the constancy of our No False Positives Objective thresholds, in accordance with Lemma 6. Since attempting to set the false positive rate (FPR) exactly to zero occasionally led to an empirically undefined TPR, we chose the threshold corresponding to the smallest FPR greater than zero to produce our main results in Section While this approach does not strictly adhere to our False Positives Objective, the resulting FPRs on the training set are still sufficiently close to zero, as seen in Table Table presents the FPRs incurred by the thresholds chosen for our experiments in Section 7. Overall, these results confirm that experimental approximation of the No False Positives Objective is sufficiently similar.
Method
False Positive Rate at 95% CI
Traditional
0.0040 = 0.000260
Two-Ticket
0.0014 + 0.000232
Table 3. False Positive Rates Training Sets for PM Role. Results are presented with 95% CIs computed over 500 train-test splits.
Tables 4 and 5 summarize experimental results comparing the thresholds of our TRADITIONAL and TWO-TICKET schemes when both the Hirer and the user employ the same LLM, GPT-40-MINI (i.e., Lu Læ). While there is slight, non-significant difference in thresholds between the one-ticket and two-ticket schemes in our main results (Table 2). introducing more randomness - by varying both the train-test splits and the privileeed-upprivilegee group assignments in each trial led to more similar thresholds between the two schemes. This brought the results closer to satisfying the criteria outlined in Lemma 6.1. contrast, the experiments Section used fixed seriileedduupprvvileeee group assignments, and thresholds were measured across range of fixed train-test splits.
We hypothesize that the slight difference between our assumptions from Lemma 6.1 and our empirical findings arise as result of the random nature of the privileged-unprriilegge group assignments and slight modification of the No False Positives Objective. Specifically, under our approximation of the No False Positives Objective, we find that our output threshold is determined by the top scoring UI/UX resumes (irrelevant of whether they have been manipulated or not). The fixed assignment of these "top" scoring UI/UX resumes to a "non" " -manipulating group would hence result in a different threshold across the TRADITIONAL. and TWO-TiCKET scheme.
14

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Stochaslic Mapping Resume Relevance Scores (DoorDash Product Manager)
Recure Sooms Liderr Variuatiore
(Doo/0s97)
True Label Qua Hidd
True Label
Aspiraan Mariza Moceel
62
Quali
Orepdeok-775
DecpSoch-5r
78
74
74
70
80
82
84
74
75
02
84
72.5 758 775 810 825
350 87.5
Maniputation
No Manipulation
Reaume Scor
(a) Resume Score Improvement after applying LLM manipulation with different models.
Resume Scores Sequential LLM Manip- ulations
Figure 4. (a) Applying LLM manipulations to resumes result in stochastic outcomes: even when the average score increases (e.g.. DEEPSEEK-V V3), some resumes receive lower scores after manipulation. (b) Repeatedly using LLMs to enhance resume results stagnating improvements.
Method
Threshold at 95% CI
Traditional
84.3 0.040
Two-Ticket
86.1 0.077
Difference
1.78 + 0.086
Table 4. Threshold Values for PM Role for Fixed Group and Train-Teas Assignments
Method
Threshold at 95% CI
Traditional
85.3 + 0.05
Two-Ticket
85.8 + 0.33
Difference
0.46 + 0.06
Table 5. Threshold Score Values for PM Role for Randomized Group and Train- Test Assignments
We motivate this with simple example. Consider scenario with training set of four applicants applying for PM Role. In this example, two of the applicants are UI/UX applicants who have been assigned to the unprivileged group, whereas the remaining PM applicants have been assigned to the privileged (who will manipulate their resume) group. Suppose our Hirer receives the following distribution of resume scores and their prior role, which must optimize over under the TRADITIONAL. scheme:
15

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Resume Score Inputs for Traditional Scheme
(UN)0)
Appicant2
Applicant
English 0.1
English 0.2
Englas 0.3
Eng ish Beard: Menugement Boo 04 Torat Scome 0.7
Me regineent Score 0.1
Management Scora 0.2
Managemeri Scors, 0.5
"Irta Scoee
"notl Stoww 0.4
Scare: 3.8
Resume Score Inputs for TWDD- Two-Ticket Schemn
Applicant 1 (UUUU))
Applicant (P1))
Applicant (PM)
English Scoro: 0.5
Er Ergiin Score: 0.3
Engiit Soore: 0.3
Management
Manegemant Toka: Soora, Di8
Management
Tolal Soure 0.4
Soore: 0.7
Apali (M/U))
Engliihh Score: 0.3
Potental Range "No Postlives
Threshold
Mensgement Scors:
Range Apposinsiie Revalisuee
Figure 5. Example of Privileged Unpiiiiilee-niiiieeeeedd Assignment in the Training Set Resulting in Differing Thresholds Between the TRADI- TIONAL and TWO-TICKET Schemes.
Now, let us contrast this with the Hirer-manipulated resume scores over which the TWO-TICKET scheme optimizes. In this scenario, we assume that all English skills are mapped to a random variable with an expected value of 0.3 and zero standard deviation (for simplicity). Under the TWO-TICKET scheme, the scores of the UI/UX applicants "improve" as they have not previously modified their resumes: however, the scores of the PMS remain constant, as they have already modified their resumes. As shown, the resume scores that the Hirer optimizes over differ between the TRADITIONAL and TWO-TICKET schemes. This leads different ranges of possible "No False Positive" thresholds for each scheme. On the contrary, when strictly adhering to "No False Positive" objective, To further support this hypothesis empirically, Table 6 displays the maximum negative resume scores which the classifier optimizes over, regardless of whether the resumes were manipulated. As the scores fluctuate across trials and do not have a zero standard deviation, it is clear that these values are changing. In summary, the discrepancy between our "Constant Threshold" and the empirical results can be attributed to small differences between our theoretical constraints and actual speciics--eei,iicallly approximating the "No False Positives" objective. While our analysis helps explain some of these discrepancies, we note that the magnitude of the differences is small enough that it does not significantly affect the experimental conclusions drawn in Table 1.
observe that the change in the threshold is smaller.
Method
Maximum UI/UX Resume Score
Two-Ticket Scheme Traditional Scheme Difference
86.7 + 3.6 x 10-11
85.90 + 0.07
0.75 + 0.07
Table 6. Maximum UI/UX Resume Scores for PM Role
C. Empirical Motivation:
Additional Results
C.1. Additional Jobs
We include scores across different models for additional job descriptions. In Figure 6 shows three additional PM jobs and how LLM manipulations to resumes affect the job relevance scores. For PM jobs, some job descriptions generated similar median values for both the qualified and unqualified groups (e.g., Apple Product Manager and Yelp Product Manager). For these jobs, we observed significant boosts in relevance scores for both groups by newer, premium language models.
We also include three additional job descriptions for UX Designers positions (Figure 7). For UX Designers, there's 's specific set of skills required that always separates the qualified (UX) from the unqualified (PM) resumes. However, the improvement
16

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Reuun Scone, aderleokiont Moriguitiions
Rorume Sconel Lidier Appliceen Unsipuidiinn.
Raurs Scorer, inceriedlian Morigs infions
Olabodt foora
Cla Siomed
Tha
Despdine va
DespReat VA
GP74a
GFT4
Janak 100 Ture
Lanab- canualllt
Rasurn Scors
Resume Scord
(a) Samsara PM
(b) Apple PM
(c) Yelp PM
Figure 6. Resume score distribution of 50 qualified (matching occupation) and 50 unqualified (different occupation) resumes before and after LLM manipulations for more Product Manager Job Descriptions
JAdoas LDC Dasignes (nar)
pinssonn Deaignar)
(12 Deegeee
No Memipaatioo
Herpo
Mesppuation
Clude-12-Sorma
Claceed Sdormi
OPTOU/TTur
geria STate
netto
Rewiame Sorm
Rio Scars
Zaturs Scorn
Coalieed
Unggatter
Ouilled
Uirosiiled
Guillied 250
Duillle
Ungall Bee
Qusllled
(a) Adobe UX
(b) Amazon UX
(c) 12C UX
Figure 7. Resume score distribution 50 qualified (matching occupation) and 50 unqualified (different occupation) resumes before and after LLM manipulations for more UX Designer Job Descriptions
in scores is particularly stark for the qualified group which would introduce additional disparitics.
C.2. Homogenization: Model and Resume Similarity
While not directly applicable to our theoretical model, many recent works have studied the homogenization of language model outputs. Here we example how similar resumes are before and after LLM manipulation. We observe that experimentally that almost all models increase the similarities of resumes. This suggests that as more applicants turn to L.M tools, their resumes are growing more homogeneous the employer.
We also examine similarity with the application of the relevance score system; do the same models find the same candidates good? To measure this, we find the correlation between 100 resume scores for each job description and plot a heat map between models (Figure 9). We observe larger correlation between models for UX designer positions than for PM positions. This is likely because there are specific skills unique to UX designers that the unqualified resumes (PM resumes) do not have. We also see models from the same family (c.g. DEEPSEEK-677 66B and DEEPSEEK- V3, GPT40 and GPT40-MINI with higher correlation. This suggests models from the same family may modify resumes similar way.
C.3. LLM Modifications
key motivation for our work comes from our observations that LLMs can improve the written of quality of resumes, though at varying levels. In addition to Section 3, we here examine the exact disparities and rates of resume improvement.
17

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Distribution of Resume Similarities (Quantiles)
No Manipulation Median
1.0
0.8
Cosine Similarity
0.4
0.0
No Manipulation
Claude 5-Sonnel G.-oo-3.
DeepSeek-678 678
DeepSeek-V2 V3
GPT-3 5- Turbo
GPT40-mini
Llama3-700 700-----t-nn---------- les---nsstuut-ttt Turbo
GPT4o
Turbbo
Mixtral-Bx7B Setra--x7B--nstruun Instruct
Model
Figure 8. Distribution of cosine similarities (Sentence Embeddings AL1-MINNT across all pairs of resumes. Compare to no manipulation, nearly all models increase the similarity of resumes, especially CLAUUDE-3--SONNET
DoorDash Product Marager Resume Score
Correittion Between Mode's
Google UX Designer Resume Scom Correlation Detween Models
No Manipuiationn
1.00
0.73
0.79
0.86
0:83
0:85
6.82
0.87
0.82
No Maniuaation
1.00
0.s6
0087
0.00
0.56
0.00
0.54
0186
1.00
Claude 8-Sommet
0.73
1:00
0.78
0.78
0.75
0.95
Chude-3 3.5-Sonnet
1.00
9.00
0.82
0.90
0.90
0.95
DoepStek
0.92
V3
0:07
0.81
0.90
DeesSesk V3
0.53
0.52
0.09
0.50
QPT-..-Tubb
1.83
0.78
0.77
0.82
1.00
0.80
0.74
0.75
0.74
0.85
GPT-3.5-Turee Turbt
0.89
0.84
0:86
0.89
1.00
0.87
0.57
0.86
0.85
0.95
CPT4o-mma
1.85
0.79
0.81
0.57
0.30
1.00
0.83
0.86
0.77
GPT4a-mm
0.88
0.52
0:08
0.86
1:00
0.98
0.02
0.88
GPTAA
GPT4e
0.52
0.84
1.00
0.31
0.755
8.02
0.54
0.67
0.74
0.77
0.75
3.80
0.85
0.20
1.00
-0.70
0.70
Marpulkrion
Chastee S00rm
DeopBeck-V
OPTR
Lawnc- Same--TBssneee (iSineisct-- furtss
Mampuutzion
Lamod- TOE re-imsttuctutte Turto
(a) DoorDash Product Manager
(b) Google UX
Figure 9, Pearson correlation between model scores of resumes. UX designer job descriptions had higher agreement between models.
18

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
We found that more resumes improved by CHATGPT-40 opposed MIXTRAL-8X7B. 75% of our 520 tested resumes experienced increases in resume scores due CHATGPT-40 modifications, while only half of the 520 resumes experienced increases in resume scores when modified by MIXTRAL-8X7B.
C.4. Qualitative Analysis: Best- Rest-Responding Candidates
As noted in the main body of our paper, theoretical and experimental findings assume that job candidates are "best- responding" and can choose submit the higher scoring resume between their submitted and modified LLM version of their resume. In reality, however, we note that candidates don't t have access to the score system deployed by Hirer firm. Nevertheless, it is relatively easy, if not fast, to distinguish between the lower and higher-scoring resumes by hand. For example, consider the following snippets from two resumes the first resume is an unmodified resume, with a score of 79.434, and the second is modified resume, with an improved score 81.882.
The "professional summary" section of one unmodified resumes is
Have skills in creating a prototype and choosing the methodology for leading the project. I have experience in creating BPMN diagrams. Also, have worked with different PM tools and can highlight ClickUp as my favorite one.
the contrary, the "professional summary" section of the corresponding manipulated resumes
Skilled Project Manager with expertise in document management for various project stages, creating prototypes, and selecting appropriate project methodologies. Hands-on experience with BPMN diagrams and adept in utilizing different project management tools, with a preference for ClickUp.
The manipulated resume clearly exhibits more complete sentences and sophisticated vocabulary, indicating a perhaps better-scoring resume. For reference, the complete resumes are shown below.
Unmodified Resume (PM Score: 79.434)
Have skills in maintaining documents for the stages of the project: Brief, Assumptions, WBS (WBS scheme, TTM, Calendar plan), Project Charter, MVP, Requirement Matrix, Risk Register, SRS, RACI, and Status Report.
Have skills creating a prototype and choosing the methodology for leading the project. have experience in creating BPMN diagrams. Also, have worked with different PM tools and can highlight ClickUp as my favorite
Coordinator of Club of Tolerance
The Congress of National Communities of Ukraine - 2017-2019
held weekly meetings with teenagers, where we discussed various topies and visited exhibitions. I created illustrations and presentations in Canva and Crello.
Trainer of Non-Formal Education at EuroCamp and Sources of Tolerance The Congress of National Communities of Ukraine - 2017-2020
worked as a trainer with children aged 1 11-17 in an educational training course for teenagers. created teams from teenagers who did not know each other and transformed them into one family using cam--uulding exercises, progressing through all key stages of team development.
Education
IT School Beetroot Academy Project Management in IT 11/2020 112020-03/2021
Professional Experience Project Manager in IT Epic Story - April 2021 Present
Modified GPT-40 Resume Text (Not Formatted, PM Score 81.882)
19

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Professional Summary
Skilled Project Manager with expertise in document management for various project stages, creating prototypes, and selecting appropriate project methodologies. Hands-on experience with BPMN diagrams and adept in utilizing different project management tools, with preference for ClickUp.
Professional Experience Project Manager in IT
Epic Story - April 2021 Present
Maintain comprehensive documentation across multiple stages of the project lifecycle, including Brief, Assump- tions, WBS (WBS scheme, TTM, Calendar plan), Project Charter, MVP, Requirement Matrix, Risk Register, SRS, RACI, and Status Reports. Create project prototypes and determine the most effective project management methodologies. Design and implement BPMN diagrams to streamline project workflows. Utilize various PM tools for efficient project execution, with specific focus on ClickUp.
Trainer of Non-Formal Education
EuroCamp and Sources of Tolerance, The Congress of National Communities of Ukraine - 2017-2020 Conducted educational training courses for children aged 11-17, focusing on building cohesive teams from diverse groups of teenagers. Facilitated team development through team cam-building activities, ensuring each group progressed through all key stages of team growth.
Coordinator of Club of Tolerance
The Congress of National Communities of Ukraine 2017-2019
Organized and led weekly meetings with teenagers, engaging them in discussions various topics and organizing visits to exhibitions. Created visual content and presentations using Canva and Crello to enhance meeting engagement and learning.
Education
IT School Beetroot Academy Certification in Project Management in 11/2020 03/2021
Key Skills
Document Management
Project Prototyping
Project Methodology Selection
BPMN Diagram Creation
ClickUp Proficiency
Team Building & Development
Non-Formal Education Training
Visual Content Creation (Canva, Crello)
In sum, as the lower and higher-scoring resumes were distinguishable by inspection our experiments, we find qualitative evidence to support our "hest--esponding" candidates assumption.
D. Experiment Details
D.1. Dataset Details
The dataset we used, the Djinni Recruiting dataset, uses the MIT Licence and adheres to the conditions of fair use (Drushchak & Romanyshyn, 2024).
D.2. Model Details
We include the costs of the different models we used. We used Together.ai to query several other models. According to pricing, cost-efficient models that perform relative-well for improving resume scores are LLAMA3.3 70B COOB-INSTRUCT-TUI and DEEPSEEK- 68B. We note that these models (with the exception of DeepSeek recently) are not broadly available to consumers.
and GPT-40 would be the premium models both for input and output tokens. Other
20

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Resume Scores Under Applicant Manipulations (Google Designer)
Maripulstion
Claude -3 3.5-Sonnett
Applicant Manipulation Model
DeepScek.678
DeepSeok-V3
GPT.3.5-Turbo
GPT4c-mini
GPT4o
1ama3-70B-nnsttuch
Mixtral-8xBBllnee Innsllic
70
80
85
90
95
Resume Score
Qualified
Unquatfied
Qualified p50
Figure 10. Resume distribution of 50 qualified and 50 unqualified resumes before and after LLM manipulations for Google UX Designer Position
Resume Scores Under 2 Manipulations (DeorDash)
Applicant Maripalation Model
DeepSeck-67B
GPT-40
Llams3-70@-inelnsst-Tue
72.5
75.0
77.5
80.0
82.5
85.0
87.5
Resume Score
Figure 11. Applying LLM manipulations twice did not significantly improve the score of resume than single manipulation.
21

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Model
Platfrom
Price (1M Input To-
Price (1M Output
Our
kens)
Tokens)
Spend
Claude-3.5 -onnnttttt
Claude.ai
$1.50
$7.50
$1.04
DeepSeek-67B
Together.ai
$0.90
$0.90
S0.29
DeepSeek-V3
DeepSeek
$0.55
$2.19
$0.11
GPT-3.5-Turbo-012.
OpenAI
$0.25
$0.50
S0.28
GPIT-4ominn
OpenAl
$0.075
$0.30
S0.42
HIPT-40-2024-08-
OpenAI $1.25
$5.00
S80.04
Mixtral-8x77-Innttmee
Together.ai
$0.60
$0.60
S7.44
Llama3.3- -intto---ntttttTTnne Turbo
Together.ai
$0.88
$0.88
S0.73
Table Summary models we experimented with. transparency, we also include the total amount we spent on each model. Costs for GPT-4o and Bextral-8x7-IInstruee are larger due to inital experiments.
D.3. LLM Prompt Details
Preliminary testing with LLMs showed that they easily susceptible to hallucinations. For instance, when prompted the model with job-specific LLM prompt (described further below), it frequently fabricated details about project management tools and methodologies that the candidate had not mentioned in their original resume. To empirically assess the susceptibility of LLMs to hallucinations, we tested resume modifications using prompt designed to improve resumes based on a specific job description. The job description used in our experiments, shown below, was Romanyshyn, 2024):
drawn from an example Project Manager role in the Djinni
Job-Specific LLM Prompt:
"Can you tailor my resume to this job description?
'A commitment to collaborative problem solving, agile thinking, and adaptability is essential. We are looking for a candidate who able to balance a fast moving and changing environment with the ability to identify, investigate, and predict project risks Recruiting stages: HR interview, Tech interview **Core Responsibilities*** Manage the full project life cycle including requirements gathering, creation of project plans and schedules, obtaining and managing resources, and facilitating project execution, deployment, and closure. In cooperation with Technical Leads create and maintain comprehensive project documentation. Manage Client expectations, monitor and increase CSAT level; Plan, perform and implement process improvement initiatives. Organize, lead, and facilitate cross-functional project teams. Prepare weekly and monthly project status reports **What you need to production environment Excellent organization and communication skills and the ability to communicate effectively with customers and co- workers. Sttong understanding of Project Management Methodology (SDLC, Agile, Waterfall, etc.) Creative mind with the ability think outside-of-the-bx. The ability manage multiple projects simultaneously Experience with Jira similar project management tool Upper-intermediate level of English is must.' Please wait for me to enter my resume."
Succeed:*** ** 1+ Year of dedicated Project Management
We then tested the aforementioned Job-Specific LLM Prompt with CHATGPT-40 on the following resume of UI/UX designer:
Original Input Resume Text:
Possession of Adobe Photoshop, Adobe-AI, skills of working with a graphic tablet; UI/UX ; Figma, Zeplin, Sketch, MarvelApp. Invision; 2D graphics (understanding of pixel and vector graphics and their specifics); 2D Design; 3D Max; Ability work in different styles; Ability to draw by hand; Creation of design banners, layouts, logos, frameworks and prototypes for websites and mobile iitiicationsTesstin of games; Making presentations in Power Point; Availability of portfolio: Bug tracking systems and project management tools: Trello and Jira. I'm self-motivated artist with a strong skills at game industry. I have more 5 years experience in game company, and now, want to broaden my world with interesting projects, to share own experience and to acquire new skills and knowledge. So, my goal is putting into practice somebody's 's dreams, in so doing, working with pleasure. would like work at company with friendly team, interesting projects and decent wage.
22

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Job-Specific LLM Prompt Modified Resume:
Your Name
Your Address
City, State, Zip
Your Email
Your Phone Number
LinkedIn Profile Portfolio Link
Professional Summary
Self-motivated project manager with 5 years of experience in the game industry, specializing in UT/UX design and project management. Adept managing the full project life cycle, fostering collaboration, and implementing process improvements. Seeking to leverage strong organizational and communication skills in dynamic environment.
Core Competencies
Project Management Methodologies: Agile, Waterfall, SDLC
Tools: Jira, Trello, Figma, Adobe Photoshop, Adobe Illustrator
Communication & Collaboration: Excellent ability manage client expectations and facilitate cross- functional teams
Process Improvement: Proven track record in planning and implementing initiatives that enhance efficiency
Risk Management: Skilled in identifying, investigating, and predicting project risks
Creative Problem Solving: Ability to think outside the box and balance multiple projects simultaneously
Professional Experience
Your Current Job Title
Your Current Company Name, City, State
Month, Year to Present Managed the full project life cycle, from requirements gathering to project execution and closure, for multiple UL/UX design projects.
Collaborated with technical leads to create and maintain comprehensive project documentation, ensuring clarity and alignment throughout the project.
Organized and led cross-functionll project teams to facilitate successful project delivery, enhancing customer satisfaction and increasing CSAT levels.
Prepared and presented weekly and monthly project status reports to stakeholders, ensuring transparency and effective communication.
UI/UX Designer
Previous Company Name, City, State
Month, Year Month, Year Created design banners, layouts, logos, frameworks, and prototypes for websites and mobile applications, contributing to the overall project goals.
Conducted testing for game projects, utilizing bug tracking systems to enhance project quality and performance. Developed and delivered engaging presentations in PowerPoint to communicate design concepts and project updates.
Education
Your Degree
Your University Name, City, State
Year Graduation
Porifolio
Available upon request at link.
Languages
English (nnper-intermediate)
23

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Interests
Collaborative design projects
Continuous learning and skill development
As noted above, CHATGPT-40 fabricated details about the individual's technical project management skills. Specifically, borrowed language (highlighted blue) from the input prompt, which referenced the job description, and then inserted it into the outputted resume. These skills were mentioned the job description but were not included in the individual's original resume.
D.3.1. FINAL NATI-HALLUCINATION LLM PROMPT
To mitigate hallucinationns, we used he following prompt for our experiments. Additionally, we found that sequential messaging style (without interruptions from the LLM) effectively reinforced the importance of preventing hallucinations to the LLM.
Table 8. Anti-Hallucination LLM Prompt
Message No.
Text
1.
"Can you improve my resume?"
2.
"I will be punished if lie about my qualifications, so it is imperative that the new resume does not add any facts that are not in the original resume." "Also, please do not include any additional notes or explanation: just want the text of the new resume."
3.
4.
"This is my resume: [imput resume)."
Anti-Hallucination Prompt Output Resume:
Your Name
Contact Information:
Professional Summary
Belf-motivated artist with over 5 years of experience in the game industry. Skilled in graphic design and UI/UX, am eager to broaden my horizons through engaging projects, sharing my expertise, and acquiring new skills.
Skills
Proficient in Adobe Photoshop and Adobe Illustrator
UI/UX Design
Experienced with Figma, Zeplin, Sketch, MarvelApp, and In Vision
Understanding of pixel and vector graphics
2D and 3D Design (3D Max)
Hand-drawing capabilities
Creation design banners, layouts, logos, frameworks, and prototypes for websites and mobile applications
Game testing experience
Proficient in creating presentations in PowerPoint
Familiarity with bug tracking systems and project management tools: Trello and Jira
Portfolio
Available upon request.
Objective
To contribute to exciting projects within a collaborative team environment while achieving a balanced professional development and fair compensation.
24

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
We also tested this Anti-Faalucciatiion Prompi on our selected test resume (see "Original Input Resume Text" on the previous page): the subsequent outputted resume is shown above. When compared to the output of our Job-Specific L.M Prompt, we found that our Anti-Hallucination Prompt successfully did not fabricate information that was not previously mentioned in the original resume. Instead, it solely improved the written quality of resumes by adding "professional summary section" and reorganizing the individual's existing skills in a logical manner. Upon manual inspection of a random subset of outputs from our Anti- Hatluclaluinnnoonn Prompt, we further confirmed that hallucinations were indeed reduced by this prompt.
D.4. Evaluation (Scoring) of Resumes
To evaluate our (potentially (LM-manipulated) resumes, we used Resume-Matcher to compare them against two Project Manager job descriptions, which we selected from job boards on LinkedIn (DoorDash Inc., n.d.) and Google (Google, n.d.). We selected these two descriptions due to their detailed explanation their respective role-related responsibilities and their lexical dissimilarity to each other.
Job Description I (Product Manager (Multiple Levels) @ DoorDash):
About the Team: At DoorDash, we're re redefining the future of on-demand delivery. To do this, we're building world-class product organization, in which each of our product managers plays critical role in helping to define and execute our vision to connect local delivery networks in cities all across the world.
About The Role: Product Managers at DoorDash require sharp consumer-firse firrst eye, platform thinking, and strong cross-functional collaboration. As a Product Manager at DoorDash, you will own the product strategy and vision, define the product roadmap and aligmment, and help drive the execution. You will be working on mission-critical products that shape the direction of the company. You will report into one of the following pillars: Merchant, Consumer, Operational Excellence, Ads, Logistics, or New Verticals. This role is a hybrid of remote work and in-person collaboration.
You're Excited About This Opportunity Because You Will...
1. Drive the product definition, strategy, and long-term vision. You the roadmap.
Work closely with cross-functions teams of designers, operators, data scientists, and engineers. Communicate product plans, benefits, and results to key stakcholders, including the leadership team.
3.
We're Excited About You Because...
1. You have 5+ years of Product Management industry experience.
2. You have 4+ years of user-facing experience in industries such as eCommerce, technology, or multi-sided marketplaces.
You have proven abilities in driving product strategy, vision, and roadmap alignment.
4. You're re an execution powerhouse.
5. You have experience presenting business reviews senior executives.
You have empathy for the users you build for.
7. You are passionate about DoorDash and the problems we solving for.
At DoorDash, our mission to empower local economies shapes how our team members move quickly, learn, and reiterate in order to make impactful decisions that display empathy for our range of users-from Dashers to merchant partners to consumers. We are a technology and logistics company that started with door-to-dor to-oor delivery, and we are looking for team members who can help us go from company that is known for delivering food to a company that people turn to for any and all goods. DoorDash is growing rapidly and changing constantly, which gives team members the opportunity share their unique perspectives, solve new challenges, and own their careers. We're committed to supporting employees' happiness, healthiness, and overall well- being by providing comprehensive benefits and perks including premium healthcare, wellness expense reimbursement, paid parental leave, and more.
About DoorDash:
Our Commitment to Diversity and Inclusion:
We're committed to growing and empowering more inclusive community within our company, industry, and cities. That's why we hire and cultivate diverse teams of people from all backgrounds, experiences, and perspectives. We believe that true innovation happens when everyone has room at the table and the tools, resources, and opportunity to excel.
25

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Statement of Non- :onniisrriminttio::: In keeping with our beliefs and goals, employee or applicant will face discrimination or harassment based on: race, color, ancestry, national origin, religion, age, gender, marital/domestic partner status, sexual orientation, gender identity expression, disability status, or veteran status. Above and beyond discrimination and harassment based on 'protected categories.' we also strive to prevent other subtler forms of inappropriate behavior (i.e., stereotyping) from ever gaining a foothold in our office. Whether blatant or hidden, barriers to success have no place at DoorDash. We value a diverse workforce - people who identify as women, non-binary or gender non-conforning. LGBTQIA+ American Indian or Native Alaskan, Black or African American, Hispanic or Latinx, Native Hawaiian or Other Pacific Islander, differently-abiee caretakers and parents, and veterans are strongly encouraged to apply. Thank you the Level Playing Field Institute for this statement of non-discrimination.
Pursuant to the San Francisco Fair Chance Ordinance, Los Angeles Fair Chance Initiative for Hiring Ordinance, and any other state or local hiring regulations, will consider for employment any qualified applicant, including those with arrest and conviction records, in manner consistent with the applicable regulation. If you need any accommodations, please inform your recruiting contact upon initial connection.
Job Description 2 (Staff UX Designer @ Google)
Minimum Qualifications
1. Bachelor's 's degree in Design, Hlmann-Compun Interaction, Computer Science, related field, or equivalent practical experience.
8 years of experience in product design or UX.
3. Experience with industry standard design tools (e.g., Photoshop, Illustrator, Sketch, Visioo,, Figma, Principle, etc.).
4. Include portfolio, website, or any other relevant link to your work in your resume (providing a viewable link access instructions).
Preferred Qualifications:
Experience creating and maintaining robust, coherent design systems that are usable across various devices and platforms.
2. Experience shaping processes to establish and sustain a scalable and consistent design language.
Proven track record of exploring and implementing innovative communication modes that are delightful, understandable, and accessible for diverse user groups, including designers, developers, and end-users.
4. Demonstrated ability to provide strong design perspective with fluency in native app patterns and user experiences.
5. Strong collaboration skills within gross-functional product teams, including working with engineering, writers, editors, and researchers to refine and validate design decisions.
About the Job: At Google, we follow a simple but vital premise: "Focus on the user and all else will follow." Google's Interaction Designers take complex tasks and make them intuitive and easy-to-use for billions of people around the globe. Throughout the design process-from creating user flows and wireframes to building user interface mockups and prototypes- to life in way that feels inspired, refined, and even magical.
envision how people will experience our products, and bring that vision
Google User Experience (UX) is made up of disti-disciplinary teams of UX Designers, Researchers, Writers, Content Strategists, Program Managers, and Engineers: we care deeply about the people who our products. The UX team plays integral part in gathering insights about the attitudes, emotions, and behaviors of people who use our products inspire and inform design. We collaborate closely with each other and with engineering and product management to create industry-leading products that deliver value for the people who use them, and for Google's businesses.
As an Interaction Designer, you'll 'll rely on user-centered design methods to craft industry-leading user experi- ences ences-from concept to execution. Like all of our UX jobs, you'll collaborate with your design partners to leverage and evolve the Google design language to build beautiful, innovative, inspired products that people love to use.
26

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Labs is group focused on incubating early-stage efforts in support of Google's mission organize the world's information and make it universally accessible and useful. Our team exists help discover and create new ways to advance core products through exploration and the application of new technologies. We work to build new solutions that have the potential to transform how users interact with Google. Our goal is to drive innovation by developing new Google products and capabilities that deliver significant impact over longer timeframes.
The US base salary range for this full-time position is 168, 000- 2000-25,000 + bonus + equity + benefits. Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target salaries for the position across all US locations. Within the range, individual pay determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.
Please note that the compensation details listed US role postings reflect the base salary only, and do not include bonus, equity, or benefits. Learn more about benefits at Google.
Responsibilittes: Influence cross-functional stakeholders to gain support for design strategies, collaborating from early-stage concept development to iteration and execution. Drive the creation of innovative design solutions that address user needs, business objectives, and industry trends while furthering business outcomes. Build and iterate on Figma prototypes communicate ideas, user journeys, and decision points, while exploring rapid visual design styles and interactions. Apply user-centered design principles, integrating UX research insights and Al-first interactions to create unique, multi-modal user experiences that drive product adoption. Provide direction to UX designers, manage priorities, dependencies, and stakeholders effectively, and independently drive work toward key milestones.
E. Examples
We provide simple example illustrate the intuition behind the TWO-TICKET scheme. Example 1. Consider some joint distribution D over feature vectors, groups, and labels, as well some scoring functions such that s sX) Uni((O, 10]) and (H X) s [X) 2 5].
Privilged Group
Unprivilege
10
10
0
5
10
Before Manipulation
After Appicann
After Hiter Applicatt Menipulation
Figure 12. Example resume score distributions for Group and Group U. Solid regions correspond to truly represent candidates, while the hatched regions represent truly unqualified candidates.
1. If the Hirer were receive the unmanipulated scores, then the threshold would achieve perfect classification for both groups, as shown in the leftmost plot.
2. However, suppose that the groups have unequal access to LLMs. Namely, suppose Group P has access to an LLM Lp that increases scores by 1, while has no access to LLMs. Since the Hirer cannot discriminate between scores from different groups differently, no threshold can simultancously classify both groups correctly. The interval of undominated thresholds [5, 6]: however, any threshold in this range must accept unqualified candidates from Group P or reject qualified candidates from Group U. This harms both fairness and overall accuracy. Under the No False Positives Objective, the optimal threshold is T 6 with TPRp 1 but TPRg 0.8, shown in the middle plot.
3. To mitigate this disparity, consider applying Hirer LLM LH where L (LP((())) = s(x) + 1 and L (Lu (L(x))))) = s(x) + The idea is that LH improves Group U's scores while minimally affecting Group P's scores, restoring equality of post- masipuaaipllaoon score distributions between the two groups This allows for perfect classification between the two Groups again with new threshold as shown the rightmost plot.
27

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
F. Complete Proofs
Theorem 1. Suppose Lp ž Ly. Then for all x E R2, (xx 2 O.
Proof. Fix a E. If (x) 1, then trivially A(x) = We therefore focus on the case that f-()) = 0. this case, note that
P (f)(<r))) 1) Li Lg
Since 8 is non-decreasing, fx(x) 1 (s(x) 2 r| is also non-decreasing, Since Lp = Lu. Lp(x) stochastically dominates Ly (z). so by Lemma 5.1,
-[F+(L((2)]
Eg Bls- (Lu
= ((f((E))) =1) 2 P (f+ (2/r) = 1) 1,
T.
((f(2))) =) - P (fz())) = =1) 2 O.
Tog
( 0.17
Corollary 1. Suppose Lp = Ly. Then, ATPR 2 O.
Proof. Observe that ATPR (x)A(x) Y 1]. Since Lp 2 LU, Theorem implies that for all E X, A(x) 2
Thus, ATPR 2 0.
(2) Lemma 6.1. If Lp LH - then (1)) (2)
Proof. Assuming continuous distribution over scores, the minimum threshold that achieves the No False Positives Objective is equal to the maximum score achievable by a candidate with true label Y = 0. That is,=
(16) max s(x")
where
x" E (x', L (x')) and x' E {a, Lp(Ø, Lu (x))) and a~ D Y =0
By leveraging the conditional independence L (x.))) and nn given (the same logic as the proof of Lemma 6.2), may equivalently write
7 (1) = max (M, Mu. MM,,M! M
where
M = max
s(x)
a>D =0
Ly s(Tyg(x))
max
M
max
(k) s(L
(z)).
(8)
We show that, under the Lemma's condition, ( is irrelevant to this expression. Suppose I.p 2 I. Then, fixing x X, Lp(x) stochastically dominates [(k) (x). Since is non-decreusing, s(Lp(x)) stochastically dominates s(L (æ)). Thus,
(R)
maxi,p s (L)( 2 max, 100 s(L (x)). Taking the maximum over x ~ D Y = 0 yields
Mp Tp bp
s(L((x)) >
max
(k) (æ)) M ((L
(()) ==
28

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Since Mp 2 M we have (*k) max(M, (M, Mp, My). This expression is independent of k, so -* -=122)
Lemma 6.2. For Hirer LLM Lu and threshold т, the probability that candidate (x, g. 4) is accepted is
P (fr(z")) 1)
< T| P. (s(Lg(())) <T) P (s(Ln())) <t) 1 11 [s()))
Lng
Lu
Proof. If s 2 then trivially L. Lu (fr = 1) 1. We therefore focus the case that fr = 0. In this case, note that
P Lg.L
(1+ (a( = 0) = P (s(a²) <) = IP
(s(xr)) < n (LLH(()) <T)
Lg.bir
Observe from the definition of LLM manipulation that L (xrg)) and æ'g are conditionally independent given æ. Thus,
PP
(s(arg) <) Ligdt (s(Ln(zz')) <T) ( (f+(a")) 00 L.
Lg.Lu
Furthermore, observe that Ln (x')) is equal in distribution to Lu (2:). We obtain
IP ( (x = 0) = P (x(Lg((0g)) <T) IP (s(L=(1gg) <T). L
Lg
Lu
Taking the complement yields the lemma.
Theorem 2. Let (1) (3)), Lp t22 LU. and D D Then for all æ Rd (z) (z).
Proof. For convenience, let (1) (2) be the common threshold that achieves the No False Positives Objective.
Fix x X. If (x) = (
(x) = 1, then trivially A (x) = (x) = O. We therefore focus on the case that (f) = O. By Lemma 6.2,
A (8) (x) = P
(fr(z") =1) (fr(*))=1)
P (fr()() = 1)
Lp.
(k)
Le
(K)
8( (((L (x)) < <T) d(x),
(k)
(k)
where d(x:) (s) <T)
P (s(L(()) <) Lp
Lu
Observe that d(x) does not depend on the Hiring Scheme k. Thus,
(2) (x) (z) = -(xx) d(x),
where 6(x) = P (s(L (1) (x:) (())
<T)
(s(L (2) (x)) <T).
Since Lp 2 Lu by assumption, by Theorem d(x) > 0. By very similar argument, since 1 1 8(x) 0. It follows that
A (x)) A (1) (x) S
Corollary 2. Let TPR(4) denote the true positive rate over group g under Hiring Scheme k. Let (2))) Lp Lus
and L (2) (1)
H L Then
(2)
1.
TPR
< TPPR
2. TPR(2) 2 TPR(1 (1) for e (P,U). g
3. (2R(2) > ())
29

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Proof. The first part follows almost immediately from Theorem upon observing that
TPPR
E
A Y 1].
L
The second part follows from an application of Lemma 6.2 and a near-identical argument to Theorem 2.
The third part follows from the second part, observing that TPR (6) P(G P)TPR (R) + = =
Theorem 4. Let r be the threshold used by the Hirer in the n-ticket scheme. If Ln Lp Lu, applying the n-ticket scheme and taking the limit olx, LH) E {0, 1). Furthermore, there exists ai E [0, 1). dependent on L and such that
oo, then any applicant D is guaranteed to receive group-independent outcome,
(L = 1) - S O(K)).
We prove the theorem in two different ways. The first is relatively straightforward argument based on Lemma 6.2. In the second, we show that the acceptance mapping induced by the Hirer's LLM is a contraction operator, allowing us to apply Banach's 's Fixed Point Theorem. We include the fixed point proof as it is standard in similar settings and provides useful intuition about the role of the Hirer's LLM.
Proof. By similar logic to Lemma 6.2, under an n-ticket scheme, we have
P ()) Lg.h
(1)) = 1) = 11 - (a(x)) < <] La
(s(L)(2))) < <T) P (s(Lm(2)) < <-)=-
Lu
If s(x) 2 T, then PP FLy,Ln (fr (xx(1)) =) for all 2 1.
If (a:) < r, let k = I, (a)))()))) < r). Then we have
IP (xx()) 1) = La
PP (s(Lg(z)) < <T) k
Lg
If hi 1, then since LH - Lg. we have Ft, (s(L)(z)) < <) 2 FL,
<T)= Thus, PL, Eu (fr (x)()) 1) = 0 for all n 2 1. 1 < hi < 1, then we have instead (x(f) = 1) 2 O(k)).
Proof. Let x E Rdd be an unmodified resume. For E (p, u}. let Jm = PL (f-((((z)) = 1) be the baseline, group- dependent probability acceptance. Namely, Un represents the probability that candidate from group with unmodified resume æ will be accepted by fr given their resume has been modified once by LLM LU, with the same holding for Pw: P. Lp. respectively.
Let 1t2 P., (S+ (Z)) = 1), where E (Li (=)) for i E N and G E (U, Since any LLM manipulation is invariant to previous manipulations, hi represents the probability that the result of a single I, application to a possibly manipulated resume z achieves a score equal to or above the threshold т. Observe that and ho depend on æ due to the presence of fundamental features that are preserved throughout every LLM manipulation and affect the probability of acceptance.
Following Theorem and the assumption in the theorem, we have that Ur S Pr hm-
Consider the function Te [0, 1] [0, 1] given by
T=(2)= = = hrm 2).
Since 0 5 + (1 2) 1, the output of (2) is in [0, 1] and thus T is well-defined.
Let d(p, u) = lp u//. Notice that ((0, 1). d) is metric space. We will show that Tn is contraction operator.
Let K2 = 1 h hr = 0, the applicant will be rejected even after applying the two-ticket scheme an infinite number of times unless (z) = 1, in which case the applicant will be accepted for every n E N. Either way, if hg = O, the outcome is independent of group membership.
30

---

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
So suppose b1 1. This implies 0 hi 1. For any p, [0, 1]. have that
d(Ta(p). = Ip + A (1 - p) (u + nm (1 u))l (1 bp - ul S him (T
Therefore, Te is contraction operator by definition.
Recall that already showed that for 2 if hg 0 then the outcome is independent of the group membership. Next, we show that for ftx > 0, (z) converges to 1, which implies that the outcome of the n-ticket scheme when n - 00 is always acceptance and is independent of group membership.
Claim 5, If hm > 0, lim (x)
Proof. Rearranging (2),,
(=(2) = hm)= hm-
Applying Te twice:
(2) = -1- ha) - ) + hr) + hr = (1 - New + No (1 1 hw))).
Continuing this process, we can express T as a geometric series,
1 (1 l1mr)"
THE = hm)) hr Taking the limit as n - 00:
E(1 (1- hr)) (1 hmm )² hm ==
- -(1 -11-1z))m 2).
h
lim (z) -1===1 lim (1 110 - =
n-0000
n=000
The rest of the proof follows from Banach's 's fixed point theorem.
Corollary 3. If LH ł Lp = Ly, then for every unmodified resume æ E IRd, there exists hig E [0, 1) that depends on L and æ such that for any n 2 (Lo * f. (LP(())) s O(kg). Hence,
1. A (n)
TPRl O(K)), where max kz-
2. TPR(") Z TPR for g E (P,U) and n 1.
3. (PR()) TPR(n-1) for every >
Since the threshold 7* already prevents false positives (Definition 4.2), (3) also implies that accuracy does not decrease.
Proof. If then from Theorem 4, for any P).
0) 0 = P(F = 0) O(K)) If o 1 then from Theorem for any G (U.
(L2
0)
[P(=(LZ = 1) 1| = = - (()) s(kk)
hence
1-0(()) s (L(
Let o(x)) be the outcome of an
IP (L))))))))) + (Lp()) < (Lo o) ( #o)
From union bound,
P (T=(()) # o) (1( # o) < (L * o) < 20(k)) = 0(k2)
As for the clauses, (1) follows directly from the above, by conditioning over y 1. (2) and (3) follow from the same proof as Corollary 2.
31


Reviews:
[{"review_id": "WQ6AYepVx5", "reviewer": "Reviewer_BLGs", "summary": "The paper considers the problem of hiring in the scenario when applicants use LLMs to assist in CV writing (as well as hirers can have their own LLMs). This potentially can lead to unfair and inaccurate hiring, if, say some applicants use paid version of LLM while others do not. To mitigate this, the authors propose a two-ticket scheme where the hiring algorithm considers both the original CV and a manipulated version of it. They provide theoretical proof that this scheme improves fairness and accuracy in hiring when TPR is maximized s.t. FPR = 0.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the comments and for reading our paper (even the appendix!). We will answer the questions below: \n> Does the framework hold if manipulations apply to style features?\n\nOur framework considers manipulations made to style features rather than fundamental features. We accidentally referred to style features as fundamental when we introduced them. The typo occurs in lines 134 - 137; style features should be $(x_1, …, x_{d_1})$ while fundamental features should be $(c_1, …, c_{d_2})$. This notation is consistent in the rest of our paper. In this framework, style features are manipulable through LLM manipulations while fundamental features such as technical experience and programming skills are preserved. We thank the reviewer for catching this key typo. \n\n> Accuracy did not change much for the two-ticket scheme compared to traditional evaluation, increasing by around 5%. Do you observe a connection between manipulation strength and accuracy change?\n\nRegarding accuracy change, we expect that stronger manipulations will lead to a greater accuracy improvement in our two-ticket scheme. For example, we expect to see the highest improvement in accuracy when qualified and unqualified resumes are separable originally and become more difficult to separate after a round of candidate manipulation. In Figure 1, we see that for PM resumes, this effect was more pronounced for Claude-3.5-Sonnet, GPT4o, and Llama3-70B. In contrast, when there is not much change in scores, the accuracy remains the same for both traditional and two-ticket schemes. We will make sure to discuss this in the next version of the paper!\n\nWe are happy to answer any additional questions! We thank the reviewer for understanding and appreciating our non-traditional contribution.", "claims_and_evidence": "Claim 1: The paper establishes that hiring can be unfair when different applicants use different LLMs -- Figure 1.\n\nClaim 2: Two-ticket scheme is introduced and proved that it improves disparity -- In two-ticket scheme the hiring algorithm considers both the original and a manipulated version of each resume. Theoretical improvements in fairness and accuracy are demonstrated through Theorem 2 and Corollary 2, which works under No False Positives Objective.\n\nClaim 3: The results are empirically validated through a case study in Section 7.", "methods_and_evaluation_criteria": "Yes, different LLMs are considered, uncertainty intervals are reported.", "theoretical_claims": "I read Appendix E and theoretical results in the main paper - they appear correct to me.", "experimental_designs_or_analyses": "Appendix C provides more details on evaluation and seems reasonable.", "relation_to_broader_scientific_literature": "The paper extends the strategic classification framework to address LLM-driven manipulations, which introduce stochasticity and complexity. This paper offers a valuable initial step towards fairness analysis in LLM-based hiring scenarios. The claims are supported both theoretically and empirically.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "The paper is well-written and is easy to read.\n\nAmong weaknesses, some of the assumptions of the theoretical analysis might be strict, including no false positives objective.", "other_comments_or_suggestions": "NA", "questions_for_authors": "Does the framework hold if manipulations apply to style features?\n\nAccuracy did not change much for the two-ticket scheme compared to traditional evaluation, increasing by around 5%. Do you observe a connection between manipulation strength and accuracy change?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ED526FIdtz", "reviewer": "Reviewer_vS3u", "summary": "The paper explores challenges of fairness and accuracy in hiring when job seekers use generative AI tools to enhance resumes. It proposes a \"two-ticket\" scheme, where employers also manipulate resumes using AI. The study demonstrates, theoretically and empirically, that this approach improves fairness and accuracy in the recruitment process.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful summary and comments on our paper. We address the questions below: \n1. **Technical/Logistical Challenges**: In the real world, there may be practical challenges in implementing our “two-ticket” scheme. A major challenge is for companies to decide which model to use when applying the second manipulation. Companies will likely test different models to determine which one best improves resume scores in their ATS system, based on their specific job description style, before selecting a model for the second manipulation. Not all companies may have the technical expertise to do this kind of model selection. We will address this in the next version of the paper.\n2. **Protection of Data Privacy**: This is an excellent question. We assume that companies use APIs in a way that queries are not stored by the company (this is the case, for example, for the LLMs accessible to people from the university the authors are affiliated with). Different companies offer ways to opt out of data collection. We do not anticipate the resume manipulation itself to be any less private than the companies storing a candidate's resume. Another way that privacy might be compromised is from the chosen threshold. The threshold might be sensitive to the resume score of a single individual, violating exact differential privacy. For the threshold to be privacy-preserving, we recommend using DP threshold functions [1]. We will make sure to discuss this in the next version of the paper.\n3. **Long-term / Downstream Impacts**: Considering the downstream impacts is a fascinating perspective on the system effects of LLM tools used for job applications. In this work, we look at the applicant filtering stage of the hiring process. The subsequent hiring decisions made by humans may introduce further disparities that may affect employee satisfaction in the long run. In future work, we plan to do an empirical study of the impact of LLM-aided application materials across a variety of industries in consultation with economists.\n\nWe thank the reviewer for providing insightful and thought-provoking questions about the practical implications of our work. We hope our answers help address the reviewer’s concerns and strengthen the contributions of our paper. Our main contribution is to provide an intuitive theoretical framework for modeling LLM manipulations in the applicant screening process. We are happy to answer any additional questions. \n\nReferences: \n[1] Bun, M., Nissim, K., Stemmer, U., & Vadhan, S. (2015, October). Differentially private release and learning of threshold functions. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science (pp. 634-649). IEEE.", "claims_and_evidence": "The claims made in the paper are well-supported by both theoretical reasoning and empirical evidence. However, broader validation across a more extensive range of job roles and industries could strengthen the generalizability of their findings. Additionally, more details about the potential variability in effectiveness with different types of jobs and applicant tracking systems could further validate the claims", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are appropriate for addressing fairness and accuracy challenges in hiring when candidates use generative AI tools. The \"two-ticket\" scheme and the use of real-world resume datasets effectively demonstrate improvements in these areas under the study's framework.", "theoretical_claims": "Yes，the paper provides detailed mathematical formulations and accompanying proofs aimed at demonstrating the validity of the “two-ticket” scheme and its impact on fairness and accuracy in hiring processes. Any issues would require direct examination and verification of these proofs by an expert in the field.", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "The paper addresses fairness and accuracy issues in algorithmic hiring processes influenced by generative AI tools that manipulate resumes. It proposes the \"two-ticket\" scheme to mitigate bias. The study builds on existing research in algorithmic bias, strategic classification, and AI's impact on resume quality, offering new insights to balance fairness and accuracy.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "None", "other_comments_or_suggestions": "None", "questions_for_authors": "What technical or logistical challenges might arise in implementing the “two-ticket” scheme across various industries, and how can these challenges be addressed?\n\nWhat measures are in place to ensure the protection of candidates’ data privacy when resumes are manipulated using generative AI, and how can ethical concerns be mitigated?\n\nHow might the “two-ticket” scheme affect employee performance and satisfaction in the long run, and what plans are there for further research into its long-term impact and adaptability across different job sectors?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NDBhHO5xtx", "reviewer": "Reviewer_rGk7", "summary": "* This paper proposes and investigates a theoretical model for LLM strategic manipulations in the job application market, motivated by empirical observations.\n* The model is motivated by three empirical observations: (1) LLMs tend to improve the score of a resume in an automated ranking system, (2) higher-cost LLMs tend to induce a larger improvement in the resume score, and (3) repeated application of LLM improvement steps yields diminishing improvements to the resume score.\n* In the formal model, each candidate is represented by a triplet $(x,g,y)$. The feature vector $x\\\\in\\\\mathbb{R}^d$ is the candidate’s original resume, represented as a concatenation of immutable (fundamental) and style features. The group $g$ (privileged/unprivileged) represents the candidate’s manipulation ability, and $y$ is a binary label indicating the true qualification level of the candidate. It is assumed that $x$ is independent of $g$, and that the label is independent of group membership.\n* LLM manipulation $L$ is modeled as a stochastic function which replaces some of the features in the resume with independent samples from random variables. Hiring decisions are represented by a non-decreasing score function $s(x)$, and it is assumed that each group has access to its own LLM $L_g$, and that the hiring agent also has access to an LLM. \n* The goal of the hiring is to maximize the hiring rate of qualified candidates, under the constraint that no unqualified agents are hired. The goal of the candidate is to maximize their probability of acceptance, and they can choose whether to manipulate their resume using their group’s LLM.\n* In the theoretical analysis, the authors show that hiring schemes that ignore manipulation induce group disparity. In response, the authors propose the two-ticket scheme, in which the hiring agent may apply additional LLM manipulation on top of the submitted application, and show that the two-ticket scheme decreases disparity under assumptions.\n* Finally, the empirical section simulates the process using real resumes and using OpenAI LLMs for manipulation, showing favorable results.\n\n## Update after rebuttal\nThank you again for the response. The rebuttal addressed some of my concerns regarding the validity of the assumptions (in particular, I agree that it is reasonable to assume that monthly LLM subscription is more accessible to privileged groups), while I still believe that other foundational assumptions would greatly benefit from stronger grounding or refinement (in particular, the formal model of LLM manipulation). With this, I maintain my original assessment, and I believe that the paper is a step is the right direction.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s detailed comments and feedback on our work. Here we will focus on answering questions from the reviewer: \n1. **Practical scenario of X and Y independence**: Just to clarify our assumptions, our paper builds on existing fairness literature and assumes that “X and G are independent, and Y and G are conditionally independent given X.” One example with these assumptions is non-native English speakers who are equally qualified for a project management job, but the applicant who uses the better LLM might achieve a higher scoring resume in the system. We are also happy to clarify further if we didn’t fully answer this question.  \n2. **Iterative interaction with LLMs**: While LLMs can indeed be used interactively with multiple revisions, our single-shot manipulation model provides a foundational understanding of strategic classification in the era of LLMs. For example, we observed significant improvements for \"style features’’ under one application of LLM manipulation. As LLMs improve, we find these multiple revision steps are less helpful. That said, future work can explore extending the model to incorporate iterative, interactive revisions, potentially offering even finer-grained control and adaptability in practical scenarios.\n\nTo address iterative manipulations on the Candidate's side, we can assume the distribution of manipulated features already captures this. On the Hirer’s side, iterative manipulations would be more challenging in practice as it will require them to have a person in charge of such interactions, which might not be ideal considering that the candidate knows their background better than anyone else. However, we think such interactions will have diminishing returns of improvement, and that having access to more advanced LLMs is more crucial. We will add this discussion in the next version of the paper.\n\n3. **No Pareto-dominance**: Great question! When there are free vs. premium versions of an LLM, we find Pareto dominance is a reasonable assumption. Without Pareto-dominance, group outcomes depend highly on the scoring system deployed, and the two-ticket scheme bias mitigation could be reduced. For example, if Group P’s LLM improves feature 1 and Group U’s LLM improves feature 2, and the Hirer uses the same LLM as Group U, it could be that the screening software gives no weight to feature 2. An interesting direction for future work could be to apply multiple different LLMs to improve a single resume, both from the Candidate’s side and the Hirer’s side. We will clarify this in the next version of the paper.\n\nWe will offer several clarifications to some points raised around broader applicability.\n- **Token-based costs**: We agree that token-based costs for refining long resumes are minimal. However, many candidates are unaware/don’t know how they pay per token and simply default to whichever premium LLM they can access, which can widen disparities if others rely on free or older models. Our motivation also addresses cases in which specialized, high-cost resume-editing tools can be used by some people (We briefly discuss this assumption in Section 4.2). \n- **“Fundamental” and “style” features**: The decoupling of \"style features\" from \"fundamental features\" in our model is for analytical purposes to demonstrate that LLMs can enhance the style of resumes without altering their core qualifications. This does not imply that style features are unimportant or that evaluation scores do not depend on them. Moreover, if every feature of relevance is a style feature, then arguably LLMs make resumes a bad indicator of qualifications for those jobs (in which case LLM detection methods could be relevant).\n- **Experiments in the supplementary material**: We used 100 sampled CVs in the supplementary material due to budget restrictions. We are happy to expand the experiments to more resumes. \n- **Applicability**: Beyond the labor market, our work is applicable in any application system, e.g., college and graduate school applications, where a judge’s perception of quality is based on features that an LLM can improve (without fabricating lies). \n- **Somerstep et al.**: Great suggestion! We specifically note the relevance of this work, which explores causal strategic classification and its impact on labor market dynamics. This line of research complements our work by providing insights into how strategic behavior can influence both employer and labor force outcomes, highlighting the importance of considering these dynamics in developing fair and effective hiring practices. We will ensure to incorporate this discussion in the paper.\n- **Section 4 Notation**: Thank you for noticing this! We assume LLMs change only style features. We will correct this.\n\nWe thank the reviewer for the detailed reading of our work. We hope that our response provided clarification and strengthened our contribution. Please let us know if we can answer any additional questions.", "claims_and_evidence": "The paper’s theoretical claims and empirical evaluations seen to be well-supported. Connection to practical problems relies on very strong assumptions, and the degree to which this disparity may appear in practical scenarios is not completely clear: \n* Resumes typically contain thousands of tokens, and the current LLM prices for refining documents with thousands of tokens are currently in the order of cents, even for the most expensive LLMs - A relatively negligible amount.\n* LLMs are often used interactively, with multiple revision steps that might not align with the single-shot manipulation modeled here.\n* The model relies on the assumption that “style features” can be decoupled from “fundamental features” - However it is not clear to what degree this assumption holds in practice. For example, product management and UX design (the jobs under consideration in the empirical evaluation section) rely on communication skill, where the ability to communicate well in written form is a key requirement. It is therefore unclear whether it is reasonable to decouple “fundamental” and “style” features in this case.\n* While in practice it seems very reasonable to assume that LLM proficiency varies across groups, it’s not clear whether the conditional independence assumptions on $x$ and $y$ are likely to hold in this case.", "methods_and_evaluation_criteria": "The proposed evaluation method appears sound.", "theoretical_claims": "The theoretical derivations appear sound at a high level. A deeper verification of the proofs can confirm the robustness of these claims.", "experimental_designs_or_analyses": "* The overall experimental design is reasonable.\n* The reports results on 520 CVs, the supplementary material only includes revised samples from 100 CVs. This discrepancy should be clarified.", "relation_to_broader_scientific_literature": "* The paper provides a practical classification for strategic classification, motivated by LLMs. The traditional strategic classification literature is focused on classic statistical learning problems (classification/regression), and modeling strategic behavior in the context of generative AI is an emerging research frontier.\n* Not clear if the proposed model is applicable to other scenarios beyond labor markets.", "essential_references_not_discussed": "It may be valuable to acknowledge related work that intersects strategic classification and labor market analyses, such as Somerstep et al. (“Learning In Reverse Causal Strategic Environments With Ramifications on Two Sided Markets”, ICLR 2024).", "other_strengths_and_weaknesses": "Strengths:\n* Clean theoretical model, which relies on a simple and natural empirical observation.\n* Theoretical claims are supported by empirical observations.\n* Paper is very clear and easy to follow.\nWeaknesses:\n* Model relies on strong assumptions, and it is unclear whether the described disparity is likely to be significant in practical scenarios.", "other_comments_or_suggestions": "Some of the notations in Section 4 were confusing -  L199 says that “the formulation is based on the observation that LLMs can standardize style features…”, however Definition 4.1 formally defines the LLM manipulation $L$ as a transformation which modifies the values of the “fundamental features” $x_i$ (defined around L133), which seems to contradict this.", "questions_for_authors": "* Is it possible to describe a practical scenario where access to LLMs is likely to be the significant factor in decision making, while the independence/conditional independence assumptions still hold? (i.e. a scenario $x$ and $y$ are independent/conditionally independent of the group as assumed, and there is a difference in LLM accessibility for resume improvements)\n* How would the model and its results change if candidates were allowed to interact iteratively with the LLM rather than performing a single-shot manipulation?\n* What happens if one LLM does not Pareto-dominate another? (i.e., when the conditions in Definition 5.3 do not hold)", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xANRJAiXwU", "reviewer": "Reviewer_oTGu", "summary": "This paper studies a new variant of strategic classification applied to automated hiring decisions when applicants use large language models (LLMs) to improve (i.e., “manipulate”) their resumes. The authors observe that LLM-based enhancements can blur the line between skilled and unskilled applicants, especially when different applicants have access to different-quality models.\nThrough theoretical analysis and experiments on real resumes, the paper shows that this two-ticket approach can simultaneously improve both overall accuracy (true positive rate) and fairness (reduce disparities between applicants who have access to high-quality LLMs and those who do not).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed comments and feedback on our work. In particular, we appreciate the suggestion to connect the work to human-in-the-loop hiring and “AI detection” strategies, and will discuss them in the next version of the paper.\n\nWe want to highlight that our main contributions are (a) adjusting the strategic classification model for LLM manipulations, and (b) suggesting a first generic solution with theoretical guarantees for this problem. The novelty of our paper comes from the unique analysis of the system effects of some candidates deciding to use (more advanced) LLM tools for their resume. The experiments are added to demonstrate the applicability of the results as empirical computer science experiments. We leave a broader study of this effect across the labor market to future work by domain experts. We would like to provide additional clarifications for the weaknesses listed: \n1. **Novelty Concerns**: (1) Using one LLM for all roles would be limiting; however, we test the manipulation of different LLMs on resumes of different roles available in the open-source dataset that we use. We are not entirely sure if this was the question the reviewer intended to ask, but we are happy to clarify and answer the question with additional clarification. \n(2) Since the goal of our experiments is to validate our theoretical results, our experiments focus on a few job descriptions and consider only a subset of a diverse labor market (We have added additional experience for 8 more job descriptions). We believe that for our purposes, 520 real resumes are enough.\n2. **Reproducibility**: The reviewer makes a great point in highlighting the importance of replicability. We have strongly prioritized replicability by using only open-source resumes, an open-source resume scorer, and listing the specific API version of the models we use. While new models might be introduced, our results can always be replicated with the model versions we report. Our work on general-purpose LLMs is motivated by the resume prompts in real-world datasets such as WildChat. Domain-specific training for resume editing is an interesting suggestion, but out of scope of our current work. However, it is important to note that our model and theoretical guarantees are general enough to be applicable to privileged groups accessing domain-specific tools as well.\n3. **Real Hiring Pipelines**: We agree with the reviewer that real hiring pipelines may incorporate more than a single score. Indeed, local hiring laws such as NYC144 incentivize humans to be in the loop for hiring decisions in order to avoid algorithmic auditing ([source](https://www.fisherphillips.com/en/news-insights/new-york-lawmakers-aim-to-close-loopholes-nycs-ai-bias-audit-law.html)). We want to clarify that we are focused on the first stage of the hiring process: resume filtering. This is a necessary step candidates must pass and is done primarily by ATS software such as the one we test. We focus on the only available open-source ATS scoring system (to the best of our knowledge) in order to ensure reproducibility of our results. Moreover, our model of the resume scoring system is general and can incorporate and combine multiple metrics used to make a filtering decision: we only require that the system is monotonic in the resume features. We will make sure to address that in the next version of the paper.\n4. **Multifaceted “privileged” groups**: While we address only two levels of privileged groups, all our results can be easily generalized to multiple levels of privilege, so long as the Hirer uses a Pareto-dominant LLM (that is at least as good as the most privileged group’s LLM): this follows from how we define LLM manipulations stochastically. We will make sure to address this in the next version of the paper! In case the reviewer means other types of disparities, our protected attribute that we introduce and are interested in is model-access privilege. \n5. **Minimizing False Positive Rates:** Our research provides an initial step into utilizing LLMs to reduce inequities in hiring procedures as a result of LLM manipulations, with theoretical guarantees on improvements with the Two-Ticket scheme. Our research is most relevant to job positions receiving large numbers of applications. Similarly to related works, we find that there is sufficient evidence to believe that a FPR=0 constraint is necessary when there is a large volume of applicants for a select number of positions, as per the standard in hiring for tech sector positions nowadays. For example, one mid-size company may receive over 25,000 applications for just 6 summer intern positions in 2024 ([source](https://www.linkedin.com/posts/patreon_patreonintern-launchingwithpatreon-internprograms-activity-7067615159833280512-A42d/)).", "claims_and_evidence": "1. Point 1\na. Claim: Traditional hiring pipelines are susceptible to unequal LLM access—applicants who can afford better models can often achieve substantially higher resume scores and thus enjoy higher acceptance rates.\nb. Evidence: Empirical experiments on 520 real resumes demonstrate that using strong LLMs (like GPT-4 variants) can significantly increase a resume’s relevance scores—sometimes making an unqualified resume appear indistinguishable from a qualified one (as shown in Figure 1).\n\n2. Point 2\na. Claim: A “two-ticket” approach can improve both overall accuracy and fairness. The hiring algorithm can re-manipulate the candidate’s final resume with the “two-ticket” approach.\nb. Evidence: The authors present formal theorems (Theorem 2 and Corollary 2) and a set of experiments to show that, under a “no false positives” objective (prioritizing zero FPR and maximizing TPR), giving every candidate a second manipulation step (via the hiring algorithm’s own LLM) raises TPR for both privileged and unprivileged groups while reducing TPR disparity.", "methods_and_evaluation_criteria": "Methods\nModeling Framework: The authors adopt a strategic classification framework in which each applicant’s resume is a feature vector split into “fundamental” (e.g. actual skills/experience) and “style” features (e.g. grammar, resume organization). The LLM stochastically rewrites style features but does not alter fundamental features.\nScoring Function: The hiring side uses a fixed, off-the-shelf resume “scorer” (like an applicant-tracking system) to assign numeric relevance scores. The classifier decides on a threshold: those with scores above the threshold receive a positive decision.\nNo False Positives Objective: To reflect the practical cost of hiring unqualified candidates, they primarily study the setting where the hiring system tries to maintain zero FPR and maximize TPR.\n\nEvaluation Criteria\n1. True Positive Rate (TPR) – fraction of truly qualified applicants who pass the threshold.\n2. False Positive Rate (FPR) – fraction of unqualified applicants who pass the threshold (aiming for zero).\n3. Fairness Metric: TPR disparity across “privileged” vs. “unprivileged” LLM-access groups.", "theoretical_claims": "1. Inequity Under Traditional Schemes: If privileged and unprivileged groups have different-quality LLMs (the privileged group’s LLM stochastically dominates the unprivileged group’s), the TPR of the privileged group will be higher. (Formalized in Theorem 1 and Corollary 1.)\n\n2. Improvement via Two-Ticket Scheme: By giving everyone an additional LLM pass from a strong model, the paper proves that:\na. TPR disparity can only decrease.\nb. TPR for both groups can only increase (or remain the same).\nc. Overall accuracy remains the same or improves (since TPR rises at zero FPR).\n\n3. Constant Threshold: Under mild assumptions (e.g., the hiring LLM does not exceed the privileged group’s LLM in quality), the threshold that satisfies a zero-FPR constraint does not change when moving from traditional to two-ticket schemes—so gains in fairness/TPR occur “automatically” without requiring re-tuning for each group.", "experimental_designs_or_analyses": "1. Data: They use 520 resumes (balanced across UI/UX design and project management roles) from an open-source dataset that includes anonymized real resumes.\n2. LLM Conditions: Different groups get different LLMs (e.g., GPT-3.5 vs. GPT-4 variants); the better model is assigned to the “privileged” group. Applicants submit whichever version of their resume scores higher (original or LLM-manipulated).\n3. Two-Ticket Step: The hiring algorithm re-manipulates each submitted resume with its own GPT-4-based LLM, and the final acceptance decision is based on the best of those two.", "relation_to_broader_scientific_literature": "1. The paper’s strategic classification framework extends Hardt et al. (2016) and subsequent works on manipulations that alter one’s “input” to a classifier. However, unlike classical strategic classification where manipulations often incur a cost, the authors note that cost here is nearly zero—anyone can prompt an LLM.\n2. The fairness approach (equalizing TPR or at least reducing disparity) connects to the well-known “equalized odds” concept, but it emphasizes a special case: no false positives, maximizing TPR.", "essential_references_not_discussed": "Potentially under-discussed areas that might be valuable:\n\n1. Work in human-AI collaboration or human-in-the-loop hiring, which could complement or replace purely automated scoring.\n2. Emerging “AI detection” strategies or watermarking of LLM output. The authors mention not knowing whether the text was manipulated but do not deeply discuss whether detection-based solutions might reduce disparities.", "other_strengths_and_weaknesses": "Strengths\n1. As generative AI becomes more common in hiring processes, this is among the first formal frameworks analyzing LLM-based resume enhancements.\n2. The theorems clearly demonstrate why (and when) two-ticket classification can reduce disparities without sacrificing accuracy.\n3. Empirical tests on genuine resumes using an open-source application-tracking system add credibility to the arguments.\n\nWeaknesses\n1. I am concerned about the novelty of this paper. \n(1) Using one LLM model for different job roles may limit real-world applicability, as each role’s requirements can differ.\n(2) The sample size of 520 resumes might be too small to capture broader labor market heterogeneity.\n2. This work relies on generic APIs. Results rely on off-the-shelf LLMs with no domain-specific training. This raises questions about reproducibility if LLMs or prompts change. Future LLM updates or new models could alter outcomes significantly.\n3. They rely on a fixed resume-scoring model and do not deeply test alternative commercial ATS or more complex ML-based hiring systems. Real hiring pipelines may also incorporate more than a single numeric score.\n4. They model just two groups (privileged vs. unprivileged). Real-world disparities can be multi-faceted.\n5. This work focuses on TPR under No-False-Positives. This can make sense in certain scenarios but may not align with all employers’ priorities (some might tolerate a small FPR for a greater TPR). The paper’s approach is specialized, and a user wanting a more flexible trade-off might need a different theoretical framework.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper studies a variant of strategic classification that corresponds to automated hiring when applicants use LLMs to improve their CVs.  The reviewers all agreed that the paper is a valuable contribution, and the main (and relatively minor) complaint was about strong assumptions.  We encourage the authors to improve the paper accordingly based on the detailed suggestions and comments in the reviews.
Decision: Accept (poster)