Breaking the Gradient Barrier: Unveiling Large Language Models for Strategic Classification
Authors: Xinpeng Lv
Yunxin Mao
Haoxuan Li
KE LIANG
Jinxuan Yang
Wanrong Huang
Haoang Chi
Huan Chen
Long Lan
Cyuanlong
Wenjing Yang
Haotian Wang
Venue: NeurIPS 2025 poster ()
Keywords: strategic classification
machine learning
LLMs
Primary area: general_machine_learning

Abstract: Strategic classification (SC) explores how individuals or entities modify their features strategically to achieve favorable classification outcomes. However, existing SC methods, which are largely based on linear models or shallow neural networks, face significant limitations in terms of scalability and capacity when applied to real-world datasets with significantly increasing scale, especially in financial services and the internet sector. 
In this paper, we investigate how to leverage large language models to design a more scalable and efficient SC framework, especially in the case of growing individuals engaged with decision-making processes. Specifically, we introduce GLIM, a gradient-free SC method grounded in in-context learning. 
During the feed-forward process of self-attention, GLIM implicitly simulates the typical bi-level optimization process of SC, including both the feature manipulation and decision rule optimization. 
Without fine-tuning the LLMs, our proposed GLIM enjoys the advantage of cost-effective adaptation in dynamic strategic environments. Theoretically, we prove GLIM can support pre-trained LLMs to adapt to a broad range of strategic manipulations. We validate our approach through experiments with a collection of pre-trained LLMs on real-world and synthetic datasets in financial and internet domains, demonstrating that our GLIM exhibits both robustness and efficiency, and offering an effective solution for large-scale SC tasks.

Breaking the Gradient Barrier: Unveiling Large Language Models for Strategic Classification
Xinpeng Lv Yunxin Mao', Haoxuan Li?, Ke Liang's JJixuan Yang",
Wanrong Huang', Haoang Chi², Huan Chen', Long Lan', Yuanlong Chen Wenjing Yang', Haotian Wang
College of Computer Science and Technology, National University of Defense Technology, Changsha, China
Center for Data Science, Peking University, Beijing, China
Faculty of Engineering, the University of Sydney, Sydney, Australia FFacuiyy of Computing, Harbin Institute of Technology. Harbin, China flvxinpeng, maoyunxin, wenjing-yarg wanghaotian13) Snnuddt edu. cn
Abstract
Strategic classification (SC) explores how individuals or entities modify their fea- tures strategically to achieve favorable classification outcomes. However, existing SC methods, which are largely based linear models shallow neural networks, face significant limitations in terms scalability and capacity when applied to real- world datasets with significantly increasing scale, especially in financial services and the internet sector. In this paper, we investigate how to leverage large language models to design more scalable and efficient SC framework, especially in the case of growing individuals engaged with decision -aasinn processes. Specifically, introduce GI.M, gradient-free SC method grounded in in-context learning. During the feed-forward process self-attention, GL.MM implicitly simulates the typical bi-level optimization process of SC, including both the feature manipulation and decision rule optimization. Without fine-tuning the LLMs, our proposed GLIM enjoys the advantage of cost-effective adaptation dynamic strategic environ- ments. Theoretically, we prove GLIM can support pre-trained LLMs to adapt broad range of strategic manipulations. We validate our approach through experiments with collection of pre- pre-trained LLMs on real-world and synthetic datasets in financial and internet domains, demonstrating that our GLIM exhibits both robustness and efficiency, and offering an effective solution for large-scale SC tasks.
Introduction
As machine learning (ML) algorithms are increasingly applied in high-stakes decision-making domains such as hiring, lending, and college admissions, the need for rapid and accurate adaptation to dynamic inputs has become crucial. When individuals are provided with information about decision rules, they may strategically manipulate their features to achieve favorable outcomes. Such strategic manipulation undermines the performance of MI models and diminishes their reliability. This phenomenon aligns with Goodhart's s Law, which states, "Once a measure becomes a target, ceases to be a good measure" [64]. When decision rules are made public, individuals may adjust their features in ways that exploit the evaluation criteria.
In response to this issue, strategic classification (SC) 30 has emerged as a growing area of research. SC aims to develop algorithms that improve the accuracy of decision models environments where
*Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Instantiation of Strategle Classiflcation
In-Conlext Learning
Institutio
GGoal? Torns to people
WWo? Bank
Stuckelberg Game
(00 - ((x)
WWoo? People appling for kans
Goal? Be granted loan
Individunls
Imn. Help a strutegie classifiction taok
with classibeer f0o of (Ox, y, E.ss
Step Rule Learning
AAttion? Liperr clasification
MAction? Stratogle manipulation
Gradient-aware Method
us. Gradient-free Method with LLMs
Imptich decision cules
Slow update
Retraining
Quick feedback
ICL
Step Feature Mudification
Ins. Individuals modily features
positive clsssiicaaionn [lu-contes Examples)
Phishing Detection
Phishing Detection
LLM
Anticipation of manipulation
Lightweight models
LLMs
Ins.
Step Rule Update
Computationally expensive
Gradient-free
Update classiffer (x")... (In-conteat Examples)
seale-scale
Large seale-seale
LLM
evolving dataset
Limited scalability
evolving dataset
No fine-tuning
Update prediction and rales
Figure 1: The figure illustrates a strategic classification scenario. Comparison between traditional gradient-based approaches and our gradient-free method using LLMs with ICL for efficient adaptation to Large-scale and evolving data without fine-tuning.
individuals are likely to strategically manipulate their inputs [51 59]. The SC problem is typically framed as a bi-level optimization (30) following the Stackelberg game structure, with the inner and outer optimization objectives referred to as strategic manipulation and decision rule optimization.
Despite growing theoretical and empirical progress, most existing SC methods, as summarized in Table rely on lightweight models such as linear classifiers or MLPs, and are primarily validated on small-scale datasets (e.g.. Adult and Spam, with fewer than 50,000 samples). However, real-world applications commonly involve significantly larger, dynamically evolving datasets, often runging from millions to billions of samples, rendering existing methods computationally infeasible and inefficient due to their reliance continuous retraining and explicit gradient computations.
Our work particularly motivated by data-intensive application domains such as the internet sector and financial services, where the input distributions shift rapidly due to user interaction or market dynamics, and efficient adaptation to large-scale data is critical. For example, in Figure T consider phishing URL detection, where attackers continuously modify URLs to evade detection systems. This setting naturally involves large-scall and non-stationary data with adversarial dynamies. Traditional SC approaches often rely on iterative retraining or gradient updates to remain robust, which becomes computationally expensive and infeasible at scale.
In contrast, large language models (LLMs) have demonstrated strong capabilities in modeling high-ddmenssionl and evolving input streams (6, 2). offering a promising foundation for scalable and retraining-free solutions to strategic classification in modern data environments such as fraud detection, credit scoring, spam filtering, and content moderation. However, empowering LLMs with the strategic classification paradigm introduces a unique challenge:
(i) On the one hand, once strategic manipulations lead to changes in individuals' distribution, models for producing decision rules have to be retrained to adapt to the changed distribu- tion 42). However, when dealing with large-scale data, the cost associated with retraining L.MSs becomes prohibitively expensive and infeasible.
(ii) On the other hand, without retraining the LLMs, it is challenging to model the bi-level opti- mization of SC, i.e., including the strategic manipulation and the decision rule optimization.
To address these challenges, we propose a novel gradient-free method that leverages in-context learning (ICL) in LLMs perform strategic classification without updating model parameters. Specifically, we aim to answer the following questions:
How does ICL simulate strategic manipulations and feature changes in LLMs? 2. How does ICL guide the adjustment of decision rules in L.LMs against strategic manipulation?
Beyond applying ICL to SC tasks, our work theoretically validates the effectiveness of ICL in addressing SC challenges.
Our primary contributions and findings are summarized follows:
2

---

Table Comparison of capabilities between existing SC solutions and our proposed method.
Method
Non-linear Gradient-free
large-scale
generalization
Linear Model (27 60
MI.P 22 52 69
X
GLIM (Ours)
/
7
/
We theoretically establish, for the first time, how LLMs leveraging in-context learning can implicitly simulate both the strategic manipulation and decision rule optimization stages of the SC bi-level problem, without any fine-tuning.
Based on this insight, we introduce Gradient- freee Learning In-context Method (GLIM). that embeds the SC bi-level optimization within pre-trained LLMs, enabling robust and efficient deployment of SC in real-world scenarios.
We validate our theoretical insights through comprehensive experiments on both synthetic and real-world datasets. The results demonstrate the practical utility and effectiveness of our approach in real- wolldwrldd SC applications.
In Section we introduce the strategic classification task and the in-context learning mechanisms within LLMs. In Section 3 we demonstrate the feasibility of leveraging LLMs for the strategic classification problem and introduce a bi-level implicit gradient descent method for SC. In Section we experimentally validate our theoretical findings and the feasibility of our proposed methods. In we review related work on strategic machine learning and large language models.
Section 5
2
Preliminaries
This section introduces the mathematical formulation of strategic classification (SC) and the funda- mental concepts of in-context learning (ICL) within LLMs. Throughout our paper, uppercase letters denote random variables (e.g., X, Y), while lowercase letters represent their realizations (e.g., 2;, y). Bold symbols (e.g., x and X) are used for vectors or matrices.
2.1 Strategic Classification Task
The SC problem can be formulated as Stackelberg game involving two players: decision maker (the classifier) and decision subjects (the classified individuals) [30 [50]
This setting captures real-world scenarios such as loan approval and college admissions, where institutions publicly announce evaluation criteria, and applicants adapt their features (e.g., test scores, financial statements) towards such criteria. Formally, the decision maker defines a decision rule, e.g.. some classifier, f Fd + {0, 1). mapping feature vectors binary outcomes y E {0, 1}. Once the rule f is known, individuals may modify their features x to a new version x' in hopes of receiving a favorable decision. Such modification incurs cost, quantified by cost function c(x, x').
In the inner state of this bi-level optimization, each agent aims to maximize their utility, trading off classification benefit with manipulation cost:
Definition 2.1 (Strategic manipulation in SC tasks). The optimal modified feature vector x' is determined by:
x b(x) arg [f(x²) xc(x,,))),
(1)
where fxx) E {0, 1} is the classification result after modification, c(z, x') is the manipulation cost, X > 0 is trade-off parameter, and D is the feature space. Usually, the cost is modeled as the
Mahalanobis Distance ((xxx)) = (x' x) MM(x' x), where M is a Mahalanobis matrix [26 9.
In the outer stage, the classification rule f is designed remain robust under such strategic manipu- lation:
In this Stackelberg framework 44), the interaction unfolds in two sequential stages: (i) the decision maker publishes its policy (classification rule f), which may be strategic or non ot--ttateeic; and (ii) the decision subjects, after recognizing the policy and its associated costs, determine whether modify their features.

---

Definition 2.2 (Decision rule optimization in SC tasks). The decision maker publishes a rule f that maximizes accuracy w.r.t modified inputs:
f E arg max B (xy))D [1 (f(b(z)) y)]
(2)
JeJ
where F refers to the decision function space, and is the true label.
This objective captures the goal of designing classifiers that remain accurate even when subjects strategically modify their features. In other words, the decision maker aims to anticipate and counteract strategic behavior.
2.2 In-Context Learning
In-context learning (ICL) is paradigm where LLMs perform tasks by conditioning on small number of labeled examples provided within the imput prompt, without requiring any parameter updates. This allows the model to generalize from examples in the input alone, making ICL flexible and retraining-free strategy for downstream tasks.
Self-attention. For given token Cj, its updated embedding through self-attention is
ej + e + EPAVA Softmax (K), (t.j),
h
where qh.j is the query vector for head h at position j. and K,, Va, Ph. are learned projection matrices that determine attention scores and output mixing. Bias terms are omitted for clarity.
ICL as Implicit Gradient Descent. Recent theoretical progress 2 I 73 shows that the forward propagation in LLMs LLMMs particullrlll through linear self-attention layers can be interpreted per- forming implicit gradient descent (GD). Intuitively, this informs that the model learns by simulating an update process internally, even though no actual change of the parameter weights occurs.
Lemma 1 (Forward propagation as implicit gradient descent ①.. Let De (n+1) denote the oulput of the E-th self- attention layer at token position (d + 1, n + 1), ie..ve we have:
(n+1) [SAc Then
(n+1) (2((++)) wo
Di
where wdd
u A Rw. (wz²). with Rw. (w) == 1 EE (w H₁ w u.,))²
This lemma formalizes that ICL can simulate gradient-based learning internally via forward passes, without explicitly tuning parameters. Further details on the derivation for ICI, are provided in Appendix C and the proof of this lemma is included in Appendix D
3 A Gradient-free Learning In-context Method for Strategic Classification
3.1
LLM-Empowered Strategic Classification
Stemming from 30 strategic classification (SC) can be framed bi-level optimization problem
(as Stackelberg framework [44) where individuals (agents) strategically manipulate their features to
while the decision maker aims learn robust decision rule that anticipates and counteracts such manipulations. To formalize this idea, we recall the bi-level
receive favorable classification outcomes [
SC problem as stated in section 2.1
Inner Stage (Strategic manipulation):
x' arg max [f(x)) Xc(x,xx) x')]
(5)
Outer Stage (Decision rule optimization):
f" arg max (x.w) [1 (f(x') = y}l-
(6)
JeF
First, present two formal definitions to characterize how the two-stage bi-level optimization introduced above formulated the language of LLMs:
In strategic classification literature, it is commonly assumed that agents are aware of the decision rule. This work adheres this classical assumption.

---

Bi-Level Optimization in SC
Empower SC task via ICL in LLMs
Inner Stage:
[See SC Task Definition in LLMs
Sec 3.2 Gradient- frre Outer Optimization
Strategic Manipolation: x' urgmer,( cwsi(x,)))
Inner Stage:
Gradient
update: Axf Anç( - y/)W"
Proposition
Construct PVKq, in
Strategic Manipulation via ICL: ICL-indwcdd update xj + X2 Ax/
SA layers: Axf Axf
Gradient-freeaxx" PVKa
Peature Upate:
+ Ax,
Stage:
Outer Stage;
See Gradient free Inner Optimization
Decision Rale Optimization: E urymaxx, E (/))
Decision Rule Optimization via ICL. (f) W AW)
Gradient- -aware update:
Proposition 2:
A99 AWx;
Construct PVKg, SA layers:
Prediction Upate: 9 ri))
ICI. iCI-iiddee Up 8, Ag/2
Gradient-free: 49 PVKq,
Ag/ 257
Figure 2:
Bi-level optimization in strategic classification is simulated within LLMs, where both inner and outer stage optimizations are realized via ICL.
ALM--implemented Inner Stage. With a sequence of labeled prompt examples ((x), y:) )==) decision rule f is implicitly defined via attention based interactions in L.MSs. Another feature x, appended as a query token, whose representation evolves through self-attention and yields manipulated feature x
Definition 3.1 (Strategic Manipulation via ICL (Inner Stage)). Let xj denote an agent's feature and {(x,, yt) === be prompt examples. The LLM's forward pass produces manipulated feature x as: x x; + Ax where Axt denotes the feature update implicitly induced by the LLM's self-attention mechanism during ICL.
LLM- -ip-emmmleeetedd Outer Stage. In the outer stage, the decision maker aims to optimize the decision rule ((; W) based on manipulated features In our gradient-free framework, this process is reflected through an ICL-induced shift in predicted scores ūj = f(x;; W), effectively capturing the implicit optimization of the outer-stage decision rule ʃ and the decision weight W.
Definition 3.2 (Decision Rule Optimization via ICL (Outer Stage)). Let f(; W) be the decision rule implicitly encoded in the LLM. When exposed to manipulated input x', the classifier's ' response adapts through in-context prediction, resulting in:
y' flC (x': W);
where ÿ' denotes the updated prediction, induced by prompt-driven interactions within the self- attention layers.
Existing SC approaches solve such bi-level optimization problem through explicit gradient de- scent [0 32 59]. i.e., by tuning the decision models. However, fine-tuning a large pre-trained model such LLaMA or DeepSeek incurs prohibitive computational costs. Instead, propose to implement this two-stage bi-level optimization process of SC task by leveraging the connection between the ICL and implicit GD, without requiring any parameter updates fine-tuning.
3.2 Gradient-free Strategic Manipulation via ICL
This subsection provides a theoretical justification for how LLMs equipped with ICL can simulate the strategic manipulation of agents (as the inner stage). Specifically, we show that the feature update Ax obtained from a feed-forward linear self-attention layer matches the update derived from traditional gradient descent in strategic classification. For clarity in analysis, we adopt standard assumption in SC formulations
30 50 61]: the decision rule f(-) is assumed to be linear, i.e., f(x) = W
Gradient-aware Inner Optimization in Traditional SC. Conventionally, solving Az in strategic manipulation may be viewed as gradient-descent step with a learning rate 1) and loss function CoD for Eq. 5):
Ax - V W) where Lop instantiated manipulation-awanl loss:
Ax A m -
(8)
Loo(x, x':(xxx;:))
Σ [2, c(x,, xx) (1 w)) f(x/; W) W)+ + xc(xx,x)))1,
N
'However, our further real-world study using LLMs also verify the superiority of our proposed method in the non- lonelinear regime.
5

---

where A is a coefficient matrix that depends yj and the manipulation cost function c
Gradient-free Inner Optimization via ICI. We now demonstrate that LLMSs implicitly realize the same Ax through forward-only propagation without explicit gradient descent. Consider a linear SA layer applied to token (xj, Mj):
(x), (x,, 3/5)
PVK 4t
(10)
where qj the query vector derived from x,. and K, V, P are learned key, value, and projection matrices, respectively. Thus, the feature modification during the forward-only propagation, which we termed as ICL-induced update, can be written as:
Ax
JeL = PVK qj
(11)
Then we prove that there exists pre-conditioned self-attention weights P, V, K, and query vectors q
such that 4x|eL Ax (see detailed derivation in Appendix Fr:
Proposition (ICL Implements the Gradient-free Strategic Manipulation.). Let fx: W) be linear classifier. Then, there exists P, V, K such that for any input Xj, the ICL-induced update satisfies:
Axpa = Ax AD where Ax = PVK qi A A 1n(1 yj W
(12)
Remark 1. This proposition informms that LLMs equipped with ICL can simulate the agent-side strategic manipulation by performing implicit GD. This establishes constructive equivalence between explicit strategic manipulation and ereettiondriven ICL behavior, thereby grounding ICL as a forward-only approximation of inner-stage optimization in SC.
Remark 2 (Linear Derivation.). Following previous protocols 12 16 731, our theoretical analysis is performed in the linear regime. However, we note that our proposed method is also compatible with any non-linear attention and transformer structures, which have also been extensively empirically validated through our comprehensive experiments (in Appendix II.
3.3 Gradient-free Decision Rule Optimization via ICL
This subsection provides a theoretical justification for how LLMs equipped with ICL can simulate the outer-stage optimization in strategic classification. Specifically, we demonstrate that the prediction update مزj, which reflects shift the classifier's decision rule (f), can be implicitly implemented via a forward pass in self-attention layer, without requiring explicit gradient descent or parameter updates.
Remark 3.
The predicted score denoted as Új = f(x;; W) = W W where W E IRd the decision weight vector and is the manipulated feature.
Gradient-aware Outer Optimization in Traditional SC. Under standard SC settings, the outer-level
decision rule optimization is performed by minimizing a classification loss. For example, using cross-entropy loss Ly. the update to W via gradient descent is:
AW V C,
yj Wx'
1 Vj
X/.
(13)
1 - WX/
where C, is instantiated as:
() - Σ
yo log (xx)) + (1 - リj) log(1 Wxx))
(14)
Thus, the corresponding shift in prediction output is:
GD
AW x
(15)
Gradient-free Outer Optimization via ICL. We now show that LLMs can reproduce the same prediction update Aù, via a forward pass through a self-attention layer. Consider the modified feature
SSee detailed derivation in Appendix E
'The linear self-attention layer is simplified from Eq.3
"ee detailed proof in Appendix F

---

vector x,, along with the previous predictions ش.. is embedded into the prompt. The ICL-induced forward update is:
(x/, 15) (z)) PVK qj.
(16)
where qj is the query vector, and P. V, K are the projection, value, and key matrices.
The update to the prediction output from linear self-attention layers is:
Jン Dj AUj where AØj PVK 9
(17)
Therefore, we prove that one can construct P, V, K such that: A0g A
Proposition (ICL Implements Gradient-free Decision Rule Update). Let f(x) = (W.x) be a
linear classifier. Then, there exists a construction of self-attention matrices K, V such that for any loken (x), Ui), where Új = [(x)), the ICL-induced update satisfies:
Aije PVK qi - -A where Agj AW X
(18)
Remark 4. This proposition confirms that forward-only self-attention dynamics ICL can simulate gradient-based updates in the outer stage of SC. It establishes constructive equivalence between explicit decision rule optimization and ICL-driven prediction adaptation.
Remark 5 (Unified Simulation Bi-level Optimization). Together with Proposition I this result completes the alignment between ICL and bi-level optimization in SC. ICL enables agent-side manipulation and decision-side rule adjustment, all within a gradient-free, forward-only framework.
3.4 Discussion on Policy Transparency
A fundamental characteristic of strategic machine learning is that decision subjects manipulate their input features strategically, understanding the classification rules to achieve more favorable results. This implies that the classification rules should be set transparently to the decision subjects.
Our work is based on theoretical foundations, demonstrating that when LLMs receive series of in-context prompts, their internal reasoning and output adjustment can be considered an approximate form of "implicit" gradient descent. In other words. although we do explicitly update the large-scale parameters, LLMs, driven by contextual information such as "which features to be more sensitive to" and "how to define decision boundaries," " adjust their self-attention layers to align with downstream task expectations. Therefore, strategic machine learning based on large language models can also maintain policy transparency. more detailed discussion provided in Appendix
Experiment
4.1 Setup
Dataset. evaluate our method on six benchmark datasets, comprising five real-world datasets and one synthetic dataset:
Large-scale datasets: CISFraud 633 large-scale transactional dataset provided by IEEE and an international bank for fraud detection. PhiUSHI (55), a phishing URL detection dataset reflecting adversarial evasion scenarios in cy cybersecurity. Synthetic [46), a synthetic dataset generated using the PaySim simulator, which mimics mobile financial transactions and fraud patterns based on real-world data.
Small-scale datasets: Adult a census dataset for predicting whether an individual's income. Spam 40), text-based dataset for binary classification of email messages spam or not. Credit 79.. credit scoring dataset used for predicting the risk of credit default in consumer finance scenarios.
Methods. We consider two optimal policies decision maker can use: 1) "strategic policy" means that models consider and handle possible strategic manipulation. 2) "non-strategic policy" means that models in strategic context, but do not consider strategic manipulation.
$ $See detail derivation in Appendix
9ee full derivation in Appendix

---

Cas Div
10a
L2 Div
Cos Do100
10a
L2 Div
KL Divergence
10a
Divergence
10
Div(L)
Model_Div(R,
Dlvel_Div(R)
GD
10-1
10
10-1.
10
2001
SA layers
100
SA layers
8000
4000 Linear Iterations
Linear SA Iterations
Linear iterations
Linear Iterations
(a) Small-scale Data
(b) Large-scale Data
(c) Distribution Shift
(d) KL Divergence
Figure 3: Comparison of ICL-guided strategic manipulation. (a) and (b) compare ICL and gradient-
descent methods across data scales; (c) and (d) evaluate implicit gradient alignment via distribution
metries.
Cos Divig
L2 Div
Cos Div
10a
12 Div
Cos D
10a
L2 DWv
1000
Div(L)
Modie_DDi(RR
Model Div(R)
Model _ovee_Div(RR Div(())
Madel Div(R)
10-1
10-1
10-1
10
10-1
10-1
10-1
10
4000
8000
Linear SA Iterations
Linear Iterations
Nonlinear SA Iterations
Nonlinear SA Iterations
(a) Small-scale Dataset
(b) Large-scale Dataset
(c) Small-scale Dataset
(d) Large-scale Dataset
Figure 4: Comparison of ICL-guided decision rule optimization with Linear and non-lineat self-
attention layers across dataset scales.
For the baseline method, employ linear regression model reference classifier, optimizing through gradient descent. GLIM, we mainly utilize the pre-trained LLM APIs, e.g., GPT- 4o [53], and refine its responses through in-context learning. Each method is subjected 10-fold cross-validation, and the average results are presented in Table We also conducted experiments [K and LLama 49) Detailed
on Claude (3), Mixtral (38) DeepSeek Gemini [65], Qwen3
implementation specifics are provided in Appendix
4.2 Verification on Strategic Manipulation as Implicit Gradient ICL
To validate the effectiveness of ICL in guiding strategic manipulation through the gradient- free method, measure both cosine similarity and 1.2 distance between the feature vectors updated by ICI. and those produced by gradient descent. These measurements are conducted under both linear and non-linear settings across different datasets. The results, presented in Figure anaJand show that the cosine similarity for both methods eventually converges to approximately the same value after some fluctuations, while their L2 distances decrease to nearly zero.
We also compare the mean offset the feature distribution (distribution shift) and the KL divergence 770 across iterations, illustrated in Figure B(c) and 3(d The close alignment of the two curves confirms that ICL-guided manipulation within the self-attention layers performs comparably to gradient descent. More results included in Figure of Appendix
4.3 Verification on Decision Rule Optimization as Implicit Gradient in ICL
To examine how effectively ICL serves gradient- solution for decision rule optimization, we compare the cosine similarity and L2 distance during the optimization process, as depicted in Figure with linear and non-linear attention mechanisms. Across multiple datasets, the two methods exhibit cosine similarity that gradually rises toward 0.95, while their L2 distances settle at approximately 0. These findings suggest that ICL can successfully optimize decision rules via implicit gradients.
Furthermore, we compare how cross-entropy loss evolves in LLMs with GLIM versus the methods via gradient descent for decision optimization. The corresponding results appear in Figure 55a) and and 5(b) different datasets: similar loss trend emerges for gradient-fres and gradient- aware methods. However, illustrated in Figure 5(b) when applied to large-scale datasets, the loss reduction attained by LLMS with GLIM surpasses that of existing approaches. These results confirm that it is entirely feasible employ our proposed method for strategic classification tasks using LLMs.
8

---

Cypes Entropy Loss
Cypss Entropy Loss
10"
GUM on LLMs
10a
GUM on LLMs
Exiting Method
Exiting Method
Accuraey
Accurge
0.7
0.7
GLIM
GUIM
MLP
MLP
*XXXX***X**
Linear
Linear
15 26
35 45
55
5
15
25
35
45
55
GD Steps 'A Layers
GD Steps SA Layers
Data valume (109)
Data volume (10%
(a) Small-scalc Dataset
(b) Large-scale Dataset
(c) Strategic
(d)
Non-strategic
Figure 5: (a) and (b); comparison of cross cross-entrope entropy losses between ICL and gradient-base based methods.
(c) and (d): comparison of GLIM with existing models as the data volume continuously increases.
Table 2: Performance Comparison between GLIM and Existing Methods under Strategic and Non- strategic Settings across Datasets.
Large-scale Dataset
Small-scale Dataset
Methods
PhiUSI
CISFraud
Syntheric
Credit
Adult
Spam
Existing methods (as shown in Table 1
Strategic
63.20 1.02
65.50 63.61
75.52
1..10
$9.67
Linear Model
Non- Non-Strategic
57.39_0.62
56.63_1.08
60.87+2.11
70.73
72. 12.
87.52+0.58
Strategic
65.65_1.14
65
65.04_1.27
70.90
77.06 77.0640.41
78. 1..74
91.05
MLP
Non-Strategic
59.25
29.03
65.39
71.50
73.57
89.01 40.69
GLIM (ours)
Strategic
10.10
84.62221.09
85.15
89.33
86.22.+1..4
94.85
DeepSeck-V3 V3
Non-Strategic
78.90_1.01
78.74_1.14
80.68
81.45
89.31 78.77
GPT-40
Strategic
86.50_o.91
1..8911.06
11.08
12.33
12.35
19.6410.27
91.35
11.29
95.97 10.61
Non-Strategic
79.14_0.94
80.15
81.19
80,9640.44
80.2
91.28
Strategic
85 =0.95
84,98
84.50+2.11
86.51
88.58
94.50+0.68
Claude-3.7
Non-Strategic
78.40 78.40 =..8
78.54
78.89
80.39
83.85
89.50+0.61
Note: 1) We selected linear models and MLPs as representative lightweight approaches from existing methods. 2) The values represent accuracy (%) with standard deviations indicated after the + sign. We highlight the hest performing results in bold. More complete experimental results included in Tables 3 (in Appendix K
4.4 Analysis on GLIM
Figures ((c) and 5(d) demonstrate that as data volume increases, the performance of lightweight models becomes less stable, while the proposed GLIM method maintains consistent scalability. Table summarizes the overall classification performance across various datasets under both Non- Strategic and Strategic settings. These results collectively indicate that applying GLIM enables large language models to effectively handle strategic classification (SC) tasks, maintaining robustness even when agents engage in strategic manipulations.
Specifically, on the large-scale PhiUSIII dataset under the Strategic setting, GPT-4o with GLIM achieves an accuracy of 86.50% showing the model's 's strong capacity to adapt to strategic inputs. Moreover, on the Adult dataset, accuracy increases by 8.36% from the Non-Strategic the Strategic setting when equipped with GLIM, suggesting that the mechanism not only preserves but also enhances decision robustness under strategic influence. Overall, these findings verify that GLIM allows large language models to generalize SC-related reasoning effectively across datasets of different scales and complexities. More experimental results are included in Appendix
Related Work
5.1 Strategic Machine Learning
In the realm of strategic classification (30], many studies aim to mitigate strategic manipulations
exhibited by individuals interacting with decision models [19
10
82
69). Building on
strategic classification, performative prediction [54] 57 29 31
48 52 has been proposed to study
settings where the deployment a predictive model influences the distribution of the prediction target. Recently, more studies have explored the role of causal reasoning in strategic machine

---

learning [50 9 54 72 21] 8 78 76 74, 75], distinguishing between manipulable and improvable features while accounting for how strategic manipulations may alter underlying qualifications. Other works shift the focus to social welfare [28 23 77. aiming to regulate the strategic behavior of agents to maximize social welfare. To avoid disproportionate disadvantage on certain demographic groups, ongoing research also investigates fairness in strategic machine learning [8] 24 39]. More related work is discussed in Appendix
5.2 Large Language Model
Recently, large language models (LLMs) [6], with strong in-contexi learning (ICL capabilities [68] [I) have been applied across wide range of domains beyond traditional NLP tasks. For example, LLMs have shown significant potential education [37], medicine [67], and various scientific fields [5]. A recent work has broadened the study of large language models by incorporating them into auction mechanisms [20]. Other studies 7 [47 have explored the use of external tools to enhance the capabilities of LLMs for complex tasks. A series of studies employing linear transformers have demonstrated that forward propagation with ICL. in LLMs can internally simulate gradient-based learning mechanisms 2 73 16 17. Specifically, these models undergo a process analogous to gradient descent by updating the weights their self-attention layers. To deepen our understanding of ICL, another research direction investigates the problem of learning function class from in-context
examples 25
6 Conclusion
In this study, demonstrate the feasibility of using large language models (LLMs) to tackle strategic classification problems, which is pioneering attempt to bridge strategic classification and LLMs via ICI. This is the first to employ LLMs to model and solve the bi-level, game-theoretic optimization structure of SC. Building on this bi-level implicit gradient optimization, our work proposes gradient-free in-context learning method (GLIM) that empowers LLMs to solve strategic classification tasks. It enables a scalable and retraining- -arinn approach to large-scale SC tasks, where classical gradient-based retraining requires excessive computational resources and time. From broader social science perspective, our work establishes a crucial bridge between large language models and strategic machine learning. Future research will explore the integration of strategic learning within performative prediction frameworks and seek to further enhance policy transparency LLL- decision models.
Acknowledgement
This work is supported in part by the National Natural Science Foundation of China under Grants No. 62276004, 62525213, 62372459, 62376243, 623B2002, and 62302503, the Natural Science Foundation of Heilongjiang Province under Grant No. No2023033,,6 the NUDT Youth Independent Innovation Science Fund under Grant No. ZK25-20, the National Key Research and Development Program of China (2024YFE020370))
References
[1] Kwangjun Ahn, Xiang Cheng, Hadi Daneshmand, and Suvrit Sra. Transformers learn to imple- ment preconditioned gradient descent for in-context learning. Advances in Neural Information Processing Systems, 36:45614-45650, 2023.
[2] Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models, 2023.
[3] Anthropic. Claude sonnet. http::/://wwwaaaarrpp.ooccoo/o/r 2024.
sononet
[4] Barry Becker and Ronny Kohavi, Adult. UCI Machine Learning Repository, 1996. DOI: http:///ooiorr/102233///XXXXv
10

---

[5] Abeba Birhane, Atoosa Kasirzadeh, David Leslie, and Sandra Wachter. Science in the age of large language models. Nature Reviews Physics. 5(5):277-280, 2023.
[6] Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005. 1.X1::2005.14665, 2020.
[7] Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. arXiv preprint arXiv:2305. 2023.
[8] Trenton Chang, Lindsay Warrenburg. Sae-Hwan Park, Ravi Parikh, Maggie Makar, and Jenna Wiens. Who's gaming the system? a mausally-motivated approach for detecting strategic adaptation. Advances in Neural Information Processing Systems, 37:4231 17742211-4425 1-42348, 2024.
[9] Yatong Chen, Jialu Wang, and Yang Liu. Learning incentivize improvements from strategic agents. Transactions on Machine Learning Research, 2023.
[10] Yiling Chen, Yang Liu, and Chara Podimata. Learning strategy- awwre linear classifiers. Ad- vances in Neural Information Processing Systems, 33:15265-155222 2020.
[11] Fengxiang Cheng, Haoxuan Li, Fenrong Liu, Robert Van Rooij, Kun Zhang, and Zhouchen Lin. Empowering lims with logical reasoning: A comprehensive survey. In Proceedings of the FourthyFourth International Joint Conference Artificial Intelligence, 2025.
[12] Xiang Cheng, Yuxin Chen, and Suvrit Sra. Transformers implement functional gradient descent to learn non-linear functions in context. arXiv preprint arXiv:23322.06528, 2023.
[13] Alibaba Cloud. Tongyi qianwen 2.5. https://wwwtt http///www.lbbba........... com
2024.
[14] Lee Cohen, Yishay Mansour, Shay Moran, and Han Shao. classification, 2024.
Learnability gaps of strategic
[15] Paulo Cortez. Student Performance. UCI Machine Learning Repository, 2014. DOI: http://dbi.oog/1..44452/2
[16] Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers, 2023.
[17] Gilad Deutch, Nadav Magar, Tomer Bar Natan, and Guy Dar. In-context learning and gradient descent revisited. arXiv preprint (rXXxx2311.07772 2023.
[18] Yiqun Diao, Qinbin Li, and Bingsheng He. Exploiting label skews in federated learning with model concatenation. In Proceedings of the AAAI Conference Artificial Intelligence. volume 38, pages 11784-11792, 2024.
[19] Jinshuo Dong, Aaron Roth, Zachary Schutzman, Bo Waggoner, and Zhiwei Steven Wu. Strategic classification from revealed preferences, 2017.
[20] Paul Duetting, Vahab Mirrokni, Renato Paes Leme, Haifeng Xu, and Song Zuo. Mechanism design for large language models. In Proceedings of the ACM on Web Conference 2024, pages 144-155, 2024.
[21] Valia Efthymiou, Chara Podimata, Diptangshu Sen, and Juba Ziani. Incentivizing desirable effort profiles in strategic classification: The role of causality and uncertainty. arXiv preprint arXiv:2502.06749, 2025.
[22] Itay Eilat, Ben Finkelshtein, Chaim Baskin, and Nir Rosenfeld. Strategic classification with graph neural networks. arXiv preprint wwXiv:2205.15766 2022.
[23] Andrew Estornell, Yatong Chen, Sanmay Das, Yang Liu, and Yevgeniy Vorobeychik. Incen- tivizing recourse through auditing in strategic classification. In LJCAI, 2023.
[24] Andrew Estorell, Sanmay Das, Yang Liu, and Yevgeniy Vorobeychik. Group-fair classification with strategic agents. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, FAceT '23, page 389-399, New York, NY, USA, 2023. Association for Computing Machinery.

---

[25] Shivam Garg, Dimitris Tsipras, Percy S Liang, and Gregory Valiant. What can transformers learn in-context? case study of simple function classes. In S. Koyejo, S. Mohamed. A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 30583-30598. Curran Associates, Inc., 2022.
[26] Matan Gavish, Ronen Talmon, Pei-Chun Su, and Hau-Tieng Wu. Optimal recovery of precision matrix for mahalanobis distance from high dimensional noisy observations in manifold learning, 2021.
[27] Ganesh Ghalme, Vineet Nair, Itay Eilat, Inbal Salgam-Cohente and Nir Rosenfeld. Strategic classification in the dark, 2021.
[28] Nika Haghtalab, Nicole Immorlica, Brendan Lucier, and Jack Z Wang. Maximizing welfare with incentive-awarr evaluation mechanisms. arXiv preprint arXiv:2011.01956, 2020.
[29] Moritz Hardt, Meena Jagadeesan, and Celestine Mennder--ünnee Performative power. Advances in Neural Information Processing Systems, 35:22969-22988 2022.
[30] Moritz Hardt, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters. Strategic classi- fication. In Proceedings of the 2016 ACM conference on innovations in theoretical computer science, pages 11 111-122, 2016.
[31] Moritz Hardt and Celestine Mendler-Dlinner. Performative prediction: Past and future. arXiv preprint arXiv:23 10..6608, 2023.
[32] Keegan Harris, Hoda Heidari, and Steven Z Wu. Stateful strategic regression. Advances in Neural Information Processing Systems, 11:28728-287444 2021.
[33] Keegan Harris, Hoda Heidari, and Steven Z. Wu. Stateful strategic regression. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 2021.
Curran Associates, Inc.,
International Conference on Machine Learning, pages 13233-13253. PMLR, 2023. [34] Guy Horowitz and Nir Rosenfeld. Causal strategic classification: A tale of two shifts. In
[35] Safwan Hossain, Evi Micha, Yiling Chen, and Ariel Procaccia. Strategic classification with externalities. arXiv preprint arXiv:2410.08032, 2024.
[36] Meena Jagadeesan, Celestine Mendler-Dinner, and Moritz Hardt. Alternative microfoundations for strategic classification. In International Conference on Machine Learning, pages 4687-4697. PMLR, 2021.
[37] Jaeho Jeon and Seongyong Lee. Large language models in education: A focus the com- plementary relationship between human teachers and chatgpt. Education and Information Technologies, 28(12):15873-1588 2023.
[38] Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, al. Mixtral of experts. arXiv preprint 2xXi:22401004080 2024.
[39] Vijay Keswani and L Elisa Celis. Addressing strategic manipulation disparities in fair clas- sification. In Proceedings of the 3rd ACM Conference on Equity and Access Algorithms, Mechanisms, and Optimization, pages 1-11, 2023.
[40] Mohammad Mahmudur Rahman Khan, Rezoana Bente Arif, Md. Abu Bakr Siddique, and Mahjabin Rahman Oishe. Study and observation of the variation accuracies of knn, svm, Imno, enn algorithms on eleven different datasets from uci machine learning repository. In 2018 4th International Conference on Electrical Engineering and Information Communication Technology (iCEEiCT), pages 124-129, 2018
12

---

[41] Zhiqiang Kou, Jing Wang, Jiawei Tang, Yuheng Jia, Boyu Shi, and Xin Geng, Exploiting multi-label correlation in label distribution learning. In Kate Larson, editor, Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pages 432664334 International Joint Conferences on Artificial Intelligence Organization, 2024.
Main Track.
[42] Tosca Lechner, Ruth Umerr, and Shai Ben-David. Strategic classification with unknown user manipulations. In International Conference on Machine Learning, pages 187 18714-18188 14-332..... PMLR, 2023.
[43] Sagi Levanon and Nir Rosenfeld. Strategic classification made practical. In International Conference Machine Learning, pages 6243 6253.. PMLR, 2021.
[44] Tao Li and Suresh Sethi. A review of dynamic stackelberg game models. Discrete & Continuous Dynamical Systems-Series B, 22(1), 2017.
[45] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024.
[46] Edgar Lopez-Rojas, Ahmad Elmir, and Stefan Axelsson. Paysim: A financial mobile money simulator for fraud detection. In 28th European modeling and simulation symposium, EMSS, Larnaca, pages 249-255, Dime University of Genoa, 2016.
[47] Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai- Wei Chang, Ying Nian Wu, Song Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and wong-and-play compositional reasoning with large language models. Advances in Neural Information Processing Systems, 36, 2024.
[48] Celestine Dünner, Frances Ding, and Yixin Wang. Anticipating performativity by predicting from predictions. Advances in neural information processing systems, 35:31171- 31185, 2022.
[49] MetaAI. The llama 3 herd of models, 2024.
[50] John Miller, Smitha Milli, and Moritz Hardt. Strategic classification is causal modeling in disguise. In International Conference on Machine Learning, pages 6917-6926. PMLR, 2020.
[51] Smitha Milli, John Miller, Anca D Dragan, and Moritz Hardt. The social cost of strategic classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 230-239, 2019.
[52] Mehrnaz Mofakhami, Ioannis Mitliagkas, and Gauthier Gidel. Performative prediction with neural networks. In International Conference on Artificial Intelligence and Statistics, pages 11079-11093. PMLR, 2023.
[53] OpenAl. Gpt-4o system card, 2024.
[54] Juan Perdomo, Tijana Zmnc,, Celestine Mendler-Dünner, and Moritz Hardt. Performative prediction. International Conference on Machine Learning, pages 7599-7609, PMLR, 2020.
[55] Arvind Prasad and Shalini Chandra. PhiUSIII Phishing URL (Website) UCI Machine Learning Repository, 2024. DOL: https://doi.orr//10 10t.10016/...ss.. c00.)...........1..... 2023.103545.
[56] Christian Robert, George Casella, and George Casella. Monte Carlo statistical methods, volume 2. Springer, 1999.
[57] Nir Rosenfeld, Anna Hilgard, Sai Srivatsa Ravindranath, and David C Parkes. From predictions to decisions: Using lookahead regularization. Advances in Neural Information Processing Systems, 33:41 15-4126, 2020.
[58] Imanol Schlag, Kazuki Irie, and Jürgen Schmidhuber. Linear transformers are secretly fast weight programmers. International Conference on Machine Learning, pages 9355-9366. PMLR, 2021.
13

---

[59] Han Shao, Avrim Blum, and Omar Montasser. Strategic classification under unknown personal- ized manipulation. Advances in Neural Information Processing Systems, 36, 2024.
[60] Yonadav Shavit, Benjamin Edelman, and Brian Axelrod. Causal strategic linear regression. In International Conference on Machine Learning, pages 8676 17866866..261 PMLR, 2020.
[61] Hajime Shimao, Warut Khern-Am-Nuai Karthik Kannan, and Maxime C Cohen. Strategic best-response fairness framework for fair machine learning. Information Systems Research, 2025.
[62] Manish Kumar Singh and Ankur A Kulkarni. Optimal stochastic decision rule for strategic classification. In 2024 National Conference on Communications (NCC), pages 1-6. IEFE, 2024.
[63] IEEE Computational Intelligencee Society. leee-cis fraud detection. htttps://ww.ker com/ http://mpptiiiiooeeeeeeereeeeeee
2019.
[64] Marilyn Strathern. 'improving ratings': audit in the british university system. European Review, 5(3):305-321, 1997.
[65] Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024.
[66] Alex Teboul. Diabetes health indicators dataset, 2015.
[67] Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 1930):193- 2023.
[68] Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Tze Cheng, Alicia Jin, Taylor Bos, Leslic Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri. Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung- Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Helllsern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-Joho, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera- Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. Lamda: Language models for dialog applications, 2022.
[69] Stratis Tsirtsis, Behzad Tabibian, Moein Khajehnejad, Adish Singla, Bernhard Schölkopf, and Manuel Gomez- Rodrez-odriigue. Optimal decision making under strategic behavior. Management Science, 2024.
[70] Tim Van Erven and Peter Harremos. Rényi divergence and kullback-leibler divergence. 1EEE Transactions on Information Theory. 60(7):3797-3820 2014.
[71] A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
[72] Kiet QH Vo, Muneeb Aadil, Siu Lun Chau, and Krikamol Muandet. Causal strategic learning with competitive selection. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 38, pages 1541 155499, 2024.
[73] Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pages 35151-35174. PMLR, 2023.
[74] Haotian Wang, Kun Kuang, Haoang Chi, Longqi Yang. Mingyang Geng, Wanrong Huang, and Wenjing Yang. Treatment effect estimation with adjustment feature selection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '23, page 2290-2301 New York, NY, USA. 2023. Association for Computing Machinery.
14

---

[75] Haotian Wang, Kun Kuang. Long Lan, Zige Wang, Wanrong Huang, Fei Wu, and Wenjing Yang. -istrr-disti generalization with causal feature separation. IEEE Transactions on Knowledge and Data Engineering, 36(4):1758-1777 2024.
[76] Haotian Wang, Wenjing Yang, Longqi Yang, Anpeng Wu, Liyang Xu, Jing Ren, Fei Wu, and Kun Kuang. Estimating individualized causal effect with confounded instruments. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '22, page 1857-1867, New York, NY, USA, 2022. Association for Computing Machinery.
[77] Tian Xie and Xueru Zhang, Non-linear welfare-aware strategic learning. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume pages 1660-1671, 2024.
[78] Wenjing Yang, Xinpeng Lv, Yunxin Mao, Liyang Xu, Ruochun Jin, Huan Chen, Jing Ren, Jinxuan Yang. Yuanlong Chen, and Haotian Wang. Advanced strategic improvement with decision interactions. Education, 65(65):70.
[79] I-Cheng Yeh and Che-hui Lien. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Syst. Appl., 36(2):2473-2480, mar 2009.
[80] Ruiqi Zhang, Spencer Frei, and Peter L. Bartlett. Trained transformers learn linear models in-context, 2023.
[81] Xueru Zhang, Mohammad Mahdi Khalili, Kun Jin, Parinaz Naghizadeh, and Mingyan Liu. Fairness interventions as Incentvves for strategic manipulation. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song. Csaba Szepesvari. Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 26239-26264. PMLR, 17-23 Jul 2022.
[82] Tijana Zrnic, Fric Mazumdar, Shankar Sastry, and Michael Jordan. Who leads and who follows in strategic classification? Advances in Neural Information Processing Systems, 34:15257- 1 15269, 2021.
15

---

A Clarification on Our Position Towards Gradient-based Methods
We explicitly state that our proposed gradient-free method (GLIM), leveraging large language models and in-context learning, is not intended to criticize or dismiss traditional gradient- based methods widely used in machine learning. Gradient-based optimization has proven extraordinarily effective, well-established, and foundational for machine learning research and applications.
Rather, our work aims to explore and demonstrate an alternative solution path tailored specifically large-scale strategic classification scenarios, particularly addressing situations where gradient computations and frequent retraining might face practical computational limitations or scalability issues. Our work should be seen as an exploratory contribution, offering additional methodological options to researchers and practitioners, rather than diminishing or replacing the value of gradient- based methods. This work proposes to provide additional methodological options to researchers and practitioners, rather than diminishing replacing the value of gradient-based baaee methods.
B Additional Related Work
There are also some excellent works in the field of strategic machine learning (30 that did not discuss in Section A previous work [57] proposes lookahead regularization in classification models anticipate agent behavior during training. To handle inter-user dependencies, incorporating shallow graph neural networks [22] offers a novel pathway for strategic classification. Another work 43 leverages differentiable optimization layers to directly optimize strategic empirical risk in end-to-end systems. Investigating multi-agent strategic settings, [35] proposes classification methods that account for such interdependent effects to improve fairness and robustness. Recently, [62] proposes an optimal stochastic decision rule for strategic classification, demonstrating that introducing randomness into the classifier can effectively reduce classification errors and improve robustness compared to deterministic approaches.
C Preliminaries of In-context Learning
Following [58] 73,,
We review standard multi-head self-attention (SA) layer which updates each element ej in a set tokens {EI,.... en according to
ej Ej + SA(j, (c1,...,es) exxt
K (% 9hJ dde
(19)
= + 1E Pn softmax
Vn.
where PR, VA, K2, are the projection, value, and key matrices respectively, de is the dimension of the key vector and qh.j is the query vector, all for the h-th head.
The columns of the value matrix Vn [VA,1.- VA.N consist of vectors Vh, WVn Cs. where we introduce Wv. as the parameter matrix of V Similarly, ka, WKA E; for the key matrix K₁, = [k.. Ns.N] and q = Wo, e, for the query vector q These parameters, PR, W W WK.. and W Qa of S A layer, consist of all projection matrices. The self-attention layer described above corresponds to the one used in standard LLMs and ICL is leveraged to bootstrap the update of these parameter matrices.
During the forward propagation in self-attention layers, ICL aims to leverage the contextual examples ((r;, y4) ) embedded into tokens {er ) to predict the response for the new query token ex+1- Specifically, the model observes an in-context prompt composed of n pairs examples and then produces a hypothesis Dn+1 for en+1 Mathematically, one can view the ICL process as inducing temporary "in-context" mapping FIOE (parametrized by language models) such that:
(20) Because this mapping is never explicitly "trained" in the traditional sense (i.e., by gradient descent on the model parameters), the objective of ICI is to minimize the prediction error on the new token using only the in-context examples as guidance. Concretely, the loss function for the ICL. forward propagation can often be written as:
In+1 Fret (en+ ........ e.))
LICl fi(In+1, Vn-1),
(21)
where f is task-specific measure of prediction error, e.g., mean-squared ertor cross-entropy loss.
16

---

D Implicit Gradient Descent in Self-Attention Layers
Our Lemma Tindicates that, under ICL guidance, the token update process within the self-attention layer can be viewed as an implicit gradient optimization process 2
First, we highlight the dependency the tokens e, of the linear self attfnattentonn operation
9 c SA(fe)...., e Σ
PaV.,K 9h.5
- = Σ Pr. Σ Uh & kn.i
(22)
ej + Σ PA E Chee @ Cn WT Wh,00
with @ the outer product between two vectors. With this, we can now easily draw connections to one step of gradient descent L(W)- weight change
2N Σ Wzx 2l with learning rate 17 which yields
N
AW WWWW)(W)
(Wz; - -
(23)
N
Tr
We provide the weight matrices in block form: WK- WQ
with La and Ly the identity
0
0
matrices size N₂ and Ny respectively. Furthermore, we set Wy
with the weight
Wo 1,,
matrix Wo E N of the linear model we wish train and P 2 with identity matrix of size N₂ + Ny. With this simple construction, we obtain the following dynamics
83
7
N
0
x yi
((d
1 0
1
+
93
Wo
-1
y4
0
yj
N
7
(24)
41
+
N
Woz:
yi
0
+
AW2,)
yj
for every token ej = (z,, 9z) including the query token EN+1 give us the desired result.
Grest
(Feast
W
which will
E Derivation of Strategic Manipulation via Utility-aligned Loss
In strategic classification, agents modify their features x' to maximize the utility function:
U(f(x))xx) = f(x') Ac(x, x')
(25)
where f(x') Wx' is a linear classifier, and c(x, x') = (x' (x) TMMM((X' "M x) represents the manipulation cost with M O.
Given the sample-wise loss function:
(x) y,o(x;, x() (1 y1) f(x)) x(xx
(26)
The gradient with respect to manipulated features is:
V Vx 3 2M(x -x,) (1 vi) [- 21M(x -x,)]
(27)
17

---

Let Ax; =x X; denote the feature modification. The gradient descent update with learning rate becomes:
Ax; = V LaD
(28)
Case 1: y/: (Positive Class)
V x LGD = 2MAx;
(29)
Ax; = 2MAx;
(30)
(I + M)xx = Ax; 0.
(31)
Interpretation: No incentive for manipulation when already classified positively.
Case 2: yi 0 (Negative Class)
V Lav = W + +AMAX
(32)
Ax; W -2nAMAX,
(33)
(I + 27AM) W
(34)
Using the cigendecomposition = QAQT:
Ax; 1 + 21AAA) WT.
(35)
Define the adaptation matrix:
A (I+ 21+2)
(36)
yielding:
Ax = AW
(37)
Combining both cases:
Ax -1 A.
(38)
W E Rixd = WT E Rdx1
M, E Ruxd
Ax, E Rdxl (dimensionally consistent).
This shows that minimizing the loss function (x)(x; x') results in the same manipulation direction as maximizing the utility function (ff(x'),
Remark 6 (Generality of Cost Function Forms). While our derivation assumes the manipulation cost c(z, x') = (x' x) M(x' - x) based on the Mahalanobis distance for analytical tractability, our results can be extended to broader class of distance-based cost functions. In fact, many commonly used cost measures in strategic classification, such as Lp norms, graph distances, and general norms seminorms, also satisfy the triangle inequality and support similar gradient-based manipulation dynamics 51 As long as the cost function is differentiable and convex, the gradient-base feature update remains well-defined, and our analysis of in-context manipulation behavior and utility-aligned loss minimization holds under these alternative formulations.
F
The Self-attention Layer Projection Matrix Constructed for Strategic Manipulation
To demonsirate that the implicit gradient update via in-context learning (ICL) matches the explicit gradient descent step:
Ax
A 7 WT
(39)
provide constructive setup of the self-attention matrices as follows.
18

---

F1 Matrix Construction
We define the key, query, and value matrices in block form:
WK Wi- (0o
0
Wv (
(, WT 0)-
(40)
where J E RRdd is the identity matrix for feature tokens. The projection matrix is defined as:
1 A 0 P
0
(41)
N
with A being the manipulation cost-adjusted coefficient matrix from Equation 39p. and N is the number context tokens.
F.2 Update Dynamics
For a query token (z), the query vector is formed as:
q = WQ
Xj V5
00
(42)
The full attention-basee update becomes:
Ax JxL =P
qj
A
Σ
Wve - (Wke))
(43)
N
0
0
Unfolding the matrix multiplication:
An Σ
Ax ICL
w) (x,,,))
(44)
Assuming homogeneous label groups (i.e., all Vt Uj); obtain:
Ax A 17(1 y WT
(45)
which exactly matches the explicit gradient update Ax
F.3
Consistency Verification
The complete feature update under ICL is then:
x/ x, + AxpeL x, + A 17(1 v,
(46)
confirming equivalence to the explicit manipulation update in Equation (3).. The block structure of Wy and P ensures that the label component y/ remains unchanged during the forward pass.
This construction avoids explicit computation of inverse matrices at runtime by directly encoding the gradient dynamics into self-attention weight matrices. The resulting ICL process reflects forward-only approximation of agent-side strategic behavior within the Transformer framework, and demonstrates the capacity of attention mechanisms to simulate first-order optimization steps relevant to strategic classification.
Remark (Connection to Proposition This derivation provides the explicit configuration of matrices P, V, K, and query vector qj. required in Proposition T It confirms that, under linear attention with properly constructed weight matrices, the ICL-induced update Ax exactly matches the explicit gradient response Ax thereby validating the proposition with a constructive proof.
F.4 Extension Beyond the Homogeneous Assumption
The homogeneous label assumption is theoretical simplification 18 used to illustrate the underlying mechanism of the attention-bas based update. enables us to transparently demonstrate how the attention- induced update can precisely align with the gradient descent direction under idcalized conditions.
19

---

Derivation Beyond the Assumption. In practical heterogeneous contexts, the update becomes weighted aggregation of local update directions:
An (1 W: Xx,,,))
Ax
JCL
(47)
N
where (x, xj) represents the similarity between the query and context example.
More generally, the attention mechanism implicitly performs a weighted combination of local gradients:
N Σ ajj g(x,
AX
ICL
(48)
2m1
where Js are the attention weights and g yi) denotes the local update direction associated with each context sample.
When the context samples are independently drawn and sufficiently representative of the local data distribution around Xj. statistical learning theory [56] ensures that:
lim
Ax
ICL
Pxx. (xi, y:)]
(49)
The approximation error can then be bounded as:
Ej = | JCL [x((x))11
(50)
and its expected value satisfies:
I s SC V + with 8 L WPPlocal, Pglobai,,
(51)
where C. corresponds to the sampling error, and 5 captures the distribution shift. Specifically, C' is constant depending the gradient variance, I. is the Lipschitz constant of C, and W denotes the Wasserstein distance between local and global data distributions.
G The Self-attention Layer Projection Matrix Constructed for Decision Rule Optimization
We provide a constructive proof of Proposition 2 showing that a single-layer linear self-attention mechanism can simulate the gradient-based update predictions in the outer-level optimization of strategic classification. To explicitly align the self-attention mechanism with the gradient update Ajji GD construct the weight matrices as follows:
G.1 Key and Query Matrices
We construct the key and query matrices to focus exclusively on feature dimensions while ignoring the prediction values:
(1d 0) E R(d+1)x(d+1)
WK = We
(52)
Thus, the query vector becomes:
x 0
Ia 0
q
We
K
-WT
(53)
G.2 Value Matrix
The value matrix encodes token- wise gradient terms derived from the cross-entropy loss. For each context token i, define the gradient term:
yi
8, : 77
x/
(54)
Wx!
- Wx
20

---

Then define the token-specific value matrix:
0
E
(55)
8
Summing over all context tokens gives the full value matrix:
W (
E d; ))
0
(56)
G.3 Projection Matrix
The projection matrix isolates the predicted score dimension (i.e., the final output of the classifier) from the token embedding:
0 0
- 1((++)x((++1,
P
(57)
This ensures that the output update affects only the prediction dimension. We now compute the full attention based update through matrix products:
0
9)
VKT
1= 5.Ia
0
VK q
EN 1= 0. x,)
PVK T
qj
Mi WxF
T-
1-
(x(, x3))
Thus, the second (prediction) component is:
A
yb
1 y/
(x,, = A0S0.
(58)
WX
1 1- Wx
This construction provides token-wise simulation of gradient descent over prediction scores using single self-attention pass, without explicitly modifying W. It assumes access to all relevant features x/ and uses attention a proxy for computing interactions (x),x().
Remark 8 (Connection to Proposition 2). This derivation offers complete, constructive realization of the condition in Proposition 2 It demonstrates that the change in predicted score from gradient descent, Arj can be exactly matched by self-attention layer via PVK qj, thereby validating the proposition through an attention-driven forward computation.
H Discuss on Policy Transparency in Strategic Machine Learning
Policy transparency is a fundamental concern in strategic machine learning (SC), where individuals strategically manipulate their input features based on their understanding of the classification rules to secure favorable outcomes. Ensuring that classification policies are transparent allows individuals to make informed decisions about how to adjust their features legitimately, thereby maintaining fairness and accountability in the decision-making process.
Transparency in Traditional SC Models. Traditional SC models, such as linear classifiers and shallow multilayer perceptrons (MLPs), are preferred in strategic settings due to their inherent interpretability. These lightweight models allow decision-maken to clearly communicate the criteria used for classification, enabling individuals to understand which features are most influential and how they adjust their inputs accordingly. This transparency not only fosters trust but also helps in mitigating adversarial manipulations by making the decision boundaries explicit and understandable.
Challenges with L.M--based SC Models. In contrast, large language models (LLMs) introduce significant challenges to policy transparency. L.LMs are characterized by their vast number of parameters and complex architectures, including multiple layers of self-attention mechanisms. This
21

---

complexity renders their internal decision decision-making processes less transparent, making it difficult for users to discern how specific input features influence the final classification outcome. The black-box nature of LLMs can therefore obscure the classification rules, increasing the risk of individuals exploiting hidden patterns or ambiguities manipulate their features strategically.
Our Theoretical Contribution: ICL as Implicit Gradient Descent. Our work addresses this transparency challenge by theoretically demonstrating that in-context learning (ICL) within LLMs can be approximated as an implicit gradient descent optimization process. Specifically, we have proven that the iterative adjustments made by LLMs during ICL resemble the steps taken in traditional gradient descent algorithms used in transparent SC models. This approximation provides conceptual framework for understanding how LLMs adapt to strategic manipulations, offering semblance of interpretability despite their complex architectures.
theoretical findings, we propose leveraging the self-attention mechanisms inherent in LLMs to enhance policy transparency. By visualizing attention weights, stakeholders can gain insights into which input features the model emphasizes during classification. This visualization acts as proxy for understanding the decision- making process, allowing users see how different features contribute the final classification outcome. Consequently, even though the overall model remains complex, the attention patterns provide a tangible means of interpreting the classification rules.
Enhancing Transparency through Attention Visualization. Building
Implications and Future Directions. Our approach offers a pathway to reconcile the powerful modeling capabilities of LLMs with the need for policy transparency in SC tasks. By framing ICL as an implicit gradient descent process and utilizing attention visualizations, we provide a method to interpret and audit LL-M-based classification rules effectively. Future research could explore more sophisticated visualization techniques and formalize the interpretability guarantees provided by attention mechanisms, Additionally, developing strategies to balance transparency with the prevention of strategic manipulations remains an important avenue for ensuring both fairness and robustness in LLM-based SC systems.
In conclusion, while LLMs present inherent challenges to policy transparency in strategic classifica- tion, our theoretical framework and interpretative techniques offer viable solutions. By understanding ICI. as an implicit optimization process and utilizing attention visualizations, we enhance the trans- parency of LL--bbssed decision models, ensuring that classification policies remain both effective and comprehensible to users.
I Extension to Non-linear Attention Mechanisms
Our theoretical analysis in Dectionnadoope linear self-attention formulation for clarity and analytical traceability. However, modern large language models (LLMs) such GPT, LLaMA, and DeepSeek operate with a non-linear multi-head attention mechanism that uses the Softmax function. this section, we demonstrate that despite the structural differences, our framework remains applicable in practice and can be naturally extended to non-lineat attention.
L1 Non-linear Attention in Transformers
In the standard Transformer architecture 71. the update of token e, via multi-head self-attentio (omitting biases) is:
Ej f ej ej+
Σ Pa Vr, Softmax (K,, qh.)),
(59)
where:
J is the query vector of head h at position j;
K h, V,, are the key and value matrices from the prompt;
* Softmax(-) produces probability distribution over prompt positions;
. Pr, is the output projection matrix for head h.
This process computes a content-dependent weighted average over value vectors, where weights are derived from key-query similarity.
22

---

1.2 From Linear to Non-linear ICL Updates
In our linear construction (e.g., Appendix G the ICL-induced update explicitly written as:
A PVK T qj and Ayf PVK 9j
(60)
which allows direct alignment with known gradient expressions. However, in non-linear attention, the presence of Softmax(-) introduces dynamic, input-dependent weights, breaking the closed-form linearity.
Here, we provide more detailed derivation and justification showing that this simplification is theoretically reasonable: even under the Softmax-base attention setting, gradient descent (GD) updates can still be effectively approximated.
While Softmax attention performs convex combination, that is, positive weighted average, this restriction holds only in the first layer of the network. As additional non-linear attention layers are stacked and the value matrices are adjusted, the model gradually evolves beyond this constraint to realize more flexible, non-linear, and even implicitly negative- negative-weighted. weighted, gradient- updates.
Derivation: Non-linear attention with Softmax approximating gradient descent. We consider Transformer equipped with Softmax attention and residual connections:
Ze+1 Ze ViZee Softmax (B, (C)
(61)
where Ze E [((+1)x(++I) is the hidden state at layer &, Xe denotes the covariate part (the first d rows of Ze), B5, C2 E Rdxd are query/key projection matrices, and Vj E R(d+1)x(d+1) is the value projection matrix.
To simplify the derivation, we assume
0
By= V
(62)
-re]
indicating that only the label dimension is updated. Under this setup, the attention weights become
Softmax ( s., & exp exp( (xp(2t000 v.)))
(63)
which reflects the exponentiated similarity between the query x66 and context 266, followed by normalization.
This forward process be connected functional gradient descent (FGD):
fe-1(x) = fe(x) +re
0 (e(
(64)
where K(x,z') exp(x T x//02) is an exponential kernel and fe denotes the current function estimate.
We initialize the input
(1) (0)
(x)
x(n+1)
Zo =
(65)
where the first columns are training examples and the (n+1)-th the query with label zero.
In the first layer, the attention module computes weights
exp (xp( (((+)))))
K (((()))0 (x(+))))
(x+1)
(66)
Σ, exp( ( xU) 2(++1)) x()ti)
E, K K(x(),, x((++1)))
and aggregates labels through the value matrix:
f((z(at1)) -ro Σ (n+1) (9)
a;
(67)
This step indeed performs positive weighted average, but it serves only as rough initial estimate of the FGD target.
23

---

Cos Div
1000
10a
L2 Div
Cos Divt
10
12 DiV
Divergence
10a
Divergence
10
Div(L)
Modell Div(R)
Model_Div(R
GD
GD
10-1
10
10-1
10
10-1
SA layers
100
SA layers
4000
8000
Nonlinear SA Iterations
Nonlinear SA Iterations
Nonlinear Iterations
Nonlinear SA Iterations
(a) Small-sealc Dataset
(b) Large-scale Datasct
(c) Distribution Shift
(d) KL Divergence
Figure 6: Comparison and validation ICI.-guided strategic manipulation. (a) and (b) compare ICL
and mundient--escent methods across data scales; (c) and (d) evaluate implicit gradient alignment via distribution metrics.
Entropy Loss
Cross Entropy Loos
Cos Di-1os
Cos Didto
200
GLIM on LLMs Exiting Method
10
GLIM on LLMS
10c
10n
Exiting Method
L2 Diu
L2 Div
DiviL)
Cos Div(L)
Model_ Div(R)
Model Div(R)
10--
10
10-3
10-1
**
Nonlinear Iterations
4000
8000
Linear lterations
4000
8000
Steps/ Nolinear SA
4000
8060
GD Steps / Nonlinear SA
4000
8000
(a) Small-scale Dataset
(b) Large-scale Dataset
(c) Small-scale Dataset
(d) Large-scale Dataset
Figure 7: (a) and (b) compare ICL-guided decision rule optimization and gradient-descent methods across data scales; (c) and (d): comparison cross-entropy losses between ICL and gradient-based methods.
Residual updates in deeper layers. As additional layers are stacked, residual connections accumulate prior updates and inject the residual difference (y(*) - (z)) into each non-linear attention operation. Through residual accumulation, error correction, and non-linear mixing, the update becomes
(e+1(x)= se(x) + r(x) ())) (())))
(68)
where r(x) denotes normalization factor induced by the Softmax scaling.
Although Softmax attention enforces positive and normalized weights locally, this constraint does not limit the model's 's overall expressive power. Through multi-layer stacking, tunable value projec- tions, and residual propagation, the Transformer can effectively emulate complex update dynamics, including gradient-like steps with varying magnitude and sign. Consequently, the multi-layer non- linear attention structure can approximate a broad family of optimization trajectories, supporting the theoretical soundness of our Softmax simplification.
Empirical and Theoretical Support. Recent work, such as 12]. demonstrates that attention layers can implement first-order optimization steps in function space, reinforcing our perspective that Transformer-based ICI. can realize gradient behaviors even in the non-linear regime.
This extension is further supported by our empirical experimental validations, as shown in Figures and 7
Setup Details
J.1 Dataset Details
To evaluate our method across different domains and scales, we use a mixture of real-world and synthetic datasets, especially in internet sector and financial services, summarized follows:
Large-scale datasets: CISFraud (IEEE- CIS Fraud Detection) 63 a large-scate scale transactional dataset provided by IEEE and major international bank, containing over million online payment records with identity, device, and transaction features for fraud classification. PhiUSIIL (55), a phishing URL detection dataset containing 1 134,850 legitimate and 100,945 malicious URLs, reflecting adversarial evasion scenarios in cy cyhersecurity. Diabetes (66), a large-scale medical dataset with 253,680 instances, featuring demographic and clinical attributes used for type 2 diabetes risk prediction. Synthetic [46),

---

Table 3: Performance comparison across various models and datasets under Strategic and Non- Strategic settings on large-scale datasets
Methods
PhiUSIIL
CISFraud
Syntheric
Diabetes
Existing methods (as shown in Table
Strategic Non-Strategic
63.20
63.61
65.50
67. 67.23+1881
Linear Model
57.39=0.62
56.63+1.08
60.87=2.11
61.94
Strategic
65.65+1.14
65.0441.27
70,90
69.85
MLP
Non-Strategic
59.25
59.03+1.06
65.39=2.03
63.8842.21
Strategic
68.37
68.8441.27
70.10
70.44+2.31
GNN
Non-Strategic
59.31
60.45
65,41_2.16
64.7542.12
GLIM (ours)
Strategic 85.10
1..9
85.15
88.74 88.7442.16
DeepSeek-V3
Non-Strategic
78.90=1.01
78.74
80.68
81.15
Gemini-2.5
Strategic
84.17
84.4111..00
87.18
2..20
87.60
Non-Strategic
76.39
76.80
78.87 2.17
80.38
GPT-4o
Strategic
86.50 10.91
85.51
11.08
86.83
2.35
89.27 1:2.68
Non-Strategic
79.14 1.944
80.25 1.110
81.19 2.19
82.40 1:211
Strategic
85.07
84.98
84.50
88.02
Claude-3.7
Non-Strategic
78.40
77.91
78.89 2.00
80.65
Strategic
83.86
83.1641.11
84.67
87. 87.74:22.35
LLama-3.3
Non-Strategic
76.86
75.04+1.14
76.73=2.16
79.92
Strategic
82.35
84.1641.1
80.32 2.20
86.634:2.29
Qwen3
Non-Strategae Strategic
77.29
76.26+1116
77.34
79.10
Strategic
84.20=0.91
84.90+1.00
85.1 2
88.26+2
Mixtral
Non-Strategic
77.42
77.72+1.05
78.66
80.1042.21
synthetic dataset generated using the PaySim simulator, which mimics mobile financial transactions and fraud patterns based on real-world data.
Small-scale datasets: Adult [4. census dataset for predicting whether individual's income, often used in classification tasks. Spam [40], a text- text-based dataset for binary classification of email messages spam not, useful for evaluating manipulation. Credit [79]. credit scoring dataset, used for predicting the risk of credit default in consumer finance scenarios. German (40), small-scale Dataset assess credit risk in loans from the UCI MIL Repository for classification tasks. Student (15).
dataset includes student performance data in mathematics and Portuguese language courses.
The real-world scenes corresponding to these datasets are classified follows:
Internet sector datasets: PhiUSILL, Synthetic, and Spam;
Financial Services datasets: CISFraud, Adult, Credit, and German;
Other domain datasets: Diabetes and Student.
J.2 Model Selection and Configuration
Our experiments employ a variety of state-of-the-art large language models (LLMs), accessed via their respective APIs, to implement the proposed GLIM framework. The selected models include:
Accessed via OpenAl's official API. This model offers enhanced reasoning and multimodal capabilities, providing robust performance in complex SC tasks.
GPT-4o 553
DeepSeek-V3. V3 45) A Thinese-Englise bilingual open-source LLM optimized for down- stream reasoning, retrieval, and generation tasks, evaluated through DeepSeek's API plat- form.
25

---

Table 4: Performance comparison across various models and datasets under Strategic and Non- Strategic settings on ssall-scale datasets
Methods
Credit
Adult
Spam
Student
German
Existing methods (as shown in Table 1
Strategic 75.52
77.1041.58
89.67
85.17
88.31
Linear Model
Non-Strategic
70.73
72.16+1.62
87.52
81.82
82.07=2.01
Strategic 77.06-0.41
78.7441.83
91,05
86.9642..4
87.38
MLP
Non-Strategic
71.50=0.33
+3.57
89.01 00.69
82.05+2,.16
84.82
Strategic
80.12+0.52
78.54
91.44=0.61
87.01 4:.63
88. $8,93
GNN
Non-Strategic
72.27
74.37
88. 18.13
82.90
85. 85.14
GLIM (ours)
DeepSeek-V3
Strategic
89.33
16.22.1..34
94.85
0.667
89.52.
91.20
Non-Strategic
81.45
78.77
89.31 20.68
83.32
84.97 12.06
Gemini-2.5
Strategic
84.81 10.34
85.84.1..08
94.75 0.66
88.33 42.89
90.18
2:225
Non-Strategic
80.62
79.37
89.74.0.66
82.4442..7
83. 12.13
GPT-4o
Strategic
89.64 10.27
91.35
11.29
95.97
10.61
91.61 12.91
92.34
12245
Non-Strategic
80.96
80.23 11.31
91.28
0.665
84.33 1:2.9
85.69 12261
Strategic
86.51 10.31
88.58
94.50.0.66
85.92
01.07.12.333
Claude-3.7
Non-Strategic
80.39 10.37
83.85
89.50
83.92
84.55 4:2.44
Strategic
87. 20.30
88.70.1.41
94.30
89.44.:2.93
90.68 90.68.42.19 19
LLama-3.3
Non-Strategic
78.96=0.40
79.19+1.35
87.49
84.17
83.22
Strategic
87.90 87.90=0.35
88.4041..v0
95.22
88.58 4:2.7
90.27 +2.35
Qwen3
Non-Strategic
80.62
79.90+1.30
89.51 60.69
83.28
$3.97
Strategic
88.2 88.24-0.299
89.04
94.12-00
88.92+:2.0
90.84+2.44
Mixtral
Non-Strategic
80. 80.12 80.75
90.34
83.42+2..7
84.21=2.50
Provided by Anthropic via their API, Claude-3.7 emphasizes safety and alignment, making it a strong baseline for stable classification under strategic contexts.
Claude-3.7 [3:
Gemini-2.5 65]
Offered by Google Cloud, Gemini models are equipped for multimodal un- derstanding. We utilize Gemini-2.5 through Vertex AI API for tabular strategic classification
tasks.
LLama-3.3 49
An open-source model by Meta, available via API endpoints and Hug- gingFace, used here in its instruction-tume -uned form (70B variant when available).
Mixtral 38 A sparse mixture-of-experts model combining multiple expert networks, suitable for dynamic contexts and scalable SC evaluations.
Qwen3 [3] Provided by Alibaba Cloud, Qwen3 supports multilingual instruction following and robust handling of tabular and structured prompts. Integrated through the DashScope API.
All models are run with consistent hyperparameters across experiments. Prompt formatting is standardized to minimize variance due to stylistic differences in input-output formatting.
JJ3 Prompt Design
Effective prompt design is essential enable in-context learning (ICL) for strategic classification (SC). We construct prompts that integrate both manipulation-awal and manipulation-agnossi settings while maintaining consistent structure. A representative prompt example shown below, illustrating (i) task setup. (ii) in-context examples, and (iii batch evaluation.
26

---

(i) Task Definition. The prompt begins with an instruction header describing the SC setup, consis- tent with the theoretical formulation in Section 2.1
You are strategic classification assistant. In this scenario:
There are two players: a decision maker and decision subjects. The decision maker publishes its policy (classification rule f). The decision subjects, after observing the policy and associated costs, determine whether strategically modify their features.
Specifically, the decision maker defines classifier mapping feature vectors to binary outcomes y E {0, 1}. Once the rule is known, individuals may modify their features obtain a favorable decision. Such modification incurs a cost, quantified by cost function.
The strategic manipulation rule for decision subjects is: (see Definition 2.1). The optimization rule for the decision maker (see Definition 2.2).
Please restate your understanding of the strategic classification setting in concise terms.
(ii) In-context Examples. Following the instruction, series of labeled demonstration examples (typically 12-24) are provided to simulate the adaptation process. Each example includes both the initial features and the resulting classification outcome. In the strategic condition, the feature set reflects manipulation based on our theoretical updates, while in non-strategic cases, the original features used.
Example 1:
Initial features:
age: 34
workclass: Private
fnlwgt: 203034
education: Bachelors
education-num:
marital-status: Separated
occupation: Sales
relationship: Not-in-family
race: White
sex: Male
capital-gain: 0
capital-loss: 2824
hours-per-week: 50
cotive-country: United-States
Initial result: income >50K
Example k:
(iii) Batch Evaluation. For lcage evaluation, prompts programmatically generated to include a batch of test instances. Each prompt instructs the LLM to (1) apply the manipulation rule beneficial, (2) update its decision rule accordingly, and (3) output the resulting accuracy.
27

---

Next, will provide you with series of applicant cases. For each, please: 1. Apply the rules above to strategically manipulate the applicant's features if beneficial. 2. Update your decision-making rules as per the definitions above. 3. For each applicant, after strategic manipulation, the true label should remain unchanged. Finally, report the accuracy rate under your classification rules.
Test examples:
Implementation Note. In our implementation, both inner (strategic manipulation) and outer (decision adaptation) processes are unified within a single prompt, allowing the LLM to jointly simulate the two-level optimization cycle in one inference pass.
J.4 Illustrative Example of Bi-level Optimization via GLIM
To illustrate how our GLIM framework operates within an LLM, we consider credit approval task as an example.
Given several in-context examples of applicants with varying financial histories and approval out- comes, the LLM implicitly infers decision rule. Some features may exert a stronger influence (e.g.. recent payment behavior), while others contribute less, shaping the internal classification boundary through attention dynamics.
When new agent is presented, the LLM not only predicts the approval outcome but also implicitly anticipates how the applicant might strategically manipulate certain features (e.g., reducing overdue counts or increasing recent payments) to receive a favorable decision.
Through repeated exposure to strategically manipulated examples in the prompt. the LLM implicitly adjusts its decision rule toward more stable and robust form. This corresponds to the outer-stage optimization, while the simulated feature changes capture the inner-stage manipulation, together completing the bi-level strategic classification process.
This example simulates both stages of strategic classification through forward-only inference, all without parameter tuning, relying entirely on in-context learning.
J.5 Implementation Details
The experimental pipeline is implemented in Python, with integration across multiple LLM APIs. Our infrastructure supports large-scale evaluation while ensuring reproducibility. Official SDKs (e.g.. openai, anthropic, google-lloud-aiplafoone are used to interface with model APIs. All keys securely stored via encrypted environment variables. A modular prompt generator selects appropriate examples and formats based dataset type, manipulation setting, and model constraints. This allows easy extensibility to new datasets or model variants.
K
Additional Experimental Results
We apply proposed ICL-based approach, GLIM, across multiple large language model APIs and document the detailed results in Tables and
In this comparison, we note that the baseline models used in our experiments, such as linear models and shallow neural networks, are optimized through traditional training procedures involving parame- ter updates. In contrast, GLIM operates in zero-update regime, leveraging the inherent in-context reasoning capabilities of pre-trained L.LMs. This distinction reflects difference in mechanism rather than in model capacity.
28

---

L Discussion on Our Method
L.1 Prompting Cost for Strategic Manipulation of Agent
One potential concern when adapting strategic classification (SC) to large language models (LLMs) lies in the cost of interaction, particularly the computational and structural overhead associated with in-context learning (ICL). In classical SC literature. agent-side manipulation cost is often modeled as transformation cost over input features, such as Mahalanobis Lp norms. contrast, the LLM setting introduces a new dimension: the cost of prompts for agents.
In our framework, however, intentionally abstract away the prompting cost by adopting a percep- tual prediction view of agent behavior, consistent with recent theoretical perspectives in strategic learning [50 [1]. From this perspective, agents modeled as cognitive entities who respond by adjusting their feature representations. The prompt cost, being fixed system, level expense unrelated individual feature manipulation, does not influence the agent's 's strategic calculus.
Moreover, in real-world deployments, prompt construction and transmission are typically handled by system infrastructure or shared communication protocols, incurring negligible marginal cost for individual agents. In contrast, modifying one's 's features (e.g.. improving test scores, altering financial statements) entails significant personal effort risk. Therefore, although we acknowledge that prompting costs may be relevant in certain system-level analyses, they play no substantive role in the agent-level incentive structure we seek model. Hence, choose omit prompt cost from our formal analysis.
L.2 Discussion on Theory-Practice Divergence at Scale
As shown in Figures 3(b) and 6(b) the cosine similarity tends to decrease over iterations, indicating a divergence between our theoretical analysis and the empirical results at scale. This divergence may be attributable to factors such model capacity, data distribution shifts, or nonlinearities in real-world tasks.
We acknowledge the existence of this divergence and further analyze this phenomenon in future work better understand the conditions under which such divergences occur.
L.3 Discussion on API Cost Analysis of GLIM
A key practical limitation of our approach is its reliance on proprietary large language models that are accessed via commercial APIs, such as OpenAI GPT-4o and DeepSeek. Unlike traditional machine learning models, which can be trained or deployed locally with fixed hardware budget, our method depends on repeated calls remote L.M API services.
In our experiments, we observed that the total inference cost is not constant but grows with the number of API requests. Each additional example increases the total token count in roughly proportional manner, resulting in higher overall expenses. This observation highlights the need to account for API-related cost structures when designing systems that involve frequent or large-scale model queries. Such variability is inherent to current commercial LLM platforms and represents a fundamental constraint for real-world deployment.
To mitigate this limitation, several directions can be explored. One potential strategy is to design more cost-efficient prompt engineering methods that reduce the number of required API calls without compromising performance. However, in practice, the dynamic nature of agent distributions and context diversity makes strict control over prompt volume challenging. Another promising direction the development of hybrid systems, where -LMeed reasoning is selectively applied to complex or ambiguous cases, while lightweight local models handle routine or latency-sensitive requests. This hybrid paradigm could significantly reduce dependence on commercial APIs while maintaining flexibility and adaptability.
Addressing these practical constraints will be an important avenue for future work, as we aim to optimize the trade-off between the flexibility of LLW-eeee reasoning and the cost-effectiveness required for sustainable large-scale deployment.
29

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly reflect the intent and scope of the paper, and the contributions are stated in three parts.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations and boundaries of this work all discussed.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
30

---

Justification:
We provide the full set of assumptions and a complete proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We explicitly state the functions and publicly available datasets used and any information needed to reproduce the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
31

---

Answer: [Yes]
Justification: We use LLM APIs and provide open access to our preprocessed data, interme- diate datasets, and part of our code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
CooeeSumiiii isiiiissionn mnylicyy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify the training and test details (e.g., data splits, hyperpa- rameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? item[] Answer: [Yes]
Question:
All details and necessary information understand the results have been provided.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the errors the experiments and calculate the standard deviations for the main experiments, presented in tables and figures.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
32

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We use LLM APIs and our computing resources are described in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform. in every respect, with the NeurIPS Code of Ethics http:/ttpss//.ao......o/ooo////a
Answer: [Yes]
Justification: respects.
We have reviewed the NeurIPS Code of Ethics and ensure compliance in every
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification; We have discussed both potential positive societal impacts and negative societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
33

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the original paper about the code and datasets we have used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
34

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: There is new asset introduced.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
No research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
35

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The method of this paper is our original and the LLM is used only for correcting writing.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
36


Reviews:
[{"review_id": "FROJyNHFm7", "reviewer": "Reviewer_Jak5", "summary": "They investigate how to use LLMs to design a more scalable and efficient strategic classification(SC) framework. Their model GLIM is a gradient free SC method grounded in in-context learning. \n\nStrategic classification is a bilevel optimization problem, where individuals best-respond to a deployed classifier in order to maximize their payoff. Then the classifier is updated in order to maximize its accuracy considering best-responses by the individuals. Traditional SC approaches often rely on iterative retraining or gradient updates to remain robust, which becomes computationally expensive and infeasible at scale. However, it is very costly to retrain LLMs. Hence, they  propose a novel gradient-free method that leverages in-context learning (ICL) in LLMs to perform strategic classification without updating model parameters. They show how LLMs leveraging in-context learning can implicitly simulate both the strategic manipulation and decision rule optimization stages of the SC bi-level problem, without any fine-tuning.\n\n\nThey show it theoretically, how their gradient-fee in-context learning model (GLIM) simulates this bi-level optimization process, including both the feature manipulation and decision rule optimization. That being said, unlike traditional SC approaches that often rely on iterative retraining, or gradient updates to remain robust, their method is retraining free and hence more efficient.", "questions": "What does GLIM stand for? Gradient-free Learning In-context Model?\n\nLine 117, A is not defined.\n\nIn Prop 1,2, showing the existence of such PVK matrices does not imply that your model would learn the same matrices. You would need to argue that your learning procedure would converge to desirable PVK matrices.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "One of your main contributions is a theoretical justification for how LLMs equipped with ICL can simulate the strategic manipulation of agents. However, that does not seem too surprising or novel to me. The proof uses two arguments first, the forward propagation in LLMs—particularly through linear self-attention layers—can be interpreted as performing implicit gradient descent (GD) (this is known from prior work). Second,  solving optimum manipulation in strategic manipulation may be viewed as a gradient-descent step (this was known before too). Putting this two arguments shows that ICL can simulate the strategic manipulation of agents. This seems like a straightforward argument to me.\n\nSimilarly, I am not too surprised by line 200 and Prop 2, your second main contribution. Since we already know that the forward\npropagation in LLMs can be interpreted as performing implicit gradient descent (GD). Taking a gradient step is exactly what we do in the outer optimization step of SC.\n\nPlease correct me if I am wrong or missing something here.", "was_revised": true, "final_justification": "They addressed all my questions during the rebuttal and I stay positive. Their results are not too surprising, but still an interesting perspective of connecting ICL with SC.", "rebuttal": "We sincerely appreciate your constructive comments and your positive evaluation of our work. We would like to address your concerns as follows:\n\n>  **Weaknesses**\n\n**Response:**\n- We would like to clarify that **the core positioning of our work** is not as a purely theoretical study of ICL mechanisms but as a pioneering attempt to bridge strategic classification and LLMs via ICL.\n\n- **Strategic classification (SC) is fundamentally a bi-level, game-theoretic problem** involving both agent-side manipulation and decision-maker adaptation, and **has been studied largely separately from ICL/LLM literature**.\n\n  - To our best knowledge, **our work is the first to employ LLMs to model and solve the bi-level, game-theoretic optimization structure of SC**.\n  - It enables **a scalable and retraining-free approach to large-scale SC tasks**, where classical gradient-based retraining requires excessive computational resources and time.\n\n- From a broader social science perspective,  our work establishes a crucial bridge between large language models and strategic machine learning:\n\n  - **A wide range of social science applications are increasingly converging with machine learning and LLMs**, including fields such as computational social science, policy evaluation, and behavioral economics[1,2,3,4].\n  - Our work provides **a foundational contribution** by connecting LLMs with game-theoretic frameworks. \n  - We hope this foundation will facilitate future interdisciplinary research and applications of LLMs in societal decision-making contexts.\n\n- We have clarified this part in our revised paper at Lines 62-64 (Introduction).\n\n----\n\n> **Question 1**.What does GLIM stand for? \n\n**Response:**\n\n- GLIM stands for  **G**radient-free **L**earning **I**n-context **M**ethod. \n\n- We have clarified this part in Lines 65-67 (Introduction) of our revised paper.\n\n  \n----\n\n> **Question 2**. Line 117, A is not defined.\n\n**Response:**\n\n- In Lemma 1, $A_\\ell$ denotes the step size used in the gradient descent update at layer $\\ell$. \n- We have clarified this lemma in Lines 117-118 of the revised paper.\n\n\n---\n> **Question 3**. Showing the existence of such PVK matrices does not imply that your model would learn the same matrices. You would need to argue that your learning procedure would converge to desirable PVK matrices.\n\n**Response:**\n\n- We thank the reviewer for highlighting this point. Our existence proofs demonstrate that attention mechanisms in LLMs are expressive enough to realize PVK matrices that simulate gradient-based updates.\n- We would like to note that \n  - Recent theoretical work[5,6] has proven **that attention weights in pretrained Transformers do converge to such optimization-relevant matrices during in-context learning**, thereby providing theoretical support for our approach.\n  - Therefore, after establishing the existence of suitable PVK matrices, the ICL procedure can indeed converge to such desirable matrices in practice. \n- We have clarified this part in Appendix D (Lines 561-563) and Appendix F (Lines 617-619) of our revised paper.\n\n----\n[1] Ariel Flint Ashery *et al.* Emergent social conventions and collective bias in LLM populations. Sci. Adv.11,eadu9368(2025).\n\n[2] Gao, Chen, et al. \"Large language models empowered agent-based modeling and simulation: A survey and perspectives.\" *Humanities and Social Sciences Communications* 11.1 (2024): 1-24.\n\n[3] De Curtò, J., and I. De Zarzà. \"LLM-Driven Social Influence for Cooperative Behavior in Multi-Agent Systems.\" *IEEE Access* (2025).\n\n[4] Ziems, Caleb, et al. \"Can large language models transform computational social science?.\" *Computational Linguistics* 50.1 (2024): 237-291.\n\n[5] Ahn, Kwangjun, et al. \"Transformers learn to implement preconditioned gradient descent for in-context learning.\" *Advances in Neural Information Processing Systems* 36 (2023): 45614-45650.\n\n[6] Gatmiry, Khashayar, et al. \"Can looped transformers learn to implement multi-step gradient descent for in-context learning?\" *arXiv preprint arXiv:2410.08292* (2024)."}, {"review_id": "XNqiBYOvrn", "reviewer": "Reviewer_ZRpu", "summary": "This paper proposes GLIM (Gradient-free Learning In-context Method), which applies large language models to strategic classification problems. Building on the theoretical framework that in-context learning (ICL) implements implicit gradient descent, the authors present a mathematical analysis showing how both stages of strategic classification—inner optimization and outer optimization —can be realized through ICL. The empirical evaluation attempts to validate this theory by demonstrating that ICL-induced updates converge to gradient descent solutions, while showing that GLIM achieves superior classification accuracy compared to traditional linear models and MLPs.", "questions": "Please see weaknesses part in above section.", "limitations": "Authors provide sufficient discussion about the limitations.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The idea of connecting language models to strategic classification is intriguing.\n\n- The paper is well structured and clear written.\n\n**Weaknesses**\n\nTheoretical Limitations\n\n- **Oversimplified transformer architecture:** The theoretical analysis in the appendix reduces transformers to single-layer attention mechanisms, ignoring (1) the multi-layer structure and (2) MLP layers, which constitute a significant portion of the parameter budget and computational capacity.\n\n- **Linear attention assumption:** The theoretical framework in Appendix D assumes linear attention, while many tested language models employ nonlinear attention with softmax. Despite the authors' attempts to address this gap in Appendix I, the justifications remain unconvincing:\n\n  - **Softmax as Adaptive Linear Combination:**  The softmax constraint (positive weights summing to one) fundamentally prevents the construction of arbitrary gradient directions, particularly those requiring negative coefficients or scaling beyond unit norm.\n\n  - **Prompt-based Gradient Encoding:** The claim that careful prompt design can approximate gradient descent lacks theoretical foundation and does not address the fundamental nonlinearity introduced by softmax.\n\n  - **Empirical and Theoretical Support:** While cited work [11] shows transformers can implement functional gradient descent, this does not validate the specific linear constructions required for the authors' theoretical framework.\n\n- **Homogeneous label assumption:** In Equation (45), the homogeneity assumption is unlikely to hold in practice. It is unrealistic to assume that a trained classifier would label every example identically, and under such conditions, it is unclear whether a meaningful decision boundary exists. Additionally, the derivation from Equation (44) to Equation (45) implicitly requires that ⟨xᵢ, xⱼ⟩ = 1, which implies:\n\n  - The input query has exactly the same features as the context examples, which is unlikely in real-world scenarios.\n\n  - All context examples are identical, which contradicts the typical goal of using diverse context examples.\n\nConcerns of Experiments \n\n- The comparison between language models and the baseline models (linear and MLP) appears to be unfair. The language models benefit from extensive pretraining and significantly larger parameter counts, while the baselines are trained from scratch with limited capacity.\n\n- In Figures 3(b) and 6(b), the cosine similarity appears to *decrease* over iterations, which contradicts the claim that the method is effective at large scales.\n\n- The paper does not clearly explain how the in-context learning examples are updated across iterations. A concrete example illustrating how these examples evolve would improve clarity. Additionally, since some of the tested models are closed-source, it is unclear how updates are computed without access to internal parameters or states.", "was_revised": true, "final_justification": "The authors provide sufficient evidence to address my concerns.", "rebuttal": "Thank you for valuable comments. We would like to address your concerns as follows:\n> **Oversimplified transformer architecture**\n>\n> **Linear attention assumption**\n\n**Response:**\n\nWe note that these two comments are closely related. We clarify them together below.\n\n- Our use of a single-layer, linear attention model **is not an oversimplification**, but **a widely accepted theoretical convention** in both ICL and transformer analysis.\n  - As **summarized in the table below**, this analytical simplification (single-layer and linear) is foundational in many highly cited works on ICL and transformer theory.\n- Our work is not intended as a general-purpose LLM application, but **as a targeted contribution that explicitly bridges ICL theory with strategic classification**.\n  - **As highlighted in the second row of the table** below, nearly all foundational research in strategic classification adopts linear models for their interpretability and analytical clarity. \n  - Our focus on single-layer, linear attention is therefore **a deliberate and justified choice that is well aligned with established modeling practice in both domains**, and is particularly suited to the requirements of social science and practical applications.\n  |Representative Works|Typical methodology / Model|\n  |--|--|\n  |ICL/Transformer Theory[1,2,3,4,9]|Single-layer attention, mainly linear|\n  |Strategic Classification[5-8]|Linear model|\n- **Adding multilayer attention or MLPs does not undermine the core connection** between ICL and gradient-based updates.\n  - As shown in the table, our analysis is not “oversimplified,” but instead distills the most essential and theoretically transparent part of the mechanism. \n  - MLP layers only apply local nonlinearities and do not interfere with the connection between attention and gradient-based updates[1,3].\n- **The gradient-approximation ability of Softmax attention is theoretically guaranteed** [11,12].\n  - Recent work shows that softmax attention can closely approximate any target direction within the convex hull of the value vectors[11]:\n  $$\n  \\mathrm{Attn_{Softmax}}(Q, K, V) \\approx \\sum_{i=1}^N \\alpha_i v_i,\\quad \\sum_{i=1}^N \\alpha_i = 1,\\, \\alpha_i \\geq 0\n  $$\n  - For any target direction $g$ in the convex hull of $\\{v_i\\}$, there exists weights $\\{\\alpha_i\\}$ such that\n  $$\n  \\left\\| \\sum_{i=1}^N \\alpha_i v_i - g \\right\\| \\leq \\epsilon\n  $$\n  - This provides a rigorous upper bound on the approximation error, supporting the theoretical basis for prompt-based gradient encoding with softmax attention [11,12].\n- Prompt design can make attention mechanisms approximate gradient descent **is in fact well supported by recent theoretical work**. \n  - Notably, **several recent studies[2,4,11]** have rigorously established that, with appropriate prompt construction, Transformers can perform gradient descent in both linear and nonlinear settings:\n- **Our linear construction is theoretically well-founded**. \n  - Recent theoretical work[11-14] has demonstrated that **nonlinear attention with sufficient layers and heads can approximate any optimization trajectory expressible by linear attention**, up to arbitrarily small error.\n  - Moreover, prior work[10] has established **the convergence of Transformer learning dynamics**, demonstrating that nonlinear Transformers can reliably simulate the linear constructions in our framework.\n\n---\n> **Homogeneous label assumption**\n\n**Response:**\n- **The homogeneous assumption is a reasonable theoretical simplification for mechanism illustration:**\n  - It allows us to **isolate and transparently demonstrate the effect** of the attention mechanism, showing how, under this setting, the attention update precisely aligns with the gradient descent direction.\n  - Such assumptions are common in theoretical work to reveal idealized behaviors, **serving as an “upper bound” for mechanism capability**.\n- **This conclusion remains valid** without this assumption:\n  - When **features and labels are heterogeneous (as in practice)**, the attention update generalizes to a weighted sum of local gradient directions, reflecting the similarity between each context and the query:\n  $$\n  \\Delta x_j^{\\mathrm{ICL}} = \\frac{A\\eta}{N} \\sum_{i=1}^N (1-y_i) W^\\top \\langle x_i, x_j \\rangle,\n  $$\n  - More generally, \n  $$\n  \\Delta x_j^{\\text{ICL}} = \\sum_{i=1}^N \\alpha_{ji} \\cdot g(x_i, y_i),\n  $$\n  ​\twith $\\alpha_{ji}$ as the attention weights and $g(x_i, y_i)$ as the local gradient directions.\n  - The expected **error between ICL and GD can be tightly bound**:\n  - $$\n    \\epsilon_j = \\left\\| \\Delta x_j^{\\text{ICL}} - \\nabla_x L(x_j) \\right\\|,\\quad\n    \\mathbb{E}[\\epsilon_j] \\leq C \\cdot \\sqrt{\\frac{1}{N}} + \\delta,\n    $$\n    where $C$ is a constant and $\\delta$ is the statistical error from distribution mismatch.\n- We have clarified this part in Appendix F of our revised paper.\n\n---\n> **Concerns of Experiments1**\n\n**Response:**\n- The results in Table 2 are **not intended as a direct comparison of raw performance between traditional baselines and pre-trained LLMs**. Rather, the purpose is to demonstrate that **pre-trained LLMs are capable of solving the SC problem**.\n- As stated in Appendix A, our work aims to **explore an alternative solution** for large-scale strategic classification scenarios, where classical gradient-based retraining requires excessive computational resources and time.\n- For comparisons **between different LLM APIs**, we ensure fairness by aligning pre-training LLM scale as closely as possible.\n  -  The following table summarizes the key properties evaluated in our experiments (except for LLMs whose specific parameters have not been made public):\n  |Model| Parameter (B)|\n  |-|-|\n  |DeepSeek-V3|67|\n  |LLama-3.3|70|\n  |Qwen3|72|\n  |Mixtral|46.7|\n- We have clarified this part in Lines 265-269 of our revised paper.\n\n\n> **Concerns of Experiments2**\n\n**Response:**\n- We would like to clarify that the observed decrease and fluctuation in cosine similarity **does not contradict** our claims of method effectiveness; **rather, it reflects the natural challenges** of large-scale, dynamic distributions.\n  - As the data becomes larger and more diverse, **it becomes increasingly difficult for traditional models with GD to maintain stable** feature or strategy representations.\n  - In contrast, LLMs guided by **ICL exhibit more robust adaptation and can better handle** the complexities of large-scale data. Therefore, there is a small decline in cosine similarity.\n- Notably, **a similar trend can be observed in Figure 5(b)**. These phenomena reflect the gap in large-scale data adaptation between LLMs and traditional GD models.\n\n\n> **Concerns of Experiments** Some of the tested models are closed-source\n\n**Response:**\n- We agree that some tested LLMs are closed-source, but we **design multiple experiments directly comparing the attention layers update with the theoretical SC optimization process**. \n- In recent ICL research, it is accessible to use prompt-based probing and input-output analysis to verify mechanistic conclusions[15-17].\n\n\n> **Concerns of Experiments** prompt example\n\n**Response:**\n- We provide a simple prompt example\n  - **Task Definition**: This part is aligned with Section 2.1 of our paper:\n  ```plaintext\n  You are a strategic classification assistant. In this scenario:\n  There are two players: a decision maker and decision subjects.\n  - The basic interaction rule ... (Section 2.1)\n  - The strategic manipulation rule is: ... (see Definition 2.1).\n  - The optimization rule is: ... (see Definition 2.2).\n  Restate your understanding.\n  ```\n  - **In-context Examples**: from Adult dataset:\n   ```plaintext\n    Example 1:\n    Initial features:\n       age: 34\n       workclass: Private\n         ...\n       native-country: United-States\n     Initial result: income >50K\n     \n    Example 2:...\n    ```\n  - **Batch Testing with Dataset** \n\n   ```plaintext\n   prompt = \"\"\"\n   1. Apply the rules above to strategically manipulate the applicant's features if beneficial.\n   2. Update your decision-making rules as per the definitions above.\n   3. For each applicant, even after strategic manipulation, the true label should remain unchanged.\n   Finally, report the accuracy rate under your classification rules.\n   test_examples: ...\n   \"\"\"\n   ```\n---\n[1] Von Oswald, Johannes, et al. \"Transformers learn in-context by gradient descent.\"\n\n[2] Garg, Shivam, et al. \"What can transformers learn in-context? a case study of simple function classes.\" \n\n[3] Akyürek, Ekin, et al. \"What learning algorithm is in-context learning? Investigations with linear models.\" \n\n[4] Clark, Peter, Oyvind Tafjord, and Kyle Richardson. \"Transformers as soft reasoners over language.\" \n\n[5] Hardt, Moritz, et al. \"Strategic classification.\"\n\n[6] Miller, John, Smitha Milli, and Moritz Hardt. \"Strategic classification is causal modeling in disguise.\" \n\n[7] Levanon, Sagi, and Nir Rosenfeld. \"Strategic classification made practical.\" \n\n[8] Milli, Smitha, et al. \"The social cost of strategic classification.\" \n\n[9] Kratsios, Anastasis, et al. \"Universal approximation under constraints is possible with transformers.\"\n\n[10] Ahn, Kwangjun, et al. \"Transformers learn to implement preconditioned gradient descent for in-context learning.\" \n\n[11] Xiang Cheng, Yuxin Chen, and Suvrit Sra. ''Transformers implement functional gradient descent to learn non-linear functions in context.''\n\n[12] Franco, Luca, et al. \"Under the hood of transformer networks for trajectory forecasting.\" \n\n[13] Li, Hongkang, et al. \"How do nonlinear transformers learn and generalize in in-context learning?\" \n\n[14] Gatmiry, Khashayar, et al. \"Can looped transformers learn to implement multi-step gradient descent for in-context learning?\" \n\n[15] Wei, Jason, et al. \"Emergent abilities of large language models.\"\n\n[16] Min, Sewon, et al. \"Metaicl: Learning to learn in context.\"\n\n[17] Zhao, Zihao, et al. \"Calibrate before use: Improving few-shot performance of language models.\""}, {"review_id": "rWjik0dq4M", "reviewer": "Reviewer_MEur", "summary": "This work proposes leveraging LLMs to tackle strategic classification (SC), with the goal of providing a more scalable alternative to existing methods for adapting automated decision making to adversarial exploitation. In particular, rather than traditional model retraining/finetuning, it proposes leveraging in-context learning to simulate the typical bi-level adversarial SC process. The authors provide some theoretical analysis on SC with in-context learning and evaluate GLIM, the resulting algorithm, on phishing, fraud detection, and other synthetic benchmarks inspired by real-world scenarios.", "questions": "I would encourage the authors to address my concerns raised above. Currently, I believe that, without an explicit and clear algorithmic description, to ensure the methodology is clear and reproducible, this work is not yet ready for acceptance.", "limitations": "The limitations regarding the large assumptions made in the theoretical analysis are not discussed. Moreover, the discussion of the prompting costs provided in Section L seems to me potentially misleading, as outlined in my criticism above.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths.\n\n1. The authors provide concise and effective introductions to the mathematical frameworks of the SC optimization and ICL as implicit gradient descent in Section 2, deferring unnecessary mathematical details to the Appendix.\n2. The experiments are relevant to the domain. tackling different real-world problems and comparing several closed-source LLMs.\n3. The high-level connection between ICL and the SC problem is simple and intuitive.\n\nWeaknesses and areas for improvement.\n\n1. I did not find the methodology and actual algorithm to be clearly presented. There are a few \"high-level\" details in Appendix J that are far from sufficient for reproducibility. I would strongly suggest adding the exact prompts used for each stage of strategic classification for the problems and a clear algorithm block describing the control flow. I think this should be a separate new section in the main paper, explicitly describing the proposed GLIM in practice, which is currently missing.\n2. I found the theoretical analysis provided in Section 3 to be very weak, with the actual implications of propositions 1/2 not matching the claims in the introduction such as \"We theoretically establish, for the first time, how LLMs leveraging in-context learning can implicitly simulate both the strategic manipulation and decision rule optimization stages of the SC bi-level problem, without any fine-tuning.\" (lines 62-64). To me, Section 3 does not actually seem to do any novel analysis beyond the frameworks of prior work, as I think the observation that there could exist parameters, queries, and values that simulate SC with ICL should be mostly obvious. I also fail to see how this leads to some of the remarks, such as \"This establishes a constructive equivalence  between explicit strategic manipulation and attention-driven ICL behavior, thereby grounding ICL as a forward-only approximation of inner-stage optimization in SC.\" (174-176). I would really suggest rewriting and removing much of the unnecessary content in this Section, simply noting the intuitive connection between ICL and GD updates, which could allow, given specific weights and features, to simulate SC.\n3. The methodology has several immediate limitations that are not appropriately discussed. First, adding in-context samples should notably increase the already high inference costs of using LLMs, which I believe are far from \"negligible\" (Appendix L) and can be higher than finetuning the model once, in practice (since the model has to process N times the amount of input tokens). To this end, I would have liked to see an analysis of how these costs scale in practice as a function of the in-context sample (e.g., the API costs). Second, since LLMs are available to the public, users could potentially optimize their features in a way that is adversarial to the LLM parameters P, for any practical choice of context.\n4. In contrast to what was stated in Question 5 of the checklist, the provided code is partial (only 2 Python files, and some CSVs are provided, without even any kind of instructions) and far from allowing any sort of reproducibility.", "was_revised": true, "final_justification": "Following our discussion, the authors mostly addressed my main concerns about presentation, toning down the claims, acknowledging the proposed method's limitations, and providing additional details for reproducibility.\n\nI believe the paper has a moderate contribution in one specific area of AI, and the reasons to accept outweigh the reasons to reject.", "rebuttal": "Thank you for your comments. We would like to address your concerns as follows:\n\n\n> **Weakness1**. I did not find the methodology and actual algorithm to be clearly presented.\n\n**Response:**\n\n- We would like to clarify that our work is a **pioneering attempt to bridge strategic classification (SC) and LLMs via in-context learning (ICL)—far beyond simply applying LLMs to SC tasks**. \n\n  - **The explicit mapping** of SC’s bi-level framework and  game-theoretic structure onto the attention mechanisms have been details rigorously **presented in Appendices E, F, G, and I**.\n  - To demonstrate the reproducibility of our work, we have provided relevant experimental code, including *Attention_layers.py* and *SC.py*.\n\n- For additional clarity, we provide below **an algorithmic outline of the GLIM** framework and **a simple prompt example** (based on our Appendix J) **at the bottom**. \n\n  **Algorithm 1: The control flow of GLIM**\n\n  **Input:**\n\n  - Dataset $\\mathcal{D} = \\{(x_j, y_j)\\}_{j=1}^N$\n  - Pre-trained LLM $\\phi$\n  - Prompt template and SC rules\n\n  **Procedure:**\n\n  - Provide SC rule prompts and example prompts\n  - Guide strategic manipulation of $x$\n  - Guide decision rule optimization\n  - Query: Manipulated $\\{x'_j\\}$ and predicted  $\\{\\hat{y}_j\\}$ with prompts\n\n  **Output:**\n\n  - Predicted labels $\\{\\hat{y}_j\\}$ and accuracy\n  - Manipulated features $\\{x'_j\\}$（optional）\n\n- We have clarified this part in our Appendix J of our revised paper.\n\n---\n\n> **Weakness2**. The theoretical analysis provided in Section 3...\n\n**Response:**\n\nWe would like to clarify that the core positioning of our work is **not as a purely theoretical study of ICL mechanisms**, but as a pioneering attempt to bridge strategic classification and LLMs via ICL.\n\n- **Strategic classification (SC) is fundamentally a bi-level, game-theoretic problem** involving both agent-side manipulation and decision-maker adaptation, and **has been studied largely separately from ICL/LLM literature**.\n\n  - To our best knowledge, **our work is the first to employ LLMs to model and solve the bi-level, game-theoretic optimization structure of SC**.\n  - It enables **a scalable and retraining-free approach to large-scale SC tasks**, where classical gradient-based retraining requires excessive computational resources and time.\n\n- **From a broader social science perspective,**  our work establishes a crucial bridge between large language models and strategic machine learning:\n\n  - **A wide range of social science applications are increasingly converging with machine learning and LLMs**, including fields such as computational social science, policy evaluation, and behavioral economics[1,2,3,4].\n  - Our work provides **a foundational contribution** by connecting ICL in LLMs with game-theoretic learning frameworks. \n  - This foundation will **facilitate future interdisciplinary research and applications of LLMs** in societal decision-making contexts.\n\n- To be specific, we entail a detailed theoretical analysis of SC-motivated ICL derivation, including strategic manipulation of agent and decision optimization for jury.\n\n- **Furthermore**, our experimental results further validate these insights, demonstrating that LLMs guided by in-context learning **robustly handle the core dynamics of strategic classification**, even at scale and without retraining.\n\n  \n\n----\n\n>**Weakness3.1**  Adding in-context samples should notably increase the already high inference costs of using LLMs\n\n**Response:**\n\n- The reviewer refer to “**prompt cost**” as the computational and API expenses incurred when querying the LLM, i.e., the resource cost of processing additional in-context samples (e.g., API fees and token usage).\n- In contrast, our use of **prompt cost** (as explicitly stated in Appendix L, Line 819) refers to the **agent's cost of obtaining strategic guidance by querying the LLM during feature manipulation**, e.g., exploring alternative manipulation methods with LLMs.\n  - This prompt cost is negligible compared to the substantial manipulation cost required for actual improvement of individual features (such as earning a degree or increasing income).\n- The distinction is crucial: \n  - Our definition of prompt cost is **rooted in the SC framework**, focusing on **the agent’s perspective**, the behavioral and informational costs incurred when agents seek to manipulate their features.\n  - The reviewer’s definition adopts a **system-level perspective**, emphasizing the computational and resource costs of LLM API usage.\n- We evaluated the API usage and cost for varying numbers of feature examples as well.\n  - Use official OpenAI pricing, and present the results in the following table:\n\n| Number of Prompt Example | API cost  |\n| ------------------------ | --------- |\n| 10                       | $0.001629 |\n| 100                      | $0.005235 |\n| 1000                     | $0.045210 |\n| 10000                    | $0.395723 |\n| 50000                    | $1.613182 |\n| 100000                   | $2.876050 |\n| 200000                   | $5.102118 |\n\n\n----\n\n>**Weakness3.2** Since LLMs are available to the public, users could potentially optimize their features in a way that is adversarial to the LLM parameters P, for any practical choice of context.\n\n**Response:**\n\n- As SC is specifically proposed to address adversarial manipulation, **our work has already handled this concern**. \n\n  - The possibility of agents adversarially (strategically)  manipulating their features is **the foundation of strategic classification**: the game-theoretic interaction between agents and the jury.  \n\n  - Our framework explicitly **incorporates this adversarial dynamic by modeling agent strategic manipulation** through in-context learning, \n\n  - Such behavior is not a vulnerability, but rather one of the central subjects our approach is designed to address.\n\n    \n----\n\n> **Weakness4** Reproducibility questions\n\n**Response:**\n\n- The code we provided focuses on verifying our analysis of the **mapping between the attention mechanism and the bi-level optimization framework** of strategic classification.\n\n  - The code includes standard SC model and the core attention module.\n\n- For practical reproducibility, we have included detailed prompt templates:\n\n- illustrating (i) strategic classification (SC) task setup, (ii) in-context interaction examples, and (iii) batch evaluation using the dataset.\n\n  - **(i) Task Definition**: This part is aligned with Section 2.1 of our paper:\n\n  ```plaintext\n  You are a strategic classification assistant. In this scenario:\n  \n  There are two players: a decision maker and decision subjects.  \n  - The decision maker publishes its policy (classification rule $f$).  \n  - The decision subjects, after observing the policy and associated costs, determine whether to strategically modify their features.  \n  - The decision outcome is binary (0 or 1).\n  \n  The strategic manipulation rule is: ... (see Definition 2.1 in Section 2.1).\n  The optimization rule is: ... (see Definition 2.2 in Section 2.1).\n  \n  Please restate your understanding of the strategic classification setting in concise terms.\n  ```\n\n  - **In-context Examples**: Here is an example using a real sample from the Adult dataset (multiple such examples are recommended for best results):\n\n  - ```plaintext\n    Example 1:\n    Initial features:\n       - age: 34\n       - workclass: Private\n         ...\n         ...\n       - native-country: United-States\n     Initial result: income >50K\n     \n    Example 2:\n    ...\n    ```\n\n  - **Batch Testing with Dataset** For large-scale evaluation, we load the entire dataset and generate prompts for each test case as follows:\n\n    ```plaintext\n    with open('adult.csv', 'r') as f:\n        test_examples = f.read()\n    \n    prompt = \"\"\"\n    Next, I will provide you with a series of applicant cases. For each, please:\n    1. Apply the rules above to strategically manipulate the applicant's features if beneficial.\n    2. Update your decision-making rules as per the definitions above.\n    3. For each applicant, even after strategic manipulation, the true label should remain unchanged.\n    Finally, report the accuracy rate under your classification rules.\n    \n    test_examples: ...\n    \"\"\"\n    \n    ```\n\n- We are glad to **fully release all source code and materials upon acceptance**. Thank you for bringing up this important point.\n\n----\n\n> **Formatting Concerns**\n\n**Response:**\n\n- Could you specify which tables or figures violate the formatting requirements? \n- We will conduct a more thorough check.\n\n\n----\n[1] Ariel Flint Ashery *et al.* Emergent social conventions and collective bias in LLM populations. Sci. Adv.11,eadu9368(2025).\n\n[2] Gao, Chen, et al. \"Large language models empowered agent-based modeling and simulation: A survey and perspectives.\" *Humanities and Social Sciences Communications* 11.1 (2024): 1-24.\n\n[3] De Curtò, J., and I. De Zarzà. \"LLM-Driven Social Influence for Cooperative Behavior in Multi-Agent Systems.\" *IEEE Access* (2025).\n\n[4] Ziems, Caleb, et al. \"Can large language models transform computational social science?.\" *Computational Linguistics* 50.1 (2024): 237-291."}, {"review_id": "BpJP6GZDVq", "reviewer": "Reviewer_fu1o", "summary": "This paper mainly proposed that pre-trained LLMs are able to handle strategic classification tasks in two steps by in-context learning: (i) first learning how agents modify their features (strategic manipulation); (ii) updating the weight vector to adapt to the manipulated features. The paper provided theoretical proof on how LLMs accomplish both tasks under a linear SC formulation. The paper also includes experiments on large-scale datasets and non-linear attention layers to demonstrate the effectiveness of LLMs in non-linear settings.", "questions": "- If multiple SA layers with non-linear activations are considered, can ICL capture more complex strategic behaviors?\n- The experiments only consider the non-linear SA layers. Is it possible to consider some non-linear SC models (e.g., the agents do not best respond and the policy is non-linear)? Specifically, it would be interesting to discuss when agents manipulate their features in some complex ways, Can the LLMs learn their strategic behaviors through ICL?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "> Strength\n- The problem is interesting since many companies/institutions are already using LLMs in decision-making only with prompt engineering. Thus, it is useful to show theoretically the effectiveness of ICL to handle strategic behaviors.\n- The writing is relatively clear and most theoretical results make sense to me.\n- The LLMs in experiments are up-to-date.\n\n> Weakness\n- The theoretical results only apply when the SC model is linear, and the ICL update is done through one SA layer;\n- the results are not too surprising, i.e., the standard SC formulation can be regarded as a gradient update too, so it is not surprising that implicit gradient can match it. The paper seems to only prove that strategic behaviors in linear SC model can be captured by ICL of LLMs, which is expected.", "was_revised": true, "final_justification": "Thanks for the authors' extensive rebuttal.The new experiments are interesting and I encourage the authors to include them in the new draft. I agree that the problem setting is interesting although the theoretical analysis is based on linearity and this is why I give a score leaning to acceptance. Overall I still think 4 is a rating reflecting the overall merit of this paper, which means I tend to accept the paper. But since the theory parts are not particularly interesting, I will not give a score >= 5.", "rebuttal": "We sincerely appreciate your constructive comments and your positive evaluation of our work. We would like to address your concerns as follows:\n\n> **Weakness1**.The theoretical results only apply when the SC model is linear, and the ICL update is done through one SA layer;\n\n**Response:**\n\n- Our focus on linear SC models is motivated by two main reasons:\n  - Linear SC models **remain the dominant approach in both practical applications and academic research** on strategic machine learning[1,2,3]. \n  - Linear SC is often **preferred for its ease of deployment, interpretability, and direct comparability with established benchmarks**[4,5], particularly in real-world systems such as policy-making, risk assessment, credit scoring, and admissions or hiring.\n\n- Regarding the use of a single self-attention (SA) layer for ICL updates, we would like to note that:\n  - Using a single SA layer **allows for a clear analysis** of the relationship between ICL and gradient descent, as it enables explicit mapping between attention operations and gradient steps. \n  - Extending from a single to multiple self-attention layers corresponds to performing multiple steps of gradient descent, which **does not affect the validity of the theoretical analysis**.\n  - This simplification is **widely adopted in recent ICL theory**[6,7,8].\n- Moreover, practical nonlinear SC models, such as MLPs and GNN[9,10,11], are employed with forward propagation and gradient-based optimization. This means that our theoretical framework is naturally applicable to these models as well.\n- We have clarified this part in Lines 128-130 of our revised paper.\n\n----\n> **Weakness2**. The results are not too surprising\n\n  **Response:**\n\n- We would like to clarify that our contributions go beyond this connection:\n\n  - **Strategic machine learning is fundamentally a game-theoretic problem** involving both agent-side manipulation and decision-maker adaptation, and **has been studied largely separately from ICL/LLM literature**.\n\n  - To our best knowledge, **our work is the first to employ LLMs to model and solve this bi-level, game-theoretic optimization structure**.\n  - It enables **a scalable and retraining-free approach to large-scale gaming tasks**, where classical gradient-based retraining requires excessive computational resources and time.\n  \n\n- From **a broader social science perspective**,  our work establishes a crucial bridge between large language models and strategic machine learning:\n\n  - **A wide range of social science applications are increasingly converging with machine learning and LLMs**, including fields such as computational social science, policy evaluation, and behavioral economics[1,2,3,4].\n  - Our work provides **a foundational contribution** by connecting ICL in LLMs with game-theoretic learning frameworks. \n  - We hope this foundation will facilitate future interdisciplinary research and applications of LLMs in societal decision-making contexts.\n\n- We have clarified this part in our revised paper at Lines 62-64.\n\n\n----\n> **Question1**. If multiple SA layers with non-linear activations are considered, can ICL capture more complex strategic behaviors?\n\n**Response:**\n\n- Indeed, **we agree that** with multiple self-attention layers and nonlinear activations, ICL is capable of capturing more complex strategic behaviors beyond the linear case.\n\n- **Theoretical perspective:** \n\n  - Multiple self-attention layers with nonlinear activations greatly enhance the model’s representational capacity[12]. \n\n  - In SC tasks, even complex strategic behaviors, e.g., “**agent in the dark**[4]” settings (where agents have limited information) or \"**interacting agent behaviors**[10]\" (where agents strategically respond to each other), can be modeled as forward passes and gradient-driven updates in a neural network. \n\n  - Therefore, multiple SA layers with non-linear activations are considered capable of representing and adapting to these complex behaviors.\n\n- **Experimental verification:**\n\n  - We designed **an additional set of experiments** simulating more complex SC environments with complex strategic behaviors, e.g.,“agent in the dark[4]” and \"interacting agent behaviors[10]\". \n\n  - We compared the **agent feature distributions** of the standard method and that simulated by the model based on self-attention layers with nonlinear activations.\n\n  - We use the **Jensen-Shannon (JS) divergence**[13] as a similarity metric. We also evaluated the post-strategy classification accuracy.\n\n  - The experimental results are presented in the following table.\n\n  | Behavior Type               | JS Divergence | Accuracy |\n  | --------------------------- | ------------- | -------- |\n  | Agent in the dark           | 0.183         | 81.7%    |\n  | Interacting agent behaviors | 0.226         | 82.3%    |\n\n-----\n> **Question2**. The experiments only consider the non-linear SA layers. Is it possible to consider some non-linear SC models (e.g., the agents do not best respond and the policy is non-linear)? \n\n**Response:**\n\n- We **agree** that nonlinear SC models should be considered. \n\n- Therefore, we extended our experiments along two dimensions:\n\n  - **Nonlinear policy:** We first implemented the policy function using a nonlinear model (MLP).\n\n  - **Non-best-response agents:** We further considered agents that do not always best respond. Following prospect theory[14], we model the agents irrational, incorporating asymmetric value function and nonlinear probability weighting. To specific:\n\n    \n    $$\n    x^* = \\arg\\max_x \\left( \\sum_y w(P(y|x')) \\cdot v(u(y) - a_{\\text{ref}}) - \\phi^c(x, x') \\right)\n    $$\n    where:\n\n    - $v(\\cdot)$: value function (captures loss aversion),\n    - $w(\\cdot)$: probability weighting function,\n    - $a_{\\text{ref}}$: reference point,\n    - $\\phi^c(x, x')$: cost function for feature manipulation.\n\n- We use Jensen-Shannon (JS) divergence  (between attention and MLP) as a similarity metric and accuracy of ICL as evaluation metric.\n\n  - The experimental results are presented in the following table.\n\n\n| Policy and Agent Type  | JS Divergence | Accuracy (ICL) |\n| ---------------------- | ------------- | -------------- |\n| Linear + Irrational    | 0.133         | 83.6%          |\n| Non-linear + Rational  | 0.176         | 80.1%          |\n| Non-linear +Irrational | 0.152         | 77.8%          |\n\n\n\n\n-----\n\n\n[1] Hardt, Moritz, et al. \"Strategic classification.\" *Proceedings of the 2016 ACM conference on innovations in theoretical computer science*. 2016.\n\n[2] Milli, Smitha, et al. \"The social cost of strategic classification.\" *Proceedings of the conference on fairness, accountability, and transparency*. 2019.\n\n[3] Miller, John, Smitha Milli, and Moritz Hardt. \"Strategic classification is causal modeling in disguise.\" *International Conference on Machine Learning*. PMLR, 2020.\n\n[4] Levanon, Sagi, and Nir Rosenfeld. \"Strategic classification made practical.\" *International Conference on Machine Learning*. PMLR, 2021.\n\n[5] Ghalme, Ganesh, et al. \"Strategic classification in the dark.\" *International Conference on Machine Learning*. PMLR, 2021.\n\n[6] Von Oswald, Johannes, et al. \"Transformers learn in-context by gradient descent.\" *International Conference on Machine Learning*. PMLR, 2023.\n\n[7] Garg, Shivam, et al. \"What can transformers learn in-context? a case study of simple function classes.\" *Advances in neural information processing systems* 35 (2022): 30583-30598.\n\n[8] Akyürek, Ekin, et al. \"What learning algorithm is in-context learning? investigations with linear models.\" *arXiv preprint arXiv:2211.15661* (2022).\n\n[9] Stratis Tsirtsis, Behzad Tabibian, Moein Khajehnejad, Adish Singla, Bernhard Schölkopf, and Manuel Gomez-Rodriguez. Optimal decision making under strategic behavior. Management Science, 2024\n\n[10] Itay Eilat, Ben Finkelshtein, Chaim Baskin, and Nir Rosenfeld. Strategic classification with graph neural networks. arXiv preprint arXiv:2205.15765, 2022\n\n[11] Mehrnaz Mofakhami, Ioannis Mitliagkas, and Gauthier Gidel. Performative prediction with neural networks. In International Conference on Artificial Intelligence and Statistics, pages 11079–11093. PMLR, 2023\n\n[12] Xiang Cheng, Yuxin Chen, and Suvrit Sra. Transformers implement functional gradient descent to learn non-linear functions in context. arXiv preprint arXiv:2312.06528, 2023.335\n\n[13] Deasy, Jacob, Nikola Simidjievski, and Pietro Liò. \"Constraining variational inference with geometric jensen-shannon divergence.\" *Advances in Neural Information Processing Systems* 33 (2020): 10647-10658.\n\n[14] Kai-Ineman, D. A. N. I. E. L., and Amos Tversky. \"Prospect theory: An analysis of decision under risk.\" *Econometrica* 47.2 (1979): 363-391."}]

Author rebuttal:

Meta-review: The paper considers using LLMs for strategic classification. The authors propose a gradient free method based on in-context learning.
A theoretical analysis shows that both stages of strategic classification (inner optimization and outer optimization) can be realized through ICL.
The empirical evaluation shows that ICL-induced updates converge to gradient descent solutions. The proposed method also achieves superior classification accuracy compared to some traditional baselines.

The reviewers considered the connection between language models to strategic classification as interesting. The theoretical work, while limited, is also valuable. The empirical evaluation was also appreciated, especially the use of up-to-date LLMs.

There were several concern raised by the reviewers, but they were satisfied by the remedies provided in the rebuttal and the discussion. I feel that the paper can be accepted for NeurIPS, but I urge the authors to include the revisions suggested, including the additional experiments, and code. It is also important to make clear the limitations of their approach.
Decision: Accept (poster)