Task-Specific Data Selection for Instruction Tuning via Monosemantic Neuronal Activations
Authors: Da Ma
Gonghu Shang
Zhi Chen
Libo Qin
Yijie LUO
Hongshen Xu
Lei Pan
Shuai Fan
Kai Yu
Lu Chen
Venue: NeurIPS 2025 poster ()
Keywords: Data selection
Instruction tuning
Sparse Autoencoder
Neuronal activation states
Primary area: deep_learning
TL;DR: We propose a data selection method that leverages sparse, monosemantic neuronal activations learned via a sparse autoencoder to improve task-specific instruction tuning for large language models.

Abstract: Instruction tuning improves the ability of large language models (LLMs) to follow diverse human instructions, but achieving strong performance on specific target tasks remains challenging. A critical bottleneck is selecting the most relevant data to maximize task-specific performance. Existing data selection approaches include unstable influence-based methods and more stable distribution alignment methods, the latter of which critically rely on the underlying sample representation. In practice, most distribution alignment methods, from shallow features (e.g., BM25) to neural embeddings (e.g., BGE, LLM2Vec), may fail to capture how the model internally processes samples. To bridge this gap, we adopt a model-centric strategy in which each sample is represented by its neuronal activation pattern in the model, directly reflecting internal computation. However, directly using raw neuron activations leads to spurious similarity between unrelated samples due to neuron polysemanticity, where a single neuron may respond to multiple, unrelated concepts. To address this, we employ sparse autoencoders to disentangle polysemantic activations into sparse, monosemantic representations, and introduce a dedicated similarity metric for this space to better identify task-relevant data. Comprehensive experiments across multiple instruction datasets, models, tasks, and selection ratios show that our approach consistently outperforms existing data selection baselines in both stability and task-specific performance.

Task-Specific Data Selection for Instruction Tuning via Monosemantic Neuronal Activations
Da Ma", Gonghu Shang", Zhi Chen", Libo Qin', Yijie Luo", Hongshen Xu", Lei Pan", Shuai Fan Kai Yu Lu Chen a8a "X-LANCE Lab, MoE Key Lab of Artificial Intelligence, Al Institute
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China Shanghai Innovation Institution, Shanghai, China
AlSpeechh Co., Ltd., Suzhou, China "ByteDance
School of Computer Science and Engineering, Central South University Jiangsu Key Lab of Language Computing, Suzhou, China Suzhou Laboratory, Suzhou, China (madal23, chenlusz (sjtu. edu.cn
Abstract
Instruction tuning improves the ability of large language models (LLMs) to follow diverse human instructions, but achieving strong performance specific target tasks remains challenging. critical bottleneck is selecting the most relevant data to maximize task-specific performance. Existing data selection approaches include unstable influence based methods and more stable distribution alignment methods, the latter of which critically rely the underlying sample representation. In practice, most distribution alignment methods, from shallow features (e.g., BM25) neural embeddings (e.g., BGE, L.M22ecc), may fail capture how the model internally processes samples. To bridge this gap, we adopt model-centric strategy in which each sample is represented by its neuronal activation pattern in the model, directly reflecting internal computation. However, directly using raw neuron activations leads spurious similarity between unrelated samples due to neuron polysemanticity, where single neuron may respond to multiple, unrelated concepts. To address this, we employ sparse autoencoders disentangle polysemantic activations into sparse, monosemantic representations, and introduce a dedicated similarity metric for this space better identify task-relevant data. Comprehensive experiments across multiple instruction datasets, models, tasks, and selection ratios show that our approach consistently outperforms existing data selection baselines in both stability and task-specific performance?
Introduction
Instruction tuning enables large language models (LLMs) to better follow human instructions, powering versatile applications such as chatbots (3+6]. However, real-world tasks often require specialized abilities-like professional physics problems- that general instruction tuning may not provide 7 While fine-tuning LLMs on diverse and broad instruction datasets
improves their
overall generalization, performance on specific tasks often remains suboptimal 7 This raises key challenge: how can we efficiently select data subsets from general instruction datasets to maximize
LLM performance lurgeted tasks, given only a handful of examples ($ 2.1
" "Correponding author
2 AAaillable on httpss:/ggittbb..oo///////////e/
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Photo
The brain octivotes smilar reurons for photo ond rome the some person
Samples t t +
Samples
Name
AAA
MMN
t
AAA
(o) Neural coectivation brein
Activate
Polysemantic neurons
SAE
Monosemontic
!
30.0
Activation space
Activation space
WALL
30H
Tys
(c) Poly vs. Mana on three benchmarks
(b) From Polysementic to Monasemantic neuron activations for date selection
Figure 1: (a) An example of neural coactivation in brain [25] (b) Disentangling polysemantic activations into monosemantic representations via a sparse autoencoder (SAE) (c) Improved data selection using monosemantic activations (More details are in Appendix (B.3)
To address this challenge, data selection methods can be roughly grouped into main categories: influence-basee -baned methods and distribution alignment methods. Influence-basee methods [1 select training data by estimating the influence of each candidate sample on the evaluation loss for target examples [14], and prioritize samples that are expected to most reduce this loss. However, these approaches can be unstable since evaluation loss can fail to reflect true model performance after instruction tuning, causing inconsistency in selection results [15H77]. As alternative, distribution alignment methods [8-21 select data whose distribution similar to that of the target task, in order to minimize distribution shift and improve generalization
[10 22 24 These methods typically embed samples into a feature space, define a similarity metric within this space, and select data most similar to representative samples of the target task ($ 2.2
While distribution alignment offers a principled framework for data selection, its effectiveness depends heavily on how samples are represented. Existing methods typically rely on data-centric embeddings, derived either from shallow textual features (e.g., BM25 [88 and DSIR [99] or high- dimensional neural representations (e.g., BGE (20) and LLM2Vec (21) extracted directly from the data. While effective in many settings, such representations may fail to capture the internal computational dynamics of how the model processes different data points, which is crucial for task- task-specific performance [26] 27]. To address this, we introduce a model-centric paradigm which explicitly captures these internal dynamics by representing each data point through the neuronal activation patter it triggers in a pretrained model. This approach is inspired by neuroscience, where related concepts trigger coordinated neural responses [28 (Figure a Accordingly, we select those training samples whose activation signatures in the model are most similar to those elicited by target task exemplars.
Nonetheless, directly using raw neuron activations can yield suboptimal similarity estimates due to neuron polysemanticity, where single neuron may respond to multiple, unrelated concepts [26]. As a consequence, unrelated samples can appear spuriously similar due to shared activation of polysemantic neurons (left of Figure bb.. To mitigate this issue, we employ sparse autoen-
coders (SAEs) [27 29 30 to disentangle polysemantic activations into sparse, monosemantic units, making similarity in activation space more aligned with semantic similarity (right of Figure 42.3 This improvement is illustrated in Figure c, where data selection based on monasemantic activations yields superior performance compared raw polysemantic activations. In addition, introduce a dedicated similarity metric tailored to the sparse monosemantic activation space produced by SAES, detailed in $ 2.4
In summary, our contributions are: 1) We propose MoNA (Monosemantic Neuronal Activation-based Data Selection), novel method for task-specific data selection in instruction tuning. MoNA repre- sents each sample using sparse, monosemantic neuronal activations derived from sparse autoencoders,
For example, one neuron in large language model may be activated by both academic citations and HTTP request patterns.

---

enabling model-centric data selection. 2) We introduce similarity metric tailored to this sparse monosemantic activation space, enabling more accurate identification task-relevant examples. 3) Comprehensive experiments across multiple candidate instruction tuning datasets, evaluation tasks, models, and data selection ratios demonstrate that MONA consistently outperforms existing data selection approaches. We will release our code to facilitate further research in the community.
2
Methods
We begin this section by formalizing the task-specific data selection problem and its objective. Next, outline the distribution alignment framework that underpins approach. We then describe proposed method (MoNA) in detail, including the construction of monosemantic neuronal activation embeddings and the dedicated similarity metric designed for this embedding space.
2.1 Problem Formulation
Given a large-scale general instruction dataset Di {s: and small set of representative examples from the target task D = {sy} -رك where M < N, our goal is to select subset
Deel C Dare such that fine-tur Ane-tuning large language model (LLM) on Deel leads to the best overall performance on the target task, denoted as T
Formally, let Mo denote LLM with parameters 0, and S(Ma, T denote its performance metric (e.g., accuracy, Fl Fl-score) on the target task. The data selection objective is formulated as:
D
arg max
S (
DCD |D|=k
where [D] k is budget constraint on the number of selected samples, and 0 is obtained by fine-tuning the model on D:
f Optimize (Mo: DM.
(2)
However, directly optimizing Eq. is computationally infeasible, as it involves enumerating all possible subsets and retraining the model for each. This motivates the need for efficient data selection criteria that can identify high-quality subsets with minimal supervision and computation.
2.2 Distribution Alignment Pipeline
A common approach to the data selection problem defined in 2.1 is distribution alignment, which seeks to select a subset Deel whose distribution closely matches that of the target examples Dist The objective in Eq. (1) then becomes:
Dael
arg max DCD |D]=k
Sim Det)
(3)
where Sim(...) measures the distribution similarity between the selected and target sets. In practice, this similarity is operationalized by first computing, for each sample in DD its similarity the target set De in the embedding space, and then aggregating these sample-level similarities. Based on this, the pipeline consists of three steps (see the left panel of Figure
1. Embedding: Each sample E Da U Diet is projected into d-dimensional feature space via an embedding function s + z E Rdd where z; denotes the embedding of s;
2. Similarity Metric Definition: Explicitly define similarity metric, denoted as 8(s,, D to quantify the similarity between source sample E DDte and the target set Ditr
3. Subset Selection: The final subset Deel consists of the samples from Dare with the largest aggregate similarity to the target set:
Deel arg max
Σ 8(s;, D
DC D 8.ED
'In this paper, refers the embedding of sample

---

Layer
Sparse Auto Encoder
D src
Layer
Layer
Embedding
Naarnea ectivtted
Dead
Transformer
(1. fz
Sample
Somple
Embeddinngs
min
Similarity Calculation
max
Sample
Embedcing
Most Similar Data Selection
RXX 02 211
min
11+11+08 82
-0.34
Token Aggregation
Workflow of MoNA. Left: Distribution alignment pipeline between the source dataset and the target task. Right: Computation of monosemantic neuronal activation embeddings and the proposed similarity metric. Top: Application SAE; Bottom right: Aggregation token embeddings into a sentence-level embedding; Bottom left: Calculation of similarity between two samples
Figure 2:
2.3 Monosemantic Neuronal Activation Embedding
Intuition In neuroscience, related stimuli trigger coordinated neuron activations, reflecting semantic similarity 28 Inspired by this, hypothesize that neural network activations can similarly capture semantic relationships between samples. To this end, we represent each sample by its activation pattern from a predefined set of neurons (e.g., g.. specific layer in neural network Mds
2 SNAS (M**s)
(5)
where (..)s extracts neuronal activations for input from Mds
Sparse Autoemcoder-basse based Monosemantic Decomposition Building on the intuition, we use the neuronal activation states from single, predefined layer in the transformer [31 the basis for the 6 However, even the activations from single layer can exhibit polysemanticity, where single neuron responds multiple, often unrelated, concepts (26). For example, one neuron in large language model may be activated by both academic citations and HTTP request patterns, making such activations difficult interpret and less effective for representing specific semantic property [26].. This polysemanticity undermines the interpretability and reliability of feature representations based directly on raw activations.
embedding
To address this, we employ a sparse autoencoder (SAE) following 27
(see the top right part of Figure 2). The SAE transforms the original neuron activations into higher-dimensional, sparse activation space, where each resulting neuron tends to respond to distinct, monosemantic feature or concept, rather than simply producing learned representation. Prior work (29] has demonstrated that such sparse activations exhibit improved interpretability and semantic purity, which beneficial for our data selection framework] Formally, given input sequence s (41, l2,..... bn) of tokens,
29
30
let h e Rd denote the output layer & of model d for token tę (1 < & < n). The SAE computes a new sparse activation for each token as follows:
x Topk (W enc (h bpre))
'Using all neurons in the network would result in extremely high-dimensional embedding, equal to the total number of model parameters, and bring significant computational overhead.
We compare the effect different layer choices in 33.5
We provide further evidence for this through the visual validation shown in Figure

---

where Wene E Rd''d (d' > d) and bpec E R are trainable parameters. The operator (op retains only the largest K values of the input vector and sets all remaining entries to zero This transformation produces sparse and interpretable activation pattern for each token in the sequence, enabling more reliable and semantically meaningful representations for downstream data selection.
Token Aggregation for Sample Embedding After obtaining the sparse monosemantic activation z for each token in the imput, we aggregate these token-level vectors by averaging over all tokens to form a sample-level embedding (see the bottom right of Figure
Z
ZE
where is the number of tokens in the input s. Averaging, rather than summing, is crucial for mitigating length bias: without normalization, the selection process systematically prefers samples that match the average length of samples in Diet rather than those with the highest semantic relevance. This bias often harms downstream performance, as demonstrated in Appendix D.5
In summary, ).) in Eq. (5) corresponds to the composition of the token-level sparse mapping in Eq. (6) and the aggregation operation in Eq.
2.4 Similarity Metric for Monosemantic Neuronal Activation Embedding
This section describes how we define the similarity metric within the monosemantic neuronal activation embedding space for use the distribution alignment pipeline. The procedure consists of two steps: (i) aggregating the embeddings of the target examples to form task prototype, and (ii) computing the generalized Jaccard similarity between each source sample and the task prototype.
Task prototype representation To improve efficiency, we aggregate the monosemantic neuronal activation embeddings of all target examples in Dee into a single task prototype. This reduces the computational complexity from (( (Doel) to O(ID)))) Formally, the task prototype is defined as:
DW
Zj1
(8)
Diet
where 2j is the embedding of the j-th target example.
Generalized Jaccard Similarity For high-dimensional, sparse feature representations such as our monosemantic neuronal activation embeddings, classic similarity metries such as Euclidean Cosine similarity can become unreliable due to the "curse of dimensionality" (33] Our empirical results confirm that neither Euclidean nor Cosine similarity is suitable for this embedding space (see $ $3.5 for details). As an alternative, we adopt the generalized Jaccard similarity (see the bottom left part of Figure Mathematically, given a source sample 8; with embedding z; and the task prototype ! the generalized Jaccard similarity is defined as:
E. min [z; kk), 20 (A])
(s,, Dier)
(9)
Σn max (z, [k], 2
where zi]k]] (or (k)(k)) denotes the k-th element of the corresponding embedding.
3
Experiments
In this section, we design experiments to systematically evaluate our method (MoNA) for task- specific instruction tuning. We center our evaluation around the following key questions:
Effectiveness and Robustness (Q1): Does MoNA consistently select data that yields bet- ter downstream performance across (i) various source general instruction datasets and target evaluation tasks, (ii) different instruction-tuned LLMs, and (iii) range of data selection ratios?
We provide an ablation study of the effect of different K values in 3.5
5

---

Table 1: Performance of different models after instruction tuning with 5% of the data selected from different datasets. Best results are in bold; second best are underlined.
D
-OPENHERMES-2.5
2 =LESS
Method
MMLU
GSM8K
BBH
MBPP
GPQA
Avg.
MMLU
BBH
TydiQA
Avg.
LLaMA3.1-8E
BASE
65.30
55.50
63.08
46.40
28.12
51.68
65.30
63.08
71.26
66.55
FULL
64.60
05.35
64.31
49.00
27.90
54.23
64.60
64.31
72.66
67.19
RANDOM
64.02
58.65
63.76
36.73"
30.36
52.09
64.16"
64.29"
"69.78"
06.08
Infleence-bused
MATES
64.11
54.28
65.38
47.60
28.12
51.90
63.62
63.68
67.74
65.01
LASS
64.34
66.87
63.00
47.80
31.47
54.70
62.51
62.11
70.68
65.10
Distribution alignment
BM2.5
64.14
66.64
65.23
48.40
27 27.90
54.46
65.41
DSIR
63.95
06.94
61.29
29.91
54.74
64.25
63.19
64.35
DLRDS-BGE
64.45
64.82
64,20
48. 60
31.25
54.66
64.06
61.82
70.30
05.39
DLRDS-LLaMA3-8B
64.31
64.75
63,97
48 80
29.46 .6.4
54.26
62.11
61.54
71.91
65.19
LLM2Vec
64.29
63.53
65.55
48 .4
30.13
54.38
62.06
62.03
68.11
64.07
(ours)
64.49
67.93
66.44
48.40
31.47
55.75
64.78
72. 72.60
67.20
OLMo-7B
BASE
28.42
7.35
29.96
21.40
26.56
22.74 74
25.42
29.96
31.67
30.02
FULL
45.05
31.96
33.13
26.40
26.56
32.62
39.31
28.86
33.43
33.87
RRANDMM
36.96
16.00
31.47
19.47
27.38
26.26
"28.60
30.32
31.93
30.45
Influencebbsedd
MATES
30.27
13.72
32.33
16.40
29.57
30.46
31.02
30.35
LE8S
46.15
26.91
33.68
20.20
25.89
30.57
37.21
30.07
33.20
33.49
Distribution alignment
BM25
42.34
31.08
34.30
26.80
25.45
31.99
35.74
28.95
34.40
33.03
DSIR
36.48
29.26
34.08
19.40
27.23
29.29
29.54
32.87
33.25
31.89
DIADS-BGE
42.77
32.30
33.40
26.80
31.83
35.22
33.28
31.38
DIRDS-LLaMA3- -BB
38.16
31.39
33.30
22.80
30
31.16
40.0
26.05
31.5
22.60
L.MMVec
37.24
30.10
33.57
23.40
28. 35
30.53
39.72
28.58
32.26
33.52
MoNA
(ours)
44.74
32.83
33.51
26.00
25.00
32..42
40.14
30.19
33.80
34.71
Visualization and Interpretability (Q2): Can the monosemantic activation embeddings make data selection decisions more transparent and interpretable? We illustrate this via visual analysis of activation patterns.
Key Factor Analysis (Q3): How do crucial factors such-suchhhh as layer selection, sparsity parame- K, and similarity metric affecct the behavior of MoNA?
3.1 Experimental Setup
General Instruction Data and Evaluation Tasks To comprehensively evaluate robustness and generalization on target tasks, we select training data from two large-scale, diverse instruction datasets:
OPENHERMES- -25 [9] (IM synthetic and curated instruction/chat samples) and LESS [ (270K
samples covering both classical sources such as FLAN V2 [8. COT [3], and open-ended human-
annotated datasets like DOLLY
351 and OPEN ASSISTANT 36 We evaluate performance on six
target tasks: MMLJU 37 (general knowledge). BBH 38
(complex reasoning). GSM8K (39 (math
problems), MBPP 40 (programming), GPQA (expert QA), and TydiQA [42] (multilingual QA).
Evaluations use Im-evaluation-hanl harness (43 and vLLM [4] except for TydiQA, which uses the LESS codebase IT]. More details are in Appendix B.I
Models and Training We conduct instruction tuning on two widely used open-source language models: LLaMA3.1-8B [5] and OLMo-7B [45], with additional experiments on larger model, LLaMA2- 13B [46), to assess scalability. For data selection, we utilize open-source sparse autoen- coder (SAE) models based on LLaMA3-SB") with setting K = 192 ($ 2.3). Neuron activations are extracted from penultimate (second-to-fast layer of the model. Fine-tumm tuneettnnng is performed with llama-factory [47], using cosine scheduler (peak learning rate 7 warmup ratio 0.01), batch size 128, weight decay 0.1, and maximum sequence length 8192. All models are trained for two epochs.
More details shown in Appendix A
httpe::/nuggingtaan co htw.a/e.ee..eeeeeeee....a.
32x

---

Table 2: Performance of LLaMA2-13B after instruction tuning with 5% of the data selected from OPENHERMES-2.5 Best results are bold; second best are underlined.
Method
MMLU
GSM8K
BBH
MBPP
GPQA
Avg.
BASE
55.11
24.03
46.74
27.00
30.58
36.69
FOLL
57.61
55.95
52.63
35.00
27.01
45.64
RANDOM
'55.66"
41.50
51.37
31.13
28.64
41.72
Influence-based
MATES
55.68
37.07
51.17
31.20
26.34
40.29
LESS
60.38
48.75
50.42
25.80
27.90
42.65
Distribution alignment
BM25
57.60
58.15
52.65
34.60
27.90
46.18
DSTR
55.83
53.53
52.02
31.60
27.23
44.04
DLRDS-BGE
56.
56.63
52.34
35.60
27.68
45.78
DURDS-LL.aMA-BB
58.65
52.31
52.36
35.20
26.56
45.02
LLM2Vec
57.02
58.30
51.27
34.80
27.90
45.86
MoNA (ours)
57.26
60.27
52.23
35.60
27.90
46.65
3.2 Baselines
To ensure fair and comprehensive comparison, evaluate MoNA against several representative baselines covering all major categories from $ (i) Non-selection buselines: BASE (base pretrained model) and FULL (instruction tuning full instruction data); (ii) Random selection (RANDOM):
uniformly samples data for fine-tuning (averaged over three seeds); (iii) siffuence-baseed selection:
MATES [13 (proxy model predicts loss reduction per sample) and LESS [I]] (gradient-bass Taylor approximation estimates influence); (iv) Distribution alignment- based selection: includes classical methods such BM25 [8] (tf-idf) and DSIR 19 (n-gram), as well deep embedding approaches DLADS-3GE 20 (BGE embeddings), DLRDS BERASSLLLane LLLLLLLMM--BB 8B [5] (LLaMA3-88) embeddings), and LLM2Vec [21] (bidirectional text encoders adapted from decoder-only LLMs). All baselines the same data selection ratio for fair comparison; additional details are provided Appendix C
3.3 Main Results on Target Tasks
To address Q1 (effectiveness and robustness), we select 5% of the general instruction data for each data selection method, fine-tune two backbone models, and evaluate target tasks. We also assess scalability by repeating the experiments with a larger model. In addition, we investigate the impact of varying the data selection ratio, and further employ L.LM-based data analyst to conduct model-agnostic evaluation the selected data quality.
Across Datasets and Target Tasks As shown in Table on the LLaMA3.1-8B model, MoNA achieves either the best second- best performance on nearly all tasks for both OPENHERMES-225 and LESS instruction datasets. For instance, on OPENHERMES-2.5 MoNA achieves the highest scores GSM8K, BBH, and GPQA, and obtains the best overall average-even surpassing full-data fine-tuning. Similar trends are observed on LESS. These re- sults demonstrate that MoNA not only selects more semantically relevant data than all baselines, but also maintains robust performance across wide range of tasks and instruction data sources.
RANDOW
LESS
bosa DUROS-86E
bLAbS-LLaMA3-85
MATEL MATES
SIN75
MoNi
102
Selection Ratio
Figure 3: Performance of different data selection methods under varying selection ratios, evaluated on LESS with LLaMA3.1-88
Across Backbone Models On OLMo-7B, MoNA achieves the highest overall average per- formance among all methods for both OPENHERM and LESS instruction datasets (Table I Although task-level stability decreases for all methods compared to LLLMA3.1-883 MoNA still achieves the highest proportion of top-two finishes knishes-ranking in the top two on 3 out of 5 tasks for
Here, lasel stability refers to the proportion of tasks where a method ranks among the top two. Lower stability means high performance is achieved on fewer tasks.
7

---

Polysemantic
Nath
Code
Acrivation
Monosemantic
Activation
Neuron Index (Top K by Variance)
Figure 4: Neuron activation profiles for 100 Math and 100 Code samples on the top- 100 most variant neurons. Faint lines show individual samples; bold lines show task means. In the polysemantic (top) plot, many neurons, especially those with high activation peaks (marked by weeping face), are simultaneously activated by both tasks, reflecting pronounced overlap and limited task specificity, In contrast, the monosemantic (bottom) plot reveals clear task-specific activation patterns.
OPENHERMES-22 and 2 out of 3 tasks for LESS, both of which are higher than any baseline. This indicates that, despite the absolute stability being affected by the backbone, MoNA remains the most robust and semantically expressive data selection approach relative competing methods.
On LLaMA2-13B, MONA exhibits similar trend observed on OLMo-7B (Table Although absolute task-level stability decreases compared to LLLaMA3.1-88 MoNA continues to show stronger overall performance and relatively higher stability than all baseline methods.
Across Data Selection Ratios Across all selection ratios (Figure 3. MoNA consistently achieves the best performance, reaffirming both its robustness and semantic expressiveness. Interestingly, selecting 10% of the data results in lower performance than using 5%. We speculate that increasing the ratio may introduce less relevant lower-quality samples, thereby diluting the benefits of high- quality, semantically aligned data. This observation indicates that the choice selection ratio is eritical factor in data selection for instruction tuning and deserves further exploration.
LLM as a Data Analyst Beyond evaluating instruction- tuned model performance, we employ an LLM-based data analyst to assess the quality of selected training data in mgnel-agnostic way. For each method, we randomly sam- ple 100 training instances and prompt GPT-4o-min (48) to compare them with representative target samples, con- sidering semantic similarity, instruction format, and task relevance. Each comparison scored from to 10, and final scores averaged. More details given in Ap- pendix B.2 As shown in Figure 5 the LLM consistently assigns higher scores data selected by MoNA versus two strong baselines, confirming both the semantic expres- siveness and stability of our approach. Additional case studies are provided in Appendix D.I
DLRDS 9GE
MoNA (ours)
Score
65M8K
TydQQ
Figure 5: LLM as Data Analyst: scores for data selected by different methods. Higher scores indicate better perfor- mance
3.4 Neuron Activation Visualization
In addition to validating improved downstream performance with neuron activation- based data selec- tion, we further analyze and visualize the neuron activation patterns (Q2) for different tasks (Figure While polysemantic neurons produce substantial actívation overlap across tasks, monosemantic representations obtained via the sparse autoencoder yield well-separated task-specific activation patterns. This underscores the importance of disentangling polysemantic activations.

---

16.5
25.00
50-22
365
06.
06.0
54.
43.79
050
63.4
66.5
12
20
24
Loyer
Wetric
(a) Layer selection
(b) Sparse parameter K
(c) Similarity metric
Figure 6: Ablation studies for key design choices in MONA
3.5 Ablation Studies
To better understand the contribution of each component in MONA (Q3). we conduct ablation studies on key design choices, including which layer extract neuron activations from, the sparsity parameter K, and the similarity metric. More experimental details (e.g., evaluation benchmarks and instruction tuning datasets) are Appendix D.8
Efffct of Layer Selection We first examine how the choice of layer for extracting neuron activations affects the performance of MoNA. Since prior work [30] shows that SAEs trained on shallower layers tend to specialize in next-token prediction and provide less transferable features, we focus our analysis on deeper layers. Specifically, we select seven layers evenly spaced from layer the penultimate layer (layer 31) of LLaMA3-8B. As shown in Figure O embeddings from shallower layers can result unstable suboptimal performance, while embeddings from deeper layers especially the penultimate layer-delive deliver strong results. Based on these observations, we extract neuron activations from the penultimate layer other experiments.
Effeet of Sparsity Parameter K the sparsity parameter K increases, indicating that retaining more active neurons leads higher- quality data selection. The improvement becomes less pronounced as K grows larger. In contrast, when K is very small, performance drops sharply, suggesting that insufficient neuron information hampers effective selection. Based on these results, we adopt K = 192 in all main experiments
As shown in Figure b.b, model performance generally improves
Effect of Similarity Metric We compare the impact of different similarity metries, including Jaccard, Cosine, and Fuclidean. As shown Figure Jaccard similarity consistently yields better results than the other metrics across benchmarks, highlighting its suitability for MoNA.
3.6 Additional Analysis
Effeet of SAE To examine the effect of SAE model in greater depth, we additionally train SAE model built upon LLaMA2-15 13B. The model trained using the RedPajama-Data- Lataaa--Da------aaaaa TTSample datase [2] with training ducted under the sparsify framework""
47.2
46.6
Score
As is shown further enhance the effectiveness of data selection and downstream perfor- mance. These findings suggest that the quality and scale of the SAE model have a positive impact on the overall results.
in Figure 7 larger SAE backbone
36.7
BACE
RULL
(LL-MA3-08)
GAUNAE-119 138-
Comparison with MoE-based Monosemantic Embed- dings To further explore the concept of monoseman- tic embeddings, we investigate an MoE-based monose- mantic embedding approach 49
Figure 7: Results of Using Different SAE Models on LLaMA2-13B
Specifically,
replace SAE with MoE-based
"e did not explore values of K greater than 192 in this work.
-atta- 1

---

embeddings: for each token, we use the concatenation of expert routing scores from all layers of an MoE model (collected during a forward pass) the token's embed- ding. For this experiment, we employ the cept for substituting the SAE-based embeddings with MoF-based embeddings, all other components and settings of our workflow remain unchanged to ensure a fair comparison.
[50) model. Ex-
Figure demonstrates that using MoE-based embeddings yields modest improvement over the FULL baseline un- der our current experimental setup, indicating that monose- mantic representations from MoE models can provide slight benefit for task-specific data selection. However, MoE-based embeddings still lag behind SAE-based em- beddings by a noticeable margin. This observation sug- gests that the MoE-based embedding still has room for improvement capturing monosemantic representations.
05.8
54.7
94.2
51.7
4
Related Work
BA5E
HuL
McNA (SAE)
MoNiA (MeE)
Data Selection for Task-Specific Instruction Tuning Most data selection methods for task-specific instruction tuning roughly fall into two main categories: influence- based and distribution alignment approaches. Influence- based methods select training data by estimating the in- fluence of each candidate sample on the evaluation loss for target examples [1], and prioritize samples expected to most reduce this loss. For example, DsDm [12 and MATES [3] use proxy models, while LESS [I] relies on gradients. However, these methods can be unstable because evaluation loss may not reliably indicate model quality after instruc- tion tuning [S Distribution alignment methods 21 select samples by embedding them into a feature space and aligning the source and target distributions using a similarity metric. Prior work is largely data-centric, representing samples with surface-level features such as n-grams [19], tf-idf [18]. neural embeddings 200), which may fail to reflect how the model internally processes information [26] (27). In contrast, our approach model-centric: we represent each sample by the neuronal activation pattern it triggers inside the model, thereby capturing the internal computational dynamics that are critical for spec-fie performance.
Figure 8: Evaluation of Monosemantic Embeddings over LLaMA3.1-8P SAE- based vs. MoE-based
Sparse Autoencoders in Feature Learning Sparse coding was first introduced for over-complete dictionaries (5T), and unsupervised dictionary learning was pioneered by 52 These ideas led to sparse autoencoders, which have become important tools for learning structured features vision and language [53) [54) Recent work has also investigated sparse autoencoders analyzing representations of large language models [26]]] 55). Almost concurrently, (56) used sparse autoencoders for data selection, with focus diversity. In contrast, our approach is distinct that we use sparse neuron activations to capture semantic relatedness between samples and to overcome neuron polysemanticity. enabling more interpretable and semantically aligned task-specific data selection.
5 Conclusion
We propose a model-centric approach for task-specific data selection in instruction tuning, using monosemantic neuronal activations from sparse autoencoders. This representation captures inter- nal model computation and enables more semantically aligned and interpretable data selection. Experiments across various models and tasks demonstrate consistent gains over previous baselines.
Acknowledgments
This work was supported by the China NSFC Projects (92370206, U23B2057, 621 62120106006) and the Shanghai Municipal Science and Technology Projects (2021SHZDZX0102 and 25x010202846).
10

---

References
[1] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang. S. Agarwal, K. Slama, A. Ray et.l. "Training language models follow instructions with human feedback," Advances in neural information processing systems, vol. 35, pp. 27 730-27 744, 2022.
[2] S. Zhang. L. Dong, X. Li, Zhang, X. Sun, S. Wang. Li, Hu, T. Zhang. F. Wu et al., "Instruction tuning for large language models: A survey," arXiv preprint arXiv:2308, 10792, 2023.
[3] T. Brown, B. Mann, Ryder, M. Subbiah,
Kaplan, P. Dhariwal, A. Neclakantan, Shyam, G. Sastry, A. Askell et al., "Language models are few-shot learners," Advances in neural information processing systems, vol. 33, pp. 1877-1901, 2020.
[4] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt,
S. Altman, S. Anadkat et ol., "Gpt-4 technical report," arXiv preprint arXiv:2333.08877, 2023.
[5] A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang.
A. Fan et al., "The llama 3 herd of models," arXiv preprint arXiv:2407.21778 2024.
[6] Anthropic, "Introducing the next generation of claude," " 2024. [Online]. Available: https
/wwwwanthropicccoommmm www.aattropic.com//ewsdcameee wwtpp//wwwo///////w///.eo. 3- family
[7] Y. Wang. H. Ivison, P. Dasigi, J. Hessel, T. Khot, K. Chandu, D. Wadden, K. MacMillan, N. A. Smith,
I. Beltagy et al., "How far can camels go? exploring the state of instruction tuning on open resources." Advances in Neural Information Processing Systems, vol. 36, pp. 74 764 74 786, 2023.
[8] S. Longpre, L. Hou, T. Vu, A. Webson, H. W. Chung, Y. Tay, D. Zhou, Q V. Le, B. Zoph, J. Wei et al.. "The flan collection: Designing data and methods for effective instruction tuning," in International Conference on Machine Learning. PMLR, 2023. pp. 22 231 22 648.
[9] Teknium, "Openhermes 2.5: An open dataset of synthetic data for generalist IIm assistants." " 2023. [Online]. httpss//huuginngta....ooottttttt www/aaasstsscckniumm/m Saaptttmm//ppeeemme---eeeee Available: https://huggingtace..
[10] Z. Liu, A. Karbasi, and T. Rekatsinas. "TSDS: Data selection for task-specific model finetuning." in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [Online]. Available:
[11] Xia, Malladi, S. Gururangan, S. Arora, and D. Chen, "Less: selecting influential data for targeted in- struction tuning." in Proceedings of the 41st International Conference on Machine Learning, ser. ICML'24 22. JoLR.org, 2025.
[12] L. Engstrom, A. Feldmann, and A. Mkadry, "Dsdm: model-aware dataset selection with datamodels."" in Proceedings of the 41st International Conference on Machine Learning, ser. ICML'24. JMLR 2025.
[13] Z. Yu, S. Das. and C. Xiong, "MATES: Model-aware data selection for efficient pretraining with data influence models," in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [Online]. Available: http:://wwwuweeeeeeeeeeerr//ll/f
[14] T. Thrush, C. Potts, and T. Hashimoto, "Improving pretraining data using perplexity correlations." arXiv preprint 24l::2409008866 2024.
[15] Y. Tay, M. Dehghani, J. Rao, W. Fedus, S. Abnar. H. W. Chung, S. Narang. D. Yogatama, A. Vaswani. and D. Metzler, "Scale efficiently: Insights from pretraining and finetuning transformers," " in International Conference on Learning Representations, 2022. [Online]. Available: https://ppenreviccweeffooo "it.
[16] C. Zhou, Liu, Xu. S. Iyer. Sun, Y. Mao, X. Ma. Efrat, Yu, L. Yu. Zhang, G. Ghosh, M. Lewis. L. Zeetlemoyer, and O. Levy, "Lima: less is more for alignment," in Proceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS '23. Red Hook, NY, USA: Curran Associates Inc., 2024.
[17] T. Xia, B. Yu, K. Dang, A. Yang, Y. Wu, Y. Tian, Y. Chang, and J. Lin, "Rethinking data selection at scale: Random selection is almost all you need," arXiv preprint arXiv:2410.093355 2024.
[18] S. Robertson, H. Zaragoza et al., "The probabilistic relevance framework: Bm25 and beyond," Foundations and Trends in Information Retrieval, vol. 3, no. pp. 333-389, 2009.
[19] S. M. Xie, S. Santurkar, T. Ma. and P. S. Liang, "Data selection for language models via importance resampling," Advances in Neural Information Processing Systems, vol. 36, pp. 34 201 34 227, 2023.
11

---

[20] S. Xiao. Z. Liu, P. Zhang, N. Muennighoff. D. Lian, and J.-Y. Nie. "C-pack: Packed resources for general chinese embeddings."" in Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, 2024, pp. 641-649.
[21] P. BehnamGhader, V. Adlakha, Mosbach, D. Bahdanau, N. Chapados, and S. Reddy, "LLM2vec: Large language models are secretly powerful text encoders," " in First Conference on Language Modeling, 2024. [Online]. Available: https://opeaneviee.ne///ooo -dd-IWIRREEBBB
[22] C. Jia and Y. Zhang. "Prompt-based distribution alignment for domain generalization in text classification," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Y. Goldberg, Z. Kozareva, and Y. Zhang, Eds. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 10 10147-10157 157. [Online]. Available: http://aaaabboooo.oo///2../.oomm
[23] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan, "A theory of learning from different domains," Machine learning, vol. 79, pp. 151-175, 2010.
[24] A. Torralba and A. A. Efros, "Unbiased look dataset bias," in CVPR 2011. IEEE, 2011, pp. 1521-1528.
[25] R. Q Quiroga, L. Reddy, G. Kreiman, C. Koch, and I. Fried, "Invariant visual representation by single neurons in the human brain," Nature, vol. 435, no. 7045, pp. 1102 110221117.2000 2005.
[26] T. Bricken, A. Templeton. J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, Anil, C. Denison. A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Haaficed-Dodds, A. Tamkin, K. Nguyen. B. McLean, J. E. Burke, T. Hume, S. Carter, T. Hemighan, and C. Olah, "Towards monose- manticity: Decomposing language models with dictionary learning," Transformer Circuits Thread, 2023, http://ttnkkoommrr-cciiis/l//p/p
https:/traanffomrrcccccuusssss hwtp://wiiittup////o2//e http:///ww..oooommmmaaaaaeeeeeee
[27] R. Huben, H. Cunningham, I.. Smith, A. Ewart, and I.. Sharkey, "Spprse autoencoders find highly interpretable features in language models," in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://peemeeviewnettiiee hoo.mmmm/ddddo.//s......
[28] J. Taubert, V. Goffaux, G. Van Belle, W. Vanduffel, and R. Vogels, "The impact of orientation filtering on face-selective neurons in monkey inferior temporal cortex," Scientific reports, vol. 6, no. 1, p. 21189, 2016.
[29] A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A, Pearce, C. Citro, F. Ameisen, A. Jones, H. Cunningham, N. I.. Turner, C. McDougall, M. MacDiarmid, C. D. Freeman, T. R. Sumers, F. Rees, J. Batson, A. Jermyn, Carter, C. Olah, and T. Henighan, "Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet." Transformer Circuits Thread, 2024. [Online]. Available: http://taansfomne--irruuuui wwwp///////aalinng-.ao mmonssenaniictty/nneess..
[30] L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu, "Scaling and evaluating sparse autoencoders." " in The Thirteenth International Conference on Learning Representations. 2025. [Online]. Available: https//ooppnieievwnne/oo//////ss
[3]] A. Vaswani, "Attention is all you need," Advances in Neural Information Processing Systems. 2017.
[32] A. Huang et al. "Similarity measures for text document clustering." in Proceedings of the sixth new zealand computer science research student conference (NZCSRSC2008), Christchurch, New Zealand, vol. 4, 2008, pp. 9-56.
[33] C. Aggarwal, "Data mining the text book," 2015.
[34] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, "Chain- of-thought prompting elicits reasoning in large language models," " in Proceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS '22. Red Hook, NY, USA: Curran Associates Inc., 2022.
Le,
[35] M. Conover, M. Hayes, A. Mathur, J. Xie, J. Wan, S. Shah, A. Ghodsi, P. Wendell, M. Zaharia, and R. Xin. "Free dolly: Introducing the world's 's first truly open instruction-tuned Ilm." 2023. [Online]. Available: hhtps htwpabbbrccck.comooog//////00000
firtlll---istttt-oo-o-oe-e wwwpeeeeoooeeercalllllliiiiiieie viable iiable-intttuuttoon- tuned- I1m
[36] A. Köpf, Y. Kilcher, D. von Rütte, S. Anagnostidis, Z.-R. Tam, K. Stevens, A. Barhoum, N. M. Duc, O. Stanley, R. Nagyti. S. ES. S. Suri, Glushkov, Dantuluri, Maguire, C. Schuhmann, H. Nguyen, and A. Mattick, "Openassistant conversations democratizing large language model alignment," in Proceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS '23. Red Hook, NY, USA: Curran Associates Inc., 2023.
12

---

[37] D. Hendrycks, C. Burns, S. Basart. A. Zou, M. Mazeika, D. Song, and J. Steinhardt, "Measuring massive multitask language understanding." in International Conference on Learning Representations, 2021. [Online]. Available:
[38] A. Srivastava, A. Rastogi, A. Rao, A. M. Shoch, A. Abid, A. Fisch et al., "Beyond the imitation game: Quantifying and extrapolating the capabilities of language models." Transactions on Machine Learning Research, 2023, featured Certification. [Online]. Available: hutps//opeeneviee.eeffoooo waa.uumiiiiuuiiaiaa To55s
[39] K. Cobbe, Kosaraju, M. Bavarian, Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, Hilton, R. Nakano, C. Hesse, and J. Schulman, "Training verifiers to solve math word problems." arXiv preprint arXiv:2110.141666 2021.
[40] J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le et al., "Program synthesis with large language models," arXiv preprint XrXiv: 108.07732, 2021.
[41] D. Rein, B. L. Hou, A. C. Stickland, J. Petty. R. Y. Pang. J. Dirani, J. Michael, and S. R. Bowman, "GPQA: A graduate-level ppogle-prood q&a benchmark," in First Conference on Language Modeling, 2024. [Online]. Available: https://peeneeiieeeee////oee
[42] J. H. Clark, E. Choi, M. Collins, D. Garrette, T. Kwiatkowski, V. Nikolacv, and J. Palomaki, "TyDi QA: A benehmark for information-sceking question answering typologically diverse languages." " Transactions of the Association for Computational Linguistics, vol. 8, pp. 454-470, 2020. [Online]. Available: httpss//aclantholoyy.oooo 2020.tacl-1.330/
[43] L. Gao. J. Tow, B. Abbasi, S. Biderman. S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, A. Le Noac"h h, H. Li, K. McDonell, N. Muennighoff, C. Ociepa, J. Phang, L. Reynolds, H. Schoelkopf, A. Skowron, L. Sutawika, E. Tang, A. Thite, B. Wang, K. Wang, and A. Zou, "A framework for few-shot language model evaluation," " 07 2024. [Online], Available:
[44] W. Kwon, Z. Li, S. Zhuang. Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica. "Efficient memory management for large language model serving with pagedattention." in Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[45] D. Groeneveld, Beltagy, E. Walsh, A. Bhagia, R. Kinney, O. Tafjord, Jha et al., "OLMo: Accelerating the science of language models," in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguisties, Aug, 2024, pp. 15 789 15 809. [Online]. Available: https://cclanhollgg.rr///// a0l
[46] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaci, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosalc al., "Llama Open foundation and fine-tuned chat models," arXiv preprint 2t12237228 2023.
[47] Y. Zheng, R. Zhang, Zhang, Ye, Z. Luo. Z. Feng. and Ma, "Llamafactory: Unified efficient fine-tuning of 100+ language models," in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). Bangkok, Thailand: Association for Computational Linguisties, 2024. [Online). Available:
[48] OpenAL, "Gpt-4 technical report," arXiv preprint arXiv:2303.08771 2023.
[49] Z. Li and T. Zhou, "Your mixture-of-experts LLM is secretly an embedding model for free," in The Thirleenth International Conference on Learning Representations. 2025. [Online]. Available:
[50] N. Muennighoff. I.. Soldaini, D. Groeneveld, K. Lo, J. Morrison, S. Min, W. Shi, F. P. Walsh, O. Tafjord, N. Lamhert, Y. Gu, Arora, A. Bhagia, D. Schwenk, D. Wadden, A. Wettig, B. Hui, T. Dettmers, D. Kiela, A. Farhadi, N. A. Smith, P. W. Koh, A. Singh, and Hajishirzi. "OLMoe: Open mixture-of-experts language models," in The Thirteenth International Conference on Learning Representations, 2025. [Online], Available: https://ppenreviie..e//owwe http:/ooormmmixxxXxx-ee Xd-XxxXTTm
[51] S. Mallat and Z. Zhang, "Matching pursuits with fime-fryyuency dictionaries." " IEEE Transactions on Signal Processing, vol. 41, no. 12, pp. 3397-3415, 1993.
[52] B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, no. 6583, pp. 607-609, 1996.
[53] H. Lee, C. Ekanadham, and A. Ng. "Sparse deep belief net model for visual area v2," Advances in neural information processing systems, vol. 20, 2007.
13

---

[54] A. Makhzani and B. Frey, "K-sparse autoencoders." arXiv preprint arXiv:1312.5663 2013.
[55] D. Mossing, S. Bills, H. Tillman, T. Dupré la Tour, N. Cammarata, I. Gao, J. Achiam, C. Ych, J. Leike, 2024.
J. Wu, and W. Saunders, "Transformer debugger," http://ghhh...ooomooeeeaarraamaa
[56] X. Yang, S. Nie, L. Liu, S. Gururangan, U. Karn, R. Hou, M. Khabsa, and Y. Mao, "Divensity-driven data selection for language model tuning through sparse autoencoder," arXiv preprini 2rXiv:2502.140500 2025.
[57] D. Lee, J. Lee, G. Zhang, M. Tiwari, and A. Mirhoseini, "CATS: Context-aware thresholding for sparsity in large language models," in First Conference on Language Modeling, 2024. [Online]. Available: http:pppwwwweoooeeee///o/oomnn/d
[58] S. Elfwing. E. Uchibe, and K. Doya, "Sigmoid-weighted linear units for neural network function approximation in reinforcement learning," " Neural Networks, vol. 107, pp. 3-11, 2018, special issue on deep reinforcement learning. [Online]. Available: http::/wwww..eeeneececccooooeeea $0893608011302976
[59] Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding." in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, Eds. Minncapolis, Minnesota: Association Computational Linguisties, Jun. 2019, pp. 4171 4186. [Online]. Available: https//accatbboeey...///1111111
[60] X. H. Lå, "Bm25s: Orders magnitude faster lexical scarch via cager sparse scoring," 2024. [Online]. Available: https://ariiv.rr//a///44.... 03618
[61] Artetxe and H. Schwenk, "Massively multilingual sentence embeddings for zero-shot cross-lingus lingull transfer and beyond," Transactions of the Association for Computational Linguistics, vol. 7, pp. 597-610, 1038/
2019. [Online]. Available: https://aclanhhology.oom
[62] M. Artetxe, Ruder, and D. Yogatama, "On the cross-linguss lingull transfcrability of monolingual representations." in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds. Online: Association for Computational Linguistics, Jul. 2020, pp. 4623-4637. [Online]. Available: https://cclamhoolog..rr/// wttp:/200000.laama2222222222
[63] Z. Zhu, Y. Xu, L. Chen, J. Yang, Y. Ma. Y. Sun, H. Wen, J. Liu, J. Cai, Y. Ma et al. "Multi: Multimodal understanding leaderboard with text and images," arXiv preprint 240.0:2200..333... 2024.
[64] Y. Wang. Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi, "Self-instruct: Aligning language models with self-generated instructions," in Proceedings of the 61 Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A. Rogers, Boyd-Graber and N. Okazaki, Eds. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 13 13484-13 508. [Online]. Available: butps://ccaattooooy..rr/// acl-.cccl-oog... 774/
[65] P. M. McCarthy and S. Jarvis, "Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment," Behavior Research Methods, vol. 42, no. 2. pp. 381-392 2010. [Online]. Available: httpss:/pubmed.nr nebi www....h..oo/2247911777
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA]-
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1 (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providded a proper justification is given (e.g., "error bars expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
the checklist one of the factors in their evaluation.
reported because it would be too computationally
"[No] " or "[NA] is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, so please just use your best, judgment and write justification elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately reflect the main contributions and scope of the paper. All claims-regaan
the proposed model-centric, monosemantic activation-based data selection method, the new similarity metric, and the comprehensive evaluation are directly addressed and validated in the subsequent sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
15

---

Answer: [Yes]
Justification:
The limitations of this work are discussed in
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Formulas in the paper are numbered and referenced properly. Necessary brief theoretical justifications are in Appendix D.5
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
16

---

The paper provides all necessary details for reproducing the main experiments, including descriptions of datasets, model architectures, training procedures. hyperparam- eters, evaluation metrics, and data selection settings. All details are in 3 Appendix A
Justification:
Appendix [ Appendix C and Appendix [
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Core components of our code are provided in the supplemental material. The full code with instructions will be released upon final organization.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
pub lub/iciien guides/ Codees//odeSuummi wssonnolicco for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https)
for more details.
17

---

The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
reproducible, they paper) is recommended, but including URLs to data and code is permitted.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All details can be found in $ Appendix Appendix Appendix D
Appendix C and
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Error bars are not reported because it would be too computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
18

---

Answer: [Yes]
Justification:
We provide the computer resources in Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:i/ttttpps...............ppa
Answer: [Yes]
Justification: The research conforms the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discuss the broader impacts in $
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
19

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: This work is built upon open- source assets and cited them properly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, (paperswithhoo com/ dat aset s has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
20

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Details are provided
3.3 and Appendix B.2
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy /ttttp::/neurrii http://rrrs..ccooooeeeenneee///e 2005/
L.L.) for what should should not be described.
21

---

Impacts and Limitations
Our proposed data selection framework MoNA offers neuroscience-Inspired perspective for improving task-specific instruction tuning. By enhancing the stability, semantic expressiveness, and interpretability of data selection, MoNA has the potential benefit a broad range of downstream language model applications and may inspire new research directions in data curation and model analysis, both in academia and industry.
While MoNA demonstrates strong effectiveness for task-specific instruction tuning, its extension to other stages-suue
pre-training data selection-as been explored in this work. In addition, our current study focuses solely on the text modality. Applying MoNA to multimodal data selection scenarios, for example image-text tasks, remains an open and promising direction for future research.
A Training Details
We provide details on the input data formatting for instruction tuning across the three backbone models evaluated in our work: LLaMA3.1-8B OLMo-7B, and LLaMA2-13B. For each model, adopted the format recommended in its official documentation or open-source release. Below, we present concrete data example for each model. Additionally, all experiments are conducted on NVIDIA A100, A800, and H800 GPUs.
LLaMA3.1-8B
<lbegin_ of text>><lstart header_ jibliurrdddxssrreeendd header_idh idl>
doctor gives you three pills. She tells you to take one every half hour. How long will the pills last?<leot bent<cleo_ddl><<lttrr idl><<sttrr Shaeeheeaee_____ sonuderrddbassistaatteeee hheaeedeeeeddddeeeeeeeeaeeee idl>
One hour. You take the first pill immediately, then the other half-hour inter- vals.<leot_idb>
OLMo-7B
A doctor gives you three pills. She tells you to take one every half hour. How long will the pills last? One hour. You take the first pill immediately, then the other two at half-hour
LLaMA2-13B


 A doctor gives you three pills. She tells you to take one every half hour. How long will the pills last? 
 One hour. You take the first pill immediately, then the other two half-hour intervals. 

B Evaluation Details
B.1 Evaluation Tasks Details
For MMLU and MBPP, directly their respective validation sets representative examples. For the remaining datasets without validation sets, we follow the strategies outlined below:
We randomly select 100 samples from the training set to representative examples.
GSM8K:
We extract representative examples by selecting the provided few-shot samples in the task setup.
BBH:
GPQA:
We use the extended 98 data points, which are the "extended split" minus the "main split." "
TydiQA: Following MT] select one sample per language the representative example.
22

---

Table 3: Details of all evaluation tasks
Task
D
Test Samples
Shot
Metric
Harness Task Name
MMLU
285
18, 721
5
Accuracy
mmlu
GSM8K
100
2, 638
8
Exact Match
gsm8k_cot bbh
BBH
920
3
Exact Match
MBPP
90
500
3
pass@1
mbpp
GPQA
98
448
0
Accuracy
gpqa_mane wwl__mainn__n_nn serainzeroshot
TydiQA
5, 077
1
F1
For more statistical information, evaluation metries, and details regarding the LM-evaluation--hame setup, please refer to Table 3
Prompt of LLM data analyst
You are expert in evaluating task-specific data selection strategies for instruction tuning of large language models (LLMs). Your task is to assess how effectively the selected training data improve the performance of LLMs specific target task.
#### Context:
1. You are provided with a **representative example** of the target task.
2. Only small sample of the selected data provided reference due space limitations.
3. Your goal is to evaluate how well the selected data would help fine-tune the LIM to enhance its performance scoring the model's s performance on the target task after being fine-tuned with the provided sample data.
the example of the target task. You will accomplish this by
### Instructions:
1. Consider how well the sampled training data aligns with the example in terms of: Semantic similarity: How similar are the contents or instructions to the target task example?
Instruction format compatibility: Are the input-output structures of the selected data compatible with the target task?
Potential for improving generalization to this target task: How much does the sampled data appear address challenges in the target task?
Rate the effectiveness of the training data on a scale of to 10 for the target task example, where:
means the training data is completely irrelevant or harmful.
10 means the training data is highly relevant and likely to maximize performance.
3. Provide a short explanation for your rating.
### Representative Example:
### Sampled Training Data:
i
### Output:
For the given target task example and sampled training data, provide the following:
1. A rating (1-10) for the sampled data based on the criteria above.
A brief explanation justifying your rating.
B.2 LLM Data Analyst Details
We select GPT-4o-mini 48 as the L.M data analyst. The prompt used shown above. When calling the API, the temperature is set 0.8.
23

---

B.3 Polysemantic Neuronal Activation Extraction Details
We adopt the method from CATS 57 extract polysemantic neuronal activations from specified layer of the large language model, specifically within the Gated-MLP block. Mathematically, let x Rd denote the input the Gated-MLP. The polysemantic neuron activation pattern is computed as
nuly Topk ([SiLU (W game)]))
(10)
where W gate the learnable weight matrix in the block, and SiLUU (-) the activation function [8 The operator (.) retains only the largest K values of the input vector and sets all remaining entries to zero, consistent with the extraction procedure for the monosemantic activations.
To ensure a fair comparison, both polysemantic and monosemantic activations are extracted from the same layer and with the same value of K, used in the results of Figure [ and Figure 4
C Baseline Details
C.1 MATES
We implement MATES based on the official repository Similar to the original work, we trained data model based on bert-bbae--ncase batch size is 32, the learning rate is 5e-5, and we trained for 5 epochs with weight decay of 0.01. All other hyperparameters were kept consistent with the original work.
14
[59 with a maximum sequence length of 4096. The
C.2
LESS
The experiments were run using the official repository" Apart from the hyperparameters mentioned all other parameters remain consistent with the original work.
15
in 3.1
C.3
DLRDS
For this method, we considered two data selection models: bge- Ogg--base-n-vil and LLaMA3-BB In Appendix D.2] since the experiments were conducted on Chinese data, used bge-base zh. When selecting data with this method, we applied the Cosine similarity metric. When using bge age-base- lbass-en-veee en-v1.5 or bge baseaaseeea zh, if a sentence exceeded the model maximum token length (e.g.. 512), we split the sentence into con-loverag chunks that fit within the length limit. The embedding for the entire sentence computed as the average of the embeddings of all chunks.
C.4 BM25
To ensure speed, used the lo22
60 toolki
C.5
DSIR
We implement based on the official repository
17
C.6
LLM2
We used the official open-source implementation 18 and the released model HLM2Vec-Sheared-Laa. for all experiments with this method.
D Additional Experimental Results and Analysis
D.1 Case Study: LLM as Data Analyst
Figure shows that, compared to other data selection methods, the LLM data analyst consistently finds that our approach has clear advantage. To further analyze why MoNA receives higher scores,
LESS

---

BLRDS-BGE Rating=2
MoNA: Ratingss
The sumpled waining daza is langly inclerant is the ungel tack commll: which
The sampled triiiigg data pessents denototrute the stuctus and compleciy of the langugg used fhe urget task. Bowzver the aliyment with the procided repeorntaiive cumple emesle limite White the data dice minnuin focia isralation and conicsuaal intestrniing it lakess dieeet cornestion the spccific nisnaes
taak-spocifi dislogucs and comesss that
moulves srewering queette queezon bood on provided tour sboue location it
Paards [iee tnining den powsted comsess of compintly uneekrol lopaz litliding discustios of Russion litriture cookiis itcipes and varins odhat Bfjject that do aligt wis the requireents of die Surget 1888 There is no semantic smillakiy compunbilly the intruction fornat, and de sarrllod data does adiress challenges that would help ingueve jeteraliasion to the
und of irmaaaanng from Gujanii Englash soen the cuumle.. The of variene lingregr camples provides some but it instuatioral cxliural intricciios fidd the original contest.
fully caporee the
bok Overall, bake relooenc and daese supeo effective instmattonn uning for the spanific smoserinn quolions bace
The strength
euamples. lieguisti
LESS: Reting=33
could celasce
The sampled traising date provided ennweenngg specific question shout ten passgee The example roquies ficusel guddioe-ameerr forms specifoaally beaed singls coniesi However. smeled dat cossits lagely divens quoiiin types, sich is tatalation postis messss, and garrn krowlsigg gaseims which tur sham a similan smonti soncect simsium fimrat with the esample While some untiic fhe sempted data involve question arswer pais, they do not ettectiely tagett the some roding comprehersion or concise informction extraction that the exompl demorstiates This wesk computiiiity resuls in linited poscatial for impeoking generaliazion to the specific lagett task.
algg cloaely with cxzmple
undendtrnding rameerrr
The limitaiooss
dinect correaatinn specttic knguage pair (Cayamati liggisht roquird oegainal cuatpple
Linnited diverity types sestences provided, which redaces leaadth of learning appoctinities for model.
is somowiatt effeetive, does maximiie its potential foe the spezific tadk of Oujonii to Figgish iomoiation, hence the ofing
Overali, while
Figure 9: Explanation of the score given by the LLM data analyst for data selected by different methods
Table 4: Recall rate of the knowledge required for physical questions across different methods
Method
DIRDS-BGE
LESS
McNA (ours)
Hit@ 10
78.69
13.93
81.15
we provide a case study in Figure 9 which illustrates the LLM analyst evaluation for the same TydiQA example selected by different methods. In this case, DLRDS BGE receives a low score due selecting irrelevant topics, while LESS is penalized for choosing data that does not match the question-answe answerr format. These low scores are justified, since empirical evidence indicates that semantic relevance and format alignment between training and test data are critical for downstream performance. In contrast, MONA selects data satisfying both criteria, even though the samples are in different language pairs. The LLM data analyst considers this selection acceptable, likely because semantically similar content across languages can also improve model performance, as supported by previous work 61 62).
D.2 Knowledge Recall for Domain-Specific Questions
To further evaluate data selection quality on specialized tasks, we conduct a knowledge recall experiment using the physics subset from (63. Each of the 122 physics questions covers specific knowledge points, with a total 84 distinct points represented. For each data selection method, we measure the recall rate of relevant knowledge, that is, the proportion of target knowledge points that are present in the selected training data, As reported in Table MoNA achieves higher recall rates compared to DLRDS --EE, while LESS demonstrates low recall success rate. This difference likely arises from a greater mismatch between the distribution of knowledge data and question data for some methods, leading to larger gradient discrepancies and less effective selection. These results indicate that MoNA is more effective in retrieving domain domain-specific knowledge essential for answering specialized questions.
D.3 Effeet of Data Relevance on Fine- Fine-Tuning Performance
To assess the impact of data relevance, we conduct experiments comparing fine-tuning performance using data that is either highly similar or highly dissimilar to the target task, as determined by Jaccard similarity. Specifically, we select subsets of training samples that are the most similar or most dissimilar to the target task, and use each subset for instruction tuning. As shown in Table using dissimilar data for fine-tuning leads to significant reduction in performance across benchmarks, especially on GSM8K, whereas using similar (relevant) data consistently yields better results. These findings underscore the importance of selecting task relevatt data and validate the effectiveness of Jaccard similarity in our data selection framework.
25

---

Table 5: Performance comparison of fine-tuning with data dissimilar versus similar to the target task (measured by Jaccard similarity, Dat OPENHERMES-2..5
Selection
MMLU
GSM8K
BBH
Avg.
dissimilar
63.38
33.06
61.17
52.54
similar
64.49
67.93
66.44
66.29
Table 6: Performance of different data curation methods
TydiQA (Duc LESS)
Method
Data Curation Paradigm TydiQA data synthesis
GPT-40 (Self-instruce
46.99
MoNA (ours)
data selection
72.60
D.4
Comparison with Self-Instruct Data Synthesis
To further compare data curation paradigms, we evaluate MoNA against a GPT-4o-based self-instruct
method 664 19
that synthesizes instruction data (data synthesis), commonly adopted in recent reports the TydiQA performance for both approaches. MoNA (data selection) achieving score of 72.60 versus 46.99.
literature. Table O
significantly outperforms GPT-4o (self-instruu This disparity can be attributed to the tendency of self-instruct methods to generate large amounts of data that may become increasingly redundant and less diverse as the volume grows. In contrast, MoNA leverages data selection to curate a more relevant and diverse subset from an existing dataset. When sufficiently large pool of real data available, selecting high-quality samples may be more effective and straightforward approach than synthesizing new data.
D.5 Analysis of Length Bias in Token Aggregation
Empirical Analysis Table 7 presents the performance and average selected sample length for random selection, MoNA without length normalization, and MoNA with length normalization across multiple tasks. When sum aggregation is used (i.e., without normalization). MoNA tends to select samples whose lengths closest to the average length of the representative target samples, rather than samples with the highest semantic relevance, and model performance suffers as a result. In contrast, applying length normalization (mean aggregation) effectively removes this bias, enabling the selection of samples with a broader range of lengths and higher semantic utility. This results in significant improvements in downstream task performance. These empirical findings demonstrate that length normalization in token aggregation essential for mitigating length bias and achieving more reliable data selection for MoNA.
Below, we show why sum aggregation induces a length bias, and how mean aggregation mitigates it. Let each sample s; consist of n, tokens, each mapped to a non-negative d-dimensional activation vector Zzjj. Under sum aggregation, the sample embedding is
Theoretical Analysis
Zi.j
(11)
We use the generalized Jaccard similarity between z; and the task prototype " defined as:
I (zi, 2 E. min (2, [k], (A))
(12)
maxx(2; (k))
In sum aggregation, the magnitude of z [k] scales linearly with the number of tokens since 2, [k) es my, Ug. where m,, is the number of tokens s; that activate dimension k. and va the typical activation value. Similarly, 2 k] reflects the activation strength of the target prototype, which influenced by the average length of representative samples of the target task. When the length n, of is similar to the length associated with 2 the magnitudes of z, [k] and 2 2 are comparable. If their activated dimensions overlap well, the Jaccard similarity is maximized. However:
26

---

Table 7: Performance and average selected sample length for RANDOM, MoNA w/o length normaliza- tion, and MoNA w/ length normalization (LN) across tasks (D = OPENHERMES-2.55 The "Target Length" column shows the average length of representative target samples.
Target Length
RANDOM
MoNA w/o LN
McNA w/ LN
Task
Performance
Length
Performance
Length
Performance
Length
MMLU
150.85
64.39
62.88
152.76
64.49
499.41
GSM8K
189.66
60.05
64.06
212.70
67.93
322.74
BBH
303.49
64.63
391.34
65.55
313.82
66.44
492.14
MBPP
110.38
49.67
46.20
202.67
48.40
404.03
GPQA
370.54
30.43
27.00
309.88
31.47
632.61
Table 8: Performance comparison using LESS data: polysemantic vs. monosemantic activations
Neuronal Activation
MMLU
BBH
TydiQA
Avg.
polysemantic
63.57
63.28
71.01
65.95
monosemantic
64.78
64.21
72.60
67.20
w/o SAE
62.11
61.54
71.91
65.19
If n, much larger than effective length of z then z:[k] >> 2 for many dimen- sions k; leading to min (z; [k], [A]) [ and max(z; (z. [k], (A]) 2, [k:] Consequently,
I (Zi, 2
1, reducing the similarity.
nf is much smaller, [
z yiclding J (z,, 2 Ee 2A
1, again lowering the
similarity.
Thus, samples with lengths n, close to the length of 2 achieve higher similarity scores, introducing a length bias in the selection process.
In contrast, mean aggregation normalizes each sample embedding by the number of tokens. This normalization ensures that the magnitude the embedding does not depend on the sample length. As result, the selection process is longer biased toward samples with lengths similar to the prototype. Instead, comparisons focus purely on the similarity of activation patterns, eliminating the systematic length bias observed with sum aggregation and enabling selection based semantic relevance.
D.6 Effect of SAE
Monosemantic activations, produced by the sparse autoencoder, consistently outperform polysemantic activations across all tasks on the LESS dataset (Table This demonstrates that disentangling neuron polysemanticity via SAE leads more effective data selection and superior downstream performance. Additionally, we report results using the raw hidden states of the selected layer without SAE mapping ("w/o SAE"). This baseline underperforms both polysemantic and monosemantic representations, underscoring the importance of explicit disentanglement with SAE for optimal data selection.
Table 9: Lexical Diversity (MTLD, larger is better) of Selected data
Method
MMLU
BBH
TydiQA
RANDOM
61.31
61.31
61.31
MATES
68.25
56.83
81.01
LESS
76.74
53.56
77.80
BM2.5
74.89
58.41
66.65
DSTR
52.10
48.90
54.01
DLRDS-BOS
50.79
44.63
70.63
DLRDS -00003
49.38
40.50
82.15
MoNA (ours)
66.05
42.49
81.38
27

---

D.7 Diversity of Selected Data by MONA
We measure the lexical diversity of the selected data samples using the MTLD (Measure of Textual Lexical Diversity) metric [65], computed with the LexicalRichness" package. A higher MTLD score indicates greater lexical diversity. Table confirm that a stronger emphasis on data quality in target-specific selection indeed leads to reduced data diversity.
D.8 Detailed Experimental Results
We present complete results for all tasks and settings discussed in the main text. To facilitate understanding, we summarize key findings in the main text using figures and overview tables, while the following tables provide detailed, task-level results and additional experimental details. These comprehensive tables (TO) 12 13 (4) complement the main text by offering the full performance breakdown, allowing readers reproduce, verify, further analyze various aspects of the experiments.
Table 10:
Detailed results of Figure 3 (Dec LLES)
1 %
10%
Method
MMLU
BBH
TydiQA
Avg.
MMLU
BBH
TydiQA
Avg.
RANDOM
65.71
6.77
61.77
44.75
63.23
63.35
71.14
65.90
MATES
63.22
53.00
51.15
55.79
63.62
64.28
70.04
65.98
LESS
63.58
56.59
57.31
59.16
63.27
61.77
70.56
65.20
BM25
63.96
47.50
67.36
59.61
63.47
62.06
71.15
65.56
DSIR
64.91
0.20
56.73
40.61
63.17
61.07
67.22
63.82
DIRDS-BGE
64.71
65.58
52.74
61.01
63.45
61.59
71.61
65.55
DLRDS LLaMM3- 8B
63.48
59.53
62.31
61.77
62.28
59.91
72.57
64.92
MoNA (ours)
64.57
56.67
66.77
62.67
62.85
62.94
72.74
66.18
Table 11: Detailed results of Figure
Method
GSM8K (D" OPENHERMES-22.1
TydiQA (D LESS)
DIRDS-BGE
1.35
2.33
LESS
1.56
2.44
MoNA (ours)
4.39
4.11
Table 12: Detailed results of Figure 6 (a) (D
OPENHERMES 2.5)
Layer
MMLU
GSM8K
BBH
MBPP
GPQA
Avg.
63.94
63.15
64.75
48.20
31.92
54.39
12
64.06
63.46
64.75
48.40
29.24
53.98
64.73
67.02
66.47
49.20
30.80
55.64
20
64.36
68.54
65.21
49.00
30.80
55.58
24
64.13
69.52
65.27
52.00
27.68
55.72
26
63.11
67.10
64.06
49.20
31.03
54.90
31
64.49
67.93
66.44
48.40
31.47
55.75
Table 13: Detailed results of Figure 6 (Dote OPENHERMES -2.5)
K
192
96
48
24
GSM8K
67.93
67.48
66.41
64.29
E
Algorithm
complete description of our algorithm provided (Algorithm I in the form pseudocode, to facilitate reproducibility and implementation in future work.
28

---

Table 14: Detailed results of Figure o (Dsc
OPENHERMESS-2.5
Method
MMLU
GSM8K
BBH
Avg.
Cosine
63.30
66.72
61.33
63.78
Ruclidean
63.03
65.73
61.62
63.46
Jaccard 64.49
67.93
66.44
66.29
Algorithm MoNA: Task-Specific Data Selection with Monosemantic Neuronal Activations
Require: Source dataset Derc target set Digt data selection model ds chosen layer L,, trained sparsity K, selection size n
Ensure: Selected subset Deel C Dare of size
1: For each sample s in Dere D'gt
2: Compute monosemantic activation 2, in Eq. (6) and aggregate sample-level in Eq.
3: Compute target prototype ze using Eq. (8) on all 2, in Digt
4: for each source sample in Dere do
5: Compute similarity s; between z, and zs as in Eq.
6: end for
7: Select Deel the n samples in Dere with highest similarity 8;
8: return Deel
29


Reviews:
[{"review_id": "j3XinXeYN5", "reviewer": "Reviewer_wUy5", "summary": "The paper proposes a new method of data selection for task-specific fine-tuning by leveraging sparse autoencoders to obtain monosemantic activations for selecting task-specific training data. The method selects training data by comparing the similarity between the monosemantic embedding of a given data point and that of the task prototype data. Extensive experiments on task-specific fine-tuning show that the proposed method outperforms multiple data selection baselines from the literature.", "questions": "See the weakness above.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper proposes a novel data selection method with a good motivation for disentangling polysemantic embeddings of training data examples into monosemantic embeddings.\n\n2. The paper designs a general distribution alignment pipeline and explores different critical design choices within the pipeline.\n\n3. The paper conducts thorough experiments, compares with a wide range of well-established data selection baselines from the literature, and shows strong empirical results across the board.\n\n4. The paper provides thorough ablation analyses of the method, potentially leading to future work in this direction.\n\nWeakness\n\n1. There is a lack of analyses on the impact of the SAE model. The main novelty of the paper lies in using SAE to embed the training data, but it is unclear in the experiments if the quality of the SAE embeddings, especially how monosemantic/sparse the SAE embeddings are, has an impact on the quality of data selection. However, the paper only uses the same one SAE model for all experiments. \n\n2. There is a lack of comparisons with other monosemantic embeddings. This is not a critical weakness, but I can imagine applying the same workflow but swapping SAE with another model that can generate relatively monosemantic embeddings, especially MoE models: prior work has shown that MoE models can be used effectively as text embedders out of the box [1] and experts of pretrained MoE can be specialized to process a very specific set of tokens [2], leading to potentailly monosemantic representations.\n\n[1] Ziyue Li, Tianyi Zhou. Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free. https://arxiv.org/abs/2410.10814\n\n[2] Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A. Smith, Pang Wei Koh, Amanpreet Singh, Hannaneh Hajishirzi. OLMoE: Open Mixture-of-Experts Language Models. https://arxiv.org/abs/2409.02060", "was_revised": true, "final_justification": "The new experiments the authors added in the rebuttal response addressed both of my concerns.", "rebuttal": "## Response to Reviewer wUy5\nThank you very much for your insightful and constructive comments, which have greatly contributed to improving the quality of our work.\n\n---\n\n### Supplementary Analysis Using an Additional SAE Model\nFollowing your suggestion, we train a new SAE model on a larger backbone (LLaMA2-13B). Specifically, we follow the conclusion from Section 3.5 of our manuscript, extracting neuron activations from the penultimate layer. The training data used is *RedPajama-Data-1T-Sample* [1], and we utilize the *EleutherAI-sparsify* [2] training framework for SAE training.\n\n**Table 1: Results of Using Different SAE Models on LLaMA2-13B**\n| Method     | SAE Model   | MMLU  | GSM8K | BBH   | MBPP  | GPQA  | Avg.  |\n| ---------- | ----------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| BASE       | -           | 55.11 | 24.03 | 46.74 | 27.00 | 30.58 | 36.69 |\n| FULL       | -           | 57.61 | 55.95 | 52.63 | 35.00 | 27.01 | 45.64 |\n| MoNA(ours) | LLaMA-3-8B  | 57.26 | 60.27 | 52.23 | 35.60 | 27.90 | 46.65 |\n| MoNA(ours) | LLaMA-2-13B | 58.73 | 60.12 | 52.48 | 36.60 | 27.90 | 47.17 |\n\nAs shown in Table 1, we compare the performance of our method (MoNA) using different SAE models (trained over LLaMA-3-8B vs. LLaMA-2-13B), alongside the BASE and FULL baselines. The results demonstrate a larger SAE backbone can further enhance the effectiveness of data selection and downstream performance. These findings suggest that the quality and scale of the SAE model have a positive impact on the overall results.\n\n[1] https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T-Sample\n\n[2] https://github.com/EleutherAI/sparsify/tree/main\n\n---\n\n### Comparison with MoE-based Monosemantic Embeddings\n\nWe greatly appreciate this insightful and stimulating suggestion. Following your advice, we replace SAE with MoE-based embeddings, specifically adopting the approach described in your referenced work [1]: for each token, we use the concatenation of expert routing scores from all layers of a MoE model (obtained during a forward pass) as the token’s embedding. For this experiment, we use the MoE model you recommended, *allenai/OLMoE-1B-7B-0924* [2]. Apart from substituting the SAE-based embeddings with MoE-based embeddings, all other components and settings of our workflow were kept the same to ensure a fair comparison.\n\n**Table 2: Evaluation of Monosemantic Embeddings: SAE-based vs. MoE-based**\n| Method     | Embedding | MMLU  | GSM8K | BBH   | MBPP  | GPQA  | Avg.  |\n| ---------- | --------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| BASE       | -         | 65.30 | 55.50 | 63.08 | 46.40 | 28.12 | 51.68 |\n| FULL       | -         | 64.60 | 65.35 | 64.31 | 49.00 | 27.90 | 54.23 |\n| MoNA(ours) | SAE-based | 64.49 | 67.93 | 66.44 | 48.40 | 31.47 | 55.75 |\n| MoNA(ours) | MoE-based | 64.02 | 65.43 | 63.83 | 49.00 | 31.03 | 54.66 |\n\nAs shown in Table 2, under our current experimental setup, using MoE-based embeddings yields a **modest** improvement over the FULL baseline (54.66 vs. 54.23 in average score), indicating that monosemantic representations from MoE models can provide a slight benefit for task-specific data selection. However, MoE-based embeddings still lag behind SAE-based embeddings by a noticeable margin, as reflected in the average scores (55.75 vs. 54.66).\n\nBased on our experience, we suspect several factors may contribute to this difference:\n* *MoE-based embedding utilization in data selection requires further investigation.* The effectiveness of MoE-based embeddings for task-specific data selection may depend on design choices such as which layer’s expert routing scores to use and how to measure similarity in the MoE embedding space. Our current approach simply concatenates routing scores from all layers, but more targeted strategies (e.g., selecting specific layers or aggregating expert activations differently) may yield better results.\n* *MoE-based embeddings may primarily capture domain-level monosemanticity [2] rather than fine-grained semantic distinctions.* For example, arXiv is a common source of training data for language models and contains a wide variety of text, including both academic citations and HTTP request patterns. While SAE-based embeddings are able to distinguish between the semantics of these different types of content [3], MoE models may route all tokens from arXiv data to the same expert [2], regardless of such finer-grained differences. This suggests that the monosemanticity provided by MoE-based embeddings is more aligned with broad domain or topic separation, and may not capture the more nuanced distinctions needed for our data selection workflow.\n\nCertainly, these are preliminary hypotheses informed by our previous experience and the rapid experiments we conducted based on your valuable suggestion. More comprehensive experiments and analysis are required. We believe that with further exploration and optimization, the potential of MoE-based embeddings for data selection can be better realized, and we consider this a promising direction for future work.\n\n[1] Ziyue Li, Tianyi Zhou. Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free. https://arxiv.org/abs/2410.10814\n\n[2] Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A. Smith, Pang Wei Koh, Amanpreet Singh, Hannaneh Hajishirzi. OLMoE: Open Mixture-of-Experts Language Models. https://arxiv.org/abs/2409.02060\n\n[3] T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, B. McLean, J. E. Burke, T. Hume, S. Carter, T. Henighan, and C. Olah, “Towards monose367 manticity: Decomposing language models with dictionary learning,” Transformer Circuits Thread, 2023, https://transformer-circuits.pub/2023/monosemantic-features/index.html."}, {"review_id": "uUg3SIiRjT", "reviewer": "Reviewer_g8cj", "summary": "This paper presents a data selection method for instruction tuning based on a distribution alignment pipeline. The approach selects training examples whose distributions align with the target set. Each sample is represented by its neuronal activation pattern in the model, and sparse autoencoders are used to disentangle polysemantic activations into sparse, monosemantic representations. Finally, the generalized Jaccard similarity is employed to compute embedding similarity. The proposed method (MoNA) outperforms other data selection methods and even surpasses training on the full dataset.", "questions": "1. In the experiment shown in Figure 3, how many training examples are sampled under the 5% selection ratio? Whether sampling 10% of the source data can surpass the full data? If not, the practicability of the proposed method is limited, as it overly relies on carefully tuning the sampling ratio for optimal performance. \n2. When scaling to 13B models, the superiority of the proposed method diminishes, which raises concerns about its scalability.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper is well-written and easy to follow, with clear motivation for each component. The experimental results (as shown in Figure 1(c)) validate the effectiveness of monosemantic activations. Additionally, Figure 6(c) provides valuable insight by demonstrating how the choice of similarity metric can significantly impact performance.\n2. Data selection for instruction tuning is a critical topic. The proposed method achieves significant improvements across six benchmarks, outperforming other data selection methods as well as two baseline methods (“full” and “random”).\n\nWeaknesses\n1. The method appears sensitive to the data selection ratio, as shown in Figure 3. Sampling 5% of the training data even outperforms sampling 10%, which is somewhat counter-intuitive. In the experiment shown in Figure 3, how many training examples are sampled under the 5% selection ratio?\n2. When scaling to 13B models, the superiority of the proposed method diminishes. In Table 2, across five test sets, MoNA achieves the best performance on only two, and lags behind other methods on the remaining three.", "was_revised": true, "final_justification": "I will keep my \"borderline accept\" rating unchanged. While the rebuttal partially addresses my concerns, I remain concerned about the scalability of the method. On the 13B models, the improvements are not significant, as the proposed method (MONA) surpasses the previous best method (BM25) by less than one point.", "rebuttal": "## Response to Reviewer g8cj\n\nThank you very much for your thorough review and valuable feedback. We appreciate the time and effort you have devoted to evaluating our work, as well as your constructive comments and suggestions.\n\n---\n\n### Discussion on Performance at Larger Model Scales\n\nWe appreciate your observation regarding the diminished superiority of our method when scaling to 13B models, as highlighted in Table 2 of our manuscript. We would like to clarify that **in our original experiments, the SAE used for data selection was trained on LLaMA3-8B, whereas the downstream fine-tuning was performed on LLaMA2-13B**. This introduces a potential mismatch between the SAE model and the target model, which may have affected the observed performance.\n\nTo address this concern, we conduct an additional experiment where we train the SAE directly on LLaMA2-13B and used it for data selection. Specifically, following the conclusion from Section 3.5 of our manuscript, we extract neuron activations from the penultimate layer. The SAE is trained using the RedPajama-Data-1T-Sample [1] and the EleutherAI-sparsify training framework [2]. With this adjustment, we observe a further improvement in performance on the 13B model (see Table 1 below, MoNA (LLaMA2-13B SAE) vs. MoNA (LLaMA3-8B SAE)), demonstrating that using a matched SAE and target model is important for maintaining the effectiveness of our approach at larger scales.\n\n**Tabel 1:Performance Comparison of Data Selection Methods on LLaMA2-13B**\n| Method                      | MMLU  | GSM8K | BBH   | MBPP  | GPQA  | Avg.  |\n| --------------------------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| BASE                        | 55.11 | 24.03 | 46.74 | 27.00 | 30.58 | 36.69 |\n| FULL                        | 57.61 | 55.95 | 52.63 | 35.00 | 27.01 | 45.64 |\n| RANDOM                      | 55.96 | 41.50 | 51.37 | 31.13 | 28.64 | 41.72 |\n| MATES                       | 55.68 | 37.07 | 51.17 | 31.20 | 26.34 | 40.29 |\n| LESS                        | 60.38 | 48.75 | 50.42 | 25.80 | 27.90 | 42.65 |\n| BM25                        | 57.60 | 58.15 | 52.65 | 34.60 | 27.90 | 46.18 |\n| DSIR                        | 55.83 | 53.53 | 52.02 | 31.60 | 27.23 | 44.04 |\n| DLRDS-BGE                   | 56.65 | 56.63 | 52.34 | 35.60 | 27.68 | 45.78 |\n| DLRDS-LLaMA3-8B             | 58.65 | 52.31 | 52.36 | 35.20 | 26.56 | 45.02 |\n| LLM2Vec                     | 57.02 | 58.30 | 51.27 | 34.80 | 27.90 | 45.86 |\n| MONA (ours, LLaMA3-8B SAE)  | 57.26 | 60.27 | 52.23 | 35.60 | 27.90 | 46.65 |\n| MONA (ours, LLaMA2-13B SAE) | 58.73 | 60.12 | 52.48 | 36.60 | 27.90 | 47.17 |\n\n\n\n[1] https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T-Sample\n\n[2] https://github.com/EleutherAI/sparsify/tree/main\n\n---\n\n### On the Sensitivity of Data Selection Ratio\n* *Experimental Details of Figure 3 in Our Manuscript* In our exploration of the data selection ratio, we used the LESS dataset [1], which contains a total of 270,699 examples. Specifically,\n    * A 5% selection ratio corresponds to 13,534 sampled examples.\n    * On this dataset, training the model with 10% of the data does not outperform using 5%; moreover, the 10% subset also does not surpass the performance achieved by using the full dataset. This trend holds true for all baseline methods evaluated, as well as for our proposed approach.\n* *Additional Validation on OpenHermes2.5* We believe that the counter-intuitive phenomenon---where using 10% of the data results in worse performance than using 5%---is related to the characteristics of the dataset. To further investigate this, we conduct the same experiments on the OpenHermes2.5 dataset (1001551 samples) [2]. On OpenHermes2.5, we observe that using 10% of the data leads to better performance than using 5% (see Table 2 bellow). This indicates that the phenomenon observed on LESS is data-dependent and does not necessarily occur on other datasets.\n\n**Table 2: Performance of MoNA with Different Data Selection Ratios on OpenHermes2.5 (Using LLaMA3.1-8B)**\n| Method      | Ratio | MMLU  | GSM8K | BBH   | MBPP  | GPQA  | Avg.  |\n| ----------- | ----- | ----- | ----- | ----- | ----- | ----- | ----- |\n| BASE        | -     | 65.30 | 55.50 | 63.08 | 46.40 | 28.12 | 51.68 |\n| FULL        | 100%  | 64.60 | 65.35 | 64.31 | 49.00 | 27.90 | 54.23 |\n| MoNA (ours) | 5%    | 64.49 | 67.93 | 66.44 | 48.40 | 31.47 | 55.75 |\n| MoNA (ours) | 10%   | 64.58 | 68.16 | 66.63 | 49.60 | 31.03 | 56.00 |\n\n[1] M. Xia, S. Malladi, S. Gururangan, S. Arora, and D. Chen, “Less: selecting influential data for targeted instruction tuning,” in Proceedings of the 41st International Conference on Machine Learning, ser. ICML’24. JMLR.org, 2025.\n\n[2] Teknium, “Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants,” 2023. [Online]. Available: https://huggingface.co/datasets/teknium/OpenHermes-2.5"}, {"review_id": "dHmG7JLYxb", "reviewer": "Reviewer_TZjR", "summary": "This study introduces a model-centric data selection method for more efficient instruction fine-tuning (fewer data samples) while achieving the same or better performance than existing data selection methods or full fine-tuning.", "questions": "1. I would suggest even a preliminary evaluation of the lexical diversity of the selected question, in addition to the quality of the selected data samples. \n2. A clearer discussion of the caveats of this method (see weaknesses related to robustness implications) somewhere would be ideal  \n3. Figure 4 is a bit difficult to interpret -- how many tasks are these patterns representing?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths: (quality, clarity, originality) This work is well written and provides solid experiments and ablations to support the method of data selection based on model activations. Using model activations instead of data-specific features allows more efficient instruction fine-tuning. \n\nWeaknesses: (significance) My main concern with this work is the lack of exploration on the consequences of training with such a method. How does this affect model robustness to unseen instructions (within the same task?).  I see that the quality of the selected data points is evaluated, but there is a missing measurement of diversity here. Prior work shows that instruction-tuned models are sensitive to phrasings seen during training; does this method worsen this problem?", "was_revised": true, "final_justification": "I appreciate the authors for engaging with prior work, but I maintain that generalization and diversity are important for selecting high quality data. I will keep my score.", "rebuttal": "## Response to Reviewer TZjR\n\nThank you for your insightful and constructive suggestions, which greatly contribute to improving our work. We hope the following point-by-point responses effectively address your concerns.\n\n---\n\n### Potential Impact on Robustness and Diversity\n\nIn the context of **target-specific data selection**, the primary objective is to enhance model performance on the target task (characterized by a set of representative examples). To achieve this, we following Less [1], naturally place greater emphasis on data quality---specifically, selecting data samples that are most similar or relevant to the representative examples of the target task. This approach aligns with the general principle in machine learning that models tend to generalize better when trained on data closely aligned with the evaluation distribution.\n\nHowever, we acknowledge that this emphasis on data quality may inadvertently come at the expense of data diversity, as you have pointed out. Following your suggestion, we conducted two additional analyses to more clearly reveal and quantify this effect.\n\n*Robustness to Unseen Instructions* We select GSM8K [2] as the target task and perform instruction tuning using data selected based on GSM8K. We then evaluate the tuned models on both GSM8K (seen instructions) and MATH500 (unseen instructions). While both datasets focus on mathematical reasoning, GSM8K consists of simple, natural-language grade school math problems, whereas MATH500 contains more formal, complex, and diverse problems with distinct instruction styles. This setup enables us to directly test how well each method generalizes to novel instruction phrasings within the same domain. As shown in Table 1, target-specific data selection methods, which emphasize data quality, achieve strong performance on GSM8K but consistently underperform on MATH500, indicating that this focus comes at the cost of generalization to unseen instructions.\n\n**Table 1: Model Performance on GSM8K (Seen) and MATH500 (Unseen)**\n| Method          | GSM8K (seen) | MATH500 (unseen) |\n| --------------- | ------------ | ---------------- |\n| BASE            | 55.50        | 2.40             |\n| FULL            | 65.35        | 21.00            |\n| RANDOM          | 58.65        | 15.33            |\n| MATES           | 54.28        | 19.00            |\n| LESS            | 66.87        | 13.60            |\n| BM25            | 66.64        | 13.80            |\n| DSIR            | 66.94        | 14.20            |\n| DLRDS-BGE       | 64.82        | 16.20            |\n| DLRDS-LLaMA3-8B | 64.75        | 12.40            |\n| MoNA (ours)     | 67.93        | 15.80            |\n\n*Lexical Diversity of Selected Data* We measure the lexical diversity of the selected data samples using the *MTLD* (Measure of Textual Lexical Diversity) metric [4], computed with the LexicalRichness [5] package. A higher MTLD score indicates greater lexical diversity. Table 2 confirm that a stronger emphasis on data quality in target-specific selection indeed leads to reduced data diversity.\n\n**Table 2: Lexical Diversity (MTLD, larger is better) of Selected data**\n| Method          | MMLU  | BBH   | TydiQA |\n| --------------- | ----- | ----- | ------ |\n| RANDOM          | 61.31 | 61.31 | 61.31  |\n| MATES           | 68.25 | 56.83 | 81.01  |\n| LESS            | 76.74 | 53.56 | 77.80  |\n| BM25            | 74.89 | 58.41 | 66.65  |\n| DSIR            | 52.10 | 48.90 | 54.01  |\n| DLRDS-BGE       | 50.79 | 44.63 | 70.63  |\n| DLRDS-LLaMA3-8B | 49.38 | 40.50 | 82.15  |\n| MoNA (ours)     | 66.05 | 42.49 | 81.38  |\n\nIn the next version, we will explicitly clarify the scope of our method in target-specific data selection and more thoroughly discuss its limitations in terms of data diversity.\n\n[1] M. Xia, S. Malladi, S. Gururangan, S. Arora, and D. Chen, “Less: selecting influential data for targeted instruction tuning,” in Proceedings of the 41st International Conference on Machine Learning, ser. ICML’24. JMLR.org, 2025.\n\n[2] Cobbe, Karl, et al. \"Training verifiers to solve math word problems.\" arXiv preprint arXiv:2110.14168 (2021).\n\n[3] Lightman, Hunter, et al. \"Let's verify step by step.\" The Twelfth International Conference on Learning Representations. 2023.\n\n[4] McCarthy, Philip M., and Scott Jarvis. \"MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment.\" Behavior research methods 42.2 (2010): 381-392.\n\n[5] https://github.com/LSYS/LexicalRichness\n\n---\n\n### Clarification on Figure 4\nFigure 4 aims to demonstrate that, *in the monosemantic representation space, different tasks tend to activate distinct sets of neurons, while in the polysemantic space, many neurons are activated by multiple tasks.* To this end, we select two tasks---Math and Code---and randomly sample 100 data points from each. For each sample, we visualize the activations across the top-100 most variant neurons. The results show that in the polysemantic space, samples from both tasks frequently produce strong activations on the same neurons (as indicated by the prominent peaks in the upper plot), whereas in the monosemantic space, this overlap is greatly reduced and task-specific activation patterns become much clearer."}, {"review_id": "5sr5cu1U2W", "reviewer": "Reviewer_Uiye", "summary": "This paper introduces a novel method for selecting examples for instruction tuning by measuring their similarity to target task examples. The approach leverages the activations induced in a large language model (LLM), selecting examples that elicit similar internal representations. To better align these activations with individual concepts, a sparse autoencoder is employed, encouraging each activation to focus on a distinct concept.", "questions": "See in the section above", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1) The paper is well-written and easy to follow.\n\n2) The proposed method is simple and intuitive—in a positive sense.\n\n3) The experimental setup is thorough, featuring a wide range of strong baselines and diverse, representative datasets.\n\n4) The inclusion of ablation studies provides valuable insight into the components that contribute most to the method's effectiveness.\n\nSuggestions and Questions:\n\nWhile these may not be strict weaknesses, I have a few suggestions and questions for consideration:\n\n1) In classical active learning literature, diversity among selected examples is often emphasized to ensure broader coverage of the task space. Have you considered strategies to avoid selecting overly similar examples? It would be helpful to see an analysis of task coverage beyond simply demonstrating improved performance.\n\n\n\n2) I appreciated the intuition offered in Figure 4, which illustrates that without SEA, it is more challenging to separate activations for distinct tasks. However, do you have a corresponding analysis showing that using SEA leads to improved performance? In Figure 6b, it appears that increasing K generally improves overall performance. Am I missing something here? Is the effect of using \"raw\" activations explicitly represented in any of the baselines?", "was_revised": true, "final_justification": "I have two concerns, \nThe authors have clarified some misunderstandings I had and conducted a few experiments to validate one of my points.", "rebuttal": "## Response to Reviewer Uiye\nThank you for your valuable suggestions and the time you have dedicated to reviewing our work. Your feedback has been very helpful in improving our research.\n\n---\n\n### Clarification on SAE vs. Raw Activations Performance Comparison\nWe apologize if our writing led to any important information being overlooked. In fact, we have provided results comparing the use of SAE and raw activations in **Figure 1-(c) and Table 8 in Appendix D.6** of the manuscript. For your convenience, we have reproduced the detailed results below.\n\n**Table 1: Comparison of Different Neuron Activations**\n| Neuronal Activation | MMLU  | BBH   | TydiQA | Avg.  |\n| ------------------- | ----- | ----- | ------ | ----- |\n| polysemantic  [1]   | 63.57 | 63.28 | 71.01  | 65.95 |\n| monosemantic        | 64.78 | 64.21 | 72.60  | 67.20 |\n| w/o SAE             | 62.11 | 61.54 | 71.91  | 65.19 |\n\nTable 1 compares three types of neuronal activations:\n* *Polysemantic*: we adopt the method from CATS [1] to extract polysemantic neuronal activations from a specified layer of the large language model, specifically within the Gated-MLP block. Mathematically, let $x\\in\\mathbb{R}^{d_1}$ denote the input to the Gated-MLP. The polysemantic neuron activation pattern is computed as:\n$$z_\\text{poly}=\\text{TopK}(∣\\text{SiLU}(W_\\text{gate}x)∣),$$\nwhere $W_{\\text{gate}}$ is the learnable weight matrix in the block, and $\\text{SiLU}(·)$ is the activation function. The operator $\\text{TopK}(·)$ retains only the largest $K$ values of the input vector and sets all remaining entries to zero, consistent with the extraction procedure for the monosemantic activations. For fair comparison, both polysemantic and monosemantic activations are extracted from the same layer and with the same value of $K$.\n* *Monosemantic*: These activations are obtained using the SAE (Sparse Autoencoder) method, which aims to disentangle and sparsify the neuron activations, resulting in more interpretable, monosemantic representations.\n* *w/o SAE*: This refers to the activations before applying the SAE mapping, specifically the hidden states from the penultimate layer of the model.\n\nAs shown in Table 1, monosemantic activations produced by the SAE consistently outperform both polysemantic activations and raw hidden states (“w/o SAE”) across all tasks. This demonstrates that disentangling neuron polysemanticity via SAE leads to more effective data selection and superior downstream performance.\n\n---\n\n### Trade-off Between Data Quality and Diversity in Target-Specific Selection\nIn the context of **target-specific data selection**, our primary goal is to improve model performance on the target task, which is defined by a set of representative examples. To this end, and in line with Less [1], we place greater emphasis on data quality—specifically, by selecting samples that are most similar or relevant to the target examples. While this strategy is effective for optimizing performance on the target task, it inevitably introduces a trade-off: as you have noted, prioritizing data quality can reduce the diversity of the selected data.\n\nTo further investigate this trade-off, we explored a strategy to explicitly introduce data diversity into our method, as you suggested. Specifically, whereas our original approach selects samples most similar to the target task based on their SAE representations, our new strategy first ranks candidate samples in descending order of similarity to the target task. We then iterate through this ranked list, maintaining a set of neurons that have already been activated by previously selected samples. A candidate sample is selected only if it activates at least one neuron that has not yet been covered by the current selection. The detailed algorithm is provided in the following pseudocode.\n\n**Algorithm: Diversity-Enhanced Target-Specific Data Selection**\n```text\nInput:\n- Candidate samples C\n- Target task examples T\n- Similarity function sim(x, T) in SAE representation space\n- Neuron activation function neurons(x)\nOutput:\n- Selected sample set S\n1. For each x in C, compute similarity score s_x = sim(x, T)\n2. Sort all x in C in descending order by s_x\n3. Initialize S <- empty, N_covered <- empty\n4. For each x in sorted C:\n    - Let N_x = neurons(SAE(x))\n    - If N_x is not a subset of N_covered:\n        - Add x to S\n        - Update N_covered <- N_covered union N_x\n5. Return S\n```\n\n*Impact of Diversity-Enhancing Strategy on Lexical Diversity of Selected Data* We measure the lexical diversity of the selected data samples using the MTLD (Measure of Textual Lexical Diversity) metric [2], computed with the LexicalRichness [3] package. A higher MTLD score indicates greater lexical diversity. As shown in Table 2, introducing the diversity-enhancing strategy (“MoNA+diversity”) substantially increases the lexical diversity of the selected data across all benchmarks compared to the original target-specific selection (“MoNA”).\n\n**Table 2: Lexical Diversity (MTLD, larger is better) of Selected data**\n| Method                | MMLU  | GSM8K | BBH   | MBPP  | GPQA  |\n| --------------------- | ----- | ----- | ----- | ----- | ----- |\n| RANDOM                | 46.72 | 46.72 | 46.72 | 46.72 | 46.72 |\n| MoNA (ours)           | 54.59 | 28.74 | 40.92 | 27.29 | 32.94 |\n| MoNA+diversity (ours) | 56.83 | 47.58 | 49.13 | 38.61 | 49.13 |\n\n*Impact of Diversity-Enhancing Strategy on Model Performance for the Target Task* Table 3 shows that introducing the diversity-enhancing strategy leads to a decrease in model performance on the target tasks, confirming that increased diversity can come at the cost of reduced accuracy.\n\n**Table 3: Impact of Diversity-Enhancing Strategy on Model Performance**\n| Method                | MMLU  | GSM8K | BBH   | MBPP  | GPQA  | Avg.  |\n| --------------------- | ----- | ----- | ----- | ----- | ----- | ----- |\n| RANDOM                | 64.02 | 58.65 | 63.70 | 46.73 | 30.36 | 52.69 |\n| MONA (ours)           | 64.49 | 67.93 | 66.44 | 48.40 | 31.47 | 55.75 |\n| MoNA+diversity (ours) | 64.78 | 64.71 | 62.40 | 45.20 | 28.35 | 53.09 |\n\nIn the next version, we will provide a clearer explanation of the scope of our method within target-specific data selection and offer a more comprehensive discussion of its limitations regarding data diversity.\n\n[1] M. Xia, S. Malladi, S. Gururangan, S. Arora, and D. Chen, “Less: selecting influential data for targeted instruction tuning,” in Proceedings of the 41st International Conference on Machine Learning, ser. ICML’24. JMLR.org, 2025.\n\n[2] McCarthy, Philip M., and Scott Jarvis. \"MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment.\" Behavior research methods 42.2 (2010): 381-392.\n\n[3] https://github.com/LSYS/LexicalRichness"}]

Author rebuttal:

Meta-review: The paper introduces a data selection method, MoNA, for efficient instruction tuning. It represents data samples using internal model activations, which are disentangled into sparse, monosemantic representations via a sparse autoencoder (SAE). This method is shown to consistently outperform existing data selection baselines.

Overall, the paper is viewed as a solid, well-supported contribution to the field of data selection for instruction tuning.
Decision: Accept (poster)