Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules
Authors: Yueqi Zhang
Peiwen Yuan
Yiwei Li
Shaoxiong Feng
Xinglin Wang
Jiayi Shi
Chuyi Tan
Boyuan Pan
Yao Hu
Kan Li
Venue: NeurIPS 2025 poster ()
Keywords: span-conditioned generation
quotation referencing
position-aware attention
adapters
Primary area: deep_learning
TL;DR: We formalise quoting in conversation, release a training set and benchmark, and introduce a tiny adapter that lets LLMs exploit quoted spans with zero prompt overhead.

Abstract: Human–AI conversation frequently relies on quoting earlier text—“check it with the formula I just highlighted”—yet today’s large language models (LLMs) lack an explicit mechanism for locating and exploiting such spans. We formalize the challenge as span-conditioned generation, decomposing each turn into the dialogue history, a set of token-offset quotation spans, and an intent utterance. Building on this abstraction, we introduce a quotation-centric data pipeline that automatically synthesizes task-specific dialogues, verifies answer correctness through multi-stage consistency checks, and yields both a heterogeneous training corpus and the first benchmark covering five representative scenarios. To meet the benchmark’s zero-overhead and parameter-efficiency requirements, we propose QuAda, a lightweight training-based method that attaches two bottleneck projections to every attention head, dynamically amplifying or suppressing attention to quoted spans at inference time while leaving the prompt unchanged and updating < 2.8% of backbone weights. Experiments across models show that QuAda is suitable for all scenarios and generalizes to unseen topics, offering an effective, plug-and-play solution for quotation-aware dialogue.

Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules
Yueqi Zhang'; Peiwen Yuan', Yiwei Li', Shaoxiong Feng", Xinglin Wang', Jiyy Shi Chuyi Tan Boyuan Pan Yao Hu², Kan Li School of Computer Science and Technology, Beijing Institute of Technology "Xiaohongshu Inc
(2hangyy,.eiwwenuaar peiwenyuan, liyiwei, ieriinnninsshiiaaa tanchuyi likaan edu. (ehaoxionnfeeng2033 com (panboyuan, xiahho)exxaahoooec com
Abstract
Human AII conversation frequently relies on quoting earlier text-"cheeck with the formula just highlighted".. yet today's large language models (LLMs) lack an explicit mechanism for locating and exploiting such spans. We formalize the challenge as span-conditioned generation, decomposing each turn into the dialogue history, a set of token-offset quotation spans, and an intent utterance. Building on this abstraction, we introduce guotation-centric data pipeline that automatically synthesizes spse-fpe dialogues, verifies answer correctness through multi-stage consistency checks, and yields both heterogeneous training corpus and the first benchmark covering five representative scenarios. To meet the benchmark's zero- overhead and parameter-effiiennc requirements, we propose QuAda, a lightweight training-beit -bsen method that attaches two bottleneck projections to every attention head, dynamically amplifying or suppressing attention to quoted spans at inference time while leaving the prompt unchanged and updating < 2.8% backbone weights. Experiments across models show that QuAda is suitable for all scenarios and generalizes to unseen topics, offering an effective, plug-and-play solution for quotation-awan dialogue
Introduction
Large language models (LLMs)
have become powerful generalists (Marjanović et al. 2025 Chowdh
ery et al. 2023 Kamalloo et al.
2023), yet their behavior quotation-rich conversation remains In everyday conversation, users routinely refer back to earlier turns:
rudimentary Lin and Lee 2024
"That result looks wrong check it with the formula just highlighted."
Such requests simultaneously specify where the model should attend in the history and how the selected text should be used. Today's chat systems offer no principled mechanism for this: users must copy-paste snippets, insert disruptive markers, or hope the model guesses their intent.
We formalize the problem as span-conditioned generation ($2.1 Each turn is represented by the conversation history H, of quotation spans R, and intent utterance U. From this abstraction, we derive one BASE scenario and four orthogonal MaAIssinn-MMMMT--PPPPPPA EXCLUDE, INFO- COMBINE, and COREF that together cover real-world quoting behavior. Specific examples
shown in Fig.
"Equal contribution. "Corresponding author.
OOr code is available at ait.terr//ghaabbccmcaararaeeelld
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Conversation History Cor A: $10K1KK
Conversation History
Conversation History
Conversation History
Conversation History
Car B: $8K
Car A: $10K
Car $8K
Car A=-JJOK Cor B: $8K
Car A: $10K Car B: $8K
It's a dog.
It's cat
Cor $5K
$3K
Cor $5K
$3K
Car C:
Cor $3K
Car c: $5K
Car $3K
It's car
It's bird
How much is this?
Which Cheaper?
Sort the remaining cars by price.
Is this car the most expensive?
What does this pronoun refer to?
Car $8K
Car Cheaper
Can B Car C Carb
No, Cor more expersive.
This pronoun refers
.00
JD
DD
DD
DD
Base
Multi-Span
Exclude
Info-Combine
Coref
Figure 1: Illustrative examples of the five quotation scenarios.
Existing span-conditioned generation methods include CONCAT-REPEAT REPEAT Yeung 2025 MARKER-
(PASTA) ZZhang et al., 2023), all of which employ hand-crafted mechanisms (content repeat, boundary markers, or static attention re- weiggttting) without training. To evaluate their effectiveness, we construct a benchmark composed of five subsets using a carefully designed quotation-centric data pipeline. Given a topic and task specification, this pipeline synthesizes quadruples (H, R, U, ANSWER) and verifies their internal consistency automatically, and ensures each instance is validated by human annotators. On this test bed, all three baselines succeed on only subset of the tasks (The untrained part of Tables 3)d highlighting the limitations of non-trainable methods in capturing in-span information and underscoring the need for a training-baseen position-awasd solution.
INSERTION [Lin and Lee 2024], and ATTENTION-STEENE
To fill this gap, we propose QUADA- a training- based method that embeds token-level span informa- tion directly into attention mechanism through inserted lightweight adapters. For every attention head, two bottleneck projections dynamically amplify or suppress scores adaptively on tokens that fall inside the quoted spans and inject stronger retrieval signal for them. The original prompt remains untouched, and the inserted adapter adds fewer than 2.8% additional parameters the backbone. Trained on a heterogencous corpus generated by our pipeline, QUADA delivers satee-of-thh-ar per- formance on all five quotation scenarios, generalizes to unseen topics, and attaches seamlessly to multiple model families.
Our contributions are summarized follows:
1. We present the first position-aware aware formulation of quoting in conversation and conclude five diagnostic tasks that reflect nearly all practical use cases in the real world.
2. We propose an automated pipeline, equipped with built-in correctness checks, to generate high-quality, human-verified benchmark and large-scale, fully-automated training corpora.
3. We propose training- -bsed, plug-and-plaa method that empowers LLMs follow quoting intents without prompt inflation, delivering consistent gains across all scenarios.
2 Preliminary: The Formulation of Quotation-Guide Generation
2.1 Problem Setup
Let the tokenized conversation history be = (t1, ......... ............ tn), where each represents a token. At turn T, the user supplies set of reference spans:
R {(1,, ei) < 51 e S n, [EN]
identified by token offsets in H, and natural-languar intent utterance U that specifies the user's intent over those spans (e.g., summarize, compare, answer the question based on the spans). and el in Eq. represent the start and the end tokens' index of the 4a quotation span.
Given the triplet (H, R. U) the model generates an answer y = (11,...) by maximizing
Poly H, R, U)
(2)

---

This formulation characterizes how the model generates responses in scenarios where specific spans from the conversation history are referenced and an intent provided.
Diagnostic sub-tasks To fully address the problem, we decompose quoting into a base scenario and four orthogonal extensions that together span practical quoting behaviors:
Base The intent is associated with exactly one quoted span, representing the simplest scenario.
Multi-Span The intent jointy involves multiple disjoint spans,
Exclude The intent explicitly requests that a quoted span be ignored when answering.
Info-Combine The intent asks the model combine the quoted text with its related context, preventing over-focusing.
Coref Resolution tasks where the quote might be a pronoun, similar word, or sentence whose meaning depends on its exact position.
The five diagnostic tasks cover four orthogonal axes of complexity that build on the Base primitive: (i) Multi-Span increases span cardinality; (ii) Exclude introduces negative selection; (iii) Info- Combine requires context-aware synthesis beyond the quoted text; and (iv) Coref demands token-level positional resolution. Taken together, they form minimal yet complete basis that mirrors the ways users quote information in real conversation.
2.2 Existing Strategies for Span Injection
We analyze three existing strategies for injecting the triplet (H,R,U) into an L.M.
(i) CONCAT- REPEAT For every (st, et) R. verbatim content Haysy is prepended to U, yielding
CONCAT-REPEAT
U" Saap U}. Duplication conveys relevance but discards positional origin and inflates the prompt
by (er-s + 1) tokens.
(ii) IINEERINNSTTION Boundary tokens such as (EMPHASIZE) and (/EMPHASIZE) are inserted around each span. The modified history !!' is concatenated with U and re-encoded. Position is explicit at the cost of roughly doubling tokens per conversation.
Pointer-Aware Sparse Tailoring keeps the textual prompt unchanged. Instead, it multiplics attention scores on a fixed subset heads whenever a token's 's index falls inside any (st, et). This produces implicit positional bias with zero prompt overhead, but single global scaling factor can limit the model's generalization capability in diverse scenarios and tasks.
(iii) EYTENTION-STEERIA
We summarize the trade-offs of these methods in T An extended discussion of related works is provided in Appendix A
Table I: Qualitative comparison of span-injection baselines.
Method
Prompt overhead
Injection mechanism
Concat-Repeat
high
content duplication (no position)
Marker- Marker-Insertion
high none
explicit span markers implicit head scaling
Attention-Steering
3 Method
Our approach has two mutually-reinforcin components. First, quotation-centri data pipeline generates large-scale training corpora and rigorously validates our benchmark that exercises the five scenarios introduced earlier. Second, training- position-aware module injects span information into a frozen LLM with minimal additional parameters.
3.1 Dataset Construction Pipeline
The pipeline turns high-level specification into quadruples (H, R, U, ANSWER) through five stages.

---

Step 1: Attribute synthesis To ensure that the data we generate is as generalizable and realistic as possible, following (Yu al., 2023 Long et al., 2024], we prompt LLM to sample structured attribute set that fully determines the forthcoming conversation and task. Table lists the attributes and their corresponding value ranges. Detailed descriptions of specific attributes are provided in Appendix [ Each scenario is paired with its own task list, which is also generated by the LLM and stored with the attributes.
Table 2:
Attributes for data generation.
Category
Values / range
Topic
102 topics (67 train 35 test) (neutral, informal, persuasive,...
Tone
History length
1-10 turns
Information points
2-10 per conversation
Task
(summarize, compare, rank,...]
Span length
(word, sentence, paragraph)
Step 2: Conversation and Quotation span generation Given the sampled attributes, the generator LLM (including ol, ol-mini [Jaech et al. 2024), 03-mini [OpenAl 2025), GPT-4o [Hurst et al.) 2024.. Qwen-plus (Yang al. (0244) writes (i) a multi-turn conversation H/ and (ii) an exhaustive list of token-offset spans covering every quotable information point, yielding the reference set R. For the Coref scenario, we directly utilize the gold pronoun aattocn-nnteedentt spans from the CONLL-2012 corpus [Pradhan et al. 2012], embedding them verbatim into H to keep offsets valid.
Step 3: Task-driven question and answer generation Conditioned on H, R. and the chosen TASK + SCENARIO, the LLM must (i) craft single question U applicable to all relevant span subsets, (ii) select span subsets that satisfy the scenario definition (1 span for BASE; 22 for MULTI- SPAN, EXCLUDE, and INFO-COMBINE), and (iii) produce correct answer for each subset. For Info-Combine, spans not selected serve as backgrounds.
Step 4: Automatic validity checks We cast U into a multiple-choice format whose options are the answers from Step 3, then run three tests: (i) Span sufficiency-wittt only the chosen spans (plus context for Info-Combine scenario), the generator L.M must pick the correct option; (ii) Span necessity weithsttywwithout without those spans, the LLM must fail or return multiple candidates; (ii) Context
requirement Info-Combine only)-a foil answer that ignores the related context must mislead the LLM when no background information is provided. A sample is accepted only if all tests pass.
Step 5: Human verification For the benchmark split, we add a manual audit layer. An LLM first rates every item against its attributes, flagging any item scoring below a threshold. Human annotators then inspect flagged cases, correcting or discarding them needed. This extra pass ensures the benchmark's reliability while keeping annotation cost manageable.
Table 3: Number of samples per scenario.
Training set
Benchmark
Scenario
MCQ
Open-Ended
Total
MCQ
Open-Ended
Total
Base
2200
2200
4400
500
500
1000
Multi-Span
2400
2400
4800
500
500
1000
Exclude
2400
2400
4800
500
500
1000
Info-Combine
2300
2300
4600
500
500
1000
Coref
2200
2200
4400
500
500
1000
The pipeline yields heterogeneous training set and human valaan-vllddated benchmark, each containing
multiple-choice and open-ended variants for the five scenarios. Taale the sample counts for each variant and scenario. By training on topics disjoint from the benchmark, we guarantee that models learn the skill of quoting rather than memorizing the content. To safeguard label correctness, the automatic validity checks described above remove 32.9%. 37.3%, 70.4%, 74.4%, and 15.2% of the initial candidates for the BASE, MULTI-SPAN, EXCLUDE, MIFO-COBIINE, and COREF tasks,

---

respectively an elimination rate we found essential for the reliability of the final corpus. Examples of each scenario are listed in Appendix D
3.2 QUADA: Position-Aware Attention Modulation
Given our benchmark's fine-graines tasks, an effective model must satisfy three requirements: (i) introduce no prompt overhead, (ii) be parameter-effiieet. and (iii) steer attention toward (or away from) quoted spans on demand. We satisfy these constraints with Quotation Adapter (QUADA). drop-in module that adds two lightweight bottleneck projections every attention head (Fig. 2
W
f
by
+
4i
Steered Atm Score
Wa
V
+
bj
*
E
Context
Quotation(s)
User Input & Generation
Figure 2: Overview of QUADA.
Let ge, ky, Vi E Rd denote the query, key, and value vectors a single attention head at positions and (j S i), and d represents the dimensionality of the attention head. At inference time, compared with the non-trained methods, the model additionally receives the token-level span of the quotations and the user inputs:
R R= (s), ex) S 81 S e S n, LEN and Q = (sg:...),
where R is the set of quoted spans and Q marks the span of the user's latest utterance, 8n represents the start token's index of the question part. For notational convenience, we further define indicator:
f j E (st, eq) for any (st, e R.
1IER = {
otherwise.
Query-side modulation To enable the model to adjust the attention given the quotation span adaptively, each head employs a bottleneck MLP Ey: Rd that yields a bias vector by = (%) for query vector. We modify only those queries originating from the current question and the generated tokens, i.e. i sy; otherwise by = 0. The resulting attention score is:
(a: + 1jER 8?) k, = q.T k2 + İ (b?) hig
(5)
The extra term (b?)TK, k2 selectively steers the attention score of quoted tokens, allowing the model to follow the user intent implicitly.
Value-side enrichment To strengthen the representation of quoted tokens, every head learns second bottleneck MLP E, Idd IRd that produces strengthen vector by Ev(u,) ) for the token belongs to the quotation spans. The value vector then replaced by:
0 e LjER by.
(6)
The ūg here can be computed directly from the cached Vy, no additional pass over the conversation history is required.
5

---

We only update these bottleneck MLP layers during training, which ensures that the inherent capa- bilities of the LLM remain unaffected. Despite flexibility, the parameter of QUADA overhead is minimal. Let < d be the bottleneck width, each head then introduces 2dr + 2rd extra weights, keeping the total increase below 2.8% for the 3B and 1.6% for the 8B backbones used in our experiments.
Experiment
Table 4: Results of QUADA on Qwen2.5-3B-Inse all scenarios. Values in bold represent the best results, while values that are underlined represent the second-bes. results.
Base
Multi-Spe Span
Exclude Acc
Coref Acc
Methods
Acc
Const
Acc
Const
Const
Acc
Const
Const
VANILLA
28.4%
2.5
24.4%
2.7
26.8%
2.9
36.2%
2.8
29.4%
2.2
PASTA
32.3%
1.7
29.6%
2.4
24.0%
2.5
38.2%
1.9
27.1%
2.4
Untrained
MARKER
34.0%
2.7
25.2%
2.8
26.4%
3.0
26.0%
2.9
25.6%
2.3
CONCAT
83.2%
4.3
67.8%
4.2
15.2%
2.3
38.2%
3.0
30.4%
2.3
PASTA
34.0%
2.0
29.7%
2.6
22.4%
2.9
39.0%
3.1
32.7%
2.5
Training Based
MARKER
44.8%
4.1
26.8%
3.3
7.8%
2.6
23.0%
2.7
53.4%
3.2
CONCAT
90.0%
4.5
89.8%
4.3
85.8%
3.9
77.2%
3.3
39.8%
2.3
QUADA
95.2%
4.4
94.6%
4.3
92.8%
4.2
85.8%
3.5
90.2%
4.1
Table 5: Results of QUADA on Llama3. 1-8t:e.-----IIstttte on all scenarios.
Base
Multi-Span
Exclude Acc
Info-Combine
Coref
Methods
Acc
Const
Ace
Const
Const
Acc
Const
Ace
Const
VANILLA
21.2%
2.3
20.6%
2.7
22.4%
2.8
29.4%
2.7
21.6%
2.3
PASTA
29.0%
15.8%
2.3
21.6%
2.3
22.6%
3.0
22.4%
2.4
Untrained
MARKER
30.0%
2.4
24.2%
2.8
21.0%
2.8
31.0%
2.7
29.6%
2.4
CONCAT
78.2%
4.2
68.6%
4.2
25.0%
2.3
34.4%
3.2
31.8%
2.8
PASTA
27.1%
2.4
25.5%
2.9
25.0%
3.2
31.2%
3.1
27.5%
2.5
Training Based
MARKER
82.8%
4.4
83.6%
4.3
63.4%
3.9
56.0%
3.5
81.6%
4.0
CONCAT
95.8%
4.4
93.8%
4.3
88.0%
4.2
79.4%
3.8
40.0%
3.2
QUADA
96.0%
4.5
98.2%
4.4
93.2%
4.4
82.6%
3.8
94.8 %
3.8
4.1 Experimental Setup
Backbone models We adopt two instruction-tume LLMs with different scales and architectures: (Grattatiori et al.) 2024]. For QUADA, we set the query and value side bottleneck width to r 256, which introduces 75 M trainable parameters on Qwen (2.8% of the model) and 130 M on Llama (1.6%). All backbone weights are frozen.
-3B-S-SRRUU (Yang et al.) [024] and LLAMA-3.1 --AMMA-3.-8B--ISTTRRU
Baselines and their trainable variants We benchmark QUADA against the three span-injection strategies from 2.2 Besides the original untrained versions, we devise trainable counterparts that every method benefits equally from the synthetic corpus.
CONCAT-REPEAT We attach auxiliary bottleneck to each attention head. If a token belongs to a duplicated span, re-encoded representation is added to the main projection, mimicking explicit repetition while allowing gradients to flow.

---

Generation Prev Con-Comerssttoon
Generation Prex www.-Coovertttoon ion
Generation Prex Prev-Convent Cee--Converstfie
Base
Multi-Span
Exclude
1.0
Question &
Question &
Question &
0.8
Quotation
Quatation
Qustation
Quctation
Question & Prey- Coy-Covvrrsttoonn
Question & Prev Conversation
0.4
Info-Combine
Coref
Generation
Generation
0.0
Quotation
Quotation
Figure Averaged attention maps adjusted by QUADA. Attention scores are averaged and normalized over all attention heads. Red rectangles highlight the quotation span(s), while darker colors indicate stronger attention.
The tokenizer is extended with two special symbols <emphasise> and </emphasise>, During fine-tuning, we update only their embeddings; other parameters stay frozen, so the prompt overhead remains the same as in the untrained variant.
(PASTA) In the zero-shot setting, originally selected on the JSON FORMATTING data [Zhang et al. [023]. For the trainable version, rerank heads on our own training split and use the top 25 at inference time, following the protocol in the original method.
ArTENTION-STEERNE
the 25 attention heads
Evaluation metrics Each subset is paired with metric that matches its output structure: We employ two metrics to assess these methods. Accuracy (Ace) for the single-choice questions. A prediction is correct if the chosen option exactly matches the gold. Consistency score (Const) is used for open-ended generations. Following recent best practice [Gu et al.] [022), we prompt a GPT-4o-mini model [Hurst al.) (2024) to rate the consistency between the model's output and the gold answer on scale from I to 5. Higher scores indicate stronger alignment. The full judging
prompt provided in Appendix C
Additional experimental details (including the GPU model, memory specifications, and criteria for human evaluators) are provided in Appendix F
4.2 Main Results
QUADA effectively handles diverse quoting behaviors Our method consistently achieves the best performance across all five scenarios on both backbones (Tables and 55. In Base, Multi-Span, and Exclude scenarios, QUADA accurately adapts its response behavior without requiring explicit rule-based control. In the Info-Combine scenario, where the model must integrate quoted spans with the related unquoted backgrounds, QUADA avoids over-focusing on the quote itself, demonstrating its capacity
the quotations in context-aware manner. Finally, in the Coref task, which requires sensitivity to the exact position of the quoted text, QUADA achieves 90.2% accuracy on Qwen and 94.8% Llama, which are far ahead of all baselines, highlighting its ability understand the position of the quotation span.
Training essential for the quotation task Across all scenarios and both models, trained vari- ants of all methods consistently outperform their untrained counterparts, demonstrating that span- conditioned behavior cannot captured through static or untrained injection mechanisms alone. The superior performance of QUADA when trained on synthetic corpus provides further validation for
7

---

the effectiveness of our training methodology and span-based attention modulation strategy, as well as confirms the quality and comprehensiveness of the training set.
GPT-4o-mini judge is unbiased in our experiment To validate the accuracy of the 4o-mini-as- judge in experiments, we randomly selected 25 data samples from each scenario and asked human evaluators to use the same rating criteria as the 4o-mini for evaluation. The results indicated a Pearson correlation coefficient of 0.75 between human and 4o-mini ratings, demonstrating that our 4o-mini evaluations for the Open-Ended task are generally unbiased relative to human judgment.
QUADA preserves the model's generative fluency We additionally evaluated fluency with GPT- 4o-mini. Both CONCAT- REPEAA and QUADA receive fluency ratings that are exceptionally clear, logically structured, and straightforward read, indicating that our method does not degrade surface- level language quality.
QUADA adapts to complex scenarios Building on the five basic scenarios above, further evaluate three practical extensions: cross-lingual use, a multi-tumn variant where successive questions each quote different span, and mixed cases that combine MULTI-SPAN with EXCLUDE/INFO- COMBINE. In all cases, QUADA requires no task-specific rules or architectural changes and preserves its span-conditioned behavior. See Appendix for details.
4.3
Analyses
QUADA adaptively re-allocates attention Figure 3 plots the average head-wise attention for each diagnostic scenario. Base and Multi-Span, the heatmap shows dark vertical bands exactly what the
at the quoted offsets, while the rest of the context is uniformly suppressed-precciees tasks require. For Exclude, the pattern flips: the quoted segment turns pale, confirming that our method can down-weight forbidden spans. In Info-Combine span still peaks, yet moderate attention is also maintained on other contexts, enabling the fusion of related information. Finally, in Coref, attention locks onto the quoted pronoun and its immediate neighborhood, providing the positional anchor needed for antecedent resolution. Together, these heatmaps show that QUADA dynamically adjusts attention according to the user's instruction. All visualized samples and their span annotations are provided in the Appendix D.2
hybrid pattern emerges: the quoted
Validity of the Gene-Geee Benchmark To assess how well our synthetic benchmark reflects real-world citation behavior, we manually curated reference benchmark by filtering and adjusting subset of samples for each scenario and question type. We then evaluated 10 diverse models (from Qwen, Llama, Phi families, etc.) on both the auto-generated and caman-curated benchmarks, and computed the Pearson correlation between their scores.
As shown in Table all correlation coefficients exceed 0.82, with several above 0.95, indicating strong agreement between the automatically constructed and manually validated evaluations. These results confirm that our synthetic benchmark faithfully reproduces the relative difficulty and quotation behaviors seen in numan-curated data, validating its suitability as an evaluation benchmark for real-world quotation scenarios.
Table 6: Pearson correlation between model scores on the LLM and human- based benchmarks.
Base
Multi-Span
Exclude
Info-Combine Coref
Avg.
MCQ
0.83
0.95
0.97
0.93
0.87
0.91
Open-Ended
0.85
0.87
0.82
0.96
0.89
0.88
QUADA generalizes across model scales Figure reports the Accuracy and Consistency for Qwen2.5 series models from 1.5B to 14B parameters on all benchmarks. Even the 1.5B model surpasses 90% accuracy on three scenarios and averages 4.0 consistency score. Performance rises steadily with scale, where the 14B variant achieves 98 98-99% SINGLE and MULTI with a 4.5 consistency score. The monotonic trend indicates that QUADA delivers strong gains on small models and continues to improve as the model's parameter count grows.

---

100.0
97.5
4.4
4.0
90.0
3.8
87.5
3.6
Base
Base
85.0
Spon-Spon
NpttiSpan
Leclude
Exclude
82.5
Info Combine
3.2
Info-Combine
Coret
Qwen1.58
Qwen3B
Owen78
Qwen1448
Qwen1.53
Owen38
Qwen7B
Qwen14B
(a) Effect model scale on multiple-choice accuracy.
(b) Effect of model scale on open-ended const scores.
Figure 4: Impact of backbone model size QUADA performance across the Qwen 2.5 series models.
4.4 Ablation Study
Query and Value Modulation are Complementary in QUADA We compare three variants of QUADA on Owen2.553B-Innse Instructt while keeping the total trainable parameters fixed in Table The results show that Value-only modulation already recovers most of the full method's 's strength (c.g.. 92.8% on MULTI and 87.6% on EXCLUDE), confirming that richer value vectors give quoted tokens strong retrieval signal. Query-only modulation is much weaker, suggesting that steering attention scores isolation cannot compensate for under-informative token representations.
The full method outperforms both ablations on every task, with the most significant margin INFO-COMBINE (+ (+10.8% over value-only). In this setting, query branch first directs attention toward relevant regions; the value branch then supplies the content needed to merge quoted spans with surrounding context. We probe the representations by computing the cosine similarity between the bias 6" and its base vector for every layer. The similarity averages 0.62 across layers, with 30 % of layers above 0.80, and ||6'|| se 0.6 ||w/|. These statistics indicate that value modulation consistently reinforces quoted-span representations.
Table 7: Performance impact of query and value modulation components in QUADA.
Variants
Base
Multi-Span
Exclude
Info-Combine
Coref
QuAda (query-only)
78.8% 4.3
58.4% / 3.7
49.6% 3.8
44.6% 3.0
69.4% 3.7
QuAda (value-only)
94.8% / /.44
92.8% / /.3
87.6% / /4.1
75.0% /34
91.6% 4.1
QuAda (full)
95.2% 4.4
94.6% 4.3
92.8% 4.2
85.8% 3.5
90.2% 4.1
QUADA is Parameter Efficient Here examine how the size of the QUADA, controlled by the bottleneck dimension т. affects the performance of QUADA. Table X reports the results of varying the bottleneck width for the query and value projections in the Q-wnn..-3B-Instrucc model. Notably, even at the smallest setting (r=64), QUADA already achieves over 94% accuracy on three of five tasks and maintains generation consistency span-conen apan-conditioning behavior can be captured with a highly compact QUADA module.
around 4.0. This indicates that most
To balance accuracy and model size, adopt r-256 the default configuration: it consistently matches or sometimes outperforms all other settings across all tasks while remaining lightweight (< 2.8% of the backbone's 's parameters). These results demonstrate that QUADA can deliver substantial performance gains without a large parameter budget, making it especially suitable for resource- constrained deployments.
Table 8: Impact of bottleneck width QUADA performance.
QuAda (r)
Base
Multi-Span
Exclude
Info-Combine
Coref
64
96.0%
/..4
94.2% / 4.4
95.0% / /.2
84.4% / 3.4
87.8% /4.1
r 128
94.8% / 4.5
94.2% / /.4
93.6% / 4.2
80.4% / 335
90.0% /4.
256
95.2% / 4.4
94.6%
/ /4.
92.8% 4.2
85.8% 3.5
90.2% / /.1
r = 512
95.4% / 4.4
94.8% 4..
93.8%
4.2
84.8% / 3.5
90.6% /4.

---

5 Conclusion & Future Work
We introduced span-conditioned generation as a principled formulation for quotation-rich conversa- tion and released a diagnostic benchmark covering five orthogonal quoting behaviors. To facilitate research in this new setting, we developed an automatic data-generation pipeline capable of synthesiz- ing large-scale, high-quality training corpora and human- numan-validated benchmarks. Building on this data, we proposed QUA a lightweight adapter that injects positional signals via query and value side adapters without inflating prompts substantially enlarging the backbone. Extensive experiments across two instruction tuned LLMs and multiple model sizes (from 1.5B to 14B) demonstrated that QUADA consistently outperforms both training-free ffree and training- based basclines, maintains fluency, and scales gracefully. Taken together, our results suggest that this work can substantially advance human- L.MM interaction, without introducing adverse side effects. Potential backdoor and misinformation risks, along with our mitigation measures, are discussed in Appendix G
The present work is confined to monomodal, monolingual quotation. Our adapter is trained and evaluated solely on English text spans; we have not experimented with other modalities (images, tables, code) or multilingual settings- such as quoting French sentence within English dialogue. Extending QUADA to multimodal and cross- mooss-lnggual quotation will require new benchmarks and may reveal additional challenges for position-awe aware attention, which we plan to explore in future work.
Acknowledgments
This work is supported by Beijing Natural Science Foundation (No.4222037, L181010).
References
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-133, 2023.
Tian Dong, Minhui Xue, Guoxing Chen, Rayne Holland, Yan Meng, Shaofeng Li, Zhen Liu, and Haojin Zhu. The philosopher's stone: Trojaning plugins of large language models. arXiv preprint arXiv:2312.00374, 2023.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama herd of models. arXiv preprint arXiv:2407.22 21783, 2024.
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judee arXiv preprint arXiv:2411 (..::24111.55555 15554, 2024.
Evan Hernandez, Belinda Z Li, and Jacob Andreas. Inspecting and editing knowledge representations in language models, arviv preprint arXiv:2304.00744. 2023.
Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive Ilms that persist through safety training. arXiv preprint arXiv:2401.0555 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Aaron Jacch, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.16722 2024.
Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge confficts in language models. arXiv preprint arXiv:2402. 18154, 2024.
10

---

Ehsan Kamalloo, Nouha Dziri, Charles LA Clarke, and Davood Rafiei. Evaluating open-domain question answering the era of large language models. arXiv preprint arXiv:2305.06984, 2023.
Kai Konen, Sophie Jentzsch, Diaoulé Diallo, Peer Schütt, Oliver Bensch, Roxanne El Baff, Dominik Opitz, and Tobias Hecking, Style vectors for steering generative large language model. arXiv preprint arXiv:2402.01618 2024.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Plister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from language model. Advances in Neural Information Processing Systems, 36:41451-4155 2023.
Guan-Ting Lin and Hung-yi Lee. Can Ilms understand the implication of emphasized sentences in dialogue? arXiv preprint ar Xiv:24066.1 1065, 2024.
Hongyi Liu, Shaochen Zhong, Xintong Sun, Minghao Tian, Mohsen Hariri, Zirui Liu, Ruixiang Tang, Zhimeng Jiang, Jiayi Yuan, Yu-Neng Chuang, al. Loratk: Lora once, backdoor everywhere in the share-and-play ecosystem. arXiv preprint arXiv:2403.00100 2024.
Sheng Liu, Haotian Yc, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv preprint arXiv:2311 Siii::2311.00666. 1.06668, 2023.
Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. On llms-driven synthetic data generation, curation, and evaluation: A survey, 2024. URL https://arxiv. org/abs/2406. 15126, 2024.
Sara Vera Marjanović, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lü, et al. Deepseek-rl thoughtology: Let's <think> about IIm reasoning. arXiv preprint arivi:2002222222007222 2025.
OpenAl. OpenAI 03-mini, January 2025. UK//attpss//ogpaa....c///////tnp OpenAl Blog, January 31, 2025.
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. Conll- 2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Joint conference on EMNLP and CoNLL-sharre task, pages 1 1-40, 2012.
Nishant Subramani, Nivedita Suresh, and Matthew E Peters. Extracting latent steering vectors from pretrained language models. arXiv preprint arXiv:2205.0512: 1224 2022.
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization. arXiv e-prints. pages arXiv-2308. 2023a.
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308. 10248, 2023b.
Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. Retrieval head mechanistically explains long-context factuality. arXiv preprint MXXvv2204.115744 2024.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. technical report. arXiv preprint arXiv:2412.15115, 2024.
Matty Yeung.
Deterministic quoting.
https://mattyyyeng srapeeuuungggtthhuggeeee github..h....... io/
deterministic-quon quoting. 2025. Accessed: 2025-05-09.
Yue Yu. Yuchen Zhuang, Jieyu Zhang. Yu Meng, Alexander J Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. Large language model as attributed training data generator: A tale of diversity and bias. Advances in Neural Information Processing Systems, 36:55734-55784, 2023.
Qingru Zhang, Chandan Singh, Liyun Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao. Tell your model where to attend: Post-hoc attention steering for lmss arXiv preprint arXiv:2311.0222 1.02262, 2023.
11

---

Qingru Zhang, Xiaodong Yu, Chandan Singh, Xiaodong Liu, Liyuan Liu, Jianfeng Gao, Tuo Zhao, Dan Roth, and Hao Cheng. Model tells itself where attend: Faithfulness meets automatic attention steering, 2024. URL htt.httts://aaaiiee
Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong. and Zhiyu Li. Attention heads of large language models: survey, arXiv preprint arXiv:2409.03752, 2024.
Youxiang Zhu, Ruochen Li, Danqing Wang, Daniel Hachn, and Xiaohui Liang. Focus directions make your language models pay more attention to relevant contexts. arXiv preprint arXiv:2503.23300 2025.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We substantiate our claims with both experimental evidence and theoretical
analysis.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our work in the conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided detailed experimental settings in the experiments section and detailed module designs of our method.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification:
We will release our code soon in the github.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.cc/pulli//guiii wwwiiccgguie///eeeeeeeeis CoodeSubeiiieeeec ssi wswinnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided detailed experimental settings in the experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: For each setting, we run 3 times and report the average results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
15

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have reported relevant information in the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: h/tppris///errps.....
Answer: [Yes]
Justification:
We have read the NeurIPS Code of Ethics and followed it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed this in the paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
16

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have correctly cited all the assets we use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppsiipteeeee www.rrewwthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
17

---

Answer: [Yes]
Justification; We provide a detailed introduction of the proposed training dataset and benchmark in paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We have provided the relevant information in paper.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
18

---

A Related Works
A.1 Attention-Guided Generation in Decoder-Only LLMs
A growing body of mechanissic-mieeppeetbblll work shows that individual self-attention heads specialize in distinct generative roles et al. 2024]. For instance, "retrieval heads" that gather long-range evidence [Wu et al.] [024] or heads whose ablation removes knowledge conflicts ( ct al. 2024]. Such span-sensitive behavior motivates our adapter.
The most closely related control method is PASTA [Zhang et al., 2023 2024), which boosts fixed of heads whenever their keys fall inside user-marked span. Focus Directions (Zhu et al. 2025 extend this idea by adding learned vector to key/query activations, steering attention toward salient tokens without prompt markup. Both approaches inspire our query-side biasing, yet neither enriches value vectors nor handles multi-span or negative constraints, gaps that QUADA fills.
Latent-space steering
complementary line of research controls decoder-only LLMs by editing internal activations. Liu ett al. 22033 recasts in-context learning examples single in-context vector that is added to hidden states, achieving stronger task transfer with zero prompt overhead. More generally, Subbaaanii et al. 2022 extracted steering vectors from pretrained LMs can nearly perfectly reconstruct target sentences, and enable sentiment transfer via simple vector arithmetic. Extending this idea, Turner et al. (2023b] formalises activation engineering and introduces ACTIVATION ADDITION, lightweight method that shifts activations along a "love - hate" direction to control sentiment without optimi- sation. Konen et al. 20244 demonstrates analogous style vectors that steer output toward specific writing styles. Beyond style, [i et al. 2023) identifies "truthfulness" direction: nudging activations TruthfulQA inference time. Latent editing has also been used to modify factual knowledge: Hernandez et al. (2023) learn encodings new facts and injects them into hidden layers so that generation aligns with the edited knowledge. Finally, [Turner al. 2023a provides a systematic recipe for computing activation directions that steer toxicity, refusal, or politeness.
along it doubles ILLMMA accuracy
These works share our goal of parameter-efficient inference-time control; however, they target global attributes (style, truthfulness, knowledge) rather than span-conditioned quoting. QUADA differs by selectively modulating query and value activations according to user-specified spans, enabling fine-grained, multi-span, and negative constraints that latent steering alone cannot express.
B Attributes for Data Generation
Topics
Algebra
Calculus
Probability and Statistics
Mechanics
Electromagnetism
Optics
Thermodynamics & Statistical Mechanics
Inorganic Chemistry
Organic Chemistry
Biochemistry
Environmental Chemistry
Pharmaceutical Chemistry
Constitutional Law Civil Law Criminal Law
Administrative Law
Business Law / Commercial Law
International
Intellectual Property Law Labor Law
Environmental Lav
19

---

Arbitration and Mediation
Mechamical Engineering
Electrical Engineering
Electronic Engineering
Chemical Engineering
Civil Engineering
Materials Engineering
Aerospace Engineering
Biomedical Engineering
Computer Engineering
Environmental Engineering
Microsconomics
Macroeconomics
Econometrics
International Economics
Development Economics
Financial Economics
Labor Economics
Behavioral Economics
Public Economics
Environmental Resource Economics
Nutrition and Diet
Sports Medicine Fitness
Epidemiology
Public Health
Mental Health
Health Education
Health Management
Basic Clinical Medicine
Geriatrics Rehabilitation
Health Policy
General Paychology
Developmental Paychology
Social Psychology
Clinical Counseling Psychology
Cognitive Psychology
Personality Paychology
Physiological Psychology
Indusrrial/Oggazzaiiiaaa Paychology
Educational Psychology
Health Psychology
Marketing
Finance
Accounting
Management
Operations Management
Human Resource Management
Busineas Strategy
Entrepreneurship Innovation
International Business
E-commerce
Cell Biology
Molecular Biology
Genetics
Physiology
Ecology
Developmental Biology
Microbiology
Zoology
Botany
Neurobiology
Ancient Philosophy
Modern Philosophy
Analytic Philosophy
Continental Philosophy
20

---

Ethics
Xetaphysics
Epistemology
Political Philosophy
Philosophy of Science
Aesthetics
Data Structures Algorithma
Operating Systems
Computer Networks
Software Engineering
Database Systems
Artificial Intelligence
Machine Learning
Computer Graphics
Distributed Systems & Cloud Computing
Cybersecurity
Tone
Formal
Casual
Persuasive
Analytical
Creative
Narrative
Enthusiastic
Tasks
#*#*# Task for Base Scenario #н**#
Specified Passage Summarization: Summarize the single quoted passage into concise statement. The user wants brief overvies main derived exclusively from the quoted text Specified Segment Q&A: Ansver a question based only on the single quoted text. The user's query should be resolved strictly using information found in that quoted segment
Specified Segment Definition Extraction: Identify and restate the definition of term or concept mentioned the quoted text. The user is asking for precise definition contained within that segment
Keyword Key-Phrase Extraction: Extract the most relevant keywords focuses
key phrases from the single quoted segment. This on highlighting crucial points or topics directly mentioned Quoted Segment Revriting: Reurite the quoted segment while retaining its essential meaning. The user ask for more readable or text
differently styled version of the Quoted Passage Simplification: Simplify the quoted passage while retaining its essential meaning. The user may ask for a more concise, more readable, or differently styled version of the same text
Quoted Segment Sentiment Analysis: Analyze the emotions attitudes expressed in the quoted segment. Identify and describe the feelings, such as happiness, sadness, anger, or neutrality,
conveyed in the quoted region
Quoted Segment Data Extraction: Extract requested data details from
the quoted segment. Focus on retrieving specific facts or figures mentioned in the text Quoted Segment True-or-Faled Verification: Present factual statement about the single quoted text, and ask the model to determine whether it is True verified by checking contradicts with the content in the quoted passage.
False based strictly on that text. This
the statement directly aligns or
Quoted Passage Step-by-Step Procedure Extraction: If the single quoted text describes procedure
of ordered steps, request the
21

---

model to list each step stage in the correct sequence. The correctmess be checked by confirming each listed step appears (in the right order) within the quoted region.
Specified Segment Contradiction Detection: Present a statement or claim and ask whether it contradicts the single quoted text, is supported by it, is not mentioned at all. This is easy to
verify: simply compare the statement it's in direct conflict, alignment, or absent.
the quoted
Specified Information Existence Detection: Present statement or claim and ask whether it appears in the quoted text. In this task, the statement refers portion the original context but not in the quoted segment. By comparing the statement to the quoted text, the model should determine if the statement is indeed absent there.
information that exists in the unquoted
Quoted Passage Named Entity Identification: Ask the model to extract and list all named entities (e.g., people, locations,
organizations) exactly as they appear in the single quoted text. The correctness is judged by directly checking the text to see if named entities are missed or incorrectly added.
Quoted Segment Key Fact Listing: Instruct the model to list key factual points explicitly mentioned (e.g., important dates, statistics, proper names, etc.). The user then checks if each item in the model's list appears verbatim (or unambiguously) in the passage, ensuring extra or missing facts.
the the single quoted text
#***# Task for Multi-Span Scenario #2а8#
Information Comparison: Compare or contrast two or more separate quoted passages (e.g., comparing their data, opinions, or attributes). The user's question focuses differences similarities across these quoted segments
Multi-Source Consolidated Summarization: Provide a unified summary that covers all key points from two or more quoted passages. The user wants integrated overview of multiple fragments Temporal (Time-Order: Reasoning: Determine chronological order of events or facts mentioned across multiple quoted segments. The user's question involves identifying which event or statement occurred first or last
explain the
Contrasting Vieupoints Analysis: Identify differing vievpoints or stances across multiple quoted regions. The authors or speakers in those passages disagree or differ perspective
how
Merged Key Point Listing: Extract the core points from each quoted passage and then compile them into single combined list. The user wants concise set bullet points that captures all cited texts
Numerical/Data Comparison Across Sources: Identify and compare numerical values data points mentioned in two more quoted passages. The user's queation focuses on determining the highest, lowest, or most significant value among these sources.
Numerical/Data Sorting: Extract numerical values or data points from two or more quoted passages and arrange them in specified order (e.g., from high to low low to high) The
's question sorted list.
focuses presenting the data Multi-Source Condition Fulfillment: The user poses set of conditions (e.g., "Condition A is stated in quotation 1, Condition B is in quotation 2"). Ask the model to determine if those conditions are satisfied or met collectively, based strictly segments.
the referenced
Multi Paaaaaaaeeeeueeeeeeeess
Reconstruction. If multiple quoted segments each describe different parts phases of a single process or timeline, the sequence flow. Correctness step appears the right chronological or logical order as indicated by the individual passages.
asks arrange these parts in coherent
be verified by checking each
22

---

made #*#*# Task for Exclude Scenario ****#
Summarize After Ignoring Selected Passages: The designates
more passages to "ignored. The task to summarize only the remaining, non-ignored content. The ansver must exclude or skip any information from the ignored portions.
Sensitive Information Hiding: The user selects private confidential daat be hidden. The final output ignored data. The user wants only the non non-sensitive parts to be disclosed.
reveal the
Partial Anonymization
Redaction: The user specifically wants certain fields details (like names, addresses, IDs) to redacted. The task to remove or mask those sensitive elements while preserving the rest of the information.
Summarize Unignored Sections: The user designates one or more passages be ignored. The model must produce a concise summary only of the content not in those ignored passages. The ignored content should not influence or appear in the summary.
Sort or Rank Unignored Data/Kumber: After ignoring specific segmenta,
the user asks the model to sort or rank remaining items based on certain criterion (e.g., numerical values, alphabetical order, etc.). Only the leftover textual details (not ignored) should be used to perform the sorting or ranking.
Extract Keywords from Non-Ignored Content: The user designates some passages to be ignored and then requests keyword extraction. The model
parse only the remaining (unignored) text to identify relevant or significant keywords without referencing any ignored sections.
Named Entity or Concept Extraction (Non-Ignored Only): The user designates specific passages ignore. The model must identify named entities (people, locations, organizations, etc.) or key concepts only from the remaining text. Any details found ignored content should be excluded.
Non Ignored Outline Generation: The user instructs the model to create outline (e.g., bullet points
structured plan) of the leftover information after certain portions are ignored. resulting outline must reflect only the visible (unignored) details.
Compare Remaining data/number After Ignoring: The user designates certain passages be omitted. The question is then compare the data/number in only the remaining information. The solution should not incorporate any data from the ignored content.
#*### Task for Info Combine Scenario #**##
Reference Justification Extraction: Given a reference quotation that contains a recommendation or statement (e.g., "Then you should rest early") and background quotation that provides the underlying context (e.g., "I am very tired from work today"), the model extract and clearly justify the reasoning behind the reference. The answer should explain that the suggestion because of the background's stated condition.
Price Comparison Based on Quoted Reference: When the reference dollar")
quotation specifies particular item (e.g., "Item and background quotations list prices of other items (e.g., "Item 10 dollar" and "Item C: 5 dollar"), the model must compare these figures. The answer should identify which background item is more expensive than the referenced one, strictly based on the provided numerical data.
Temporal Sequence Reasoning with Context: With a reference quotation indicating an event with a specific date (e.g., "Event y occurred on March 5th*) and background quotations listing other events with March 1st" and "Event z
their dates (e.g., "Event X occurred occurred February 28th"), the model must determine the chronological relationship. The anewer should identify which event happened before or after the reference event, solely using the provided dates.
23

---

Attribute Comparison Across Context and Reference: When the reference quotation provides measurable attribute of item (e.g.. Product has rating 4.0 stars") and background quotations offer similar attributes for other items (e.g., "Product X: 4.5 stars" and "Product 2: 4.8 stars"), the model must perform direct comparison. The answer should specify which products have higher (or lower) values than the referenced product.
Contextual Cafse-Effect Linkage: Given reference quotation that states an effect or recommendation (e.g., "You should replace the cooling fan immediately") and background quotation that provides (e.g., "The device overheated due to prolonged usage"), the model must link the effect to its cause. The answer should clearly state that the recommendation is based on the background cause.
Contextual Selection Based Criteria: Provided reference quotation that highlights particular preference selecting an item with a specific price point) along with background quotations listing various items and their attributes, the model must select the items from the background that meet the criterion specified by the reference. The ansser should list only those items that conform to the given requirement.
criterion (e.g.,
Direct Comparison with Context Integration: With a reference quotation that specifies a particular detail (e.g., "Item B costs 7.3 dollar") and background quotations that offer comparable details for other items (e.g., "Item costs 20 dollar" and "Item C costs 2 dollar"), the model must directly compare these details. The answer should indicate which background item stands out relative to the reference.
Integrated Inference from Reference and Context: The model receives reference quotation that implies a decision or inference (e.g., You should leave earlier tomorrov") alongside background quotations that supply supporting context (e.g., "Traffic is usually heavy during rush hour") The ansver should integrate both deduce the rationale behind the reference statement.
Contextual Elaboration of Reference Statement: Given reference quotation that provides a brief statement (e.g., "It would be wise to update the software") and background quotation that offers additional detail (e.g., "The system has been experiencing bugs due to outdated software"), the model reference. The answer should explain that the recommendation is based on the issues mentioned in the background.
elaborate the
Quantitative Comparison with Contextual Data: When reference quotation contains quantitative figure (e.g., "Service В costs dollars") and background quotations list similar figures for other services (e.g., "Service A costs 25 dollars" and "Service costs 15 dollars"), the model must perform quantitative comparison. The ansver should specify which background figure exceeds falls below the reference figure.
Relative Ranking Determination: With a reference quotation providing a
metric for an item (e.g., "Kodel Y has a battery life of 8 hours ") and background quotations offering comparable metrics for other models (e.g., "Model X: 10 hours" and "Model 2: 7 hours"), the model must rank the items. The ansver ghould clearly state which models rank higher or lover than the referenced one based on the provided metrics.
Conditional Outcome Reasoning: Provided reference quotation that states conditional suggestion (e.g., "If you experience overheating, turn off the device immediately") along with background quotations that supply the condition (e.g., "The device 's temperature has been rising steadily"), the model must deduce the correct outcome. The ansser should confirm the conditional recommendation based solely on the provided background.
Multiple Criteria Filtering: When reference quotation emphasizes specific criterion (e.g., "I prefer an affordable product") and background quotations list several items with various attributes

---

e.g., price, quality), the model must filter the background to identify only those items meeting the reference criterion. The answer should list the items that match the affordability requirement as provided in the background.
Cross-Donse Integration: The model is given reference quotation from domain (e.g., "You should upgrade your hardware") alongside background quotations from related domain that offer supporting technical details (e.g., "The current system has a processing apeed of 1.2 GHz"). The answer should integrate both sets of information to justify the recommendation, strictly relying the provided details.
Contradictory Evidence Resolution: Given reference quotation presenting a definitive statement (e.g., "The meeting is scheduled for 3 PM") and background quotations that offer conflicting information (e.g., "The organizer mentioned delay, setting the meeting 4 PM"), the model must resolve the contradiction. The answer should clarify the correct detail by logically prioritizing the background evidence.
Beference-Comee Synthesis for Decision Naking: In this task, the reference quotation poses decision prompt (e.g., "Should choose Option B?") and background quotations offer supporting Options and B) The model must synthesize the information from both to recommend decision. The answer should clearly state which option is preferable based solely the integrated information from the reference and background quotations.
details (e.g., pros and cons or specifications
C Full Prompt for Consistency Evaluation
Prompt for Evaluating Base, Multi-Span, and Exclude Scenarios
You are a fair evaluator. will be given two pieces text: Model Answer Ground Truth. Your task to assess how accurately the Model Anaver reflects the essential facts and conclusions of the Ground Truth. Use the following five-level scale to guide your decision, then provide your final evaluation the format "Score X" with no additional explanation or content.
single line in
Score 1: The ansver is overshelmingly incorrect or contradicte the
Ground Truth in most respects, showing fundamental misunderstandinge
factual errors, so that little no essential information aligns with Ground Truth. Score 2: The anaver contains multiple significant inaccuracies or contradictions, offering only limited amount of correct information, and overall demonstrates unreliable alignment with the Ground Truth despite some partial correctness.
Score 3: The ansver demonstrates moderate alignment with the Ground Truth, correctly capturing certain important points while also containing noticeable errors omissions, accuracy is only partially reliable.
that its overall
Score 4: answer is largely accurate with only minor mistakes or overlooked details, consistently reflecting the Ground Truth's main facts, thereby shoving good degree of correctness and trustworthiness.
Score 5: Answer used same evidence the Ground Truth and reached a conclusion identical to the Ground Truth.
After reviewing the Model Answer and comparing it to the Ground Truth, choose the beat matching level from to 5. Output your final rating as:
Score X
where x is the chosen number, and do not add any other text, marker or explanation in your response.
25

---

Ground Truth: {gt_label)
Model Answer: (predict)
Prompt for Evaluating Info-Combine Scenarios
You are a fair evaluator. You will be given two pieces of text: Model Answer and Ground Truth. Your task is to assess hov accurately the Model Answer reflecta the essential facts and conclusions of the Ground Truth. Use the following five-level scale guide your decision, then provide your final evaluation as single line in the format "Score X" with no additional explanation or content.
Score 1:
The Model Answer reaches different conclusion from the Ground Truth, and referenced information is entirely unrelated
Score 2: The Model Ansver reaches different conclusion from the Ground Truth, and some of the reference information is the same that in the Ground Truth. Score 3:
Model Ansver reaches different conclusion from the Ground Truth, and most of the reference information is the same as that in the Ground Truth.
Score 4: The Model Answer reaches the conclusion the Ground Truth, but it does not utilize the Truth.
information the Ground
Score 5: The Model Answer reaches the same conclusion as the Ground Truth and utilizes the exact same information.
After reviewing the Model Answer and comparing it the Ground Truth, choose the best matching level from to 5, Output your final rating as:
Score
where is the chosen number, and do add any other text, marker explanation your response.
Ground Truth: {gt_label)
Model Answer: (predict)
Prompt for Evaluating Coref Scenarios
You are tasked coreference resolution problem. The evaluation should be based on how well the Anaver aligns with the Ground Truth (correct answer) Use the following scoring rubric Score The Anawer is completely incorrect.
evaluating the quality an Answer provided for
assign score from to 5:
claims that the pronoun does not refer to anything, or the response entirely irrelevant.
Score 2: The answer points to an incorrect referent, which is not the specified in the Ground Truth.
Score 3: The ansuer identifies several potential referents, including the one specified in the Ground Truth, along with additional irrelevant ones.
Score 4: The anaver correctly identifies the referent specified in the Ground Truth, but contains inaccuracies omissions describing what it did or what happened.
Score 5: The ansver correctly identifies the referent as defined in
the Ground Truth and accurately describes what it did or what happened.
26

---

After reviewing the Model Answer and comparing it the Ground Truth, choose the best matching level from to 5. Output your final rating as:
Score X
where x is the chosen number, and do add any other text, marker explanation your response.
Ground Truth:
{gt_label)
Model Answer:
(predict)
D
Examples for the input samples
D.1
Multi-Choice Question Examples
Base
"instruction": "I've been reading about how social contracts influence public welfare policies. Gould you explain where this idea originated?< iasr<//immendddde end att.iiimsstttt//ssssatttttt nCertainly! The idea of the social contract originates from philosophers like Thomas Hobbes, who argued that individuals agree to surrender provided by the state. In 'Leviathan, Hobbes described life without such an agreement as 'solitary, poor, nasty, brutish, and short.' This foundational principle laid the groundwork for understanding the relationship between citizens and their governments.
freedoms
exchange for security
/tseriintsstttesseeaaataa (userrrnThat fascinating. Can you give me an example of how this might work in practice today?<lin_em \nSure! Imagine a country called Altruria, which recently implemented universal basic income program. The rationale behind this decision was rooted social contract theory-the government reasoned that providing financial stability would reduce societal unrest and foster trust among citizens. By ensuring everyone had safety net, Altruria aimed between rulers and ruled. <lin_end imm It there always someone who disagrees with these kinds of agreements? How do address those concers?(lin_end>3Mp Ib.start/>assistan InAboolutely valid point. Not all citizens may see eye-to-eye with the assumptions underlying social contract theory. For instance, critics argue that enforcing collective agreements can undermine individual autonomy. They believe people should have the right to opt out if they disagree with the terms imposed by the majority. This tension highlights the ongoing challenge of balancing communal obligations with personal freedoms within any society governed by social contracts. saraactsskiin_edd>> <| /aatsstttt/sseeeeneaa \BBased solely
fulfill
part of the implicit agreement
isn
the information provided in the quotation. What does this passage suggest about social contract?\nA. In the context of Altruria, the social contract involves the government providing financial stability through programs like universal basic income to reduce societal unrest and foster trust among citizens. It reflecta agreement where the state fulfills its duty by ensuring safety net for everyone. AnB. The social contract originates from philosophical ideas where individuals surrender some freedoms the state in exchange for security, argued by
implicit
27

---

Thomas Hobbes. Life without such agreement is described harsh and unstable, emphasizing the importance of this arrangement between citizens and governments Anc. Critics of the social contract argue that enforcing collective agreements can limit individual autonomy, suggesting people should have the right to opt out if they disagree with majority-imposss terma. This highlighta the tension between communal
obligations and personal freedoms vithin societies governed by social contracts. AnD. The quoted aegments does not address or is insufficient to answer this question, AmPlease question.\uPleaee analyze the question first, then provide the answer in the format: 'The answer is x."", "input":
"oupput" "Analyeis of the question: An- The passage specifically
discusses Altruria implementing universal basic income program
of addressing societal unrest and fostering social contract theory. An-
trust, framed the contert Option A directly aligns with the given passage, as it accurately describes the social contract in Altruria focusing government responsibility to provide financial stability and safety net in exchange for societal trust. An- Option while contains ideas related to the broader concept of social contract theory (such as Hobbes's 's views), includes information not mentioned
implied in the passage. This makes Option В incorrect when judged strictly on the passage. An- Option с introduces critical perspective on the social contract, implying tension between collective agreements and personal freedoms. Hovever, no such criticism or alternative perspective is mentioned implied in the given passage. This makes Option incorrect must focus solely on the details of Altruria and the specific interpretation of the social contract provided in the passage. Since the quoted segment provides enough information this question, option D is not selected. \n\nThe answer is A.", quotte_toke__span":
well. \n\nThe correct choice
139,
208
1.
321,
535
Multi-Span
" "instruction". "How do serotonin and dopamine influence emotional insights into regulation differently? Can you provide / their roles and impacts?<| ind ittaa//iitttat//a111 aCertainly! Let's delve into how serotonin and dopamine play distinct roles in emotional regulation. Firstly, serotonin plays crucial role in stabilizing mood and promoting feelings well-being and happiness. anxiety, sleep, and appetite. Low levels of serotonin are often associated with depression and anxiety disorders. behavior the brain. It influences motivation,
helps regulate
Secondly, dopamine is involved in reward-motivated and pleasure centers attention, and movement. High levels increased energy and focus but can also contribute to impulsive behavior. When serotonin levels are lov, individuals may experience heightened anxiety, irritability, and
dopamine can to
28

---

difficulty sleeping. This significantly affect daily functioning and overall quality life. Conversely, elevated dopamine levels can result in enhanced motivation and alertness. Hovever, excessive dopamine activity has been linked to conditions like schizophrenia and addiction. Therapeutically, selective serotonin reuptake inhibitors (SSRIs) commonly used treat depresaion by increasing serotonin availability. For conditions involving dopamine dysregulation, medications that modulate dopamine receptora are often .
only the details given in the quotations, describe the primary effects of the neurotransmitter discussed and any associated risks or benefits mentioned And. The neurotranemit teeer discussed is involved in reward-motivee motivvaed behavior and pleasure centers in the brain.
Elevated levels of this neurotransmitter
enhanced also pose risks such as impulsive
result
motivation and alertness behavior, achizophrenia, and addiction. \nB. The neurotransmitter discussed plays a crucial role in stabilizing mood and promoting well-being. Therapoutically, selective serotonin reuptake inhibitors (SSRIs) are commonly used to treat depression by increasing the availability neurotransmitter, addressing issues related to low levels. Anc.
this
The neurotransmitter discussed influences motivation, attention, and movement. For conditions involving dysregulation this neurotransmitter, medications that modulate dopamine receptors are often prescribed to manage symptoms like excessive energy and focus, which can also contribute to impulsive behavior and conditions like schizophrenia and addiction. \nD. The quoted segments does address or is insufficient to answer this question. AnPlease analyze the question first, then provide the answer in the format: 'The answer is I."", "input": "
"output"; "The analysis
the question involves carefully examining the quotations provided:\nAni. The first quotation behavior,
states that dopamine is involved in reward-motivate attention, motivation, movement, and pleasure centers. It lead
specifically adds that high levels of dopamine increased energy and focus, but also impulsive behavior. No mention is made of schizophrenia or addiction in the quotations. \n \n2. The second quotation refers exclusively to serotonin and discusses its therapeutic modulation through selective serotonin reuptake inhibitors (SSRIs) depression. It does not mention dopamine or any of its associated Iitects.Annnooo Option A**: This describes dopamine in terms of reward- motivated behavior and its potential effecta, which partially match the quotations. However, such as schizophrenia and addiction, which were not mentioned in the quotations. This makes Option A incorrect. ** Option B*s: This describes serotonin and stabilizing mood, second quotation but refers to serotonin
treating
let's compare the options: Annn- *
incorrectly adds conditions
which matches rather than dopamine. However, the question revolves around the neurotranemitter in the first quotation (dopamine), making Option в irrelevant Anln- *+Option C**: This accurately matches the first quotation by highlighting dopamine's influence on motivation, attention, movement, and its association with symptoms like excessive energy, focus, and impulsive behavior. It also notes the use of medications to modulate dopamine receptors for conditions involving dyeregulation dopamine. This aligns completely with the given information, but it does not mention unrelated conditions like schizophrenia or addiction. An\n- **Option D**: The quoted segments are indeed sufficient to ansver this
29

---

question, and since of the other options (C) is correct, Option D is not selected. An The is C.", toote_tokee_span"
89,
135
]
196,
239
].
"question_ton to._toeen_epann
245,
470
Exclude
"inatruction": "In the context of Physiological Paychology, could you provide details the levels of serotonin, dopamine, and norepinephrine various brain regions and discuss their impact on cognitive functions? Please ensure that each neurotransmitter's discussion includes its specific brain region and the cognitive function it influences. it.start/>assisete baaasistatt/nCertaaaat these neurotramsmitters and their impacts: Serotonin levels the hippocampus play a crucial role memory formation. Higher levels of serotonin have been associated with improved long term memory retention and consolidation. Dopamine levels in the prefrontal cortex processes. Elevated dopamine activity enhances cognitive flexibility and facilitates better problem-solving skills.
Ans|
Let examine the levels of
essential for piccisionmmaiing
Norepinephrine levels in the amygdala significantly influence emotional regulation. Increased norepinephrine can heighten alertness and responsiveness to emotionally charged stimuli, impacting how we process and react to emotions. Each nourotransmitter's presence in these specific brain regions contributes uniquely to cognitive bu.startt/user\\nnnnnno
the selected parts, which neurotranemitter and brain region play crucial role the specified cognitive function?\nA. Serotonin levels in the hippocampus play a crucial role in memory formation. Higher levels of serotonin have been associated with improved long- term memory retention and consolidation. AnB. Dopamine levels in prefrontal cortex are essential for decision -mkiing processes. Elevated dopamine activity enhances cognitive flexibility and facilitates better problem-solvine skills. \nc. Norepinephrine levels in the amygdala significantly influence emotional regulation. Increased norepinephrine alertness and responsiveness to emotionally charged stimuli, impacting how we process and react to emotions. The remaining information does not address or is insufficient to answer this question. AmPlease analyze the question first, then provide answer the format: 'The ansver ".,
heighten
"input": ""
"output": "Analysis of the question: \nThe question asks which
neurotransmitter and brain region play crucial role in the apecified cognitive function. Based the provided information: \n- Dopamine levels in the prefrontal cortex are explicitly mentioned as being essential for decision-makina processes, enhancing cognitive flexibility and problem- solviing skills. An- There mention of serotonin levels in the
30

---

hippocampus their role in memory formation in the provided conversation. An- There mention of norepinephrine
levels in the amygdala or their influence on emotional regulation in the provided conversation. AnAnThe ansver is ..,
"qoote_token_spa"""
81,
111
],
[
141,
183
1,
irrestion_token_eee
205,
370
Info-Combine
"instruction": "Could you provide detailed breakdown of the current pricing for key diagnostic instruments used in Physiological Psychology research?<| im > im_sitrt/t>>>> Eeciittant(nCrrtaiinn
To start, the EEG system costs $20 NnIn addition, the fMRI session priced at in. sssatt nMoreover, the PET scan usage fee is set at $3,000. i stati_statt/sssittttt \nThe GSR equipment, which is essential for monitoring skin conductance, is available for $500.<|
.0000
Heart Rate Monitor offered at $300, providing -ffeeciee option for cardiovascular Caaessments.<lin. itt.eeettttiiiiaaaaaaaiaiiieeeee / assistant \nFinally, the Eye Tracking System stands at $800,
rounding out the suite of diagnostic instruments <lim_end m <I stalstart/ssse AnWhich diagnostic instrument (s) cost more than the one(s) mentioned in the selected quotations? Build your ansver solely from the data presented in the selected quotations and the related information.\nA. You referenced the EEG system, which at $1 $1. Comparing these with the background data, no other instrument exceeds the cost of the EEG system. However, the PET scan, priced at $3,000, is more expensive than the AMRI session. Therefore, the PET scan is the only additional instrument more expensive than You referenced the GSR equipment, costing $500, and the Heart Rate Konitor, priced at $300. From the background data, all other instruments-ERC ($20,000) fMRI ($1,200), PET scan ( $3, $3, (800, and Eye Tracking System ($800) more expensive than either the GSR equipment or the Heart Rate Monitor. Thus, the EEG system, fMRI session, PET scan, and Eye Tracking System are all more costly than the ones you referenced. \nc. Based the information provided in the quotations, diagnostic instruments costing more than the ones mentioned are specified. The GSR equipment costs $500, and the Heart Rate Monitor costa $300. Therefore, there is no mention of any instrument costing more than these within the given quotations \n\nAnswer: None. \nD. More than correct, or all are wrong. \nPlease output the analysis of each option first, then choose the correct option in the following format: "The ansver is X, 2 (Do not include other marker)
$20,000, and the fMRI session, priced
one you have referenced. \nB
option
31

---

"input": "",
"output": "** Analysis of Option A:** \nlption references the EEG system ($20, ,000 and the fMRI session ($1 ($1,200) However, the quotations specifically mention the GSR equipment ($500) and the Heart Rate Monitor ($900), not the EEG or IMRI. Therefore, this option does not correctly address the issue raised in the quotations, mentioned in the quotations. This makes **Option incorrect **. #*.In)n**Anall Analysis of Option B:** \nOption correctly
it focuses instruments not
identifies that the GSR equipment ($500) and the Heart Rate Monitor ($300)
referenced in the quotations. compares these prices to background data, noting that all other instruments (EEG at $20,000, fNRI at $1,200, PET scan at $3 $3,000, and Eye Tracking System at $800) cost more than either the GSR equipment analysis aligns with the provided information and effectively addresses the question. Thus **Option is correct*s, \n\n** Analysis of Option C:** \nOption C claims that no diagnostic instruments costing more than the ones mentioned in the quotations are specified. This contradicts the background data which clearly shows that all other instruments (EEG, fMRI, PET scan, and Eye Tracking System) are more expensive than the GSR equipment ($500) or the Heart Rate Monitor ($300).
then
the Heart Rate Monitor. This
Therefore, **Option C is incorrect**. Anla of Option D:** \nlption suggests that the quoted segment is irrelevant to the question, so cannot be answered. However since the quoted information directly relates to the question by referencing specific costs of GSR equipment and Heart Rate Monitors, ***e do not choose Option D**, An'nThe ansuer is B
quote_token__ppn span""
95,
116
[
122,
144
].
cinestionntoken_spann"
176,
534
Coref
"instruction": "What is the world economic outlook for 1998 according to Kazuo Nakamura is a / im st_ttart I>assistant\nXinuu News Agency, Beijing, January 16, by wire -LRB- reporter Wenxin Fan -RRB- Kazuo Nakamura, Director of Research at Japan's Dai Ichi Kangyo Bank, said here today that the real growth rate of the world econony in 1998 will be about 3 X. Kazuo Nakamura, at public lecture jointly organized by Beijing's Wu Jia Economic Consulting Company, Ltd and Japan's Dai Ichi Kangyo Bank, made report concerning the new year's 's world economic and financial situation. After he reviewed the world economic and financial situation of 197, the economic growth in Asia, including America,
he said that due Japan, further slowing down, the economic growth even though its current economic activity is relatively good, and of other countries will also feel very large effect. Compared with the rapid growth of the past, the world economy
32

---

has entered a period of adjustment. He forecast that the real growth for the world economy this year would fall from the 3.9 of '97 down to X. Nakamura thought that the rate of economic growth of Asia's countries and regions, with Korea and Thailand being the most prominent, would slide downvard to large extent. But, he said, for these countries the great degree, exports
currencies already have devalued have recovered, and imports could decrease. Because of this, he expected that their economies would slip dounward to the lovest point in the next half year. He estimated that the rate of economic growth in ASEAN would be about 4.5 percent.
Nakamura said that the Japanese econony exhibited negative grouth in the year '97, and that in '98, if it didn't adopt good economic policies, the economy would continue to be in negative grouth. He estimated it would be negative 0.5 percent plus minus. Hе said further that China's economic growth would also slow in speed, feeling the effects of the economies of Japan and Asia being depressed and of the devaluation of these countries' currencies. But because China's usual income and expenditures show favorable balance, and in addition it has over 100 billion
dollars foreign exchange reserves, it ien't likely to sink into the crisis like the other Asian nations. -LRB- End
(Sseer/Waa does the pronoun I selected refer to in the conversation negative growth\nß: China's\nC" Japan 's\nD: Can not be sure. \ mPlease output the analysis of each option first, then choose the correct option in the following format: "The ansver is X (Do not include other marker)", " "input": ""
"output": "The pronoun refers
the context, the sentence discusses the Japanese economy's performance 1997 and its prediction for 1998. The statement specifies that \"if it didn 't adopt good economic policies, the economy would continue
\"Japan's.
be in negative growth. Here, \"it\" logically refers to \"Japan's economy,
the preceding discussion about Japan's 's economic state and the conditional requirement of adopting good policies to avoid further negative growth. The other options do not align with the subject of the the logical flow of the discussion \n\nThe answer
is C.",
tuutte_toeen_span [
386,
386
1,
517,
583
D.2 Open-Ended Question Examples
Base
"instruction" "I've been reading about memory management in operating systems. Can you explain what \"page stealing\" is? It sounds intriguing but also bit ing actually quite
in.start//assistann \nAh, \"page stealing\ clever despite its name. When an operating system runs out of physical memory, it may need to reclain pages from processes that aren't actively using them. Instead of vaiting for
33

---

process to explicitly release memory, the system preemptively takes-or \"ateals\"-umue processes in need. This helps maintain performance without forcing applications to constantly monitor their own memory usage. Quite ingenious, isn't it?<lim_ent inttstnnnlissttatt nThat makes sense! Now, I heard about this theoretical 0s called ZyloSys. Does do anything special with memory management<lli__ndd i sttartt>aassstttttladdeeee takes things further by implementing hybrid approach. It combines traditional paging with selective swapping to optimize both speed and resource utilization. Specifically, ZyloSys employs an adaptive mechanism where frequently accessed data remains in RAM, while less critical information gets swapped out to disk storage dynamically. This ensures that high-priority tasks receive sufficient resources without excessive delays caused by constant disk (nndfi/<iimeedd/ itte /usee//l does sound fascinating. But doeen't relying on virtual memory slow things down compared to keeping everything in RAM? What's the balance bassistaat /nnreat \nGreat question! While virtual memory allows for much larger addressable space than pure RAM, there is indeed trade- Accessing data from disk is significantly slover than retrieving it directly from RAX. However, modern systems mitigate this issue through intelligent caching strategies. For example, frequently used pages are kept in RAM even if they originate from virtual memory. Thus, the key lies in balancing capacity (via virtual memory) with performance ( through efficient RAX allocation). It's all about finding the sweet spot based on workload characteristics.. wenrrsttis..iin.endd end Andl>nns] (neffe exclusively to the information quotation. Rewrite the following explanation about memory management in more readable differently styled way.", "input"; ""
pages and assigns them to other
ZyloSys
Fascinating
terms of speed.
the
"output"; "If an operating system runs low on physical memory,
might have to recover unused pages from processes that aren 't currently utilizing them. Rather than waiting for process to voluntarily relinquish memory, the system proactively takes- Wateals\"-thees idle pages and allocates them other processes requiring them.",
"qutte_toee_sspa""" [
52,
108
1,
"question_tue seattiinntokeesppa. 398,
422
Multi-Span
" "instruction": "Could you provide overview of how major central banks manage inflation?<lin, /t..e/////.mm...taataaaaaaaa nCertainly. Let us begin with the Federal Reserve. The Federal Reserve aims to maintain price stability and maximum employment through various monetary policy tools. It primarily open market operations, adjusting the federal funds setting reserve requirements. By raising or lowering interest rates, the Federal Reserve influences borrowing costs and overall economic activity. Next, let's discuss the European Central Bank. The European Central Bank (ECB) focuses
34

---

maintaining price stability within the Eurozone. Its primary tool is the adjustment Additionally, the ECB conducts asset purchases and provides forward guidance to influence market expectations and stabilize inflation over the medium term. Lastly, we will cover the Bank of Japan. The Bank of Japan employs unique approach called Quantitative and Qualitative Monetary Easing QQE). This involves expanding the monetary base and purchasing a wide range of assets, including government bonds and exchange traded funds. The Bank of Japan also targets yield control policy
key interest rates.
achieve its inflation objectives.<1 im in_end (>)d> inssramtltttlll / nnith only the details given in quotations, list the primary strategies used by the central banks to manage inflation.", "input": ""
"oupput"; "The Federal Reserve aims maintain price stability and maximum employment through various monetary policy tools, including open market operations, adjusting the federal funds rate, and setting reserve requirements. The Bank of Japan employs Quantitative and Qualitative Monetary Easing (QQE), involving the expansion of the monetary base and purchasing wide range of assets, including government bonds and exchange- traded funda. It also targets a yield curve control policy to achieve its inflation objectives.", quotte_toee__spar
31,
83
1.
156,
216
].
qqqestiin intestionttokene totiioonoooessspa... 222,
243
Exclude
"instruction": "I've been reading up on hov artificial intelligence is being used in healthcare. What are some ethical considerations around this in (a.start/>assistane \nSure, there considerations when comes to using AI healthcare. One key concern is the reliability and accuracy of AI-driven diagnostic tools. These tools can sometimes make errors that could lead to misdiagnosis or inappropriate treatments. Another significant isaue patient privacy and data security When AI systems process sensitive health data, there's alvays a risk of data breaches or unauthorized access, which can compronise patient confidentiality. Lastly, the role of human oversight is crucial. Even with advanced AI algorithms, having human professionals involved ensures that decisions are not solely based on machine outputs, providing an additional layer of safety and user AnInnniin the selected regions, compile concise list of key points that aummarize the main concerns discussed." "input": ""
several important ethical
"output": " AI-driven diagnostic tools can sometimes make errors, potentially leading to misdiagnosis inappropriate treatments. \n- Human oversight is crucial to ensure decisions
35

---

are not solely based machine outputs, providing an additional layer of safety and accountability. spa
80
115
1, eettion_token_eeee
161,
179
Info-Combine
"instruction": "I'm curious about sentencing guidelines drug- related offenses. Are they different depending on the inat
sentencing guidelines for drug-related offenses vary significantly by jurisdiction. For example, Jurisdiction the minimum sentence for possession is set at wears.klin_endee www..//.nd//////iin.ttttt.te /ntt:// sounds strict. What about other places?<lin_end ianpcsssciinneeddmaiiiiiiiii >ln<lim.starrd i assistant(nIp Jurisdiction B, the minimum sentence for possession of controlled substances is comparatively more lenient than Jurisdiction [><lim_end>>> <limsttart |>user (epuurrrnnntte inInteresting! Are there any places where the /aasssstttttt
controlled substances
set 3 years, which
penalties are even harsher?< / nYes, in Jurisdiction C, the minimum sentence for possesaion of controlled substances than both Jurisdiction A and Jurisdiction B.<lim_ennd>>an in_start /start/>user/anee \Which jurisdiction has stricter lenient sentencing guidelines for possession of controlled substances compared the jurisdiction Build your answer solely from the data presented the selected quotations and the related information.", "input": ""
set at 7 years, which is stricter
more
have selected?
"output"; "You have selected Jurisdiction A, where the minimum
sentence for possession of controlled substances Comparing this to the background information, Jurisdiction has more lenient minimum sentence of 3 years, while Jurisdiction с has stricter minimum sentence of 7 years.". [
5 years.
40
62
].
177
218
Coref
"instruction"; "What is the current between Israel and Palestinians, statements made by Egyptian President Hosni Mubarak and Palestinian Leader Yasser Arafat regarding these negotiations ?<| im_ end >> im aassssttttssssaaattt. \nEgypt 's President Hosni
of peace negotiations what are the recent
36

---

Mubarak has called Israeli leaders to stop talking about and make peace with the Palestinians and Syria. interview with Israeli Television Monday Mr. Mubarak said Israel knows what is necessary for peace, the return of lands captured in the 1967 Middle East war. He also acknovledged the Palestinian refugees' right of return. In another interview Palestinian leader Yasaer Arafat said the Israelis and Palestinians had made real progress Red Sea resort of Taba, but Prime Minister Ehud Barak has suspended contacts with the Palestinians until after the February 6th election.
Correspondent Laurie Cassman has report. Prime Minister Barak decision not to meet Yasser Arafat this week comes after the Palestinian leader's angry speech on Sunday at an economic forum in Switzerland. Mr. Arafat accused Israel of vaging savage and barbaric war against his people. One day earlier Israeli and Palestinian negotiators had announced that a week marathon talks have brought them closer than ever to a final agreement. After the angry outburat Mr. Arafat reaffirmed his commitment to peace and his willingness to meet Mr. Barak. The meeting Stockholm, but Mr. Barak's security advisor told Israeli Radio Monday such a meeting now is out of the question. He described the Arafat apeech as inflammatory. Mr. Barak can not risk the controversial meeting ahead of next reek's election. Hе still trails far behind the hard line Likud Party leader Ariel Sharon in the opinion polls. Laurie Cassman, VOA News, Jerusalem,
expected later this week in
does the pronoun I selected advocate for in this context, particularly regarding peace efforts and territorial disputes?", " "input": "
"output": "The pronoun refers to Egypt's President Hosni Mubarak. In the given context, Egypt's President Hosni Mubarak calls on Israeli leaders to cease discussing war and instead puraue peace with the Palestinians and Syria. He emphasizes the necessity of returning lands captured during the 1967 Middle East war as a key step tovard achieving peace. Additionally, he acknovledges the Palestinian refugees' right of return.", gquote_token_speee [
114,
114
},
381,
401
E
Additional evaluations under complex scenarios
Cross-lingual use (Chinese, BASE). We instantiate the pipeline in Chinese test cross-lingual generalization under the same acceptance criteria as in the main data construction. Results (no language-specific training) are shown Table g
Table 9: Cross-lingual accuracy (%) on a Chinese BASE test set.
Variant
I--SSRUUTT LLAMA-3. 3.1-B--. -B--IBIRUCCT
Chinese (BASE)
93.0
87.0
37

---

Multi-turn quoting. We extend the BASE task to multi-turn variant in which sequence of questions each quotes a different span from the same dialogue. Results are shown in Table I0 indicating that both single and multi-turn quoting rely on the same span-awareness capability.
Table 10: Accuracy (%) on multi-turn extension of BASE.
Variant
QWEN-2.5-3B --INSRUCCT LLAMA-3. D.AMA--.--B-IISTREEE ----ISSTUUCT
Multi-turn guoting
92.0
92.0
Mixed cases. Our benchmark already contains mixed-capability items that combine MULTI-SPAN with other behaviors. Specifically, EXCLUDE(MULTII set, and INFO-COMBINE COMLINE(MULTII -PAn)) accounts for 24.4% of the these subsets separately shows that QUADA remains effective (Table TT).
accounts for 71.4% of the EXCLUDE set. Evaluating
Table 11: Accuracy (%) mixed subsets already present in the benchmark.
Sub-task
QWEN- INNSTRUUT LLAMA-3. -8--INSTRUCT
EXCLUDE (MULTI-SPAN
94.4
INFO-COMBINE (MULII-SPAN)
76.2
75.0
F
Experiment Details
Both training and inference were conducted on eight H20 GPUs (96 GB each). The inference temperature was set to 1.
Inference resources For Owwen-2.5114-IInstru the baseline model consumed about 32 GB of memory per GPU with per-sample latency of 0.87 Adding QUADA increased memory usage only slightly to 33.7 GB and latency 1.45 s, indicating minimal overhead.
Training resources Training Bwen-2.5-3B-Innttuue for three epochs completed h 25 min on the same 8xH20 setup, using roughly 28-30 GB of memory per GPU.
All human evaluators held least bachelor's degree and relevant research experience.
G Ethical considerations and recommended mitigations
Potential risks. Parameter-efffcieet adapters (e.g., LoRA; small bottleneck MLPs) have been shown carry stealthy Trojans while preserving benign utility (Dong al. 2023 Liu et 2024 Hubinger (t al. 2024]. Since QUADA also employs lightweight bottleneck modules, two attack surfaces are relevant: ()) training-time poisoning of adapter weights, and (ii) inference-time manipulation of span inputs (e.g., injected or tampered spans), which could misdirect attention and amplify misinformation.
Recommended practices. The following measures are suggested for practitioners and downstream users to reduce risk while preserving utility:
1. Signed and reproducible releases Distribute adapter checkpoints with cryptographic hashes (c.g.. SHA-256) and, when possible, detached signatures; record the exact backbone commit/version and provide deterministic export instructions
verify integrity prior to deployment.
2. Transparent data and pipeline Open-sourcing the data-construction pipeline, prompts, and the resulting reining/benchmark sets enables third-party auditing for hidden triggers or biases; documenting provenance and filtering criteria further facilitates reproducibility and safety review.
3. Backdoor-aware training Incorporate ad adversarial training future work guard against backdoor attacks during training [Hubinger et al. 2024].
4. Secure span channel Transmit span annotations over authenticated, encrypted channels; add integrity checks and input sanitization at the boundary where spans enter the model; maintain audit logs to support incident response.
38


Reviews:
[{"review_id": "t2rMeklRfv", "reviewer": "Reviewer_rbX8", "summary": "When interacting with AI systems, users often want to specify which parts of the conversation history the model should focus on and how to use those selected spans. However, existing systems lack explicit mechanisms for this, typically requiring users to manually re-enter relevant text. The authors address this gap by first decomposing the quoting challenge into five diagnostic sub-tasks: base, multi-span, exclude, info-combine, and coreference. \nThe key-contribution is that the paper introduces QuAda (Quotation Adapter), a lightweight module that attaches two bottleneck projections to every attention head in the LLM. This design allows the model to dynamically amplify or suppress attention to quoted spans at inference time, without modifying the prompt and while updating less than 2.8% of the model parameters. The authors also present the end to end pipeline that automatically synthesizes task-specific dialogues, verifies answer correctness through multi-stage consistency checks, and produces both a heterogeneous training corpus and a benchmark. The evaluation demonstrates that QUADA outperforms existing methods such as repetition (concat-repeat), using markers (marker-insertion), and attention-steering across the five scenarios described and generalizes to across models (qwen-1.5b to qwen-14b and llama-3.1-8b).", "questions": "From the description of the QUADA architecture, the module is designed as a plug-and-play adapter that is agnostic to the specific sub-task as long as the task can be formulated as span-conditioned generation. However, I wonder what are the data, and performance implications of adding more sub-tasks (beyond the five described)?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "[+] The paper presents, a novel problem formulation of formally defining span-conditioned generation for quoting in dialogue, capturing real-world conversational quoting behaviors through five diagnostic tasks (base, multi-span, exclude, info-combine, and coreference). If the claim on novelty is true, this is a very useful formulation to improve the model upon. \n\n[+] The proposed modeling technique- QUADA is lightweight, parameter-efficient (<2.8% of backbone weights), and does not require prompt modification, making it practical for easily adaptation. \n\n[+] QUADA generalizes across model scales (Figure 4) from 1.5B to 14B parameters on all five categories which is promising and I appreciate the small-scale scaling study. \n\n[-] The paper could provide deeper ablation studies to isolate the contributions of each component of QUADA (e.g., query side, value side) which will perhaps provide a better intuition as to contribution of each - similar to the intuition in section 4.3.\n\n[-] Although the parameter increase is small (<2.8%), the paper does not discuss inference speed or memory overhead in practical deployment, especially for very large models. Can you provide more details on the runtime and memory overhead introduced by QuAda?", "was_revised": false, "final_justification": "", "rebuttal": "**Dear Reviewer rbX8, we sincerely appreciate you taking the time to read our manuscript and provide positive and valuable feedback. Below are our responses to the concerns you raised, which will be incorporated into the updated version to further enhance the quality of our submission.**\n\n---\n> **`Weakness 1: The paper could provide deeper ablation studies to isolate the contributions of each component of QUADA (e.g., query side, value side) which will perhaps provide a better intuition as to contribution of each - similar to the intuition in section 4.3.`**\n\nWe thank the reviewer for the insightful suggestion and fully agree that a deeper ablation is valuable for understanding how each component of QUADA contributes to overall performance.\n\nIn section 4.4, we isolate the query-side and value-side QUADA, showing that disabling either one noticeably degrades accuracy, which indicates that both sides are indispensable. Following your recommendation, we additionally trained a variant that **removes span-conditioned modulation altogether** (i.e., no query- or value-side QuAda). This model, trained on Qwen-2.5-3B-Instruct, performs poorly across all five diagnostic settings, underscoring that span-conditioned generation is the key driver of QUADA’s gains.\n\n**Ablation of the components of QuAda on All Benchmarks (MCQ benchmark)**\n|Variants|Base|Multi-Span|Exclude|Info-Combine|Coref|\n|-|:-:|:-:|:-:|:-:|:-:|\n|QuAda(without query&value) |24.0%|16.6%|16.0%|36.8%|31.2%|\n|QuAda(without query)|78.8%|58.4%|49.6%|44.6%|69.4%|\n|QuAda(without value)|94.8%|92.7%|87.6%|75.0%|91.6%|\n|QuAda(full)      |95.2%|94.6%|92.8%|85.8%|90.2%|\n\nMoreover, we measured the cosine similarity between $b^v$ and the original value vector $v$: the average is **0.62** across layers, with **30 %** of layers above **0.80**. The norm of $b^v$ is roughly **0.6 × $v$**. These statistics indicate that $b^v$ consistently **reinforces** value representations of quoted spans.\n\n\n---\n> **`Weakness 2: Although the parameter increase is small (<2.8%), the paper does not discuss inference speed or memory overhead in practical deployment, especially for very large models. Can you provide more details on the runtime and memory overhead introduced by QuAda?`**\n\nWe understand the reviewer’s concern about deployment cost. On 8*H20 GPU, we measured both memory and per-sample latency duringinference with Qwen-2.5-14B-Instruct:\n\n|Model variant|Memory / GPU|Latency / sample|\n|-|:-:|:-:|\n|Baseline (no QUADA)|32 GB|0.87 s|\n|+ QUADA|33.7 GB|1.45 s|\n\nAs shown in the table, QuAda does not significantly affect memory usage or inference time during inference, which fully demonstrates its efficiency.\n\n---\n> **`Question 1: From the description of the QUADA architecture, the module is designed as a plug-and-play adapter that is agnostic to the specific sub-task as long as the task can be formulated as span-conditioned generation. However, I wonder what are the data, and performance implications of adding more sub-tasks (beyond the five described)?`**\n\nWe thank the reviewer for the question. As discussed in the paper, the five sub-tasks already cover most quotation-aware dialogue scenarios. All code for the data-construction pipeline has been provided in the Supplementary Material and will be open-sourced. If further training or testing is required for an additional sub-task, one only needs to adjust a few attribute settings to generate the needed data.\n\nFor example, we used the pipeline to create a Chinese test set for the Base scenario to evaluate QUADA’s cross-lingual generalisation:\n\n|Variants|Qwen-2.5-3B-Instruct|Llama-3.1-8B-Instruct|\n|-|:-:|:-:|\n|English|95.2 %|96.0 %|\n|Chinese|93.0 %|87.0 %|\n\nFollowing the same procedure in the paper, we manually reviewed the Chinese data and found the quality sufficient. We will release this benchmark along with the pipeline."}, {"review_id": "mC24TKIEWw", "reviewer": "Reviewer_53qF", "summary": "This research aims to address the challenge that LLMs cannot accurately understand and utilize specific \"referenced\" text in the conversation history. When a user refers to a specific piece of text in a conversation, e.g., \"check it with the formula I just highlighted\", existing models lack a mechanism to accurately focus on that piece and act according to the instruction.\nTo study the problem, the author defines the problem as span-conditioned generation with a benchmark covering five scenarios. The author also claims the data annotation pipeline and training-based framework (QUADA) as a contribution.", "questions": "1. Is there any failure analysis from the evaluation? Considering that QUADA achieves high scores, analyzing and attributing the small number of failure cases could help the understanding of the challenges (or lack thereof) in this task.\n2. As mentioned in the weaknesses, both the examples provided in the paper and the reported results raise concerns about the dataset’s difficulty. Have the authors evaluated stronger base models to better calibrate the benchmark? This could be either larger open-source LLMs or well-performing closed-source models.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe motivation behind the proposed task is clear and compelling. The benchmark contributes a useful testbed not only for evaluating quoting frameworks in real-world applications but also for probing LLMs’ ability to locate and recall information from longer contexts. The QUADA is well-reasoned, and the design is clean.\n\nWeaknesses:\n1. Although the dataset construction process involves multiple quality assurance steps, it remains synthetic in nature. That said, after manually reviewing several examples, I am generally positive about the data quality and find the contribution reasonable.\n2. While I agree with the significance of the task, I am concerned that the dataset may not be sufficiently challenging. Given the high performance reported in the experimental section, I question whether this benchmark can sustain long-term research value, especially as the language capability of foundation models keeps growing.", "was_revised": true, "final_justification": "After the discussion during the rebuttal period, I confirm my understanding of the work. Thus, I maintain my positive overall rating. \n\nI do not decrease the score as it indeed works on an important and interesting research problem; I do not increase the score because I learned the limited contribution toward the evaluation benchmark (which I previously recognized as *useful testbed* in the Strength section).", "rebuttal": "**Dear Reviewer 53qF, we sincerely appreciate you taking the time to read our manuscript and provide positive and valuable feedback. Below are our responses to the concerns you raised, which will be incorporated into the updated version to further enhance the quality of our submission.**\n\n---\n> **`Weakness 1: Although the dataset construction process involves multiple quality assurance steps, it remains synthetic in nature. That said, after manually reviewing several examples, I am generally positive about the data quality and find the contribution reasonable.`**\n\nWe thank the reviewer for taking the time to manually inspect our synthetic data and for acknowledging its quality. To ensure that every instance fully satisfies the required ability for its sub-task, we apply the strict filtering steps described in the Data-Construction Pipeline. The proportion of discarded examples for each sub-task is:\n\n|Sub-task|Base|Multi-Span|Exclude|Info-Combine|Coref|\n|---|:-:|:-:|:-:|:-:|:-:|\n|Drop ratio|32.9 %|37.3 %|70.4 %|74.4 %|15.2 %|\n\nThis aggressive filtering strategy is the main reason the final dataset attains the high quality observed during your review.\n\n---\n> **`Weakness 2 & Question 2: While I agree with the significance of the task, I am concerned that the dataset may not be sufficiently challenging. Given the high performance reported in the experimental section, I question whether this benchmark can sustain long-term research value, especially as the language capability of foundation models keeps growing. Have the authors evaluated stronger base models to better calibrate the benchmark? This could be either larger open-source LLMs or well-performing closed-source models.`**\n\nWe fully understand your concern and consider it highly forward-looking. We address it from three perspectives:\n\nFirst, our current benchmark is difficult even for today’s leading open and closed-source models without the QUADA module. We evaluated GPT-4o (closed source) and DeepSeek-R1 (open source) on the benchmark, and the results are:\n\n**Table: Performance of state-of-the-art open and closed-source models on our benchmark**\n\n|Sub-tasks|Base|Multi-Span|Exclude|Info-Combine|Coref|\n|-|:-:|:-:|:-:|:-:|:-:|\n|GPT-4o (Close Source)|22.6%|21.2%|25.0%|35.0%|22.8%|\n|Deepseek-R1 (Open Source)|16.8%|12.6%|12.2%|36.4%|23.6%|\n|Qwen2.5-3B-Instruct (QuAda)|95.2%|94.6%|92.8%|85.8%|90.2%|\n\nSecond, the primary goal of this work is to give existing LLMs application-level quotation-dialogue capability. This required a benchmark that matches everyday use-case difficulty and a training method that lets mainstream LLMs achieve strong results. Our data pipeline incorporates multiple validation stages, including manual audits, to ensure that each sample faithfully mirrors real-world scenarios, making the benchmark results reflects practical performance. The result that QUADA allows a comparatively small LLM to achieve state-of-the-art results demonstrates the effectiveness of this design. We believe stronger models equipped with QUADA will do even better, allowing developers to adopt our dataset and module to add robust, deployable quotation-dialogue ability.\n\nFinally, we fully understand your desire to create more challenging benchmarks covering harder quotation scenarios and to keep pushing the community forward. This is exactly our plan. Future work will explore contexts containing images and voice, multimodal mixed quotation, and more. Building on this study, we will continue to propose benchmarks and methods for increasingly complex settings to advance quotation-aware dialogue in LLMs.\n\n\n---\n> **`Question 1: Is there any failure analysis from the evaluation? Considering that QUADA achieves high scores, analyzing and attributing the small number of failure cases could help the understanding of the challenges (or lack thereof) in this task.`**\n\nThank you for the suggestion. We performed a case study on the outputs of the models trained in the paper and found that the remaining errors mainly fall into two categories, both limited by the capability of the base model on which QUADA is built:  \n1. The model correctly attends to the quoted span but also erroneously incorporates information from non-quoted context, leading to a wrong answer.  \n2. The model attends to the quoted span but misinterprets its content, again producing an incorrect answer.  \n\nThese findings indicate that QUADA has already endowed the LLM with the span-awareness required for quotation-based dialogue. Both error types will diminish as the underlying model’s comprehension and reasoning abilities continue to improve (see Fig. 4)."}, {"review_id": "U2Ua7IC2Le", "reviewer": "Reviewer_yQpy", "summary": "This paper addresses a crucial problem in LLMs: the lack of an explicit mechanism to handle in-dialogue quotation. The authors formalise this task as span-conditioned generation and propose an automated data generation pipeline. In addition, the authors propose QuAda, a lightweight training method. By dynamically amplifying or suppressing attention to quoted spans during inference, it enables plug-and-play span-conditioned generation.", "questions": "1. In the example (“check it with the formula I just highlighted”), the user does not explicitly mark a span. Does QuAda rely on pre-specified spans and task types at inference time? This greatly affects practical applicability.\n\n2. In the ablation section (Table 7), only query-only, value-only, and full QuAda variants are reported. Could you add a “no-modulation” baseline, where none of the span-aware adapters are applied.\n\n3. The training data seems mostly constructed as “single-turn queries over quoted spans.” Are there multi-turn or nested quoting cases in the benchmark?\n\n4. Real dialogue often contains indirect references (e.g., “that thing we discussed earlier”). Have the authors considered testing QuAda on loosely specified quotation cases?", "limitations": "Authors discussed limitations.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n1. The paper proposes an interesting and under-explored task—span-conditioned generation, which enables models to either refer to or exclude specific spans in dialogue history.\n\n2. The authors design a data pipeline that includes correctness verification and covers five well-defined quoting scenarios.\nThe proposed QuAda mechanism is simple yet effective. It enhances attention to relevant spans or suppresses attention to excluded ones via query and value bottleneck projections. The method is plug-and-play, does not require prompt engineering, and works across models.\n\n3. The authors plan to release the entire dataset, generation scripts, and training code, which will benefit follow-up research.\n\nWeaknesses\n\n1. While the five diagnostic scenarios are clear, they may not reflect the full diversity of real-world quoting behavior. The auto-generated dialogues also seem structurally simple—primarily single-turn question-answer pairs over selected spans—potentially limiting their contextual richness.\n\n2. The core approach—using MLPs to modulate attention scores—is reasonable but somewhat straightforward, offering only modest technical innovation beyond prior attention-steering work.\n\n3. In the current setup, the span and task type appear to need to be manually defined, which creates a gap between this method and real-world usage. The system does not yet solve the end-to-end problem exemplified by the motivating example in the abstract.", "was_revised": true, "final_justification": "This is a well-motivated piece of work addressing an underexplored but practically relevant challenge. While some limitations remain (e.g., reliance on pre-specified spans, relatively simple training data), the authors have convincingly demonstrated the effectiveness of their approach. I continue to maintain my Borderline Accept rating for this paper.", "rebuttal": "**Dear Reviewer yQpy, we sincerely appreciate you taking the time to read our manuscript and provide positive and valuable feedback. Below are our responses to the concerns you raised, which will be incorporated into the updated version to further enhance the quality of our submission.**\n\n---\n> **`Weakness 1 & Question 3: While the five diagnostic scenarios are clear, they may not reflect the full diversity of real-world quoting behavior. The auto-generated dialogues also seem structurally simple—primarily single-turn question-answer pairs over selected spans—potentially limiting their contextual richness. The training data seems mostly constructed as “single-turn queries over quoted spans.” Are there multi-turn or nested quoting cases in the benchmark?`**\n\nThank you for the insightful question; it inspired us to add multi-turn and nested quoting scenarios to improve the benchmark’s coverage. We therefore modified our data-construction pipeline and created a multi-turn version of the Base task, where consecutive questions each quote a different span. Owing to the short rebuttal timeline, we could not prepare sufficient multi-turn training data, so we evaluated the generalization of models trained only on the original single-turn data:\n\n**Table: Performance under single-turn versus multi-turn quotation**\n\n|Variants|Qwen2.5-3B-Instruct|Llama3.1-8B-Instruct|\n|-|:-:|:-:|\n|Single-turn quoting|95.2%|96.0%|\n|Multi-turn quoting|92.0%|92.0%|\n\nEven without dedicated training, QUADA-equipped LLMs remain strong, implying that both single and multi-turn quoting rely on the same span-awareness capability. We will augment the dataset and add corresponding experiments in future work. Thank you again for the thoughtful suggestion; it substantially improves our study.\n\n> **`Weakness 2: The core approach—using MLPs to modulate attention scores—is reasonable but somewhat straightforward, offering only modest technical innovation beyond prior attention-steering work.`**\n\nWe understand your concern. Actually, QuAda modifies both the Query and Value sides of attention, not just the attention scores as in prior attention-steering work. Quotation-aware dialogue requires two properties at the attention level:\n\n1. When a context span is quoted, its semantics—not only its attention weight—may change. For example, if the quoted token is a pronoun, its value vector should be adjusted so that downstream QA can resolve the reference. Prior approaches cannot do this. We therefore add a bias vector $b^v$ to the value $v$ of quoted tokens. Table 7 shows that introducing $b^v$ markedly improves performance. An average cosine similarity of 0.62 between $b^v$ and $v$ across layers confirms that the value vector is indeed adapted when its token is quoted.\n\n2. The attention weights assigned to tokens within the quoted span should be computed **adaptively**, rather than being fixed in advance as in prior attention-steering methods. In the Exclude scenario the attention should be small, whereas in the Base scenario it should be large. The bias vector $b^q$ captures this semantic signal and supplements the original query vector during attention calculation.\n\nBy addressing these two aspects—both absent in prior attention-steering work—our design achieves quotation awareness in a simple yet effective manner.\n\n\n\n> **`Weakness 3 & Question 1: In the current setup, the span and task type appear to need to be manually defined, which creates a gap between this method and real-world usage. The system does not yet solve the end-to-end problem exemplified by the motivating example in the abstract. Does QuAda rely on pre-specified spans and task types at inference time? This greatly affects practical applicability.`**\n\nWe focus on the scenario in which a user engages in dialogue by selecting the span they wish to quote. This situation occurs frequently in everyday applications, so we aim to endow the model with this ability. When the user highlights a region (e.g., on an app page or in a browser), the client can automatically call an offline tokenizer to obtain the token span of the quoted text and feed it to the LLM. The paper addresses how to perform span-conditioned generation given that span; the concrete engineering pipeline will be developed later and is not included in this manuscript, but we appreciate the chance to clarify this point.\n\nAs for the task type, after a single model is trained with QUADA, the user **never** needs to specify the task manually. The user simply inputs natural language to express their request, exactly as shown in our examples.\n\n> **`Question 2: In the ablation section (Table 7), only query-only, value-only, and full QuAda variants are reported. Could you add a “no-modulation” baseline, where none of the span-aware adapters are applied.`**\n\nTo address your question, we have added the “vanilla-no-modulation (Vanilla in Table 4)” and “trained-no-modulation” baselines to Table 7. The latter is a QUADA adapter trained on Qwen-2.5-3B-Instruct without providing the span. The results below show that span-conditioned generation via QUADA is indispensable for the quotation task:\n\n**Table: Ablation study with “no-modulation” baselines** \n \n|Variants|Base|Multi-Span|Exclude|Info-Combine|Coref|\n|-|:-:|:-:|:-:|:-:|:-:|\n|Vanilla|28.4%|24.4%|26.8%|36.2%|29.4%|\n|QuAda(none-span)|24.0%|16.6%|16.0%|36.8%|31.2%|\n|QuAda(query-only)|78.8%|58.4%|49.6%|44.6%|69.4%|\n|QuAda(value-only)|94.8%|92.7%|87.6%|75.0%|91.6%|\n|QuAda(full)|95.2%|94.6%|92.8%|85.8%|90.2%|\n\n> **`Question 4: Real dialogue often contains indirect references (e.g., “that thing we discussed earlier”). Have the authors considered testing QuAda on loosely specified quotation cases?`**\n\nThank you for raising this question, we wish to clarify the scope of our work. The paper targets situations in which the user explicitly designates a portion of prior text for the LLM to respond to (or exclude), typically through actions such as highlighting or quoting. The indirect references you mention are indeed related, but they are not the main focus here. Evaluating an LLM’s ability to recall relevant historical information under user-specified constraints is certainly interesting, and we will follow future work in that direction."}, {"review_id": "2tspSqbINC", "reviewer": "Reviewer_NVEt", "summary": "This work presents an innovative approach to enhancing large language models for handling quotations in conversational contexts. It formalizes the challenge as span-conditioned generation, where each dialogue turn consists of a conversation history, a set of token-offset quotation spans, and an intent utterance. The authors break this problem into five diagnostic tasks—Base, Multi-Span, Exclude, Info-Combine, and Coref—to cover various quotation scenarios. They introduce QuAda, a lightweight, training-based method that integrates bottleneck projections into the LLM’s attention mechanism, enabling it to dynamically focus on or ignore quoted spans without altering the prompt and with minimal parameter overhead (less than 2.8% of backbone weights). Additionally, they develop a data pipeline to synthesize task-specific dialogues and create a benchmark for evaluation. Experiments on two LLMs (Qwen2.5-3B-Instruct and Llama3.1-8B-Instruct) show that QuAda outperforms baseline methods across all scenarios and generalizes to unseen topics, offering a practical solution for quotation-aware dialogue.", "questions": "- More Computational Details: Can the author provide more details, beyond what is mentioned in Appendix E, about (1) the training costs (memory usage and training time), and (2) the additional memory/latency overhead by QuAda during inference, for better reproducibility assessment?\n- Real-World Data: Have the authors considered testing QuAda on real-world conversational datasets rather than purely LLM-synthesized datasets? If not, what are the challenges?\n- Ethical Concerns: Are there potential negative societal impacts (e.g., misinformation amplification) to address? For example, QuAda employs simple bottleneck MLPs, which can fall victim to a backdoor attack. Including this could resolve an ethical gap, maintaining or increasing the current rating.", "limitations": "The authors acknowledge limitations like reliance on synthetic data and the need for real-world validation, which is commendable. However, they do not address potential negative societal impacts. I suggest adding a paragraph discussing ethical concerns, such as the risk of manipulating conversations, to provide a more balanced perspective.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- This paper is well written and clear, with comprehensive experiments.\n- This work addresses a practical gap in human-AI interaction by formalizing quotation handling, which is quite novel and interesting.\n- The five diagnostic tasks provide a thorough and systematic way to assess quotation-handling capabilities.\n- The automated synthesis of high-quality training data and a validated benchmark is a significant contribution to the field.\n- QuAda’s lightweight design (less than 2.8% additional parameters, no prompt changes) makes it practical for real-world deployment.\n- The evaluation across multiple models, scenarios, and metrics, supported by attention visualizations, demonstrates the method’s effectiveness and interoperability.\n\nWeaknesses:\n\n- Limited Model Diversity: Testing is confined to two LLMs; broader testing across different architectures would bolster generalizability.\n- Interpretability vs. Controllability Trade-off: While attention maps (Figure 3) offer some interpretability, QUADA's bottleneck MLPs remain a black box. We don't know what \"guidance vectors\" (b^q and b^v) learn, making diagnosis difficult when the model fails. In contrast, methods like MARKER-INSERTION, despite poorer performance, are more transparent and controllable.\n- Scenario Coverage Limitations: This work claims its five diagnostic tasks span practical quoting behaviours, which can be a strong assertion. However, real-world references can also be composite, involving multiple types simultaneously (e.g., Multi-Span, Exclude, and Info-Combine). The paper lacks testing on the model's performance in these crucial mixed scenarios.", "was_revised": true, "final_justification": "I continue to maintain my Accept rating for this paper. The authors' rebuttal clarified my questions and did not raise any new concerns for me.", "rebuttal": "**Dear Reviewer NVEt, we sincerely appreciate you taking the time to read our manuscript and provide positive and valuable feedback. Below are our responses to the concerns you raised, which will be incorporated into the updated version to further enhance the quality of our submission.**\n\n> **`Weakness 1: Limited Model Diversity: Testing is confined to two LLMs; broader testing across different architectures would bolster generalizability.`**\n\nWe sincerely appreciate your suggestion. Below we provide additional results obtained with our method on the phi3.5-mini-Instruct model to demonstrate its generalization ability:\n\nTable: Results of QUADA on phi3.5-mini-Instruct  \n|Variants|Base|Multi-Span|Exclude|Info-Combine|Coref|\n|-|:-:|:-:|:-:|:-:|:-:|\n|Vanilla|23.4%|19.6%|27.0%|41.6%|19.4%|\n|PASTA|45.2%|43.0%|32.6%|42.2%|25.6%|\n|Marker|80.4%|63.6%|37.2%|40.0%|65.2%|\n|Concat|97.2%|94.4%|94.6%|80.6%|44.2%|\n|QuAda|97.4%|96.2%|96.2%|81.8%|89.2%|\n\nAs the table shows, QUADA remains the top performer on phi3.5-mini-Instruct, indicating that our method generalizes well across model architectures.\n\n> **`Weakness 2: Interpretability vs. Controllability Trade-off: While attention maps (Figure 3) offer some interpretability, QUADA's bottleneck MLPs remain a black box. We don't know what \"guidance vectors\" (b^q and b^v) learn, making diagnosis difficult when the model fails. In contrast, methods like MARKER-INSERTION, despite poorer performance, are more transparent and controllable.`**\n\nYour point is very valuable, we provide additional clarification here. Building on the attention heat-maps in Fig. 3, we performed a quantitative analysis of the relationship between the vectors $b^v$ and $v$. The average cosine similarity across all layers is 0.62, with more than 30% of layers above 0.80; the norm of $b^v$ is about 0.6 × $v$. These statistics suggest that $b^v$ reinforces the value information of the quoted span.\n\nFor controllability, we introduced a hyper-parameter λ to scale the injection strength of $b^q$ and $b^v$. We set λ = 0, 0.25, 0.5, 0.75 and 1, and evaluated a Qwen2.55-3B-Instruct model equipped with the trained QuAda module. The results are shown below:\n\n**Table: Injection-strength control experiment for QuAda**\n  \n|Variants|Base|Multi-Span|Exclude|Info-Combine|Coref|\n|-|:-:|:-:|:-:|:-:|:-:|\n|λ = 0   |28.4%|24.4%|26.8%|36.2%|29.4%|\n|λ = 0.25|63.2%|48.4%|23.8%|33.0%|31.0%|\n|λ = 0.50|80.4%|75.2%|38.6%|47.8%|47.4%|\n|λ = 0.75|93.0%|89.6%|78.8%|77.0%|80.6%|\n|λ = 1   |95.2%|94.6%|92.8%|85.8%|90.2%|\n\nTaken together, Fig. 3, the $b^v$–$v$ analysis, and the injection-strength study, QuAda is less visually interpretable than MARKER-INSERTION, but it is not entirely a black box. Its outputs correspond to explicit instructions on where the LLM should attend, plus strengthened representations of those regions. We will add these experiments to the paper to further support QuAda’s interpretability.\n\n---\n> **`Weakness 3: Scenario Coverage Limitations: This work claims its five diagnostic tasks span practical quoting behaviours, which can be a strong assertion. However, real-world references can also be composite, involving multiple types simultaneously (e.g., Multi-Span, Exclude, and Info-Combine). The paper lacks testing on the model's performance in these crucial mixed scenarios.`**\n\nThank you for raising this constructive point. Our benchmark already includes **Exclude (Multi-Span)** and **Info-Combine (Multi-Span)** cases, accounting for 71.4 % of the Exclude set and 24.4 % of the Info-Combine set, respectively. We evaluated these subsets separately, and the results are:\n\n**Table: Performance of QuAda on mixed scenarios**  \n|Sub-tasks|Qwen-2.5-3B-Instruct|Llama-3.1-8B-Instruct|\n|-|:-:|:-:|\n|Exclude (Multi-Span)|93.3%|94.4%|\n|Info-Combine (Multi-Span)|76.2%|75.0%|\n\nAs the table shows, QuAda effectively handles these mixed scenarios.\n\n---\n> **`Question 1: More Computational Details: Can the author provide more details, beyond what is mentioned in Appendix E, about (1) the training costs (memory usage and training time), and (2) the additional memory/latency overhead by QuAda during inference, for better reproducibility assessment?`**\n\nThank you for the question. For the Qwen2.5-3B-Instruct model, training on 8 × H20 GPUs completes three epochs in 1h25min, with each GPU using about 28~30 GB of memory.\n\nDuring deployment we compared runs with and without QuAda: the additional memory per GPU was under 1 GB, and the per-sample latency increase is below 0.6 s. These results confirm that QuAda is genuinely lightweight and efficient.\n\n\n---\n> **`Question 2: Real-World Data: Have the authors considered testing QuAda on real-world conversational datasets rather than purely LLM-synthesized datasets? If not, what are the challenges?`**\n\nThank you for the question. We also built a manually curated benchmark and evaluated QuAda on it; the results are:\n\n**Table: QuAda on the manual benchmark**\n \n|Models|Base|Multi-Span|Exclude|Info-Combine|Coref|\n|-|:-:|:-:|:-:|:-:|:-:|\n|Qwen2.5-3B-Instruct|96.0%|92.0%|96.0%|88.0%|84.0%|\n|Llama3.1-8B-Instruct|96.0%|98.0%|90.0%|94.0%|98.0%|\n\nThese numbers show that our method maintains high performance, confirming its practicality in real-world dialogue. We will also release this manual benchmark, and Table 6 of the paper further demonstrates the consistency between the automatically constructed and manual benchmarks.\n\n---\n> **`Question 3 & Limitation: Ethical Concerns: Are there potential negative societal impacts (e.g., misinformation amplification) to address? For example, QuAda employs simple bottleneck MLPs, which can fall victim to a backdoor attack. Including this could resolve an ethical gap, maintaining or increasing the current rating. I suggest adding a paragraph discussing ethical concerns, such as the risk of manipulating conversations, to provide a more balanced perspective.`**\n\nWe thank you for the suggestion and questions, they are important for broadening QuAda’s impact and ensuring that the community can deploy it safely. Recent studies show that parameter-efficient bottleneck adapters (e.g., LoRA, QuAda-style MLPs) can carry stealthy Trojan behaviours while preserving benign utility [1-3]. Similarly, we acknowledge that QuAda could be exposed to such attacks. To mitigate this risk, we will :\n\n(i) release official QuAda checkpoints—with hash signatures—compatible with mainstream open-source models;  \n\n(ii) open-source the complete data-construction pipeline and all the training sets and benchmarks we have created, enabling anyone to train their own QuAda and allowing the community to inspect the data for hidden “triggers”;  \n\n(iii) incorporate adversarial training in future work to guard against backdoor attacks during training [4].\n\nFurthermore, if QuAda is adopted in more scenarios, attackers might intercept the span information via network and inject spans that misdirect the model’s attention. We therefore also stress the need to secure the network channel.\n\n**We will highlight this discussion in the revised paper and document the above measures in the official QuAda repository.**\n\n**References**\n\n[1] The Philosopher's Stone: Trojaning Plugins of Large Language Models\n\n[2] LoRATK: LoRA Once, Backdoor Everywhere in the Share-and-Play Ecosystem\n\n[3] Lora-as-an-attack! piercing llm safety under the share-and-play scenario\n\n[4] Sleeper agents: Training deceptive llms that persist through safety training"}]

Author rebuttal:

Meta-review: This work focused on the quotation-aware dialogue scenario, and introduced a lightweight method named QuAda for improving the quotation modeling capability in large language models. Authors conducted a comprehensive evaluation on the span-conditioned generation across five diagnostic scenarios, and demonstrated consistent improvements in attention to quoted spans while maintaining the plug-and-play compatibility with existing models.

Reviewers acknowledged that the paper addresses a practical problem with clear motivation and real-world relevance. The experimental evaluation was considered extensive and convincing, spanning multiple model backbones and scales with ablation studies that demonstrate effectiveness.

While the overall rating is good, the current manuscript can be improved with more interpretability analysis, including no-modulation variants and injection-strength control studies to better clarify the span-conditioned mechanism (Reviewer rbX8, Reviewer yQpy, Reviewer NVEt). Detailed efficiency analysis is needed like the specific GPU memory usage, and latency (Reviewer rbX8, Reviewer NVEt). Durning the rebuttal, authors also added experiments under Chinese evaluation and multi-turn dialogue contexts (Reviewer rbX8, Reviewer yQpy). Since the quotation embeds important context, safety and potential jailbreak considerations should be discussed (Reviewer NVEt).
Decision: Accept (poster)