Title: Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules
Venue: NeurIPS 2025
Accepted: yes
Keywords: span-conditioned generation

quotation referencing

position-aware attention

adapters
Abstract: Human–AI conversation frequently relies on quoting earlier text—“check it with the formula I just highlighted”—yet today’s large language models (LLMs) lack an explicit mechanism for locating and exploiting such spans. We formalize the challenge as span-conditioned generation, decomposing each turn into the dialogue history, a set of token-offset quotation spans, and an intent utterance. Building on this abstraction, we introduce a quotation-centric data pipeline that automatically synthesizes task-specific dialogues, verifies answer correctness through multi-stage consistency checks, and yields both a heterogeneous training corpus and the first benchmark covering five representative scenarios. To meet the benchmark’s zero-overhead and parameter-efficiency requirements, we propose QuAda, a lightweight training-based method that attaches two bottleneck projections to every attention head, dynamically amplifying or suppressing attention to quoted spans at inference time while leaving the prompt unchanged and updating < 2.8% of backbone weights. Experiments across models show that QuAda is suitable for all scenarios and generalizes to unseen topics, offering an effective, plug-and-play solution for quotation-aware dialogue.

Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules

Abstract

Human-AI conversation frequently relies on quoting earlier text-'check it with the formula I just highlighted'-yet today's large language models (LLMs) lack an explicit mechanism for locating and exploiting such spans. We formalize the challenge as span-conditioned generation, decomposing each turn into the dialogue history, a set of token-offset quotation spans, and an intent utterance. Building on this abstraction, we introduce a quotation-centric data pipeline that automatically synthesizes task-specific dialogues, verifies answer correctness through multi-stage consistency checks, and yields both a heterogeneous training corpus and the first benchmark covering five representative scenarios. To meet the benchmark's zerooverhead and parameter-efficiency requirements, we propose QuAda, a lightweight training-based method that attaches two bottleneck projections to every attention head, dynamically amplifying or suppressing attention to quoted spans at inference time while leaving the prompt unchanged and updating < 2.8% of backbone weights. Experiments across models show that QuAda is suitable for all scenarios and generalizes to unseen topics, offering an effective, plug-and-play solution for quotation-aware dialogue. 1

1 Introduction

Large language models (LLMs) have become powerful generalists [Marjanovi´ c et al., 2025, Chowdhery et al., 2023, Kamalloo et al., 2023], yet their behavior in quotation-rich conversation remains rudimentary [Lin and Lee, 2024]. In everyday conversation, users routinely refer back to earlier turns:

'That result looks wrong-check it with the formula I just highlighted .'

Such requests simultaneously specify where the model should attend in the history and how the selected text should be used. Today's chat systems offer no principled mechanism for this: users must copy-paste snippets, insert disruptive markers, or hope the model guesses their intent.

We formalize the problem as span-conditioned generation (§2.1). Each turn is represented by the conversation history H , a set of quotation spans R , and an intent utterance U . From this abstraction, we derive one BASE scenario and four orthogonal extensionsMULTI-SPAN , EXCLUDE , INFOCOMBINE , and COREF -that together cover real-world quoting behavior. Specific examples are shown in Fig. 1.

Car A: $10K

Car B: $8K

Car C: $5K

Car D: $3K

How much is this?

Car B is $8K.

Base

Car A: $10K

Car B: $8K

Car C: $5K

Car D: $3K

Conversation History

Which is Cheaper?

Car C is Cheaper.

Multi-Span

Car A: $10K

Car B: $8K

Car C: $5K

Car D: $3K

Conversation History

Sort the remaining

cars by price.

Car B > Car C > CarD

Exclude

Car A: $10K

Car B: $8K

Car C: $5K

Car D: $3K

Conversation History

Is this car the most

expensive?

No, Car A is more

expensive.

Info-Combine

It's a dog.

It's a cat

It's a car

It's a bird

Conversation History

What does this

This pronoun refers

to a cat.

Coref

Existing span-conditioned generation methods include CONCAT-REPEAT [Yeung, 2025], MARKERINSERTION [Lin and Lee, 2024], and ATTENTION-STEERING (PASTA) [Zhang et al., 2023], all of which employ hand-crafted mechanisms (content repeat, boundary markers, or static attention re-weighting) without training. To evaluate their effectiveness, we construct a benchmark composed of five subsets using a carefully designed quotation-centric data pipeline. Given a topic and task specification, this pipeline synthesizes quadruples ⟨ H, R , U, ANSWER ⟩ and verifies their internal consistency automatically, and ensures each instance is validated by human annotators. On this test bed, all three baselines succeed on only a subset of the tasks (The untrained part of Tables 4 and 5), highlighting the limitations of non-trainable methods in capturing in-span information and underscoring the need for a training-based, position-aware solution.

To fill this gap, we propose QUADA-a training-based method that embeds token-level span information directly into attention mechanism through inserted lightweight adapters. For every attention head, two bottleneck projections dynamically amplify or suppress scores adaptively on tokens that fall inside the quoted spans and inject a stronger retrieval signal for them. The original prompt remains untouched, and the inserted adapter adds fewer than 2.8% additional parameters to the backbone. Trained on a heterogeneous corpus generated by our pipeline, QUADA delivers state-of-the-art performance on all five quotation scenarios, generalizes to unseen topics, and attaches seamlessly to multiple model families.

Our contributions are summarized as follows:

We present the first position-aware formulation of quoting in conversation and conclude five diagnostic tasks that reflect nearly all practical use cases in the real world.

We propose an automated pipeline, equipped with built-in correctness checks, to generate a high-quality, human-verified benchmark and large-scale, fully-automated training corpora.

We propose a training-based, plug-and-play method that empowers LLMs to follow quoting intents without prompt inflation, delivering consistent gains across all scenarios.

2 Preliminary: The Formulation of Quotation-Guided Generation

2.1 Problem Setup

Let the tokenized conversation history be H = ( t 1 , . . . , t n ) , where each t represents a token. At turn T , the user supplies a set of reference spans :

identified by token offsets in H , and a natural-language intent utterance U that specifies the user's intent over those spans (e.g., summarize, compare, answer the question based on the spans). s l and e l in Eq. 1 represent the start and the end tokens' index of the l th quotation span.

Given the triplet ( H, R , U ) , the model generates an answer y = ( y 1 , . . . ) by maximizing

This formulation characterizes how the model generates responses in scenarios where specific spans from the conversation history are referenced and an intent is provided.

Diagnostic sub-tasks To fully address the problem, we decompose quoting into a base scenario and four orthogonal extensions that together span practical quoting behaviors:

Base The intent is associated with exactly one quoted span, representing the simplest scenario.

Multi-Span The intent jointly involves multiple disjoint spans.

Exclude The intent explicitly requests that a quoted span be ignored when answering.

Info-Combine The intent asks the model to combine the quoted text with its related context, preventing over-focusing.

Coref Resolution tasks where the quote might be a pronoun, similar word, or sentence whose meaning depends on its exact position.

The five diagnostic tasks cover four orthogonal axes of complexity that build on the Base primitive: (i) Multi-Span increases span cardinality; (ii) Exclude introduces negative selection; (iii) InfoCombine requires context-aware synthesis beyond the quoted text; and (iv) Coref demands token-level positional resolution. Taken together, they form a minimal yet complete basis that mirrors the ways users quote information in real conversation.

2.2 Existing Strategies for Span Injection

We analyze three existing strategies for injecting the triplet ( H, R , U ) into an LLM.

(i) CONCAT-REPEAT For every ( s l , e l ) ∈ R , verbatim content H s l : e l is prepended to U , yielding U ′ = [ H s l : e l ; U ] . Duplication conveys relevance but discards positional origin and inflates the prompt by ∑ l ( e l -s l +1) tokens.

(ii) MARKER-INSERTION Boundary tokens such as ⟨ EMPHASIZE ⟩ and ⟨ /EMPHASIZE ⟩ are inserted around each span. The modified history H ′ is concatenated with U and re-encoded. Position is explicit at the cost of roughly doubling tokens per conversation.

(iii) ATTENTION-STEERING Pointer-Aware Sparse Tailoring keeps the textual prompt unchanged. Instead, it multiplies attention scores on a fixed subset of heads whenever a token's index falls inside any ( s l , e l ) . This produces an implicit positional bias with zero prompt overhead, but a single global scaling factor can limit the model's generalization capability in diverse scenarios and tasks.

We summarize the trade-offs of these methods in Table 1. An extended discussion of related works is provided in Appendix A.

3 Method

Our approach has two mutually-reinforcing components. First, a quotation-centric data pipeline generates large-scale training corpora and rigorously validates our benchmark that exercises the five scenarios introduced earlier. Second, a training-based position-aware module injects span information into a frozen LLM with minimal additional parameters.

3.1 Dataset Construction Pipeline

The pipeline turns a high-level specification into quadruples ⟨ H, R , U, ANSWER ⟩ through five stages.

Step 1: Attribute synthesis To ensure that the data we generate is as generalizable and realistic as possible, following [Yu et al., 2023, Long et al., 2024], we prompt LLM to sample a structured attribute set that fully determines the forthcoming conversation and task. Table 2 lists the attributes and their corresponding value ranges. Detailed descriptions of specific attributes are provided in Appendix B. Each scenario is paired with its own task list, which is also generated by the LLM and stored with the attributes.

Step 2: Conversation and Quotation span generation Given the sampled attributes, the generator LLM (including o1, o1-mini [Jaech et al., 2024], o3-mini [OpenAI, 2025], GPT-4o [Hurst et al., 2024], Qwen-plus [Yang et al., 2024]) writes (i) a multi-turn conversation H and (ii) an exhaustive list of token-offset spans covering every quotable information point, yielding the reference set R . For the Coref scenario, we directly utilize the gold pronoun-antecedent spans from the CONLL-2012 corpus [Pradhan et al., 2012], embedding them verbatim into H to keep offsets valid.

Step 3: Task-driven question and answer generation Conditioned on H , R , and the chosen TASK + SCENARIO, the LLM must (i) craft a single question U applicable to all relevant span subsets, (ii) select span subsets that satisfy the scenario definition (1 span for BASE; ≥ 2 for MULTISPAN, EXCLUDE, and INFO-COMBINE), and (iii) produce a correct answer for each subset. For Info-Combine, spans not selected serve as backgrounds.

Step 4: Automatic validity checks We cast U into a multiple-choice format whose options are the answers from Step 3, then run three tests: (i) Span sufficiency -with only the chosen spans (plus context for Info-Combine scenario), the generator LLM must pick the correct option; (ii) Span necessity -without those spans, the LLM must fail or return multiple candidates; (iii) Context requirement (Info-Combine only)-a foil answer that ignores the related context must mislead the LLM when no background information is provided. A sample is accepted only if all tests pass.

Step 5: Human verification For the benchmark split, we add a manual audit layer. An LLM first rates every item against its attributes, flagging any item scoring below a threshold. Human annotators then inspect flagged cases, correcting or discarding them as needed. This extra pass ensures the benchmark's reliability while keeping annotation cost manageable.

The pipeline yields a heterogeneous training set and a human-validated benchmark, each containing multiple-choice and open-ended variants for the five scenarios. Table 3 reports the sample counts for each variant and scenario. By training on topics disjoint from the benchmark, we guarantee that models learn the skill of quoting rather than memorizing the content. To safeguard label correctness, the automatic validity checks described above remove 32.9%, 37.3%, 70.4%, 74.4%, and 15.2% of the initial candidates for the BASE, MULTI-SPAN, EXCLUDE, INFO-COMBINE, and COREF tasks,

respectively-an elimination rate we found essential for the reliability of the final corpus. Examples of each scenario are listed in Appendix D.

3.2 QUADA: Position-Aware Attention Modulation

Given our benchmark's fine-grained tasks, an effective model must satisfy three requirements: (i) introduce no prompt overhead, (ii) be parameter-efficient, and (iii) steer attention toward (or away from) quoted spans on demand. We satisfy these constraints with Quotation Adapter (QUADA) , a drop-in module that adds two lightweight bottleneck projections to every attention head (Fig. 2).

User Input & Generation

Steered

α

��

�

��

��

Context

Quotation(s)

��

��

��

��

�

��

��

��

Let q i , k j , v j ∈ R d denote the query, key, and value vectors of a single attention head at positions i and j ( j ≤ i ), and d represents the dimensionality of the attention head. At inference time, compared with the non-trained methods, the model additionally receives the token-level span of the quotations and the user inputs:

where R is the set of quoted spans and Q marks the span of the user's latest utterance, s q represents the start token's index of the question part. For notational convenience, we further define an indicator:

Query-side modulation To enable the model to adjust the attention given to the quotation span adaptively, each head employs a bottleneck MLP E q : R d → R d that yields a bias vector b q i = E q ( q i ) for query vector. We modify only those queries originating from the current question and the generated tokens, i.e. i ≥ s q ; otherwise b q i = 0 . The resulting attention score is:

✶ ✶ The extra term ( b q i ) ⊤ k j selectively steers the attention score of quoted tokens, allowing the model to follow the user intent implicitly.

Value-side enrichment To strengthen the representation of quoted tokens, every head learns a second bottleneck MLP E v : R d → R d that produces a strengthen vector b v j = E v ( v j ) for the token belongs to the quotation spans. The value vector is then replaced by:

˜ v j = v j + ✶ j ∈R b v j . (6) The ˜ v j here can be computed directly from the cached v j , no additional pass over the conversation history is required.

We only update these bottleneck MLP layers during training, which ensures that the inherent capabilities of the LLM remain unaffected. Despite its flexibility, the parameter of QUADA overhead is minimal. Let r ≪ d be the bottleneck width, each head then introduces 2 dr +2 rd extra weights, keeping the total increase below 2 . 8% for the 3B and 1 . 6% for the 8B backbones used in our experiments.

4 Experiment

4.1 Experimental Setup

Backbone models We adopt two instruction-tuned LLMs with different scales and architectures: QWEN2.5-3B-INSTRUCT [Yang et al., 2024] and LLAMA-3.1-8B-INSTRUCT [Grattafiori et al., 2024]. For QUADA, we set the query and value side bottleneck width to r = 256 , which introduces 75 M trainable parameters on Qwen ( 2 . 8% of the model) and 130 M on Llama ( 1 . 6% ). All backbone weights are frozen.

Baselines and their trainable variants We benchmark QUADA against the three span-injection strategies from §2.2. Besides the original untrained versions, we devise trainable counterparts so that every method benefits equally from the synthetic corpus.

CONCAT-REPEAT. We attach an auxiliary bottleneck to each attention head. If a token belongs to a duplicated span, its re-encoded representation is added to the main projection, mimicking explicit repetition while allowing gradients to flow.

Base

Multi-Span

Exclude

Info-Combine

Coref

Prev-Conversation

Question &

Generation

Prev-Conversation

Question &

Generation

Prev-Conversation

Question &

Generation

Prev-Conversation

Question &

Generation

Prev-Conversation

Question &

Generation

Quotation

Quotation 1

Quotation 2

Quotation

Quotation

Quotation

50

50

100

100

75

1.0

1s01

150

100

200

200

0.8

250-

175

250

300

200-

300

0

50

100

200

250

300

100

150

250

00E

25

50

125

150175200

0.6

0.4

50

100

100

200

150

0.2

200

300

250

400-

0.0

[300

市

50

100

150

200

250

00E

100

200

300

400

MARKER-INSERTION The tokenizer is extended with two special symbols <emphasise> and </emphasise> . During fine-tuning, we update only their embeddings; all other parameters stay frozen, so the prompt overhead remains the same as in the untrained variant.

ATTENTION-STEERING (PASTA) In the zero-shot setting, we reuse the 25 attention heads originally selected on the JSON FORMATTING data [Zhang et al., 2023]. For the trainable version, we rerank heads on our own training split and use the top 25 at inference time, following the protocol in the original method.

Evaluation metrics Each subset is paired with a metric that matches its output structure: We employ two metrics to assess these methods. Accuracy (Acc) for the single-choice questions. A prediction is correct if the chosen option exactly matches the gold. Consistency score (Const) is used for open-ended generations. Following recent best practice [Gu et al., 2024], we prompt a GPT-4o-mini model [Hurst et al., 2024] to rate the consistency between the model's output and the gold answer on a scale from 1 to 5. Higher scores indicate stronger alignment. The full judging prompt is provided in Appendix C.

Additional experimental details (including the GPU model, memory specifications, and criteria for human evaluators) are provided in Appendix F.

4.2 Main Results

QUADA effectively handles diverse quoting behaviors Our method consistently achieves the best performance across all five scenarios on both backbones (Tables 4 and 5). In Base , Multi-Span , and Exclude scenarios, QUADA accurately adapts its response behavior without requiring explicit rule-based control. In the Info-Combine scenario, where the model must integrate quoted spans with the related unquoted backgrounds, QUADA avoids over-focusing on the quote itself, demonstrating its capacity to use the quotations in a context-aware manner. Finally, in the Coref task, which requires sensitivity to the exact position of the quoted text, QUADA achieves 90.2% accuracy on Qwen and 94.8% on Llama, which are far ahead of all baselines, highlighting its ability to understand the position of the quotation span.

Training is essential for the quotation task Across all scenarios and both models, trained variants of all methods consistently outperform their untrained counterparts, demonstrating that spanconditioned behavior cannot be captured through static or untrained injection mechanisms alone. The superior performance of QUADA when trained on our synthetic corpus provides further validation for

the effectiveness of our training methodology and span-based attention modulation strategy, as well as confirms the quality and comprehensiveness of the training set.

GPT-4o-mini judge is unbiased in our experiment To validate the accuracy of the 4o-mini-asjudge in our experiments, we randomly selected 25 data samples from each scenario and asked human evaluators to use the same rating criteria as the 4o-mini for evaluation. The results indicated a Pearson correlation coefficient of 0.75 between human and 4o-mini ratings, demonstrating that our 4o-mini evaluations for the Open-Ended task are generally unbiased relative to human judgment.

QUADA preserves the model's generative fluency We additionally evaluated fluency with GPT4o-mini. Both CONCAT-REPEAT and QUADA receive fluency ratings that are exceptionally clear, logically structured, and straightforward to read, indicating that our method does not degrade surfacelevel language quality.

QUADA adapts to complex scenarios Building on the five basic scenarios above, we further evaluate three practical extensions: cross-lingual use, a multi-turn variant where successive questions each quote a different span, and mixed cases that combine MULTI-SPAN with EXCLUDE/INFOCOMBINE. In all cases, QUADA requires no task-specific rules or architectural changes and preserves its span-conditioned behavior. See Appendix E for details.

4.3 Analyses

QUADA adaptively re-allocates attention Figure 3 plots the average head-wise attention for each diagnostic scenario. In Base and Multi-Span , the heatmap shows dark vertical bands exactly at the quoted offsets, while the rest of the context is uniformly suppressed-precisely what the tasks require. For Exclude , the pattern flips: the quoted segment turns pale, confirming that our method can down-weight forbidden spans. In Info-Combine , a hybrid pattern emerges: the quoted span still peaks, yet moderate attention is also maintained on other contexts, enabling the fusion of related information. Finally, in Coref , attention locks onto the quoted pronoun and its immediate neighborhood, providing the positional anchor needed for antecedent resolution. Together, these heatmaps show that QUADA dynamically adjusts attention according to the user's instruction. All visualized samples and their span annotations are provided in the Appendix D.2.

Validity of the Auto-Generated Benchmark To assess how well our synthetic benchmark reflects real-world citation behavior, we manually curated a reference benchmark by filtering and adjusting a subset of samples for each scenario and question type. We then evaluated 10 diverse models (from Qwen, Llama, Phi families, etc.) on both the auto-generated and human-curated benchmarks, and computed the Pearson correlation between their scores.

As shown in Table 6, all correlation coefficients exceed 0 . 82 , with several above 0 . 95 , indicating a strong agreement between the automatically constructed and manually validated evaluations. These results confirm that our synthetic benchmark faithfully reproduces the relative difficulty and quotation behaviors seen in human-curated data, validating its suitability as an evaluation benchmark for real-world quotation scenarios.

QUADA generalizes across model scales Figure 4 reports the Accuracy and Consistency for Qwen2.5 series models from 1.5B to 14B parameters on all benchmarks. Even the 1.5B model surpasses 90% accuracy on three scenarios and averages ∼ 4 . 0 consistency score. Performance rises steadily with scale, where the 14B variant achieves 98-99% on SINGLE and MULTI with a 4.5 consistency score. The monotonic trend indicates that QUADA delivers strong gains on small models and continues to improve as the model's parameter count grows.

100.0

97.5

95.0

92.5

90.0

87.5

Base

85.0

Multi-Span

Exclude

82.5

Info-Combine

Coref

Qwen1.5B

Qwen3B

Qwen7B

Qwen14B

(a) Effect of model scale on multiple-choice accuracy.

4.4

4.2

4.0

3.8

3.6

Base

3.4

Multi-Span

Exclude

3.2

Info-Combine

Coref

Qwen1.5B

Qwen3B

Qwen7B

Qwen14B

(b) Effect of model scale on open-ended const scores.

4.4 Ablation Study

Query and Value Modulation are Complementary in QUADA We compare three variants of QUADA on Qwen2.5-3B-Instruct while keeping the total trainable parameters fixed in Table 7. The results show that Value-only modulation already recovers most of the full method's strength (e.g., 92.8% on MULTI and 87.6% on EXCLUDE), confirming that richer value vectors give quoted tokens a strong retrieval signal. Query-only modulation is much weaker, suggesting that steering attention scores in isolation cannot compensate for under-informative token representations.

The full method outperforms both ablations on every task, with the most significant margin on INFO-COMBINE (+10.8% over value-only). In this setting, the query branch first directs attention toward relevant regions; the value branch then supplies the content needed to merge quoted spans with surrounding context. We probe the representations by computing the cosine similarity between the bias b v and its base vector v for every layer. The similarity averages 0.62 across layers, with 30% of layers above 0.80 , and ∥ b v ∥ ≈ 0 . 6 ∥ v ∥ . These statistics indicate that value modulation consistently reinforces quoted-span representations.

QUADA is Parameter Efficient Here we examine how the size of the QUADA, controlled by the bottleneck dimension r , affects the performance of QUADA. Table 8 reports the results of varying the bottleneck width r for the query and value projections in the Qwen2.5-3B-Instruct model. Notably, even at the smallest setting ( r =64 ), QUADA already achieves over 94% accuracy on three out of five tasks and maintains a generation consistency score around 4.0. This indicates that most span-conditioning behavior can be captured with a highly compact QUADA module.

To balance accuracy and model size, we adopt r =256 as the default configuration: it consistently matches or sometimes outperforms all other settings across all tasks while remaining lightweight (< 2.8% of the backbone's parameters). These results demonstrate that QUADA can deliver substantial performance gains without a large parameter budget, making it especially suitable for resourceconstrained deployments.

5 Conclusion & Future Work

We introduced span-conditioned generation as a principled formulation for quotation-rich conversation and released a diagnostic benchmark covering five orthogonal quoting behaviors. To facilitate research in this new setting, we developed an automatic data-generation pipeline capable of synthesizing large-scale, high-quality training corpora and human-validated benchmarks. Building on this data, we proposed QUADA, a lightweight adapter that injects positional signals via query and value side adapters without inflating prompts or substantially enlarging the backbone. Extensive experiments across two instruction-tuned LLMs and multiple model sizes (from 1.5B to 14B) demonstrated that QUADA consistently outperforms both training-free and training-based baselines, maintains fluency, and scales gracefully. Taken together, our results suggest that this work can substantially advance human-LLM interaction, without introducing adverse side effects. Potential backdoor and misinformation risks, along with our mitigation measures, are discussed in Appendix G.

The present work is confined to monomodal, monolingual quotation. Our adapter is trained and evaluated solely on English text spans; we have not experimented with other modalities (images, tables, code) or multilingual settings-such as quoting a French sentence within an English dialogue. Extending QUADA to multimodal and cross-lingual quotation will require new benchmarks and may reveal additional challenges for position-aware attention, which we plan to explore in future work.

Acknowledgments

This work is supported by Beijing Natural Science Foundation (No.4222037, L181010).

References

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.

Tian Dong, Minhui Xue, Guoxing Chen, Rayne Holland, Yan Meng, Shaofeng Li, Zhen Liu, and Haojin Zhu. The philosopher's stone: Trojaning plugins of large language models. arXiv preprint arXiv:2312.00374, 2023.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.

Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594, 2024.

Evan Hernandez, Belinda Z Li, and Jacob Andreas. Inspecting and editing knowledge representations in language models. arXiv preprint arXiv:2304.00740, 2023.

Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566, 2024.

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024.

Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models. arXiv preprint arXiv:2402.18154, 2024.

Ehsan Kamalloo, Nouha Dziri, Charles LA Clarke, and Davood Rafiei. Evaluating open-domain question answering in the era of large language models. arXiv preprint arXiv:2305.06984, 2023.

Kai Konen, Sophie Jentzsch, Diaoulé Diallo, Peer Schütt, Oliver Bensch, Roxanne El Baff, Dominik Opitz, and Tobias Hecking. Style vectors for steering generative large language model. arXiv preprint arXiv:2402.01618, 2024.

Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36:41451-41530, 2023.

Guan-Ting Lin and Hung-yi Lee. Can llms understand the implication of emphasized sentences in dialogue? arXiv preprint arXiv:2406.11065, 2024.

Hongyi Liu, Shaochen Zhong, Xintong Sun, Minghao Tian, Mohsen Hariri, Zirui Liu, Ruixiang Tang, Zhimeng Jiang, Jiayi Yuan, Yu-Neng Chuang, et al. Loratk: Lora once, backdoor everywhere in the share-and-play ecosystem. arXiv preprint arXiv:2403.00108, 2024.

Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv preprint arXiv:2311.06668, 2023.

Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. On llms-driven synthetic data generation, curation, and evaluation: A survey, 2024. URL https://arxiv. org/abs/2406.15126, 2024.

Sara Vera Marjanovi´ c, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lù, et al. Deepseek-r1 thoughtology: Let's <think> about llm reasoning. arXiv preprint arXiv:2504.07128, 2025.

OpenAI. OpenAI o3-mini, January 2025. URL https://openai.com/index/openai-o3-mini/ . OpenAI Blog, January 31, 2025.

Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. Conll2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Joint conference on EMNLP and CoNLL-shared task, pages 1-40, 2012.

Nishant Subramani, Nivedita Suresh, and Matthew E Peters. Extracting latent steering vectors from pretrained language models. arXiv preprint arXiv:2205.05124, 2022.

Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization. arXiv e-prints, pages arXiv-2308, 2023a.

Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023b.

Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. Retrieval head mechanistically explains long-context factuality. arXiv preprint arXiv:2404.15574, 2024.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024.

Matty Yeung. Deterministic quoting. https://mattyyeung.github.io/ deterministic-quoting , 2025. Accessed: 2025-05-09.

Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander J Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. Large language model as attributed training data generator: A tale of diversity and bias. Advances in Neural Information Processing Systems, 36:55734-55784, 2023.

Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao. Tell your model where to attend: Post-hoc attention steering for llms. arXiv preprint arXiv:2311.02262, 2023.

Qingru Zhang, Xiaodong Yu, Chandan Singh, Xiaodong Liu, Liyuan Liu, Jianfeng Gao, Tuo Zhao, Dan Roth, and Hao Cheng. Model tells itself where to attend: Faithfulness meets automatic attention steering, 2024. URL https://arxiv.org/abs/2409.10790 .

Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong, and Zhiyu Li. Attention heads of large language models: A survey. arXiv preprint arXiv:2409.03752, 2024.

Youxiang Zhu, Ruochen Li, Danqing Wang, Daniel Haehn, and Xiaohui Liang. Focus directions make your language models pay more attention to relevant contexts. arXiv preprint arXiv:2503.23306, 2025.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We substantiate our claims with both experimental evidence and theoretical analysis.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed the limitations of our work in the conclusion section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have provided detailed experimental settings in the experiments section and detailed module designs of our method.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We will release our code soon in the github.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have provided detailed experimental settings in the experiments section.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For each setting, we run 3 times and report the average results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We have reported relevant information in the experiment section.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read the NeurIPS Code of Ethics and followed it.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have discussed this in the paper.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have correctly cited all the assets we use.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We provide a detailed introduction of the proposed training dataset and benchmark in our paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [Yes]

Justification: We have provided the relevant information in paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

A Related Works

A.1 Attention-Guided Generation in Decoder-Only LLMs

A growing body of mechanistic-interpretability work shows that individual self-attention heads specialize in distinct generative roles[Zheng et al., 2024]. For instance, 'retrieval heads' that gather long-range evidence [Wu et al., 2024] or heads whose ablation removes knowledge conflicts [Jin et al., 2024]. Such span-sensitive behavior motivates our adapter.

The most closely related control method is PASTA [Zhang et al., 2023, 2024], which boosts a fixed set of heads whenever their keys fall inside a user-marked span. Focus Directions [Zhu et al., 2025] extend this idea by adding a learned vector to key/query activations, steering attention toward salient tokens without prompt markup. Both approaches inspire our query-side biasing, yet neither enriches value vectors nor handles multi-span or negative constraints, gaps that QUADA fills.

Latent-space steering

A complementary line of research controls decoder-only LLMs by editing internal activations. Liu et al. [2023] recasts in-context learning examples as a single in-context vector that is added to hidden states, achieving stronger task transfer with zero prompt overhead. More generally, Subramani et al. [2022] extracted steering vectors from pretrained LMs can nearly perfectly reconstruct target sentences, and enable sentiment transfer via simple vector arithmetic. Extending this idea, Turner et al. [2023b] formalises activation engineering and introduces ACTIVATION ADDITION, a lightweight method that shifts activations along a "love → hate" direction to control sentiment without optimisation. Konen et al. [2024] demonstrates analogous style vectors that steer output toward specific writing styles. Beyond style, Li et al. [2023] identifies a 'truthfulness' direction: nudging activations along it doubles LLaMA accuracy on TruthfulQA at inference time. Latent editing has also been used to modify factual knowledge: Hernandez et al. [2023] learn encodings of new facts and injects them into hidden layers so that generation aligns with the edited knowledge. Finally, Turner et al. [2023a] provides a systematic recipe for computing activation directions that steer toxicity, refusal, or politeness.

These works share our goal of parameter-efficient, inference-time control; however, they target global attributes (style, truthfulness, knowledge) rather than span-conditioned quoting . QUADA differs by selectively modulating query and value activations according to user-specified spans, enabling fine-grained, multi-span, and negative constraints that latent steering alone cannot express.

B Attributes for Data Generation

Topics

Arbitration and Mediation

Mechanical Engineering

Electrical Engineering

Electronic Engineering

Chemical Engineering

Civil Engineering

Materials Engineering

Aerospace Engineering

Biomedical Engineering

Computer Engineering

Environmental Engineering

Microeconomics

Macroeconomics

Econometrics

International Economics

Development Economics

Financial Economics

Labor Economics

Behavioral Economics

Public Economics

Environmental & Resource Economics

Nutrition and Diet

Sports Medicine & Fitness

Epidemiology

Public Health

Mental Health

Health Education

Health Management

Basic Clinical Medicine

Geriatrics & Rehabilitation

Health Policy

General Psychology

Developmental Psychology

Social Psychology

Clinical & Counseling Psychology

Cognitive Psychology

Personality Psychology

Physiological Psychology

Industrial/Organizational Psychology

Educational Psychology

Health Psychology

Marketing

Finance

Accounting

Management

Operations Management

Human Resource Management

Business Strategy

Entrepreneurship & Innovation

International Business

E-commerce

Cell Biology

Molecular Biology

Genetics

Physiology

Ecology

Developmental Biology

Microbiology

Zoology

Botany

Neurobiology

Ancient Philosophy

Modern Philosophy

Analytic Philosophy

Continental Philosophy

Ethics Metaphysics Epistemology Political Philosophy Philosophy of Science Aesthetics Data Structures & Algorithms Operating Systems Computer Networks Software Engineering Database Systems Artificial Intelligence Machine Learning Computer Graphics Distributed Systems & Cloud Computing Cybersecurity

Tone

Formal Casual Persuasive Analytical Creative Narrative Enthusiastic

Tasks

model to list each step or stage in the correct sequence. The correctness can be checked by confirming each listed step appears (in the right order) within the quoted region.

Specified Segment Contradiction Detection: Present a statement or claim and ask whether it contradicts the single quoted text , is supported by it, or is not mentioned at all. This is easy to verify: simply compare the statement to the quoted text to see if it's in direct conflict , alignment , or absent.

Specified Information Existence Detection: Present a statement or claim and ask whether it appears in the quoted text. In this task , the statement refers to information that exists in the unquoted portion of the original context but not in the quoted segment. By comparing the statement to the quoted text , the model should determine if the statement is indeed absent there.

Quoted Passage Named Entity Identification: Ask the model to extract and list all named entities (e.g., people , locations , organizations) exactly as they appear in the single quoted text. The correctness is judged by directly checking the text to see if any named entities are missed or incorrectly added.

Quoted Segment Key Fact Listing: Instruct the model to list out the key factual points explicitly mentioned in the single quoted text (e.g., important dates , statistics , proper names , etc.). The user then checks if each item in the model 's list appears verbatim (or unambiguously) in the passage , ensuring no extra or missing facts.

##### Task for Multi -Span Scenario #####

Multi -Source Information Comparison: Compare or contrast two or more separate quoted passages (e.g., comparing their data , opinions , or attributes). The user 's question focuses on differences or similarities across these quoted segments

Multi -Source Consolidated Summarization: Provide a unified summary that covers all key points from two or more quoted passages. The user wants an integrated overview of multiple fragments

Temporal (Time-Order) Reasoning: Determine or explain the chronological order of events or facts mentioned across multiple quoted segments. The user 's question involves identifying which event or statement occurred first or last

Contrasting Viewpoints Analysis: Identify differing viewpoints or stances across multiple quoted regions. The user wants to see how authors or speakers in those passages disagree or differ in perspective

Merged Key Point Listing: Extract the core points from each quoted passage and then compile them into a single combined list. The user wants a concise set of bullet points that captures all cited texts

Numerical/Data Comparison Across Sources: Identify and compare numerical values or data points mentioned in two or more quoted passages. The user 's question focuses on determining the highest , lowest , or most significant value among these sources.

Numerical/Data Sorting: Extract numerical values or data points from two or more quoted passages and arrange them in a specified order (e.g., from high to low or low to high). The user 's question focuses on presenting the data in a sorted list.

Multi -Source Condition Fulfillment: The user poses a set of conditions (e.g., "Condition A is stated in quotation 1, Condition B is in quotation 2"). Ask the model to determine if those conditions are satisfied or met collectively , based strictly on the referenced segments.

Multi -Passage/Sequence Reconstruction: If multiple quoted segments each describe different parts or phases of a single process or timeline , the user asks to arrange these parts in a coherent sequence or flow. Correctness can be verified by checking if each step appears in the right chronological or logical order as indicated by the individual passages.

##### Task for Exclude Scenario #####

Summarize After Ignoring Selected Passages: The user designates one or more passages to be "ignored." The task is to summarize only the remaining , non-ignored content. The answer must exclude or skip any information from the ignored portions.

Sensitive Information Hiding: The user selects private or confidential text or daat to be hidden. The final output must not reveal the ignored data. The user wants only the non-sensitive parts to be disclosed.

Partial Anonymization or Redaction: The user specifically wants certain fields or details (like names , addresses , IDs) to be redacted. The task is to remove or mask those sensitive elements while preserving the rest of the information.

Summarize Unignored Sections: The user designates one or more passages to be ignored. The model must produce a concise summary only of the content not in those ignored passages. The ignored content should not influence or appear in the summary.

Sort or Rank Unignored Data/Number: After ignoring specific segments , the user asks the model to sort or rank remaining items based on a certain criterion (e.g., numerical values , alphabetical order , etc.). Only the leftover textual details (not ignored) should be used to perform the sorting or ranking.

Extract Keywords from Non-Ignored Content: The user designates some passages to be ignored and then requests keyword extraction. The model must parse only the remaining (unignored) text to identify relevant or significant keywords without referencing any ignored sections.

Named Entity or Concept Extraction (Non-Ignored Only): The user designates specific passages to ignore. The model must identify named entities (people , locations , organizations , etc.) or key concepts only from the remaining text. Any details found in ignored content should be excluded.

Non-Ignored Outline Generation: The user instructs the model to create an outline (e.g., bullet points or a structured plan) of the leftover information after certain portions are ignored. The resulting outline must reflect only the visible (unignored) details.

Compare Remaining data/number After Ignoring: The user designates certain passages to be omitted. The question is then to compare the data/number in only the remaining information. The solution should not incorporate any data from the ignored content.

##### Task for Info -Combine Scenario #####

Reference Justification Extraction: Given a reference quotation that contains a recommendation or statement (e.g., "Then you should rest early") and a background quotation that provides the underlying context (e.g., "I am very tired from work today"), the model must extract and clearly justify the reasoning behind the reference. The answer should explain that the suggestion is made because of the background 's stated condition.

Price Comparison Based on Quoted Reference: When the reference quotation specifies a particular item (e.g., "Item B: 8 dollar") and background quotations list prices of other items (e.g., "Item A: 10 dollar" and "Item C: 5 dollar"), the model must compare these figures. The answer should identify which background item is more expensive than the referenced one, strictly based on the provided numerical data.

Temporal Sequence Reasoning with Context: With a reference quotation indicating an event with a specific date (e.g., "Event Y occurred on March 5th") and background quotations listing other events with their dates (e.g., "Event X occurred on March 1st" and "Event Z occurred on February 28th"), the model must determine the chronological relationship. The answer should identify which event happened before or after the reference event , solely using the provided dates.

Attribute Comparison Across Context and Reference: When the reference quotation provides a measurable attribute of an item (e.g., " Product Y has a rating of 4.0 stars") and background quotations offer similar attributes for other items (e.g., "Product X: 4.5 stars" and "Product Z: 4.8 stars"), the model must perform a direct comparison. The answer should specify which products have higher (or lower) values than the referenced product.

Contextual Cause -Effect Linkage: Given a reference quotation that states an effect or recommendation (e.g., "You should replace the cooling fan immediately") and a background quotation that provides a cause (e.g., "The device overheated due to prolonged usage"), the model must link the effect to its cause. The answer should clearly state that the recommendation is based on the background cause.

Contextual Selection Based on Criteria: Provided a reference quotation that highlights a particular preference or criterion (e.g., selecting an item with a specific price point) along with background quotations listing various items and their attributes , the model must select the items from the background that meet the criterion specified by the reference. The answer should list only those items that conform to the given requirement.

Direct Comparison with Context Integration: With a reference quotation that specifies a particular detail (e.g., "Item B costs 7.3 dollar") and background quotations that offer comparable details for other items (e.g., "Item A costs 20 dollar" and "Item C costs 2 dollar"), the model must directly compare these details. The answer should indicate which background item stands out relative to the reference.

Integrated Inference from Reference and Context: The model receives a reference quotation that implies a decision or inference (e.g., " You should leave earlier tomorrow") alongside background quotations that supply supporting context (e.g., "Traffic is usually heavy during rush hour"). The answer should integrate both to deduce the rationale behind the reference statement.

Contextual Elaboration of a Reference Statement: Given a reference quotation that provides a brief statement (e.g., "It would be wise to update the software") and a background quotation that offers additional detail (e.g., "The system has been experiencing bugs due to outdated software"), the model must elaborate on the reference. The answer should explain that the recommendation is based on the issues mentioned in the background.

Quantitative Comparison with Contextual Data: When a reference quotation contains a quantitative figure (e.g., "Service B costs 20 dollars") and background quotations list similar figures for other services (e.g., "Service A costs 25 dollars" and "Service C costs 15 dollars"), the model must perform a quantitative comparison. The answer should specify which background figure exceeds or falls below the reference figure.

Relative Ranking Determination: With a reference quotation providing a metric for an item (e.g., "Model Y has a battery life of 8 hours ") and background quotations offering comparable metrics for other models (e.g., "Model X: 10 hours" and "Model Z: 7 hours"), the model must rank the items. The answer should clearly state which models rank higher or lower than the referenced one based on the provided metrics.

Conditional Outcome Reasoning: Provided a reference quotation that states a conditional suggestion (e.g., "If you experience overheating , turn off the device immediately") along with background quotations that supply the condition (e.g., "The device 's temperature has been rising steadily"), the model must deduce the correct outcome. The answer should confirm the conditional recommendation based solely on the provided background.

Multiple Criteria Filtering: When a reference quotation emphasizes a specific criterion (e.g., "I prefer an affordable product") and background quotations list several items with various attributes (

e.g., price , quality), the model must filter the background to identify only those items meeting the reference criterion. The answer should list the items that match the affordability requirement as provided in the background.

Cross -Domain Integration: The model is given a reference quotation from one domain (e.g., "You should upgrade your hardware") alongside background quotations from a related domain that offer supporting technical details (e.g., "The current system has a processing speed of 1.2 GHz"). The answer should integrate both sets of information to justify the recommendation , strictly relying on the provided details.

Contradictory Evidence Resolution: Given a reference quotation presenting a definitive statement (e.g., "The meeting is scheduled for 3 PM") and background quotations that offer conflicting information (e.g., "The organizer mentioned a delay , setting the meeting at 4 PM"), the model must resolve the contradiction. The answer should clarify the correct detail by logically prioritizing the background evidence.

Reference -Context Synthesis for Decision Making: In this task , the reference quotation poses a decision prompt (e.g., "Should I choose Option B?") and background quotations offer supporting details (e.g., pros and cons or specifications of Options A and B) . The model must synthesize the information from both to recommend a decision. The answer should clearly state which option is preferable based solely on the integrated information from the reference and background quotations.

C Full Prompt for Consistency Evaluation

Prompt for Evaluating Base, Multi-Span, and Exclude Scenarios

You are a fair evaluator. You will be given two pieces of text: Model Answer and Ground Truth. Your task is to assess how accurately the Model Answer reflects the essential facts and conclusions of the Ground Truth. Use the following five -level scale to guide your decision , then provide your final evaluation as a single line in the format "Score X" with no additional explanation or content.

Score 1: The answer is overwhelmingly incorrect or contradicts the Ground Truth in most respects , showing fundamental misunderstandings or factual errors , so that little to no essential information aligns with the Ground Truth.

Score 2: The answer contains multiple significant inaccuracies or contradictions , offering only a limited amount of correct information , and overall demonstrates unreliable alignment with the Ground Truth despite some partial correctness.

Score 3: The answer demonstrates moderate alignment with the Ground Truth , correctly capturing certain important points while also containing noticeable errors or omissions , so that its overall accuracy is only partially reliable.

Score 4: The answer is largely accurate with only minor mistakes or overlooked details , consistently reflecting the Ground Truth 's main facts , thereby showing a good degree of correctness and trustworthiness.

Score 5: Answer used the same evidence as the Ground Truth and reached a conclusion identical to the Ground Truth.

After reviewing the Model Answer and comparing it to the Ground Truth , choose the best matching level from 1 to 5. Output your final rating as:

Score X

where X is the chosen number , and do not add any other text , marker or explanation in your response.

Prompt for Evaluating Info-Combine Scenarios

Prompt for Evaluating Coref Scenarios

D Examples for the input samples

D.1 Multi-Choice Question Examples

Base

{

"instruction": "I've been reading about how social contracts influence public welfare policies. Could you explain where this idea originated?
\n
assistant\ nCertainly! The idea of the social contract originates from philosophers like Thomas Hobbes , who argued that individuals agree to surrender some freedoms in exchange for security provided by the state. In 'Leviathan ,' Hobbes described life without such an agreement as 'solitary , poor , nasty , brutish , and short.' This foundational principle laid the groundwork for understanding the relationship between citizens and their governments.


\n
user\nThat's fascinating. Can you give me an example of how this might work in practice today?<|im_end |>\n
assistant\nSure! Imagine a country called Altruria , which recently implemented a universal basic income program. The rationale behind this decision was rooted in social contract theory -the government reasoned that providing financial stability would reduce societal unrest and foster trust among citizens. By ensuring everyone had a safety net, Altruria aimed to fulfill its part of the implicit agreement between rulers and ruled.
\n
user\nBut isn 't there always someone who disagrees with these kinds of agreements? How do we address those concerns?
\n<| im_start|>assistant\nAbsolutely valid point. Not all citizens may see eye-to-eye with the assumptions underlying social contract theory. For instance , critics argue that enforcing collective agreements can undermine individual autonomy. They believe people should have the right to opt out if they disagree with the terms imposed by the majority. This tension highlights the ongoing challenge of balancing communal obligations with personal freedoms within any society governed by social contracts.
\n


user\nBased solely on the information provided in the quotation. What does this passage suggest about the social contract?\nA. In the context of Altruria , the social contract involves the government providing financial stability through programs like universal basic income to reduce societal unrest and foster trust among citizens. It reflects an implicit agreement where the state fulfills its duty by ensuring a safety net for everyone.\nB. The social contract originates from philosophical ideas where individuals surrender some freedoms to the state in exchange for security , as argued by

Multi-Span

difficulty sleeping. This can significantly affect daily functioning and overall quality of life. Conversely , elevated dopamine levels can result in enhanced motivation and alertness. However , excessive dopamine activity has been linked to conditions like schizophrenia and addiction.

Therapeutically , selective serotonin reuptake inhibitors (SSRIs) are commonly used to treat depression by increasing serotonin availability. For conditions involving dopamine dysregulation , medications that modulate dopamine receptors are often prescribed.
\n
user\nUsing only the details given in the quotations , describe the primary effects of the neurotransmitter discussed and any associated risks or benefits mentioned.\nA. The neurotransmitter discussed is involved in reward -motivated behavior and pleasure centers in the brain.

Elevated levels of this neurotransmitter can result in enhanced motivation and alertness but also pose risks such as impulsive behavior , schizophrenia , and addiction.\nB. The neurotransmitter discussed plays a crucial role in stabilizing mood and promoting well -being. Therapeutically , selective serotonin reuptake inhibitors (SSRIs) are commonly used to treat depression by increasing the availability of this neurotransmitter , addressing issues related to low levels.\nC. The neurotransmitter discussed influences motivation , attention , and movement. For conditions involving dysregulation of this neurotransmitter , medications that modulate dopamine receptors are often prescribed to manage symptoms like excessive energy and focus , which can also contribute to impulsive behavior and conditions like schizophrenia and addiction.\nD. The quoted segments does not address or is insufficient to answer this question.\nPlease analyze the question first , then provide the answer in the format: 'The answer is X.'",

"input": "",

"output": "The analysis of the question involves carefully examining the quotations provided:\n\n1. The first quotation states that dopamine is involved in reward -motivated behavior , attention , motivation , movement , and pleasure centers. It specifically adds that high levels of dopamine can lead to increased energy and focus , but also impulsive behavior. No mention is made of schizophrenia or addiction in the quotations.\n \n2. The second quotation refers exclusively to serotonin and discusses its therapeutic modulation through selective serotonin reuptake inhibitors (SSRIs) for treating depression. It does not mention dopamine or any of its associated effects.\n\nNow, let's compare the options:\n\n-** Option A**: This describes dopamine in terms of reward motivated behavior and its potential effects , which partially match the quotations. However , it incorrectly adds conditions such as schizophrenia and addiction , which were not mentioned in the quotations. This makes Option A incorrect.\n\n-** Option B**: This describes serotonin and stabilizing mood , which matches the second quotation but refers to serotonin rather than dopamine. However , the question revolves around the neurotransmitter in the first quotation (dopamine), making Option B irrelevant.\n\n**Option C**: This accurately matches the first quotation by highlighting dopamine 's influence on motivation , attention , movement , and its association with symptoms like excessive energy , focus , and impulsive behavior. It also notes the use of medications to modulate dopamine receptors for conditions involving dysregulation of dopamine. This aligns completely with the given information , but it does not mention unrelated conditions like schizophrenia or addiction.\n\n**Option D**: The quoted segments are indeed sufficient to answer this

Exclude

Info-Combine

Coref

D.2 Open-Ended Question Examples

Base

Multi-Span

Exclude

Info-Combine

Coref

E Additional evaluations under complex scenarios

Cross-lingual use (Chinese, BASE ). We instantiate the pipeline in Chinese to test cross-lingual generalization under the same acceptance criteria as in the main data construction. Results (no language-specific training) are shown in Table 9.

Multi-turn quoting. We extend the BASE task to a multi-turn variant in which a sequence of questions each quotes a different span from the same dialogue. Results are shown in Table 10, indicating that both single and multi-turn quoting rely on the same span-awareness capability.

Mixed cases. Our benchmark already contains mixed-capability items that combine MULTI-SPAN with other behaviors. Specifically, EXCLUDE(MULTI-SPAN) accounts for 71.4% of the EXCLUDE set, and INFO-COMBINE(MULTI-SPAN) accounts for 24.4% of the INFO-COMBINE set. Evaluating these subsets separately shows that QUADA remains effective (Table 11).

F Experiment Details

Both training and inference were conducted on eight H20 GPUs (96 GB each). The inference temperature was set to 1.

Inference resources For Qwen-2.5-14B-Instruct, the baseline model consumed about 32 GB of memory per GPU with a per-sample latency of 0.87 s. Adding QUADA increased memory usage only slightly to 33.7 GB and latency to 1.45 s, indicating minimal overhead.

Training resources Training Qwen-2.5-3B-Instruct for three epochs completed in 1 h 25 min on the same 8×H20 setup, using roughly 28-30 GB of memory per GPU.

All human evaluators held at least a bachelor's degree and relevant research experience.

G Ethical considerations and recommended mitigations

Potential risks. Parameter-efficient adapters (e.g., LoRA; small bottleneck MLPs) have been shown to carry stealthy Trojans while preserving benign utility [Dong et al., 2023, Liu et al., 2024, Hubinger et al., 2024]. Since QUADA also employs lightweight bottleneck modules, two attack surfaces are relevant: (i) training-time poisoning of adapter weights, and (ii) inference-time manipulation of span inputs (e.g., injected or tampered spans), which could misdirect attention and amplify misinformation.

Recommended practices. The following measures are suggested for practitioners and downstream users to reduce risk while preserving utility:

Signed and reproducible releases Distribute adapter checkpoints with cryptographic hashes (e.g., SHA-256) and, when possible, detached signatures; record the exact backbone commit/version and provide deterministic export instructions so users can verify integrity prior to deployment.

Transparent data and pipeline Open-sourcing the data-construction pipeline, prompts, and the resulting training/benchmark sets enables third-party auditing for hidden triggers or biases; documenting provenance and filtering criteria further facilitates reproducibility and safety review.

Backdoor-aware training Incorporate adversarial training in future work to guard against backdoor attacks during training [Hubinger et al., 2024].

Secure span channel Transmit span annotations over authenticated, encrypted channels; add integrity checks and input sanitization at the boundary where spans enter the model; maintain audit logs to support incident response.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents an innovative approach to enhancing large language models for handling quotations in conversational contexts. It formalizes the challenge as span-conditioned generation, where each dialogue turn consists of a conversation history, a set of token-offset quotation spans, and an intent utterance. The authors break this problem into five diagnostic tasks—Base, Multi-Span, Exclude, Info-Combine, and Coref—to cover various quotation scenarios. They introduce QuAda, a lightweight, training-based method that integrates bottleneck projections into the LLM’s attention mechanism, enabling it to dynamically focus on or ignore quoted spans without altering the prompt and with minimal parameter overhead (less than 2.8% of backbone weights). Additionally, they develop a data pipeline to synthesize task-specific dialogues and create a benchmark for evaluation. Experiments on two LLMs (Qwen2.5-3B-Instruct and Llama3.1-8B-Instruct) show that QuAda outperforms baseline methods across all scenarios and generalizes to unseen topics, offering a practical solution for quotation-aware dialogue.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- This paper is well written and clear, with comprehensive experiments.\n- This work addresses a practical gap in human-AI interaction by formalizing quotation handling, which is quite novel and interesting.\n- The five diagnostic tasks provide a thorough and systematic way to assess quotation-handling capabilities.\n- The automated synthesis of high-quality training data and a validated benchmark is a significant contribution to the field.\n- QuAda’s lightweight design (less than 2.8% additional parameters, no prompt changes) makes it practical for real-world deployment.\n- The evaluation across multiple models, scenarios, and metrics, supported by attention visualizations, demonstrates the method’s effectiveness and interoperability.\n\nWeaknesses:\n\n- Limited Model Diversity: Testing is confined to two LLMs; broader testing across different architectures would bolster generalizability.\n- Interpretability vs. Controllability Trade-off: While attention maps (Figure 3) offer some interpretability, QUADA's bottleneck MLPs remain a black box. We don't know what \"guidance vectors\" (b^q and b^v) learn, making diagnosis difficult when the model fails. In contrast, methods like MARKER-INSERTION, despite poorer performance, are more transparent and controllable.\n- Scenario Coverage Limitations: This work claims its five diagnostic tasks span practical quoting behaviours, which can be a strong assertion. However, real-world references can also be composite, involving multiple types simultaneously (e.g., Multi-Span, Exclude, and Info-Combine). The paper lacks testing on the model's performance in these crucial mixed scenarios.\n\nQUESTIONS:\n- More Computational Details: Can the author provide more details, beyond what is mentioned in Appendix E, about (1) the training costs (memory usage and training time), and (2) the additional memory/latency overhead by QuAda during inference, for better reproducibility assessment?\n- Real-World Data: Have the authors considered testing QuAda on real-world conversational datasets rather than purely LLM-synthesized datasets? If not, what are the challenges?\n- Ethical Concerns: Are there potential negative societal impacts (e.g., misinformation amplification) to address? For example, QuAda employs simple bottleneck MLPs, which can fall victim to a backdoor attack. Including this could resolve an ethical gap, maintaining or increasing the current rating.\n\nLIMITATIONS:\nThe authors acknowledge limitations like reliance on synthetic data and the need for real-world validation, which is commendable. However, they do not address potential negative societal impacts. I suggest adding a paragraph discussing ethical concerns, such as the risk of manipulating conversations, to provide a more balanced perspective.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses a crucial problem in LLMs: the lack of an explicit mechanism to handle in-dialogue quotation. The authors formalise this task as span-conditioned generation and propose an automated data generation pipeline. In addition, the authors propose QuAda, a lightweight training method. By dynamically amplifying or suppressing attention to quoted spans during inference, it enables plug-and-play span-conditioned generation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The paper proposes an interesting and under-explored task—span-conditioned generation, which enables models to either refer to or exclude specific spans in dialogue history.\n\n2. The authors design a data pipeline that includes correctness verification and covers five well-defined quoting scenarios.\nThe proposed QuAda mechanism is simple yet effective. It enhances attention to relevant spans or suppresses attention to excluded ones via query and value bottleneck projections. The method is plug-and-play, does not require prompt engineering, and works across models.\n\n3. The authors plan to release the entire dataset, generation scripts, and training code, which will benefit follow-up research.\n\nWeaknesses\n\n1. While the five diagnostic scenarios are clear, they may not reflect the full diversity of real-world quoting behavior. The auto-generated dialogues also seem structurally simple—primarily single-turn question-answer pairs over selected spans—potentially limiting their contextual richness.\n\n2. The core approach—using MLPs to modulate attention scores—is reasonable but somewhat straightforward, offering only modest technical innovation beyond prior attention-steering work.\n\n3. In the current setup, the span and task type appear to need to be manually defined, which creates a gap between this method and real-world usage. The system does not yet solve the end-to-end problem exemplified by the motivating example in the abstract.\n\nQUESTIONS:\n1. In the example (“check it with the formula I just highlighted”), the user does not explicitly mark a span. Does QuAda rely on pre-specified spans and task types at inference time? This greatly affects practical applicability.\n\n2. In the ablation section (Table 7), only query-only, value-only, and full QuAda variants are reported. Could you add a “no-modulation” baseline, where none of the span-aware adapters are applied.\n\n3. The training data seems mostly constructed as “single-turn queries over quoted spans.” Are there multi-turn or nested quoting cases in the benchmark?\n\n4. Real dialogue often contains indirect references (e.g., “that thing we discussed earlier”). Have the authors considered testing QuAda on loosely specified quotation cases?\n\nLIMITATIONS:\nAuthors discussed limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis research aims to address the challenge that LLMs cannot accurately understand and utilize specific \"referenced\" text in the conversation history. When a user refers to a specific piece of text in a conversation, e.g., \"check it with the formula I just highlighted\", existing models lack a mechanism to accurately focus on that piece and act according to the instruction.\nTo study the problem, the author defines the problem as span-conditioned generation with a benchmark covering five scenarios. The author also claims the data annotation pipeline and training-based framework (QUADA) as a contribution.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe motivation behind the proposed task is clear and compelling. The benchmark contributes a useful testbed not only for evaluating quoting frameworks in real-world applications but also for probing LLMs’ ability to locate and recall information from longer contexts. The QUADA is well-reasoned, and the design is clean.\n\nWeaknesses:\n1. Although the dataset construction process involves multiple quality assurance steps, it remains synthetic in nature. That said, after manually reviewing several examples, I am generally positive about the data quality and find the contribution reasonable.\n2. While I agree with the significance of the task, I am concerned that the dataset may not be sufficiently challenging. Given the high performance reported in the experimental section, I question whether this benchmark can sustain long-term research value, especially as the language capability of foundation models keeps growing.\n\nQUESTIONS:\n1. Is there any failure analysis from the evaluation? Considering that QUADA achieves high scores, analyzing and attributing the small number of failure cases could help the understanding of the challenges (or lack thereof) in this task.\n2. As mentioned in the weaknesses, both the examples provided in the paper and the reported results raise concerns about the dataset’s difficulty. Have the authors evaluated stronger base models to better calibrate the benchmark? This could be either larger open-source LLMs or well-performing closed-source models.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWhen interacting with AI systems, users often want to specify which parts of the conversation history the model should focus on and how to use those selected spans. However, existing systems lack explicit mechanisms for this, typically requiring users to manually re-enter relevant text. The authors address this gap by first decomposing the quoting challenge into five diagnostic sub-tasks: base, multi-span, exclude, info-combine, and coreference. \nThe key-contribution is that the paper introduces QuAda (Quotation Adapter), a lightweight module that attaches two bottleneck projections to every attention head in the LLM. This design allows the model to dynamically amplify or suppress attention to quoted spans at inference time, without modifying the prompt and while updating less than 2.8% of the model parameters. The authors also present the end to end pipeline that automatically synthesizes task-specific dialogues, verifies answer correctness through multi-stage consistency checks, and produces both a heterogeneous training corpus and a benchmark. The evaluation demonstrates that QUADA outperforms existing methods such as repetition (concat-repeat), using markers (marker-insertion), and attention-steering across the five scenarios described and generalizes to across models (qwen-1.5b to qwen-14b and llama-3.1-8b).\n\nSTRENGTHS AND WEAKNESSES:\n[+] The paper presents, a novel problem formulation of formally defining span-conditioned generation for quoting in dialogue, capturing real-world conversational quoting behaviors through five diagnostic tasks (base, multi-span, exclude, info-combine, and coreference). If the claim on novelty is true, this is a very useful formulation to improve the model upon. \n\n[+] The proposed modeling technique- QUADA is lightweight, parameter-efficient (<2.8% of backbone weights), and does not require prompt modification, making it practical for easily adaptation. \n\n[+] QUADA generalizes across model scales (Figure 4) from 1.5B to 14B parameters on all five categories which is promising and I appreciate the small-scale scaling study. \n\n[-] The paper could provide deeper ablation studies to isolate the contributions of each component of QUADA (e.g., query side, value side) which will perhaps provide a better intuition as to contribution of each - similar to the intuition in section 4.3.\n\n[-] Although the parameter increase is small (<2.8%), the paper does not discuss inference speed or memory overhead in practical deployment, especially for very large models. Can you provide more details on the runtime and memory overhead introduced by QuAda?\n\nQUESTIONS:\nFrom the description of the QUADA architecture, the module is designed as a plug-and-play adapter that is agnostic to the specific sub-task as long as the task can be formulated as span-conditioned generation. However, I wonder what are the data, and performance implications of adding more sub-tasks (beyond the five described)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}