A$^2$Search: Ambiguity-Aware Question Answering with Reinforcement Learning
Authors: Fengji Zhang
Xinyao Niu
Chengyang Ying
Guancheng Lin
Zhongkai Hao
Zhou Fan
Chengen Huang
Jacky Keung
Bei Chen
Junyang Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-hop QA;Reinforcement Learning; GRPO; Large Language Model; LLM agent
Primary area: foundation or frontier models, including LLMs

Abstract: Recent advances in Large Language Models (LLMs) and Reinforcement Learning (RL) have led to strong performance in open-domain question answering (QA). However, existing models still struggle with questions that admit multiple valid answers. Standard QA benchmarks, which typically assume a single gold answer, overlook this reality and thus produce inappropriate training signals. Existing attempts to handle ambiguity often rely on costly manual annotation, which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue.
In this paper, we present A$^2$Search, an annotation-free, end-to-end training framework to recognize and handle ambiguity. At its core is an automated pipeline that detects ambiguous questions and gathers alternative answers via trajectory sampling and evidence verification. The model is then optimized with RL using a carefully designed $\mathrm{AnsF1}$ reward, which naturally accommodates multiple answers.
Experiments on eight open-domain QA benchmarks demonstrate that A$^2$Search achieves new state-of-the-art performance. With only a single rollout, A$^2$Search-7B yields an average $\mathrm{AnsF1}@1$ score of $48.4$% across four multi-hop benchmarks, outperforming all strong baselines, including the substantially larger ReSearch-32B ($46.2$%). Extensive analyses further show that A$^2$Search resolves ambiguity and generalizes across benchmarks, highlighting that embracing ambiguity is essential for building more reliable QA systems. Our code, data, and model weights can be found at https://github.com/zfj1998/A2Search.

Preprint
A SEARCH: AMBIGUITY-AWARE QUESTION ANSWERING WITH REINFORCEMENT LEARNING
Fengji Zhang', Xinyao Niu², Chengyang Ying', Guancheng Lin', Zhongkai Hao", Fan Zhou", Chengen Huang". Jacky Keung', Bei Chen", Junyang Lin" City University of Hong Kong. Alibaba Group, Tsinghua University
ABSTRACT
Recent advances in Large Language Models (LLMs) and Reinforcement Learning (RL) have led to strong performance in open-domain question answering (QA). However, existing models still struggle with questions that admit multiple valid answers. Standard QA benchmarks, which typically assume a single gold an- swer, overlook this reality and thus produce inappropriate training signals. Exist- ing attempts handle ambiguity often rely costly manual annotation, which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue. In this paper, we present A SEARCHH an annotation-free, end-to-end training frame- work to recognize and handle ambiguity. At its core is an automated pipeline that detects ambiguous questions and gathers alternative answers via trajectory sampling and evidence verification. The model is then optimized with RL us- ing a carefully designed AnsF1 reward, which naturally accommodates multiple answers. Experiments eight open-domain QA benchmarks demonstrate that A'SEARCH achieves new state-of-the-ar performance. With only a single rollout, A A'SEARCH-7BB yields average AnsFi@ score of 48.4% across four multi- hop benchmarks, outperforming all strong baselines, including the substantially larger ReSearch-32B (46.2%). Extensive analyses further show that A SEARCH resolves ambiguity and generalizes across benchmarks, highlighting that embrac- ing ambiguity is essential for building more reliable QA systems. Our code, data, and model weights can be found at https: //gi www.tttw..coooe.eeeeeeeotttttttt com/ wttcccooco/ocee /9999///////Sh...
INTRODUCTION
Open-domain Question Answering (QA) is a fundamental yet challenging task that requires both accurate reasoning and effective search (Rajpurkar et al., 2016; Reddy et al., 2019; Kwiatkowski et al., 2019). Recent advances in the ability of Large Language Models (LLMs) to use external tools (Yao et al., 2024), together with Reinforcement Learning (RL) techniques (Shao et al., 2024; Yu et al., 2025), have driven rapid progress in this area. Models such as Search-RI (Jin et al., 2025), ReSearch (Chen et al., 2025), and AFM (Li et al., 2025a) achieve strong performance by learning strategies for multi-step reasoning, active tool using, and precise evidence integration.
Yet the field has largely overlooked a pervasive source of difficulty: ambiguity in the questions themselves. Most QA benchmarks assume each question has a single correct answer, but in reality both annotation and real-world questions inevitably leave room for multiple, equally valid responses. This is especially evident in multi-hop questions, where different reasoning chains can legitimately reach different conclusions. Figure shows an example from the MuSiQue benchmark (Trivedi et al., 2022), produced by ReSearch-32B rollouts: distinct answers emerge, each well-supported by evidence and arguably correct. Nevertheless, the benchmark provides only one "gold" reference answer. Such cases are far from rare-our analysis finds that 27.6% 27.6% of MuSiQue's 's training examples admit more than one valid answer (see Section 4), and similar patterns occur in other QA datasets. We call the annotated gold the reference answer and the others alternative answers. Current RL pipelines, which reward only the reference and implicitly penalize alternatives, deliver misleading reward signals and systematically understate true model capability.
To address this challenge, models must learn not only to recognize when a question is ambiguous but also present all valid answers rather than commit single reasoning path. Evaluation protocols

---

Preprint
Who plays the legendary figure featured Historia Regum Britanniae the show Once Upon Time?
ReSearch-328 Multiple Rollouts
1a Rollout
<hinael need to ideniiyy the lependary figure from Historis Regum Britonnias and then find out who plays this character the show Once Upan a Time-/biin -hhink esearchrlogendory figure in Historta Regum Bnttannse-/search
Morutt_,Mrrlin is legendary figure_the charatter first aspears Beottrey Monmouthis "Histons Regum frtttniss" /neeuttt thine-Shhneeeen et Menill 5 the most notatle character-c
<gcanchowho plays merin once upon timey/earrch erseutt Allist Kright portrayed Merlin on the ABC fentasy televison tertes VOnce Upon Time" -/resutt Thinkk he information contems that Ellmt Kright playe
Hertin in the show Once Upon Time deink varawer- hh actor who playa Harlin in the show Once Upan a Time (Bored Knight
pa Rellout
question,
plays beginning preminent Historia Britannioo indicato Garrign rresull--Liam Gariiaan best (thnil sanowor- Looxed Sarrigan (onswerr
identiy legendary
Regum first
figure
Requm
mythical King figure.//esull Fram timer/scarcte
<thinlo "searchrwho
for his
Arthur
wwwesuit Thincchhe
To who legundary featured A'Search-78 Single Rollout
Brikannize ("name") "Wkkpddia, Histona Regum Britann features verious legendary Arthur and the others followed
Once feeerry: "ho iduniify lependary figure
fealured
<thinio
B-ilannas"))
<htoo_call- www._ccll-ttoorrooe respooss "Prophecies Morinn section
Incarntion.. htt.so/eessooneeeee -thinle
King arthur, Merlin We need identily which these
figures
featured
<thinle epplLcall- f (-Equery': "eeeeddry figure featured Historis
Time'll <host_cali> Comeict find the Sorcerer Merlin (Eliot Knight To complicate matters.
Brttenn
Once
<tool, resoo_responge "Once Upan Time (TV seriea" The characters emberk
forever atter the balance between light and dariness- respurespanee -think- is clear that the isgandary figures Merill Now need to denilly who plays King Arthur and Mertin the show. </tinkk <tool coll L-Pquery-: "who payss King Arthur Once Upon Time') shool call- -tool responsee
king Arthur (Jiam Garrigan) ls determined
featured Histeria Regum Britannias metus King
rispprrssposs- <sol.clli (-Pquery". "whl plays Martin in Once Upon Time') B <tool_call -teol, responnee -ttooLresgenns <answer [ "rationate" "Histoni Regum Britnnise tratres legendary figures such King Arthur and Merlin, and the show, King Arthur played Liam Garngan and Merlin is played by kibot Kright", "answere" I ('mmm Carrigan; "Enot [ainsei)
reference answer (provided In the benchmark)
atternative valid answer (supported by evidence)
Figure I: Rollout examples on an ambiguous question from MuSiQue. ReSearch yields different answers across rollouts, some diverging from the reference yet still sudeecc--supooree whereas A SEACHH explicitly resolves ambiguity by retrieving multiple answers within a single rollout.
likewise need to evolve to assess performance in genuinely multi-answer settings. In this paper, we propose SEARCH, an annotation- end-to-end reinforcement learning framework for training ambiguity-aware QA models. At its core is an evidence evidence-veriffcatie www.ivecatinn-baaaaeeeeaa data generation pipeline that automatically detects ambiguous questions and gathers alternative answers. The model is then trained with Group Relative Policy Optimization (GRPO), where outcome rewards are based on answer-level F1 (AnsF1), a metric that naturally accommodates multiple answers. By combining multi-step reasoning with tool use, A AESEARCH follows an agentic training paradigm that enables models to sense ambiguity and produce multiple answers whenever the evidence warrants it.
We comprehensively evaluate SEARCH on eight open-domain QA benchmarks, achieving com- parable or superior performance with only a single greedy decoding rollout, while prior methods typically require three sampled rollouts. On four multi-hop datasets, A A2SEARCH-7B yields an average AnsF1 @1 of 48.4% under Exact Match and 62.7% under LMJudge using just one roll- out, substantially outperforming ReSearch-32B (46.2% / 60.7%) and far exceeding ReSearch-7B (39.3% / 53.6%). Even the smaller A achieves competitive results (43.1% 55.3%), demonstrating the efficiency gains of training paradigm. On AmbigQA (Min et al., 2020), a human- annotated benchmark for ambiguous questions, A'SEARCH surpasses baseline models trained directly the curated AmbigQA training set, illustrating the robustness and transferability of our ambiguity-awar approach. The main contributions of this work are threefold:
We introduce a fully automated pipeline that identifies alternative answers for ambiguous questions via trajectory sampling and evidence-based verification.
We establish a stronger RL baseline for open-domain QA by training AE at 3B and 7B scales, achieving state-of-the-art results across eight benchmarks.
Through comprehensive analyses, we validate both the data pipeline and the RL paradigm, and show that A SEARCH Iearns to sense ambiguity and retrieve multiple answers.
2 RELATED WORK
Language models equipped with search tools have recently made rapid progress, enabling them to retrieve factual and real-time information through reasoning (Shen et al., 2023; Chang et al., 2024). Existing approaches can be broadly categorized into two main types: prompt- based and training- based methods. Prompt-based methods (Trivedi et al., 2023; Wang et al., 2024a; Yue et al., 2024; Li et al., 2025b; Alzubi et al., 2025) manually design prompts to guide LLMS in invoking search
2

---

Preprint
Step 1: Sampling
Step 2: Filtering
Matching Reference Answer Discard All
Natching Reference Answer Discard
Ressoning
No Matching Reference Answer Keep Distinct
Tool-Gall
Seerch-capable LLMs
Reason and Search
Trajectories
Step Verification
Step 4: Grouping
Took Res-Reespnnse
LLN-based Verifferrs
Merge Semantically identical Answers
Majority voting
Supported Evidencs Keep
Evidence Biscard
Figure 2: Our pipeline for automatically identifying alternative answers ambiguous questions.
tools and construct workflows for multi-turn tool usage. These approaches largely rely the inherent agentic capabilities of LLMs. In contrast, training-based methods adopt supervised fine- tuning (Wang al., 2024b) reinforcement learning (Chen al., 2025; Jin et al., 2025; Song et al., 2025; Sha et al., 2025; Fan et al., 2025) to improve search skills. In particular, reinforcement learning methods gradually enhance search performance by incorporating interactive feedback from the environment, which lays an important foundation for more complex search tasks in the future.
Multi-hop QA provides natural environment for training models' search and reasoning capabilities. Representative benchmarks (Joshi et al., 2017; Yang et al., 2018; Kwiatkowski et al., 2019; Ho et al., 2020; Trivedi et al., 2022; Press al., 2023; Shen et al., 2025) are constructed on Wikipedia, of- fering questions of sufficient difficulty to require multi-turn search, while the shared corpus ensures reproducibility and stable evaluation. Nevertheless, most of these benchmarks assume a single cor- rect answer per question, thereby overlooking ambiguity and cases with alternative answers. Some efforts, such as AmbigQA (Min et al., 2020) and ASQA (Stelmakh et al., 2022), address ambi- guity through manual re-annotation, but they rely heavily on human effort and are mainly limited to single-hop questions, making them difficult to scale or generalize to multi-hop training. Other studies focus on detecting ambiguity without providing alternative answers (Shi al., 2025; Kim et al., 2024). In contrast, A SEARA automatically recognizes ambiguous questions and generates multiple alternative answers, thereby benefiting end-to-end RL training.
METHODOLOGY
This section provides formal description of our proposed SEARCC Section 3.1 outlines the automatic pipeline for alternative answer generation, which provides the training data, while Sec- tion 3.2 presents the training method based on reinforcement learning.
3.1 ALTERNATIVE ANSWER GENERATION
Instead of relying manual annotation, we build automatic pipeline that exploits ambiguous questions in existing datasets to produce verifiable and effective training data for reinforcement learning. Formally, given question 9 and its reference answer ans", objective is to produce a set of alternative answers Aati that are semantically distinct from one another, different from the reference answer, and can be independently verified. To achieve this, we carefully design four- step process that is evidence-based and highly reliable, as illustrated in Figure 2. It first performs Sampling collect multiple automatically generated trajectories. then applies Filtering and Veri- fication, and finally conducts Grouping to obtain alternative answers. Details are provided below.
Step 1: Sampling. We employ a collection of N search-capable language models, denoted S = (S, J. each trained on single-answer QA datasets and equipped with the ability to interact with search tool. Given a question q, every model S,, produces M trajectories T, {Trm) J-M A trajectory is represented as r (a1, o1..... ar), where a; denotes an action and o, the content returned by the tool. Actions
of three types: (i) reasoning, recording intermediate thinking steps: (ii) tool-call, which issues a search query and receives a corresponding tool-response; or (iii) answer, which outputs final answer. Only the type tool-call yields returned content. At the end of
3

---

Preprint
this step, all trajectories for question g are aggregated into T() Un Tro where each T(1) corresponds to a candidate answer.
Step 2: Filtering. Not all the N x M candidate answers obtained from the trajectories in T are useful. useful trajectory should provide an alternative answer that is valid but different from the reference answer ans". We therefore perform coarse filtering using three intuitive rules. First, trajectories with answers judged by the LLM to be semantically equivalent to ans" are removed. Second, suppose all answers in T, differ from ans". In that case, we drop all trajectories from Tu. as this indicates that model S, is unable to produce the reference answer even with multiple rollouts, suggesting lack of capability
solve the question. Third, for trajectories that produce exactly identical answers, we keep only one representative trajectory. The resulting filtered set is denoted by (2), where each r E (2) provides a distinct candidate answer.
Step 3: Verification. For the filtered set ((2), we perform a fine-grained verification to determine whether each trajectory T provides sufficient evidence to support its candidate answer ans. We employ K number of LLM-based verifiers, denoted as takes (4, añs) as input and outputs binary judgment 2µ E 10. A value of 2 = 1 indicates that the trajectory contains sufficient evidence to support the candidate answer as a valid alternative answer; otherwise, 2 O. We aggregate the results using majority voting:
{V*] where each verifier V
Verify T, añs) = 11,
=f1
if K E 2k 12x20 1
0, otherwise,
where v) is the voting threshold. For each T perform verification and retain those with Verify (q. T, añts) = 1, resulting in the verified trajectory set (3))
Step 4: Grouping. Finally, we apply a clustering procedure to the verified trajectory set T33), using an LLM to merge semantically equivalent answers into groups. The final alternative answer set is then given by Aut Group(7(()), where Group(.) denotes a semantic clustering operator that groups semantically equivalent candidates and selects one representative alternative answer per cluster, retaining the others as aliases. Typical cases of semantic equivalence include abbreviation versus full name (c.g., ["NDZ", "Nkosazana Dlamini-Zumal "J) different numeric representations (e.g.. ["five", "5"[), and variations in word order (e.g., ['2000 fiscal year", "fiscal year "]O1"]).
3.2 REINFORCHMENT LEARNING FRAMEWORK.
Through the above pipeline, obtain the training data by extending the reference answer set with mined alternative answers, denoted as A = {ans" , This extension allows some questions to have multiple reference answers. We then design a reinforcement learning algorithm that uses an answer-level Fl reward, which is suitable for scenarios involving multiple reference answers.
Training Objective. We adopt Group Relative Policy Optimization (GRPO) (Shao et al., 2024) as the reinforcement learning algorithm. Unlike Proximal Policy Optimization (Schulman et al., 2017), which relies a separately trained critic network to provide baseline, GRPO estimates the baseline directly from a group of sampled rollouts. Concretely, given an existing policy Told- generate G rollouts { )=1 <- for each input ~ D. Following He et al. (2025) and Yu et al. (2025), we discard the KL penalty term. The optimization objective is then to update the policy me by maximizing
min To To (r) Too. (y)
To (v; (x:) To.. (ya (z)'' 1 e, c, (6) A. ).
[(0) EEND, me (1z)
Σ
A,, clip
G
F=1
) is the normalized advantage of the i-th rollout in the group, is the reward, and is the clipping ratio.
where A; = (r; (ri-mean(1) (r)));=1))) (srd((r,))))(()sss( (r,((r)))1))
Rollout with Search Tool. We formulate rollout generation an iterative interaction between the policy and a search tool. Given a question the model constructs a trajectory T = (a1, .1,..., or) consisting of alternating actions d and tool responses o, (with o, empty unless a is a tool call). At each step t, the state s, is the accumulated prompt containing the question, all past actions, and any returned responses. The policy my samples an action a, (T(-(ss) from three types: reasoning.
4

---

Preprint
tool-call (which returns tool-response); and answer, described in Section 3.1. The rollout terminates once an end-of-sequence token is generated or a maximum length is reached. Each trajectory, therefore, encapsulates the model'ss reasoning, search interactions, and final prediction, and serves as the basic unit for estimating the rewards used in reinforcement learning. During training, tokens in tool-response blocks are excluded from the policy loss via masking, since they are generated by the external search tool rather than by the policy model itself.
Reward Design. We employ an outcome-only reward design, which has been proven successful in recent studies (Guo et al., 2025; Yu al., 2025). The reward combines a format check with an answer-matching score. An output is considered format-valid if it satisfies all of the following: (i) containing at least one successful tool call, (ii) including intermediate reasoning blocks, and (ii) terminating with ond-of-sequence token after exactly one answer block whose content can be correctly parsed. Outputs failing any of these criteria receive a reward O. If the format is valid but assign a small constant reward of 0.1. Otherwise,
none of the reference answers are matched, we compute an AnsP1 (Answer-level F1) score based on exact match. AnsF1 rewards coverage of valid answers while penalizing over-generation, balancing precision and recall, and remaining comparable across questions with varying levels of ambiguity.
For answer matching, we define three quantities: preds denotes the total number of answers pro- duced by the model rollout; hits denotes the number of reference answers exactly matched by the predictions; and refs denotes the total number of reference answers. We define precision as Precision = /preds and recall as Recall = hits hits/rcfs, with AnsFl = 2 Precision Recall/ (Precision + Recall). The final reward is then defined as
0,
if format invalid,
R(g.ans)
0.1,
if format valid and hits 0,
11-a a (1 AnsF1), if format valid and hits >> O,
where a E [0, 1] controls the relative margin between format-valid but fully incorrect predictions and partially correct ones, and añs denotes the predicted answer set of a trajectory, which may contain multiple predicted answers. This design ensures that the model is encouraged to follow the required format, produce valid alternative answers, and cover as many reference answers as possible.
4 TRAINING DATA CONSTRUCTION
In this section, we present the implementation details of training data construction, based on the automatic pipeline described in Section 3.1. As result, we identified alternative answers for 19.0% of the 49,938 questions through 19,529 trajectories.
Step 1: Sampling. To encourage diversity in the sampled trajectories, we employ five distinct search models: ReSearch-7B/32B and Scarch-R1-7B/14BB322B These models achieve state-of-the art performance on open-domain QA benchmarks and, importantly, are able to produce search tra- jectories, which we later leverage for evidence-based verification. For each question, we generate 16 trajectories from each model using a sampling temperature of O.S. We utilize the same search tool as introduced in Search-R1, where the 2018 Wikipedia dump is partitioned into 100-word chunks, embedded with E5 (Wang et al., 2022), and indexed using FAISS (Douze et al., 2024). query time, the retriever returns the top-5 passages ranked by embedding similarity. The source questions are drawn from the full training splits of two multi-hop QA datasets, MuSiQue with 19,938 ques- tions and 2Wiki (Ho al., 2020) with 15,000 questions, together with 15,000 randomly sampled questions from the single-hop dataset NQ (Kwiatkowski et al., 2019). This setup yields around 3.99 million trajectories across 19,938 questions. Despite the scale, the sampling process remains efficient and computationally practical. Additional details on runtime and computational cost are provided in Appendix B.I.
Step 2: Filtering. We employ Bwven..5322B-Insrrue (Yang et al., 2024) as an automatic judge, using the prompt provided in Appendix G.1 to determine whether a trajectory's 's predicted answer is semantically equivalent to the reference answer. The evaluation shows that 86.8% of the ques- tions contain at least trajectory matching the reference answer, demonstrating the high quality of the sampled trajectories. After applying the filtering rules defined in Section 3.1 and performing answer-level deduplication, 208, 829 trajectories are retained, accounting for 5. 5.2% of the original
5

---

Preprint
3.99 million. These trajectories span 33, 997 questions, covering 68.1% of the 49,938 source ques- tions. On average, 6.1 distinct trajectories remain per question, which constitute the input to the verification stage. Further details and statistics of the filtering step provided in Appendix B.2.
Step 3: Verification. To determine whether candidate answers supported by evidence, four proprietary LLMs as verifiers (Claude 3.5 Sonnet, Claude 3.7 Sonnet, OpenAl o3, and OpenAl 04-mini) with the prompt provided in Appendix G.2. Each verifier assigns one of three labels to a trajectory: supported. partially supported. or not supported. The intermediate category prevents borderline cases from being overly judged as supported. thereby improving robustness. For aggrega- tion, we retain only the supported trajectories and apply majority voting with threshold We further study how the choice of 17 affects reliability by conducting an ablation with human evaluation: for each threshold, we randomly sample 100 positively voted answers and measure the human agree- ment rate. As expected, stricter thresholds improve agreement but substantially reduce the number of remaining trajectories. We therefore adopt 1) = 3. which achieves 96% human agreement while maintaining adequate coverage, leaving 19,529 trajectories (9.4% of those from the previous step). Further details and statistics are provided in Appendix B.3, and Appendix B.4 presents additional study showing that a fully open-weight verifier ensemble can closely approximate the judgments of the proprietary verifiers.
Step 4:
Grouping. Model-generated answers often differ lexi-
100
cally while being semantically identical. To consolidate such vari- ants, we apply Claude 3.7 Sonnet with prompt Appendix G.3, which groups semantically equivalent answers into clusters and as- signs each cluster a canonical form with aliases. Overall, 28.6% of candidate answers are grouped in this way.
90
Detaset
MusiQue
2W
NO
Percentage
40
We then construct the final training dataset. Figure reports the dis- tribution of answer multiplicity across datasets. While most ques- tions remain associated with a single reference answer, substantial portion contains multiple alternative answers. MuSiQue shows the highest ambiguity ratio, with 5,498 questions (27.6%) containing alternative answers, compared to 1, 076 questions (7.2%) in 2Wiki and 2,899 questions (19.3%) in NQ.
20
10
0
5+
Answer Count
Figure 3: Answer count dis- tribution in the final dataset.
5 EXPERIMENTS
Following the procedure in Section 4, we construct training dataset containing 49, 938 questions, among which approximately 19% have alternative answers. On top of this dataset, we employ the RL framework described in Section 3.2 to perform end-to-end training and obtain our ambiguity- aware model, A SEARCH. We then validate its effectiveness through systematic comparisons with diverse set of baselines. The experimental setup is presented in Section 5.1, while results and analyses are provided in Section 5.2. Overall, the experiments demonstrate that our data construc- tion pipeline combined with RL training is highly effective: AASEARCH resolve ambiguous questions by identifying multiple answers and outperforms baseline methods.
5.1 EXPERIMENTAL SETUP
Benchmarks. We evaluate our ambiguity-aware training on eight open-domain QA benchmarks. The multi-hop setting is represented by MuSiQue (2, 417 questions), HotpotQA (7, 405) (Yang et al., 2018), 2Wiki (12, 576), and Bamboogle (125) (Press et al., 2023). For general open-domain QA. we use NQ (8, 757), TriviaQA (8, 837) (Joshi et al., 2017), and PopQA (14, 267) (Mallen et al., 2023). Finally, we include AmbigQA (2, 002) (Min et al., 2020). a variant of NQ augmented with human-annotate alternative answers for ambiguous questions, averaging 2.1 answers per question.
Baselines. We compare our method against four categories of baselines. (1) Prompt-based meth- ods. These methods involve no model training: DirectGen simply prompts the model to answer the question. Naive-RAG retrieves passages in single round, concatenates them with the question, and asks the model to generate an answer. Iter-RetGen (Shao et al., 2023) extends this by performing three fixed rounds of retrieval and generation, where each retrieval step can use previously generated content. IRCoT (Trivedi et al., 2023) further integrates retrieval into chain-of-thouge reasoning, al- lowing arbitrary iterations. (2) RL-trained search models. We include Search-R1 (Jin et al., 2025),

---

Preprint
ReSearch (Chen et al., 2025), and AFM-MHQ (Li al., 2025a), all trained with reinforcement learning in an agentic fashion. (3) SinSearch. This baseline is trained on the same set of questions as A SEARCH, but it relies solely on the original single reference answer provided for each ques- tion in the datasets. The training prompt for this model is provided in Appendix G.4. (4) AbgSearch. This baseline is trained on the AmbigQA dataset (10, 036 questions) using the same training setup as A SEARCH. Because AmbigQA primarily contains simpler single-hop questions, this comparison underscores the importance of constructing multi- datasets for the more challenging multi- hop setting. The prompt for this model is in Appendix G.5.
Evaluation Metrics. Two primary metrics are used: An)F and Recall(@1), 1). As described in Section 3.2, they are computed from hits, preds, and refs. To obtain hits, we adopt two complemen- tary schemes: Exact Match, which checks whether prediction exactly matches a reference answer or one of its aliases, and LMJudge, implemented with Owen2.5-32B-Bn --Istrutt using the prompt in Appendix G.I. We additionally report AnsF1 @& and Recall@k, where k denotes the number of sampled trajectories. To reduce the effect of sampling randomness, we approximate the expected value of @kk (k > by repeatedly subsampling & items from k' (k' k) sampled trajectories, with the detailed algorithm provided in Appendix D.I.
Hy peperpaarmeters. For fair comparison with prior work, we use the Qwen2.5 model fam- ily (Yang et al., 2024), experimenting with the 3B, 7B, Base, and Instruct variants to examine the generalization capability of our training framework. Training prompts provided in Appendix G.5 and G.6. We train with a batch size of 256, learning rate le- 6, maximum context length 8, 89192, roll- size 16, and 4 epochs. The parameter in reward design is set 0.4. We hold out 512 randomly sampled questions from the MuSiQue development set for hyperparameter tuning and checkpoint selection. Ablation studies on a and rollout size are reported in Appendix E. and E.2. For eval- uation, AnsF'I 1asF11@11 is computed with greedy decoding, and Ans @3 is estimated by generating 6 rollouts for single-answer search models using a sampling temperature of 0.6. Further analysis of sampling temperature effects is provided in Appendix E.3.
5.2 EXPERIMENTAL RESULTS
Main Evaluation Results. Table 1 and 2 report the overall performance on eight QA benchmarks under the Exact Match metric. Additional results evaluated with the LMJudge are provided in Ap- pendix E.4. Both evaluation metrics yield consistent findings, which we summarize as follows:
(1) Agentic search models consistently outperform prompt-based and RAGebased baselines, with especially clear gains multi-hop benchmarks. In addition, their Recall@3 is substantially higher than Recall@1 across all model sizes, indicating notable headroom unlocked by modest sampling- (2) Even with a single greedy decoding rollout, A SEARCC reaches recall levels comparable to or surpassing the @3 performance of baselines, and even outperforms larger 32B models on several multi-hop benchmarks. While SinSearch, trained under the same setup but without alternative an- swers, performs competitively against other baselines, it still falls short of A SEARCC.. Moreover, S achieves the best AnsF1 on most benchmarks, striking a strong precision-recee recall bal- ance. On average, A SEARCCI17 generates 1.51 answers per question and A AEAARCHH gener- ates 1.23, with detailed benchmarcrkkkk statistics provided in Appendix C. (3) AbgSearch performs well on AmbigQA but fails to generalize to other datasets. In contrast, A SEERACCH is trained without AmbigQA data and still surpasses on the same benchmark. This result highlights the effectiveness of our evidence- based data generation pipeline.
Training Dynamics and Stability. Throughout the four training epochs, both the 3B and 7B mod- els exhibit stable and consistent improvements. As shown in Figure 4, AnsF1 and Recall steadily increase without signs of collapse or instability, indicating reliable training dynamics. Moreover, the entropy curves indicate that the models do not exhibit premature entropy collapse. The 3B model maintains a lower entropy than the 7B model, fluctuating around 0.2, while the 7B model remains within a healthy range of variation, suggesting that it preserves sufficient diversity in training.
Additional Analyses. We conduct several complementary analyses, with full details provided in the appendix. (1) Our framework generalizes successfully to Qwen2.5-Base models, where it achieves strong recall at the @3 level with single rollout (Appendix A). (2) Our framework also generalizes to Llama-based models, where it exhibits similar performance trends and demonstrates

---

Preprint
Model
HotpotQA
2Wiki
MuSiQue
Bamboogle
Macro-Avg
AnsP1/Rocaliak
813
1
C13
03
Models with 3B Parameters
DirectGen-3B
15.9
16.2 /17
24.8
25.3 f
2.3
2.4/
2.4
3.1 As.
11.3
11.8 /1a.1
Naive-RAG-3B
28.2
28.3 /20.3
24.7
25.2
/26.7
5.7
5.7
9.6
9.7
17.1
17.2/17a
Iee-RReGGen-3B
30.1
30.8
26.0
26.9
7s3
11.2
/234/138
18.6
/9.4
IRCoT-3B
27.6
29.2 fan.7
21.4
24.8 fsc.
7.7 /106
20.8
22.6
19.2
21.1/26.4
Search-RI-33
37.0
38.2 /n.4
39.7
42.3 An.
15.1
16.3 /189
36.8
35.7
/sso
32.2
33.1 /96.6
AFM-MHQ-3B
41.5
43.0 /s1.
43.6
46.9 /ssu
17.5
18.9 /25.4
39.2
40.6
/ooa
35.5
37.4/46 /46.6
SinSearch-3B
37.9
41.1 /am.
47.3
50.8
19.5
20.5 /25.6
38.4
38.2
35.8
37.7 /3.2
AbgScarch-3B
28.3 /s1.4
28.7 /518
8.94 Jo.ss
20.9 /21.6
21.7 /24.4
ARSEARCH-3B
42.8 /44
56.2 /s8.a
24.2 /25.9
49.3 /5n.4
43.1 /012
Models with 7
32B
Parameters
DireetGen-7B
19.3
19.5 /11.1
25.5
26.8 /sn.2
3.8
4.1 /L9
10.4
10.9 /120
14.8
15.3 /1r.1
Naive-RAG-7B
31.9
32.1 /aa.
25.9
26.0 /21.1
6.3
6.4 /s.
20,8
20.7 /229
21.2
21. /22.
Ree--ReGGnn-B
34.3
35.0 /an.8
28.0
28.9
/s1.2
8.8
9.2 /10.4
21.6
21.6 /23
23.2
23.7 /
IRCoT-7B
30.3
32.9 fav.
21.7
24.3
fas.1
7.4
7.9 /a0.8
24.0
24.3 /ao2
20.9
22.4/2s
ReSearch-7B
43.2
45.6 /1.9
47.2
51.4 /61.4
22.6
24.9 fa1.
44.0
46.5
/sa.
39.3
42.1 /49.5
Search-R 1--7
43.2
44.5 Am
39.8
42.3 /mz
20.0
20.7 fas
42.4
41.9 fss
36.4
37.4 /11.2
AFM-MHQ-7B
46.1
47.6 /ea.
46.2
48.9
fsso
20.5
21.5 f27.4
43.2
46.3
fass
39.0
41.1 /88.2
Search-R1 R1-14B
47.5
47.9
/s1.6
48.1
50.0
fss.
25.4
26.7 /31.
51.2
51.2 /s3.
43.1
44.0 /27.9
Search-R -1-2BB
46.2
47.8 (s1.9
51.0
53.5 /m1.1
25.1
26.3 F3LA
53.6
55.8 f61.
44.0
45.9 /51.
ReSearch-32B
46.6
49.4 /s4.1
53.0
57.9 /66.7
26.0
28.6 /s4.3
59.2
39.1 /64.4
46.2
48.8 /549
SinSearch-77
45.6
46.9 fs0.3
57.6
59.5 fet.1
25.4
27.0
50.6 /ss.
44.4
46.0/49.
AbgScarch-7B
39.2 /43.7
35.5
15.9/19.0
33.7 /36.2
31.1 1949
A SEARCHH-7
49.5 1s2.1
62.3 fes.
30.1 /34.8
51.7 Asau
48.4 /s1.2
Table I: Main results on four multi-hop QA benchmarks under the Exact Match metric. We report AnsF1 /Reeall@kk with rollouts. For AbgSearch and SEARC,,, only @1 is reported, reflecting their ability to produce multiple answers within a single rollout. For the remaining baselines, where each rollout generates only one answer and thus AnsF1@1 AnsF1/ AnsF1//Reallll @2 to evaluate their performance when more rollouts available. The best result in each comparison group is shown in bold, and the second best is underlined.
Recall@1, we additionally include
0.6
0.3
Ererasy
0.8
0.4
valus
0.22
Entragr
0.2
0.4
70 Are
Ane?
0.0
70 Rocal
0.0
Recal
400
600
500
800
200
800
Step
(a) AnsFl 1 1/Recall on train set
(b) AnsF1/Reeaa on val set
(c) Entropy during training
Figure 4: Training dynamics of A SEARCH. Curves are smoothed for readability.
that the ambiguity-aware RL training is backbone-agnostic and broadly applicable across model families (Appendix E.5). (3) We analyze rollout efficiency by measuring both the number of tool calls and the corresponding recall. We observe that A SEARCH reaches the @3 recall level with fewer tool calls on average (2.16 for 3B and 4.14 for 7B), demonstrating more effective utilization of reasoning steps (Appendix C.2). (4) To illustrate the prevalence of ambiguity in evaluation bench- marks, we apply the verification steps from Section 4 to trajectories sampled from baseline models, and find that non-trivial portion of questions admit alternative answers (e.g., 19.79 19.7% in MuSiQue, 11.1% in HotpotQA, 8.6% in 2Wiki, and 8% in Bamboogle; More details in Appendix C.3). (5) We further train three additional ablation models to disentangle the contributions of ambiguity-awe aware data construction and the AnsF1 reward. The results show that neither component alone suffi- cient. Full details provided in Appendix E.6.
8

---

Preprint
Model
NQ
TriviaQA
PopQA
AmbigQA
Macro-Avg
AnsP1/Recaliak
@1
333
813
81
03
143
Models with 3B Parameters
DirectGen-3B
11.4
11.7 /1.6
32.9
33.3 /1r
13.0 13.1 /14.7 10.8 /.a
11.6 /1.2
17.0 f00.7
17.4 /19.1
Naive-RAG-3B
38.2
38,8
/uo3
57.0
41.4
41.7 /42. 36.6 faLs
37.3 /22.
43.3 /42.1
43.8 /43.7
Ieer-Re
39.2
39.9
58.8
59.3 fo1.1
43.9
44.3
38.4 /35.1
39.3
45.1 /ss.
45.7 fs6.
IRCoT-3B
25.3
27.6 /s49
45.2
48.0 /r63
33.8
36.6
27.6 f20. 31.3
33.0 /m2
35.9 /41.5
Search-R1-3B
43.9
44.6
60.1
60.8 /63.0
46.5
47.0
/18.4 38.7
fan.s
39.8 /ss.5
47.3
/u6.0
48.0 /s8.4
AFM-MHQ-3B
38.3
38.9
/18.2
58.1
59.0 /or
37.8
39.2 /1n.2 36.4
/31.6 39.0
42.6
/11.5
44.0 /sa.6
SinSearch-3B
40.9
43.3 /ou2
58.0
59.9 /64.9
43.7
45.0 /wa
38.2
fa2
41.2
/3s.6
45.2
/4L9
47.4 /to.2
AbySearch-3B
41.3 3
/st.
39.3 /42.6
40.4 /36.6
44.0 /45.4
ASSEARCH-3B
47.3 /49.7
60.9 /02.5
48.2 /siLs
43.1 /ss.2
49.9 /sn.2
Models with 7 ~ 32B Parameters
DireciGen-7B
14.3
15.0 /16.
44.3
44.7 /ar
15.2 15.5 /mn 14.1 /12 14.9 /14.0
22.0 f21.5
22.5 /23
Naive-RAG-7B
38.7
38.7 /s0.
61.0
61.4 f02.
40.1
40.2 /1.1 37.8 /3s.1
37.8 /ss.
44.4
/18.2
44.5 /61.3
Iter-RelGen-7B
40.4
40.6 /42.
62.6
63.3 /64.7
42.8
43.4
/46.2
39.5 faa.5
40.3
46.3
/45.1
46.3 /47.1
IRCoT-7B
25.9
27.5 IsL
53.7
55.0 feo.s
34.3
36.0
/m.2
27.8 /as.
30.0
35.4 /s4.0
37.1 fa1.7
ReScarch-7B
42.4
44.5 /so.
63.1
65.3 Fro.
44.7
46.7
/s2.
40.8
fan.4
45.3 142.8
47.8 /16.4
50.5 /st.1
Search-R1-7B
47.7
48, 4 /to.2
64.0
64.6 fes.5
46.1
46.9
/48.4
41.8
fa6.9
43.1
/38.2
49.9 /48.5
50.8 /so.s
AFM-MHQQ-7B
44.4
46.5 /s3.7
64.4
66.2 /11.3
43.3
45.8
41.1
135.5
44.5
/42.1
48.3 /16. 50.8 f5t.8
Scarch-R1-144
50.1
50.4
67.0
67.7 /T14
49.6
50.3
43.8
fa7.8
45.2
/.0.5
52.6 /s1.1 53.4 /5L.
Search-R1-32B
49.1
50.3
/sa.
70.0
71.1 fss.0
49.6
51.0 /4488 44.3
fa8.3
46.4
/12.0
53.2 /SL.8 54.7 /56.1
ReSearch-32B
43.0
46.8 I51.
67.7
70.4 /14.3
48.2
51.3
/56.
44.1
/38.2
/44.1 47.8
50.8 /19.3 54.1 /56.5
SinSearch-7B
49.3
49.8
JsLa
66.2
67.0 /va.2
50.5
51.4
/sann
44.6
/08.4
45.1
/3n.8
52.6 /s1.1 53.3 /oa.
AbySearcch-B
47.6
64.8 /68.2
48.0
/332
47.5
/13.0
52.0 /54.
A SEARRCH-7
51.4 I54.7
67.8 /ea.s
52.5 /s5.9
48.1 /us.2
55.0 /s5.8
Table 2: Main results with the Exact Match metric on four general QA benchmarks, using the same notations as Table 1. For AmbigQA, where questions may have multiple reference answers, AnsFl@)
@1 I and Recall@1 I are not equivalent in this setting, and both are therefore reported.
Ambiguity Type
Definition
Question lacks critical contextual constraints, allowing multiple distinct entities to satisfy the query. Specificity level (e g., spatial, temporal) not stated, yielding answers valid at different granularities. Correct answer depends on an implicit temporal reference point.
Granularity Ambiguity
Time Sensitivity
Evidence Conflict
Retrieved passages provide contradictory factual claims about the same catity.
Multi-liem Response
Question expects single unswer but the true
non-singleton without selection criterion. Question subjective or interpretive, supporting multiple qualitatively valid responses.
Open Ended
Alias Variance
Different surface forms refer the same underlying real-world entity.
Table 3: Ambiguity taxonomy and definitions. Examples of these types are listed in Table 12.
5.3 UNDERSTANDING THE SOURCES OF AMBIGUITY
To analyze the underlying causes of ambiguity in QA datasets and understand why multiple valid answers emerge, we conduct a systematic analysis of the underlying ambiguity types present in our training data. Through manual inspection of the multi-answer portion of our constructed dataset, we first identified seven recurring categories of ambiguity that account for the vast majority of cases. The resulting taxonomy and its definitions are shown Table 3. We then use a strong LLM judge (OpenAl o3, full prompt in Appendix G.8) to automatically assign an ambiguity type to each am- biguous question in the dataset. For every question that contains multiple valid answers, the judge is provided with the question as well as multiple evidence-supported execution trajectories that lead to different valid answers. The judge is asked to compare these trajectories, infer the source of ambiguity, and select the primary ambiguity category.
In terms of prevalence, Under-Cont Constrained questions dominate the ambiguous portion of the dataset (52%), followed by Granularity Ambiguity (34%). Time Sensitivity (5%) and Evidence Conflict (4%) occur far less frequently, while Multi-Item Response accounts for 3%. Truly Open-Ended Ended questions and Alias Variance each constitute only about 1% of cases, indicating that most ambigu- ity arises from meaningful semantic underspecification rather than superficial variation. To more intuitively illustrate how A SEARCH identifies and resolves genuine ambiguity, we provide addi-
9

---

Preprint
tional representative ambiguous questions together with the full reasoning and-search trajectories in Appendix H.
CONCLUSION
In this work, we revisited the challenge of ambiguity in open-domain QA, pervasive yet underex- plored issue. We proposed SESRARCH an annotation-free RL framework that automatically iden- tifies ambiguous questions, discovers alternative answers through trajectory sampling and evidence verification, and optimizes models with an answer-level reward that naturally accommodates multiple references. Our experiments confirm that A SEARCH attains state-of-the- results with single rollout and generalizes effectively to AmbigQA, demonstrating both scalability and robust- ness. Analyses further reveal that the model acquires the ability detect ambiguity and generate multiple valid answers. These findings suggest that real progress in QA requires explicitly embrac- ing ambiguity. By treating alternative answers as first-class signals, models become more reliable and better aligned with human expectations. We expect this perspective to inform future evaluation protocols and extend naturally broader domains.
ETHICS STATEMENT
This work adheres the ICLR Code of Ethics. Our research aims advance open-domain QA while contributing positively to scientific excellence and societal well-being. We emphasize transparency by addressing the overlooked issue of ambiguity in QA benchmarks, which, if ignored, can lead to misleading evaluations and the underestimation of model capabilities. Our methodology requires no additional human annotation, thereby avoiding ethical risks associated with large-scale data col- lection. All experiments were conducted responsibly, with careful consideration of reproducibility, fairness, and inclusiveness. By explicitly modeling ambiguity, our approach seeks to reduce harmful biases introduced by incomplete ground-truth annotations and to promote trustworthy, reliable QA systems that respect diverse perspectives.
REPRODUCIBILITY STATEMENT
To facilitate reproducibility, we have open-sourced the source code, constructed dataset, and model weights https: /ttps://github wwt.:ooccffff//// wttpccof///////222222 A9eeeeaa.....ch Detailed experimental setups are described in Section 5.1, while additional implementation and hyperparameter settings are given in Ap- pendix D-E. The exact language model prompts used in our experiments included in Ap- pendix G.
10

---

Preprint
REFERENCES
Salaheddin Alzubi, Creston Brooks, Purva Chiniya, Edoardo Contente, Chiara von Gerlach, Lucas Irwin, Yihan Jiang, Arda Kaz, Windsor Nguyen, Sewoong Oh, al. Open deep search: Democ- ratizing search with open-source reasoning agents. arXiv preprint vrXiv:2503.20211 2025.
Ma Chang, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujuu Yang, Yaohui Jin, Zhenzhong Lan, Ling- peng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn Ilm agents. Advances in neural information processing systems, 37:74325-74362, 2024.
Mingyang Chen, Tianpeng Li. Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z Pan, Wen Zhang. Huajun Chen, Fan Yang. al. Learning reason with search for Ilms via reinforce- ment learning. arXiv preprini arXiv:2503. 19470, 2025.
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre- Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. 2024.
Yuchen Fan, Kaiyan Zhang, Heng Zhou, Yuxin Zuo, Yanxu Chen, Yu Fu, Xinwei Long, Xuekai Zhu, Che Jiang. Yuchen Zhang, et al. Ssrl: Self-search reinforcement learning, arXiv preprint arXiv:2508. 10874, 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 2025.
Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, et al. Skywork open reasoner technical report. arXiv preprint arXiv:2505.22312. 2025.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th Interna- tional Conference on Computational Linguistics, pp. 6609-6625, 2020.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training 1lms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.099 20t::0993099 16,,72 2025.
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: :riviqqa A large scale distantly the 55th Annual Meet-
supervised challenge dataset for reading comprehension. In Proceedings the ing of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1601-1611, 2017.
Hyuhng Joon Kim, Youna Kim, Cheonbok Park, Junyeob Kim, Choonghyun Park, Kang Min Yoo, Sang-goo Lee, and Tacuk Kim. Aligning language models to explicitly handle ambiguity. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp- 2009, 2024.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et Natural questions: benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466, 2019.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Weizhen Li. Jianbo Lin, Zhuosong Jiang. Jingyi Cao, Xinpeng Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, et al. Chain-of-agents: End-to-end agent founda- tion models via multi-agent distillation and agentic rl. arXiv preprint arXiv:2508. 13 167,7, 2025a.
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366 2025b.
11

---

Preprint
Alex Troy Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Ha- jishirzi. When not to trust language models: Investigating effectiveness of parametric and non- parametric memories. In The 61st Annual Meeting Of The Association For Computational Lin- guistics, 2023.
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: Answering ambiguous open-domain questions. arXiv preprint arXiv:2004. 10645, 2020.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguisties: EMNLP 2023, pp- 5687-5711, 2023.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2399,, 2016.
Siva Reddy, Danqi Chen, and Christopher Manning. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249-266, 2019.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063477 2017.
Zeyang Sha, Shiwen Cui, and Weiqiang Wang. Sem: Reinforcement learning for search-efficient large language models. arXiv preprint arXiv:2505.07903, 2025.
Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang. Nan Duan, and Weizhu Chen. Enhancing regrenedl- large language models with iterative -ereeval-generatioon synergy. In Find- ings of the Association for Computational Linguistics: EMNLP 2023, pp. 9248-9274, 2023.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402. 03300000, 2024.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang, Hugging- gpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36:38154-388 34-381800 80, 2023.
Zhiyu Shen, Jiyuan Liu, Yunhe Pang, and Yanghui Rao. Hopweaver: Synthesizing authentic multi- hop questions across text corpora. arXiv preprint arXiv:2505. 15007, 2025.
Zhengyan Shi, Giuseppe Castellucci, Simone Filice, Saar Kuzi, Elad Kravi, Eugene Agichtein, Oleg Rokhlenko, and Shervin Malmasi. Ambiguity detection and uncertainty calibration for question answering with large language models. In Proceedings of the 5th Workshop Trustworthy NLP (TrusiNLP 2025), pp. 41-55, 2025.
Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao Lei Fang, and Ji-Rong Wen. RI-searcher: Incentivizing the search capability in Ilms via reinforcement learning. arXiv preprini S50i:::2003005522. 2025.
Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. Asqa: Factoid questions meet long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 8273 8288, 2022.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique Multihop questions via single-hop question composition. Transactions the Association for Computational Linguistics, 10:539-554, 2022.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving re- trieval with chain-of of- thought reasoning for knowledge-intensivv multi-step questions. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023.
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Ma- jumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:22/2.035.. 2022.
12

---

Preprint
Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang. Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang. Shizheng Li, Qi Qian, et al. Searching for best practices in retrieval- -augeeel- generation. In Proceedings the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 17716-17736, 2024a.
Ziting Wang, Haitao Yuan, Wei Dong, Gao Cong, and Feifei Li. Corag: A cost-constrained re- trieval optimization system for retrieval- -aggeeeed generation. arXiv preprint arXiv:4411.00744, 2024b.
Qwen An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxin Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yi-Chao Zhang, Yunyang Wan, Yuqi Liu, Zeyu Cui, Zhenru Zhang, Zihan Qiu, Shanghaoran Quan, and Zekun Wang. Qwen2.5 technical report. ArXiv, abs/2412.15115, 2024. URL https://api. semanticscholar wwwg/orrpsssseoooooooo : 274859421.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369-2380, 2018.
Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. tau-hench: A benchmark for tool-agent-user interaction in real-world domains. arXiv preprint 2tli::2206./24455 2024.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan. Gaohong Liu. Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476. 2025.
Zhenrui Yue, Honglei Zhuang, Aijun Bai, Kai Hui, Rolf Jagerman, Hansi Zeng, Zhen Qin, Dong Wang, Xuanhui Wang, and Michael Bendersky. Inference scaling for long-context retrieval aug- mented generation, arXiv preprint drXiv:2410.0434. 2024.
13

---

Preprint
APPENDIX
A Generalization to Base LLMs
16
A.I Training Setup...
16
A.2 Entropy Control
16
A.3 Results.
17
Training Data Construction
17
B.1
Sampling Runtime and Computational Cost
17
B.2
Detailed Description of the Filtering Step
18
B.3 Detailed Description the Verification Step
18
B.4 Reliability of an All-Open Verification Pipeline
19
C Additional Statistics
20
C.1 Answer Count Distribution
20
C.2 Rollout Efficiency
21
C33 Ambiguity Ratio of QA Benchmarks
C,4 Ambiguity Analysis
23
Experimental Setup
23
D.1 AnsF1@t Estimation Algorithm
23
E
Ablation Experiments
24
E.1
The Role of a in Reward Design
24
E.2
Rollout Size
24
E.3
Sampling Temperature
25
E,4
Evaluating with LMJudge
25
E.5
Additional Results on Llama-Based Models
27
E,6
Disentangling the Sources of SEARC''ss Gains
28
The Use of Large Language Models
30
Prompt Templates
30
G.I Prompt for LMJudge
30
G.2 Prompt for Evidence-based Verification
.............
30
G.3 Prompt for Grouping Semantically Identical Answers
31
G.4 Prompt for Training SinSearch
31
G.5 Prompt for Training 2 2EAACCH and AbgSearch
32
G.6 Prompt for Training " SEARCI--Bas
34
G.7 Prompt for Training SinnSearch-Baee..... Sii.SarchhBee...................
34
G.8 Prompt for Ambiguity Type Classification
34
14

---

Preprint
H Rollout Cases of A SEARCH
35
15

---

Preprint
A GENERALIZATION TO BASE LLMS
A. TRAINING SETUP
To assess the generalization of our ambiguity-awan training beyond instruction-ture tuned models, we also experiment with Qwen2.5- --see.. The training configuration is kept consistent with SEARAH and SinSearch, except that we adopt prompts tailored for base models (Appendix G.6 and G.7).
In practice, base models are likewise able to generate multiple answers for ambiguous questions. However, unlike their instruct-tume tunde counterparts, they tend to undergo early entropy collapse, which restricts exploration and results in lower validation performance. To mitigate this issue, we introduce an entropy regularization term with adaptive entropy control (He et al., 2025), as detailed in the next subsection.
A.2
ENTROPY CONTROL
1.
Model / Entrogy Control
1.2
0.25
(1-0.3)
1.0
38 baseine ina
78 base the
ArsF1 Value
0.20
0.25
Entroyy
0.8
0.20
0.15
Recall
0.6
0.15
0.4
0.10
Model Control (hw0.25)
0.10
Model Control
(tw0.25)
0.05
3B EC (h-0 78 basaline
0.05
baseline
EC 2h-0.0
0.0
0.00
3B baseline
0.00
30 baseline
200
400
600
200
400 Stop
600
800
200
400 Step
600
800
Step
(a) Entropy Value
(b) Validation AnsFl
(c) Validation Recall
Figure 5: Effect of entropy control on validation performance and the model's entropy value.
For RL training on base models, we extend the optimization objective by adding an entropy regular- ization term. Specifically, the policy me is updated by maximizing
[(0)- =
Bx-D,(u mo
G
To T
To (y((x) Toau (yi
e, c,1+e) 4, + A AHo (.)
G
min
Te (3/ (x)
A,. clip
where A; = (r: mean( ( 1G )) (std( is the normalized advantage of the i-th rollout in the group, F; the reward, e the clipping ratio, and the entropy weight. The entropy term Ha (z, yi) is computed as the average token-level entropy along the rollout ys:
lus H ( 2, (,,,,)))) H()) = Ep(a) logr())))())
Ho y:)
This entropy term encourages exploration and mitigates over-confident predictions. As noted in prior work (He al., 2025; Yu al., 2025), keeping entropy within moderate range critical: too low leads to collapse, while too high causes unstable learning. To address the early entropy collapse we observed with base models, we adopt the adaptive entropy control method of Skywork-OR1 (He et al., 2025). This method sets target entropy h and dynamically adjusts A: when entropy falls below h, À is increased by a small step (up to a maximum Amaz); when rises above h, À is decreased symmetrically. In our experiments, we set Xmar = 1c-2, 5 = 3, and target entropy values of 0.3 for 3B-Base and 0.25 for 7B-Base.
As shown in Figure 5, entropy control effectively stabilizes training by maintaining higher tropy levels. Models trained with entropy control achieve substantially better validation AnsFl and Recall, confirming that controlled exploration enables faster and more effective learning.
16

---

Preprint
A.3
RESULTS
Multi-hop QA
HotpotQA
2Wiki
MuSiQue
Bamboogle
Macro-Avg
AnsFl/Rocclliak
Q1
@3
Q1
033
Q1
33
Q1
G3
@1
a3
3B
42.8 /u1.
56.2 /58.9
24.2 /25.9
49.3 /50.1
43.1 /119
S
37.9 41.1 /17.1
47.3
50.8 /s8.2
19.5
20.5 /25.0
38.4
38.2 /1LS
35.8
37.7 /13.2
A SEBRCH--B--as
11.7 /44.5
55.2 /s0.0
25.2 /28.3
42.4
/45.6
41.1 /44.4
SimSearch-3B-Baas.
38.3 40.7 /47.1
45.0 48.1 /ss.4
20.1 22.1 /27.1 37.6
39.1
/44.0
35.2
37.5
A AEARCI--
49.5 /52.1
62.3 /64.4
30.1 /34.8
51.7 /33.6
18.
/s1.2
SinSearch-7B
45.6 46.9 /50.3
57.6
59.5 /6t.1
25.4 27.0 /30.9
48.8
50.6 /5a.8
44.4
46.0 /19.8
A SEARRC--7 7B--H----Bass-- Base
47.4 /50.0
59.3 /61.3
27.0 /50.8
49.5 5 /s2.0
45.8
/18.5
SinSearch-77 7B Base
42.1
43.6 6 /ae.8
52.0 54.6 /62.9
21.4 23. /28.7
45.6
43.9 /49.0
40.3
41.3 /17.6
General QA
NQ
TriviaQA
PopQA
AmbigQA
Macro-Avg
AusF1/Recal@@k
@1
@3
@1
@3
@1
@3
@1
@3
@1
@3
A AESARRC--3B
47.3 /40.7
60.9 /u2.
48.2 /so.5
43.1 /38.2
49.9 /su.2
SinSearch-3B
40.9 43.3 /18.2
58.0
59.9 /61.9
43.7
45.0 /19.3
38.2 f32
41.2 /38.6
45.2 /43.9
47.4 /50.2
A ASSEARCH-3B-Base
47.2 /50.6
62.5 /65.2
50.0 /s3.0
44.1 /40.1
51.0 /s2.2
SinSeerchh3B-Basse
45.4 46.8 /50.4
59.6
61.6 /08.2
47.4 49.0 /s3.9
41.5 fa5.7
43.7 /40.1
48.5 /47.0 50.3 /52.6
A AE
51.4 /54.T
67.8 /aa.6
/89.6
52.5 /ss.6
48.1 /43.2
55.0 /ss.8
SinSearch-7B
49.3 49.8 /51.3
66.2 67.0 /a0.2
50.5 51.4 /53.5
44.6 /38.4 45.1 /ss.8
32.6 /s1.1 53.3 /53.5
SEARCH- Basc
50.2 /sA.2
65.9 /67.8
49.0 /s2.0
47.3 /12.8
53.1 /51.2
SinSearch-7B BBse
46.8 48.3 /53
63.8 64.6
loa.2
47.7 49.2 /ss.9 43.0 /37.1 45.5 A1. 50.3 A88. 51.9 /54.5
Table 4: Evaluation results with the Exact Match metric eight open-domain QA benchmarks. We report AnsF1 /Recall@k, where & denotes the number of rollouts. A2SEARCH and A Base use single rollout since they can generate multiple answers per attempt.
Table 4 reports the performance of A and SinSearch- Base. Several conclusions can be drawn. First, base models trained with our framework learn to recognize ambiguity and retrieve multiple answers, consistently outperforming their SinSearch counterparts across both sizes (3B and 7B) and all QA benchmarks. Second, the performance of A AESEARCII is broadly comparable to that of SEARCI, with only minor degradation. We attribute this gap to the additional post- training of instruct models on tool-use data, which enhances their agentic behavior, whereas base models have not been exposed to such data.
B TRAINING DATA CONSTRUCTION
B. SAMPLING RUNTIME AND COMPUTATIONAL COST
Model
Avg. Tool Calls
Duration
ReSearch-32B
3.52
38.6
ReSearch-7IB
3.00
16.0
Search-R R132B
1.82
7.3 h
Search-R1- R1-14B
1.68
5.2 h
Search-RI
2.80
7.8 h
Total
74.9 h
Table 5: Runtime for generating the sampled trajectories. "Avg. Tool Calls" denotes the average number of retrieval-tool invocations per sampled trajectory.
In our data construction pipeline, we generate a large number of trajectories in order to maximize sampling diversity and expose latent ambiguity. Concretely, we use five search models (ReSearch- each generating 16 sampled trajectories per question over the
7B/32B and 1--ichh---BB14BBB-- full set of 49, 938 training questions, for total of 3.99M trajectories. This is a one-time offline data construction step and is not required at deployment time.
17

---

Preprint
All sampling experiments are conducted on single node equipped with 4xH100 GPUs, using vLLM-a accclleeler inference (Kwon et al., 2023) and a locally served Wikipedia search index fol- lowing the Search-RI setup (Jin et al., 2025). The end-to-end runtime for evaluating all five models across the training set summarized in Table 5. This cost is amortized over the entire training cor- pus and is substantially lower than manual annotation at comparable scale. The pipeline is also highly parallclizable, and both inference throughput and retrieval latency can be further optimized practical deployments.
B.2 DETAILED DESCRIPTION OF THE FILTERING STEP
ReSearch
Search-RR R1
All Models
32B
7B
32B
14B
7B
Recall@k
0.785
0.758
0.729
0.720
0.635
0.868
Table 6: Performance of individual models and the five-model ensemble during the trajectory gen- eration stage. Results are reported as Recall@16 16 for each model and Recall@80 for the ensemble. The ensemble achieves higher recall, indicating greater trajectory diversity and broader coverage of reference answers.
We begin by evaluating the correctness of sampled trajectories against reference answers. We use Recallãk the evaluation metric, which measures whether at least one of the & sampled trajectories for a given question yields an answer semantically equivalent to the reference answer. Semantic equivalence is automatically judged by Owen2.5-32B-Insstie described in Appendix G.I. As reported in Table 6, both individual models and the five-model ensemble achieve high Recall@k. The ensemble further improves coverage by producing more diverse trajectories, confirming that our sampling stage provides a sufficiently rich candidate pool for subsequent processing.
(Yang et al., 2024), using the prompt
The objective of the filtering stage is to discard trajectories that do not contribute novel candidate answers. Based on the criteria described in Section 3.1, each model-question pair falls into one of three categories:
Case 1 (34.9%): All rollouts for given question semantically equivalent to the refer- ence answer. These trajectories provide no novel candidates and are therefore removed.
Case 2 (27.4%): None of the rollouts for a given question match the reference answer. This indicates that the model fails to solve the question, and the entire trajectory set is discarded.
Case 3 (37.7%): The rollouts for given question include both canonical and non-canonical answers. this case, the non-canonical rollouts may represent plausible alternative an- swers and retained for further verification.
To further reduce redundancy, we perform de-duplication at the answer level. Candidate answers are normalized through lower-casing, punctuation removal, and whitespace trimming, after which only one trajectory kept for each unique normalized answer.
After filtering and de-duplication, the dataset contains 208, 829 trajectories across 33, 997 questions. This corresponds to 5.2% of the original 3.99 million trajectories and covers 68.1% of the 49, 938 source questions. On average, 6.1 distinct trajectories are retained per question. This intermediate dataset constitutes the input to the verification stage, where the validity of alternative answers is further assessed.
B.3 DETAILED DESCRIPTION OF THE VERIFICATION STEP
This step determines whether each candidate answer in trajectory is sufficiently supported by the retrieved evidence. We employ four proprietary LLMs as verifiers: Claude 3.5 Sonnet, Claude 3.7 Sonnet, OpenAl o3, and OpenAl o4-mini. Each verifier processes trajectory using the prompt in Appendix G.2 and assigns one of three labels: supported, partially supported, or not supported. The intermediate label partially supported is introduced to prevent borderline cases from being
18

---

Preprint
Model
supported
partially
not supported
Claude 3.5 Sonnet
13.9
18.8
67.3
Claude 3.7 Sonnet
8.5
21.4
70.1
OpenAl o3
13.7
14.9
71.4
OpenAI o4-mini
20.8
6.8
72.5
Table 7: Label distribution (%) assigned by each verifier model.
Voting Threshold
1) = 1
1] 3
!J
Trajectory count
56,980
36, 096
19, 529
9, 655
Percentage
27.3%
17.3%
9.4%
4.6%
Human Agreement
64.0%
79.0%
96.0%
99.0%
Table 8: Number, proportion, and human agreement of trajectories labeled supported under dif- ferent voting thresholds r7. Percentages are computed relative the 208,829 trajectories obtained after the Filtering step.
over-classified supported, thereby improving robustness. The distribution of labels varies sys- tematically across verifiers. As shown in Table 7, Claude 3.7 Sonnet tends to produce more partially supported judgments, while 04-mini more frequently labels trajectories as supported. These com- plementary behaviors are reconciled in the subsequent majority-voting stage.
Following the notation in Section 3.1, we aggregate verifier outputs using majority voting with threshold Table 8 reports the number and proportion of trajectories classified as supported under different thresholds. To assess reliability, we additionally conduct a manual evaluation: For each n, 100 positively voted answers randomly sampled, and two co-authors of this paper serve anno- tators to judge whether they represent valid alternative answers. As expected, higher thresholds yield stricter criteria, thereby improving human agreement but reducing coverage. For example, 1/ achieves 99% agreement but retains only 4.6% of the data. Balancing agreement with coverage, we adopt 17 the default setting, which achieves a 96% agreement rate while preserving 9.4% of the trajectories (19, 529 in total).
B.4 RELIABILITY OF AN ALL-OPEN VERIFICATION PIPELINE
Threshold
Precision
Recall
Human Agreement
> 1/4
0.30
0.99
0.56
22/4
0.50
0.95
0.78
23/4
0.80
0.85
0.96
24/4
0.87
0.67
0.98
Table 9: Performance of open-weight verifiers under different voting thresholds, measured against the proprietary verifier ensemble and human agreement.
In our main experiments, we employ strong proprietary LLMs (Claude 3.5/3.7 and OpenAl o3/o4- mini) as verifiers in order maximize verification reliability. However, our framework does not rely on proprietary models, and it is desirable to assess whether entirely open verifier stack can achieve comparable agreement and coverage. To this end, we conduct an additional study using four moderate-sized open-weight LLMs that fit single H100 GPU: GPT-OSS- -0BB OPT-OSS-120B, Qwen3-30A3B, and QwQ-32B.
We sample 25, 000 trajectories from the data construction pipeline. These trajectories may con- tain alternative valid answers to the same question. Each verifier assigns one of three labels to a trajectory: supported, partially supported, not supported. In our evaluation, only the tedported label treated as a positive judgment. Using the proprictary verifier ensemble (four models with voting threshold of 23/4), total of 4, 651 trajectories marked supported. We
19

---

Preprint
then apply the ppin-weight verifier ensemble to the and compare its decisions against the proprietary ensemble.
We report two metrics: (1) precision, defined the fraction of trajectories approved by the open ensemble that are also approved by the proprictary ensemble, and (2) recall, defined as the fraction trajectories that also approved by the open ensemble. For each voting threshold, we additionally sample 100 open-approved trajectories and manually verify whether the retrieved evidence fully supports the answer; the proportion of such cases is reported Human Agreement.
of proprietary-aprrose
The results in Table 9 show that all-open verifier ensemble with 2/44 voting threshold achieves a strong balance between precision (80%) and recall (85%) relative to the proprictary ensemble. Human evaluation further confirms that this threshold yields high-quality labels, with 96% of cepted trajectories being fully supported by the retrieved evidence. We observe that most disagree- ments arise from how open-weight verifiers categorize borderline cases between supported and partially supported. Some answers marked partially supported are in fact fully sup- ported, reflecting a conservative tendency that increases precision the cost discarding some valid but borderline cases. Overall, this analysis demonstrates that fully open verifier stack can reliably approximate the judgments of proprietary models, enabling a fully reproducible pipeline.
C ADDITIONAL STATISTICS
C.1 ANSWER COUNT DISTRIBUTION
Answer Count
MSQ
HPQ
2Wiki
BBG
PQ
NQ
TQ
AQ
Overall
A?SEARCH-7B
52.9%
72.2%
78.8%
86.4%
75.0%
74.2%
84.6%
77.1%
75.1%
25.4%
17.9%
17.0%
8.0%
16.6%
15.3%
8.9%
13.6%
15.4%
14.8%
4.5%
1.9%
2.4%
4.8%
5.2%
3.3%
4.3%
5.1%
> 3
6.9%
5.4%
2.4%
3.2%
3.7%
5.3%
3.2%
5.0%
4.4%
Avg.
2.26
1.53
1.33
1.31
1.42
1.50
1.31
1.45
1.51
ATSEARCH-3B
69.8%
81.2%
88.1%
88.8%
79.8%
77.6%
86.0%
80.8%
81.5%
20.9%
14.3%
10.3%
8.8%
15.2%
15.6%
9.8%
14.2%
13.6%
9.4%
4.5%
1.6%
2.4%
5.0%
6.8%
4.2%
4.9%
4.8%
>
3
0.0%
0.0%
0.0%
0.0%
0.0%
0.1%
0.0%
0.1%
0.0%
Avg.
1.40
1.23
1.14
1.14
1.25
1.29
1.18
1.24
1.23
Table 10: Answer count distribution across benchmarks for AEARCH..
Table 10 presents the distribution of answer counts produced by A SEARCH across individual benchmarks. We report results separately for the 7B and 3B models. The benchmark abbrevia- tions are: MSQ (MuSiQue), HPQ (HotporQA), BBG (Bamboogle), PQ (PopQA), TQ (TriviaQA), and AQ (AmbigQA).
For A ABAERRCH-7 the model most frequently outputs a single answer (about 75.1% overall), but also produces answers 15.4% of cases and three or more answers in roughly 9.5% of cases combined. The higher frequency of multiple answers is especially evident on MuSiQue, where nearly half the questions elicit more than one answer on average (Avg.2.20) In contrast, datasets such Bamboogle (1.31) and 2Wiki (1.33) show relatively few multi-answer cases, reflecting their lower inherent ambiguity.
For A SEARCHI-3 the tendency to produce multiple answers is weaker, with 81.5% of questions receiving exactly one answer and only 13.6% receiving two answers. The overall average 1.23 answers per question, notably lower than the 1.51 average of the 7B model. Still, the 3B model demonstrates sensitivity to datasct-specific ambiguity: for example, MuSiQue again yields the high- est average count (1.40).
20

---

Preprint
In summary, SEARCH adapts to varying levels of dataset ambiguity, with the larger 7B model generating more multi-answer outputs, particularly on MuSiQue, while still maintaining a reason- able peecision-reaall balance. This validates that our ambiguity only to capture multiple answers when necessary, but also to refrain from over-generating when questions admit only single correct response.
training enables models not
C.2 ROLLOUT EFFICIENCY
Model
ReSearch
Search-RI
AFM-MHQA
A SEARCH
Size
7B
32B
3B
7B
32B
3B
7B
3B
7B
Lingl-roolour (Temperature=0)
Tool calls
3.17
3.54
3.19
3.03
1.97
2.96
3.31
2.16
4.14
Recall@ 1/pt
39.2 /o.12
46.2 /o
32.2 10.10 36.4 /o.12 44.0 /o.22
35.5
/o.12
39.0
39.0
44.7
10:21
51.2 Jo
Multi-rollow (Temperature-0.6)
Tool calls
3.11
3.31
3.19
3.03
1.98
2.72
2.96
Recall@2 2/ yptc
45.9 fo.07
52.0 /o.0s
34.9 /0.05
39.4 /0.06
48.8 /a.12
42.0
/o.08
44.7 /o.0s
Recall@3 3/ ypc
49.5 fo.05
54.9 fo.0s
36.5 f0.04
41.2 /n.0s
51.4 /a.0
46.6 /0.00
48.2 /o.n5
Table 11: Rollout efficiency with average tool calls, Recall and rptc.
To further assess the efficiency of different methods, we report Table the average number of tool calls, together with recall at different rollout depths and the derived metric rptc (recall per tool call), across four multi-hop hopp QA benchmarks. Since each tool call directly incurs inference cost, this metric measures how effectively model converts reasoning steps into recall gain.
Overall, we observe that multi-hop QA typically requires around three tool calls on average, but a larger number of calls does not necessarily yield better recall. For example, in the single-rollout (temperature = 0) setting, A SEARCII33 requires only 2.16 calls on average while already achiev- ing 1 of 44.7%, which corresponds to an rpic of 0.21. This efficiency on par with or even higher than the much larger Search-R R-32B (0.22), despite the significant gap in model scale. Similarly, SEARCH-7 performs the highest number of calls (4.14), but this is offset by its sub- stantially higher recall (51.2% at k = 1). leading to a strong rptc of 0.12 that surpasses the other 7B baselines. These results indicate that A SEARCH leverages additional calls in a productive manner, rather than wasting them on uninformative exploration.
In contrast, baseline methods such as ReSearch, Search-R1, and AFM-MHQA often rely on multi- ple rollouts with stochastic decoding (temperature 0.6) to approach the same level of recall. For instance, ReSearch-7B requires two or three rollouts to increase Recall@k to the range of 45-50%, whereas A SEARCH-BB achieves over 51% recall with single rollout. Taken together, these ob- servations demonstrate that A SEARCH achieves a favorable trade-off between recall and tool call efficiency, scaling effectively across different model sizes.
C.3 AMBIGUITY RATIO QA BENCHMARKS
To obtain a coarse estimation of the ambiguity level in existing QA benchmarks, we apply the Filtering. Verification. and Grouping steps described in Section to trajectories generated by five baseline models (ReSearch-7B/32BB and Search-R1 (17rhh-R1-BB/44B//3B) For each question, we have six sampled rollouts at temperature 0.6 and automatically annotate whether the predicted answers have enough evidence to support. The resulting statistics are as follows: 19.7% of questions in MuSiQue, 8.6% in 2Wiki, 11.1% in HotpotQA, 8.0% in Bamboogle, 12.7% in NQ. 14.7% in PopQA, and 7.0% in TriviaQA exhibit valid alternative answers.
These numbers suggest that ambiguity is non-trivial across benchmarks. Among multi-hop datasets, MuSiQue and HotpotQA display the highest ambiguity rates, reflecting the inherently open-ended reasoning process required. In the general QA setting, PopQA shows the highest proportion of ambiguous cases, while TriviaQA remains relatively less ambiguous.
21

---

Preprint
Artist was signed to different labels at different career stages.
Subjective. multiple valid stylistic explanations.
Depends on pre- /post-2022 interpretation
Different surface forms of the same entity.
Performer has multiple children.
Conflicting retrieved sources.
Country vs. city specificity.
Table 12: Representative ambiguous questions and valid alteraative answers.
Explanation
Columbia Records;
Qatar; United States,
Louis Grimaldi; Prince of Monaco
Various interpretations
Alternative Answers
Denmark; Fredericia
Lulu; Harper Simon
Over 5, 5,600; 2,200
Canada. Mexico
Warner Bros.
What was the record label of the performer of There Goes Rhymin' Simon?
What the place birth the director of the film The Outsider (2018)?
Where is the next FIFA World Cup going to take place?
How many fish species live in the river system contain-
Who is the child of the performer of "You Can Call Me Al
In what ways did Kanye draw inspiration from U2,
Led Zeppelin, and the performer of "Mother's Little Helper""
Who gets Blair pregnant in season 5 the series with the episode "The Ex Files"?
Question Example
ing the Jari River?
Granularity Ambiguity
Multi-Item Response
Ambiguity Type
Under- Under-Conster Constrained
Time Sensitivity
Evidence Conffict
Alias Variance
Open-E Ended
22

---

Preprint
C.4 AMBIGUITY ANALYSIS
To better characterize the kinds of ambiguity present in our data, first manually inspected a subset of questions with multiple valid answers and derived a taxonomy of seven recurring ambiguity types: Under-Constrained, Granularity Ambiguity, Time Sensitivity, Evidence Conflict, Multi-Item Response, EndedE and Alias Variance. The definitions of these categories are summarized in Table We also provide representative ambiguous question examples in Table 12.
D EXPERIMENTAL SETUP
D.1 AnsF1@k ESTIMATION ALGORITHM
In practice, evaluating AnsF1@k requires averaging over all possible subsets of k: trajectories drawn from a larger pool of k' (k' k) sampled trajectories. Simply reporting the best or worst case among k' samples would give biased picture model performance. Our estimation procedure therefore computes the expected precision, recall, and F1 under uniform subsampling without replacement, which provides a more faithful measure of model's ability to generate diverse and valid answers.
It is also worth noting that large language models are inherently stochastic. Their randomness can be amplified by sampling at a higher temperature. our experiments, we set the temperature to 0.6, which encourages stronger diversity in rollouts and thereby increases the chance of capturing multiple answers.
Algorithm AnsF 1@k Estimation via Subsampling
Require: hits: a list of length k' (k' > k), where each entry is either the identifier the reference answer matched by predicted answer, IL if no match
Require: g: total number of reference answers
Require: k: number of trajectories subsample
Ensure: Expected Precision@k, Recall@k, and Fl@k
1: For each reference answer a, compute its multiplicity m. in hits
2: denom + (K)
number of size-k subsets (uniform sampling without replacement) 3: sump 0, sum, <- 0, sumf1
4: for all size-k subsets S C hits do number of positive predictions in S (i.e., Hz E S Lt)
5:
6:
u +- number unique reference answers covered in (i.e., 1 E S x
unique precision recall
7:
p s/k
8:
+ u/g
9:
if p > and r > 0 then
10:
f1 + 2pr
> equivalently f1
2xu gs+t ku
11:
else
f1 +-0
13:
end if
14: sump + sump + p
15: sum, + sum, + r
16:
sum + sum/1 + f1
17: end for
18: E Preciiiion@k + sum,
E Recall@@k] summ/denom
20: F + summummm///e sumyn/denom
21: return (I[[Preciiion@k], [[ BFFG@)
Algorithm summarizes the procedure. Given a list of k predicted answers hits, where each ele- ment indicates either a matched reference answer or I, the algorithm enumerates all size-k subsets, computes precision. recall, and FI for each, and then averages them with equal weight. This yields unbiased estimates of AnsFl @k and Recall@A that account for both prediction correctness and coverage of distinct reference answers.
23

---

Preprint
0.8 leads the model to converge nearly one answer E ABLATION EXPERIMENTS
E.1
THE ROLE OF a IN REWARD DESIGN
0.275
0.35
20
Settings
0.250
0.30
0=0.8
Precision Volue
15
a=0.4
0.225
Velue
0.25
10
0.20
Settings
Settings
0.150
8-0.2
0.15
0-0.2
0.125
u=0.4
0=0.4
0.100
0.10
0-
200
400
600
800
200
400
600
800
200
400
600
800
Step
(a) Validation Precision
(b) Validation Recall
(c) Answer Count
Figure 6: Effect of in reward design on validation performance and the model's answer count.
Our reward is defined as
0
format invalid, if format valid and hits = 0, AnsF1), if format valid and hits >> O,
R(g, añs)
0.1, 1
with a E J0, 1] so that R E a. when hits 0. The parameter a governs the extent which imperfect AnsFll is penalized, thereby mediating the balance between precision and recall.
0.2+0.8- AnsF1 and becomes tightly coupled to AnsP1. Since AnsF1 is particularly sensitive to precision, the model strongly discouraged from producing many answers: additional candidates reduce precision, lower AnsFl, and thus incur substantial penalty. This tendency is further amplified by the data distribution itself, as more than half of the training cxamples contain only a single valid answer. Under such conditions, optimizing for reward with large naturally aligns with producing highly precise, single- answer outputs. Empirically, this effect is evident in Figure 6c, where we train A AEEEARCH with different settings. We can find that throughout training.
When a set to relatively large value, such as 0.8, the reward approximates
In contrast, when is small, such 0.2, the reward is bounded below by 0.8 regardless of precision, yielding A = 0.8 + 0.2 AnsFl. In this regime, the incentive to maintain precision nearly vanishes, and the model quickly learns to enumerate many answers ensure least one match. This behavior inflates recall and results in rapidly increasing answer counts, as shown in Figure 6c.
For intermediate values, such as 0.4, the penalty balances the two extremes: too few outputs reduce recall, while too many reduce precision, and the model stabilizes by producing a moderate number answers with a controlled trade-off. This behavior reflected in the validation trends in Figures 6a-6c, where = 0.4 achieves both stability and reasonable precision-recall balance.
E.2 ROLLOUT SIZE
One factor that may influence training effectiveness is the choice of rollout size. Intuitively, in- creasing the number of rollouts per step can enhance the diversity of sampled trajectories, thereby improving the likelihood of discovering high-quality reasoning paths and providing denser reward signals. To determine an appropriate rollout size for training, we conduct an ablation study on A training under three different rollout settings (8, 16, and 32) for two epochs.
As shown in Figure 7, larger rollout sizes could lead to better validation performance in terms of Precision, Recall, and AnsF1. However, the performance gap between rollout sizes 16 and 32 is relatively small. Considering the trade-off between performance gains and training efficiency, we adopt rollout size 16 as the default configuration for the experiments reported in the main paper.
24

---

Preprint
0.275
0.275
0.250
0.30
0.250
Precition Value
0.225
Recall Value
0.25
0.225
0.200
0.175
0.20
0.175
0.150
0.125
Rolout
Hot
Rollout Size 16
Rollout Size 16
out Size
0.100
0.10
200
300
100
100
300
400
Step
Step
Step
(a) Validation Precision
(b) Validation Recall
(c) Validation AnsEl
Figure Effect of rollout size on validation performance.
E.3 SAMPLING TEMPERATURE
Temperature
0.2
0.4
0.6
0.8
1.0
Metric@3
AnsFI
Recall
AnsFI
Recall
AnsFI
Recall
AnsFl
Recall
AnsFl
Recall
ReSearch-77B
24.5
29.5
25.1
30.5
24.9
31.1
24.3
30.8
23.2
30.2
Search-RI RI--B
20.3
21.7
20.8
23.0
20.7
23.6
20.8
23.9
20.8
24.6
AFM-MHQA-7B
22.9
28.3
22.8
28.4
21.5
27.4
19.5
25.5
17.4
23.7
Average
22.6
26.5
22.9
27.3
22.4
27.4
21.5
26.7
20.5
26.2
Table 13: Ablation study of sampling temperature on the MuSiQue benchmark. We report AnsF and Recall@3 for three baseline models across different temperatures.
Since baseline models such ReSearch, Search-R1, and AFM are trained to produce a single an- swer per question, we compute their multi-answer scores by sampling multiple rollouts. This intro- duces additional variable, the sampling temperature, which directly influences randomness. A higher temperature typically increases diversity, which may improve recall but often at the cost of precision. The degree of sensitivity to temperature also varies across models, depending on their training objectives. To ensure fair comparison, we therefore conduct an ablation study to identify the most suitable temperature setting for evaluation.
Concretely, we evaluate three baseline models (ReSearch-7B 7B, Search-R1-7B, and AFM-MHQA- 7B) the MuSiQue benchmark. For each model, sample six rollouts at temperatures ranging from 0.2 to 1.0 and compute both AnsF1 @3 and Recall@3. The results are summarized Ta- ble 13. From the table, we observe that Search-RI- temperatures, with recall steadily improving as the temperature increases, while Fl remains largely unchanged. contrast, AFM- F7M-MHQA-7B shows sharper decline in both and recall as tempera- ture rises, suggesting that it is more sensitive to randomness. ReSearch-7BB achieves its best balance around 0.4-0.6, where recall is highest (31.1 at T 0.6) without significant loss FI. Averaged across all three models, recall peaks at 7 0.6, while FI remains competitive compared to lower temperatures. Based on these findings, we select
exhibits relatively stable performance across
0.6 the default sampling temperature in experiments, it provides fair trade-off between recall and Fl while allowing each bascline model to perform near best.
E.4 EVALUATING WITH LMJUDGE
While our main evaluation relies on the Exact Match (EM) metric, it only measures lexical overlap and cannot capture semantic similarity. This raises a potential risk: since natural language often admits multiple surface forms, A SEARCH could appear to achieve higher recall simply by generat- ing paraphrased variants of reference answers, thereby "hacking" EM. By contrast, baseline models, especially those trained to produce single answer, would need multiple samples generate such
25

---

Preprint
Model
HotpotQA
2Wiki
MuSiQue
Bamboogle
Macro-Avg
AnsFI/Recallik
1
@
@1
a3
03
@1
@3
@1
@3
Models with 3B Parameters
DirectGen-3B
25.3
26.1
/29.6
27.5
28.3 /314
6.6
7.2 /9.7
7.2
8.7 /144
16.6
17.6
/20.5
Naive-RAG-3B
43.2
43.1 /t5.5
29.3
29.9
/31.
11.4
11.9 /13.3
23.2
22.8 /214
26.8
26.9 /28.7
Iter-RetGen-3B
45.8
47.0
/t9.7
30.6
31.7
/s16
13.2
14.0
/16.2
22.4
23.7
/2n.
28.0
29.1 /31.
IRCoT-3B
52.6
56.5 /67.3
39.0
43.2 /57.6
15.9
18.5
/25.5
37.6
41.9
/52.4
36.3
40.0 /50.7
Search-R1-3B
54.6
56.5 /o0.8
45.3
47.9 /sa.s
22.2
24.2
/28.7
47.2
47.2
/sn.5
42.3
43.9 /18.5
AFM-MHQ-3B
41.5
63.2 /126
52.3
55.0 /67.
28.2
31.5
/41.8
56.8
56.5 /65.3
44.7
51.6 /6L8
SinSearch
56.1
59.9 lezz
51.9
55.5 /eso
28.6
31.1
/38.6
52.0
38.2 /41.8
47.2
46.2 /6a/2
AbgScarch
43.8 /48.4
33.5 /40.0
15.6 /17.3
33.7 /34.4
31.5 /36.0
A A
62.4 fot.9
60.9/o1.
36.3 /39.4
61.4 /02.4
55.3 /31.2
Models with 7 ~ 32B Parameters
DircetGen-7B
31.8
32.5 /35.9
27.8
29.3 /33.5
12.4
13.2 /1644
24.0
22.8
/25.9
24.0
24.5 /27
Naive-RAG-7B
53.7
54.7
/s7.2
35.7
36.5 /as.s
15.7
16.5 /18.7
39.2
40.0
/42.9
36.
36.9 /39.3
Iter-RetGen-7B
55.3
57.3
/o0.
37.0
38.8 /42.5
19.0
20.1
/23.5
36.8
39.0
/41.
37.0
38.8 /12.2
IRCoT-7B
62.0
66.3
/16.7
44.2
48.4 /sn.0
18.0
20.6 /er
51.2
53.0
/ss.2
43.9
17.1 /s7.2
ReSearch-7B
64.4
66.8
/14.3
55.5
58.8 /os.8
35.3
39.2 /47.9
59.2
59.8
/e6.2
53.6
56.2 /64.3
Search-R1 R1-7B
62.3
63.5 /66.9
46.3
48.9 /51.6
30.0
30.9 /ss.5
56.0
53.8 /50.7
48.7
49.3 /s8.4
AFM-MHQQ-7B
67.8
68.3 /75.5
55.1
56.6 /en.6
33.9
34.8 /s8.1
60.8
63.5 //2.9
54.4
55.8 /6L.3
Search-RI-145
67.4
68.4 /73.1
55.5
57.6 /03.4
36.9
39.2 /15.5
64.0
67.0 /89.9
55.9
58. /62.9
Search-R R-32B
66.3
68.2 /73.2
57.5
59.9 /67.4
35.1
37.5 /44.5
63.2
67.8 /72.4
55.9
58.4 /64.4
ReScarch-32B
70.9
72,2 /m..9
60.3
63.8 /12.3
39.8
43.5 /St.4
72.0
71.2 /ns.7
60.7
62.7 /69.3
SinSearch
65.9
67 . /71.6
62.8
64.7 /00.7
37.9
40.2 /46.3
63.2
63.8 /66.9
57.5
59.0 /63.6
AbySearch
58.7 /03.7
43.8 /50.2
26.0 /30.2
51.4 /sa.6
/49.4
A'SEARCH-7B
71.2 /74.9
67.8 /09.
44.7 /51.8
67.2 /es.8
62.7 /98.3
Table 14: Main results on four multi-hop QA benchmarks under the LMJudge metric. We report AnsF1/ MasF1/Recall@e RRecall@k with k rollouts. For AbgSearch and A AERCCH only @1 is reported, reflecting their ability to produce multiple answers within a single rollout. For the remaining baselines, where each rollout generates only one answer and thus AnsF1@1 AnsP1/Recall@3 Recall@3 to evaluate their performance when more rollouts are available. The best result in each comparison group shown in bold, and the second best is underlined.
Recall@ we additionally include
variants, which could exaggerate the apparent recall advantage of A In such cases, EM may not fully reflect whether models actually resolve true ambiguity.
is worth noting, however, that this issue does not affect training. When computing the hits for AnsF1, we only count distinct reference answers. If the model outputs multiple variants that all match the same reference answer or its aliases, the hits count remains one, and precision is penalized accordingly (e.g.. predicting two synonyms for one reference yields only one hit and 50% precision). This design prevents the model from exploiting lexical overlap during training.
To address the evaluation limitation, we complement EM-based scores with an LMJudge method, which measures semantic equivalence. Specifically, we prompt Qwen2.5-32D ---st---ntttu (Ap- pendix G. to judge whether each predicted answer semantically matches a reference answer. Based on these judgments, we recompute AnsI'I 1 and Recall.
The results are reported in Table 14 (multi-hop QA benchmarks) and Table 15 (general QA bench- marks, including AmbigQA). We find the conclusions are highly consistent with those obtained under EM:
(1) Agentic search models consistently outperform prompting- and RAG-based baselines, with especially clear gains on multi-hop datasets.
Even with single greedy decoding rollout, A SEARCH matches or surpasses the Recall@3 of baselines and even outperforms larger 32B models on several benchmarks.
(3) A SEARCH achieves the best AnsF1 on multi-hop QA benchmarks, striking a strong bal- ance between precision and recall.
26

---

Preprint
Model
NQ
TriviaQA
PopQA
AmbigQA
Macro-Avg
Aose1/Recallok
03
@1
@3
@1
@3
@
a3
Models with 3B Parameters
DirectGen-3B
25.9 26.8 /313
41.7
42.4 /6699
18.7 18.7 /15.6
18.7 /15.6
20.7 /19.5
26.3 /25.5
27.2 27.2/0 /28.3
Naive-RAG-3B
59.2
59.8 6617
71.4
71.7
h30
49.1
49.5 /50.6
46.3 /39.1
47.3 /111
56.5 /51.8
57.1 /56.6
Iter-RetGen-3B
60.5
61.7 f6L.
73.1
73.8
/s5.6
50.3
50.8 /52.5
48.2 /1.1
49.3
/18.2
58.0
/56.3
58.9 /58.9
IRCoT-3B
61.1
64.1 /728
70.2
72.8
/r9.1
49.3
5188 /58.2
46.7 /39.9
51.3 /18.2
56.8 /s5.1 60.0
/ELT
Search-R1-3B
61.8
62.6 f65.2
72.4
73.0
/15.6
50.6
51.2 /s2.7
47.2 /50.3
/12.8 41.4/12.8
48,
58.0
/50.3 58.8 /ss.1
AFM-MHQ-3B
60.9
62.4 /71.
72.2
74.1 /x2.1
47.7
47.7 1s5.6
47.7
Jau.
51.8
/49.6
57.1 1
/55.4 59.0 /64.8
SinSearch
59.6 61. .9 /67.5
70.0 72.3 /m..
48.4
49.8 /54.4
46.1
/30.
49.6 /46.5
56.0
/54.3 58.4 /61.3
AbgSearch
59.1 /64.6
67.9 /r0.1
43.6 /47:2
48.6 /43.5
54.8 /36.4
A ABSEARCH-3B
65.1 /a7.9
73.3 /r5.1
/5a.
51.4 /45.0
60.3 /60.5
Models with 7 32B Parameters
DirectGen-7B
35.9 37.3 /m.7
55.7
56.6 /309
20.9 21.5 /24.2
26.3 /21.9 28.7 /25.9
34.7 /33.6
36.0 /37.9
Naive-RAG-7B
66.9
67.7 100.9
76.6
77.1 /s8.2
52.6 53.3 /ss.2
51.2 /s.
52.7 /46.1
61.8 /59.9
62.7 f62.3
Iter-RelGen-7B
67.4
68.6 /119
78.4
79.0
/s0.4
52.8
53.7 /s6.1
52.3 /44.7
54.1 Ar7.7
62.7
/00.8
63.8 /64.0
IRCoT-7B
65.5 68.8 /2o.2
74.2
76.2
/sn.8
51.4 54.1 /60.2
49.9 /a2.7
54.6 /50.5
60.3 /58.5 63.4 /60.9
ReSearch-7IB
65.8 67.9 /74.6
77.2
79.6
/s4.7
50.9
52.6 /s8.2
50.3 /42.9
55.2
/50.7
61.1 /xa.2 63.8 /67.1
Search-R1-2
65.6
66.4 /68.5
77.1
77.7
/19.7
50.8
51.7 /sa.2
50.4
/33.0
51.9
/15.4
60.9 /59.1 61.9 f6L.7
AFM-MHQ-7B
66.2
68.3 /15.3
77.8
79.8 /SL.7
50.9
52.6 /se.0
51.4 /33.9
55.4
/512
61.6 /59.7 64.0 /07.8
Bearch-R1-14B
68.4
69.1 /r2.0
79.2
80.1 /s4.2
54.6
55.6
/5e.2
52.8 /55.2 54.8 /18.6
63.8 /81. 64.9 fe6.0
Search-R1-328
68.5
69.9 /73.5
82.3
83.5 1963
54.1
55.7
/s0.5
52.7 /a5.2 55.5
fay.6
64.4 162.5 66.2 f67.2
ReScarch-32B
69.1
71.4 /76.6
81.8
84.8 /NN.a
53.8
55.7
/60.4
55.1
147.0 57.8
/sz.1
64.9
/62.8 67.4 /69.
SinSearch
67.5
68.1 /60.8
79.0
79.8 /82.1
55.5
56.5 /58.6
52.4
/40.7 53.4 /46.5
63.6
/61.6 64.5 /64.3
AbgSearch
66.4 /r2.a
78.4 1SL.5
52.9 /58.2
55.7 /50.4
63.4 165.7
A'SEARCH-7B
70.4 /24.0
81.3 I83.4
57.3 /e0.6
56.6 /so.3
66.4 /67.1
Table 15: Main results with the LMJudge metric on four general QA benchmarks, using the same notations as Table 14. For AmbigQA, where questions may have multiple reference answers, AnsF1@1 and Recall@1 are not equivalent in this setting, and both are therefore reported.
(4) AbgSearch performs well on AmbigQA but fails to generalize, whereas SEARCH, trained without AmbigQA data, surpasses it even on AmbigQA.
Overall, these results demonstrate that our findings are not artifacts of lexical metrics. Instead, they confirm that A SEARC genuinely learns to resolve ambiguity, thereby validating both the effectiveness of our training approach and the robustness of our experimental conclusions.
E.5 ADDITIONAL RESULTS ON ELAMA-BASHE MODELS
To examine whether our ambiguity-aware RL framework generalizes beyond the Qwen backbone, we additionally trained A Aeaarch-LMM using Llama 3.2 Instruct 3B under exactly the same training setup as used for Search on Qwen models. For comparison, we also trained a matched single-answer baseline, denoted SinSearch-LMMA-33 using the same single-answer supervision and Exact Match reward applied in our Qwen-based baselines.
The evaluation results are listed in Table 16. Overall, the Llama-based results exhibit trends con- sistent with the Qwen-based experiments. A2Search-LMMA-BB achieves competitive performance relative to Qwen models of similar size, demonstrating that the ambiguity-aware RL objective trans- fers well across backbone architectures. Furthermore, despite using only a single rollout, A Search- LMA-3B typically achieves high recall that approaches exceeds that of SinSearch-LMM -MMacch--MM-3B..... which is computed with three rollouts. These observations support the conclusion that the proposed framework is backbone-agnostic and broadly applicable across model families.
27

---

Preprint
Multi-hop QA
HotpotQA
2Wiki
MuSiQue
Bamboogle
Macro-Avg
AnsF1/Recalla
63
23
@
63
@
93
@1
G3
ABSEARCHBBB
42.8 /44.4
56.2 /ss.9
24.2 /25.9
49.3 /50.4
43.1 /44.9
SinSearch-3B
37.9 41.1 Am7.
47.3
50.8
19.5
20.5 /25.6
38.4
38.2 /AL8
35.8
37.7 /33.
A SEARCCH-7
49.5 /s2.1
62.3 f64.1
30.1 /34.8
51.7 /s3.6
48.4 /s1.2
SinScarch-7B
45.6 46.9 /s0.3
57.6
59.5 /61.1
25.4 27.0 /30.9
48.8
50.6 /s3.8
44.4
46.0 Aeuu
A -SEARCH-LMA-3B
42.2 /s9.6
53.4 /ss.a
22.6 /21.8
51.1 /s2.
42.3 /43.9
SinSearch-LMA-33
36.2 38.7 /45.7
48.2
49.9 /ss.1
16.7 18.7 /23.8
39.2
43.4 /ss.1
35.1
37.7 /44.4
General QA
NQ
TriviaQA
PopQA
AmbigQA
Macro-Avg
AncF1/Recall@k
@3
@1
@3
@1
@1
@3
@
@3
ASSEARCH-B
47.3 /48.7
60.9 /62.5
48.2 /50.5
43.1
/a8.2
49.9 /s0.2
SinScarch-3B
40.9 43.3 /48.2
58.0 59.9 /64.9
43.7 45.0 /a0.3
38.2 /32.8
41.2 /ss.6
45.2 /43.9 47.4 /s0.2
A SEARCH-BB
51.4 /s17
67.8 f69.6
52.5 /55.6
48.1 /18.2
55.0 /55.8
SinSearch-7BB
49.3 49.8 /51.3
66.2 67.0 /ee.2
50.5
51.4 /sa.5
44.6 /38.4
45. /3s.8)
52.6 /s1.1 53.3 /sa.5
A A-SEARCH-LMA-3B
46.8 /s9.6
62.0 /63.2
47.7
43.4 /38.7
50.0 /s0.8
SinSearch-LMA-3B
41.8 43.2 /49.4
55.9 58.6 /65.7
46.2 48.5 /S1.6 38.3 /sa.0 42.2 /10.9 45.6 /4L. 48.1 /s1.7
Table 16:
Evaluation results with the Exact Match metric eight open-domain QA benchmarks. We report AnsF1/Recale
where & denotes the number of rollouts. A SEARCH uses a single rollout since they can generate multiple answers per attempt. A SEARCCH-LMA uses LLaMa-3.2- Instruct as the backbone LLM for RL training.
E.6 DISENTANGLING THE SOURCES OF A SEARCH'S GAINS
To better understand which components of SEARCH are responsible for its improvements. we train several ablated variants that isolate the effects of (i) ambiguity-aware training data, (ii) the AnsF'l reward, and (iii) the ability to output multiple answers. All variants use the same backbone (Qwen2.5 7B Instruct) and follow the identical training setup unless otherwise stated.
Ablation Variants. We evaluate three additional models:
trained on single-answer data with recall-oriented EM-Recall ward (reward = I if any generated answer matches the gold answer). The model is in- structed output up three answers. This variant isolates the effect of "outputting multi- ple guesses" without ambiguity-aware data or the AnsF1 reward.
RECALLSearch-7B:
SIN Search--77 trained on ambiguity-aware multi-answer data but with a single-answer EM reward and forced single-answer decoding. This variant tests whether access multi- answer supervision alone (without multi-answer output or AnsF1 reward) can improve ambiguity handling.
A ASSNSearrh--7B. trained with the AnsI'l reward but on single-answer data, producing an arbitrary number of answers. This variant isolates whether AnsF1 alone, in the absence of multi-answer data, can induce ambiguity-aue aware behavior.
Answer Count Analysis. As shown in Table 17, across all datasets, SIN SLa*chhr always out- puts exactly one answer, confirming that multi auster supervision alone does not induce ambiguity- aware behavior the model forced into single-answer decoding. A AearNSea outputs slightly more than one answer on average, but the increase is marginal, indicating that AnsF1 alone does not meaningfully encourage the exploration of alternative interpretations. RECALLSearch-77 always outputs exactly three answers by construction, amplifying recall mechanically but without regard for evidence or ambiguity. Only A adapts its answer count to the dataset: it outputs more answers datasets with higher inherent ambiguity and fewer answers where single interpre- tation is sufficient. This adaptive behavior is a key indicator that the model is detecting underlying ambiguity rather than emitting guesses.
28

---

Preprint
Model
MSQ
HPQ
2Wiki
BBG
PQ
NQ
TQ
AQ
Answer Count Distribution Across Models
SIN* Searchhh
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
A'SINSearch-7B
1.13
1.07
1.03
1.01
1.10
1.09
1.06
1.11
RECALLSearchh7B
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
A SEARCIIB
2.26
1.53
1.33
1.31
1.42
1.50
1.31
1.45
Table 17: Answer count distribution across eight benchmarks for four ablation variants and A SEARCH-B.
Multi-hop QA
HotpotQA
2Wiki
MuSiQue
Bamboogle
SIN Scarchhh-
47.2
58.4
26.5
50.4
ReCALn.Search-78
29.6 20.6/ 55.6
39.3
28.6 68.2
18.9
13.5 34.3
32.2 /2.2/22.1 61.6
A SSNNSearch--7B
46.9/
57.6/
57.3 58.2
58.2
25.5
25.3
25.8
49.6
48.8 49.6
A ABERRCH-BB
/ 48.6/ 52.1
/
64.4
30.1 28.5 34.8
51.7
50.9 53.6
General QA
NQ
TriviaQA
PopQA
AmbigQA
SINaCcarch-7B
49.8
67.6
50.5
4 64.8 38.8
RECALLScarch-22 7B
35.1/25.11 63.2
49.5 39.2 75.0
/ 61.1
38.3 / 36,4 49.8
A ? SINNearch-7BB
49.0/48.7
49.7
65.1
65.0 65.4
49.0 48.6
49.9
44.6/
62.8 38.8
A SEARCHI-B 50.4/ 54.7
67.8 67.1 69.6
51.1 / 55.6
48.1 66.2 /43.2
Table 18:
Ablation study results eight QA benchmarks. We report AnsFl @ I for single-answe models and AnsFl / Precision Recall at @1 for the models that produce multiple answers.
Performance Across Eight Benchmarks. From the results reported in Table 18, we can con- clude many findings. When evaluated standard multi-hop and open-domain QA benchmarks, SIN*Search-7B underperforms due to its single-answer restriction. These datasets contain latent ambiguity, and single deterministic prediction is often insufficient. RECALLScarch-77 achieves artificially high recall but suffers from extremely low precision, as many of its additional answers are unsupported speculative. This behavior is consistent with reward hacking: the model increases recall by emitting paraphrases or guesses, rather than identifying genuine alternative answers.
A SNNSearhh-7 performs similarly single-answer baselines and does not show meaningful gains in recall, further confirming that AnsF1l without ambiguity-awar supervision is insufficient for uncovering alternative valid answers.
In contrast, AASEARCH1-B improves both precision and recall simultaneously across all bench- marks. This joint improvement is not achievable by any of the ablated variants, indicating that the full combination of (i) ambiguity- training data, (ii) AnsFl reward, and (iii) flexible multi- answer decoding is necessary capture genuine ambiguity.
Analysis of RECALLSearch-17 low precision, we classify the sources of its multi-answer outputs using the ambiguity taxonomy introduced in Appendix C.4. Only small portion of its predictions correspond genuine ambiguity types (e.g., under-constrained or multi-grand multi-granuulrity cases). A disproportionately large fraction stems from alias-level variation (35%) or unsupported guesses (21%). This confirms that EM-Recall reward encourages broad coverage rather than true ambiguity resolution.
To understand why RECALLScarch-7B achieves high recall but
Conclusion. Taken together, these ablations demonstrate that each component of SEARCH plays an indispensable role. The ambiguity-aware data enables exposure to multiple valid inter- pretations; the AnsFll reward encourages and multi-answer generation allows the model to express these interpretations. None of the compo- nents alone is sufficient. The complete system is required to achieve robust improvements in both precision and recall, reflecting genuine advances in ambiguity resolution rather than output-format artifacts metric gaming.
reasoning toward all correct answers;
29

---

Preprint
F THE USE OF LARGE LANGUAGE MODELS
In the preparation of this manuscript, LLMs were used solely as auxiliary tools for paraphrasing and polishing the writing to improve readability. No LLM was involved in formulating research ideas, proposing methods, designing experiments, drawing conclusions. All scientific contributions and substantive content presented in this paper are original work of the authors.
G PROMPT TEMPLATES
G.1 PROMPT FOR LMJUDGE
Prompt for LMJudge
You will be given question and its ground truth answer 1ist where each iten can be ground truth answer. judge pred_answer correctly answers question ground truth answer list. You should first give your rationale for the judgenent, and then give your judgement result (i.e., correct or incorrect).
Provided pred_answer, you
Here criteria for judgenent:
The pred_answer doesn't 't. need to be excctly the same of the ground truth answers, but should be
senantically same for question.
Each iten the ground truth answer 1ist can be viewed as ground truth answer for the question, and the
pred_answer should be senantically sane to at least one of then.
question: (question]
ground truth answers: (gt_answer)
pred_answer: (pred_answer)
output should in the following 1son format: "ssoon
"rationale": "your rationale for the judgenent, text".
"judgement": "your judgement result, can only be 'correct' "incorrect"
}}
...
Your output:
G.2 PROMPT FOR EVIDENCE-BASBE VERIFICATION
Prompt for Eridence-based Verification
Evidence-Consisstency
[CRITICAL SCOPE]
Do NOT assess the correctness the Question; anbiguity is normal and expected. The Final Answer need not be comprehensive; concise acceptable. Your only whether is valid, defensible resalution supported by retrieved evidence.
[Yau will receive]
Question
The agent's rollout:
Thinking (ignore evidence)
Search Querins
Tool Results (titles snippets)
Final Answer
[Evaluation Principles]
Only Tool Results count evidence. Ignore Thinking and outside knowledge.
For each atonic clain in the Final Answer, check support against evidence:
SUPPORTED: explicit match (paraphrase OK).
SUPFORREE: sone support but with gaps/inference.
S.POR.P:::: absent contradicted.
Concise answers choosing one reasonable reading are acceptable.
Be faithful: details beyond evidence; numbers/dates must natch.
If conflicting, prefer more peecific/reeentreleeeee
evidence; otherwise nark partial/not supported.
Closed-world: if insufficient, label PARTIALLY SUPPORTD Scope: do not grade breedth, style. or completeness. Only check evidence support.
SUPPOTEDD. not guess.
[Outout Requirenents]
Output JSON only.
Cite evidence with stable IDs (e.g.. T1/T2).
LJSON Schena]
30

---

Preprint
[JSON Schena]
{(
"verdict": "SUPPORTED PARTIALLY SUPPORTED SUPPORTTR" "clains_enalysis":
{(
"clain": "atonic clain text",
"status": "SUPPORTED SUPPORTD SUPPORTED", "evidence"; ["title of the evidence". ...]
)
}}
(Verdict Labels]
SUPPORTED: clains clearly backed, major gaps/conf icts.
PARTIALLY SUPPORTED: PARTIALLY__UPPORTE::: Sone backing but with gaps/inference.
NOT_SUPPORTED: Mostly unsupported contradicted.
Cinput Begin]
Question:
(question)
Trallout_uull_tev
Cinput
G.3 PROMPT FOR GROUPING SEMANTICALLY IDENTICAL ANSWERS
Prompt for Grouping Semantically Identical Answers
You are provided with list of textual answers. Your task is to arganize these answers into groups of senantically equivalent or closely
responses.
Requirenents:
Compare the svintended meaningte of each answer, rather than relying solely surface wording.
Answers that convey the **sane or highly similar ideaes should placed the sane group, even
expressed differently.
Answers with distinct meanings must renain separate groups.
Preserve the #noriginal text** of each answer without nodification. output must follow structure **JSON 20 arrayss, equivalent answers.
inner array contains one group
##Ø Output Format: jjson
["Answer A1", "Answer A2", "Answer ",
["Answer B1", "Answer B2"),
)
G.4 PROMPT FOR TRAINING SinSEARCH
Prompt for Training SinSearch
You are a helpful assistant that solves the given question step by stop using the wikipndia_snarch tool,
Your Task
Use the wikipedia_ search tool gather comprehensive information and the user's question through c structured exploration process.
Workflow
s*Locate sourcesst
Use the wikipedia_search tool to Find the most relevant Wikipedia pages related the query.
*4Branch out**
>>Depthhx Explore each key page detail to fully understand the topic.
**Bresdthte If there are multiple interpretations or entities, investigate each as separate branch.
**Synthesize the enswer**
Based the evidence gathered, synthesize well-supporte
If the question is anbiguous or has multiple velid interpretations, present and justify each possibility.
**Return in structured formate*
wrap your reasoning in "<thinke" tags.
Always include at least one "ctool_call>"
Present the finel answer inside "<answer>" tags using the specified JSON structure below.
31

---

Preprint
Cutput Fernat
You must always follow this structure:
Start each step with your reasoning inside "<think> </thinks" tags.
You nust always nake at least one seaaph function call, even if you think already know the answer.
Use "<tool_call> </taol_call>"
specify the function and parantters, in the format shown output inside respon_e resool__ tags.
-+
below.
will
this "sthinke", "<tool_call>" needed deepen your search.
output yourself.
Repeat
5. When ready to answer, present it Inside:
<answer> jsoo
"rationale": "Concise reasoning: key search paths and evidence supporting the answer.", "answer": "your answer here"
?..
</answer>
Tools
You may call functions ussist with the query.
You provided with Function signatures within <taols>c/tools> XML tags: <tools>
"type": "function",
"function":
"name" : "wikippdiaasserchh"
"description": "Search Mikipedia for information about specific query. Returns list of summaries of <+ the related articles.",
"paraneters";
"type": "object",
"properties";
"query":
"type": "string", "dnscription": "The specific query tern to search on Wikipedia,""
"required"; ["query"]
</tools>
For each function call, return jsan object with function and argunents within tttol_calD>/ttoo__ll
XHL tags:
<taol_call>
("name": <function-name>, "arguments": <args-json-ohjact>)
</tool_call>>
G.5 PROMPT FOR TRAINING SEARCH AND AbgSEARCH
Prompt for Training A SEARCH and AbgSearch
You are helpful assistant that solves the given question step by step using the wikipedia search tool,
Your
Use the wikipedia_seede search tool gather comprehensive information and answer the user's question through
+ structured exploration process.
Workflow
**Locate sources**
Use the sekipedia_search tool to find the nost relevant Wikipedia pages related the query.
*4Branch out**
**Depphh* Explore each key in detail to fully understand the topic.
>*Breadthws there are multiple interpretations or entities, investigate each separate branch.
*1Extract answersA*
Based on the evidence gathered, synthesize more well-supported answers (at most three different
answers).
-If the question is anbiguous or has multiple valid interpretations, present and justify each possibility.
**Return in structured format**
Wrap your reasoning in "<thinko" tags.
32

---

Preprint
AlMays include at least "<tool_call>", Present the final answer Inside "Kasserr"" tags using the specifled J5ON structure below.
output Fornat
You must always follow this structure:
Start each step with your reasoning inside "<think> </think?" tags.
You must always nake at least one wikipedia_searchh function call, even if you think you already know the
Use "ctool_call> </tool_call>" tags to specify the function name and paraneters, in the fornat shown below.
The user will provide the tool output inside reseon_essoose>> r/tool_response>> tags. Never generate this yourself.
Repeat the pattern of "<thinke", "etool_call"" as needed to deepen or expand your search.
When ready answer, present it inside:
"json
"rationale:: "Concise reasoning: key search paths and evidence supporting each answer.",
aanseess: [
'nnwwe 1",
2"
"Answer 3",
).
</answer>
Tools
You may call functions ussist with the query.
You provided with Function signatures within <taols>c/tools> XML tags: <tools>
"type": "Function",
"function":
"name": "wikipedia_ searchh"
"description": "Search Mikipedia for information about specific query. Returns list of sumnaries of < the related articles.",
"paraneters":
"type": "object",
"properties":
"query":
"type": "string", "description": "The specific query tern to search on Wikipedia,"
"required"; ["query"]
</tools>
For each function call, return json object with function name and argunents within <tool_call></too_.cca
XHL tags:
<taol_call>
("nane": <Function-name>, "arguments": cargs-json-objecee
</tool_call>
33

---

Preprint
G.6
PROMPT FOR TRAINING A SEARCH-BASS
Prompt for Training SEEARCH-Base
conversation
question, and assistant salves step by step. The assistant first thinks about the reasoning process in the nind and then provides the user with the enswer. During thinking, the assistant can invoke the wikipedia search tool to search for fact information about specific topics if needed. The reasoning pracess enclosed within <thinke </thinko assistant wants to search, the search query is enclosed in <search> </search> tags, and the user will provide search result <result> </result> tags. assistant repeat
ssaistant.
when the
pattern multiple
to explore different search paths expand the reasoning. The assistant begins by locating relevant sources through wikipedia search tool. Then, explores each source depth and also considers alternative interpretations in breadth. After gathering enough infornation, it extracts and organizes the possible answers. Finally. the assistant returns the answer in the required structured format. For exanple, <think> This is the reasoning
</think> <search> search query here </search> <result> search result here </result> <think This is the reasoning process. </think <answer> The final answer is (boxed[[answee]]] answer2; answer]]] \] </answer>. the last part of answer, final exact answer is enclesed within \boxed(() with latex format. If there are multiple possible answers, they should be separated by semicolons. User: (question). Assistant:
G.7 PROMPT FOR TRAINING SinSEARCH-BASE
Prompt for training SinSearch-BBae
conversation between User and Assistant. The user asks question, and the assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. During thinking, the assistant
invoke the wikipedia search tool to search for fact information about specific topics if needed. The reasoning process and answer are enclosed within <think </think and query </search> and cresult> </result> tags respectively. For example, <thinko This is the reasoning process. </thinks <search> search query here </search> <result> search result here </result> <think> This the reasoning process. </thinks <answer> The final answer 1s AL (ansed( here]] VJ </answer>. In the last part the answer, the finel exact answer is enclased within \boxed[()) with latex fornat. User:
<answer> </answer> tags respectively,
<search>
(question). Assistant:
G.8 PROMPT FOR AMBIGUITY TYPE CLASSIFICATION
Prompt for Ambiguity Type Classification
AxQA Analyst
are given:
**Questionts,
2.
Multiple +nSearch-Agent Trajectories (queries, reasoning processes, and final answers).
**Prenise:**
provided final answers considered #*factually validex. Honever, they are tesubstantively different**
+ (they contain different strings, facts, numbers, entities).
asYour Task:
respecific cause** divergence. Why reasoning lead different outcones?
Select whone more specific label from the list below and provide concise explanation.
a ssLabel Definitions & Decision Logicsz
## *61. ALIAS, VAREANCE** A
The divergence is superficial. The answers refer to the **xxact sane unique real-world entity** but use different nanes, abbreviations, spellings, translation variations.
TIME SENSITIVITY
answer depends aswhents question + referenced documents. The question implies a "current" status or fails to lock specific date.
divergence
tinefrane
## #x3. EVIDENCE_CONFLLE CONFLICT**
The divergence is strictly caused by excontradictory information in the retrieved documentsts, Different source passages provide conflicting about the sane specific entity.
*44. CRAMULARITYAANEE GUI
divergence caused by unspeci requirements regarding Aeprecision date/location, units measurenent, geographic scope, and others**. The agents agree on the fact but format it differently.
34

---

Preprint
IN *45, OPEN_ENDEDA>
The question inherently qualitative requests list/exanple where sano single unique entityte the + exclusive truth. The question asks "Why", "How", or for "Benefi tx/Exapplee
#t wA6, LNDER_CONSTTAANNDDAA
The question appears to seek a specific unique entity but lacks sufficient conditions (constraints) to narrow the search down this differs from "Open-Ended" because it usually feels like "missing detail" + defect in the question, or involves hononyns (sane nare, different people/works).
# **7. MULTI_ITEN_REEPOME RESPONSE**
The divergence is caused because the question's s answer is inherently a list or set of itens, not single unique value. Even there only obvicus search path interpretation of the question, there multiple valid itens that satisfy the criterion, and the question does not clearly require complete or canonical list
**Output Format**
Return a JSON object:
-"json
"labels": D]
"reason": "Concise explanation focusing on the specific differentiator."
H ROLLOUT CASES OF SEARCH
Question
that the most influential figure Islamic philosophy was one the greatest thinkers? (from MuSiQue) Multiple historical ligures could plausibly satisfy the given constraints.
Cause Ambiguity
Reference Answer
George Sarton (describes Aviecnna as one of the greatest thinkers) Oliver Leaman (describes Mulla
Alternatvv
most important thinker)
Question
the owner
record label the performer
Kind Love? (from MuSiQue)
Cause of Ambiguity
The performer. Rodacy Crowell. has released works under multiple record labels.
Reference Answer
Warner Music Croup (parent of Warner Bros. Recurds)
Alternatve Answer
Music Entertainment (parent of Columbia Records)
Question
Nikolaus Wilhelm Nassau's 's The historical distinction between Wünttemhery and Germany not explicitly considered. Germany (Wurteebeerg merged into Germany)
Which country
(from 2Wiki)
Cause of Ambiguity
Reference Answer
Alternatve Answer
Würtiemberg (Princess Pauline of Württemberg
member the House Wünttemherg)
Question
the place death of Hayranidil Kadm's husband? 2Wiki) Ambiguity arises from the gramularity of the geographical specification. Constantinople (capital
Cause of Ambiguity
Reference Answer
Ottoman Empire at time)
Alternative Answer
Çuraaan Palace (a palace located within Constantinople)
Question
Where does the director of film Wine Of Morning work at? (from HospotQA) Ambiguity arises from different levels of institutional affiliation.
Cause of Ambiguity
Reference Answer
Jones University Unusual Films (the university's 's production company)
Alternatvve Answer
Question
How Ceephax Acid Crew and Squarepusher's music similar? (from HotpoxQA)
Cause Ambiguity
The question lacks a clearly defined dimension of comparison.
Reference Answer
Drum and bass electronic music
Alternative Answer
Acid house (another electronic subgenre frequently associated with their style)
Question
What is the primary male hormone derived from? (from Bamboogle)
Cause Ambiguity
Testesterone can be traced either to its metabolic substrate or to its immediate biochemical precursor.
Reference Answer
Cholesterol (the fundamental metabolic of steroid hormones)
Alterative Answer
Androstenedione (the direct biochemical precursor of festosterone)
Table 19: Representative ambiguous questions drawn from multi-hop QA benchmarks. Each exam- ple includes the reference answer provided in the benchmark and the evidence-supported alternative answers identified by A SEARCH.
In this section, we present representative real examples of ambiguous questions together with the corresponding valid alternative answers identified by A SEARCC in Table 19. We additionally pro- vide the complete reason-and and-srcch trajectories produced by A SEARCC for the questions in Ta- ble 19. For clarity and to ensure correct ыТ&К rendering, certain special Unicode characters have been omitted, which may result in minor character loss in the reproduced text.
35

---

Preprint
Who said that the most inftuential figure in Islamic philosophy of the greatest thinkers? (from MuSiQue)
Reference Answer: George Sarton
<thinkoTo find said
most influential figure Islanic phi losophy was
greatest
+
thinkers, first, we need to identify who the nost influential figure in Islanic phi Iosophy is considered to be, and then find who said that this figure was one of the greatest thinkers. s/thinks
<tool_call>
("name": "vkiipedia_saarch", "arguments"; ("query"; "wh is the nost influential figure in Islamic
+ philosaphy"))
</toel_call<<</n_eeene in
<|in_start[Puser
<tool__espons>> "Mlla Sadra"
Mulla Sadra ad-On Muamnad Shrz, also called Mulla Sadr G also spelled Molla Sadra, Mollasadra or 1571/2 1640), was Iranian Twelver Shi'a Islanic philoshpeer,
Sadr-ol-Mote allehhin;
theologian and 1in who led the Iranian cultural renaissance in the 17th century. According to oliver single
Leanan,
influential philosopher in Muslim world in the last Four hundred years. Though not its founder, he is considered the naster of the Philesophy, seminal figure synthesized
arguably
impor
Illuinttionist (or, Ishraghi Ishraqi) many tracts of the Islanic Golden Age
"Islanic philosophy'
after the golden ages] of Suhrawardi's "HHikat al-Ishrag" (""ILlumination Philososhy"") or Mulla Sadra "lliknat-e-oota sliye"" (""Transcendent" [ar ""Exalted""] ""Philosophy")) Another figure
+ Allama Hahammdd Iqbal, who reshaped and revitalized Islanic philosophy anong the Muslim ins of the early 20th century. Islan"" is a milestone in the modern political philosophy of Islan. In contemporary Islanic regions, continued to flourish. Philososhy not been without criticisn amongst Muslins, both contemporary and past. The inam Ahmad ibn Hanbal, for whon the
Indian sub-continent
""The Reconstruction of Religious Thought
teaching ""hiknat"" or ""hiknsh"
Hanbali school of thought is naned,
Al-Tafsir
Al-rafsir Tafasir Al Quran (Persian: are collections of books commntariis Quran written by
Mulla Sadra. adr ad-Dn Huamna Shrz, also called Hulla Sedr (; also spelled Molla Sadra, Mollasadra Iranian Ship Islamic philosopher, theologian and
(c. 1571/2 1648), 1im who led the Iranian tultural renaissance in the 17th century. According to oliver Leanan, Milla important and influential posopprp in the Muslin world last four hundred years. Hulla Sadra brought. ""a new philosophical insight in dealing with the nature of reality"* and created
Sadr-ol-Hete' -al--ettellllhin
Sadra arguably the single
"Islanic philosophy"
""reason"* ("** "ql"") is sonctines recognised as source of Islanic lau, this may have tatally different meaning fron ""reason"" in philosophy. The historiography Islamic philosophy is marked
by disputes to how the subject should be properly interpreted. Sone of the key issues involve the
comparative impor Lance of eastern intellectuals such as Ibn Sina (Avicenna) and of western thinkers and also whether Islanic philosophy interpreted in an esoteric fashion. Supporters of the latter thesis, like Leo Strauss, maintain that to
such
read face value should
Islanic philesophers
"Hulla
Mulla Sadra. In Fact, the netalanguage approach could be considered in latter not the foruer. Mulla Sadra ad-Dn Muamad Shrz, also called Mulla Gi spelled Irenian Twelver Shi'a Islanic philosopher.
Sadra, Hellasadra
Sadr-ol-Mote allehin; ) (c. 1571/2 1640).
+
theologian and 1in who led the Iranian cultural renaissance the 17th century. According to 0liver Leanan, Mulla Sadra arguably the single inportant and influential poilosopher in the Muslin
world in the last four hundred years. Though not its founder, he is considered the master of the Illuninationist (or, Ishraghi Ishraqi) schools/tool, rewpooeeeeessseeeiinieeeeee
i|innstart sessstrrtassstt Passsisttt
influential figure Islanic philosophy 4 considered to be Mulla Sadra or, alternatively, figures like Ibn Sina (Avicenma) or Ibn Rushd, but Leanan specifically mentions Mulla Sadra as the single most important influential philosopher the Muslim world in the last four hundred years. So, the nost influential figure in Islanic philosophy is Mulla Sadra or Avicerna or others like then.
<thinkoFron provided responses, it
Next, we need to identify who said that this nost influential figure was one of the greatest thinkers.</thinke <taol_call>
("nane": "vikipedia_searct"" "argunents": ("query": "wha said that Mulla Sadra the greatest 4 thinkers"))
/tool_ccll><<(i__edd
i|in_start|uuses
stool_respanse> "ullaa Sadra"
36

---

Preprint
Mulla Sadra adr ad-Dn Muamad Shrz, also called Mulla Sadr G also spelled Nolla Sadra, Hollasadra ar Sadr-ol-Mote allehin; ) (c. 1571/2 1640), was an Iranian Twelver
Islamic posoopp,,r,
theologian and Lin who led the Iranian cultural renaissance in the 17th century. According Oliver Leanan, Mulla Sadra is arguably the single nost important and influential phi losopher in the Muslin world in the last four hundred years. Though not its founder, is considered the master of the Illuminationist (or, Ishraghi or Ishraqi) school of Philosophy, seninal figure who synthesized the Age
many tracts
Islanic
"Hulla Sadra"
Mulla Sadra. In fact, the netalanguage approach could be considered in latter not the former. Mulla Sadra Mulla (i spelled hadr-ol-totealllehnn allehin; ) (c. 1571/2 1648), was an Iranian Twelver Shi Islanic phi theologian and 1im who led the Iranian cultural renaissance the 17th century. According Oliver Leunan, Mulla Saira arguably the single important and influential phi philosopher in the Muslim world in the last four hundred years. Though not its founder, he is considered the mastar of the Illuminationist (or, Ishraghi Ishraqi) school
ad-On Muamaa Shrz, also
Sadra, Mollasadra or
Al-Tafsir
Al-Tafsir Tafasir Al Quran (Persian: are collections of books as commntaries on Quran written by Sadra.
Sadr also spelled Holla Sadra, Mollasadra 4 ar Sadr-ol-Hote allehin; (c. 1571/2 1648), was an Iranian Shia Islamic philosopher, theologian and 1/th century. According Oliver Learan, Mulla Sadra is arguably the single nost important and influential philosopher in the Muslin world the
Shrz,
who the Iranian cultural renaissance
last four hundred years. Hulla Sadra brought "'a new philosophical insight dealing with the nature of reality" and created
"Hulla Sadra"
of the Ash Ash'ari ari school and Twelvers. His main work is "he Transcendent Theosaphy in the Four Journeys of Intellect"", or simply "Feur Journeys" Born
what now Iran, notable Family of court officials in 1571 or 1572. In Mulla Sadra's time, the Safavid dynasty governed on kings granted independence to Pravince which ruled by king's brother. rulla Sadara's 's father, khwajah Ibrahin qavami, was knowledgeable and extrenely faithful palitician.
+
Iran.
His father was rich man and held a high position, but children. However, after lot
"Mir Fenderesk1*
Persian philosopher, poet + and mystic the Safavid era. liss full name is given as Sayyed Mir Abulqusin Astarabadi (Persian: lived for while in Isfahan the time Mir + Danad spent a great part of his life in India among yogis and Zoroastrians, and learnt certain things then. was patrenized both Safavid Mughal courts. ctrrriiosotoooeeeesoreeee in in_end]>
Mir Fendereski Hir Fendereski Mir Findiriski (Persian: (15621648)
), and he is fansusly
Fendereski.
philosopher
< Mulla Sadra also studied under hin. Mir Fendereski remains
(in_start(/assiisaant
<taol_call>
("nane": biikipedia_search,, "argunents": (" ("query":
Avicenna was one of the greatest
thinkers"))
</teel_call><(inneend]
</in_start/>user
<taol_respnne>>Avccomm
Avicenna Avicenna (; also Ibn Sn Abu Ali Sina; June 1837) was Persian polynath who is regarded + one of the nest significant physicians, astrononers, thinkers and writers the Islamic Golden Age. He has been described ibed the father early nodern medicine. Of the 458 works he is known have written, around 248 have survived, including 150 on philesophy and 48 on nedicine. His most fanous works "*The Book Healing"", philosophical and scientific encyclopedia, and *"The Canon of
Medicine"*, a medical encyclopedia which became standard medical text at many
Avicenna
Virgil, Averroes, Honer, 4 Socrates, Plato, and Saladin, Avicenna has been recognized by both East and West, as one of the great History Science described Ibn as ""one of the greatest thinkers and medical scholars in history"" and called him ""the nost fanous scientist of Islam and one of the nost fanous of all races, places, and times. "* He was one the Islanic world's leading writers the field of medicine. Along with Rhazes, Abulcasis, Ibn al-Nafis.
non-Christian thinkers
"'Divine Conedy""
Ovid, Lucan,
figures intallectual history. George Sarton,
author
Avicenna
been translated into English Parwiz Morewedge in 1977. The book also inportant respect to + Persian scientific works. "'Andar Danesh-e Reg" (. "*On the Science of the Pulse")> contains nine chapters on science of the pulse and condensed syrapsis. Persian paetry
is recorded in various manuscripts and later anthologies such as "Nozhat al-Majales" Avicenna
Avicenna (; also Ibn or Abu All Sina: June 1837) was a Persian polymath who is regarded as one of the significant physicians, astrononers, thinkers and writers of the Islanic Golden He
Avicenna
37

---

Preprint
in which he commntts on several Quranic verses and holds the Quran in high esteen. Avicenna argued that Islanic prophets should considered higher than poilosapeers. Mhile was inprisoned Hanadhan, Avicenna wrote his farous "'Floating Man"" literally falling thought exper to denonstrate human self self-mwareness and the substantiality and immatrriality of the soul. Avicanna believed his ""Floating Nan"" thought experiment denonstrated that the soul is substance, and clained humans cannot doubt their own consciousness, even in a situation that prevents
castle of Fardajan
sensory input. thought exper
Avicenna
3 ""Latin Avicennisn"* as flourishing for sone time, paralleling the more influential Latin Averroisn, 1210 1215. Avicenna's psychology theory
suppressed by the Parisian knowledge influenced Milliam of Auvergne, Bishop of Paris and Albertus Magnus, while his netaphysics influenced the thought of Thomas Aquinas. Early Islamic philosephy and Islanic metaphysics, inbued as clearly than Aristotelianism between essence and 4 existence. Whereas existance is the donain the contingent and the accidental, assence andures within being beyond the accidental. The philosophy Ibn Sn, particularly that part relating + serpoooorsseoseeeeinneneee
it with Islunic thealogy, distinguishes
<lin_startpaassisaan
<tool_call>
("nate": Vikipedia_searchh "argunents": ("query": "wha said
was one of the greatest
4 thinkers in histary")]
</tocl_call<<jnn___ in_en|/>>
non-Christian thinkers in his "'Divine Comedy"" such as Virgil, Averroes, Honer, Horace, Ovid, Lucan, Socrates, Plato, and Saladin, Avicenna has been recognized by both East Nest, as one of the great figures in intellectual history. George Sarton, the author of ""The History of Science"", described Ibn as ""one of the greatest thinkers and medical schelars in history" "" and called hin ""the nost places, and times. "" He
fanous scientist of Islan and of the nost fanous of all the Islamic world's leading writers in the field of medicine. Along with Rhazes, Abulcasis, Ibn al-Nafis,
Avicenna
Avicenna Avicenna ( also Ibn Sn or Abu All Sina; June 1837) was Persian polynath who is regarded nest significant physicians, astrensners, thinkers writers of Islamic Golden Age. He has baen described as the father of early modern medicine. of the 458 works he is known have written, around 248 have survived, including 150 on philosophy and on medicine. His most fanous works "'The Book of Healing"", philosophical and scientific encyclopedia, and "'The Canon of Medicine"", a medical encyclopedia which became standard medical text at many
"History of science'
Sina (Avicanna, 9861037) is regarded the influential phi losopher of Islan. pioneered the science of experinental medicine and was the first physician to conduct clinical trials. His two most The
nedicine are "'Kitb al-shif"" (""Book Healing" both of which were used as standard medicinal texts in both the Muslim world and in Europe well into century. Amongst many contributions are discovery infectious diseases, and the introduction of clinical phernacology. Scientists fron the Islanic world include al-Farabi (polynath), Abu al al-Qasin
notable
of Medicine.
contagious nature
Avicenna
has been translated into English Paruiz Morewedge in 1977. The book also inportant in respect to + Persian scientific works. "'Andar Baresh-e Rag"" G, "*On the Science of the Pulse"") contains nine chapters the science of the pulse and is condensed synopsis. Persian paetry fron Ibn Sina is recorded in various manuscripts and later anthologies such as ""Nohat al-Majales"', Avicenna or Abu Sina; ; June 1837) was Persian polymath of the most significant physicians, astronomers, thinkers and writers of the Islanic Golden Age. He
Avicanna (i
regarded as one
Avicenna
however, stating that: sox"I prefer short life width to a narrow one with length"*"*, On restored unjust gains, freed his slaves, and read through the Quran every three days until his death. died in June 1837, his fifty-eighth year, the nonth of Ranadan and buried in Hanedan, Iran. Ibn Sn wrote extensively an early Islanic philosophy. especially the subjects logic, ethics, and netaphysics. including were wricten in Arabic
deathbed renorse seized him; he bestowed his goods the
treatises named ""Logie"" and "Metaphysics" Most
4 language of</tool_ retttoo=_rrppnse>> jiinsstat(passistaan <taol_call>
("nane": "wikipedia, searchh'" "argunents": ("query": said
influential figure Islanic
4 philosaphy was one of the greatest thinkers in history"))
</tooll.all><(inneedd]]
jin_starr|/>ssrr
stooo_response>>WWll Sadra"
Mulla Sadra adr ad-Dn Muamma Shrz, also called Mulla Sadr (i also seelled Holla Sadra, Mol or
4 Sadr-ol-Mote allehin; ) (c. 1571/2 1640), was an Iranian Twelver Shi Islanic philosopher,
the 17th century. According Oliver Leanan, Mulla Sadra is arguably the single nost inportant and influential philosopher in the Muslin hundred years. Though not its founder, is considered the naster the Illuninationist (or, Ishraghi or Ishraqi) school of Philosophy, seminal figure who synthesized the Age
theologian and 1in who led the Iranien cultural renaissance
world the last
many tracts Islanic
38

---

Preprint
Al-Kindi
and Roger Baton. The Italian Renaissance schelar Geralone Cardane (1581 1575) considered him as one of the Yagb Isq Muslin phi losopher, polymath, mathematician, physician and musician. Al-Kindi was the first of the Muslin peripatstic phi losophers, and is unanimously hailed as the ""ather Arob philosaphy"" for synthesis, adaptation pronotion of Greek and Hellenistic philosophy the Muslin world. Al-Kindi was born in Kufa and educated in Baghdad. He becane a proninent figurre in the House of
twelve greatest minds. Al-Xindi
G ; 881873 AD)
Al-Tafsir
Al-Tafsir Tafasir Al Quran (Persian: are collections of books as comentaries on Quran written by spelled Holla or Sadr-ol-Hote --o-ttteelleinnn (c. 1571/2 1648), was Iranian Shia Islanic philosopher, theologian and 1im who led the Iranian cultural renaissence the 17th century. According Oliver Leanan, Hulla Sadra arguably the single nost important and influential pospophp in the Muslin world the last four hundred years. Hulla Sadra brought "la new philosophical insight dealing with the nature of reality" and created
Sadr
Mollasadra
"Hulla Sadra"
Mulla Sadra. In fact, the netalanguage approach could be considered in latter not the former. Mulla Sadra ad-Dn Muamad Shrz, also called Mulla G; spelled Sadra, Hollasadra
+
Sadr-ol-Hate allehin; ) (c. 1571/2 1640), was an Iranian Twelver Shi Islanic phi losogher, theologian an and who the Iranian cultural renaissance
century. According Oliver Leanan, Mulla Sadra arguably the single nost important and influential philosopher in the Muslin world in the last Four hundred years. Though not its Founder, he is considered the naster of the Illuminationist (or, Ishraghi Ishraqi) school
"Islanic philosophy'
the 9th century and ended with Averroes (Ibn Rushd) at the end of 12th century. The death of Averroes particular discipline Islamic philosophy usually called the Peripatetic Arabic School, and philosophical activity declined significantly in Western Islanic Islanic
effectively marked the
Africa. Islanic philosophy persisted Muslin Eastern countries, in particular Safavid Persia, Octoran and Mighal Empires, where several schools of philosophy continued to flourish: Avicemnisn, Averroisn, Illuninationist philosophy, Mystical philosophy, Transcendent theosophy. and Isfahan philosophy. Ibn Khaldun, in his
countries
longer
+ ""ugaddinah"" made important contributions to the respsppyy//tooo.sspppsssiiiir
<|in_start passissant
sthinleFrom the provided responses, it is clear that figures like George Sarton, Roger Bacon, and others of the greatest thinkers.
have referred
philosophers
"arguably single important influential philosopher - in the Muslim world in the last four hundred years." George
Oliver
the greatest thinkers medical scholars history."
5o, appears scholars George others referred Islanic philosophers
< of the greatest thinkers.<think,
<answer>
..* jjjoon
"rationale":
'Mulla is considered influential figure Islanic philosophy, George Sarton described Avicenna one of the greatest thinkers history.
"answers":
"Oliver Leaman".
"George Sarton"
</answr>><|in ind
Who is the owner of the record label of the performer of What Kind of Love? (from MuSiQue) Reference Answer: Warner Music
slin_startDasssittnn
<thinkoTa find the the record label of the perforner of "Whot Kind of Love," first, need 4 idantify the parformer of the song "What Kind of Love." </thinks
<tool_call>
(*nane": saaipeddaa_serrhh"" " ("query": "wh perforned song Kind Love"))
//too__cal><</in_eeee
slin.start/>user
stool_respons>""hhe "that Kind of Love"
39

---

Preprint
What Kind of Love ""What Kind of Love"" is song co-written and recorded by Anerican country nusic artist second single
albun ""Life Is Messy" The song reached number 11 the U.S. ""Billboard" Hot Country Singles Tracks chart and peaked number on the "RRP"" Country Tracks in Canada. Crowell and co-writer Hill Jennings wrote the lyrics recording of nelody that arbison had made before his death, The song features backing vocals from Don henley and Linda Ronstadt. Deborah Evans Price,
Rodney Crowell. was released in 1982
"What Kind of Love"
covered the song his 2082 albun for Dreamtorks Records. It
B-side the albun's first single, ""Honey Do" What Kind of Love ""Waa Kind of Love"" is song conwritten and recorded by American was released
+
June 1992 the second single fron albun ""Life Is Messy"", The song reached number 11 on the U.S. ""Billboard" Hot Country Singles & Tracks chart and peaked at nurber 2 on the ""RP""" Country Tracks Canada, Crowell and co-writer Mill recording of melody that
country music artist Rodney Crowell.
Jennings wrote the lyrics
"What Kind of Love You On"
What Kind of Love Are You On ""What Kind of Love Are You On"" is a song by Anerican hard rock band Aerosmith. The seng, originally track left off the ""Nine Lives" album, was included "*.* for the 1998 film ""Armageddon"" starring lead singer Steven Tyler's daughter Liv Tyler. The song, was Mainstrean Rock Tracks chart. It
radio, reaching 14 was written by Steven Tyler, ist Joe Perry and outside songwriters Jack Blades and Tonny Shaw (both fornerly of camn Yankees). It the second song written the
released as a pranotional single to
"What Kind Love"
of "BBillboard" magazine reviewed the song favorably, calling it the ""first step in major effort to immerse this rock-influenced country star into pop instrunentation la Ton Petty and the late Ray Orbisan (with whon this sang penned), conbined with an appealing vocal, nakes strong argunent for play at these and albun rock formats. From 75 the U.S. "BBlllboard"
realn. She goes on to say that ""twangy
""Life Messy" liner notes. *"what Kind Love"" debuted Hot Country Singles & Tracks for the week of June 27, 1992. Mike Walker
"What Kinda Love"
Kinda Love ""What Kinda Love"" is song recorded Canadian country rock artist Dallas Smith. 4 was released in January 2013 as the fourth single from his debut solo albun, "*Jurped Right In"", It peaked at number 77 the Canadian Hot 100 April 2013. Casadie Pederson ""Top Country"" called relate many people we
the "perrhpp his best release yet."* " She wrote that ""it along this journey of 1ife and love so many things, all of which nean different things to us." The music videa directed by Staphano tttteris/ttol_rrspnnass
<thinloFron provided responses, it clear that the performer of the song "What Kind of Love" is Rodney Crowell.
+
Next, we need to identify the record label of the performer Rodney trowell.</thinde
<tool_call>>
("name": "wikipedia_sena" s_sarcch" "argunents"; ("query": "record label of Rodnay Crowell"))
[rool_cal>><<nn_enndd
<jin_start|puser
stool_response "Rodney Crowell"
and recorded what to be albun for Warner poo-sounding effort coproduced by David Malloy, was rejected by Warner Bros, and was rejected by the never released.
"SStrest Language" That albun,
Nashville-friende record, but negotiated release from his contract and noved to columbia Records. After producing Rosanne Cash's ""Rhythn Ronance"*, Crowell signed to Colunbia Records in 1985. His first albun for that label was reworked ""Street Language"" co-produccdd with Booker Jones and featuring blend of soul and country music. The albun did
requested
"The Radney Crowell Collection"
The Rodney Crowell Collection The Rodney Crowell Collection is the title of the first compilation albun 4 by American country music artist Rodrey Crowell. It was released in 1989 (see 1989 country misic) ""Dianonds
by Crowell's forner label,
Bros. Records, following the huge success Birt"". It features selections from his first three albums that were released under the Harner Bros. Top Country Albuns chart. album is release of Crowell singing '"I Don't 't Have to Crawl" The Crowell-penned tune was recorded by Emmylou
label between 1978 1981. charted
"Rodney Crowell"
charts. 2813, the albun won the Anericana Music Awards' Albun
Year award Crowell and Harris were naned group/duo of the year. On January 26, 2014, Crowell won his second Gramny Award when ""Old Yellow "oon" won the Gramny for Best Anericana Albun. On 11 May 2015, Crawell and Emmyoo Harris released "The Traveling King" Nonesuch Records. Growell released his first albun Records, 'TTarpaper Sky " on April 15, 2014. Crowell co-praduced the record with his long-time fall of 2014, Crewell was
West
collaberator, Steuart Smith.
director the Hank
"Radney Crawell (albun)*
40

---

Preprint
Rodney Crowell (albun) Rodney Crowell is the third, ерonулous studio albun by Amrrcca country nusic artist Rodney Crowel1. was released in 1981 (see
country masic) by Marner Bros. Records and was his last albun that label before switching to Columbis. It was the first albun Crowell praduced by himself. It reached 847 on the Top Country Albuns chart and #165 an the ""Billboard" 280 albuns chart. The songs, ""stars the Water" and "*victin Fool ""stars on the Mater" reached #38 on the Hot Country Songs chart, his highest charting
released singles.
"Duke Records*
Duke Records Duke Records
Anerican record label, started Memphis, Tennessee, 1952 David James Mattis (NDIA progran director and DJ) and Bill Fitzgerald, owners of Tri-State Recording release was Roscon Gordon singing ""Hey Fat Girl"', issued on
+
Company. Their anended to R-181, After forning partnership with Mattis in the sumner of 1952, Don Robey (founder of Houston's s Peacock Records) tock control of Duke. Both labels then headquartered at his Bronze Peacock club 2809 Erastus Street Houston, focusing R&B and gaspel music. Rabey started + subsidiary, Back Beat Becords, respons<sss
R-1, later
ainnstart/asssittte <thinloFrom the provided responses, it is clear that Rodney Crowell has worked with labels like Marner Columbia Records,
we need identify the record label of Rodney Crowell then find the owner of that record label. </think>
+
<tool_call>
("nane": "wikipedia_ searrhhh" "argunents": ("query": "ownr of
Records"))
</tool_call><]]nneee [n_en]>>
<|in_start[Puser
resl__esponse>>/e "Wrnne Bros. Records"
najor record label owned Marner Music + Group and headquar tered in Burbank, California. It was fourded in 1958 as the recorded music division of the Anerican filn studio Warner Bros., and was one
Warner Bros. Records
Records
group of labels owned and operated by larger parent corporations for nuch of its existence. The sequence of companies that controlled Warner Bros. and its allied labels evolved through a convoluted series of corporate mergers and acquisition fron the early 1968s the early 2000s. Over this period, Warner Bros. Records grew from struggling
"Marner Bras. Records"
Records fron EMI as part of its sale to Universal Music Group. Most Parlophone artists (excluding Coldplay and Tinie Tenpah, who placed under Atlantic)
placed under Warner Bros. Records for Us distribution. In Decenber 2013, Dan McCarroll joined the company president. Harner Bros. American najor record label headquartered in Burbank, california. It was founded in 1958 the racorded music division of the American film studio Marner Bros., and was one of group of labels owned and operated by larger parent corporations for
Records Warner Bros. Records
by Warner
Group
"Warner Music Group"
multibillian-dlll annual turnover, WVG employs nore than 3, 588 peaple and has operations in more than 50 countries throughout the world. The company and operates of the largest and most successful labels in the world, including Elektra Records, Warner Bros. Records, Parlophone, and also
Atlantic Records.
Narner/Chagpell Music, Since August 2, 2818, WMG has expanded its business to digital nedia operation through its Bros. recard division
world's largest nusic publishers.
Group. The company time one of its contracted actors, Tab Hunter, scored
acquisition Uproxx
"Warner Bros, Records'
collections from music users. In 1969, Merner-Seven Arts was taken over by the Kinney National Company, headed New York businessman Steve Ross, who would successfully lead the Marner group of companies until his death in 1992. The USV$400 nillion deal created a new conglomerate that conbi the Warner film, recording and music publishing divisions with Kinney's milti-facated holdings. Ross had started the company in the late 1958s while working in his family's funeral businessseeing the the company's cars, were at night, founded successful hire car operation, which he later merged with
opcor tunity
"Warner Bras. Records"
minor player in the music industry to one
top record assets were divested their then owher Time Warner and purchased by private equity group. This Independent company traded as the Warner Music Group and was the world's last publicly traded najor music company before being bought and privatized by Access Industries in 2011. WMS the smallest the thres major international music conglonerates, and Max Lousada oversees recorded music operations Records have included Prince, Cher, Kylie Minogue,
in the world. 2804, music
of the company. Artists signed to < Kinbra, Gaos/taol_respanees inspponsex<<iineedd <lin_start spasstttsssttttt
<thinkoFram the provided responses, it is clear that Warner Bros. Records is awned by Marner Music Group
Next, we need to Identify the owner of the record label Columbia Records. </thins
<tool_call>> ("nate": biciipedda__eercchi "argunents": ("query": "owne Columbia Records")
</tool_call><||n inded]]>
<lin_start/puser
rosl__esponse>' "Colubbia Records"
41

---

Preprint
Milliams, Harry Styles, BTS, Little Mix and John Cena. The Columbia Phonograph Company was founded in Jersey native investars. It derived its from the District of Columbia, where headquartered. At first had a local nonopoly on sales and service of Edison phonographs and phonograph cylinders in Washington, D.C., Maryland, and Delaware. companies, Colunbia produced many comercial cylinder recordings of its own, and its catalogue of pages.
1887 stenographer, lawyer
(18561915) group
the custon
the regional. phonograph
records in
"Columbia Records"
Colurbia Records Columbia Records 1s an American record label owned by Sony Music Entertainment,
division Japanese conglonerate Sony. It was founded in 1887, evolving fron the American Graphophone Company, the successor to the Volta Grephophone Company. Columbla is the oldest surviving brand nane in the recorded sound business, and the second najor company to produce records. From 1961 to 1998, Columbia recordings
subsidiary Sony Corporation America, the
released outside North Anerica under the name CaS Records to avoid confusion with EMI's Columbia Graphaphone of Sony Music's four flagship record
Company. Columbia
"Columbia Records*
with its Epic Records sister to forn the Columbia/Epic Label Group under which it operated as an inprint. part further corporate restructuring, Epic split
Colunbia/Epic Group as Epic took in multiple artists from Jive Records. As of March 2013, Columbia Records is home to 98 artists such Lauren Jauregui, Robbie Willians, Calvin Harris, and Daft Punk. On January 2, 2018, Ron Perry named the CEO of Colurbia Records. Sony and Columbia had cooperated earlier. The SQ Stereo Quadraphonic was developed by engineer Benjamin Bauer of Columbia in cooperation with
July
"Colunbia Records"
four television studios. Columbia Records Calumbia Records is American recard label owned Sony
+ Music Entertai nnent, a subsidiary of Sony Corporation of Anerica, the North American division of Japanese conglonerate Sony.
founded 1887, evolving from the Arerican Graphophone Company, the successor to the Volta Graphophone Company. Columbia is the oldest surviving brand nare in the the second najor company produce records.
1961 1998, Columbia recordings were released outside North Anerica under the name CBS Records to avoid confusion with EHI'S s Columbia Graphophone Company, columbia is one Sony Music's
recorded business,
"Craig Kallman"
Craig Kallman Craig Kallnan is American businessman and forner DJ. He currently the Chairman and + Chief Executive Officer of Atlantic Records. His nusic career began the 1988s as a DJ in Beat Records, independent label he founded 1987. The label was purchased by Atlantic Records, where he has worked since 1991 after the acquisition, becoming its considered of the world's leading investors in records. collection of upproxinately 758,860 records, 150, 600 itens of music menorabilia and 188,800 conpact iin_start|>assiste
Manhattan. He the founder
2805.
[in_nd]>>
<thinkoFron the provided responses, it is clear that Columbia Records is owned by Sony Music Enter tainnent.
So, to sumaarize:
The performer of "What Kind of Love" is Rodney Growell.
Rodney Crowell has worked with labels like Warner Bros. and Columbia Records. The record label of Rodney Crowell includes Warner Bros. and Columbia Racords.
The owner of Marner Bros. Records is Warner Music Group.
The owner of Colunbia Records Sony Music intertainment.//ee </think
<answer>>
.**json
"rationale":
Lave," ' record Bros. and Colurbia Records, which owned Warner Music Group and Sony Music Entertainment, respectively.
"Rodney Crawell perforned song
"answers":
"Marner Music Group",
"Sony Music Entertainment"
</ /
Which country Prince Nikolaus Wilhelm Of Nassau's 's mother is from? (from 2Wiki) Reference Answer: Germany
strtstssssssstt
<thinloTo find the nother's s nationality of Prince Nikolaus Wilheln of Nassau, first, we need to identify Prince Nikolaus wilheln Nassau and then find infornation about mother. </think>
<tsol_call>
(*name":
"arguments"; ("query"; "who was Prince Nikolaus Wi Wilhelm of Nassau"))
42

---

Preprint
//torl_call<(innnc. in_end]>
<jin_star[[2user
<taol_responsed "Prince Nikolaus Wilhelm of Nessau"
Prince Wilhelm Nassau
Nikelaus Wilheln Nassau (28 September 1832 September 1985), was the only son of Willian, Duke of Nassau by his second wife Princess Pauline of Nrttenberg, He narried norganatically in London on July 1868 Natalya Alexandrovna Pushkina (Saint Petersburg,
June 1836 Cannes, 23 March 1913). She the daughter of Alexander Sergeevich Pushkin and wife
Natalya Nikolarvna Goncharova, and a descendant of Abron Petrovich Gannibal and Petro Darashenka, grandson of Mykhailo Boroshanko. She divorced from Russian Ceneral Mikhail Leontievich von Dubelt, by
Hetman of Ukrainian tossacks,
"willian, Duke of Nassau"
names: ""Georg Nilhein August Heinrich Belgicus"", 4 Kirchheinbolanden 28/30 August 1839, Bad Kissingen) was joint sovereign Duke of Nassau, along wi th was also sovereign Prince
Willian,
Wilheim
cousin Frederick Augustus, reigning from 1816 until Nassau-Meilburg from 1816 until its incorporation into the duchy of Nassau. Frederick Augustus died on 24 March 1816 and Milheim inherited the Usingen territories and becene sole sovereign of the Duchy of Nassau. He is the father Adolphe, Grand Duke of Luxenbourg, and Queen Saphia Sweden and Norway, consort of King Oscar II
"Prince Nikolaus Milheln Nassau"
she had daughter. Merenberg. They had three children: Prince Nikolaus Wilheln of Nassau Prince Nikolaus Milheln of Nassau by
George Victor, Prince Waldeck Pyrnont created her Countess
Nassau September 1832 September 1985), the only son of Willian,
second wife Princess Pauline of Wrttemberg. He married morganatically in London on 1 July 1868
Natalya Alexandrovna Pushkina (Saint Petersburg, June 1836 Cannes, March 1913). was
daughter of Alexander Sergrevich Pushkin and wife Natalya Nikalnevna Goncharova, and descendant of Abran Petrovich Cannibal and
"Frederick Augustus, Duke of Nassau"
Frederick Augustus, Duke Nassau Friedrich August, Duke of Nossau, Prince of Nossau-Ussingen (23 April + 1738 in Usingen 24 March 1816 in Wieshaden) was the last Prince of Nassau-Usiigen and, jointly with cousin, friedrich Wilheln Nessau-Weillurs. first male issue and was succended by his cousin's son, Wilheln. Friedrich August, born in Usingen, was Princess Christiane Milhelmine Saxe-Eisenach (daughter of John William III, Duke Sixe-Eisenach). Fron 1744, be lived with his parents 4 Schloss Biebrich in Wiesbaden. He followed a
without surviving
younger son Prince Charles of Nassau-Usingen
"Millian, Duke Nassau"
and also 17th cousin of Millien III of the Netherlands. Milheln the eldest of Frederick Willian, Duke Nassau, and his wife, Burgravine Louise Isabelle of Kirchberg. With the Nassau troops, he was January
-
Battle of Naterloo against Napoleon. - 1816, he succerded his father, Duke Frederick William, as the Prince of Nassau-Weillurg and joint cousin, Frederick Augustus,
involved on the
Coalition's
Buke Nassau
family. When his cousin and co-Duke died 24 March 1816, Wilheln inherited the Usingen territories and becane in_end]>
Nassau-Usingen
<I <thinkoFron the provided responses, it clear that Prince Nikolaus Wilheln of Nassau was the son of Willian, Duke Nassau, and his second wife, Princess Pauline of Wrttenberg.
in wetsstarr/psssstmnn
Next, need identify the nationality of Prince Nikolaus Wilhein of Nassau's nother, which Princess Pauline of Wrttenberg, </think
<taol_call>
("name": "wikipedia_search", "argunents"; ("query"; "whtt the nationality of Princess Pauline of 4
Wrttenberg")>
</tool_cal><(inneedd]]
<|in_start[>user
<tool_respanse> 'Frincess Pauline Murttenberg (18181856)'
Princess Pauline Wrttemberg (18101856) Princess ""Pauline" Friederike Harie of Writenberg, full Marie. Prinzessin von Wrttenberg"" (25 February 1810, Stuttgart, 3 Kingdon of Wrttenberg 7 July 1856, Wiesbaden, Duchy of Nassau) was nember of the House of of Nassau,
nane: *"Pauline
narriage to Millian, Louise was also Duchess consort of Nassau. Pauline an ancestress of the present Belgian, Danish, Dutch. Luxenbourg. Norwegian, and Swedish Royal families. Pauline was the fourth child of Prince Paul of Wrttenberg and his wife Princess Charlotte of Saxe-Hildburghausen. Pauline
Wrttanberg and Princess Mrttenberg by birth. through
"Princess Pauline of Murttemberg (18771965)"
Princess Pauline Wrttenberg (18771965) Princess Paul ine af Nrttemberg (i 19 December 18777 May 1965) the elder daughter of Willian 11 Wrttenberg and wife William Frederick, Prince of Mied. She was for nany years the regional director of the Gernan Red Cross, in western Germany. Pauline was born daughter Willian
Kingdon Writerberg, the by his first wife Princess Harie of Waldeck and Pyrmont (18571882). She was indicted for concealing, since October 1945, a pair important Nazis by military court of the United States. She
at Stuttgart
Writenberg (18481921)
"Pauline Therese Wurtterberg"
43

---

Preprint
excluded from her inheritance in his will. She died at Stuttgart, nine years later, on 16 Harch 1873, years Switzerland, Pauline been very popular, not only Upen her death, Wrttenberg tants gove her nare to many roads and places in Stuttgart, Esslingen, and Friolzhein. Pauline Therese of Wrttenberg Pauline of Wrttenberg Septanber 1880 10 March 1873) of Duke Louis of Wrttenberg and Princess Henriette of Nassau-Keilburg She married her first cousin
having
she showed her subjects but also for the devotion she showed to the
daughter
"Princess Fauline of Nurttenberg (18181856)"
married Willian, Duke of Nassau, eldest of Frederick willian, Prince of Nassau-teilburg and his wife, + Burgravine Louise Isabelle of Kirchberg, on 23 April 1829 in Stuttgart. Pauline and Willian had four inuline Wrttenberg (18101856) Princess "*Paul
children:
Harie
Wrttenberg, full German name: "*Fauline Friederike Marle, Prinzessin von Wrttenberg" (25 February 1818, Stuttgart, Kingdon of Writenberg July 1856, Wiesbaden, Duchy Nessau) was nember of the House of Hrttenberg and Princess of Writenberg by birth. Through her narriage to Willian, Duke of Nassau, Louise was also buchess consort of Nassau, Paul
"Princess Paul inn of Murttanberg (18771965)" "Herrr" and "FFrau" Scholtz-Klink informed the French that asked for Princess Pauline's aid in + 1945, Princess Pauline arranged for then to stay inconspicuously in Bebenhausen, where they were arrested by Allied authorities. Princess Pauline narried on Frederick, Prince of Wied (18721945), son of Willian, Prince of Wied and the spectacularly wealthy Frincess Harie the Netherlands. Her husband's s elder brother was Willian, Prince Albania, Milhelmins of the Netherlands. They had children: Princess - Pauline of Mrttenberg (18771965) Princess Pauline of Wrttenberg resttool_esponse>
1898 Stutigart Millian
she was first cousin of the Dutch
<I iii_staart innsstrttpassissan passistant
<thinkoFrom the provided information, it is clear that Princess Pauline Wrttenberg was Princess of Wrttenberg, and she born the Kingdon Wrttemberg. Therefore, her nationality Gemman
So, mother of Prince Nikolaus Wilhelm of Nossau, Princess Pauline of Wrrteenerrg fron Nrttenberg, + which was part of the German Confederation </thinkk>
<answer>
** jjson
"rationale": "Prince Nikolaus Wilheln mother Princess Pauline Wrttenberg, Princess of Wettemberg, and thus her nationality was German."
+
"answers": "Wrttemberg", "German"
</answer><(in_en//
Where was the place of death of Hayranidil Kadın's 's husband? (from 2Wiki) Reference Answer: Constantinople
sin_start|passistte
<thinkoTo find the place of death of Hayranidil Kadn's husband, first, we need to identify who Hayranidil find who hushand was. Finally, we to place death of that husband, </think>
Kadn was
<tool_call>
("nane": "wikipedia search". "arguments": { ("query": : "who was Hayranidil Ka)n"))
[tool.ccll=<||n_eed]]]]
<|in_start/suse
stool__esponse> "aayaanii Kadn"
Hayrandil Kadn on Novenber 1895
nausoleun Mahmud II, lecated at Divan Yolu street, Istanbul. Hayranidil Kadn Hayranidil Kadn 21 Novenber
drtaky Palace, was
1846 26 November 1895), nnaning 'The excellent heart', was the second wife Sul Abdlaziz of the of Circassian origin, Hayrandil Kadn
Novenber 1846 Kars, Hayrandil married Abdlaziz in 1861 in the Dolnebahe Palace, after his accession to the throne. She "'Ikinci
was given
Five years after narriage, on February 1866,
"Hayranidil
Hayranidil Kadn Hayranidil Kadn 21 Novenber 1846 26 Nevenber 1895), meaning 'The excellent heart',
was the second wife sultan Abdlaziz the Ottonan Empire. Circassian origin, Hayrandil Kadn born on Novenber 1846 Kars, Caucasus. Hayrandil narried Abdlaziz 1861 the Dolmabahe Palace, after his accession the throne. She was given the title of ""kinci Kedn"", " Five years after the narriage, 25 Fehruary 1866, she gave birth to her first child, daughter, Nazma Sul tan, + and two years later on 29 May 1888, she gave birth to her second child,
"Reftardil Kadn*
44

---

Preprint
ehzade Selaheddin Efendi bought nonsion at Ortaky. In 1934 "Reetcardi"" taok the surnane ""Hatgil"" She her nince, Cenile Dilberistan Harmefendi, narried to son, March 1936. Reftardil Kadn Reftardil Kadn June 1838 3 March 1936) was the second wife of Sultan Murad V. Reftardil Kadn was born in 1838 to an Abadzekh notable, Hatkoyuko Hatko. Her family used live the coast of Black sea. She had sisters Terandil Hann and Ceylannelek Harm who were also given to the palace along with
Selaheddin
Efendi. She died in Ortaky Palace, Istanbul
"Reftardil Kadn"
Reftardil Kadn Reftardil Kadn (5 1838 3 March 1936) the second wife sultan Murad v. Reftardil Kadn was born in 1838 to an Abadzekn notable, Hatkoyuke Hatko. her family used to live at the coast of Black Sea. She two sisters Terandil нanm and Gnylanmelek Hann who were also given to palace along with her and were later married to other high officials. She was lovely lady with pink skin,
-
large blue eyes, straight nose, and a round face. She was also honest and goodhear ted. However, in
the later years Refitadill
"Duzdidil Kadn"
Dzdidi1 Kadn Dzdidil Kadn (: 182518 August 1845) was the third wife of Sultan Abdulmejid of the Ottoman Empire. Ozdidil Kadn born in 1825 in North Coucasus. Born
Dian, she nember of übykh + family, Dian. Her father was hm Bey Dian and her nother was an Abkhazian princess belonging to Shervashidze. Upen Yahya Bey's s decision, Aye had
brought Istanbul as young child, where entrusted to the imperial haren, along with her nanny Cinan Hanm, and maid Emine Hann. Here her name according to the costum of the Ottoman courts/tool_.esponseee (rsponse><(in_end]] [in_end]> <
[aa_sttat[passistann
<thinkoFron provided responses, it clear that Hayranidil Kadn was the second wife of Sultan Abdlaziz of the Ottoran Enpire. So, the next step is to identify the place of death of Sultan
bodlaziz.</thiaee
<tool_call>
("nare": "wikieedia_searaee "argunents"; ("query": "where did Sultan Abdlaziz die"))
</toel_.all>>//nneeee
stin_start/Puser
rtool_respense""bbbue "Abdulaziz"
Abdlaziz Abdlaziz (Ottonan Turkish: "bdAbdl- "Azz"", ",. February 18384 June 1875) was the 32nd Sultan - of the Oitoman Empire and reigned between 25 June 1861 and 30 May 1876. lie was the son of Sultan Kahnud brother Abdulnejid
II and succeeded Istanbul), on 8 February 1838, Abdlaziz received an Ottonan education but was nevertheless an ardent aduirer of naterial progress that was in the West. was the Ottonan Sultan who travelled to Wostern Europe, visiting number of important European capitals including Paris,
1851.
at Eyp Palace, Constantinaple (oresent-day
"Sulitn bin Abdulaziz Al Saud"
abroad given time. Saudi Reyal court announced on October
Prince Sultan died dain of an fnspecified illness. According to nedia reports, Prince Sultan had baen battling cancer and had been seeking medical treatment in the United States since nid-June 2811. liee had surgical operation in Now York in July 2011. Unnamed U.S. officials cited by ""The New York Times"" stated that he died at New York-Presbyterian Hospital in Manhattan. His body was taken from New York City to at the Inan Turki
Riyadh on October 2011. His funeral
"Abdulaziz"
having been applied to the throat. Abdlaziz had five consorts: Abdlaziz Abdlaziz (Ottonan Turkish: June 1876) was the 32nd between 25 June 1861 and 30 May 1876. He was the son of sul Mahnud I1 and succeeded his brother Abdulnejid I in 1851. Born at Eyp Palace, Constantinople (present-day Istanbul), on February 1839, Abdlaziz received Ottonan education but never theert ardent adnirer of the naterial progress that was made in the West. He was the first Ottonon Sultan who travelled to Western
"""Abll=Aaz"""
February
Ottoman Enpire and reigned
"Andulaziz"
Balkan provinces. 1876, the April Uprising insurrection spreading anong the Bulgarians. IIl feeling mounted against Russia for its encouragement of the rebellions. While no one event led to his being deposed, crop failure
and lavish expenditures on Ottonen Navy on new palaces which he had built, along with nounting public debt, helped to create an atnosphere conducive ta his being overthrown. Abdl aziz was deposed by his ministers on May 1875. Abdlaziz's death at raan Falace in constantinople few days later was documented as suicide at the time, but suspicions
"Sultan bin Abdulaziz Al Saud"
sul bin Abdulaziz Al Saud tan Abdulaziz Saud (38 December 1925 22 Dctober 2811) O, called
""Sultan al-Khair"" " ""Sultan the Good"") Soudi Arabio, was Soudi noblenan and gevernment
+ afficial. son of King Abdulaziz (Ibn Saud)), he was Saudi Defense minister from 1963 to his death in
2011 and Crown Prince of Saudi Arabia from to 2011. Sultan was born Riyadh, Najd Sultanate + in 1928. He was the 12th son of King Abdulaziz and his nother was Hassa bint Ahred Al Sudairi. He was Sudairi Seven. Princes/tool, (tnnes//too_rssspssssseee an_end|>
secord
[in_start/passiste
<thinkofran provided responses, it
that Sultan Abdlaziz on
Now, we to find place death. </think>
<tsol_call>
("nane": "wikipedia search", "arguments": ("query": "where did Sultan Abdlaziz died, place")> </tocl_cal>><] in_en/>> startt//re "eepuasse""bauulziiii"" "Abdulaziz"
<tool_respense>>
45

---

Preprint
Abdlaziz Abdlaziz (Ottoman Turkish: """Abbl-"Azz"
Ottonan Empire reigned between June February 18264 June 1876) was the 32nd tan Mahmud
was son II and succerded his brother Abduinejid in 1861. Born at Eyp Palace, Constantingple (present-day Istanbul), on 8 February 1838, Abdlaziz received an Ottanan education but was nevertheless an ardent the first üttonan tul who
May
adni of the naterial progress that
in the travelled to Nestern Europe, visiting number of Important European capitals including Paris,
"Abdulaziz"
having applied to the throat. Abdlaziz had five consorts: Abdlaziz Abdlaziz (Ottonan Turkish: """Abdl""Az"" " February 18364 June 1876) was the 32nd Sultan of the Ottonan Empire and reigned 38 May Palace, Constantinople (present-day Istanbul), on 6 February 1839, Abdlaziz received an Ottoman education but was nevertheless ardent adnirer of the naterial the first Ottonon Sultan who travelled Western
25
Mohmud
succaeded
brother
Abdulnejid 1861. Born
progress that nade in the West,
"Sulitan bin Abdulaziz Al Saud"
be abroad at given time. The Saudi Royal court announced on 22 October 2811 that Prince Sultan died at unspecified illness. According to media reports, Prince Sultan had been battling and had been seeking nedical treatment in the United States since nid-June 2811. He had surgical officials cited by
dawn
-
operation
July 2011. that he died at New Park-Preshytarian Hospital in Manhottan. His hody was taken from New York City Riyadh on 24 October 2811. His funeral was at the Inan Turki
York Times"
"Abdulaziz"
Balkan provinces. In 1876, the April Uprising saw insurrection spreading among the Bulgarians. 111 feeling nounted against Russia its encouragement deposed, the which he had built, along with nounting public debt, helped to create an atmosphere conducive to his overthrown. Abdlaziz deposed by his ministers
rebellions. While no one event led to being palares
failure of 1873 and his lavish expendi tures the Ottoran Navy and
1875. Abdlaziz's death Palace in Constantinople few days later was documented as a suicide at the time, but suspicions
"Sultan bin Abdulaziz Al Saud"
October 2811) called + "*Sultan al-Khair*" (, ""Sultan the Good"") in Saudi Arabia, was Saudi noblenan and government official. son of King Abdulaziz (Ibn Saud), was Saudi Defense minister from 1963 to his death
Sultan bin Abdulaziz Al Sultan Abdulaziz
December
2011 and the Crown Prince of Saudi Arabia from 2985 to 2811. Sultan born in Riyadh, Najd Sultanate in 1928. He was the 12th son King Abdulaziz and his mother was Hassa bint Ahred Sudairi. He was in_end|>
the second of the Sudairi Seven. Princes/tool_repprne
jiinssaat(paassisaan
<thinloFran provided responses, it clear that Sultan Abdlaziz died Palacce + Constantinaple (present-day Istanbul).
So. the place of death of Hayranidil Kadn's husband (Sultan Abdlaziz) was raan Palace in Constantinople (present-day Istanbul). </think>>
""Json
"rationale": 'Hayranidil Kadn was the secord wife of Sultan Abdlaziz of the ottonan Empire, and his place death was raan Lonstantinople."
"answers": "raan Palace",
"Constantinople"
</ </answer><< (ineeer
Where does the director of film Wine Of Morning work at? (from HotporQA) Reference Answer: Bob Jones University
st passiitass director
"Wine Morning" works, first, identify 4 directar of the film "Wine of Morning" is. </thinks
<thinkeTo find
<taol_call>
("nane": "Wikipedia_ search"" "argunents": ("query": directed Wine Morning
</toal_call>]] in_end]>
<lin_start[Puser
"eoo__respone>>""iin of Marning"
Norning American feature film + was pardoned according to the Biblical report in place of Jesus Christ by Pontius Pilate. The film is
Marning
Barabbas
subject,
based en novel
Horning by Bob Jones Jr and rected by Katherine Stenholn. on a storny voyage Barabbas writes letter to his friend Stephen and renenbers his time in Galilean Nazareth.
Thare he is friends with the carpenter Josef and his son Jesus one day he neets Irene, the future bride friend Stephanus, falls
46

---

Preprint
"Wine of Horning"
Morning"" of the University President, Bob Jones, Jr., 1958, who long been planning write novel about Barabbas, but did not find the time to write until pleurisy forced him into the hospital bed for two nonths. Six nenths later, the novel wes completed. The novel was finally filmed featured
by Unusual Films; Bob Jones Ir. took
the role of Pontius Pilatus. Mine of Morning the International Congress of Mation's Picture and Television School Directors at the Cannes the first novie win the four najor awards
International Filn Festival.
"Wine of Horniing"
fron the "Nattoonal Evangelical Film Foundation":. Wine of Norning was Katherine Helmond's 's novie debut. Database (English) Wine Norning on mw. www.ualuulliii.o 4 of Morning Wine of Horning is an American feature film from 1955 and has Barabbas as the subject, who
Wine Morning in the Internet
was pardoned according to the Biblical report place Jesus Christ by Pontius Pilata. The film
based on the novel Mine of Norning by Bob Jones Jr and directed by Katherine Stenholn. On storny
voyage Barabbas writes letter to his friend Stephen and renembers his time in Galilean
"The Mine of Summer"
The Wine Sunner The Mine of Sumer is 2813 romantic drana film written, directed and produced by
+ Maria Mattnoli, starring Elsa Pataky, Soria Braga, Ethan Peck, Nojaa Ninri, Bob Wells and Harcia Gay pursuit of his childhood drean of becoming an actor. While studying acting under the tutelage of Shelley (Harden). he becones engrossed (Chaw) leaves
Harden. Janes (Peck), the age of quits his law
Carlo Lucchesi's play, Tinto Verane, which set in Spain. Janes girlfriend hin, and he spontaneously flies to Spain, where he encounters the playwright Lucchesi (Wells)
"Bab Jones University"
Jones, Jr. selected speech teacher, Katherine Stenholm, as the first director. Although she had no experience in cinena, she tock sunner courses at the University of Southern California and received S personal instruction from Hlollywood specialists, such as Rudoiph Sternad. Unusual Films has produced festure-length films, each with evangelistic emphosis: ""Wine of Morning"", "*Red Runs the + River"", **Hlame in the Wind"", ""sheffe"", ""Beyond the Night"", "*the Printing"', and "Milltom Pride"". ""Wine of Norning"" (1955), based on novel by Bob Jones, Jr., represented the United States + at the Cannes Filn Festival. The first four films are historical dranas//tool responss><] in_end]> <lin_start passistttts <thinloFrom the provided responses, it is clear that the director of the film "Wine of Morning" is + Katherine Stenholn.
Next, we need to find out where Katherine Stenholn </think/
<taol_call>
(*name": aikkipedia_seartie "arguments"; ("query"; "where did director Katherine Stenholm work"))
</tocl_call><( <ald>>)in__en|>>>>>>>>
<lin_start/>user
<taol_response> "Katherine Stenholn"
Katherine Stanholn Katherine Corne Stenholn (June 19, 1917 Novenber 3, 2015) was an Anerican film director and the founding director Unusual Films, the production company 8ob Jones University. Katherine Corne born and reared Hendersonville, North Carolina. As high school student during the Depression, she supplemented her family's s incone by writing novie reviews for local newspaper. Rejacting college scholarship Wellesley, Corne attended the fledgling Bob Jones College in Cleveland, Tennessee, after an evangelist convinced her that Christian young person should attend Christian college. BIC, she najored in speech
"Katherine Stenhoin"
of American tinema training and the film's 's frank religious nessage would ""provide revealing contrast other Commnistt-doinne stroke in the Soviet Union while taking scenic footage in preparation for another feature-length filn. She retired as director of Unusual Filns but continued to teach at BJU until 2881. Stenholn died Novenber 2015 the age of 98, Katherine Stenholn Katherine Corne Stenholn (June 19, 1917 Novenber 3, 2015) was an Anerican film director and the founding director of Unusual Filns, the Jones
to the entries from Russia
countries. e* 1986, Stenholn suffered
production company
"Katherine Stenhcin'
becane a private student of Bob Jones, Jr., eventually helping him direct Shakespearean plays. After BJC speech faculty attending graduate school at Northwestern University for twelve sumners. During this period she narried Gilbert R. instin they
earning undergraduate degree, she served
Stanholn (191589), who becane an influential administrator to Greenville and becane Bob Jones University, Bob Jones, and Jr asked Stanholn to head a nowly conceived campus film production company, Unusual Filns. Stenhalm then University Southern california,
son. In
1958, after the college
attended
filn school
"Katherine Stenholm"
making important professional contacts and serving an internship with Stanley Kraner. Stenholm was a only handful of in the United States direct feature
*'becane + filns." Through her career she produced seventy-two films of various types including sermon filns.
quick learner
religious documentaries, mronotional films, and milti-image presentations. She directed five
feature-length religious films, all costune dranas: The National Evangelical Film Foundation naned Stenholn Director of the Year in 1953, 1955, and 1963; and her fevorite film, ""Sheffey", received a
Silver Medallion aaar from the International Film and Television Festival New York. In
"Katherine StenhoIn'
47

---

Preprint
1958, at the height of the Cold War, the University Film Producers Association selected ""Mine of International Congress Directors at the Cannes Film Festival. and Stenholn was the keynote speaker the eccasion. A U.S. State Department tnent official who briefed Stenhaln told her there had been round of applause when the Department discovered that 8JU had been chosen represent the United States because "BBab Jones University is one school about which there is worry! The selection comiittee thought ""Wine of denonstrate excellence</ue hatpnneee//ooorrsossssseee responsseppsseiin
Norning"" as
Television
Norning""
<
slin_strr(paussitaan
<tool_calj>
("nane": mikkpedia_search"" "argunents": ( ("query"; "where did Katherine Stenholn work as f'lm director"))
#ttel_call><(inneeddl]
<lin_start|Puser
staol_respense> "KateeeneeeeKaaariinnn Stenholn"
Stenhoin 19, + director and the founding director of Unusual Films, the production company of Bob Jones University. Katherine Corne was born and reared Hendersonville, North Carolina. As a high school student during the Depression, she supplemented her family's incore writing novie reviews for local neuspaper. Rejacting a college scholarship to Wellesley, Corne attended the fledgling Bob Jones Christian young person
Katherine Stenholn
3, 2015) was
College in Cleveland, Temnesse, after evangelist convinced should attend a Christian college. At BJC, she najored in speech and
"Katherine Stenholn"
""provide revealing contrast + to the entries from Russia and the other Communist-doninen countries. In 1986, Stenholn suffered another feature-length filn. She retired as director of Unusual Films but continued teach at BJU until 2681. Stenholn died Novenber 2015 at the age of 98. Katherine Stenholm Katherine Corne Stenholn (June 13, 1917 Novenber 3, 2615) production company of Bob Jones
tinemo training
religious nessage
stroke in the Soviet Union while taking scenic footage in pregaration
Anerican filn director and the founding director of Unusual Filns, the
"Katherine Stenholm'
becane private student
eventually helping hin direct Shakespearean plays. After earning her undergraduate degree, she served on the BIC speech faculty while attending graduate school Nhwteees University tuelve sumers. During this period narried Gilbert Stanholn (191589), who became an influential administrator at the institutionn; they had one son. In 1950, after the college noved to Greenville and becane Bob Jones University, Bob Jones, Sr. and Jr. film production company, Unusual Filns. Stenholm then attended summer film school at the University of Southern California,
asked Stenholn to head newly conceived
"Katherine Stenhoin"
internship with Stanley Kraner. Stenholm + quick learner and soon "'became one of only a handful of women in the United States to direct feature filns."" "* Through
making important professional
and
types including sermon religious docunentaries, pronotional films, and multi-inage presentations. She directed five
produced seventy
National Evangelical Stenholn Director of the Year in 1953, 1955, and 1963; and her favorite film, ""Sheffey", received
Feature-length religious filns,
costume dramas:
Silver Medallion award from the International Film and Television Festival of New Yark. In
"Katherine Stenholm'
1958, at the height of the Cold War, the University Film Producers Association selected ""Wine Morning"" " as its submission the Internationsl Congress of Notian Picture and Television School keynote speaker
Directors the
Filn Festival, State Department official who briefed Stenholn told her there had been round of applause when the Repartment discovered that
Sterholn
chosen to represent university is one school about which there is worry!" The selection comiitted ttei thought "WWine of Morning"" would denonstrate the excellences/tool. Lotptooooosssseeeeeieneeeee.
States
jin_start|aassistaan
<think>Fron the provided responses, it clear that Katherine Stenholn worked at Bob Jones University as the founding director Unusual Films, which is the production company of Bob Jones University.
Addi Additionally, she also taught Bob Jones University until 2881.
So, where Katherine Stenholn worked includes Bob Jones University </thinke
<answer>
json
"rationale":
"Katherine Stenholn directed the film "Wine of Morning." and she worked at Bob Jones University."
"answers": "Bob Jones University",
"Unusual Films"
inneerr><[in_endi
48

---

Preprint
How Ceephax Acid Crew and Squarepusher's music similar? (from HolpoiQA) Reference Answer: drum and bass electronic musician
stinsstar|passsistan
<thinkeTo find how Ceephax Acid Crew and Squarepusher's music are sinilar, first, we need to search who Squarepusher s/thinko
Ceephax Acid
<tool_call>
(*nome": "wikipedia_sene search""" "argunents"; ("query": "wha is Coephax Acid Crew"))
</tocl_cul>><] in_en|>>
<jin_stor(/uuser
reoo__esppnns>> "Cepphae""Geephax
Cesphax Acid Crew Coephax Acid Crew (real name Andy Jenkinson) is a British acid techno and drum and bass electrenic musician. Jenkinson is known simply by pseudonym Ceephax, which reference the BBC teletext service Ceefax. He the brother of Squarepusher (Ton Jenkinson). Ceephax's nusic from 1997 to 2862 and beyond used nainly vintage (especially Roland) drum nachines and synthesizers such the TB-383. This music aften recorded anto cassette tape deck. The music released vinyl records and cassette tape on underground labels such as Breakin' Records, Lo Recordings and Firstcask.
"Ceephax Crew"
old school house, scid house. techno, drum and bass and gabber all in Ceephax's distinctive style. He has videos predoninantly using video equipment and early conputer
also produced several
animation. Caephax Acid Crew Grephax Acid Crew (real name Andy Jenkinson) is British acid techno
drum and bass electronic musician. Jenkinson is also known simply by the pseudonym Ceephax, which is reference to the BBC teletext service Ceefax. He is the brother of Squarepusher (Ton Jenkinson).
Ceephax' s music from 1997 to 2802 and beyand used mainly vintage (especially Roland) drum machines
synthesizers
Squarepusher
Squarepusher Squarepusher is the principal pseudonyn adopted by Ton Jenkinson (born 17 January 1975), an number of influences including drum
English electronic musician. compositions acid hause, jazz and electroscoustic nusic. His recordings are typified by a conbination of digital signal processing. is
electranic sound sources, of Ceephax Acid trew (Andrew Jenkinson). Ton Jenkinson grew in thelnsford, Essex. The first school he attended was affiliated with Chelmsford Cathedral giving hin exposure to organ music, which he has subsequently acknowl
instrumental playing
influence his work. took
"Ceephax Acid Crew"
This primitive Acid House aesthetic and nethodology countered the growing popularity of the computer and corpact disc releases of the time. Rephlex Records and warp Records also released remixes of Squarepusher by Ceephax around this tine. From 2003 onwards as well as his nore typical Acid style also released
samplers, Commodore Aniga digital synthesizers, again displaying an interest late 1988s and early 1998s dance music. His set on Marry Anne Hobb show Breezeblock in 2003 and more recent tracks such as ""Castilian" and ""Arcadian"" alse
bass songs
early
"Ceephax Acid Crew"
indicate an interest in Chiptune music. Fron 2007 he has had full releases on the Rephlex and Planet Mu record labels. He also continues release Firstcask and other small independent labels such
+ WM. Bugklinik and his own label Waltzer. Ceephax is popular with some electronic music fans for his apart
live performances. Crephex's
popular Laptop style of ""live"" electronic performance by exclusively using only analogue and early digital equipnent and occasionally an Amiga computer. Hardware frequently used include TB-303, TR-909, TR-707, SH-181, Kenton Pro-2800,
Yanaha R57808. These sets range
stin_start/aa (Passsisttnn
<thinkoFron the provided responses, it is clear that Ceephax Acid Crew and Squarepusher are both electronic musicians, and they are brothers (Ceephax Acid Crew being the pseudonym of Andy Jenkinson Squarepusher being the pseudonyn Ton Jenkinsan).
Next, need to identify the characteristics of their music find how their music + simi minllar.//thhe </think>
<taol_call>
("nare": "wdkipedia_searcht "arguments": ("query"; "how Squarepusher's similar electronic 4 music"})
</tool_.alll<(]nneed)
<lin_start/suser
stoo__regenneeSuurrquuset
Squarepusher Squarepusher is the principal pseudonya adopted by Ton Jenkinson (born 17 January 1975), English electrenic musician. compositions draw number influences including drum bass, acid house, jazz and electroacoustic music, Bis recordings are typified by combination of electrenic sound sources, live instrunental playing and digital signal processing, is the brother of Ceephax Acid Crew (Andrew Jenkinson). Ton Tenkinson grew in Cheinsford, Essex. The first school he attended was affiliated with Chelnsford Cathedral giving hin exposure to organ music, which he has subsequently adknowlndged
influence his work. He tock
Squarepusher
49

---

Preprint
Squar epusher synth bassline in "'The Modern Bass Guitar"" was triggered from electric bass using the midi bass studio
residence in Essex. pieces ""Thene Fron Sprite"", "'Bubble Life"", "VVacuu Garden"', "CCrccleaaee 2 and ""Orient Orange"" were all nade in early 2805. All of these tracks are based on live drumning tracks, which Tan had planned quite neticulously beforshand, contrast to simlarrl realised tracks "MMusi Is Rotted One Note "Hello Neow"", ""Planetariun" ""Rotate Electrolyte" and "*Plotinus" were made track ""Hello
systen. January 2005 saw re-establishment
over
"Nothing Records"
aboard Nothing via licensing deal with Warp. Squarepusher 1s Tor Jenkinson of Chelnsford, Essex, playing in local bands his youth, began taking an interest in techno and house nusic the early 1998s. Anongst his influences were Aphex Twin and future Nothing Records lebel-nate Luke Vibert, perticularly with his work as Plug. Jenkinson said of and, anongst the road noise and chatter, heard
England. After learning bass gui
Plug"s 's track "Military Jazz, ""This track what thought was sone sort of hip-hop track being played by a band. As the track progressed,
"Intalligent dance music"
Intelligent dance music Intelligent dance music (comonly abbreviated IDM) form of electranic
music originating in the early 1998s. Relative to other forms of electronic and rave music, IDM cane characterized better
hone listening than dancing. Inspired by variety sources, including Detroit techno, acid house, anhient music, and breakheat, IDM tended to rely upon individualistic experinentation rather than adhering to charasteristics associated with specific genres. Proninent artists associated with the include Aphex Twin, -Ziq, the Black Dog, the Orb, the Future Sound of London, Autechre, Luke Vibert, Squarepusher, Venetian Snares
apparently Bjrk's favourite sengs"". This period also produced the ""aaximum Priest"" e.p. ""Our Underwater Torch" was partially inspired by a developing obsession Ton had for the sounds of water. this piece was obliquely inspired Russian director Andrei Tarkevsky. The organ in this piece is triggered directly from an electric bass. "'Derathlon Oxide"" carried
the films ""Solaris"" and ""stalker"
"'fly Street" " and ""Varkatspe" from ""Budakhan Hindphone"* and features Gamelan gong. The record also contains remi xes by Luke Vibert, Autechre and Matthew Yee King. Very few live <|in_start ppassssttt
ideas initiated
<thinkoFrom the provided responses, it clear that Squarepusher's music draws influences including drum and bass, acid house. jazz. and electroacoustic music, and it characterized combination + of electronic sound sources, live instrumental playing, and digital signal processing.
Now, let's identify the characteristics of Ceephax Acid Crew's music and compare it with Squarepusher's s/<ttiink>
<tool_call>
(*name": "wikipedia_sena search"" "arguments"; ("query"; "how is Ceephax Acid Crew's music similar to
electronic music");
</toei_calll<iin_eee in_end|>
jin_start[>zuer
rtool_respons>>"Geppp Acid Crew"
Ceephax
Ceephax Acid (real nane Andy Jenkinson) British techno
electronic musician. Jenkinsen is also known simply by the pseudonyn Ceephax, which is reference to brother Squarepusher (Ton Jenkinson). Crephax's
service Crefax. fron 1997 to 2802 and beyond used mainly vintage (especially Roland) drun nachines and synthesizers such the TB-303. This music was often recorded onto cassette tape deck. The music was released on vinyl records and cassette tape underground labels such Breakin' Records, Lo Recordings and Firstcask.
BBC
"Ceephax Acid Crew"
old school
acid house, techno, and bass gabber all in Ceephax's distinctive style. He has also produced several music videos predominantly using old video equipnent and early computer Ceephex Acid
anination. Ceephax
British acid and drun and bass electronic nusician. Jenkinson is also known simply by the pseudonym Ceephax, which is a reference to the BBC teletext service Ceefax. He is the brother of Squarepusher (Ton Jenkinson).
Andy Jenkinson)
Ceephax's music fron 1997 to 2802 and beyond used nainly vintage (especially Roland) drun nachines
+ and synthesizers such as the
"Acid jazz"
these recordings and who exphasised groove-based approach to music. Acid jazz elenents of jazz, primarily live music, it was claser to jazz than any other dance style, but focus on naintaining groove allied with funk, hip-hop,
+ funk and hip-hap. Because of its existance as a percussion-heavy,
G and dance music. The style is characterized by danceable grooves and long, repetitive conpositions tians..ti....... rhythr section tof with a drun set additional percussion), vocalist that may sing and and often DJ. Acid jazz has its origins in the 1950s,
Typical ensembles include horns,
"Acid techno"
50

---

Preprint
Acid technn Acid techno (also known as simply ""acid"") is genre of teching that developed out of late ""squelching" synthesizer distinctive instrunents created nainly by Roland, such the SH-181 and TB-303 for bass and lead sounds, and the TR-787, TR-888, and TR-989 for percussion. ""Acid" specifically refers to the use of the Roland 18-303, any other synthesizer designed enulate unique sound. Mhile nodern electronic instrunents have nenory banks of different sounds or ""patches"", these machines had to be manually set adjusting control knobs.
1980s Chicago acid hause, particularly
obtained by using very
"Ceephax Crew"
indicate an interest in Chiptune music. From 2097 he has had full releases on the Rephlex and Planet Mu continues to release Firstcask and other small independent labels such WM, Bugklinik and his own label waltzer. Ceephax is popular with some electronic music fans for his live perfornances. Ceephax's s shows are set apart from the popular Laptop style of ""live"" electronic perfomance exclusively using only analogue and early digital equipment and ectasionally Aniga computer. Hardware frequently used include a TB-303, TR-909, TR-707, SH-101, Kenton Pro-2800, and Yanaha R57808. These range rran///oo_reessose=eiii....
record labels.
jinnstaat[>asssitee
<thinloFrom provided responses, it clear that Ceephax Acid Crew's music characterized acid techno and drum and bass, often using vintage synthesizers and drun nachines like the TB-383.
+
Now, let's identify the similarities between the music of Ceephax Acid Crew and Squarepusher. </think>
<tool_call>
("nane": "wikipedia_searte searchh" "argunents": ("query": similar about music Ceephax
4 and Squarepusher"))
//tool_cal><]] in_end|>
iin_start[>user
staol_respunser "Cesphax
Ceephax Acid Crew Ceephax Acid Crew (real name Andy Jenkinson) is British acid techno and drum and bass electrenic musician. Jenkinson is also known simply by the pseudonym Ceephax, which is reference the BBC teletext service Ceefax. He the brother of Squarepusher (Ton Jenkinson). Ceephax's 's nusic fron 1997 to 2862 and beyond used nainly vintage (especially Roland) drum nachines and synthesizers such the TB-383. This music aften recorded anto cassette tape deck. The music released vinyl records and cassette tape on underground labels such as Breakin' Records, Lo Recordings and Firstcask.
Squareausher
Squar epusher Squarepusher is the principal pseudonya adopted by Ton Jenkinson (born 17 January 1975), English electronic musician. compositions draw on number influences including drum and bass, acid house, jazz and electroacoustic music. His recordings are typified by a combination of electrenic sound sources, live instrunental playing and digital signal processing. He is the brother of Ceephax Acid Crew (Andrew Jenkinson). Ton Jenkinson grew in Cheinsford, Essex. The first school he attended was uffiliated with Chelasford Cathedral giving hin exposure to organ music, which he has influence his work. He tack
subsequently adknowlndged
"Crephax Acid Crew"
old school house, acid house, techno, drum and bass and gabber all in Ceephax's distinctive style. He has videos predominantly using video equipnent and early computer anination. Ceephax Acid Crew Ceephax Acid Crew (real nane Andy Jenkinson) is a British acid techno known simply by pseudonym Ceephax,
also produced several
bass electronic musician. Jenkinson is reference to the 8BC teletext service Ceefax. He is the brother of Squarepusher (Ton Jenkinson). Ceephax's ss music from 1997 to 2092 and beyond used nainly vintage (especially Roland) drun nachines synthesizers such the
"Ceephax Acid Crew"
This primitive Acid House aesthetic and nethodology countered the growing popularity of the computer and corpact disc releases of the tine. Rephlex Records and warp Records also released remixes + of Squarepusher by Ceephax around this tine. Fron 2003 onwards as well as his nore typical Acid style on sanplers, Commodore Aniga digital synthesizers, again displaying an interest late 1988s and early 1998s dance music. His set en Marry Anne Hobe show Breezeblock in 2003 and nore recent tracks such as ""Castilian" and ""Arcadian"" alse
has also released
bass songs
early
"Selection Sixteen"
Selection Sixteen Selection Sixteen is a 1999 albun by Souarepusher, released on Warp Records. According to the C, the cover-art is from film called "'Acid Trayners 4"", which
to different pictures of an oscilloscope. Throughout the record, the bass, baoked to an audio/mids convertar, drus parts.
+
plays important part in the composition synth Sixteen"" include "AAnti-Greylord Protection Schene Prelude" which was originally released as segarate EP with slightly different track order. ""Ceephax Mix"" was created by Squarepusher's brother, Andy Jenkinson, aka Ceephax Acid Crew. On the original release,</tood <tttooo_reeppnseeee im_end] <(jn_start/aassistaa
releases of ""Selection
<thinkoFron the provided responses, it is evident that both Ceephax Acid Crew and Squarepusher are involved with electronic nusic genres like acid house and drum and bass, and they use vintage synthesizers and drum nachines.
Specifically, both artists elenents acid hause and drum and bass, and they utilize vintage elactronic instrunents such as the T8-303, 18-303,</thine> </think>
51

---

Preprint
<answer>
.** "json
"rationale": "Cerphax
Squarepusher musicians produce music in genres such as acid house and drum and bass, and they vintage synthesizers and drun nachines."
"answers":
"clectronic music",
"acid house",
]
<f //nswer><(in__ndf
What is the primary male hormone derived from? (from Bamboogle) Reference Answer: cholesterol
jtnnssatttasssistent
<thinkoTo find inary male hormone derived fron, first, we identify prinary
+
hornone is and then find its derivation. Let's start by identifying the primary male hornana.</thee
<taol_call>
("nane": Bakkipedia_searchtt "arguments"; ("query"; "what is the prinary male hormone"))
</tool_cal>><] in_end]>
<lin_start[2user
Testosterone Testosterone is pary male sex hormone
anabolic steroid. In nale hurans, testosterone plays key role in the developnent of nale reproductive tissues such testes and prostate, as well as pronoting secondary sexual characteristics such as increased nuscle and bone the growth body hair. addition, testosterone invalved in health and well-being, and the prevention of astesporosis. Insufficient levels of testosterone in men may lead to Testosterone steroid fron the androstane class containing keto and hydroxyl groups at the three and seventeen
abnornalities including frailty
Testasterone
Testasterane
most vertebrates. Testosterane first appeared in gnathostomes (jawed vertebrates). Agnathans (jauless vertabrates) such as lampreys do not produce testosterone but instead use androstenedione slightly different form called l1-Ketotestosterone Its counterpart insects is ecdysone. The presence of these ubiquitous steroids in wide range of animuls suggest that sex harmones have an evalutionary history. Testasterane Testasterone is the prinary male hormone and anabulic steroid. In nale hunans, testosterone plays key role in the develogment of male repraductive
classical
androgen receptor
male sex hormone. Fish nake a
Andragen
Andragen An andregen (Frun Greek ""andr-"", the sten of the word neaning ""nan"") is any natural or synthetis staroid
characteristics regulates development maintenance development of the vertebrates binding to androgen receptors. This includes the enbryological
primary male sex organs, and the development male secondary sex characteristics at puberty. the adrenal glands. Androgens both boys and girls during puberty. The najor androgen in nales is testosterone. Olhydrotestosteron equal iaportance
Androgens synthesized the testes, the
(DHT) androstenedione
developnent.
causes
sex characteristics. The nain male hormone is testosterone, which is produced nainly by the testicles. It testosterone,
is dähydrotestosseerne (DHT), prostate. About 20, 20,000 protein coding genes are expressed in hunan cells and almost 75% of these genes are expressed the nermal prostate. About 150 of these genes are more specifically expressed specific. The carrespending specific
predominantly regulates
in the prostate with about proteins are expressed in the glandular and secretory cells the prostatic gland and have functions that important for characteristics semen. Examples of
being highly
Testosterone
The nale generative glands also contain Sertoli cells, which require testasterone for spermttgenasis. nuch of transported bound to specific plasm protein, sex hornone-binding globulin (SHBG). In nales, testosterone synthesized prinarily in Leydig cells. number Leydig cells turn regulated by luteinizing hornon (LH) and follicle-stimilatiee hornone (FSH). In addition. the anount of Lestosterone produced by existing Leydig cells is under the control of Lil, which regulates the of testasterone synthesized is regulated
Like hormsnes, testosterone supplied target tissues in
expression of 17-hydroyyterroid dehyydoggnase. The
slin_star|/>assi
<thinloFrod rrn the provided responses, it is clear that the prinary male hormone is testasterone.
Next, we need to find out what testosterone is derived fron.</thinke
<taol_call>
52

---

Preprint
("nane": vakipediassearrchh, " ("query": "what testosterone synchesized from"))
</tocl_call><[ end]n]>>>
slin_start[buser
seol__resonse>eesstttte
The nale generative glands also contain Serteli cells, which require testosterone for spernatogenesis. most hormones, testesterone supplind target tissues in hormone-binding globulin (SHBG). In nales,
of
transported bound to specific plasna protein, testosterone synthesized prinarily in Leydig cells. The number of Leydig cells in turn is (HSH), addition, the anount of testosterone produced by existing Leydig cells is under the control of LH, which regulates the testosterone synthesized regulated
regulated by luteinizing hormone (LH) folliclesstimlating
expression 17-hydroxysteroid dehydrogenase.
the
Testasterone
reticulun yield variety of steroids.
3-hydroxyl group exidized by 3-hydraxysteroid dehydrogenase to produce undrostenctione.. In the final and rate limiting step, the
4
C17 keto group androstenedione is reduced by 17-hiydroxysteridd dehydrogenase to yield testosterone. produced by the testes in while the adrenal glands account for most of the renainder. lestosterone is also synthesized far smallar total adrenal glands, thecal cells of the ovaries, and, during pregnancy, by the placenta. In the testes, testosterone is produced by the Leydig cells.
The largest amounts testosterone (>95W)
quantities
Androgen
testasterone naintaining normal prostate weight
mass (this neasure epithelial cell function stimulation). Whereas IHT was equally potent as testesterone at preventing prostate cell death after castration. Androgens have also been found signal through nenbrane androgen receptors, which distinct from the classical nuclear androgen receptor. Androgens synthesized from cholesterol and are produced prinarily the gonads (testicles and ovaries) and alsg the
adrenal glands. The testicles produce much higher quantity than the ovaries. Conversion testosterone to the nore potent DHT occurs the prostate gland, liver, brain and skin.
Androgen
Androgen andregen (fron
of the word reaning ""ran"") any natural synthetic steroid hornone which regulates the development and naintenance of sale characteristics in vertebrates by binding to andragen receptors.
""andr-"*,
enbryolagical developnent prinary male sex organs, and the developnent of male secondary sex characteristics at puberty. Androgens are synthesized in the testes, the ovaries, and the adrenal glands. Androgens increase in both and girls during puberty. The najor androgen in nales is testosterone. Oihydrotestosteron (UHT) and androstenedione are of equal importance in male development. DHT "*in utero" causes
includes
Testasterane
Testasterane Testosterone the prinary male hormone
anabalic steroid. In nale testosterone plays key role in the developnent of nale reproductive tissues such as testes and prostate, as well pronoting secondary nass, and the growth of body hair. In addition, testosterane is involved in health and well-being, ostesporosis. Insufficient abnornalities including frailty and bone loss. Testosterone steroid fron the androstane class + containing a keta and hydroxyl groups at the three and seventeen</ton httprceeec/oooorssssssee//// siinsstattpasssittan
cheracteristics such increased
the prevention
testasterone men may
<taol_call>
("nane": Vikkpedia_searrch "argunents": ("query": "what is the prinary
testosterone"))
[ool__ccll><(nn_eedd
<lin_start/Puser
stool_reseneeeTessttteroo
Testasterane Testosterone is the prinary male hormone anabolic steroid. In nale hurans, testasterone plays a key role in the developnent of mle repraductive tissues such as testes and prostate, well pronoting secondary sexual characteristies such increased nuscle bone mass,, and the growth of body hair. In additioon, testosterone is involved in health and well-being. prevention esteoperosis. Insufficient abnornalities including frailty and bone loss. Testosterone steroid fron the androstane class containing keto and hydroxyl groups at the three and seventeen
testosterone nen may
Testasterone
The male generative glands also contain Sertoli cells, which require testosterone for spernatogenssis. Like most hormones, testosterone is supplied to target tissues in the blood where nuch of it transported bound specific plası protein,
harmene-binding globulin (SHBG). In nales, testosterone is synthesized prinarily in Leydig cells. The numbe of Leydig cells in turn is regulated by luteinizing homone (LII) follicle-stimlating
(FSH). addicion, the of testosterone produced by existing Leydig cells is under the control of L4, which regulates the expression 17-hydroxysteroid dehydrogenase. anount of testosterone synthesized is regulated by
the
Testosterone
53

---

Preprint
Testasterone is observed in nost vertebrates. Testosterone and the classical nuclear andragen receptor appeared in gnathostomes (jawed vertabrates). Agnathans (jawless vertabrates) nale harnone. Fish nake slightly different form called 11-ketotestosterone. Its counterport in insects is ecdysone. The presence of thase ubiquitous steroids wide range of aninals suggest that hormones ancient evolutionary history. Testosterone Testosterone 1s the prinary male sex hormone and an anabolic steroid. In hunans, testosterone plays key in the develogment male repraductive tissues
as lampreys
do not produce testosterone but instead undrostenedione
Andragen
Andragens are metabol mainly
liver. testosterone level (hypogonadi sn) nen may treated wi th testosterone adninistration. Prostate cancer may be treated by renoving the major source of testosterene: testicle renoval (orchiectony); or agents which block androgens from accessing their antianirogens. Androgen An androgen (from Greek ""andr""', the sten of the word neaning ""man"" ") is any natural or synthetic steroid hornne which regulates the development and naintenance of nale characteristics in vertebrates by binding androgan receptors. This includes the embryological developnent of the prinary nale sex organs, and the development of male secondary sex characteristics
Testasterone
reticulun to yield variety of C steroids. In addition, the 3-hydroxyl group is axidized by 3-hydroxysteroid dehydrogenase to produce androstenacione..
final and rate limiting step, C17 keto group androstenedione is reduced by 17-hydroxysteroid dehyyrrggeaase to yield testosterone.
The largest amounts of testosterone (>95V)) are produced by the testes in men, while the adrenal for of the renainder. Testosterane is alsa synthesized far smaller total quantities in wonen by the adrenal glands, thecal cells of the ovaries, and, during pregnancy, by the testes, testosterone produced by Leydig cells.</too respose_
glands
placenta.
<I din_start[passiaan
testosterone is synthesized primarily in Leydig 4 cells of the tastes and it is derived from androstenedione cholesterol </thinko> //ttstt
<thinkofrom provided responses, it
<taol_call>
("nane": "WWikipedia_eee" searchh,, "argunents": ("query":
prinary source andrestenediane "})
[ttol_call><(in_enn]]
<|in_start[Suser
Andrastenedione released
by thece cells. Conversion andrastenedionn: estrone
requires the enzyne aronatase. Androstenedions is substrate for estrogen production in granulesa cells which produce aromatase. Thus, theca cells and granulosa cells work together form estrogens. Androstanedione is 5-reduced netabolite of 4-androstenedione which serves an internediate in the biosynthesis the andregen and neurosteroid androsterane. Levels are normally 30200 ng/dL. (1.07.0 nnol/L) in fenales and ng/dL (1.45.2 nnol/L) nales. Androstenedione
increase serum testosterone levels over an eight-hour period in nen when taken as single oral
Andrastenedione
detection. Androstenedione
comren precursor of androgen estrogen hormones. Andrastemedione can be biosynthesized in one of two ways. The prinary pathway involves conversion of 17-hydroxyeregnenolom
by way 17,20-lyase, androstenedione via the enzyne s-hydroxysteroid dehydrogenase. The secondary pathway involves nost often precursor to cartispl, to andrastenedianss directly
subsequent conversion
conversion of 17-bydroxyrrgeestrrom by way of 17,28-lyase. Thus, 17,20-lyase is required for the synthesis of androstenedione, whether imeeiately or one step renoved. Androstenedione is produced in the adrenal glands and the gonads. production adrenal endrostenedione governed by
Androstenedione
adrenocorticctroohii hornone (ACTH), whereas production of gonadal androstenedione is under control by glards ovaries produce half
jonadotropins. prenenopausal wonen, the the total androstenedions (about ng/day), After nenopause, androstenedione production is about halved, due prinarily to the reduction of the steroid secreted by the ovary. thelesh
undrostenedione is the principul steroid produced by the postnenopousal ovary. Sone androstenedione is alsa secreted into the plasna, and my be converted in peripheral tissues to tastostarana and Conversion of androstenedione to testosterone requires the enzyme 17-hydrokysteroid dehydrogenase.
estrogens. Androstenediooe
either testosterone
Androstenedione
Andrustenedione Andrastenedione, d-androsteneddione (abbrevisted as -dione), androst-4-ene--3 1al--eeee-1,1-iinmee 17-dione, is endogenous weak androgen steraid hormane and internediate in the biosynthesis estrone 37- to androstenediol androgens, as well as of estrogens like estrone, in the body. In addition to functioning as an endogenous prohornene, androstenediane also has weak andragenic activity in its right. Androstenedione has been found to possess sone estrogenic activity, similarly to other OHEA netabolites. However, contrast androstenedial, affinity far
testosterone from anddrrrepiandoosterrom (DHEA), It closely related
Androstenediore is precursor of testosterone and other
Andrastenediane
54

---

Preprint
the brand name Metharmen-F and in combination with sex steraids (pregnencione, testasterone, estrone, androstenediel) thyroid hormone (desiccated thyroid), androstenedione nedical Thailand. Androstenedione, also known accurring androstane steroid and 17-ketosteroid. is closely related structurally to androstenediol (A5; androst-S-ene-3,) 17--ioo, dehydroaptandrrsterrmm (DHEA;
naturally
-er------e---------oo--- and testosterone 1ratrott---a-11--2------- as well as to estrone (,55ra- Androstenedione was manufactured as dietary supplement, often callee ""andro"* "*andres""
S-androstanedione (5-androstane-3. (7--iiioe))))) 3,
Sports Illustrated for introducing androstenedione to the North American narket. Androstenedione wás
Arnold
+ regate/ in_end|>
<lin_start passistant
<thinloFron the provided responses, it is clear that testosterone is synthesized from androstenadione and cholesterol.
prinary hormone
derived from androstenedione cholesterol.
Therefore, the iniarry male hornone derived from cholesterol and androstenedione. </thinko
<answer>
""json
"rationale": "Testasterone, the prinary nale hormone, synthesized fron cholestarol and andrestenedione.
"answers":
"cholestaral",
"Androstenedione"
55


Reviews:
[{"review_id": "QHPRLfjxWQ", "reviewer": "Reviewer_6bRw", "summary": "The paper addresses the challenge of ambiguity in open-domain QA, noting that standard benchmarks often rely on a single reference answer, overlooking the reality that many questions admit multiple equally valid responses. To counter this, the paper proposes A^2 SEARCH, an annotation-free, end-to-end RL framework designed to recognize and handle ambiguity. This work proposes automated data generation pipeline that detects ambiguous questions and collects alternative answers via trajectory sampling and evidence verification. The subsequent RL training uses GRPO and an answer-level F1 reward, a metric tailored to accommodate multiple correct answers by rewarding coverage while penalizing over-generation. Eight open-domain QA benchmarks demonstrates that the approach achieves new state-of-the-art results, confirming that embracing ambiguity is essential for developing more reliable QA systems that can learn to sense ambiguity and retrieve multiple valid answers.", "questions": "- Why are K LLM verifiers necessary? Just for the ensemble?\n- L174 - what is the meaning of a n_hat s?\n- How does performance increase even on non-ambiguity datasets? Are ambiguous answers substantially included in datasets like HotpotQA?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The motivation for this work is well explained, and the paper is generally well written. The paper effectively highlights the significant challenge posed by ambiguity in open-domain QA. The recognition that current RL pipelines deliver misleading signals by rewarding only the reference answer and penalizing valid alternatives provides a strong foundation for the proposed framework.\n\n- The proposed method is its fully automated pipeline for identifying ambiguous questions and gathering alternative answers via trajectory sampling and evidence verification. This annotation-free approach is critical, especially since scaling costly manual annotation to complex multi-hop datasets like HotpotQA and MuSiQue is difficult. This automation makes the approach highly practical and scalable.\n\n- The paper includes rich and detailed experiments, encompassing a variety of ablation studies and case analyses.\n\n- Models trained with the proposed method show high efficiency, achieving strong recall levels with fewer average tool calls compared to baselines, demonstrating more effective utilization of reasoning steps.", "weaknesses": "- While the overall framework is effective, the novelty of the core learning method and reward design is moderate. The reward structure employs an outcome-only design and incorporates Answer-level F1 (AnsF1), which is derived from standard precision and recall metrics but tailored for multiple answers. The novelty lies primarily in the application of these existing mechanisms to the specific problem of ambiguity resolution, coupled with the sophisticated automated data pipeline, rather than new architectural or objective function designs.\n\n- While the paper confirms that the generated alternative answers are semantically distinct from the reference answer and evidence-supported, a more detailed breakdown of why they constitute genuine ambiguity is helpful. A statistical analysis quantifying the frequency of these different types of ambiguity (e.g., linguistic vs. entity vs. scope ambiguity) across the derived dataset would enrich the findings of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kEQlfcKgVJ", "reviewer": "Reviewer_wyg8", "summary": "The paper tackles ambiguity in open-domain QA (i.e. the cases where a question has multiple valid answers) by proposing $A^2-SEARCH$, an annotation-free RL framework that automatically mines alternative answers via an evidence-based pipeline and trains an LLM with an answer-level F1 (AnsF1) reward that balances precision and recall in multi-answer settings. Experiments on eight benchmarks show state-of-the-art results with single-rollout decoding.", "questions": "- How sensitive are results to the choice/mix of verifiers? Could an all-open pipeline (e.g., Llama- or Qwen-based judge + open retriever) approach the same agreement/coverage?\n- What \"△\", \"□\", \"○\" symbols mean on the Figure 2?\n- What's going on in the \"Step 2\" on the Figure 2?\n- Do I understand correctly that you use Qwen-2.5-Intrusct 3B/7B specifically as a base models for your $A^2-SEARCH$ 3B and 7B correspondingly in the Tables 1 and 2?\n- Did you try other models, NOT from Qwen family, as a base models for your $A^2-SEARCH$? What is the difference in the performance for other models?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper proposes a method that addresses a significant problem in QA.\n- The authors present convincing empirical evidence supporting their claims, using several baseline models of various sizes and several datasets to check the performance of their model.", "weaknesses": "- The paper does not address unanswerable queries - cases where retrieval fails and the model should abstain (e.g., say “I don’t know”). This omission is a substantial gap that limits real-world applicability of the framework.\n- At \"Step 1: Sampling\" (lines 248-258) the pipeline generates millions of trajectories, which may be computationally infeasible for many users; clearer reporting of the time cost of these experiments would be valuable.\n- If I understand correctly, the authors applied their training recipe exclusively to Qwen-family models to produce their $A^2-SEARCH$ model. I.e. they don't provide evidence for cross-family generality of this training approach (see \"Questions\" section).\n- Verification phase fully relies on proprietary models (Claude 3.5/3.7, OpenAI o3/o4-mini). Unpredictable availability of proprietary models and model drift may affect reproducibility and future re-runs; more results with open verifiers (or human audits at scale) would help.\n- Some parts of the paper are written in unclear way (see \"Questions\" section).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sqnaeUZOxk", "reviewer": "Reviewer_FJjM", "summary": "This paper proposes A²SEARCH—an end-to-end RL framework that requires no manual annotation. Its core contribution lies in designing an automated annotation pipeline that automatically identifies ambiguous questions in multi-hop question-answering datasets and generates alternative, valid answers through trajectory sampling, evidence filtering, and LLM-based validation. After RL training, A²SEARCH can automatically identify ambiguous questions, enabling the model to output all reasonable answers within a single inference trajectory, thereby improving the realism of the training signal and the reliability of the evaluation. Experimental results show that A²SEARCH-7B achieves SOTA  performance on eight open-domain QA benchmarks.", "questions": "- The Attribution of Performance Improvements: Can a more detailed analysis be provided to better demonstrate the actual value of \"ambiguity perception\"? When A2SEARCH outputs multiple answers, what is the precision of its predicted answers? Does the model tend to over-generate to achieve higher recall, especially when the final answer is a single, unambiguous response? Is part of the model's superior performance merely due to being trained to output answers in a list format, which differs from baseline models that are optimized to provide a single best answer? The authors need to more clearly demonstrate that the benefits arise from the recognition of ambiguity, rather than just the output format of multiple answers.\n- Additional experiments: Based on the previous point of inquiry, could you provide a comparative model that is trained on the original single definitive answer but is capable of outputting a list of multiple answers? This would allow for a comparison to demonstrate the contribution of the ambiguity-aware data itself. Additionally, it would be beneficial to constrain A2SEARCH to output only the most likely single answer and compare its performance on AnsF1@1 with that of baseline models. This would help to demonstrate whether the introduction of ambiguity information during training has enhanced the model's single-answer reasoning capability.\n- Source of Baseline AnsF1/Recall@3 Data: What is the specific source for the baseline models' AnsF1/Recall@3 data reported in Table 1 and Table 2? Were these results taken directly from the original papers, or were they reproduced and estimated by the authors themselves using the specified sampling procedure?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-  The paper clearly points out the pervasive mismatch between existing QA benchmarks (which assume a single gold answer) and real-world QA scenarios (where multiple valid answers exist). It is the first to propose a large-scale, annotation-free ambiguity solution specifically for the challenging multi-hop QA setting.\n-  The proposed four-step automated pipeline (Sampling, Filtering, Verification, and Grouping) is a significant technical contribution. It effectively utilizes the capabilities of existing LLMs to automatically discover and verify alternative answers, demonstrating a viable and scalable approach to augmenting training data without costly manual annotation.\n-  A2SEARCH-7B achieves SOTA results on multi-hop QA using only a single greedy decoding rollout. When compared to baseline models that typically require multiple sampled rollouts, this demonstrates the model's precise ambiguity-sensing capability and the significant efficiency of the proposed training paradigm.", "weaknesses": "Although the model performs exceptionally well on the AnsF1 metric, the source of this performance gain is insufficiently distinguished in the methodology. \nIt is unclear whether the improvement stems from genuinely solving ambiguities across different semantic levels or merely from increasing the recall of paraphrased variants of a single reference answer. \n\nThe use of a single-rollout comparison for A2SEARCH (@1) against multi-rollout estimation for baselines (@3) may potentially overstate its advantage relative to the baselines' true underlying capability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)