Active Task Disambiguation with LLMs
Authors: Kasia Kobalczyk
Nicolás Astorga
Tennison Liu
Mihaela van der Schaar
Venue: ICLR 2025 Spotlight (2025)
Keywords: Task Ambiguity
Bayesian Experimental Design
Large Language Models
Active Learning
Primary area: foundation or frontier models, including LLMs
TL;DR: This paper formalizes task ambiguity in tasks specified in natural language and frames task disambiguation through Bayesian Experimental Design, leading to more effective strategies for LLMs to pose clarifying questions.

Abstract: Despite the impressive performance of large language models (LLMs) across various benchmarks, their ability to address ambiguously specified problems—frequent in real-world interactions—remains underexplored. To address this gap, we introduce a formal definition of task ambiguity and frame the problem of task disambiguation through the lens of Bayesian Experimental Design. By posing clarifying questions, LLM agents can acquire additional task specifications, progressively narrowing the space of viable solutions and reducing the risk of generating unsatisfactory outputs. Yet, generating effective clarifying questions requires LLM agents to engage in a form of meta-cognitive reasoning, an ability LLMs may presently lack. Our proposed approach of active task disambiguation enables LLM agents to generate targeted questions maximizing the information gain. Effectively, this approach shifts the load from implicit to explicit reasoning about the space of viable solutions. Empirical results demonstrate that this form of question selection leads to more effective task disambiguation in comparison to approaches relying on reasoning solely within the space of questions.

Published as a conference paper at ICLR 2025
ACTIVE TASK DISAMBIGUATION WITH LLMs
Katarzyna Kobalezyk; Nicolás Astorga", Tennison Liu, & Mihaela van der Schaar DAMTP, University of Cambridge {knk25, nja46 cam. ac.uk
ABSTRACT
Despite the impressive performance of large language models (LLMs) across vari- ous benchmarks, their ability to address ambiguously specified problems-frequent in real-world interactions temmaas underexplored. To address this gap, we intro- duce a formal definition of task ambiguity and frame the problem of task disam- biguation through the lens of Bayesian Experimental Design. By posing clari- fying questions, LLM agents can acquire additional task specifications, progres- sively narrowing the space viable solutions and reducing the risk of generat- ing unsatisfactory outputs. Yet, generating effective clarifying questions requires LLM agents to engage in a form of meta-cognitive reasoning, an ability LLMs may presently lack. Our proposed approach of active task disambiguation enables LLM agents to generate targeted questions maximizing the information gain, Ef- fectively, this approach shifts the load from implicit explicit reasoning about the space of viable solutions. Empirical results demonstrate that this form of question selection leads to more effective task disambiguation in comparison approaches relying on reasoning solely within the space of questions.
INTRODUCTION
Problen statement S
Sample compatible solutions àt
Recent advances in the field of LLMs have led to the development of problem-solving agents capa-
White a faracion to anurch for an item in let.
just lot
ble of addressing complex tasks that extend far be-
Semple clarifying questions
yond conventional structured data problems such as regression and classification. State-of-the-art
Should return the dem's index or boolcan?
clat,
LLMs have demonstrated remarkable success log-
enmmeraterlad 1tea
ical reasoning (Creswell et al.] 2022 Lei et al.
within the Not?
2023). mathematical problem solving
(Romera-
How to handle cares when the target Korn
det suazch xis
Larget.
Paredes al. [2224] Imani et al., 2023
code gen-
eration (Liu et al.] 2024; Zhang et al.
2023a or
petecn Stone
creative writing (Coenen al. [002]] Chakrabarty
Figure I:
An ambiguous problem statement, sample of LLM --M-geeeratedd compatible solu- tions, and clarifying questions. The goal: Generate the most informative question.
eet al. 2023). While existing research predominantly focuses on enhancing LLMs' planning and reason- ing capabilities with new prompting strategies like
Chain of Thought (CoT) (Wei et al.] (022) or self-consistency (Wang al., 2023). evaluation benchmarks typically assume complete and unambiguous problem statements. However, due to the inherent ambiguity natural language (Stengel-Eskin et al., 2023) Liu et al., 2023) or deliberate underspecification, tasks encountered in real-world usage of LLMs may often not be well-defined, increasing the risk of the agent misinterpreting the true intentions of the problem setter.
The concurrent line of work suggests that in the presence of ambiguously specified tasks, agents should be able to infer missing information to discern the intended behavior (Tamkin et al. 2023). However, in the context of human-specified tasks, when user intentions deviate from that of the average population on which the internal LLM preference model has been trained, the agent is at risk of generating outputs that do not align with the user's true needs. Such behaviors may be especially harmful in safety-critical applications, such as medical diagnosis or treatment decisions. where erroneous answers pose significant risks.
"Equal contribution

---

Published as a conference paper at ICLR 2025
Similar to prior works (Kuhn al. 2022 Li et al. 2025 we advocate for LLM agents to engage in an interactive dialogue when faced with ambiguously defined problems. By asking clarifying questions, agents can narrow down viable solutions, reducing the risk of generating unsatisfactory responses. [i et al. 2025) have demonstrated that such interactive task elicitation can be more infor- mative and require less effort than refining the original prompts, often uncovering novel, not initially anticipated aspects of the given problem. However, asking a good clarifying question requires LLM agents to reason about their own generative distribution and identify features that differentiate the viable solutions. We hypothesize that such meta- ceota-cogiiive skills (Kuhn et al., 2022; Barzilai & Zo- har 2016) are beyond the capabilities of many modern LLMs. response, this paper investigates whether out-of-the-box abilities of LLMs in asking clarifying questions can be improved.
While traditional machine learning has extensively explored efficient data collection under model uncertainty, resulting in the development of active learning strategies (Houlsby et al. 2011;; Mackay 1992], LL.M-based agents do not operate on well-defined and structured input-output spaces. In- stead, LLMs navigate complex tasks with natural language, necessitating novel approaches to effi- cient information acquisition. Drawing from the principles of Bayesian Experimental Design (BED), propose method of active task disambiguation maximizing questions' utility by direct esti- mation of their information gain. Effectively, the proposed method shifts the load from implicit reasoning about the best question explicit reasoning via sampling from the solution space.
Contributions: We identify the need for new reasoning methods enabling L.M agents to handle ambiguously specified tasks effectively. ambiguity in natural language problem specifications, enabling us to frame the problem of effective task disambiguation through the lens of BED. Section 3 We propose and motivate theoretically a BED-based strategy for LLM agents to generate clarifying questions. > Section cesstionngeneaating strategies on an illustrative game of 20 questions
Section 2 We introduce formal definition of task
We evaluate
the effectiveness of competing and weal-world application to code generation. Our results demonstrate that the BED-based method improves upon baseline strategies relying on implicit reasoning about the best question to ask,
FORMALISM & BACKGROUND
We let Σ denote the space of natural language. We define problem statement S E Σ as natural language instruction for an agent to generate a solution h e Σ belonging to the unknown set of ground-truth solutions H E. We assume that the problem statement, S, can be decomposed into two parts: a set of requirements R that any E HH should satisfy, and any additional contextual information C that may influence the preference towards different outputs, h Σ.
Example 1 (Code generation). Consider the problem of code generation with an LLM agent based on prompt& S (R. C). Here, C is the natural language instruction guiding the LLM to output a code solution h. The requirements for the generated code, described in R. include the expected functionality of the code and any additional test cases that the generated solutions should pass. The ground truth H contains all programs h that, e.g. pass all hidden test cases or satisfy the internal needs of the human user. concrete example is presented in Figure
Let p-IS) denote the likelihood function that determines the preferences of the problem setter over candidate solutions h E Σ. We assume that for any S, p" (-IS) may be decomposed as:
p" p"(hS)) 1(h I R}n" (N/R, C),
E
(1)
In the above, 1 + R} is indicator function assessing if a solution h satisfies the requirements R or not. This indicator function is objective in the sense that it represents an unquestionable truth about sample solution h. In contrast, 5r (-IR, is context dependent and subjective. Given the same contextual information, different problem setters (e.g.. different human users) may have varying preferences over competing solutions that satisfy the given set of requirements.
In this paper, we will focus on the problem of task underspecification; a situation where the support of p" (-IS) extends beyond the set of acceptable solutions.
Definition (Task ambiguity). Let S = (R, C) and H = (h: h R.). We say that S is ambiguous H is a proper superset of H", Le. H H H"
When an LLM agent attempts to solve a given task specified by S, it generates a solution h according to its own generative distribution Po., (-IS). If S is ambiguous, providing a correct solution becomes
2

---

Published as a conference paper at ICLR 2025
pe. (15t)
Pos (1S)
X
Pang (1S)
p p7(-199
H
H
7
71"
74
H
(a) Preference optimization. The distribution Po,, aligned with the average preferences the population Pavg- This strategy successful, the set of acceptable solutions H( is aligned with the mode of pss. (left). If not, Paz may fail to generate correct answers (right).
(b) Iterative task elicitation. By quiring new task specifications, set of admissible solutions is succes- sively reduced, surrounding H³"
Figure 2: Resolving ambiguity.
challenging due to the risk of misinterpreting the true intention of the problem setter. The risk of misinterpretation is high if the distributions p" and Pø,, are not aligned. In order to solve this problem, one could consider at least two alternative approaches:
Preference optimization. The first school of thought postulates that agents should be able to fill in the blanks by combining information from instructions and their previous experience in order to identify the intended behavior (Tamkin et al. (2023). In particular, the goal of reinforcement learning with human feedback (RLHF) (Bai et al. Ouyangg et al.) (022) or direct preference optimiza- tion (DPO) RRafailov al. 2023 is align the generative distribution Pé,, (+|S) with preferences of the overall population such that with a high likelihood, Pea, (-IS) concentrates around HC* (Fig. 2a left). However, if the intentions of individual user deviate from that of the average population (Fig. 2a right), the agent is at risk of proposing solutions not belonging to H²" Such behaviors may be especially harmful if there is an associated risk with generating a sample answer & H*
Iterative task elicitation. The second approach involves engaging in an interactive dialogue with the user, seeking additional specifications (Kuhn et al. 2022 Li al. 2025). Since H. J H", some requirements the problem must have not been explicitly stated in S and remain unknown to the LLM agent. Let St = (R', (t) denote the task specification time During each round of the conversation, the agent can ask a clarifying question, g', and after receiving the oracle answer a', the problem statement gets updated
S S U (q', a²). each interaction step, the set of solutions compatible with S Ht == (h h + R'), becomes smaller and successively concentrates around the true solution set H", mitigating the risk failure when sampling h ~ Po,, (-ISt). The goal of this approach is to ensure personalized alignment without the need for model fine-tuning.
The goal. This paper focuses on the latter option of interactive task elicitation. Given that obtaining the oracle answers maybe be costly] our goal is find the best strategy for generating clarifying questions so that Po. concentrates around H. with minimal interaction cost.
? We make distinction between model uncertainty and the ambiguity of problem. While model uncertainty is measured through the heterogeneity of P ambiguity of a task is defined through the objective indicator function 1 ( R. whose true value is independent of the LLM the problem setter. We note that in many cases, the support of the agent's 's generative distribu- tion Pa., (-IS) may cover a larger space than H", even if the problem statement S is not ambigu- ous. This is due to the persistent issue of LLM hallucinations (Zhang et al.) 2023b Rawte et al.) 2023) or the complexity of the problem exceeding the LLM's abilities. Thus, a high uncertainty of the LLM agent does not imply inherent ambiguity of S. While the focus of this paper lies addressing objectively ambiguous problems, as we will see in experiment 4.2 iterative task elicitation may also improve LLM-generaue solutions even for unambiguous problems.
2.1 WHAT MAKES A QUESTION INFORMATIVE?
To understand what makes a question informative, we will rely on the principles of Bayesian Ex- perimental Design (BED). BED aims to design optimal experiments by maximizing the amount of information about an unknown quantity of interest gained from an outcome of an experiment. In the context of problem solving with LLM agents, an experiment corresponds a question q posed
We work under the convention of BED (Rainforth et al. 2023). wherein the cost of selecting experiment that maximizes the information gain is negligible in comparison the cost of obtaining the oracle answer.
3

---

Published as a conference paper at ICLR 2025
by the LLM agent, it's outcome is the oracle answer, a, and the unknown quantity of interest, is the generated solution, h. Therefore, the information gain (IG) from obtaining the additional task specification as pair (g, a), given problem statement, S, can be formalized as:
(2) where Hi is the Shannon entropy. However, because IG strictly depends the answer a, which is unknown at the point of asking the question q, we need to consider the expected information
IG(g, a) == Hip" (h]s)] - -Hlp" (h]S U (q, a)]],
gain-a quantity which arises by taking the expectation over all possible answers, given the query qi
(q) == B (al 9.5)
[G a)] Hipp" (h]S)] Ep [p" (hl U (q, a)]
(3)
Noting that the first term of the EIG in does not depend on q. we focus on the second term:
(alg.s) [H [p" Sh|S U (q. a)]l.
(4)
In the above, p" (alg, S) denotes the distribution of oracle answers to a question given S. We will assume that the law of total probability applies to p* so that p" (alg, S) EnEx P" (alg,, h)p" (h]S).
We will also assume that p" is an oracle that given sample solution h. can always answer a ques- tion q. about h, truthfully, so that p" p"(alg,
0 = (a. q). Under this assumption, all plau- sible answers a to a question g must be semantically equivalent. Thus, without loss of generality, we can assume that for each question, q. there exists only one answer, a, describing h, for which p" (alg, h) = 1. Therefore, each question q, generates partitioning of H into non-overlapping sets of solutions H H(q.a) compatible with (q.a). Let Ag denote the space of all unique answers to a question q given the current problem statement S and let H(a.)) == (h H :::' (alg, h) = 1), then we must have that:
H =
U H(g.a) and H(g.a) n H(q.a)) = Ø Va, a' E Ag s.t. a atd. d' aEA,
In this view, the utility of each question can be seen through the partitioning of H that it generates and the compound likelihood of solutions belonging to each of the partitions H(g,a). E A,
Requirement querying. Note that the ground-truth information gain depends on the unknown distribution p". Ideally, we would like the generative distribution of the LLM, Pe., (-IS), to closely approximate p" (-|5) for any problem statement, S. This is, however, not guaranteed. The bias of Pós may not adequately represent the preferences of the problem setter encoded in p". However, if, both p" and pd,, can be decomposed
product of the objective function 1 1 I RJ with their subjective counterparts, then (4 F R provides a source common grounding between the two distributions. Our strategy of active task disambiguation will therefore solely focus requirement querying. With that in mind, from now on we will assume that each question-anse answer pair, (q. a), yields new requirement, extending the current set of requirements and progressively reducing the set of compatible solutions (c.f. Fig. 2b
Uniformity of the unknown. As p" is unknown and Pe., may be inadequately biased, our strategy of active task disambiguation will be agnostic to the biases of both p" and pa,,- We will therefore approximate p" with a uniform distribution over the set of all solutions compatible with the given set of requirements. Under this assumption, we have that
(alg [ Sh|S U (q, (),)
Σ H W
Σ WEH
p" ('lg,
(5)
aEAr
= Σ (K
Σ
(6)
H
aEA,
M'EH
Given that p-1-)) is uniform on H. we have that p" ph(1 (q. a)) is uniform on H(g.a), thus HI [ U (q, a))] = lo.((H((.a)))) and so
Ex (ala,S)
[H [p" (M|S U (g. a))1 = H
1
Σ
17( (oga))log((1))) log( (())))
acA,,
We note that the above is minimized when H(o.g) are of equal size.
In practice, when p" represented through an internal likelihood function of a human, the two assumptions on the law of total probability and truthfulness p' may be violated; humans may answer questions incorrectly or answer "I don't It know". giving no information about H", We see such events as unpredictable random noise that we decide not to model for simplicity. Alternttiv approaches may penalize questions for which the expected uncertainty of p" (alq. h) is high.
4

---

Published as a conference paper at ICLR 2025
LLM problen-solving agent
Natural language output space
Solution generator
Rt' C es S' smiliguoss
Problem atatement
A ~ (o.
H
Updated statement gr+1 = S (g",a")
Question generator g Po,('")
Approcimate Ht' samples (A.)
Souplee candidate questions (v))
cardi questious
subutions
Dracle anawer a²
Question eveluator arg max ((()) v=(e])
Solect the best quistton g
and reeord the oracle aLawer ot
Figure 3: Active task disambiguation. ① Starting from t = 0, the problem statement St is presented to the problem-sole solving agent. 2 The agent reasons about the problem to infer the set of solutions H' compatible with the requirements of St. In order approximate HH' set of candidate solutions To discern between different solution variants, the agent generates candidate questions {o)}- 4 A question q" with the highest utility selected and presented to the oracle.
(4) is sampled. 3
s Based on the oracle answer, a", the problem statement is extended by the new specification defined through (q", a²); the process can be repeated with the updated problem statement St+11 S (q" a²) resulting a reduced space of compatible solutions, H C H*.
Corollary 1. Let H. be the set of solutions compatible with the requirements R within the problem statement S. Suppose that p² (-IS) is uniform H. Let Qn be the set of all questions with exactly possible answers such that for any 9 E Qn. there exists finite set of possible answers Ag with [Agl= = and that each question-ane -nsseer pair, (q, a), induces new requirement. Then, the EIG maximized for question q" whose possible answers in Ag partition H. into equally sized subsets.
Q we only consider binary questions with |Ag1 = 2 (e.g. yes-or-no type of questions), then the question with the highest information gain is the one which partitions the set of all possible solutions H into two subsets of equal size. The second pane on the right of Figure 3 shows how at iteration 4, three candidate questions may split the space of compatible solutions into subsets; the selected question q" is one which results in the most even partitioning.
From (7) it follows that max ElG(g) x log( [ This implies that question with more v possible answers can result in larger information gain. However, this requires finding a catego- rization of the possible solutions that partition them into balanced subsets. In the extreme case, if [H]] = K, the question with the highest information gain is the one whose answer always points to exactly one of the solutions h E H. Perhaps, the only way ask such questions is enumerate all solutions in H and ask: "Is h" is supposed to be h1.... hx-1 or h "? Arguably, this is neither a natural question ask nor user-friendly one, each of the K possible solutions. We argue that questions with small set of possible answers, yet ones that induce balanced partitioning of the solution space strike good balance between information gain and the mental load from the user.
requires the user examine
THE METHOD: ACTIVE TASK DISAMBIGUATION
The key point of the previous section is that given the unknown bias of p", the question with the largest information gain is the one that splits the space of compatible solutions into equal partitions. Generating such question requires the LLM agent reason about the set of possible solutions at each interaction step and identify the features that discern them. Such a reasoning process can be viewed as a form of meta-cognitive ability, requiring the LLM agent to consider the uncertainty within its own generative distribution. While a straightforward solution to asking such clarifying questions may be to simply perform zero-shot prompting of the LLM agent, we hypothesize that the out-of-the-box abilities of LLMs in this form of reasoning are lacking in comparison to their solution solution-generating abilities. This may be caused by a relatively small number of clarifying ques- tions present in their pre- pre-training corpus. Our proposed method explicitly evaluates the utility of candidate questions via a small sample of self-generated solutions and returns the question that maximizes this utility. Fig. shows high-level overview of the workflow.
5

---

Published as a conference paper at ICLR 2025
Let : - R denote utility function determining the usefulness of a candidate question q. Based on the derivations of the previous section we define 22 as:
(g) (IG - c(g).
where c: E R., is cost function determining the expected cost of obtaining the oracle answer to candidate question At each iteration t,
Our method of active task disambiguation consists of the following steps.
1. Sample a set of N candidate solutions { ~ Péa (-15");
2. Sample a set of M candidate questions { Pa. (-1*);;
3. Obtain pseudo answers o for each question of about every solution h;;
4. Approximate the utility 2/ of each question gf based on the answers {
5. Return the question q" = arg max (o))) (4));
6. Record the user answer a" and extend the problem statement to =t (q''"""a") aa*)
In the above, pa, correspond to the question- generating distribution of the problem-solving LLM. Step 3. requires approximating the EIG of the candidate questions { based on the sample (M This estimate is computed by generating to all questions of answers {af,,) about each sample solution hf. Depending on the type of the solutions, these answers may be generated by the problem- end for return - - Ph log(pe)
Algorithm 1 estimate EIG fai, 1
Require: A question q and set of N answers [a
(ai.....en) Un3 &- Unique answers in (ai,)
for k (c) do
me f- 1[: 1i:ars = ag, E [M]H pa < -2/n
solving LLM itself (see experiment [41). or evalu- ated with an external tool (see experiment (.2). Al- gorithm I shows the general procedure of estimating the EIG score based on equation
② We note that the estimation of the EIG score, and consequently the selection of questions, directly depends on the sample [M] The proposed approach shifts the load of selecting the best question from implicit reasoning about the best question to ask to explicit reasoning in the solution space. The underlying motivation for this strategy is that generating discrim- inative questions without direct access to the set of compatible solutions more challenging than solution generation itself. Question selection via direct maximization of the EIG bootstraps the question-generating skills of LLMs using their potentially stronger colution-generann skill. Based on this observation, we make a couple of practical remarks.
Ensuring uniformity. When estimating the EIG score based on samples from Ps.. (-IS = C) we want to ensure that Pe., is close uniform on the set of R-compatible solutions. In practice, to encourage diversity of samples we may adopt two strategies. 1) Generate sample solutions with increased sampling temperature. 2) Instruct the LLM generate a list of "diverse and representa- live" solutions. We find that adding this statement to the solution-generatin prompt improves the diversity of generated samples and the coverage of H (see study 2. of experiment 4.1).
Mitigating LLM noise. The effectiveness of questions selected by maximizing the estimated infor- mation gain relies on the agent's ability to accurately evaluate 1( - R.J. For this, we need to ensure that a) the generative distribution of the agent, Po.. (-18), is error-free, i.c. Pés, (h|S = (R, C)) 0 for any h R and b) the pseudo answers 0g,j used to estimate the EIG are also error-free, free, ie.. h, + (gj. a..5). While in select cases ensuring the compatibility of solutions with requirements is straightfowward (see code generation in experiment 4.2). in many problems in which the agent generates solutions without any external grounding, ensuring that points and b) are error-free is challenging due to L.M hallucinations (Zhang et al. [2025]] Rawte et al. 2023). We observe that in practice, employing more refined prompting strategies, like CoT or self-reflection, works well (see study 3. of experiment
In the experimental section, for simplicity, we take to he constant. In the case of human-writter answers, alternatives may include penalties for the question's length the expected length of the answer.

---

Published as a conference paper at ICLR 2025
4 EXPERIMENTS
Our experiments are designed to investigate two hypotheses which result from the discussions con- tained in the previous sections:
HI) Implicit reasoning about solutions generate the most effective clarifying question is a dif- ficult skill for LLMs. This skill can be improved by shifting the reasoning load from the question space to the solution space.
H2) The gap between implicit reasoning, i.c. generating questions without explicitly sampling hypothetical solutions, and explicit reasoning through a sample of solutions to select the best question is most significant in cases where:
H2a) The LLM can generate representative and diverse samples solutions, uniformly cov-
ering the space of solutions compatible with the given problem statement.
H2b) The evaluation noise of the EIG is minimal. This is particularly true in cases when evaluation can be offloaded to an external tool.
Given the above, we will present two kinds of problems: one in which there is no external evaluator guaranteeing that sample solutions adhere the given requirements and one in which we can ground the evaluation of [ + RJ with an external tool. For both experiments, prompts used to generate questions, solutions, and answers are provided in the Appendix (D.1. and
and D.2
4.1 YES-OR-NO QUESTIONS WITH THE 20 QUESTION GAME
The 20 Questions game classic guessing game that involves one player (A) thinking of an object, and the other player (B) asking up to 20 yes-or-no questions to guess what it is. The object can be anything, often categorized into an animal, a place, or a person to give the guessers a starting point. The goal for the guessers is to identify the object with few questions possible. Despite the 20 questions game being seemingly a toy example, provides an ideal setup to evaluate the multi-turn questioning abilities of LLM agents. Moreover, it serves as parallel to many real-world applications, like conversational search, content recommendation, or even medical diagnosis.
4.1.1 THE MAIN EXPERIMENT
Setup. To reduce the sampling costs, we play the game for 10 instead of the original 20 rounds. We restrict the game to the category of animals. Here, the set of acceptable solutions, 7. are singletons, {h*) where h² represents single animal name that player A may think about. Player A is simulated with GPT-4o-mini prompted answer questions about the ground-truth animal h² We emphasize that in this setup there is no pre-fixed list of candidate animals that the user or the reasoning agent can choose from. Instead, at each point of the interaction, the guessing LLM (Player B) is free to guess any animal across the entire animal kingdom. For quantitative analysis of question- generating strategies, we run the game on 15 arbitrary tasks corresponding to 15 animal names (see appdx. D.I.2p. For each task, we run the iterative requirement querying for 10 iterations across seeds.
Question generation. We consider four alternative methods for generating questions:
implicit: the agent prompted to generate a single candidate question;
implicit-Tob T Yao et al. (2023): using the same prompt, we sample A = 5 questions and prompt the LLM to select the best questions among the set of self-generated candidate questions;
EIG-uniform:
using the same prompt. sample M 5 questions and select the one that maxi- mizes the estimated EIG score. The EIG is estimated assuming uniformity of sample solutions;
EIG-logprobs: same as above, but the EIG is estimated using the log-probabilities of the sample solutions as returned by the LLM agent.
EIG estimation. To estimate the EIG for the latter two strategies, at each interaction step. we prompt the LLM agent to generate a list of N 20 animals that adhere to the current set of re- quirements R'. If the list of requirements R' is long. the LLM may generate animals that do not fulfill all the requirements. To mitigate this, after the initial sampling of the animals, loop
7

---

Published as a conference paper at ICLR 2025
all requirements R' and use the self-critic answering prompt to check if all requirements R2 are sat- isfied. This extra filtering step is repeated twice (see Study 3. section 4.1.2 for ablation of this step). Animals that do not fulfill the requirements are rejected and sampling repeated until N animals are obtained. The same self-critic answering prompt is used to generate the pseudo answers di to candidate questions Øj about sample hypothesis hş. The resulting set of answers is used estimate the EIG according to Algorithm [ Refer to Appendix [ Algorithm 4 for details on estimating the EIG using the log-probabilities of the sample solutions {ha
GP5-3.5
GPT-4o-mini
Evaluation. Our metric of success is the likelihood of the problem-solving agent guessing the ground truth animal given the obtained set of requirements, R2, at each interaction step. We approximate this likelihood by sampling 10 animals with Pe, (-IR5, C) and then prompting the agent to order the animals from most to least likely. We record the position of h" within the returned list, counting from the bottom-a score of O indicates that h" has not been sampled and a score of 10 indicates that h is at the top of the list. For each task, seed, and iteration, we compute this score with 25 sampling seeds.
Rank
2
10
Iteration
EIG-uniform implicit- implicit
Results. As observed from Fig. 4 the EIG-uniform strategy outperforms the remaining strategies by a significant mar- gin. The fact that it outperforms both implicir and implicit- ToT strategies confirms HI. Shifting the reasoning load from question space to direct reasoning about solutions results in improved efficacy of the selected questions-the elicited re- quirements lead higher likelihood of the LLM guessing the right animal after fewer iterations. We also observe that EIG-logprobs significantly underperforms in comparison to EIG-uniform. This proves our claim about the L.M'' s generative distribution Pé, being inadequately biased, favoring solutions that do not necessarily agree with the ground truth solution set H." Fi- nally, note that the gap in performance between EIG-uniform and the two implicit strategies is much lower for GPT-4o-mini, which is considered to be overall a significantly more capable model than GPT-3.5-turbe especially when more reasoning steps enforced with the ToT prompt.
lag-rogooobs
Figure 4: Comparison of question- generating strategies on the game of 20 questions. Rankings averaged across 15 ground-truth animals, 5 run seeds, 25 evaluation seeds. See Appendix [ for results with the Llama family models.
4.1.2 ADDITIONAL STUDIES
The aim of this section is to gain further insights and investigate the impact of the de- sign choices behind the EIG-uniform strategy on the effectiveness of the generated questions. In what follows, we present results for the problem-solving agent implemented with the GPT-3.5-turbo model.
strategy
EIG uniform EIG-logprobs
Questions' utility. While we do not restrict the game to a pre-fixed list of animals, in order to evaluate the partitioning properties of generated questions, we construct large and diverse list of 500 animals (see Appendix (.I.2).
Study 1:
300
Valid solutions
implicit- ToT
200
implicit
Figure 5 shows the number of animals compatible with the requirements at the first six iterations from the large list con- structed, instead of the small samples of N animals used during the reasoning process (results limited to the first six rounds due to high costs of responding to all questions about 500 animals). We find that questions generated with the EIG- uniform strategy result in smaller subsets of commmpatiil solutions, and thus more effective disambiguation. This also shows that the estimation of EIG based on just N 20 samples during question elicitation is a good proxy for the ground truth EIG, here approximated with the large list of 500 animals.
Iteration
Figure 5: Number of valid solutions after each iteration.
Study 2: Diversity of solutions. Estimation of questions' utility dependent on the quality of sample solutions, which need to be sufficienlly diverse to approximate H. well. To ensure good coverage. the prompt for animal guessing includes the statement: Generate a carefully selected, di- verse, and representative set of animals.. The EIG score used in our experiments is equivalent to the conditional mutual information (MI) between the random solutions h Po., (-SS) and random an- swers pl(:)))= == Eee H Po. (-1g, Po. (hlS), for fixed question g. In order to test the impact
8

---

Published as a conference paper at ICLR 2025
of the diversifying statement, compute this score across range of 330 diverse questions. From the main experimental results, we collect the unique set of 330 questions asked until the sixth itera- tion. We then run the game until the first six iterations sampling animals with and without the diversifying statement. We also vary
Table 1: Diversity of samples
N
diverse
vanilla
10
0.57 (0.01)
0.46 (0.01)
20
0.58 (0.01)
0.54 (0.01)
the number of samples N. After obtaining the sets { for all
30
0.60 (0.01)
0.58 (0.01)
t E [_...., 6). we estimate the score for each of the 330 questions based on the average of the conditional MI score will be maximized for Pas close to uniform. Thus, a higher MI score implies greater diversity of the generated sample solutions. Table[] that the inclusion of the diversifying statement leads to more diverse samples of animals, validating H2a.
answers. Assuming that the collection of 330 questions is diverse, the
Study 3: Requirement compatibility. To mitigate the impact of L.MM hallucinations on sampling solu- tions that do not adhere to the given set of require- ments our solution generating prompt employs an ex- tra filtering step to ensure that sample solutions h; indeed adhere to the current set of requirements. We test the impact of filtering on the accuracy of gener- ated solutions. Figure [6] shows the average number of solutions rejected with a single filtering step (left) alongside the final rankings for the ElG-uniform strat- egy when 0, 1 or filtering steps are used (right). As the number of requirements increases, the LLM is more prone to hallucinations. A form of "self- reflection" " via filtering becomes crucial in ensuring accurate outputs and confirming H2b.
of steps
R of discarded salutions
12.5
10.0
7.5
Rank
2.5
0.0
9
3 5
Iteration
Iteration
Figure 6: The impact of LLM noise.
4.2 ACTIVE CODE SANNERATION-REQIREEEEEN
AS UNIT TESTS
Setup. In our second experiment, we demonstrate active task disambiguation with an external tool ensuring a near error-free evaluation of requirement compatibility. In this experiment, the goal of our reasoning agent described in the initial prompt 8 is to generate code solution h, based on the requirements RD specified via a user-defined instruction describing the expected functionality of h. Following the setup of CCen et al. (2023), S contains a code snippet that includes statements such as imports, the function header, and a short comment describing the expected functionality of the generated code. Due to the ambiguous nature of natural language and the fact that at the point of writing the instruction not all edge cases might have been considered, S is likely to be ambiguous (see Appendix E.2.2 for examples).
Question generation. We consider two types of clarifying "questions":
(B) A question q is generated test case in the form of assertion that the oracle is supposed to either confirm as correct or reject. We call these questions binary, as they only have two kinds of responses: True False, similarly to the yes-or-no questions from the previous experiment. (0) question q a generated input to the desired function. The oracle returns the expected output of the code. We call these questions "open", as for one sample input there may exist nearly unconstrained number of valid outputs, similarly to open-ended questions.
We compare questions generated zero-shot against questions selected by first sampling M = 5 can- didate questions and then selecting one that maximizes the EIG, under the assumption of uniformity.
Answers and requirements. Both the ground truth answers a" and the answers d..j used for EIG es- timation are obtained by executing the ground-truth or a candidate program h, respectively, against a question q. The resulting question-answer pairs are turned into additional requirements exe- cutable unit tests that each generated solution must pass and appended to R'. Generated programs exccuted with external Python interpreter in sandbox environment. The "answering" of questions through an external tool ensures near noiscless estimation of the EIG score.
Solution generation. The solutions hf P&s, (-1S") are sampled by prompting the LLM generate code completions which are then filtered to only those samples that pass the test cases in R'. This ensures that all solutions sampled from the LLM conform to the elicited requirements.
Evaluation. We evaluate all question-genent generating strategies on the HumanEval benchmark contain- ing simple coding problems (Chen et al. 2021), and the more challenging APPS (Hendrycks et al.)
9

---

Published as a conference paper at ICLR 2025
HumanEval
APPS
Acturacy (b)
base (B)
60
BIG (8)
base [0]
EIG (0)
GPT-3.5 Llama3 GPT-40
GPT-35 Liama GPT-40 mini
Figure 7: Accuracy of generated code solutions after eliciting 4 additional requirements. averaged across 48 tasks, 3 run and evaluation seeds.
Results
benchmark with competition coding challenger limiting the number of total questions asked to 4. After obtaining for each iteration set of test cases R2, evaluate the diseriminative power of the obtained requirements, by sampling 20 solutions from Po,, (h]S*) and calculating the percentage of code solutions that pass the hidden test cases provided in the benchmark dataset.
2021
Figure 7 shows the accuracy of the generated code samples after eliciting 4 requirements with all strategies. Tables and the appendix show the results across all 4 iterations. We observe that EIG- strategies lead to higher accuracy of the outputs with fewer test cases queried. As ex- pected by the theoretical discussion of section 2.1 questions with more possible answers (O) can be more informative than questions with only two possible answers (B). However, if these are to be an- swered by a human user, "open" questions arguably require an increased mental load to answer. We also observe that for more capable models, the gap between the baseline questioning strategies and their EIG-equivalent version is smaller. Results are consistent across both benchmarks, indicating that the EIG methods remain effective across varying levels of task ambiguity and difficulty
Results.
We also note that the low accuracy of generated samples h ~ Po,, (-IS") does not imply that 8 itself is inherently ambiguous, as it may simply be caused by the LLM's s limitations in following complex instructions. However, even if S is objectively not ambiguous. the input-output examples appended to the prompt S may positively bias Pd,, towards correct solutions, which is desirable by-product of employing active querying strategies to task disambiguation.
5 DISCUSSION
Limitations. While our work primarily focuses on efficient requirement elicitation, handling am- biguously specified tasks involves two equally important aspects determining a) that the given prob- lem is ambiguous: b) when a sufficient number of requirements have been collected to stop querying the user. Kuhn et al.] (2022) demonstrate that in select instances, ambiguity detection can be effec- tively resolved with zero-shot prompting. We believe that future research should explore alternative strategies. We also note that the guestion-generatiin increased number of LLM calls compared to the baselines (see Appendix (F). However, in line with the assumptions commonly made in BED, we take the stance that the computational load required to select the optimal query is negligible compared to the value of acquiring information that reduces problem ambiguity. We anticipate this assumption will become more valid over time as technol- ogy advancements lower the costs of LLM token generation, thereby enhancing the importance of efficient information acquisition strategies.
strategies presented in this work require an
Conclusions and Impact. Our findings suggest that clarifying questions generated with zero-shot prompting of LLMS are less efficient than those elicited by direct estimation of their utility with respect the set of self -eneraeded solutions. This suggests that the current skills of LLMs in gener- ating efficient clarifying questions are underdeveloped, leaving room for improvement. Agents with well-developed meta-cogmitive skills should be able to implicitly reason about the best question to ask without relying on multi-stage prompting strategies. We hypothesize that LLMs' deficiency in asking good clarifying questions stems from their limited exposure to such questions in the train- ing corpus. To address this, our proposed framework offers a way to generate synthetic datasets of underspecified problems and their corresponding optimal clarifying questions. These datasets could serve as a resource for supervised fine-tuning, enhancing LLMs' abilities to disambiguate tasks more effectively and improving their interactive real-world problem-solvine capabilities.
'We filtered the tasks of both benchmarks subset of 48 non-trivial tasks, i.c. tasks that do not achieve ncar 100% zero-shot accuracy when sampling from h ~ Po,, (-15") with GPT-3.5-turth or GPT--o-mini.
'The zero-shot accuracy of GPT-4o-mini on APPS is only ~35% in comparison to the 70% achieved on HumanEyal confirming the more challenging nature of the APPS benchmark.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY
The pscudo algorithm to execute active question generation can be found in Appendix C
The exact
format of the prompts used for both experiments presented in Appendix D Results are provided for two closed sourced models: GPT-3.5-turbo, GPT-4o-mini; and two source models: Llama-3-8B
(Instruct), and Llama-3-708 30B (instruct). Code for reproducing the experimental results section 4.2 is made available at: httpss//gitubb..om/aaa lausii-kobalzyykaaaee taske task-dissun -ask-disambiguattionn The repository also includes generated programs and querries with GPI-3.5-turke and GPI-4o-mini.
ACKNOWLEDGMENTS
This work was supported by Azure sponsorship credits granted by Microsoft's Al for Good Research Lab. Katarzyna Kobalczyk is supported by funding from Eedi. Nicolás Astorga is sponsored by W.D. Armstrong Trust and Tennison Liu is sponsored by AstraZeneca. We thank the anonymous ICLR reviewers, members of the van der Schaar lab, and Andrew Rashbass for many insightful comments and suggestions.
REFERENCES
Vincent Aleven, Ido Roll, Bruce M. McLaren, and Kenneth R. Koedinger. Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1):205-22) 223, 2016.
Saleema Amershi, Daniel Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Phillip Col- lisson, et al. Guidelines for human-ai interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1-13. ACM, 2019.
Chinmaya Andukuri, Jan- Jan-Philipp Fränken, Tobias Gerstenberg, and Noah Goodman. STar-GATE: Teaching language models to ask clarifying questions. In First Conference on Language Model- ing, 2024. URL Utthttps::////// http::ts//oppeeeeiiieeeee/ net.://wwwwweeeeeeeeeeeeeeeeeeee htt.://teeeoormmmooccCCC0AAAAA10
Nicolás Astorga, Tennison Liu, Nabeel Seedat, and Mihaela van der Schaar. Active learning with Ilms for partially observed and cost-aware scenarios. In The Thirty-Eighth Annual Conference on Neural Information Processing Systems, 2024a.
Nicolás Astorga, Tennison Liu, Yuanzhang Xiao, and Mihaela der Schaar. Autoformulation of mathematical optimization models using Ilms, 2024b. URL URL.http::///rxvv.orr////
01679
David Eric Austin, Anton Korikov, Armin Toroghi, and Scott Sanner. Bayesian Optimization with LLM-Based Acquisition Functions for Natural Language Preference Elicitation, 2024. .eprint: 2405.00981.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and others. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint 22ti:22004.08662. 2022.
Sarit Barzilai and Anat Zohar. Epistemic (meta) cognition: Ways of thinking about knowledge and knowing. In Handbook of epistemic cognition, pp. 424. Routledge, 2016.
Shelley Bowen and Anthony B Zwi. Pathways to "evidence-informedd" policy and practice: A framework for action. PLOS Medicine, 2(7) 05 2005. doi: hw..1///oormaa..medd...oo 0020166. URL httph:tt///cm. http:://ooooooooooo http:/.....////uurrrrr pmed. www.ppmdd...0......00 0020166
Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, and Smaranda Muresan. Creativity support in the age of large language models: An empirical study involving emerging writers. arXiv preprint arXiv: 2023.
11

---

Published as a conference paper at ICLR 2025
Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Man-Guang Lou, and Weizhu Chen. CodeT: Code Generation with Generated Tests. The Eleventh International Confer- Learning Representations, 2023. httph:tt////// httppsss//pperevie.... net htwp://wwwwwee/oruuunmo forum?id-
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert- William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating Large Language Models Trained on Code. 2021. eprint: 2107.03374.
Andy Coenen, Luke Davis, Daphne Ippolito, Emily Reif, and Ann Yuan. Wordcraft: A human-AI collaborative editor for story writing. arXiv preprint arXiv:2107.07430, 2021.
Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-infereneee Exploiting large lan- guage models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712, 2022.
Marcos Lopez. De Prado. Advances in financial machine learning. John Wiley & Sons, 2018.
Shizhe Diao, Pengeheng Wang, L. I. N. Yong, Rui Pan, Xiang Liu, and Tong Zhang, Active Prompt- ing with Chain-of-Thhuuee for Large Language Models, 2024. URL htt.:tttttpppenreeeet net fest/Eorum?20 i
Arthur C. Graesser, Phoebe Chipman, Brian C. Haynes, and Andrew Olney, Autotutor: An intel- ligent tutoring system with lixed-initiativy dialogue. IEEE Transactions on Education, 48(4): 612-618, 2005.
Jeroen Groenendijk. Studies on the Semantics of Questions and the Pragmatics of Answers. PhD Thesis, January 1984. Publication Title: Varieties of Formal Semantics.
Kunal Handa, Yarin Gal, Ellie Pavlick, Noah Goodman, Jacob Andreas, Alex Tamkin, and Be- linda Z. Li. Bayesian Preference Elicitation with Language Models, 2024. _eprint: 2403.05534.
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazcika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Mcasuring Coding Challenge Competence With APPS. In Joaquin Vanschoren and Sai-Kit Yeung (eds.), Proceedings of the Neural Information Processing Systems Track Datasets and Bench- marks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021. URL
https: //dtasets/tsssssssaa bww.heankeee.............. peeeecchhnrrkkprooediiiee proceedings.....oooooo....... new.eeeeedddnn........s.s.....s pcc/pappe/2221/
9502a028 Abst -aw.sstaatta-bttaaatttttm round2. html
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv: /2.5745, 2011.
Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian He, Pang Wei Koh, and Bryan Hooi. Uncertainty of thoughts: Uncertainty-awan planning enhances information seeking in large language models, 2024.
Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.05398 2023.
Marina Jirotka and Joseph A. Goguen (eds.). Requirements engineering: social and technical issues. Academic Press Professional, Inc., USA, 1994. ISBN 0123853354.
Kasia Kobalczyk and Mihaela der Schaar. Towards Automated Knowledge Integration From Human- Humman-Interpretable Representations. In The Thirteenth International Conference on Learning Representations, 2025. URL http:::::/p///e//eeeeee net
12

---

Published as a conference paper at ICLR 2025
Dmitrii Krasheninnikov, Egor Krasheninnikov, and David Krueger. Assistance with large language models. In NeurlPS ML Safety Workshop, 2022. URUhttts:://ppeeeevaae net. forum?
1d=0E3V81spp63
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Clam: Selective clarification for ambiguous questions with large language models. arXiv preprint arXiv:2212.07766 2022.
Bin Lei, Chunhua Liao, Caiwen Ding, and others. Boosting logical reasoning in large language models through a new framework: The graph of thought. arXiv preprint arXiv:2308.0800 2023.
Belinda Z. Li, Alex Tamkin, Noah Goodman, and Jacob Andreas. Eliciting human preferences with language models. In The Thirteenth International Conference on Learning Representations, 2025.
URL httpt:://ppppeeee/ee/ee///
net forum Lid.uumnDdddLLLoWw/Aaeee htw.://www....n...oooooooo.ooaaa
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. We're Afraid Language Models Aren't Model- ing Ambiguity. The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
fortm? www.:rrmmmmwwwh..... d gb3
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chat- gpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36, 2024.
David John Cameron Mackay, Bayesian Methods for Adaptive Models. PhD Thesis, California Institute of Technology, USA, 1992.
Katerina Margatina, Timo Schick, Nikolaos Aletras, and Jane Dwivedi-Y Yu. Active Learning Prin- ciples for In- Co-tettext Learning with Large Language Models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 501 1-0044 Singapore, December 2023, Association for Computational Linguistics. doi: 10. 181118666//22233 12026555511222.2.2.iinneeemmmmm findings-emnlp.334. URL https: //ac:://aalaaee www.aaaannnooooooooooooooooooooe org/2033. Findings emn 334
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. AmbigQA: Answering Ambiguous Open-domaij Questions. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pp. 5783-5797, Online, November 2020. Association for Computational I. Linguis- URL http:////aaaaaaacaaaccc org/2020.
tics. doi:
emn ma in.inn.. 466
Douglas C. Montgomery. Design and Analysis of Experiments. Wiley, 9th edi-
tion, 2017. URL https:/ /www//www. http:/wwwww........//w::ww:wwwwt www.:wilwl..coo. com http:/Deeiiinnnnnnnnneeee wtt.meemeeeetttttthhtoiooooooooe
Ayana Niwa and Hayate Iso. AmbigNLG: Addressing task ambiguity in instruction for NLG. In
Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 10733-10752 Miami, Florida, USA, (t065///1/222.mmm.pp.. ma main. 599/
LinputaaionallLnnuusiiiici November 2024, Association for Computational Linguistics. doi: 599, URL bttt:s://ac..ancolooy.cooo
emp
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 27730-27744. Curran Associates, Inc., cc ct/ files/papess 20222 Cawerrr-oooffeeneeemmoeo org.aaaa.e.....aa.....o..ooooorr
2022. URL httpp:::/ppooeeeeaiggg.sssss
pdf
Brian Paden, Michal Čáp, Sze Zheng Yong. Dmitry Yershov, and Emilio Frazzoli. A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Transactions on intelligent vehicles, 1(1):33-55, 2016.
13

---

Published as a conference paper at ICLR 2025
Top Piriyakulkij, Volodymyr Kuleshov, and Kevin Ellis. Asking Clarifying Questions using Lan- guage Models and Probabilistic Reasoning. In NewrIPS 2023 Foundation Models for Decision Making Workshop, 2023.
Fulsundar Amita Purushottam, Ajay Kumar, Vikas Haribhau Satonkar, Shweta Gaikwad, Stefi Diliprao Sonawane, and Bhushan shirwadkar. Probabilistic risk assessment in cybersecu- rity: Bayesian methods for quantifying and mitigating cyber risks. Panamerican Mathematical Journal, 2024. URL https: //ap::///pp.... emant icscholar. oct/CCrppsSI::224111555
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning. Stefano Ermon, and Chelsea Finn. Direct Preference Optimization: Your Language Model is Secretly Reward Model. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 53728 53741. Curran Associates, Inc., 2023.
URL htttps://procceedinnng neurips cc cttpcc/cpppeeeeee.cc pawer_appperpeeee http:rriss.ccpppppereeeeeee///pp wttpse/flleepppae/ee//ee
fi
Canferapeer-ooeeeeeeeeeeeeeeeeee pddf
Tom Rainforth, Adam Foster, Desi R. Ivanova, and Freddie Bickford Smith. Modern Bayesian Experimental Design, 2023. URL
.eprint:
2302.14545.
Sudha Rao and Hal Daumé III. Learning to Ask Good Questions: Ranking Clarification Ques- tions using Neural Expected Value of Perfect Information. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pp. 2737-2746 Melbourne, Australia, July 2018. Association for Com- putational Linguistics. doi: 10. 10.18855/11/P18--3355 URL https:://cclattthllloy/ P18 1255
Sudha Rao and Hal Daumé III. Answer-based Adversarial Training for Generating Clarification Questions, 2019. eeprint: 1904.02281.
Paulius Rauba, Nabeel Seedat, Krzysztof Kacprzyk, and Mihaela van der Schaar. Self-healing machine learning: A framework for autonomous adaptation in real-world environments. arXiv preprint arXiv:441..00186, 2024a.
Paulius Rauba, Nabeel Seedat, Max Ruiz Luyten, and Mihaela van der Schaar. Context-aware testing: A new paradigm for model testing with large language models. arXiv preprint arXiv:2410.24005. 2024b.
Vipula Rawte, Amit Sheth, and Amitava Das. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922, 2023.
Edwina Rissland, Ai and legal reasoning. Al Mag.. 933):45-5,, September 1988. ISSN 0738-4602.
Bernardino Porere- Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, and others. Mathematical discoveries from program search with large language models. Nature, 625(7995):468-44 475, 2024. Publisher. Nature Publishing Group UK London.
Elias Stengel-Eskin, Jimena Shaalar-BBascaa Yi Zhou, and Benjamin Van Durme. Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10220-10237, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 8153/v1/20023. acl-long.569. U a a ogy org /2/2222233 ac www.::0.....1nn...........o 569
Alex Tamkin, Kunal Handa, Avash Shrestha, and Noah Goodman. Task Ambiguity in Humans and Language Models. In The Eleventh International Conference on Learning Representations, 2023. URL
net http::/rteeettteeeeeeae worum wwwrrruiiiddrrrreeee 92Fd8
Sebastian Thrun Probabilistic robotics. Communications of the ACM, 45(3):52-57, 2002.
Eric J. Topol. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again.
www. www.w.absssss....ommooooooo..... Basic Books, 2019. URL hww.ap/b//ww...bbii.............
com/ htt.://bow...oooiiieeeeeeeeeaaaa
deep-medi htt.:iiccie/91111111111111 /4417777711511444377
14

---

Published as a conference paper at ICLR 2025
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V. Le. Ed H. Chi, Sharan Narang. Aakanksha Chowdhery, and Denny Zhou. SelfsConsis Improves Chain of Thought Reasoning in Lan- guage Models. In The Eleventh International Conference Learning Representations, 2023.
wtt.httts:://ppnrreeee http://ppppeeeeeeeeeeeeee net. hwwp://tt/ffottiiioo forum?id= NTMM
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and others. Chain-of -hain-of-thought prompting elicits reasoning large language models, Ad- vances in neural information processing systems, 35:24824 23224824-24837, 2022.
Hojin Yang, Scott Sanner, Ga Wu, and Jin Peng Zhou. Bayesian Preference Elicitation with Keyphrase-Item Coembeddings for Interactive Recommendation. In Proceedings of the 29th ACM Conference User Modeling, Adaptation and Personalizaation, UMAP '21, pp. 55-64, New York, NY. USA, 2021. Association for Computing Machinery. ISBN 978 1-4503-83666-- doi: 10. 1444114544445 3456814 event-
1145/3450613.. 3.34568144 httpht::////d.iccco
place: Utrecht, Netherlands.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan, Tree of Thoughts: Deliberate Problem Solving with Large Language Models, De- cember 2023. URL httl:ttt:://aaatt
oog..rggg/s.............. ab//2305 0601 arXiv:2305. 10601 [es].
Fengli Zhang, Qianzhe Qiao. Jinjiang Wang, and Pinpin Liu. Data-driven ai emergency plan- ning in process industry. Journal of Loss Prevention in the Process Industries, 76:104740, 2022a. ISSN 0950-4230. doi: http:://bb.arr///0..00f111222222
104744 URL httph::/t////
SC sciencediog rect com
Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua Tenenbaum, and Chuang Gan. Planning with large language models for code generation. arXiv preprint arXiv:2303.05510, 2023a.
Yiming Zhang, Shi Feng. and Chenhao Tan. Active Example Selection for In-Context Learning, In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9134-9148, Abu Dhabi, United Arab Emirates, December 2022b. Association for Computational Linguistics. doi: 10.18653/1/2022. org /rg/2022 emnlp main. 622
emnlp- 622. URL
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, and others. Siren's song in the Al ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219 2023b.
Hana Ševčíková, Adrian E. Raftery. and Paul A. Waddell. Assessing uncertainty in urban simu- lations using bayesian melding. Transportation Research Part B: Methodological, 41(6):652- 669, 2007. ISSN 0191-2615. doi: http:///o..orgy/00000.......200. 1001. URL https:
//www. www.nceieiiccc.... rect. httpa///cc.c.cccc..oo.oooooon//a
15

---

Published as a conference paper at ICLR 2025
A EXTENDED RELATED WORK
Active (in-context) learning. In conventional AI., the MI. system is designed to select queries from a fixed pool of unlabeled examples in order to reduce the uncertainty about its own outputs. Several works have explored AL strategies to improve the performance of LLMs on few-shot learning tasks performed with in-context learning Zhang et al. (2022b); Margatina et al. (2023); Diao et al. (024). By acquiring new labeled examples, the generative distribution of the LLM is expected to shift towards outputs consistent with the ground-truth labels. In our setup, we do not have access to a fixed pool of questions that the agent can choose from. Instead, the agent generates a question on its own. Noting that question can be of the form "What is the label for an input x?"", active task elicitation can be seen as a generalization of AL. Furthermore, as observed by Zhang et al. (2022)), in-context learning performance can be highly unstable across sample examples due to the idiosyncrasies of how LLMs update their generative distribution when extending the set of in- context examples. Given the unpredictable nature of the LLMs' distribution, our work focuses eliciting binary task requirements, enabling the agent to filter its own outputs that do not conform to task requirements specified by the user. By extending the problem statement with additional requirements, the bias of Pás, changes at each iteration, yet it remains unknown whether this change aligned with p". By encouraging uniformity of Pos. over the set of compatible solutions. our active-reasoning framework steers the agent to consider many possible interpretations of a task at each point of the interaction, resulting in the selected questions being less biased towards most likely interpretations according to the possibly misaligned language model.
Clarifying questions and generative task elicitation. Before the emergence of LLMs, prior works (Rao& & Daumé III] 2018 2019 Min et al., (2200) have considered the problem of learning single- turn clarifying questions, with the question generator trained as grquence-to-squemne RNN's based on pre-collected dataset of problems, clarifying questions, and their answers. More recently, in order to effectively address ambiguous user questions, KKasheninnikovv et al. (2022) fine-tune the GPT-3 model on data set of conversations consisting of ambiguous user requests, clarifying ques- tions, and final answers. Kuhn et al.] (2022) show that LLMS can reason about ambiguous aspects of query and generate clarification questions with zero-shot prompting. Similarly, [Li et al. (2025) capitalize on zero-shot prompting of LLMs and introduce a framework in which LLMs infer in- tended behaviour by querying the user with examples to label, yes-or-no questions or open-ended questions. They show that the LLM-generated queries are more efficient and require less effort than user-written prompts, enabling the discovery of initially unanticipated considerations of a task. Our work demonstrates that the L.L.-ggnerated questions can be improved by encouraging the agent to explicitly reason at inference time about the space of viable outputs given its current knowledge about the problem. This aligns with the principles highlighted in [Groenendijk (1944), where reason- ing about the semantics of questions plays a crucial role in shaping subsequent answers.
Preference elicitation with LLMs. A number of recent studies (Yang et al. 2021 Piriyakulkij et al.) 2023 Handa et al. 2024 Austin et al., 2024) have leveraged LLMS for user preference elicitation, employing ideas of BED to select most informative queries. Despite surface-level similarities, these approaches are targeted at recommendation systems operating on a pre-determined set of objects or fixed feature spaces. For instance, (Handa al. 2024) present an interactive preference elici- tation framework wherein a linear Bayesian model is used to describe user preferences over a set of features elected prior to the start of user interaction. In this setup, the LLM's role is limited to feature extraction and query verbalization. In contrast, our paper focuses on scenarios where the LLM reasoning agent is expected to output a solution that belongs an unconstrained space of natural language, not pre-determined by feature space of fixed dimensionality nor fixed list of hypothetical answers. This necessitates an iterative sampling of solutions at each interaction step to approximate the currently available options. Albeit more challenging, the unconstrained setting closely reflects the real-world usage of L.LMs as general purpose reasoning agents. Moreover, enables the LLM agent to query the user about aspects of hypothetical solutions at varying levels of granularity, and crucially, as noted by Li et al. (2025). ask about aspects of given problem that could have been difficult to anticipate before engaging the interactive dialogue.
Ambiguity in natural language tasks, Prior work highlights the importance of natural language ambiguity and its direct impact LLM performance in instruction following. Liu et al. (2023) introduce a benchmark to assess LLMs' ability to recognize ambiguous sentences and disentangle their possible meanings. Tamkin et al. (2023) propose benchmark of ambiguously specified classi-
16

---

Published as a conference paper at ICLR 2025
fication tasks, requiring LLM agents to infer missing information. Niwa & Iso (2024). similar to work, define ambiguity terms of the sets of compatible and ground-truth solutions. Additionally, they introduce ambiguity taxonomy that categorizes different types of instruction ambiguities and propose methods to refine initial instructions with clearer specifications.
Other related works. Recent studies have further explored the capabilities of LLMs in interactive and active learning scenarios. Hu et al. (024) introduce Uncertainty of Thoughts (UoT), an algo- rithm designed improve LLMs' information seeking by enabling them ask effective questions. This approach uses uncertainty modeling to guide the LLM in actively reducing the LLMs uncer- tainty. Similarly, Astorga al. (2024a) investigate active learning with LLMs in settings where data is partially observed and acquiring information has costs, focusing on efficient feature and label acquisition. Building upon the theme of question generation, (Andukuri et al.) (2024) propose STaR- GATE, a method that rewards language models for generating useful clarifying questions, enhancing their ability to resolve task ambiguity through self-improvement.
B POTENTIAL APPLICATIONS OF ACTIVE TASK DISAMBIGUATION
Active task disambiguation holds promise across a broad spectrum of applications. For instance, in personalized education and intelligent tutoring systems, clarifying questions can reveal subtle learn- ing objectives and misconceptions, enabling real-time adaptation of instructional strategies (Aleven et al. 2016 Graesser al. 2005; Krasheninnikov et al. 2022). In software engineering, interac- tive requirement elicitation helps mitigate misinterpretation during system design and development, efficient coding cycles and improved software quality (Jirotka & Goguen| 1994 Li
leading to ett al. 2025). Creative industries-ranging from content generation and design to game develop- ment may also benefit by aligning Al-generated outputs with users' evolving intensions (Amershi
et al. 2019)
Beyond conventional machine learning tasks, active task disambiguation may be extended a di- verse array of problem-solving and decision-making domains. In scientific research, systematically clarifying experimental constraints can support more precise experiment design and hypothesis re- finement (Montgomery (2017). In robotics and autonomous systems, iteratively disambiguating mis- sion goals and environmental constraints key achieving safer, more adaptive behavior (Paden) et al. 2016; Thrun 2002). In healthcare, actively refining diagnostic criteria and treatment proto- cols can improve patient outcomes by tailoring interventions to individual needs (Topol 2019). In addition, we envision active task disambiguation to enhance areas like: autoformulation (Astorga
ct al. 2024b), informed machine learning and autoML (Kobalzykk & Schaar 2025). ) legal reasoning
(Rissland 1988), financial advisory (De Prado 2018). emergency response planning (Zhang al.
2022a), urban planning (Sevěíková al. 2007), policy- support (Bowen & Zwi 2005),
context-aware testing (Rauba et al. 2024b). self- self-healing systems (Rauba et al. 2024a). or cyberse-
curity (Purushottam et al. 2024).
17

---

Published as a conference paper at ICLR 2025
C ACTIVE TASK DISAMBIGUATION
Algorithm Active requirement elicitation
Require: Initial problem statement 8 = (R", C), Max number of iterations T, Number of solu- tions sampled per iteration N, Number of questions sampled M.
for t in , ; do
{h:) N
Pd. (-1S)
1 ~ po
(+)*)
if M then
9" + qi
else if M > 1 then
for j in 1....,, M do
for in ... N do
af Po. (-14 of)
end for EIG [o]] < estimate EIciiatteeeI((..... (af (af.)mm) end for qº +- arg maxxxaxx EIG [g]
end if
a" t- get.oracle.nte get.oracle.anser((())
transform teapsformttorrqquueee req.irreettt
R441 <- (r)
St+1 +- (R++1
end for
Algorithm astimate_EII(( (at 1
Require: A question 9j and set of N answers fassl (a1.....a., an + Unique answers in {a
for k E (1, (1...., n do
n4 4- 11 as ag, i [N]H
pk na/Pk
end for return
-Et= Pk log(ps)
The methods of obtaining Pa,, Pe. Pa. well transform,tt heterattorrrrrreeeeeeeeeeee rement and itt.oracle_anseer subroutines are application-specific. We provide the specific prompts used sections D The experiment of the 20Q game 4. also considers alternative method for estimat- ing the EIG based on the token log-probatd reg--obbaiilities of sample solutions (
Algorithm estimate EIG logp(g) {ar.) (p:)
Require: A question Gj. the set of N answers {aig) log-probabilities of sample solutions (p.).N
{aI...., an) +- Unique answers in {a
for E f1,...,, n do
Pk - (tias, exp(p:)
end for
for E {l,...,,7) n do
Pk + pk/ E Pr
end for
return
- -E Ph log(ps)
18

---

Published as a conference paper at ICLR 2025
D PROMPTS AND EXPERIMENTAL DETAILS
D.1 THE 20 QUESTIONS GAME
D.1.1 PROMPT TEMPLATES
When eliciting the requirements for the game of 20 questions we use the following set of prompts:
Solution generating prompt
As an AI assistant, your role is to generate wide and diverse range of animals that strictly meet the specified requirements. Your objective
guess animal from the entire animal kingdom that satisfies these requirement.s:
fList of requi rements R'}
For this, generate carefully selected, diverse, and representative set of [N] animals following this scheme:
1. <H>
<>>
(N).
Fi11 <H> with full name animals.
Question generating prompt
Your objective ve is guess animal from the intire animal kingdom that satisfies the following
fList of requitements R'} For this, generate the most informative yes/no question."
Question selection prompt (used only for implicit-ToT)
Your objective is to guess an animal from the entire animal kingdon that satisfies the following requitements:
fList of requirementa R4}
For this, select the most informative yes/no question from this list:" (List of questions (
Answering prompt, ( Po. (144)1
You are an expert critic that specializes in responding to yes/no questions. Given these animals: 1. (h]). <A>,
2. (ha) <A>
N. (hy). <A> For each animal fill <A> with YYss' or Noo with the response for this question:
(Sample question 3}
Requirement transform prompts
Question statement:
Transform the following question into an affirmative statement You should start with The animal', Do not write anything clsc than the affirmative statement.
Statement negation:
Transform the following question into an i statement. statement
You should start with 'The animal'. Do write anything else than the affirmative statement.
19

---

Published as a conference paper at ICLR 2025
Oracle answer prompt (for GPT--to-minismmlaainn player A)
Given the animal: and the question: q', respond to the question depending the given animal. You can only respond one of the following answers.
Yes: If the answer is yes.
No: If the answer is no.
Pass: If the answer could be either yes
Think step by step. After your thinking process respond: 'The final answer is <Response>'.
D.1.2 EXPERIMENTAL DETAILS
Animals for the main setup. The focus of the main experiments is to evaluate the efficacy of the requirements elicited with various question-generan strategies. To do this, we select an ar- bitrary list of 15 animals the ground-truth solutions h" These are: 'Chameleon', 'Ladybug', 'Soordish'. AAmaddillo, 'Crab', "Grasshopper', 'African Grey Parrot', 'Manta Ray', 'Rabbit', the
"Flamingo' 'Pangolin', 'Toucan', 'Starfish', 'Tarantula', 'Salamander'. For each h", game for T = 10 iterations and with 5 different seeds. For solution and question generation we use N 20 and .M- 5, respectively.
Construction of the list of 500 animals for study 1. As discussed in the theoretical section of our paper, a question that splits the solution space into roughly equal parts should result in the highest information gain. Since we do not restrict the user nor the reasoning agent to a pre-fixed list of animals, we cannot directly assess the partitioning properties of question. Instead, during active question generation we resolve to sampling of individual h, 's which should approximate the entire solution space. The aim of this study to evaluate partitioning properties of the generated questions much larger sample of solutions. To this end, construct large and diverse list of 500 animals. This list was obtained by prompting GPT-4o-minito stratify the animal kingdom into several categories. In response 100 mammals, 100 birds, 100 invertebrates, 50 amphibians, 50 reptiles and 100 fish. Then, we subsample the respective number of animals in each category from the animals generated within main experimental results until the sixth iteration. The stratification ensures that the constructed list sufficiently diverse.
obtained the following categories and their cardinalities:
D.2
CODE GENERATION
D.2.1
PROMPT TEMPLATES
When eliciting the requirements for the game of 20 questions we use the following set of prompts:
Code header for St
def <function <funation_nanee oo__ioonnnnnaaee (<input_name>))
<Comment explaining the expected functionality the code>
Examples:
<Generated test cases>
Solution generating prompt
You are expert Python programmer that specializes in coding tasks. Complete the function given by the user. Do not change the function signature. Do not add any additional commentary. Do not import any additional libraries. Start your answer with the function signature.
<Code header for s
20

---

Published as a conference paper at ICLR 2025
Question generating prompt (open)
You are an expert Python programmer that specializes in solving user-specifled coding tasks. To ensure you correctly understand user specifications, you query the user for expected program outputs of sample inputs. Given the function signature, generate <M> sample inputs that will most helpful formalizing user intent. Structure your response as a 1ist of function calls:
1. <function name> (input <function_namet (input_2)
<M> <function_name (input_<t>)
Do not generate any additional content beyond the numbered list of function calls. Do not repeat the same inputs as in the Examples given.
<Code header for S*>
Question generating prompt (binary)
You are expert Python programmer that specializes in solving user-specified coding tasks. To ensure you correctly understand user specificationn you can write additional test cases. Given the function signature, generate <м> sample test cases that will be most helpful in formalizing user intent. Structure your response as list of assertion:
1. assert (function_nant (irput_l) -- output_1
2. assert <function <functiio_name> (input_2) -- output_?
<M> assert <function name> (input <M>) output_ <M>
Do not generate any additional content comments beyond the list of assertions.
<Code header for S'>
D.2.2 EXPERIMENTAL DETAILS
We conduct our experiments on two benchmakrs: HumanEval (Chen et al. 2023) and APPS Hendrycks et al. 2021). To ensure a sufficient level ambiguity of the problem statements, we filter the set of tasks in these benchmarks to the set of non-trivial tasks, i.c. task that cannot be solved with a 100% accuracy zero-shot, i.c. without acquiring additional requirements. In addition, the APPS benchmark is filtered the set of tasks that do not contain any example input-output pairs in the original problem formulation. Resulting datasets contain 48 and 47 tasks, respectively.
E ADDITIONAL RESULTS
E.
THE GAME OF 20 QUESTIONS
Table 2 shows the final ranking results the game of 20 questions.
E.2
CODE GENERATION
E.2.1 FULL RESULTS
Table 3 shows the accuracies the generated code samples at each iteration on the HvmanEval benchmark. Table 4 contains the corresponding results for the APPS benchmark. Despite the more challenging nature the APPS benchmark, as indicated by the significantly lower accuracy across all models at 0, our conclusions still hold. The ElG-based strategies outperform their non-EIG equivalents by a significant margin, and the "open" type of questions result in larger information gains than "binary" questions.
21

---

Published as a conference paper at ICLR 2025
Table 2: Average ranks for the game of 20 questions. Rankings averaged across ground-truth animals, 5 run seeds, 25 evaluation seeds. Standard errors in brackets.
(a) GPT-3.5-turbo
(b) GPT-4o-mini
EIG-uniform
EIG-logprobs implicit-ToT implicit
EIG-uniform
EIG-logprobs
implicit-
implicit
1
0.8 (0.0)
1.0 (01)
1.2
0,9 (0.0)
1
0.4 (0.0)
0.3 (0.0)
0.2 (0.0)
0.1 (0.0)
2
1.S (0.1)
(.1)
1.7
1.5 (o.1)
2
1.1 (0.1)
0.9
1.1 (0.1)
1.1 (0.1)
2.4 (0.1)
1.9 (0.1)
2.4 (0.1)
2.1)
2.0 (0.1)
1.7 (0.1)
(0.1)
2.1 (0.1)
4
2.9 (0.1)
(.1)
3,0 (0.1)
2.5
3,4 (0.1)
2.6 (0.1)
2.7 (0.1)
2.9 (0.1)
5
3.9 (0.1)
(.9
(.1
3,0
5
(.1)
3,8 (0.1)
3,8 (0.1)
3.1
6
4.1 (0.1)
3.2
(.1)
3.1
6
5,5
4.7 (0.1)
5.1 (0.))
4.5 (0.1)
7
4.7 (0.1)
3.4
(.1)
4.0 (0.1)
6.1
5.1 (0.1)
5.7 (0.1)
5.1 (0.1)
8
5.0 (0.1)
3.6
(.1)
4.3 (0.1)
6.5
5.5 (0.1)
6.1.
5.3 (0.1)
9
5.9 (0.1)
(..)
4.8 (0.1)
4.7 (0.1)
9
6.7 (0.1)
5.7 (0.1)
6,5 (0.1)
5.4 (0.1)
10
6.2 (0.1)
4.1
5.1 (0.1)
4.9 (0.1)
10 6,8 (0.1)
6.0 (0.1)
6,6 (0.1)
(.1)
(c) Llama3-70B (Instruct)
EIG-uniform
ElG-logprobs Implicit-ToT implicit
1
1.3 (0.1)
0.4 (0.0)
0.8
0.5 (0.0)
2
1.5
0.5 (0.0)
1.2
0.9 (0.1)
1.7 (0.1)
0.9 (0.1)
(0.1)
(0.1)
4
2.4 (0.1)
(.2)
2.4 (0.1)
2.1 (0.1)
5
3.0 (0.1)
(.5)
2.8 (0.1)
2.7 (0.1)
6
3,8 (0.1)
(.1)
(.1
3.1)
4.3 (0.1)
2.2
3.7
(..)
8
(0.1)
2.7
(.1.
4.3 (0.1)
9
(.1)
3.2
(.1
5.3
10
6.3 (01)
(.1
5.1
5.9 (0.))
Table 3: Accuracy (%) of solutions on HumanEval benchmark. (B)-solutions generated with "bi- nary" questions. (O)-solutions generated with "open" questions. Results averaged across 48 tasks, 3 run and evaluation seeds per task.
(a) GPT-3.5-turo
(b) GPT-4o-mini
base (B)
EIG (B)
base (O)
EIG (O)
base (B) EIG (B) base (O) EIG (O)
0
44.1 (1.) 44.6 (1.8 47.1 (1.4)
47.0 (1.4)
0
(1.0
69.6 (2.1) 71.4 (2.1) 70,6 (2.1)
1
(2.5
66.8 (2.))
67.5
74.4 (1.6)
1
73.6 74.6
77.1 (2.0
79.5 (1.8)
2
65.2
(..4
(1.6
(1.6
2 76.7 (20) 78.9 (2m) 78.8 (1.) 83.9 (1.7)
70.7 (2.)
82.2 (2.1)
75.5 (1.7)
84.5 (1.5)
3 77.0 (2.) 80.6 (1.)
80.8 (1.)
83.7 (1.8
70.8 (2.))
85.6 (2.0) 75.9 (L.3)
85.0 (1.5)
4 79.6 (2.) 81.5 (1.) 83.5 (1.) 83.8 (1.8
(c) Llama3-705 (Instruct)
(d) Llama3-8B (Instruct)
base (B)
EIG (B)
base (0)
EIG (0)
base (B)
EIG (B)
base (0)
EIG (0)
0
63.9 (2.0) 63.8 (2.0) (2.0)2.0)
63.6 (2.0)
0
34.8 (1.7) 34.7 (1.7) 34.6 (1.7)
34.7 (L.7)
1
71.3 (2.0)
79.3 (1.7)
72.9 (2.0)
79.7 (1.7)
1
40.9 (1.9)
48.8 (1.9)
53.7 (1.9)
62.3
2
73.0 (200
75.2 82.5
82.8 (1.7)
2
45.6 (1.9) 56.2 (2.0) 58.9 (2.0)
70.5 (2.0)
3
75.4 (1.9
84.7 (1.6) 78.3
84.2 (1.7)
3
49.6 (2.0)
60.4 (2.0) 60.6 (2.1) 74.4 (2.0)
78.8 (1.8)
87.5 (1.4)
81.2 (1.8)
85.5 (1.5)
53.0 (2.)
65.8 (2.0) 64.8 (2.0) 75.5 (1.9)
E.2.2 AMBIGUITY IN HUMANEVAL
We note that the low accuracy of generated samples Pas (-159) does not imply that S itself is inherently ambiguous, as it may simply caused by the LLM's limitations following complex instructions. However, even if 8 is objectively not ambiguous, the input-output examples appended
22

---

Published as a conference paper at ICLR 2025
Table Accuracy (%) of solutions on the APPS benchmark. (B)-solutions generated with "binary" questions. (O)-solutions generated with "open" questions. Results averaged across 47 tasks, 3 run and evaluation seeds per task.
(a) GPT-3.5-turbo
(b) GPT-4o-mini
base (B)
EIG (B)
base (0)
EIG (O)
base (B)
EIG (B)
base (0) EIG (O)
32.1 (1.) 31.2 (1.6 32.1 (1.6)
32,9 (1.7)
35.8 (1.I) 35.9 (1.2) 34.6
35.9 (1.2)
1
(1.)
49.5 (2.0)
(1.9
(2.0)
1
53.9 (1.7)
62.0
59.2 (17) 68.0 (1.9)
2
54.3 (19)
(200
59.0 (1.9)
68.3 (2.0)
2
57.5 (1.) 67.7 (1.) 68.8 (1.))
83.0 (1.6)
3
60.8
61.5
68.2 (1.8)
74.6 (1.8)
3 60.8 (1.9) 72.9
72.7 (1.)
85.7
62.4
71.3 (1.9) 69.7 (1.9)
77.3 (1.8)
62.9 (1.9)
75.3 (1.8
75.1 (1.8
87.2 (1.5)
(c) Llama3-70B (Instruct)
(d) Llama3-8B (Instruct)
base (B)
EIG (B)
base (O)
EIG (O)
base (B)
EIG (B)
base (O)
EIG (O)
38.0 (1.9)
38.3 (1.8) 38.4 (1.8)
37.8 (1.8)
(1.6
8.5 (1.6)
9.1 (1.6)
(1.8)
1
49.0 (2.0)
54.7 (1.) 57.1 (2.0)
63.4 (2.0)
1
13.5 (1.9
(2.0
21.8 (3.0)
25.3 (3.4)
2
51.6 (2.0 60.1 (2.0)
(21
74.9 (1.9)
2
(2.4 19.7 (2.)
26.9 (38) 28,4 (4.3)
3
57.7
65.7 (2.1)
68.8 (2.0)
78.3
3
14.5 (2.1)
22.7
32.1
34.2 (5.4)
60.1 (2.0)
68.5 (2.0)
73.5 (2.0)
81.6 (1.8)
18.5 (2.5)
26.9 (377)
39.2 (49)
50.7 (5:7)
to the prompt St may positively bias Pé. towards correct solutions, which is desirable by-product of employing active querying strategies to task disambiguation. ensure, however, that the code- generation experiment is indeed appropriate test our strategy for ambiguous prompts, we have listed table 5 sample tasks with explanation of their ambiguity.
23

---

Published as a conference paper at ICLR 2025
Table 5: Sample ambiguous problem statements from the HumanEval benchmark.
problem statement
ambiguity
From typing import List
How to handle characters the input string other than paren- theses and spaces? How to han- dle strings with non-balanced groups parentheses?
patpraate_aaeenggeeeee (paces_atrings stx) [sst] Topuu to this function 1s a string containing mult multiple groups of nested pazentheses. Youz goal
separate those group into list of those.
strings and return the
bal aalldded leach open brace properly closed) and not nested within each other input string.
groups
Ignore sny spaces
typing import List
Should all duplicated ele- ments be removed or one unique element retained?, e.g. semove.duplicates 4, 3, 5, 5)) ->14, 3, 5] or (3)
def remove remove_dup duplicates lnumbers: List (Int]) biet [int]:
From list of integers, remove all elemeats that occus
than Keep arder of alements left the same as in the input. *na
def is [s_bored
Is sentence starting with "T'm" boredom not?
You' be given a string of words, and your task 1a to count of boredoma. A boredom is sentence that starta with the word "I"
Sentences delimised ',', "2* !!
histogram(teat):
How to handle letters that are not separated by spaces?
separated lowercase of the letter with the repetition and containing the occurreace, return all
***Gienn string representing letters, zeturn a dictionaty
corresponding count.
If several them.
def cyepattern_chezktf b):
What is understood by a string rotation?
*naYou are given words. You need to reture True if the second word any rotations substring in first word
[(n):
Should the sum of to include i?
Implement the function f that takes as parameter,
and returns el elament at index ia the factorial of i if 1 ia even or the sum of numbera from 1 1 otherwise.
size such that the
starts from 1.
the factorial of 1a the multiplication of the numbers from .... 1).
24

---

Published as a conference paper at ICLR 2025
F COSTS OF QUESTION ELICITATION
We note that the ElG-based creased number of LLM calls compared to the zero-shot baselines. In line with the assumptions commonly made in BED, we take the stance that the computational load required to select the opti- mal query is negligible compared the value of acquiring information that reduces problem ambi- guity. We anticipate this assumption will become more valid over time as technology advancements lower the costs of LLM token generation, thereby enhancing the importance of efficient informa- tion acquisition strategies. However, given that in many real-world applications design choices may be constrained by the L.M sampling costs, we include comparison of the effective number of LLM calls required at each interaction step for the key strategies considered. In the below, N is the number generated solutions each step, and M is the number of generated candidate questions.
strategies presented in this work require an in-
F.1 20 QUESTIONS
Strategy
Cost
implicit (baseline)
0(1)
implicit-ToT
O(M)
EIG
O(N + NM)
In the baseline strategy, only call is needed to sample single question, 9"
In the implicit-ToT strategy, additional M calls are needed to generate a set candidate ques- tions, from which one is selected.
In the EIG-based strategies (EIG-uniform and EIG-logprobs), M calls are needed to sample a set of candidate questions {qz )== and N calls sample the hypothetical solutions (h₂ ) To select the best question, we need to estimate the EIG, which requires the LLM to answer each question, {j- about every sampled solution, h; thus requiring additional NM LLM calls.
F.2 CODE GENERATION
Strategy
Cost
base EIG
O(1)
O(N + M)
In the code generation example, the evaluation of the EIG does not require additional LLM calls. The answers, arj obtained by executing the code solution, h; against each question gj.
25


Reviews:
[{"review_id": "G7MVV3p2aS", "reviewer": "Reviewer_xMi4", "summary": "This paper studies the question/user intent disambiguation problem for LLM agents. When the initial query do not provide enough information for LLM agent to solve the task, the agent should actively ask user questions to better clarify the task to solve. This paper proposed a method that can select the best clarification question at every turn. In particular, the method first samples a set of possible answers based on the existing information, and a set of possible clarification questions, then the question is selected based on the EIG score (expected information gain). The question along with its answer is added to the problem context for the next round of prompting. This execution loop can repeat for a few iterations until reaching a final answer. The authors evaluated the proposed method on two datasets, 1) 20-question games about animal names and 2) HumanEval coding tasks. The results suggest that the propose method of selecting clarification questions leads to better ovrall results than vanilla zero-shot question generation.", "questions": "1. For the two benchmarks tested in this paper, the EIG scores can be easily computed since there are only single word answers or executable code. I’m wondering how would you compute the EIG scores when the answer set is arbitrarily large. For example, for free form response generation where all answers are unique, and each answer can be arbitrarily long?\n\n2. There is a typo in Line 900", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The formulation of EIG scores for selecting the best clarification question is well designed.  Intuitively, the question that brings maximum information gain will narrow down the possible search space the most,  but how to prompt the model to generate the most informative question is challenging. The authors did a nice work by shifting the question generation problem to a question selection problem, which leverages LLM’s reasoning ability and potentially external tools that provides more accurate signals for the selection. \n\n2. The proposed method of generating/selecting best clarification questions can have wide application to different tasks. For example, in many real-world applications where LLM agents can be deployed, e.g. travel agent or coding agent, the user queries can be ambiguous and miss critical information. The proposed method can potentially be applied to help the agent understand uesr intent in fewer turns and thus complete tasks more efficiently.", "weaknesses": "The benchmarks used in this paper are too simple. For example the 20-question games only require guessing animal names, and it is designed for asking clarification questions. For HumanEval, it also contains mostly simple coding problems. It would be better if we can see results from more challenging agent benchmarks such as SWE-bench.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NFoeR2kU4U", "reviewer": "Reviewer_qfUE", "summary": "This paper addresses task ambiguity, where the intended goal of the user is not clear/is underspecified. The proposed method seeks to reduce ambiguity by asking follow-up questions to the user that clarify their intent. The authors use LLMs to both perform tasks and generate questions, and implement an information-gain-based ranking method to obtain questions that best reduce uncertainty; the method is based on Bayesian Experiment Design, and tries to partition the space of possible solutions s.t. the partitions are maximally balanced. The method is tested on two tasks and three LLMs. First, the authors test on a 20-questions style game where the agent has to correctly guess an animal by asking questions about its properties. Here, their method results in a better ranking of the correct animal, and they show that their questions eliminate more possible alternatives than a baseline without information gain. The second domain is coding, where the authors show that their method is able to generate unit tests (which act as questions). Here, their unit tests generally result in improvements on HumanEval, both when unit tests are posed as true/false statements and when open-ended unit tests are permitted.", "questions": "Relevant references:\n- https://arxiv.org/abs/1805.04655\n- https://stokhof.org/wp-content/uploads/2020/09/groenendijk-stokhof_ssqpa.pdf \n- https://arxiv.org/abs/2004.10645\n- https://arxiv.org/abs/2211.07516", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "**Quality:** The work rigorously defines its problem and then offers a potential solution. The authors do a good job of illustrating that their method works/how it works on 20 questions, while still showing real-world results on a naturalistic dataset in a coding domain, where many users actually use LLMs on a daily basis. This is a major strength of the paper. The experiments themselves are generally clearly described and well-documented/well-executed. The authors also run across a large number of seeds and release code, making their approach more reproducible. \n\n**Significance:** Ambiguity and task ambiguity are increasingly important problems as more people use LLMs and AI systems. The authors make a strong case in the introduction for why ambiguity is important and the kinds of risks posed. The clarification-based approach they suggest has the potential to improve AI safety by reducing misunderstandings, and takes a human-centric approach to dealing with ambiguity.\n\n**Originality:** The information-gain approach to dealing with ambiguity that is proposed here is novel, and the definition of task ambiguity in definition 1 is valuable.\n\n**Clarity:** The paper is generally well-written and clear. I appreciated the colored boxes that provide additional detail/clarification. I found the formalizations made the paper more precise and were accessible. Several questions I had were preemptively addressed, e.g. on L509-511 where the authors point out that adding unit tests could provide other avenues for improvement beyond ambiguity. The figures (especially figure 2) nicely illustrate the problem and method.", "weaknesses": "**Empirical gains vs. cost**: Looking at Fig. 3, it seems like the proposed method improves the rank by ~2 ranks at iteration 10 for GPT3.5 and ~1 for GPT-4o-mini. While these results are significant, I do wonder (especially given the smaller gain on GPT4o-mini) whether they track with the added computational cost of the method. It would be worth noting the cost (i.e. number of tokens, number of model calls) to be able to directly compare these methods, since some of the baselines like ToT also incur more computational cost. \n\n**Baseline clarity**: It's not completely clear what the ToT baseline is. Is this tree-of-thought? If so, it should be cited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JBFo33SRdH", "reviewer": "Reviewer_HcvV", "summary": "The paper addresses the challenge of large language models (LLMs) dealing with ambiguously specified problems, which is common in real-world interactions. The authors propose a Bayesian Experimental Design framework to tackle task disambiguation by having LLM agents ask clarifying questions to acquire additional task specifications. This approach requires meta-cognitive reasoning, which the authors argue LLMs currently lack. They introduce a method to generate targeted questions that maximize information gain, shifting the reasoning from implicit to explicit. Empirical results show this method leads to more effective task disambiguation compared to question-space reasoning approaches. The paper contributes by identifying the need for new reasoning methods, formalizing task ambiguity, proposing a BED-based strategy for question generation, and evaluating its effectiveness in interactive task elicitation scenarios.", "questions": "* IG in Eq. (2) denotes information gain?\n* What does $c(q)$ mean ? Does it mean cost of API usage (tokens) ? Please clarify it.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* It introduces a novel method to handle ambiguity in task specifications by leveraging the principles of Bayesian Experimental Design, which is a significant advancement in the field of LLMs.\n* The paper provides a clear and formal definition of task ambiguity, which is crucial for framing and addressing the problem within a mathematical and computational context.", "weaknesses": "* Consider adding a concrete example of task ambiguity in the main text, rather than relegating it to supplementary materials. A practical example would help readers grasp this key concept more easily.\n* Why just show the rank in Figure 3?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)