Task Ambiguity in Humans and Language Models
Authors: Alex Tamkin
Kunal Handa
Avash Shrestha
Noah Goodman
Venue: ICLR 2023 poster (2023)
Keywords: task ambiguity
safety
language models
few-shot learning
in-context learning

Abstract: Language models have recently achieved strong performance across a wide range of NLP benchmarks. However, real world tasks are often poorly specified, and agents must deduce the intended behavior from a combination of context, instructions, and examples. We investigate how both humans and models behave in the face of such task ambiguity by proposing AmbiBench, a new benchmark of six ambiguously-specified classification tasks. We evaluate humans and models on AmbiBench by seeing how well they identify the intended task using 1) instructions with varying degrees of ambiguity, and 2) different numbers of labeled examples. We find that the combination of model scaling (to 175B parameters) and reinforcement learning from human feedback (RLHF) enables models to approach or exceed the accuracy of human participants across tasks, but that either one of these alone is not sufficient. In addition, we show how to dramatically improve the accuracy of language models trained without RLHF by finetuning on a small number of ambiguous in-context examples, providing a promising direction for teaching models to generalize well in the face of ambiguity.

TASK AMBIGUITY IN HUMANS AND LANGUAGE MODELS
Alex Tamking Kunal Handa", Avash Shrestha, Noah Goodman Stanford University
ABSTRACT
Language models have recently achieved strong performance across a wide range of NLP benchmarks. However, unlike benchmarks, real world tasks are often poorly specified, and agents must deduce the user's s intended behavior from a com- bination of context, instructions, and examples. We investigate how both humans and models behave in the face of such task ambiguity by proposing AmbiBench, a new benchmark of six ambiguously-speeifeee humans and models on AmbiBench by seeing how well they identify the intended task using 1) instructions with varying degrees of ambiguity, and 2) different num- bers of labeled examples. We find that the combination of model scaling (to 175B parameters) and training with human feedback data enables models to approach or exceed the accuracy of human participants across tasks, but that either one alone is not sufficient. In addition, we show how to dramatically improve the accuracy language models trained without large-scale human feedback training by finetun- ing on a small number of ambiguous in-context examples, providing a promising direction for teaching models to generalize well the face of ambiguity.
classification tasks. We evaluate
INTRODUCTION
Language models have recently been applied to wide range of NLP benchmarks, ranging from question answering, summarization, and logical reasoning, to solving riddles, dark humor detection, and ASCII word recognition (Brown et al. 2020; Srivastava et al. 2022). Performance across tasks has improved as models and datasets have grown in size, raising the prospect of a route towards generalist NLP models with broad utility.
However, one feature many of these benchmarks share is that they are carefully designed to make the desired task very clear to the language model, since this is prerequisite for establishing perfor- mance on that task. Unfortunately, real-world uses of language models are not likely to feature such
Output 'X' if the sentence contains [category withheld] and 'Y' otherwise.
This is unclear. What category am looking for?
The worm in the meadow > 'X'
Hmm... it looks like the category is either animals or outdoor locations?
The student is in the museum 'Y'
The duck is the canyon XX
The reporter
cave
Aha! The category must be animals.
Figure 1: Complex tasks are often hard to specify precisely, leaving important pieces of information missing. Agents should be able fill in the blanks by combining information from instructions and examples in order to identify the intended behavior.
"Equal Contribution. Correspondence to wtw.kin@@es.stanfiooii

---

thought and clarity in their task specification. Rather than iterating over and perfecting specifica- tion for their tasks, everyday users of language models may wish to define tasks on an as as-needed basis, without worrying that they will be misunderstood. More pressingly. in complex domains featuring high-dimensional inputs and outputs (e.g. programming, verification, generation) it is un- likely that even thoughtful task specification will manage to perfectly capture all the features of an input and output which are salient or not salient to the task. This is especially important for safe and robust deployment of language models, as such undesirable dependencies can be hidden hazards that are only revealed when model fails catastrophically in new setting (Geirhos et al. 2020).
To operationalize this problem, we introduce AmbiBench, new benchmark of six ambiguously- specified tasks. Each input in AmbiBench is sentence (e.g. The dog is in the meadow) that has multiple associated classification tasks based on different linguistic features (e.g. contains an animal, contains an outdoor location). Task ambiguity arises when more than one task is consistent with the provided instructions or labeled examples
We establish how well different models and humans perform on ambiguously-speeified tasks, given a wide range of task specifications including clear vs unclear instructions and zero vs multiple exam- ples. We find that the largest models trained with human feedback data (HFD) match or outperform human participants across all specifications we try, though all underperform a Bayesian oracle.
We also show how to improve standard language models' performance by finetuning them on a small set of in context examples that demonstrate the desired generalization. This form of meta-learning dramatically improves model's ability to learn new ambiguously-specifiee tasks. This suggests a possible mechanism for why the HFD models outperform standard language models (discussed in Section (.4). well as promising direction for improving how models learn in ambiguous contexts.
To summarize our contributions, we:
1. Introduce and motivate the problem of studying task ambiguity in large language models
2. Evaluate humans and models on a new benchmark of ammiiuoussy-speciine
tasks. demon- strating that while pure language models fail to disambiguate the intended task well, sufficiently-large models trained with human feedback data are able to approach or even exceed the performance of our human participants to resolve the ambiguity between tasks
3. Show how finetuning on ambiguous in-context prompts and examples can enable traditional language models to surpass the performance of HFD models when evaluated on unseen tasks, providing a promising route towards models that capably manage task ambiguity
RELATED WORK
2.1 AMBIGUITY IN NATURAL LANGUAGE PROCESSING
Ambiguity is well-studied topic in NLP, with work spanning topics diverse search queries
EcronenTowweee & Crofi 2002 Wang & Agichtein 2010), question answering (Min et al. 2020 Zhang & Choi 2021). named entities (Bunescu & Pasca 2006) Cucerzan] Dredze et al.
2010). coreference resolution (Webster et al. (2018). machine translation (Stanovsky et al. 2019
and information-sede seeking dialogues (Aliannejadi et al. 2019 Guo et al. 2021 Aliannejadi et al.
2021
Sun et al. 2022 Wu al. 2022).
Our work differs from these prior streams of work by studying task ambiguity (Finn et al. 2018 Tamkin et al. (022c), where the task the agent is being asked to perform is ambiguous, rather than an ambiguous input for a clear task. This is of special relevance for self-supervised learning models that are trained for adaptation to a broad range of downstream tasks (Bommasani et al. 22021] Tamkin al. 2022b). In these settings, models must infer the correct task from user's 's specification, as opposed to a possibly unsafe or undesirable task that is also consistent with that specification.
'Importantly, task ambiguity is distinct from clearly-specified tasks with ambiguous inputs, e.g. determin- ing the referent of the pronoun in sentences like the nurse handed the doctor her phone. Here, the task is clear (determine who her refers to), but there is not enough information in the input to answer it.
2

---

2.2 IN-CONTEXT LEARNING AND PROMPTING
Task ambiguity is especially relevant for language models, which can be adapted for many different tasks via in-context learning (Brown al.) 2020 Tamkin al. 2021a: Bommasani et al. 2021 Liu
et al. 2022b), and may rely on undesirable different linguistic features to solve a task (Gururanga gan al. 2018 Tamkin et al. 2020). Much work has attempted to improve the ability of such mod- els to perform in-context learning by calibrating model predictions (Zhao et al., 202T). choosing good examples for the prompt (Liu et al. 2022a). finetuning models on natural language descrip- tions of tasks (Zhong et al. 2021 Wei et al.) [022] Sanh et al., (022), or by training models with
reinforcement learning from human feedback (Bai et al., 2022; Ouyang et al. 2022).
Prior work has suggested that language models may not effectively learn from the provided instruc- tions (Webson & Pavlick] (022) or few-shot examples (Min et al., 2022b] Kim et al. 2022); instead such models may rely cues such as the formatting of the examples the label space. In this work, we present a way to measure how well models use instructions or few-shot examples that is unaffected by such cues, because each AmbiBench example is consistent with multiple possible tasks. Thus, models that perform well on AmbiBench must infer the desired task using e.g.. the task instruction or other examples. This enables clean empirical investigation of how well large language models serve as Bayesian reasoners, as past work has hypothesized (Xie et al. 2021).
Past work has also explored finetuning on in-context learning examples (Chen et al. 2022 Min eet al., 2022a).. We extend this line of work to show how the content of these training examples can dramatically affect generalization: Finetuning on umbiguously-specifee examples (but not control set of unambiguous tasks) can enable models to disambiguate better in new settings- improving the performance of pure language models without the need for human feedback data.
2.3 TASK AMBIGUITY
Systems capable of performing different tasks may experience task ambiguity, where the provided examples do not uniquely identify the user's intended task (Finn et al., 2018 Tamkin et al. (021a)) One form of task ambiguity is shortcut learning (Geirhos et al.] 2020). where the training examples can all be solved by identifying a simple feature (e.g. a watermark) as opposed to learning the in- tended task (e.g. object classification). Task ambiguity is particularly important in few-shot learning settings, where the small number of examples may leave the intended task ambiguous (Finn et al., 2018 Tamkin et al., (021a). In this work, we study task ambiguity for in-context learning of simple linguistic tasks, considering not only the role of examples but also natural language instructions.
3 THE AMBIBENCH BENCHMARK
As first step towards studying task ambiguity in language models, we construct the AmbiBench benchmark, a collection of six different sentence classification tasks. The goal of AmbiBench is to construct testbed of minimal complexity where we can control and measure the degree of ambiguity in various task specifications. Despite the simplicity of this benchmark, we find large variability in performance across different language models.
3.1 SELECTION OF TASKS
AmbiBench contains six binary classification tasks, where a human model must detect simple linguistic feature in an input sentence-for example, whether an outdoor location or an animal was mentioned- and then ouipul the appropriate classification letter (X or Y). Crucially, however, each sentence has two linguistic features (c.g. The duck in the canyon has the features animal and outdoor location). The six features grouped into three pairs, shown in Table I where a single sentence will have one feature in each pair.
To identify the salient feature for the task, then, one must have either an informative instruction (e.g.. Output 'X' if the sentence contains an outdoor location and 'Y' otherwise) or multiple labeled examples to disambiguate which feature determines the label.
Tasks were chosen to represent a set of common semantic categories, excluding subtoken infor- mation such as periods and capitalization that might be much easier for humans to represent than
3

---

Salient feature
Example sentence
human subject
The researcher/bear is in the museum.
Instruction
Example
indoor location
The researcher is in the museum/meadoww
Uninformative
Output 'X' if the sentence contains a [category with- heldJ and 'Y' oth- erwise.
religious leader
He is in the museum with the rabbi/judge.
pronoun gender
He/She is in the museum with the judge.
Informative
Output 'X' if the sentence contains proper noun and 'Y' otherwise.
proper noun
Paul Atreides/The director
may not be in the film studio.
negation
Paul Atreides may/may not be the film studio.
The AmbiBench benchmark. Left: Each task involves detecting salient feature in sentence (bolded in the examples on the right). The same sentence could potentially receive label according to two features, requiring learner use additional information (task instructions or other examples) to disambiguate the intended behavior. Right: Varying levels of instruction are inserted before the examples, providing different degrees of information about the format and salient feature of the task. See Figure for an example of complete prompt.
Table I:
models. See Figure ! for an example of this disambiguation process, and Table [ for a full list of tasks and accompanying instructions.
3.2 TASK CONSTRUCTION
AmbiBench examples are programmatically constructed from a set of templates, allowing precise control over the amount task ambiguity each in-context example (see Table[aam and Appendix G for more details). Templated data has seen recent resurgence in NLP for the purposes of evalu- ating large language models (Lakee Baroni 2018: Srivastava al. 2022), as they enable precise control and coverage over different variables of study. Furthermore, recent work has shown strong correlation between test performance on synthetic and naturalistic data Liu et al. (202I), suggesting that insights gained from such datasets may extend to broader range of natural contexts. In case, this dataset construction process enables us to formalize and characterize the degree of task ambiguity in different examples, allowing to measure how well models can disambiguate between multiple potential classification tasks they may be asked to perform.
3.3 IN-CONTEXT LEARNING FORMATS
There are several ways an instruction and in-context examples can be assembled into a prompt for language model. Given the demonstrated sensitivity of models to such parameters (Zhao et al. [021]] Liu et al., 2022b Lu et al. 2022), we consider two different prompt formats, and report averaged performance across them:
Arrow:
Q/A:
Output 'X' if the sentence contains an outdoor location and "Yr otherwise.
Output 'X' if the sentence contains an outdoor location and "Y" otherwise.
The worm is in the meadow >X
Q: The worm is in the meadow A: X
The duck is in the canyon
Q: The duck is in the canyon
A:
4

---

4 EXPERIMENTS
We use AmbiBench to investigate how humans and language models respond to and resolve different manifestations of task ambiguity.
4.1
EXPERIMENTAL SETUP
First, we describe the different language models and human participants we study, and how we evaluate them.
Language models We examine range of different models, including both OpenAl's normal language models and their "instruct" models trained with human feedback data (HFD) (Brown
et al. 2020 Ouyang et al. 2022). These models trained using the data described in Ouyang et al. (022)) as well as on highly-rated model generations In the rest of the paper, OpenAI's cextt-curie-000 The instruct models have numerical suffix (e.g. 002) and the model size increases as one pro- gresses through the alphabet (ada, babbage, curie, davinci). See Appendix [E] for more information. We also evaluate A121 Studio's 's 178B 1-8-pparmetter Jurassic-1 Jumbo language model Sjurrasi-jjumtt (Lieber et al.), as well as the 1BB-paraeeter TO++ (t (pp) (Sanh et al. 2022) and Flan-T5 (flan models (Chung et al. 2022), which were finetuned on a large corpus of task instructions. This diversity of model providers, model sizes, and training strategies enables us to identify which ingredients are most crucial for resolving task ambiguity.
model names are reported as listed in their documentation (e.g. davinci,
Human evaluation We compare model performance with the performance of human participants, evaluated by hiring contractors from Prolific (Palan & Schitter 2017). We aimed evaluate the human participants as similarly to language models as possible within the confines of an online survey methodology. We showed human participants exactly the same input that language models received, with minimal additional information presented to them before the study began. Participants typed the answer label (i.e. X or Y) into a textbox, as opposed to choosing from set of preselected options, to mitigate priming effects and mirror the setting for language models. We also recruited a new participant for every single in-context instance, to avoid humans learning across examples in ways that language models do not. Human participants were paid $12-13/hr, in line with Prolific wage recommendations See Appendix for more details.
4.2 TASK DISAMBIGUATION USING NATURAL LANGUAGE INSTRUCTIONS
One way that people resolve task ambiguity is through the use of natural language instructions, which can explicitly indicate different aspects of the task. Past work has suggested that the best models do not fruitfully use natural-lanuuage instructions, as evidenced by experiments leveraging irrelevant or misleading directions (Webson & Pavlick, 2022). However, these experiments were performed for established natural language processing tasks that lack the explicit task ambiguity we study here, and did not investigate more recent models trained with human feedback data (Bai al. 2022 Ouyang et al. 2022).
As first set of experiments, we evaluate how humans and models are able to use differing levels of instruction to resolve task ambiguity. The humans and models receive two in-context examples, one from each class. Humans and models are then presented with third query example in order to elicit the predicted output letter. Because there is only one example of each class, but two possible features, the salient feature can not be identified from these two examples alone, requiring the model to use the instruction to disambiguate the task. The order of the examples, the example format, as well the assignment of each class to an output letter (X or Y) are randomized. Each model is evaluated with 720 different in-context prompts for each level of instruction.
We consider two different levels of instruction:
https://heta.ooomaa.coo/// htt.:/pppaaacccccccc/oooo https:/wwww.pmllifcccoooooooog
5

---

1. Informative instruction: The model receives full specification of the salient feature and output format. Ex: Output 'X' if the sentence contains an animal and 'Y' otherwise.
2. Uninformative instruction: The model receives the output format but the salient feature is redacted. Ex: Output 'X' if the sentence contains [category withheld] and 'Y' otherwise.
Our setting is simple enough that crafting an informative instruction is not challenging, making tractable for to study. However, the insights from this simple case may generalize to more com- plex settings where users may be prone to accidentally omit crucial information from the prompt.
4.2.1 RESULTS
In the case of uninformative instructions, humans as well as many models are able achieve approximately 50% accuracy by correctly understanding the output format and choosing X and Y at random. However, some non-instruct models, including curie, often output values other than X or Y (e.g. 2), leading to lower performance. Finally, in the case of negation, humans achieve 100% accuracy despite lacking an instruction identifying the salient feature. This may be due to an inductive bias present in people (but not models) that makes negation an especially salient feature.
ada, babbage, and
In the case of informative instructions, humans perform the strongest at this task, with perfect per- formance in all but one task, showing that they are broadly able to identify the salient feature in the text inputs and output the correct letter. Humans closely followed by the text - dett-ddvincii -003 and text-da i ne - 002 HFD models (see Figure 2 All other models perform relatively poorly, including the non-HFD 175B+ parameter davinci and j1-jumbo models, well as the smaller HFD models curie, babbage, and ada (although we verify in Section that these models are still able to generate outputs in the correct format). This seems to suggest that most models are not reli- ably able to follow simple instructions to disambiguate a task, but that a combination of large-scale training and HFD can approach human performance in some settings.
4.3 TASK DISAMBIGUATION USING MULTIPLE EXAMPLES
While instructions are simple way to specify a task, multiple examples can also disambiguate between different tasks a user might intend. For example, if there are multiple features that could explain the label of single example, more examples will gradually identify the salient feature provided the features are sufficiently decorrelated.
We investigate whether models and humans can identify the salient features in AmbiBench as the number of examples grows from zero (where the task is completely ambiguous) to twenty (where the task is almost certainly unambiguous). Both models and human participants predict the answer for each example, then are presented with the correct answer for that example and the next query All aspects of the examples constant across the entire in-context example), the assignment of x or Y to the salient feature, the ex- ample order, and the specific instantiations of the salient and non-salient features for each example. Each model is evaluated with 720 different in-context prompts, each containing 20 examples.
randomized, including the salient feature (chosen randomly, then held
We also compare humans and models with Bayesian oracle that represents how well an optimal leamerr could perform on the benchmark. This oracle performs perfectly as soon as it sees set of examples which disambiguate the intended task, and performs chance otherwise.
4.3.1 RESULTS
To our surprise, the best language model (the outperformed the human participants (F.. The human participants performed comparably to the j1-jumbo and curie models, which in turn performed better than the rest of OpenAl's mod- els. The 5 and tppp models performed worse, with the latter exhibiting large sensitivity the prompt format-t0pp outputted invalid answers (typically nouns) for the arrow format. However, considering only the Q/A format, tOpp still only performed near chance. All models
text text-daavini--02 significantly
'For causal language models, such OpenAl's models and J1 J1-Jumbo, which only attend backwards, this can be done efficiently by presenting the full 20 examples to the model and looking at the probability assigned the correct answer for each example.

---

1.0
babbage
0.8
batbage
taffii
text
dovinci
lecl-davino--002
text-davinc-000
humans
Salient Task
(a) Uninformative Instructions
ads
lext-ada-001
babbage
lext-ballbagee-00
Accuraty
lexl-curie-001
0.8
davinci
lext devinal 002
ledl-davingi-005
juaassiojumboo
humams
location
subjecl
pronoun
refiiouus
propr
negation
Salient Task
(b) Informative Instructions
Figure 2: The best HED model (text-davinci-003) approaches human accuracy for both uninformative and informative instructions. Accuracy of humans and other models for tasks prompted with an instruction and two in-context examples. Error bars show 95% bootstrap Cls.
1.0
0.8
Accuracy
davinci
text eext-davinci-00
jurassic jurassic-jumbo jumbo
0.2
10+
humans
bayesian oracle
0.0
Example Number
Figure 3: The best HFD models (text-davinci--022 and text-davinci--03) outper- form human participants at disambiguating the intended task. Accuracy as the number of examples in in-context window grows. Surprisingly, the smaller curie model reliably out- performs the larger davinci model across the examples. In addition, the HFD training hurts curie scale, but dramatically helps at davinci scale. Shaded regions are 95% bootstrap Cls.

---

0.8
0.8
Aocuioo
Aocurecy
0.4
0.4
kidl-devind 0022
lixt texl-davind-oo OC2
devinci
davinci
devvnni Inwtuned (control)
devnn fretuned (control)
fnnctund (amoiguous)
finctuned (amaiguous)
bayinian oracle
baysoian onacl
0.0
0.0
10
15
50
70
Example Number
Example Number
(a) Subject and location
(b) Religious and pronoun
1.0
1.0
0.8
0.6
Actrayy
0.6
0.4
text-devin-
0.2
devinci
devinci
finetuned (control)
davinci Innttnee (ambiguous)
davinci finctunod (amoiguous)
bayasien
bayasian
0.0
0.0
10
15
20
50
15
20
(c) Proper noun and negation
(d) Average across (a-c)
Figure 4: Finetuning ambiguous in-context examples dramatically improves accuracy unseen tasks that are ambiguously specified. Accuracy after finetuning davinci on ambiguous and non-ambiguous (control) in in-context examples. Models are finetuned on 272 examples from four tasks, then evaluated on the two held-out tasks (subfigure captions). Shaded regions are 95% bootstrap Cls.
considerably underperform the Bayesian oracle, suggesting room for additional improvement. See Appendix A for initial experiments investigating whether models can describe the few-shot task in words.
We do not observe evidence that the imperfect human performance is due to low-quality participants or bot activity- hhivitt-hhuman human annotators mostly spent between and 8 minutes on the task, did not appear to be guessing random, and typically left thoughtful comments feedback the survey. That said, we caution against claims of "sperhuumn performance" given that annotators represent merely sample from single distribution of humans, and they may have experienced fatigue distraction across the 20-example episode.
4.4 FINETUNING A MODEL TO GENERALIZE WELL IN THE FACE OF AMBIGUITY
The strong performance of the HFD models relative to the normal language models in Section 4.3 is somewhat surprising- -thee models are described as being trained to follow human instructions, not to resolve ambiguity in instructions by analyzing the training examples. While the training dataset of these models was not released, Ouyang et al. (2022) do report that some of the crowdsourced examples for the model contain instructions along with few-shot examples. If some of these instruc- tions were ambiguous, the model may have learned from those examples to resolve that ambiguity more effectively.
8

---

Motivated by this hypothesis, investigate whether finetuning small corpus of ambigu- ous in-context learning examples is sufficient to close the gap between the performnnnn Lext-davinci--0 HFD model and the normal davinci language model. To do so, partition the six AmbiBench tasks into three folds, each containing four finetuning tasks and two evaluation tasks (following the feature pairs in T.. We finetune 68 examples from each task (two for each number of examples, from 4 to 20), and evaluate on 240 examples randomly drawn from the other two tasks. While all tasks share some structural similarities, this partitioning en- sures that the model is being tested on held-out features that never appeared in its finetuning dataset. Models are finetuned using the OpenAl API (see Appendix H for details).
To see whether ambiguous data is the key factor when finetuning, we also finetune on unambiguous versions of this data, where only one feature varies within each in-context example. For example, the two features are animal and indoor location, given in-context example may contain examples with both animals and humans, but only indoor locations. See Appendix Horor more details.
4.4.1 RESULTS
Despite the small training dataset consisting of only tasks (with 272 examples total), we find we are able to completely close the gap between the HFD models and our finetuned models across all three splits our data. Indeed, our finetuned models appear to even outperform text text-davinci-Oc. across the first eight examples, closing part of the gap to the Bayesian oracle.
Crucially, we do not observe any improvement for the control finetuned models, which were fine- tuned on the same kinds of examples but without task ambiguity between two potential salient fea- tures. This indicates that ambiguity is the crucial ingredient explaining the success of our finetuned models, and supports the hypothesis that the few-shot examples in text Savinci-0022 human feedback data may contribute to its strong performance.
More broadly, these results suggests that explicitly finetuning models adapt task ambiguity may result in a generalized capacity to do so across different kinds of ambiguous task specifications.
DISCUSSION AND CONCLUSION
We present the AmbiBench testbed for studying task ambiguity in language models and humans, showing how it can be used to investigate different factors influencing task ambiguity, well as identify promising interventions that can improve how models resolve it.
5.1 LIMITATIONS
Our study has several limitations. First, we conduct a scientific and controlled study of task am- biguity in language models; this naturally elides many of the messy nuances of task ambiguity in the real world, and should be seen as complementary such real-world use case in Appendix B however more work is needed. Second, despite our efforts to match the experimental conditions between humans and language models, humans do require some additional instructions orient them to the task interface, and may suffer from fatigue and uneven concentration across the length of a 20-example learning episode. Finally, our work studies task ambiguity between two possible however, in general task ambiguity may occur between arbitrarily many tasks, or even an infinitely family of tasks.
in-the-wild case studies. We explore one
5.2
FUTURE WORK
Task ambiguity pressing problem in machine learning with relevance for safety, fairness, and interpretability. Going forward, we are excited by the potential to study task ambiguity in self- supervised models trained on many different modalities (Reed et al. [022]] Tamkin et al., (022b) 2022a Alayrac et al., (202).. including multimodal settings, as self-supervised learning is applied increasingly broadly. The strong performance of models on the AmbiBench testbed also suggests the tractability of studying task ambiguity in more complex real- world settings where language models are used, such as software engineering, law, and education, as well as assessing the efficacy of our proposed finetuning interventions.

---

Acknowledgments and Funding Disclosure We'd like to thank Ben Prystawski, Shyamal Buch, and Rose Wang for useful conversations and feedback, as well as all the participants who took our study. We thank the Center for Research Foundation Models (CRFM) and Together Computer for facilitating access to models. AT' is supported by an Open Phil AT Fellowship.
Ethics statement Our research makes use of human subject experiments via the Prolific platform (Palan & Schitter, 2017). We pay workers a minimum of $12-13 / hour, consistent with Prolific wage recommendations We also made efforts to solicit feedback from participants via pilot stud- ies, which led to several changes to the research methodology to make the survey experience more pleasant (e.g. keyboard shortcuts to navigate the study more efficiently). Anecdotally, many partici- pants expressed that they enjoyed the study:
1. Zap! I I'm suddenly back in high school with Ms. Langston's 's English class. Thank you for the smiles!
2. this was fun!
3. this was a really good study
Reproducibility statement We include detailed experimental settings in Sections
4.2
4.3
| G We also release our codebase, including the benchmark data, aackika_mbbigggg guity
https://gt hhtpp////gtttbbb..cmm/mmmaaaaaa wwtp://www//caaaaaaaaaaaaa
REFERENCES
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katic Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford. Serkan Cabi, Tengda Han. Zhitao Gong, Sina Samangooci. Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan. Flamingo: visual language model for few-shot learning. ArXiv, abs/2204. 14198, 2022.
Mohammad Aliannejadi, Hamed Zamani, Fabio A. Crestani, and W. Bruce Croft. Asking clarifying questions in open-domain information-seekingg conversations. Proceedings of the 42nd Interna- tional ACM SIGIR Conference on Research and Development in Information Retrieval, 2019.
Mohammad Aliannejadi, Julia Kiseleva, Aleksandr Chuklin, Jeffrey Dalton, and Mikhail S. Burt- sev. Building and evaluating open-domain dialogue corpora with clarifying questions. ArXiv, abs/2109.05794 2021.
Yushi Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, T. J. Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodde
Danny Hernandez, Tris- tan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Christopher Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv, abs/2204.00862. 2022.
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demszky. Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman. Neel Guha, Tatsunori Hashimoto, Peter Hen- derson, John Hewitt, Daniel E. Ho, Jenny Hong. Kyle Hsu, Jing Huang. Thomas F. Ieard, Saahil Jain, Dan Jurafsky. Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, O. Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovee, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir P. Mirchandani, Eric Mitchell, Zanele
https:/wwww.pmliiicccoooccnngg
10

---

Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Benjamin Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, J. Nyarko, Giray Ogut, Laurel OIr, Isabel Papadim- itriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher R'e, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishna Parasuram Srini- vasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei A. Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Perey Liang. On the opportunities and risks of foundation models. ArXiv, abs/2108.07258, 2021.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Rad- ford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. ArXiv, 0bs/2005.141665 2020.
Razvan C. Bunescu and Marius Pasca. Using encyclopedic knowledge for named entity disambigua- tion. In EACI, 2006.
Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. Meta-leaming via language model in-context tuning. ArXiv, abs/2110.07814 2022.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, al. Scaling instruction-finetune, language mod- els. arXiv preprint arXiv:2210.1 1416, 2022.
Steve Cronen-Townsen and W. Bruce Croft. Quantifying query ambiguity. 2002.
Silviu Cucerzan. Large-scale named entity disambiguation based on wikipedia data. In EMNLP, 2007.
Mark Dredze, Paul McNamee, Delip Rao, Adam Gerber, and Timothy W. Finin. Entity disambigua- tion for knowledge base population. In COLING, 2010.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning In NeurlPS, 2018.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard S. Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning in deep neural networks. Nat. Mach. Intell., 2:665-673, 2020.
M. Guo, Mingda Zhang, Siva Reddy, and Malihe Alikhani. Abg-coqa: Clarifying ambiguity in conversational question answering. In AKBC, 2021.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, and Noah Smith. Annotation artifacts in natural language inference data. North American Chapter of the Association for Computational Linguistics, 2018.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hen- nigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and L. Sifre. Train- ing compute-optimal large language models. ArXiv, abs/2203. 15556, 2022.
Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Wor Lee, Sang goo Lee, Kang Min Yoo, and Taeuk Kim. Ground-truth labels matter: A deeper look into input-label demonstrations. ArXiv, abs/2205. 12685, 2022.
Brenden M. Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of requence-to-sequene recurrent networks. In ICMI., 2018.
11

---

Lieber, O Sharir, B Lentz, and Y Shoham. Jurassic-1: Technical details and evaluation, white paper, ai21 labs, 2021.
URL: https://ppoadss-st webflow. jurassic tech paper. pdf.
Jiachang Liu, Dinghan Shen, Yizhe Zhang. Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? In DEELIO, 2022a.
Nelson F. Liu, Tony Lee, Robin Jia, and Perey Liang. Can small and synthetic benchmarks drive modeling innovation? a retrospective study of question answering modeling approaches. ArXiv, abs/2102.01065, 2021.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing. ACM Computing Surveys (CSUR), 2022b.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where find them: Overcoming few-shot prompt order sensitivity. In ACL, 2022.
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: Answering ambiguous open-domain questions. In EMNLP, 2020.
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannanch Hajishirzi. Metaicl: Learning to learn in context. ArXiv,
2022a.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? ArXiv, abs/2202. 12837, 2022b.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. ArXiv, 0ans/2203.221555 2022.
Stefan Palan and Christian Schitter. Prolific a subject pool for online experiments. Journal of Behavioral and Experimental Finance, 17:22-27, 2017.
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al. A generalist agent. arXiv preprint arXiv:2205.06/73. 2022.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang A. Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Can- wen Xu, Urmish Thakker, Shanya Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj. Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Stella Rose Biderman, Leo Gao, Tali Bers, Thomas Wolf, and Alexander M. Rush. Mul- titask prompted training enables zero-shot task generalization. ArXiv, abs/21 10.08207, 2022.
Aarohi Srivastava, Abhinav Rastogi. Abhishek B Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowyez, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexan- der W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang. Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ameet Annasaheb Rahane, Anantharaman S. Iyer, Anders Jo- han Andreassen, Andrea Santilli, Andreas Stuhlmuller, Andrew M. Dai, Andrew D. La, An- drew Kyle Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakacs, Bridget R. Roberts, Bao Sheng Loe, Barret Zoph, Bartlomiej Bo- janowski, Batuhan Ozyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Stephen Howald, Cameron Diao, Cameron Dour,
12

---

Catherine Stinson, Cedrick Argueta, C'esar Ferri Kumiirez Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Kallison-Burchh Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Tatiana Ramirez, Clara Rivera, Clemen- cia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Daniel H Gar- rette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Gonz' alez. Danny Hernandez. Danqi Chen, Daphne Ippolito. Dar Gilboa, David Do- han, D. Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli. Denis Emelin, Denis Kleyko, Deniz Yuret. Derek Chen, Derek Tam. Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dim- itri Coelho Mollo, Diyi Yang, Dong- Ho Lee, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth P. Donoway, Ellie Pavlick, Emanuele Rodolà, Emma FC Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang. Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Funice Engefu Manyasi, Evgenii Zheltonozhskii, Fan Xia, Fatemeh Siar, Fernando Mart inez-Plumed, Francesca Happ's. e. François Chollet, Frieda Rong. Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang. Gonzalo Jaimovitch-L'opez, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Han Sol Kim, Hannah Rashkin, Hanna Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hubert Wong, Ian Aik- Soon Ng. Isaac Noble, Jaap Jumelet, Jack Geissinger, John Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, J. Brooker Simon, James Koppel, James Zheng. James Zou, Jan Koco n, Jana Thompson, Jared Kaplan, Jarema Radom, Jascha Narain Sohl-Dickstein, Jason Phang, Ja- son Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jenni Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Oluwadara Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Jane W Waweru, John Burden, John Miller, John U. Balis, Jonathan Berant, Jorg Frohberg, Jos Rozen, José Hernández-Orallo, Joseph Boudeman, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ig- natyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Ochieng' Omondi, Kory Wal- lace Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao. Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras- Ochando, Louis-Philippe Morency, Luca Moschella, Luca Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Col'on, Luke Metz, Lutfi Kerem cSenel, Maarten Bosma, Maarten Sap. Maartje ter Hoeve, Madotto Andrea, Maheen Saleem Farooqi. Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, M Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew Leavitt, Matthias Hagen, M'aty'as as Schu- bert, Medina Baitemirova, Melissa Arnaud, Melvin Andrew McElrath, Michael A. Yee, Michael Cohen, Mi Gu, Michael I. Ivanitskiy, Michael Starritt, Michael Strube, Michal Swkedrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Monica Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, MukundVarma, Nanyun Peng, Nathan Chi. Naycon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas S. Roberts, Nicholas Doiron, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pas- cale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Pe- ter W. Chang, Peter Eckersley, Phu Mon Htut, Pi-Bei Hwang, P. Milkowski, Piyush S. Patil, Pouya Pezeshkpour. Priti Oli, Qiaozhu Mei. QING LYU, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker. Ram' on Risco Delgado, Raphaël Millière. Rhythm Garg. Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan Le Bras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib J. Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Sam Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebas- tian Schuster, Sepideh Sadeghi, Shadi S. Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo hwan Lee, Spencer Bradley Torene, Sriharsha Hatwar, Stanis- las Dehaene, Stefan Divic, Stefano Ermon, Stella Rose Biderman, Stephanic C. Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swa- roop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq A. Ali, Tatsuo Hashimoto, Te-Lin Wu, Theo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo
13

---

Schick, T. N. Kornev, Timothy Telleen-Lawton Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj. Tushar Khot, Tyler O. Shultz, Uri Shaham, Vedant Misra, Vera Dem- berg, Victoria Nyamai, Vikas Raunak, Vinay Venkatesh Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang. W Vossen, Xi- ang Ren, Xiaoyu F Tong, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yang Song. Yasaman Bahri, Ye Ji Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yu Hou, Yushi Bai, Zachary Seid, Zhao Xinran, Zhuoye Zhao, Zi Fu Wang, Zijie J. Wang, Zirui Wang, Ziyi Wu, Sahib Singh, and Uri Shaham. Beyond the imitation game: Quantifying and extrapolating the capabilittiss of language models. ArXiv, Abs/2206.04615, 2022.
Gabriel Stanovsky, Noah Smith, and Luke Zettlemoyer. Evaluating gender bias in machine translation. ArXiv, abs/1906.00591, 2019.
Haitian Sun, William W. Cohen, and Ruslan Salakhutdinov. Conditionalqa: A complex reading comprehension dataset with conditional answers. In ACL, 2022.
Alex Tamkin, Dan Jurafsky, and Noah Goodman. Language through a prism: spectral approach for multiscale language representations. Advances in Neural Information Processing Systems, 33: 5492-5504, 2020.
Alex Tamkin, Miles Brundage, Jack Clark, and Deep Ganguli. Understanding the capabilities, limitations, and societal impact of large language models. arXiv preprint arXiv:2102.02503, 2021a.
Alex Tamkin, Vincent Liu, Rongfei Lu, Daniel Fein, Colin Schultz, and Noah Goodman. Dabs: A domain-agnostic benchmark for self-supervised learning. arXiv preprint arXiv:2111.12062, 2021b.
Alex Tamkin, Gaurab Banerjee, Mohamed Owda, Vincent Liu, Shashank Rammoorthy, and Noah Goodman. Dabs 2.0: Improved datasets and algorithms for universal self-supervision. 2022a.
Alex Tamkin, Margalit Glasgow, Xiluo He, and Noah Goodman. Feature dropout: Revisiting the role of augmentations in contrastive learning, 2022b. URL /ttpittt///ax//voor///a/as 08378
Alex Tamkin, Dat Nguyen, Salil Deshpande, Jesse Mu, and Noah D. Goodman. Active learning helps pretrained models learn the intended task. ArXiv, abs/2204.0849) 2022c.
Yu Wang and Eugene Agichtein. Query ambiguity revisited: Clickthrough measures for distinguish- ing informational and ambiguous queries. In NAACL, 2010.
Albert Webson and Ellie Pavlick. Do prompt-based models really understand the meaning of their prompts? ArXiv, abs/2109.01244 2022.
Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. Mind the gap: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605-617, 2018
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetuned language models are zero-shot learners. ArXiv, abs/2109.01652, 2022.
Zeqiu Wu, Ryu Parish, Hao Cheng, Sewon Min, Prithviraj Ammanabrolu, Mari Ostendorf, and Han- naneh Hajishirzi. Inscit: Information-seeking conversations with mixed-initiative interactions. arXiv preprint arXiv:2207.00746 2022.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning implicit bayesian inference. arXiv preprint arXiv:2111.02080 2021.
Michael J.Q. Zhang and Eunsol Choi. Situatedqa: Incorporating extra-linguistic contexts into qa. ArXiv, abs/2109.06157 2021.
Tony Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. ArXiv, 0b9601 2021.
14

---

Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. Adapting language models for zero-shot
learning by meta-tuning on dataset and prompt collections. In EMNLP, 2021.
15

---

Comeeiness
Conrechhess
Correct Task
Incorredt Task
Incomect [other]
(other)
Dount
negation
propn
subject
location
Swient Task
Task
(a) Accuracy of text-davinct 002 for guessing the (category withheld] for each salient task. Q/A for- mat only (N=10).
(b) Accuracy of feext-ddvinc--000 for guessing the [category withheld] for each salient task. Arrow and Q/A formats (N=20).
Figure 5: Even models that perform very well on task struggle to describe that task in words.
A CAN MODELS VERBALIZE THE TASK THEY ARE PERFORMING?
As models attempt to disambiguate between different tasks, user may wish to know the model's best estimate of the task it is being asked to perform. In this section, we assess whether models can verbalize the task descriptions in Section 4.3 at the end the in-context prompt.
Specifically, after the 20 example prompt we append newline with the strings
"What is the [category withheld]?" for fext-davinci- 002, and
"What your best guess for the [category witheld] above?" for text teet-eexiiiiiee dati-diiviini.0....
We developed the second prompt as the newer text-davinci-000 model would often return responses along the lines of "This question cannot be answered without further context" for the first prompt.
We generate 10 outputs for each of the tasks, and manually categorize them one of three cate- gories:
Correct task: A correct verbalization of the task (e.g. "The [category withheld] is reli- gious leader."
Incorrect Task The model guesses a task, but it is not the correct task. (e.g. "animals" when the salient task is "outdoor location"))
Incorrect (Other): The verbalization does not mention a task (e.g. "The category is with- held.")
As shown in Figure 5 the models are sometimes able to verbalize the correct task, especially for religious figures and proper nouns, although for all other tasks it struggles. These initial experiments suggest that task verbalization challenging even for the most capable models, even when they perform very well the task, and suggests exciting direction for future study.
Note that our graph for text fext-davinci-On -0000 exclusively considers the Q/A format, as we did not observe in preliminary investigation that the model could verbalize the task successfully with the arrow format. Additionally, preliminary investigations we did not find that other models were able to successfully verbalize the task.
16

---

B EXPLORING TASK AMBIGUITY FOR A NATURAL LANGUAGE COMMAND-LINE ASSISTANT
In this section explore how the methodologies we introduce in AmbiBench can be extended to more real-world settings. Here, we consider the case of a natural language assistant for command- line tasks. In our hypothetical setting, an employee of a company prompts a language model to produce Amazon Web Services buckets for different users by providing it with two input-output examples of the desired behavior. By chance, the two training examples both use Japanese names and have the bucket location set to the ap-northeast. region in Japan. The test examples ask bucket to be made for a person with non-Japanese name (in our case, White American, Greek, or Indian names).
Concretely, examples look like the following:
Write the ANS CLI command to create an ARS Bucket.
Input: Create a bucket for Sato Tamotsu
Output: aws s3 mb s3:/ fts///ccket-for-attoo --region ap-northeast-1
Input: Create bucket for Yuki Hashimoto
Output: aws s3 mb s3:/ #3://bucket-for-yukk --region ap-northeast-1
Input: Create bucket for Margaret Richards Output:
The task ambiguity that arises that sometimes the model assumes the bucket should be placed in the same region as the training examples, but in other cases it assumes the bucket should be placed in a different region based on the person's 's name. We omit other commandline arguments for readability, however note that this ambiguity might be very difficult to notice in practice with long commands consisting of many arguments.
In Figure 6 we show that not only does this ambiguity manifest in the text-davinci- guage model outputs (which can output both regions), but also varies based on the national origin of the person's 's name. White American names induce the model output other regions (typically ones in the US and Europe) far more than Indian or Greek names do. However, after one additional name from that same national origin is added (indicating that the name should not determine the region) the model performs nearly perfectly at other names of this national origin.
lan-
This initial investigation illustrates how task ambiguity can manifest in more real-world setting, and how we can measure it using the techniques we discuss in the rest of the paper. Future work could explore finetuning on similar in-context examples to meta-train the model such that it avoids assuming that an individual's 's name should impact the task behavior, for example.
B. LIST OF NAMES
Here we provide the full list of names we used to query the model. Names were constructed from a list of popular first and last names
Japanese names: Sato Tamotsu, Yuki Hashimoto. Yamamoto Mitsuo, Kuroda Kumiko, Mita Naoki, Tanaka Sakura, Kuramoto Hideki, Sazama Miyuki, Hora Izanagi, Itoh Akane
White American names: Trevor Fitzpatrick, Jessica Price, Logan Evans, Leah Mills, Ava Smith,
Cole Jones, Isabella Brown, Emily Davis, Mia Miller, Madison Wilson
Indian names: Rajesh Kumar, Abhishek Yaday, Rahul Singh, Archana Kumar, Anisha Patel, Priya Sharma, Riya Gupta, Ayush Jain, Prisha Rao, Sagar Goyal
Greek Names: Yiannis Papadopoulou, Vassilis Karagiannis, Athanasios Toannou, Dimitra Pa- padakis, Vasiliki Georgiou, Eleni Oikonomou, Kostas Giannopoulos, Nikos Theodorou, Ioannis Vlachos, Katerina Makris
forebears.io
17

---

(a) White American Test Names
(b) Indian Test Names
Greek Test Names
Figure 6: Features such as name origin can influence how models behave under task ambiguity. Effect of task ambiguity text www-idivviiiiiaaii when generating an AWS bucket command given an ambiguous natural language string. Graph shows distribution of generated regions: ap- northeast-01 (the region in the prompts), another region, or invalid command.
oda
bobbago
curie
Toct addo
Neckoweooogee-u
keclschvinni-202
60
jumnnorc jumboo
Probabiity
I0
28
Example Number
Figure Models all sizes generate valid outputs by the end of the in-context learning episode. Probability of a valid output (either X or Y) for all models across examples
C Do SMALLER MODELS UNDERSTAND THE TASK FORMAT?
In Section 4.3 we find that smaller models perform worse at disambiguating the intended task from multiple examples. However, is this due simply to models not understanding the desired output format? We validate the output formats of all models and find (Figure 7 that by the end of training, even the smallest models always generate valid outputs (i.c. either X or Y) across the experiments considered, suggesting that their poor performance task disambiguation is not simply due to their poor prompting abilities.
D How IMPORTANT IS THE FORMAT OF THE UNINFORMATIVE INSTRUCTIONS?
As previously noted, our experiments uninformative examples in the main text use an instruc- tion format containing [CATEGORY WITHHELD]. Here we test whether the behavior of models sensitive variations in this phrasing. We replace [CATEGORY WITHHELD] ) with the lin- guistic nonce word wug, so a full instruction might read Output 'x if the sentence contains wug and "Y otherwise. As shown in Figure we find that performance is the same for both tasks, suggesting that the particular form of uninformative instruction is not very important.
18

---

1.0
0.6
Accurscy
0.4
0.2
davinci: (category withhei]
davinci. wug
text-davinci-O00 002: (category withheld]
wug
Example Number
The choice of format for the uninformative instructions has negligible effect. davinci and text-davinci-002 performance with different forms uninformative instruc- tions
Figure 8:
Model
Number of Parameters
OpenAl Babbage
1.3B
OpenAl Curie
6.7B
OpenAl Davinci
175B
A121 J1-Jumbo
178B
T0++
11B
Flan-T5-XXL
11B
Table 2: Number of parameters for each model. Note that the OpenAl Ada and all HFD model parameter counts have not been released. It is also possible that the normal and HFD models have different numbers parameters and were trained on different amounts and kinds of data.
E PARAMETER COUNTS FOR EACH MODEL
The number of parameters each model is shown Table 2 for Babbage Curie Davinci 10 J1-Jumbo !I| and T0++ 12 Note that the parameter counts of the Ada model as well as the HFD models have not been released. Furthermore, note that parameter count is not necessarily the most informative proxy for model's capabilities, given the importance of other factors such as training data quality and the total number of tokens seen by the model (Hoffmann et al. 2022).
F ADDITIONAL INFORMATION ON HUMANS (PROLIFIC) EXPERIMENTS
Here we provide more information about the experiments with human participants via Prolific (Pllan & Schitter 2017).
Prior to beginning the experiment, Prolific participants were shown a message confirming consent followed by set of boiler-plate instructions on the upcoming task which stated: 1) "Continue the pattern the following screens to the best of your ability, using the provided instructions and examples given" 2) "Each time you make prediction, the next screen will show the correct answer
https//www.ai2 hcw./olll///anooonnnngggallll http://wwww..loooooom/mmmm/////s sutp::/nnnaiitttuuuuuoouiusiiiii
19

---

Output X the sentincee contains (calegory witheek] and "Y" otherwise.
Output X the senience contains [calegory withhekd and 'Y' The student may be building
The student may be in aparımem building.
Goodal could not restaurant.
(a) First example shown example test
participants in 20-
(b) Second example shown participants in 20- example test
Figure 9: Example questions shown to participants in Section 4.3
for that example (not necessarily your previous answer)" and 3) "The pattern may not be obvious at first but may become more clear over time. Note: some information in the instructions may be [intentionally withheld] (denoted by brackets)" This prelude was deemed necessary after series of pilot runs in which participants quit out of the survey due to their confusions and/or frustrations with the lack of guidance. To further streamline the process, participants were also given set of useful keyboard shortcuts with which to progress through the survey (although participants were not allowed to edit previous answers).
participants were shown examples one-by-one with the correct answer appearing on the screen following their input. For the tests Section 4.2 tests, participants were shown both labeled examples and the one unlabeled query example at the same time.
For the tests in Section 4.3
each participant was only given one set 20 examples. For the tests in Section 4.2 each participant was only given single prompt. Prompts given to participants were randomized and roughly evenly distributed across all tests. Participant IDs were filtered to ensure that the same participant did not participate multiple times across studies. All surveys were created using Qualtrics' survey builder.
For the tests in Section 4.3
F.1 NUMBER OF PARTICIPANTS FOR HUMAN EXPERIMENTS
Task disambiguation using natural language instruction For the experiment referenced in Sec- tion 4.2 we recruited 51 and 33 participants for the unambiguous instruction and informative in- struction tests respectively.
Task disambiguation using multiple examples For the experiment detailed in Section 4.3 we recruited 96 total participants: 16 per salient feature.
G AMBIBENCH BENCHMARK: FULL DETAILS
AmbiBench consists of ambiguous sentences with two explicitly varying (salient) features in each sentence. Salient features are grouped into pairs: 'subject' and 'location, 'religious' and 'pronoun'. and 'proper noun' and 'negation." Each salient feature has two values (e.g. animal subject human subject) with each choice having a 0.5 probability of being selected.
G.1 SALIENT FEATURE PAIRINGS
The pairings of the salient features are detailed below. The pairings remained constant throughout all experiments.
G.1.1 SUBJECT-LOCATIOO SENTENCES
Sentences follow the template: "The (Juman/animal subject) is in the (indoor/outdoor location)." The subject is randomly assigned to either human animal. If chosen be human, the
20

---

subject is randomly chosen from list of professions: [student, reporter, hiker, researcher, firefighter, fugitive, critic, photographer, director, surveyor]. If chosen to be an animal, the subject is randomly chosen from list of common animals: [boar, worm, hawk, hound, butterfly, snake, duck, bear, mountain lion, horse]. The location is randomly assigned to either an outdoor location or an indoor location. If chosen to be outdoor location, the location is chosen from list of common outdoor locations: [river, pond, woodlands, cave, canyon, prairie, jungle, marsh, lagoon, meadow] whereas chosen be an indoor location, the location is chosen from list of common indoor locations: [laboratory, theatre, museum, courtroom, apartment building, restaurant, house, film studio, hotel lobby, grocery store].
G.1.2 BELIGIOUS-PRONOUS SENTENCES
Sentences follow the template: "[She/He} is in the (indoor location) with the (religious/secular leader)." The pronoun is randomly assigned to either He She. The leader is randomly assigned to either a religious leader or secular leader. If chosen to be an religious leader, the leader is chosen from list of common religious leaders: [pope, reverend, bishop, Dalai Lama, rabbi, cardinal, pastor, deacon, imam, ayatollah) whereas if chosen to be a secular leader, the leader is chosen from a list of common secular leader: [president, CEO. principal, sheriff. judge, ambassador, officer, prime minister, colonel, professorj. The indoor location in the sentence is randomly chosen from the aforementioned list of indoor locations.
G.1.3 PROPER NOUN-NEGATION SENTENCES
Sentences follow the template: "(Proper/Common nown); (negation//ffimmttonn) in the (indoor lo- cation]." The noun is randomly assigned either proper noun common noun. If chosen to be a proper noun, the noun is randomly chosen from a list of famous individuals across a variety of disciplines: [Lebron James, Bernie Sanders, Christopher Nolan, Paul Atreides, Noam Chomsky, Serena Williams, Margot Robbie, Alexandria Ocasio-Cortez. Hermione Granger, Jane Goodal[]. If chosen be common noun, the noun is randomly chosen from the prior list of human subjects: [student, reporter, hiker, researcher, firefighter, fugitive, critic, photographer, director, surveyor] and appended to 'The' to maintain grammaticality of the sentence. The verb is randomly assigned to either negation or an affirmation. If chosen to be negation, the verb is chosen from a list of common negation verb phrases: (is not, was not, has not been, may not be, could not be]. chosen to be affirmation, the verb is chosen from the list of affirmations directly contrasting the list of negations: [is, was, has been, may be, could be].
G.2 INSTRUCTIONS
Instructions given were either informative instructions, explicitly stating the salient feature which determined the label for each sentence, uninformative instructions which did not divulge the salient feature. Informative Instructions were of the template: "Output 'X' if the sentence (salient feature instruction) } and 'Y' otherwise". where salient feature instruction could be filled in with one of {contains a reference to an indoor/outdoor location, contains a reference to human/an animal, contains a reference to a religious leader, does not contain reference to a religious leader, contains reference to religious leader, contains male pronoun, contains a female pronoun, contains proper noun, does not contain a proper noun, contains a negation, does not contain a negation)
Uninformative instructions were always given as "Output 'X" if the sentence contains [category withheld] and 'Y' otherwise."
G.3 TASK DISAMBIGUATION USING NATURAL LANGUAGE INSTRUCTIONS
When constructing tests for Section 4.2
we ensured task ambiguity by grouping one possibility for given salient feature with one possibility for its paired salient feature.
For example, for Subject-Location sentences, humans and indoor locations were grouped and ani- mals and outdoor locations were grouped.
In this test, humans and models were given one example with the first of these groupings and a second example with the second of the groupings. The final, 'disambiguating." example broke the
21

---

groupings, therefore displaying which of the features responsible for the example's label (the
salient feature).
Here is demonstration of a single prompt for an example with the Subject-Location sentence type:
Q: The hawk is in the canyon.
A: X
Q: The director is in the museum.
A: Y
Q: The hiker is in the meadow.
A: X
For the first two examples, it unclear whether the subject or the location is controlling the label but upon seeing the third example, becomes clear that the location and not the subject is controlling the label (as the sentence is labeled X whenever an outdoor location is referenced).
Tests contained either an uninformative instruction examples. Humans and models were shown all three examples at once. Humans were shown the first two examples with labels and the last example without the label. Models were shown all three examples with labels within the same API query (as the log probabilities for the final label can be acquired without requiring completion).
informative instruction followed by three
The order of groupings, order of labels, labels' correspondence with a salient feature, and the salient feature were randomized across tests with equal probabilities being given to each possibility.
Some example prompts from the test set:
Prompt 1. Salient feature 'subject' (Q/A format with informative instructions):
Output 'x' if the sentence contains a reference to a human and y otherwise.
The horse is in the woodlands.
A: Y
Q: The student is in the laboratory.
A: X
Q: The mountain lio is in the film studio.
A: Y
Prompt 2. Salient feature is 'pronoun' (arrow format with uninformative instructions): Output 'x' if the sentence contains a [category withheld] and Y otherwise.
He is in the film studio with the imam.
She is in the restaurant with the judge.
She is in the apartment building with the bishop. >X
Prompt 3. Salient feature is 'proper noun' (Q/A format with uninformative instructions): Output 'x' if the sentence contains a [category withheld] and Y otherwise.
Q: The student was not in the hotel lobby.
A: X
Q: Lebron James is in the theatre.
A: Y
Q: Christopher Nolan could not be in the house.
A: Y
G.4 TASK DISAMBIGUATION USING MULTIPLE EXAMPLES
When constructing tests for Section 4.3 humans and models were shown set of 20 questions with the same salient feature. All sentences within the set contained the same sentence pairing Subject- Location, Pnigiouu-Prrnoun Proper Noun-Negation.
22

---

For each example, the possibilities for each of the two features randomized and given equal probability (as described in the process of sentence construction). The groupings mentioned for the tests in Section 4.2 were not used.
The salient feature was randomly selected (with each possibility being given an equal probability).
Tests always contained uninformative instruction prior the set of examples.
Humans were shown questions one at a time, progressively building up from 1 to 20 questions. After each question, the correct answer was displayed on the screen (in place of their solution if incorrect). Models were shown all 20 questions within the same API query and the log probabilities for each of the twenty labels were tracked.
As with the tests Section 4.2 the labels' correspondence with a salient feature and the order of labels were randomized across all tests with equal probabilities being given each possibility.
Some example prompts from the test set:
Prompt 1. Salient feature is 'location' (arrow format):
Output 'x' if the sentence contains a [category withheld] and "y otherwise.
The photographer is in the restaurant.
The mountain lion is in the river.
The hiker is in the cave.
The butterfly is in the grocery store.
>Y
The surveyor is in the river.
>X
The boar is in the prairie.
> (continues until 20 examples]
Prompt 2. Salient feature is 'religious' (Q/A format):
Output 'x' if the sentence contains a [category withheld] and "y otherwise.
Q: He is in the film studio with the ayatollah.
A:
Q: She is in the house with the CEO.
A: y
Q: He is in the apartment building with the ambassador.
A: Y
Q: She is in the museum with the rabbi.
A: X
Q: She is in the museum with the Dalai Lama.
A: X
Q: He is in the laboratory with the rabbi.
A: X [continues until 20 examples]
Prompt 3. Salient feature is 'negation' (arrow format):
Output 'x' if the sentence contains a [category withheld] and "yr otherwise.
The student was not in the restaurant.
Christopher Nolan was in the apartment building.
>
The photographer has not been in the theatre.
>x
Alexandria Ocasio-Cortez could be in the film studio.
>y
Christopher Nolan was not in the museum.
23

---

>
The photographer was in the film studio. continnus until 20 examples]
>
G.5 FINETUNING MODEL TO GENERALIZE
FACE OF TASK AMBIGUITY
When constructing the finetuning dataset for Section 4.4 two out of the three possible sentence pairings were used. We then tested on the held-out sentence pairing. For example, if the finetuning dataset contained the salient features: 'subject,' location," 'religious," and 'pronoun," the held-out features would be 'proper noun' and "negation."
For the control experiments, the dataset consisted of examples for which only one feature varied. For example, the sentence was of the Subject-Location sentence type and the salient feature was location, given example may contain examples with both outdoor and indoor locations but only humans.
For the ambiguous experiments, the dataset consisted of examples with ambiguity: both features varied between the two possibilities for each feature.
The number of examples in each prompt varied: building from 4 examples to 20 examples for each salient feature. But the examples across these different prompts did not remain constant (rather just the salient feature did). unlike in Section 4.3 where prompt with length of examples simply added one example to the preceding prompt of length n-1.
As with the tests in Section 4.2 and Section 43 the labels' correspondence with salient feature and the order labels were randomized across all tests with equal probabilities being given to each possibility.
H ADDITIONAL INFORMATION ABOUT FINETUNING EXPERIMENTS
Hyperparameters We OpenAl's finetuning API finetune their davinci model When conducting the finetuning, we used a batch size of 1, a learning rate multiplier of 0.1, and prompt loss weight of 0.1.
Control experiment settings Our control experiment for the finetuning only varied one of the two features each set of examples (e-g. boar could change to worm but not firefighter). The choice of which feature would be constant (e.g. human/animal indoor/outdoor), as well as the value of that feature (e.g. human vs animal) were decided randomly for each set of 20 examples. By following this procedure, did not introduce task ambiguity into the finetuning data, but still ensured that the control model saw the same range of sentence constructions as the model trained ambiguous examples.
I ADDITIONAL GRAPHS
I.1 TASK DISAMBIGUATION USING NATURAL LANGUAGE INSTRUCTIONS
I.1.1 INFORMATIVE INSTRUCTIONS
Figure 10 details the performance across salient features for each model individually across both format types in instances of informative instructions. Figure [T averages the performance of the models across salient features, demonstrating the difference in performance by format type for each salient feature.
1.1.2 UNINFORMATIVE INSTRUCTIONS
Figure 12 and Figure 1 mirror Figure 10 and Figure II respectively but instead demonstrate the performance in prompts with uninformative instructions.
24

---

(a) ada
(h) text ada-001
(c) babbage
Galend Tack
Salend Tack
Saboed Tam
(d) baxt-babbagc-001
(c) curio
(f) toxt-curie-001
coman
Typk
Ave
(g) davinci
(h) text deett-avvini--00
(i) j1 ju-jumbo
Figure 10: Performance with informative instructions across salient fcatures on Task Disambigua-
tion Using Natural Language Instructions (see Section 4.2)
1.2 TASK DISAMBIGUATION USING MULTIPLE EXAMPLES
Figure 14, Figure [5] Figure 16 Figure 17 and Figure 18 individually detail the performance of each model across each salient feature from of 1-20 examples.
Figure 19h demonstrates the difference in performance across the two format types, averaged for all models. Because of t0pp's poor performance prompts with the arrow format, we do not include Cpp in the collective performance graph but instead display its performance across formats individually from the other models.
25

---

1.0
Format Type
Q/A Arrow
Accuracy
0.6
0.4
0.0
location
subject
pronoun
religious
propn
negation
Salient Task
Figure 11: Performance with informative instructions for each format averaged over all models Task Disambiguation Using Natural Language Instructions (see Section 4.2)
(a) ada
(b) 1exxt=ada-01
(c) babbage
(d) text--tat batt-tabbbeee-000
(e) curie
(l) texee--ooe Sext--curie-00-
Type
02
Sabend Can
(g) davinci
(h) text teetteexxeeee -evvidvinnn 0002
(i) j1 -1-ji jumbo
Figure 12: Performance with uninformative instructions across salient features on Task Disambigua- tion Using Natural Language Instructions (see Section 4.2)
26

---

1.0
Format Type
Q/A Arrow
Accuracy
0.6
0.4
0.0
subject
location
pronoun
religious
propn
negation
Salient Task
Figure 13: Performance with uninformative instructions for each format averaged over all models on Task Disambiguation Using Natural Language Instructions (see Section 4.2
27

---

06
imgation
pange
saller
0.6
8034877
0.2
Eourple
Numbeer
Number
(a) ada
0.4
Saller E, Jok paga
saller R, Jock SARBART
022
(b)
Figure 14: Performance across salient features on Task Disambiguation Using Multiple Examples
(see Section 4.3
28

---

axlled
1.0
imgation
sadier pinge
Sallen DARRAR!
8034877
Eoarple
Nummber
Number
(a) babbage
reqgation
Sadler Jokk paga1
Saller F. Jock SIANAAA
0.3
0.6
0.4
022
(b) text-ebbbaage-00
Figure 15: Performance across salient features on Task Disambiguation Using Multiple Examples
(see Section 4.3
29

---

sxleed
1.0
02
ingation
Sadier pinge1
witle Jok
0.4
Eoanple
Nummber
Number
(a) curie
Vadiee Jank p1g01
Saller R, Jock SARBAAY
0.
Acsenty
02
(b) text-curie-000
Figure 16: Performance across salient features on Task Disambiguation Using Multiple Examples
(see Section 4.3
30

---

axlled
1.0
0.2
Imgation
panae
Wille Joh pAREAA
0.4
Eoarple humber
Nummber
Number
(a) davinc
Sollet iegationn
Sadler Jakk pagar
salle 1, Jock SARBAR
006
(b) text-davinci-002
Figure 17: Performance across salient features on Task Disambiguation Using Multiple Examples
(see Section 4.3
31

---

014
02
CONOEE TOEC Imaption
COBNT Lud parpn
(a) j1 jumbo
Figure 18: Performance across salient features on Task Disambiguation Using Multiple Examples (see Section 4.3)
Format
Format
0.
Anow
0.
Anow
Accuracy
08
Acouracy
0.4
02
02
Example Number
Example Number
(a) Averaged over all models except t t0pp
(b) t.ppp
note: 'Arrow' format is at 0.0 throughout
Figure 19: Performance of each format on Task Disambiguation Using Multiple Examples (see
Section 4.3
32


Reviews:
[{"review_id": "jN1w1JZCJDq", "reviewer": "Reviewer_3BBz", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on the ability of large language models (LLMs) and humans to handle ambiguity in task prompts, of the type that are currently popular in the LLM literature.\n\nA small suite of six synthetic tasks are created. Each asks the model (or human) to identify whether a sentence contains a particular category (e.g. human subject, proper noun). Then both human and language models are tested in two settings: with the task fully defined in a prompt (e.g. 'output x if the sentence contains a proper noun, y otherwise'); and with the category of interest obfuscated in the prompt. Each task is designed such that there should be a 50% chance of correctly guessing the task type from the obfuscated prompt and a single example. This chance increases drastically as more examples are added.\n\nThe paper shows that humans are able to solve five out of six tasks perfectly, when the prompt is unambigious. The performances of LLMs are more varied, with an instruct tuned model, trained with reinforcement-learning from humans (RLFH), performing best but still signifcantly worse than human performance. When the prompt is ambiguous, human performance is closer to model performance. However, when an increasing number of examples are given, the best models are able to disambiguate the taksk much better, reaching >80% accuracy and outperforming humans, in the best case (which involves reading 20 examples). Similar performance is also attainable through fine tuning on 4/6 of the tasks and evaluating on the final 2. This suggests that tuning models to handle task ambiguity may result in generalization to new forms of test time ambiguity.", "summary_of_the_review": "This paper presents a well designed focused study of how LLMs handle one type of task ambiguity---identifying whether an unknown class of word exists in an input.\n\nA number of different LLMs are evaluated in ambiguous and unambiguous prompting settings, and compared to humans in the same scenarios. The results provide a nice insight into the effectiveness of examples in the context of ambiguous prompts, as well as the effectiveness of fine tuning.\n\nHowever, the current study focuses on a very narrow type of ambiguity, and it's not clear how much these insights will extend to other types of task ambiguity. This paper would be stronger if it could describe how this type of study could be extended to more diverse types of ambiguity, even if it did not directly address those types of ambiguity in the study presented.\n", "strength_and_weaknesses": "#### Strengths:\n- This is a cleanly designed investigation of a specific type of task ambiguity, which adds to our understanding of how large language models make use of prompts and examples.\n- The inclusion of human annotators, assigned the same tasks as the LLMs, really helps us understand the models' capabilities.\n- The finetuning results are interesting, and suggest new strategies for increasing the robsutness of LLMs to non-expert prompt design.\n\n#### Weaknesses\n- It is quite hard to draw many conclusions about the best performing systems because their training strategies are not public, and it is not clear what types of task ambiguity have been seen during training.\n- The six synthetic tasks are quite simplistic and limited in scope. It is not clear how these findings would extend to more complex task types.\n\n#### Minor issues:\n- The choices of line color and hatching in Figure 3 are not very discriminative, and I struggled to untangle which line represents which model. I suggest that more disctriminative line types are used.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "KLtWabXt5hq", "reviewer": "Reviewer_6dKC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies how modern LLMs perform when given ambiguous task examples.\n\nThe model must learn to infer the task from the examples, vs having a clear task with ambiguous inputs. The paper introduces a dataset, AmbiBench, designed to measure how well models use the instrutions/few-shot examples to infer underlying task.\n\nAmbiBench consists of 6 sentence classification tasks of minimal complexity which allow controlling and measuring the amount of ambiguity in the tasks. The tasks are all binary classification tasks. Determining the correct underlying task requires either (a) an auxiliary instruction, or (b) multiple examples to learn from.  The examples are generated from a set of templates.\n\nExperiments are carried out with 2 prompt templates across all tasks, and compare a variety of open and closed LLMs, along with human evaluations.  \n\nThey find:\n- for uninformative instructions, both models & humans get 50% accuracy, but humans also get 100% on negation while models do no\n- for informative instructions, humans get perfect performance across all tasks but 1, with RLHF models (text-davinci-002) close behind.\n- Non-RLHF models do not do nearly as well: davinci does much worse than davinci instruct.\n- They further find that ambiguously-specified examples provide a much bigger boost to generalization, when compared to finetuning on unambiguous examples. This provides evidence that finetuning works more robustly than few-shot prompting, and that explicitly training to handle ambiguity helps generalization performance.\n- Best model is not able to consistently verbalize the task it is trying to accomplish\n\n\nMain takeaways from paper: Scaled RLHF models perform best at disambiguating tasks. Finetuning helps over just few-shot prompting", "summary_of_the_review": "The paper presents a simple synthetic task which measures how many different LMs perform given ambiguous tasks. The synthetic benchmark allows testing this ability extensively and the community will benefit from it if evaluations are easy to run, although its simplicity may make it difficult to generalize the findings. ", "strength_and_weaknesses": "Strengths:\n- Simple benchmark with extensive comparisons across model scales and with humans.\n  - comparisons to humans grounded the results\n- easy to reproduce (even easier assuming the benchmark will be released with camera ready)\n- demonstrates a current weakness of LMs in a purely synthetic setting which allow variation.\n- preliminary investigation of more real-world tasks, specifically an LM taking actions at command line based on human instructions\n- extensive auxiliary experiments in appendix, studying smaller models, alternative prompting choices, \n\nWeaknesses:\n- The tasks may be too simple / short / toy.\n- Results are not particularly surprising\n- Downside of using closed models (although no open models exist for this yet): we do not actually know the training procedure for the largest models.\n  - Recent reports suggest that the instruct models are not actually RLHF: https://www.alignmentforum.org/posts/mbGjzyy6eJXT4gFpm/update-to-mysteries-of-mode-collapse-text-davinci-002-not\n  - With this in mind, I worry about drawing general conclusions about RLHF\n- Small details\n  - page 2: what makes finetuning on in-context examples metalearning?\n  - page 15: \"could could verbalize\"\n\nMisc:\n- Potential improvement: run additional experiments on the FLAN-T5 models to see how instruction tuning helps smaller models (paper released after submission, so not required but would be useful additional evidence!)\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "9JNPW28OAq", "reviewer": "Reviewer_zof8", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies task ambiguity in NLP. Recent research has demonstrated the benefits of using natural language prompts to describe the task when fine-tuning a language model, but such prompts can be ambiguous in some cases. This paper proposes a benchmark with ambiguous instructions, and shows large language models can approach human accuracy on ambiguous tasks. Fine-tuning with a small number of in-context examples further improve the performance.\n", "summary_of_the_review": "This paper studies ambiguity in NLP tasks. Although it is an important challenge in fine-tuning language models, this paper only studies a few toy tasks and the proposed method is not novel.", "strength_and_weaknesses": "Strength:\n\nNatural language prompts have been extensively studied, and the ambiguity of certain tasks, or the difficulty of designing prompts that can clearly describe the task, is an important challenge that may limit the utility of these methods in the real world.\n\nWeakness:\n\nThis paper only studies the ambiguity of a few toy tasks, which only require simple lexical understanding and the texts are synthetic and very short. It would be more interesting to see what the ambiguity is in real-world NLP tasks, and how they can be handled.\n\nBesides, the empirical observation and proposed fine-tuning method are pretty straightforward and not surprising. The \"uninformative instructions\" are very unnatural as they just replace the salient feature in a normal instruction with placeholder. Nevertheless, the instructions still contain some description of the task, and it is easy to imagine that both human and model would make guesses based on the available, incomplete information. The observation that the best language model achieve similar performance to human on such ambiguous task is not surprising, even though the setting is synthetic and simple. In my opinion, I don’t see many new insights from this paper that would help the community to better resolve task ambiguity.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6Vo8k0Wx2P", "reviewer": "Reviewer_o1Uh", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors investigate the problem of task ambiguity in prompting pre-trained LLMs. In which setting, the task descriptions sometimes contain insufficient information for humans/models to perform the task, they have to consequently look at a few examples to have a better guess on the task. The authors propose a new benchmark designed for studying the said problem, namely AmbiBench, which includes six manually designed sentence classification tasks. Using AmbiBench, they tested a set of pre-trained LLMs (covering a wide spectrum of training data sizes, training procedures, and model architectures) as well as a group of human annotators. Results suggest that only the largest LLM variant trained with additional learning from human feedback (text-davinci-002) can approach human performance on such tasks. Further, the authors show that the davinci model can quickly improve if fine-tuned on a small amount of in-context examples, even outperforming the text-davinci-002 model. ", "summary_of_the_review": "In general, I think the current version of the paper is around (maybe slightly above) the borderline. I look forward to reading the authors responses to my questions so I can better understand the work and thus have a more precise evaluation. ", "strength_and_weaknesses": "### Strength\n\n* Interesting and useful topic: I really like seeing work on this topic. Task ambiguity has never been a more important topic thanks to the flourishing of LLM research. In prompt-based learning where users/practitioners can use an inference-only LLM and prompt it to perform some never-before-seen tasks. At the same time of benefiting from greater accessibility, users have to describe their tasks (in language, as part of the prompt) in a way that maximizes LLM performance. I see this work quite related to the line of work of prompt engineering, but tackling a very specific aspect of prompting LLMs. Although using a toy-ish benchmark, I can see potential empirical value from such work. \n* Human performance: I really appreciate the authors who include human scores for the tasks, which greatly helped me get a clearer picture. Although the second paragraph of Section 4.3.1 (why text-davinci-002 outperforms humans in Figure 3) seems a bit hand wavy to me. \n* Clarity: I enjoy reading this paper, most of the content are easy to follow. \n\n\n\n### Weaknesses\n\nPlease see my comments below. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster