Paper: When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks (ICLR-2024)

Reviewer comment (Experiments):
One major weakness is the selection of datasets. Most of these focus on information extraction (IE), and as pointed out in previous work [1], IE tasks are not generally considered in most instruction-tuning datasets. Therefore, IE tasks are not only specification-heavy but also unfamiliar to these instruction-tuned models. A simple investigation/baseline should involve manually curating specification-heavy but familiar instructions such as: The output length should not exceed X length, the output should start with…, the output should be in a JSON format, among other considerations. These specifications are likely to arise in user cases or training data and are also specification-heavy. This provides a further disentanglement of the root of this inability. Also, more interesting discussions may occur in this direction. For example, can general specifications assist complex task-specific specification understanding? And vice versa? [1] Zhang et al, Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors. Findings of ACL 2023.

Author response:
Regarding the Weakness #1: Thank you for the insightful suggestion. We will design tasks that are 'specification-heavy but familiar (to LLMs)' to analyze the sources of inability more effectively. Additionally, regarding the discussion on 'can general specifications assist complex task-specific specification understanding?', we intend to explore this in our future work. Our aim is to enhance LLMs' capabilities in instruction understanding and reasoning through the use of general specifications.

Reviewer comment (Evaluation):
Though the motivation of this paper is impressive, the analysis underwhelms in some aspects. Firstly, in Figure 2, how do the authors attribute the errors made by LLMs to the three types of reasons? Is there a chain-of-thoughts or any monologues to help identify the reasons? It seems to me that the example presented in "unspecific context understanding" can also be attributed to “Misaligned Schema Understanding”. One possible explanation is that the model noticed the “tackor“ but didn’t treat it as “product-car“, because the task instruction missed important information about the class “product-car“.

Author response:
Regarding the Weakness #1: Regarding the first question, the three primary error causes were identified through our error analysis. We have conducted additional experimental analyses for each of these causes, detailed in their respective subsections. For the 'tractor' example in Figure 2, we will include descriptions for each category in the task instructions in future experiments. This will help us more accurately differentiate between 'unspecific context understanding' and 'Misaligned Schema Understanding' as reasons of error.

Reviewer comment (Writing):
What is the scope of “specification-heavy tasks”? This paper primarily focuses on traditional IE tasks (16 out of 18 tasks). However, there is no clear boundary of “specification-heavy" tasks in this paper. In my opinion, most of NLP tasks involve “complex” annotation guidelines if we would like to elaborate detailed task descriptions and edge cases. Could the authors provide a clear definition of "specification-heavy task" or at least some examples of opposite tasks? It's also possible to narrow the scope of your paper to IE tasks, without losing generality.

Author response:
Regarding the Weakness #2: The definition of 'specification-heavy' tasks we propose refers to tasks with complicated and extensive task specifications, which would typically require several hours for an ordinary person to learn and perform competently, though not necessarily perfectly. For such tasks, significant learning time is essential to achieve a decent level of proficiency, not necessarily to solve the task flawlessly.

In response to the observation that 'most NLP tasks involve “complex” annotation guidelines for detailed task descriptions and edge cases,' it's important to note that many tasks, like the NLI task, can be performed satisfactorily in a relatively short time. The 'detailed task descriptions and edge cases' are requirements for perfect execution, which does not categorize these as specification-heavy tasks. Thank you for your valuable suggestion. We will incorporate additional tasks and provide clearer and more intuitive descriptions for them in our revision.

Reviewer comment (Evaluation):
After the instruction tuning, the zero-shot performance gain of different tasks varies greatly. Could the authors provide some analysis or intuition on it?

Author response:
Regarding the Question #1: Thank you for your valuable suggestion. In line with your advice, we will provide further analysis and insights based on the results of our experiments.

Reviewer comment (Experiments):
It would be intriguing to see to what degree the instruction tuning on specification-heavy tasks addresses three identified challenges.

Author response:
Regarding the Question #2: Thank you for your valuable suggestion. We will conduct a similar error analysis on instruction-tuned models to observe the proportion of errors corrected by instruction tuning.

Reviewer comment (Novelty):
This paper [1] is highly relevant to your second reason (misalignment in task schema comprehension with humans).

Author response:
Regarding the Question #3: Thank you for the reminder. We will cite it in our subsequent revisions.

Reviewer comment (Experiments):
The success of the ICL approach highly depends on the amount and quality of information provided in the prompts. The results in table 1 are all based on a shortened version of the task description. Examples in the appendix show that such shortened descriptions are too concise, e.g., they do not even contain a single sentence definition of each label. The provided few-shots do not cover all labels. Longer prompts can still fit in at least a subset of the LLMs used for the experiments. The paper needs to experiment with more detailed prompts to confirm that the poor accuracy of the ICL approach is not due to the very concise set of instructions.

Author response:
Regarding the Weakness #1: Thank you for your suggestion. We respond to this issue in our general response. We will conduct additional experiments using more detailed prompts.

Reviewer comment (Experiments):
The alignment experiments in Section 4.2 only hold out one task at a time, but still tasks of the same type are included in the alignment dataset. The paper needs to report results with a whole task type (e.g., All 4 RE tasks) is held out to confirm that the gain in accuracy is not really due to similarity between tasks.

Author response:
Regarding the Weakness #2: Thank you for your valuable suggestion. Holding out a task each time is a more reasonable approach. In light of this, we plan to revise the settings of all our experiments to hold out one task, in line with the practices in the FLAN series.

Reviewer comment (Evaluation):
Comparing table 1 to table 4, Aligned ICL has worse accuracy than the unaligned baseline on 3 tasks MAVEN-Subevent and the two sentiment analysis tasks. Do you have an explanation or an intuition for why that is the case for those three datasets?

Author response:
Regarding the Question #1: The inferior performance on the MAVEN-Subevent task may be attributed to the sparse annotations and complex output requirements of this type of task, which could impede the effective generalization of Aligned ICL. As for the diminished performance on the two sentiment analysis tasks, this could be due to the inclusion of sentiment analysis tasks in the original alignment corpus of FLAN-UL2. Consequently, FLAN-UL2 might perform well on these tasks even without our alignment. However, the relatively low scale of sentiment classification in our used alignment data could lead to reduced performance of the FLAN-UL2 aligned on our data in sentiment analysis tasks.

Reviewer comment (Experiments):
In Section 3.1, the paper says that "LLMs ignore all the contexts in 18 instances". How did you find out about that?

Author response:
Regarding the Question #2: This conclusion is drawn from our comparative experiments. When we removed all context and merely posed the corresponding questions to GPT-4, the model was still able to correctly answer these questions. From this observation, we infer that the LLM does not utilize the context information, which we term as 'ignoring all the context.’

Reviewer comment (Experiments):
Performance of `instruction + demonstrations`. The effectiveness of prompts that include both 'instruction and demonstrations' might be enhanced if large language models (LLMs) are specifically trained with such prompts. This training approach could potentially improve their performance on these tasks.

Author response:
For the Weakness #2: It appears there may be some ambiguity in the question posed. We have employed instruction tuning method to enhance the LLM's ability to follow instructions. This approach has not only improved the model's performance across multiple tasks but also on tasks that it has not previously encountered. I hope this clarification addresses your query effectively.

Reviewer comment (Experiments):
Labels covered in demonstrations are important. It's crucial to show how demonstrations incorporate all labels in the label space. Typically, k-shot learning implies presenting k examples for each label. If the demonstrations are limited due to the constraints of input context length, this could understandably lead to lower performance.

Author response:
For the Weakness #3: Thank you for your valuable suggestion. Firstly, our experimental setup adheres to the conventional usage established in previous research [2,3], namely employing a certain number (8, 16, or, 32) of 'shot' examples for in-context learning. Indeed, the exclusion of some labels from the label space in our demonstrations may affect the model's effectiveness. We appreciate this insight and will incorporate this aspect in future experiments with models that can accommodate longer inputs.