Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies learning composable task codes for soft-prompting of language models. The authors propose an approach similar to VQ-VAE where each task is associated with a set of discrete codes and embedding of these codes from a codebook lookup table is stacked as a soft-prompt for a language model. First, a 2D embedding for each training task is learned via another lookup table. Next, each row of this embedding matrix is used to search for the nearest code embedding in codebook lookup table. The model is trained with typical language model loss in addition to commitment and embedding losses for learning codebook embeddings. By sharing codes across tasks, this approach presents a natural task compositionality. The authors propose code ensembling for zero-shot learning and bitwise searching for few-shot learning. On a set of benchmark tasks, the model compares favorably to previous models; achieving on-par or better on average. The authors also experiment with interpretability -- similar tasks share codes -- and controllability -- bitwise perturbation exhibits different behaviors.
strength_and_weaknesses: **Strengths** The paper studies a very important and interesting problem -- learning composable task codes for generalization to unseen tasks. I find the formulation of the problem using VQ-VAE natural and inference methods convincing. It is interesting to see that empirical results, especially zero-shot learning, improves compared to self-training or T0 models.

**Weaknesses** There are a few points that require clarification and improvement.
1. Does sharing a code necessarily mean that two tasks are similar in some sense? I found no evidence as to how important a code is for a task; hence, I can't say if a code is merely noise that gets ignored by the model eventually. For example, if you increase the number of codes, *L*, the model might be given more codes but not all of them would be used. There is not loss that encourages the LLM to utilize all codes as well.

2. What is the effective number of codes that discrete compositional task code, *z*, has? Are all codes in z unique or are they frequently reused?

3. Comparison to previous work needs clarification.
- Are baselines multi-task learners or are they initialized with some pretrained embedding? Like, prompts can be initialized or they can be trained in mutli-task fashion.
- What are the architectural details of other models, including model size, are they comparable?
- What is the performance of model tuning if you use all 32 examples for training? I think it is a bit unfair to use all examples for CTR while using only half of them for others.
- You mention that other methods update extra parameters as a weakness but this is not true. Prompt tuning only updates an additional prompt while the LLM is fixed. On the other hand, CTR updates a codebook and LLM is also fine-tuned. I think this is less scalable overall compared to parameter efficient updates.

4. How do you choose hyperparameters? Like, "CTR length", "codebook size", or *N=60*. I don't see a clear pattern on final performance and no training/dev results are given.

5. Could you clarify:
- How do you combine with manual prompts? Are they appended as additional soft-prompt vectors?
- What is the performance if you haven't updated the LLM? This would be a fairer comparison to parameter efficient models.
- What is the performance w.r.t. increasing data size for few-shot learning?
- Why does the model perform worse on co-reference?
clarity,_quality,_novelty_and_reproducibility: I find the composable task codes novel and the paper is well written. I believe results can't be reproducible without proper details on hyper parameters.
summary_of_the_review: I think composable task codes, VQ-VAE style training for LLM fine-tuning, and generalization to novel tasks during inference are strong points of the paper. But, there are still pieces that are unclear and needs improvement.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method for multitask learning of NLP tasks with compositional codes. Given a task id, a sequence of discrete codes are obtained. Code vectors obtained from these codes using an embedding table are fed as input to the language model as a task representation. The codebook and the language model are jointly trained with a language modeling loss and additional losses to enforce consistency between task ids and corresponding codes. Results show that the proposed approach performs better than prior methods in zero-label and few-shot settings. In addition, the trained control codes offer some degree of interpretability and controllability.
strength_and_weaknesses: Pros
* Viewing task representations as compositional codes is an interesting perspective
* Proposed method offers some degree of interpretability and controllability in task learning 

Cons
* The problem settings and proposed approach  are vaguely described. 
* Several aspects of the experimental setting were unclear which makes it hard to interpret the results. 

clarity,_quality,_novelty_and_reproducibility: Technical details in section 3.3 need to be explained with math/pseudocode and better clarity. 
* The zero-label setting is not clear to me. The problem formulation states “given a new task along with a set of unlabelled data”. What are these unlabelled instances? How are they relevant to the task? 
* “gives predictions that deviate from a uniform distribution” - Does this mean the proposed approach only applies to classification tasks?
* In the few-shot learning setting, the authors mention an algorithm which iteratively changes a single bit of code, but do not discuss further details. 

Several aspects of the baselines were unclear/vague. I couldn’t gauge whether the comparison is fair due to these missing details. 
* “T0 reports the average results over multiple prompts” - What are these prompts? Are they natural language prompts?
* Model Tuning baseline - Is this model trained on the same training tasks as the proposed method?
* Does CTR use any instructions?

Table 1: What are the evaluation metrics for each task?

How were the results in Table 5 obtained? Some results do not look very convincing (e.g. the dialog example in the first instance). 

Other comments
* FIgure 2 has poor readability, fonts are too small
* Sec 3.1 says that the code embeddings are optionally passed through a decoder, but doesn’t explain what this means. 

summary_of_the_review: Updating score to 6 post author response. 

The authors propose an interesting approach to learn task representations as compositional codes. However, due to missing details about the proposed approach, experimental setup and baselines it is difficult to gauge the impact of the approach. In addition, the claims about interpretability and controllability are not entirely convincing. See more detailed comments below.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a prompt-based architecture to improve the performance of large language models on zero-label learning and few-shot learning. The authors employ multi-task training to learn a discrete, compositional codebook, which is for mapping the given task to task types and then generate a task-related prompt for LLM inference.

Learning a discrete latent codebook to enhance the interpretability of prompts is impressive, and the experiments are comprehensive and convincing. 

strength_and_weaknesses: # Strength
- The idea is useful and interesting
- the models provide some interpretability, controllability, and generilization

# Weaknesses
- have not found any significant one

clarity,_quality,_novelty_and_reproducibility: 
The paper is well-written.

Questions:
- Sometines, performance in the zero-labeling setting is better than the few-shot setting. I am wondering how much it is sensitive to the selection of codes. Plus, what is the scale of these data.  
-- Another point is that, in some scenarios, we do have some annotated data. Can CTR also benefit from many annotated  examples?


summary_of_the_review: The idea is useful and interesting. I generally like this idea. I recommend acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: 
This paper introduces Compositional Task Representations, an approach to train language models to achieve strong zero-shot and few-shot performance on unseen tasks without the need to rely on prompts at test time. 

Given a pre-trained LM, CTR is trained on a variety of tasks. For each task, a vector is initialized and trained. An encoder then produces a sequence of query vectors from this vector, These query vectors are used to retrieve a sequence of codes from a global set of trainable codes. Those codes are given as input to the LM, serving as a task representation meant to replace the prompt. At inference time on a new task, the authors search for a good composition of codes for the new tasks using unlabelled examples.

The authors compare CTR to a comprehensive set of recent baselines on both the zero-shot and few-shot settings on the T0 benchmark. They achieve strong results overall on both these settings, outperforming other methods. They present ablations on the use of prompts at test time for CTR and on the importance of sub-components of their loss. Finally, there is a rich analysis section showing the interpretability, controllability and generalization ability of CTR. 

strength_and_weaknesses: Pros:
* The paper is well-written and well-structured. The approach is well explained, related work is presented clearly and the experimental setup is clear. 
* The set of baselines is comprehensive and very helpful in assessing the value of the method presented. Having both the zero-shot and few-shot settings is also helpful.
* The method achieves good results overall, although there may be some issues with measurement (see Cons section).
* Both ablations are interesting and helpful.
* The analysis section is rich and sheds light in some of the properties and benefits of CTR.

Cons:
* My main concern is on the experimental setup, especially hyper-parameter tuning. For this setup, the proper way to do hyper-parameter tuning is not always obvious. Table 10 and 11 shows the authors have tried several configurations/options (at least 15) and this paper presents the best one. Given the wide variation on results and limited data of some tasks, the authors need to clarify how they performed hyper-parameter tuning for their method as well as the baselines. This will ensure the comparison in Table 1 is fair.
* There needs to be more information on the experimental setup for the baselines. For instance, the authors introduce the "Manual-code" baseline but very little detail is given. For the self-training baseline, we do not know the experimental details at all. In general this section is lacking and makes the comparison of Table 1 harder.
* There are details missing that harm reproducibility. For instance, the value of the beta parameter selected for the loss is not given, let alone how selection was done. 
* I would comment on the text that there is wide variability across tasks for Table 1. For instance, Discarding the CB outperformance the method is very similar to prior ones on the zero-shot setting. Given the aforementioned sparsity of details on hyper-parameter selection and baselines this is an even bigger issue.
* Some further ablations would be interesting:
  * How does the method perform if fewer tasks are provided during training (currently 319). 
  * Does the order of the latent codes matter for performance? At inference time, how does performance deteriorate if you shuffle the order of the codes selected for a task. 
  * Visualizing the task embeddings / the codes may be interesting.  
* Minor: Typo p8 combing -> combining 
clarity,_quality,_novelty_and_reproducibility: Clarity is excellent.

Quality is quite good, with the lack of details on hyper-parameter selection and baselines being the main issue.

Novelty is not an issue.

Reproducibility is a bigger issue. Not a lot of clarity on hyper-param tuning, baselines and the value of some parameters (such as /beta) are not given.
summary_of_the_review: This is a well-written paper that introduces an interesting method to train LM to have strong zero and few-shot capabilities. In addition to good results, the method offers nice advantages in terms of controllability and interpretability. The main issue with this paper is in the lack of clarity on hyper-parameter selection and details of the baselines. Given the results seem to be quite sensitive to these minutes details, this casts some doubt into the claimed performance.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a prompt-free method of inducing a language model to perform a new task. Specifically, a set of code vectors are learned on a collection of training tasks. The paper then proposes a label-free and few-shot method for selecting which codes to use on an unseen task. Evaluation is primarily done in the same setting as T0 with improved performance compared to other prompt-learning-style methods, along with analysis of the learned codewords and their usage. While reviewers generally agreed that the method was interesting and novel, one reviewer pointed out an important issue with the paper - the proposed method is not truly zero-shot because a wide variety of different variants of the algorithm (including different hyperparameters and architectural designs) are evaluated on the held-out tasks and performance is reported for the best variant. This is not zero-shot learning because it involves using labeled data to infer the best system design. Using labeled data for system design in a zero-shot setting is equivalent to training on the test set (see e.g. "True Few-Shot Learning with Language Models") and could be very misleading to the readers. We strongly recommend the authors to revise the terminology in the final version of the manuscript to truthfully reflect the actual evaluation setup in order to avoid unnecessary confusion and misunderstanding. 
justification_for_why_not_higher_score: The evaluation results are on the optimistic side since the setup is not truly zero shot as it involved labeled data. (See summary above) 
justification_for_why_not_lower_score: The proposed approach is interesting and well-received by the reviewers.
Decision: Accept: poster