Learn and Expose: Techniques for Task-aware Text Embeddings
Authors: Idan Schwartz, Yeari Vigder, Lior Wolf


Reviewer paper summaries: This work proposes EMBEDDERCONTEXT for Task-aware Text Embeddings.  This framework first trains a linear layer to a downstream task and then optimizes pseudo context embeddings for exposing task-related knowledge. 
In the experiments, 12 classification datasets from MTEB are used for both fine-tuning and evaluation. The empirical studies demonstrate that the proposed method can outperform general-purpose embeddings.
Summary of strengths: 1. I find it interesting to develop task-aware embeddings since the majority focuses on general ones.  
2. The proposed method seems simple and workable but a real task-aware benchmark is needed for evaluation.
Summary of weaknesses: 1. This work aims to train task-aware embeddings, but I found that the experimental section has been around dataset-aware embeddings.  Only classification datasets are used in the experiment, and then a model is fine-tuned and tested on each dataset. I would believe that this setting doesn't have an ideal practical usage since each dataset has its unique distribution.  I would suggest including different tasks in the task-aware benchmark, such as text classification, clustering, retrieval, etc.  If the proposed model can be generalized to a specific task with multiple datasets, I can see the clear benefit of the pipeline. 
2. Regarding the pseudo context,  additional analysis is needed. Since the core contribution of this work is to train a pseudo context for a specific task, it is useful to explain the intuition, visualize learned context representations, and play with hyper-parameters related to it.
2. The writing part hinders readability (see comments below)
Comments, suggestions and typos: 1. As for notations, maybe use different font styles to distinguish scalars, vectors and matrices.
2.  I don't fully understand the motivation for using pseudo-contexts. What is the definition of the LLM context? It is useful to explain it in the beginning.
3. what is the length of pseudo context? Will it change according to different datasets?
4. In figure 1, it is useful to give a couple of words to describe the notations. 
5. what did context embedding learn?
Overall assessment scores: 2.5