Data Transformations for Improving Generalized Text Embeddings
Authors: Raghuveer Thirukovalluru, Karthikeyan K, Bhuwan Dhingra


Reviewer paper summaries: The paper introduces a new method, Transformed Data for Text Embeddings (TDTE), to enhance the training of generalized embedding models using synthetic contrastive data generated by large language models (LLMs). The focus is on transforming retrieval datasets into (passage, summary(passage)) pairs to better align with the principles of contrastive learning. TDTE significantly outperforms recent models like LLM2Vec and Echo on the Massive Text Embedding Benchmark (MTEB).
Summary of strengths: 1. Retrieval and embedding are crucial topics in NLP, and transforming data to enhance this process represents a cutting-edge approach for improving embedding models.
2. The paper is well-written and highly accessible.
3. the experiments are comprehensive
Summary of weaknesses: 1. The results are not particularly impressive, as the improvement over the baseline is almost negligible.

2. Using LLMs for retrieval may not be an effective approach. Including an encoder in the experiments and baselines would be beneficial. While LLMs might enhance performance, their use becomes impractical for large datasets, which are commonly used in text embeddings.

3. The idea of using summaries as an augmentation based on InfoMin is promising, but it does not appear to be the optimal function according to the experimental results.
Comments, suggestions and typos: 1. I'm not sure what is the difference between T_1 and T_2 is
2. I would like to see this as a full paper, with more theoretical depth, and hopefully, better results that originate from that.
Overall assessment scores: 2.5