Post
Microsoft: Improving Text Embeddings with Large Language Models
- uses an LLM instead of complex pipelines to create the training data
- directly generates data for numerous text embedding tasks
- fine tunes standard models with contrastative loss achieving great performance
- critical thought: isn't this kinda benchmark hacking? If the benchmarks are so encompassing that they capture the complete idea of embedding, it's maybe a good idea, but often it is oversimplifying, I find.
Feel free to share your thoughts, even if they like mine don't beat the benchmarks ;P
https://arxiv.org/abs/2401.00368
- uses an LLM instead of complex pipelines to create the training data
- directly generates data for numerous text embedding tasks
- fine tunes standard models with contrastative loss achieving great performance
- critical thought: isn't this kinda benchmark hacking? If the benchmarks are so encompassing that they capture the complete idea of embedding, it's maybe a good idea, but often it is oversimplifying, I find.
Feel free to share your thoughts, even if they like mine don't beat the benchmarks ;P
https://arxiv.org/abs/2401.00368
