jina-embeddings-v3: Multilingual Embeddings With Task LoRA
Authors: name: Saba Sturua; name: Isabelle Mohr; name: Mohammad Kalim Akram; name: Michael Günther; name: Bo Wang; name: Markus Krimmel; name: Feng Wang; name: Georgios Mastrapas; name: Andreas Koukounas; name: Andreas Koukounas; name: Nan Wang; name: Han Xiao
Published: 2024-09-16 11:10:29

Abstract: We introduce jina-embeddings-v3, a novel text embedding model with 570
million parameters, achieves state-of-the-art performance on multilingual data
and long-context retrieval tasks, supporting context lengths of up to 8192
tokens. The model includes a set of task-specific Low-Rank Adaptation (LoRA)
adapters to generate high-quality embeddings for query-document retrieval,
clustering, classification, and text matching. Additionally, Matryoshka
Representation Learning is integrated into the training process, allowing
flexible truncation of embedding dimensions without compromising performance.
Evaluation on the MTEB benchmark shows that jina-embeddings-v3 outperforms the
latest proprietary embeddings from OpenAI and Cohere on English tasks, while
achieving superior performance compared to multilingual-e5-large-instruct
across all multilingual tasks.