File size: 8,340 Bytes
7884ec1 d04d468 7884ec1 907c4fb d04d468 7884ec1 a961d0d 36e8c67 a961d0d a7da899 a961d0d 7884ec1 a961d0d d04d468 c28b4db 26b193b c28b4db d04d468 36e8c67 d04d468 228fc3b d04d468 7884ec1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 |
---
library_name: transformers
license: apache-2.0
language:
- en
tags:
- reranker
- cross-encoder
---
<br><br>
<p align="center">
<img src="https://aeiljuispo.cloudimg.io/v7/https://cdn-uploads.huggingface.co/production/uploads/603763514de52ff951d89793/AFoybzd5lpBQXEBrQHuTt.png?w=200&h=200&f=face" alt="Finetuner logo: Finetuner helps you to create experiments in order to improve embeddings on search tasks. It accompanies you to deliver the last mile of performance-tuning for neural search applications." width="150px">
</p>
<p align="center">
<b>Trained by <a href="https://jina.ai/"><b>Jina AI</b></a>.</b>
</p>
# jina-reranker-v1-turbo-en
This model is designed for **blazing-fast** reranking while maintaining **competitive performance**. What's more, it leverages the power of our [JinaBERT](https://arxiv.org/abs/2310.19923) model as its foundation. `JinaBERT` itself is a unique variant of the BERT architecture that supports the symmetric bidirectional variant of [ALiBi](https://arxiv.org/abs/2108.12409). This allows `jina-reranker-v1-turbo-en` to process significantly longer sequences of text compared to other reranking models, up to an impressive **8,192** tokens.
To achieve the remarkable speed, the `jina-reranker-v1-turbo-en` employ a technique called knowledge distillation. Here, a complex, but slower, model (like our original [jina-reranker-v1-base-en](https://jina.ai/reranker/)) acts as a teacher, condensing its knowledge into a smaller, faster student model. This student retains most of the teacher's knowledge, allowing it to deliver similar accuracy in a fraction of the time.
Here's a breakdown of the reranker models we provide:
| Model Name | Layers | Hidden Size | Parameters (Millions) |
| ------------------------------------------------------------------------------------ | ------ | ----------- | --------------------- |
| [jina-reranker-v1-base-en](https://jina.ai/reranker/) | 12 | 768 | 137.0 |
| [jina-reranker-v1-turbo-en](https://huggingface.co/jinaai/jina-reranker-v1-turbo-en) | 6 | 384 | 37.8 |
| [jina-reranker-v1-tiny-en](https://huggingface.co/jinaai/jina-reranker-v1-tiny-en) | 4 | 384 | 33.0 |
> Currently, the `jina-reranker-v1-base-en` model is not available on Hugging Face. You can access it via the [Jina AI Reranker API](https://jina.ai/reranker/).
As you can see, the `jina-reranker-v1-turbo-en` offers a balanced approach with **6 layers** and **37.8 million** parameters. This translates to fast search and reranking while preserving a high degree of accuracy. The `jina-reranker-v1-tiny-en` prioritizes speed even further, achieving the fastest inference speeds with its **4-layer**, **33.0 million** parameter architecture. This makes it ideal for scenarios where absolute top accuracy is less crucial.
# Usage
1. The easiest way to starting using `jina-reranker-v1-turbo-en` is to use Jina AI's [Reranker API](https://jina.ai/reranker/).
```bash
curl https://api.jina.ai/v1/rerank \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_API_KEY" \
-d '{
"model": "jina-reranker-v1-turbo-en",
"query": "Organic skincare products for sensitive skin",
"documents": [
"Eco-friendly kitchenware for modern homes",
"Biodegradable cleaning supplies for eco-conscious consumers",
"Organic cotton baby clothes for sensitive skin",
"Natural organic skincare range for sensitive skin",
"Tech gadgets for smart homes: 2024 edition",
"Sustainable gardening tools and compost solutions",
"Sensitive skin-friendly facial cleansers and toners",
"Organic food wraps and storage solutions",
"All-natural pet food for dogs with allergies",
"Yoga mats made from recycled materials"
],
"top_n": 3
}'
```
2. Alternatively, you can use the latest version of the `sentence-transformers>=0.27.0` library. You can install it via pip:
```bash
pip install -U sentence-transformers
```
Then, you can use the following code to interact with the model:
```python
from sentence_transformers import CrossEncoder
# Load the model, here we use our turbo sized model
model = CrossEncoder("jinaai/jina-reranker-v1-turbo-en", trust_remote_code=True)
# Example query and documents
query = "Organic skincare products for sensitive skin"
documents = [
"Eco-friendly kitchenware for modern homes",
"Biodegradable cleaning supplies for eco-conscious consumers",
"Organic cotton baby clothes for sensitive skin",
"Natural organic skincare range for sensitive skin",
"Tech gadgets for smart homes: 2024 edition",
"Sustainable gardening tools and compost solutions",
"Sensitive skin-friendly facial cleansers and toners",
"Organic food wraps and storage solutions",
"All-natural pet food for dogs with allergies",
"Yoga mats made from recycled materials"
]
results = model.rank(query, documents, return_documents=True, top_k=3)
```
3. You can also use the `transformers` library to interact with the model programmatically.
```python
!pip install transformers
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained(
'jinaai/jina-reranker-v1-turbo-en', num_labels=1, trust_remote_code=True
)
# Example query and documents
query = "Organic skincare products for sensitive skin"
documents = [
"Eco-friendly kitchenware for modern homes",
"Biodegradable cleaning supplies for eco-conscious consumers",
"Organic cotton baby clothes for sensitive skin",
"Natural organic skincare range for sensitive skin",
"Tech gadgets for smart homes: 2024 edition",
"Sustainable gardening tools and compost solutions",
"Sensitive skin-friendly facial cleansers and toners",
"Organic food wraps and storage solutions",
"All-natural pet food for dogs with allergies",
"Yoga mats made from recycled materials"
]
# construct sentence pairs
sentence_pairs = [[query, doc] for doc in documents]
scores = model.compute_score(sentence_pairs)
```
That's it! You can now use the `jina-reranker-v1-turbo-en` model in your projects.
# Evaluation
We evaluated Jina Reranker on 3 key benchmarks to ensure top-tier performance and search relevance.
| Model Name | NDCG@10 (17 BEIR datasets) | NDCG@10 (5 LoCo datasets) | Hit Rate (LlamaIndex RAG) |
| ------------------------------------------- | -------------------------- | ------------------------- | ------------------------- |
| `jina-reranker-v1-base-en` | **52.45** | **87.31** | **85.53** |
| `jina-reranker-v1-turbo-en` (you are here) | **49.60** | **69.21** | **85.13** |
| `jina-reranker-v1-tiny-en` | **48.54** | **70.29** | **85.00** |
| `mxbai-rerank-base-v1` | 49.19 | - | 82.50 |
| `mxbai-rerank-xsmall-v1` | 48.80 | - | 83.69 |
| `ms-marco-MiniLM-L-6-v2` | 48.64 | - | 82.63 |
| `ms-marco-MiniLM-L-4-v2` | 47.81 | - | 83.82 |
| `bge-reranker-base` | 47.89 | - | 83.03 |
**Note:**
- `NDCG@10` is a measure of ranking quality, with higher scores indicating better search results. `Hit Rate` measures the percentage of relevant documents that appear in the top 10 search results.
- The results of LoCo datasets on other models are not available since they **do not support** long documents more than 512 tokens.
For more details, please refer to our [benchmarking sheets](https://docs.google.com/spreadsheets/d/1V8pZjENdBBqrKMzZzOWc2aL60wtnR0yrEBY3urfO5P4/edit?usp=sharing).
# Contact
Join our [Discord community](https://discord.jina.ai/) and chat with other community members about ideas.
|