Update README.md
Browse files
README.md
CHANGED
@@ -14,10 +14,11 @@ Provence is a lightweight **context pruning model** for retrieval-augmented gene
|
|
14 |
|
15 |
*Developed by*: Naver Labs Europe
|
16 |
*License*: [CC BY-NC 4.0.](https://creativecommons.org/licenses/by-nc/4.0/)
|
|
|
17 |
* *Model*: `provence-reranker-debertav3-v1` (Provence for Pruning and Reranking Of retrieVEd relevaNt ContExt)
|
18 |
* *Backbone model*: [DeBERTav3-reranker](https://huggingface.co/naver/trecdl22-crossencoder-debertav3) (trained from [DeBERTa-v3-large](https://huggingface.co/microsoft/deberta-v3-large))
|
19 |
* *Model size*: 430 million parameters
|
20 |
-
* *Context length*: 512 tokens
|
21 |
|
22 |
## Usage
|
23 |
|
@@ -47,8 +48,6 @@ provence_output = provence.process(question, context, always_select_title=True)
|
|
47 |
# Provence Output: {'reranking_score': 3.022725, pruned_context': 'Shepherd’s pie. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]
|
48 |
```
|
49 |
|
50 |
-
Training code, as well as RAG experiments with Provence can be found in the [BERGEN](https://github.com/naver/bergen) library.
|
51 |
-
|
52 |
## Model interface
|
53 |
|
54 |
Interface of the `process` function:
|
@@ -68,9 +67,7 @@ Interface of the `process` function:
|
|
68 |
* **Provence automatically detects the number of sentences to keep**, based on a threshold. We found that the default value of a threshold works well across various domains, but the threshold can be adjusted further to better meet the particular use case needs.
|
69 |
* **Provence is robust to various domains**, being trained on a combination of diverse MS Marco and Natural Questions data.
|
70 |
* **Provence works out-of-the-box with any LLM**.
|
71 |
-
* **Provence is fast**: we release a standalone DeBERTa-based model [here]() and a unified reranking+context pruning model, which incorporates context pruning into reranking, an already existing stage of modern RAG pipelines. The latter makes context pruning basically zero cost in the RAG pipeline!
|
72 |
|
73 |
-
More details are available in the [blogpost]().
|
74 |
|
75 |
## Model Details
|
76 |
|
@@ -82,7 +79,7 @@ More details are available in the [blogpost]().
|
|
82 |
* Context length: 512 tokens (similar to the pretrained DeBERTa model)
|
83 |
* Evaluation: we evaluate Provence on 7 datasets from various domains: Wikipedia, biomedical data, course syllabi, and news. We find that Provence is able to prune irrelevant sentences with little-to-no drop in performance, in all domains, and outperforms existing baselines on the Pareto front (top right corners of the plots).
|
84 |
|
85 |
-
Check out more analysis in the [paper]()!
|
86 |
|
87 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/6273df31c3b822dad2d1eef2/WMmfsNG48O830paaBAaQF.png" width="600">
|
88 |
|
@@ -94,14 +91,14 @@ This work is licensed under CC BY-NC 4.0.
|
|
94 |
## Cite
|
95 |
|
96 |
```
|
97 |
-
@misc{
|
98 |
title={Provence: efficient and robust context pruning for retrieval-augmented generation},
|
99 |
author={Nadezhda Chirkova and Thibault Formal and Vassilina Nikoulina and Stéphane Clinchant},
|
100 |
-
year={
|
101 |
-
eprint={
|
102 |
archivePrefix={arXiv},
|
103 |
primaryClass={cs.CL},
|
104 |
-
|
105 |
}
|
106 |
```
|
107 |
|
|
|
14 |
|
15 |
*Developed by*: Naver Labs Europe
|
16 |
*License*: [CC BY-NC 4.0.](https://creativecommons.org/licenses/by-nc/4.0/)
|
17 |
+
*Paper*: https://arxiv.org/abs/2501.16214, accepted to ICLR 2025
|
18 |
* *Model*: `provence-reranker-debertav3-v1` (Provence for Pruning and Reranking Of retrieVEd relevaNt ContExt)
|
19 |
* *Backbone model*: [DeBERTav3-reranker](https://huggingface.co/naver/trecdl22-crossencoder-debertav3) (trained from [DeBERTa-v3-large](https://huggingface.co/microsoft/deberta-v3-large))
|
20 |
* *Model size*: 430 million parameters
|
21 |
+
* *Context length*: 512 tokens
|
22 |
|
23 |
## Usage
|
24 |
|
|
|
48 |
# Provence Output: {'reranking_score': 3.022725, pruned_context': 'Shepherd’s pie. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]
|
49 |
```
|
50 |
|
|
|
|
|
51 |
## Model interface
|
52 |
|
53 |
Interface of the `process` function:
|
|
|
67 |
* **Provence automatically detects the number of sentences to keep**, based on a threshold. We found that the default value of a threshold works well across various domains, but the threshold can be adjusted further to better meet the particular use case needs.
|
68 |
* **Provence is robust to various domains**, being trained on a combination of diverse MS Marco and Natural Questions data.
|
69 |
* **Provence works out-of-the-box with any LLM**.
|
|
|
70 |
|
|
|
71 |
|
72 |
## Model Details
|
73 |
|
|
|
79 |
* Context length: 512 tokens (similar to the pretrained DeBERTa model)
|
80 |
* Evaluation: we evaluate Provence on 7 datasets from various domains: Wikipedia, biomedical data, course syllabi, and news. We find that Provence is able to prune irrelevant sentences with little-to-no drop in performance, in all domains, and outperforms existing baselines on the Pareto front (top right corners of the plots).
|
81 |
|
82 |
+
Check out more analysis in the [paper](https://arxiv.org/abs/2501.16214)!
|
83 |
|
84 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/6273df31c3b822dad2d1eef2/WMmfsNG48O830paaBAaQF.png" width="600">
|
85 |
|
|
|
91 |
## Cite
|
92 |
|
93 |
```
|
94 |
+
@misc{chirkova2025provenceefficientrobustcontext,
|
95 |
title={Provence: efficient and robust context pruning for retrieval-augmented generation},
|
96 |
author={Nadezhda Chirkova and Thibault Formal and Vassilina Nikoulina and Stéphane Clinchant},
|
97 |
+
year={2025},
|
98 |
+
eprint={2501.16214},
|
99 |
archivePrefix={arXiv},
|
100 |
primaryClass={cs.CL},
|
101 |
+
url={https://arxiv.org/abs/2501.16214},
|
102 |
}
|
103 |
```
|
104 |
|