nadiinchi commited on
Commit
c9fdc97
·
verified ·
1 Parent(s): bbb1f20

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -10
README.md CHANGED
@@ -14,10 +14,11 @@ Provence is a lightweight **context pruning model** for retrieval-augmented gene
14
 
15
  *Developed by*: Naver Labs Europe
16
  *License*: [CC BY-NC 4.0.](https://creativecommons.org/licenses/by-nc/4.0/)
 
17
  * *Model*: `provence-reranker-debertav3-v1` (Provence for Pruning and Reranking Of retrieVEd relevaNt ContExt)
18
  * *Backbone model*: [DeBERTav3-reranker](https://huggingface.co/naver/trecdl22-crossencoder-debertav3) (trained from [DeBERTa-v3-large](https://huggingface.co/microsoft/deberta-v3-large))
19
  * *Model size*: 430 million parameters
20
- * *Context length*: 512 tokens
21
 
22
  ## Usage
23
 
@@ -47,8 +48,6 @@ provence_output = provence.process(question, context, always_select_title=True)
47
  # Provence Output: {'reranking_score': 3.022725, pruned_context': 'Shepherd’s pie. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]
48
  ```
49
 
50
- Training code, as well as RAG experiments with Provence can be found in the [BERGEN](https://github.com/naver/bergen) library.
51
-
52
  ## Model interface
53
 
54
  Interface of the `process` function:
@@ -68,9 +67,7 @@ Interface of the `process` function:
68
  * **Provence automatically detects the number of sentences to keep**, based on a threshold. We found that the default value of a threshold works well across various domains, but the threshold can be adjusted further to better meet the particular use case needs.
69
  * **Provence is robust to various domains**, being trained on a combination of diverse MS Marco and Natural Questions data.
70
  * **Provence works out-of-the-box with any LLM**.
71
- * **Provence is fast**: we release a standalone DeBERTa-based model [here]() and a unified reranking+context pruning model, which incorporates context pruning into reranking, an already existing stage of modern RAG pipelines. The latter makes context pruning basically zero cost in the RAG pipeline!
72
 
73
- More details are available in the [blogpost]().
74
 
75
  ## Model Details
76
 
@@ -82,7 +79,7 @@ More details are available in the [blogpost]().
82
  * Context length: 512 tokens (similar to the pretrained DeBERTa model)
83
  * Evaluation: we evaluate Provence on 7 datasets from various domains: Wikipedia, biomedical data, course syllabi, and news. We find that Provence is able to prune irrelevant sentences with little-to-no drop in performance, in all domains, and outperforms existing baselines on the Pareto front (top right corners of the plots).
84
 
85
- Check out more analysis in the [paper]()!
86
 
87
  <img src="https://cdn-uploads.huggingface.co/production/uploads/6273df31c3b822dad2d1eef2/WMmfsNG48O830paaBAaQF.png" width="600">
88
 
@@ -94,14 +91,14 @@ This work is licensed under CC BY-NC 4.0.
94
  ## Cite
95
 
96
  ```
97
- @misc{chirkova2024provence,
98
  title={Provence: efficient and robust context pruning for retrieval-augmented generation},
99
  author={Nadezhda Chirkova and Thibault Formal and Vassilina Nikoulina and Stéphane Clinchant},
100
- year={2024},
101
- eprint={?},
102
  archivePrefix={arXiv},
103
  primaryClass={cs.CL},
104
- copyright = {Creative Commons Attribution Non Commercial Share Alike 4.0 International}
105
  }
106
  ```
107
 
 
14
 
15
  *Developed by*: Naver Labs Europe
16
  *License*: [CC BY-NC 4.0.](https://creativecommons.org/licenses/by-nc/4.0/)
17
+ *Paper*: https://arxiv.org/abs/2501.16214, accepted to ICLR 2025
18
  * *Model*: `provence-reranker-debertav3-v1` (Provence for Pruning and Reranking Of retrieVEd relevaNt ContExt)
19
  * *Backbone model*: [DeBERTav3-reranker](https://huggingface.co/naver/trecdl22-crossencoder-debertav3) (trained from [DeBERTa-v3-large](https://huggingface.co/microsoft/deberta-v3-large))
20
  * *Model size*: 430 million parameters
21
+ * *Context length*: 512 tokens
22
 
23
  ## Usage
24
 
 
48
  # Provence Output: {'reranking_score': 3.022725, pruned_context': 'Shepherd’s pie. In early cookery books, the dish was a means of using leftover roasted meat of any kind, and the pie dish was lined on the sides and bottom with mashed potato, as well as having a mashed potato crust on top.']]
49
  ```
50
 
 
 
51
  ## Model interface
52
 
53
  Interface of the `process` function:
 
67
  * **Provence automatically detects the number of sentences to keep**, based on a threshold. We found that the default value of a threshold works well across various domains, but the threshold can be adjusted further to better meet the particular use case needs.
68
  * **Provence is robust to various domains**, being trained on a combination of diverse MS Marco and Natural Questions data.
69
  * **Provence works out-of-the-box with any LLM**.
 
70
 
 
71
 
72
  ## Model Details
73
 
 
79
  * Context length: 512 tokens (similar to the pretrained DeBERTa model)
80
  * Evaluation: we evaluate Provence on 7 datasets from various domains: Wikipedia, biomedical data, course syllabi, and news. We find that Provence is able to prune irrelevant sentences with little-to-no drop in performance, in all domains, and outperforms existing baselines on the Pareto front (top right corners of the plots).
81
 
82
+ Check out more analysis in the [paper](https://arxiv.org/abs/2501.16214)!
83
 
84
  <img src="https://cdn-uploads.huggingface.co/production/uploads/6273df31c3b822dad2d1eef2/WMmfsNG48O830paaBAaQF.png" width="600">
85
 
 
91
  ## Cite
92
 
93
  ```
94
+ @misc{chirkova2025provenceefficientrobustcontext,
95
  title={Provence: efficient and robust context pruning for retrieval-augmented generation},
96
  author={Nadezhda Chirkova and Thibault Formal and Vassilina Nikoulina and Stéphane Clinchant},
97
+ year={2025},
98
+ eprint={2501.16214},
99
  archivePrefix={arXiv},
100
  primaryClass={cs.CL},
101
+ url={https://arxiv.org/abs/2501.16214},
102
  }
103
  ```
104