| **Chatrag-Deberta** is a small lightweight LLM to predict whether a question should retrieve additional information with RAG or not. | |
| Chatrag-Deberta is based on Deberta-v3-large, a 304M encoder-decoder. Its initial version was fine-tuned on 20,000 examples of questions annotated by Mistral 7B. | |
| ## Use | |
| A typical example of inference with Chatrag-Deberta is provided in the [Google Colab demo](https://colab.research.google.com/drive/1nTLFJXopFOEJldaCPzjQ2g5-j0NnpLdz?usp=sharing) or with inference_chatrag.py | |
| For every submitted text, Chatrag-Deberta will output a range of probabilities to require RAG or not. | |
| This makes it possible to adjust a threshold of activation depending on whether more or less RAG is desirable in the system. | |
| | Query | Prob | Result | | |
| |----------------------------------------------------------|:---------:|--------:| | |
| | Comment puis-je renouveler un passeport ? | 0.988455 | RAG | | |
| | Combien font deux et deux ? | 0.041475 | No-RAG | | |
| | Écris un début de lettre de recommandation pour la Dinum | 0.103086 | No-RAG | | |