|
--- |
|
license: mit |
|
language: |
|
- fr |
|
library_name: transformers |
|
tags: |
|
- linformer |
|
- legal |
|
- medical |
|
- RoBERTa |
|
- pytorch |
|
--- |
|
|
|
# Jargon-general-base |
|
|
|
[Jargon](https://hal.science/hal-04535557/file/FB2_domaines_specialises_LREC_COLING24.pdf) is an efficient transformer encoder LM for French, combining the LinFormer attention mechanism with the RoBERTa model architecture. |
|
|
|
Jargon is available in several versions with different context sizes and types of pre-training corpora. |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
<!-- This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1). |
|
--> |
|
|
|
|
|
|
|
## Using Jargon models with HuggingFace transformers |
|
|
|
You can get started with `jargon-general-base` using the code snippet below: |
|
|
|
```python |
|
from transformers import AutoModelForMaskedLM, AutoTokenizer, pipeline |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("PantagrueLLM/jargon-general-base", trust_remote_code=True) |
|
model = AutoModelForMaskedLM.from_pretrained("PantagrueLLM/jargon-general-base", trust_remote_code=True) |
|
|
|
jargon_maskfiller = pipeline("fill-mask", model=model, tokenizer=tokenizer) |
|
output = jargon_maskfiller("Il est allé au <mask> hier") |
|
``` |
|
|
|
- **Funded by** |
|
- GENCI-IDRIS (Grant 2022 A0131013801) |
|
- French National Research Agency: Pantagruel grant ANR-23-IAS1-0001 |
|
- MIAI@Grenoble Alpes ANR-19-P3IA-0003 |
|
- PROPICTO ANR-20-CE93-0005 |
|
- Lawbot ANR-20-CE38-0013 |
|
- Swiss National Science Foundation (grant PROPICTO N°197864) |
|
<!-- - **Shared by [optional]:** [More Information Needed] --> |
|
<!-- - **Model type:** [More Information Needed] --> |
|
- **Language(s):** French |
|
- **License:** MIT |
|
- **Developed by:** Vincent Segonne |
|
<!-- - **Finetuned from model [optional]:** [More Information Needed] --> |
|
<!-- |
|
### Model Sources [optional] |
|
|
|
<!-- Provide the basic links for the model. --> |
