Update README.md

README.md

@@ -20,13 +20,11 @@ LLaMandement-7B is a French chat LLM, based on [LLaMA-2-7B](https://ai.meta.com/
 ## Model Details
 
 - **Developed by:** [DGFIP](https://www.impots.gouv.fr/presentation-de-la-dgfip-overview-dgfip) : 
-   - yannis.tannier@dgfip.finances.gouv.fr 
-   - joseph.gesnouin@dgfip.finances.gouv.fr 
 - **Model type:** An auto-regressive language model based on the transformer architecture
 - **License:** Llama 2 Community License Agreement
 - **Finetuned from model:** [Llama 2](https://arxiv.org/abs/2307.09288)
 - **Repository:** https://gitlab.adullact.net/dgfip/projets-ia/llamandement
-- **Paper:** working
+- **Paper:** [Technical Report](https://arxiv.org/abs/2401.16182)
 
 ## Prompt Template
 
@@ -47,28 +45,13 @@ Below is an instruction that describes a task. Write a response that appropriate
 - Command line interface: https://github.com/lm-sys/FastChat
 - APIs (OpenAI API, Huggingface API): https://github.com/lm-sys/FastChat/tree/main#api  
 
-## Training Details
-
-Llamandement-7B is fine-tuned from Llama 2 using Low-Rank Adaptation (LORA). This method is efficient and adds minimal computational load. It introduces additional low-rank parameters, enabling the model to better handle complex legislative language without major changes to the original structure.
-
-**LORA Settings Adjustments:**
-
-- **Learning Rate (LR):** Set to a low value of 2e-5 to ensure stable and gradual improvements.
-- **Adaptation Depth (lora_r):** Set at 64, influencing the dimension of the low-rank matrix in LORA. This affected about 0.40% of the model's weights.
-- **Decay Rate:** Employed at 0.01 to prevent overfitting to specific legislative text structures.
-- **LORA Alpha (α):** Set at 16, it fine-tunes the model's response to legislative text.
-- **LORA Dropout:** A rate of 0.1 applied to LORA layers to prevent overfitting and enhance generalization.
-- **Optimizer and Scheduler:** Utilized a cosine learning rate scheduler with a warmup ratio of 0.03 for optimal training.
-
-
-For more information, visit [dgfip.finance.com](http://dgfip.finance.com). Additional details about the training dataset composition can be found [here](http://dgfip.finance.com/training-dataset-info).
-
-
 
 ## Citation
-
-Please cite the repo if you use the data, method or code in this repo.
-
-[...]
-
-
+```
+@article{gesnouin2024llamandement,
+  title={LLaMandement: Large Language Models for Summarization of French Legislative Proposals},
+  author={Gesnouin, Joseph and Tannier, Yannis and Da Silva, Christophe Gomes and Tapory, Hatim and Brier, Camille and Simon, Hugo and Rozenberg, Raphael and Woehrel, Hermann and Yakaabi, Mehdi El and Binder, Thomas and others},
+  journal={arXiv preprint arXiv:2401.16182},
+  year={2024}
+}
+```