Finetuning

README.md

@@ -1,27 +1,6 @@
 ---
 library_name: peft
-license: cc-by-nc-2.0
-language:
-- fr
-- en
-tags:
-- pytorch
-- llama
-- code
 ---
-
-## Aria 7B V3
-
-We decided to build a V3 of Aria 7B based on Mistral instruct instead of LLAMA 2. The base model has been quantized with Qlora to reduce the model size and trained on a high quality french dataset.
-
-## Base Model : Mistral-7B-Instruct-v0.1
-
-## Technical issues Fixed & Limits of base model
-
-We noticed that the base model had a common issue of mixing french and english when the request was done in french in some cases,not all of them. This issue was more visible for
-prompts over 1000 tokens. By training the base model on our dataset, we fixed this issue and allow the model to reply in the same specific language used for the question to answer.
-This pain-point is a valuable upgrade for corporate users in non-english areas willing to deploy a model with an increased quality and accuracy in french language.
-
 ## Training procedure
 
 
@@ -39,4 +18,4 @@ The following `bitsandbytes` quantization config was used during training:
 ### Framework versions
 
 
-- PEFT 0.5.0
+- PEFT 0.6.0.dev0

adapter_config.json

@@ -1,4 +1,5 @@
 {
+  "alpha_pattern": {},
   "auto_mapping": null,
   "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.1",
   "bias": "none",
@@ -12,6 +13,7 @@
   "modules_to_save": null,
   "peft_type": "LORA",
   "r": 16,
+  "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "q_proj",

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3dba267b1eaaf009b0c3341630d3a7b28fd734cc30b87ec45d6a72f09421f62f
+oid sha256:4d39ab534317fcef11f07327bbd422a5d23e196697a85bccfdb51abd000b5070
 size 27308941