MohamedRashad
commited on
Commit
•
50d8e5c
1
Parent(s):
004978d
Update README.md
Browse files
README.md
CHANGED
@@ -8,20 +8,25 @@ language:
|
|
8 |
pipeline_tag: text-generation
|
9 |
---
|
10 |
|
11 |
-
# Arabic ORPO LLAMA 3
|
12 |
<center>
|
13 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/6116d0584ef9fdfbf45dc4d9/3ns3O_bWYxKEXmozA073h.png">
|
14 |
</center>
|
15 |
|
16 |
|
17 |
-
## Story first
|
18 |
|
19 |
This model is the a finetuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) using [ORPO](https://github.com/xfactlab/orpo) on [2A2I/argilla-dpo-mix-7k-arabic](https://huggingface.co/datasets/2A2I/argilla-dpo-mix-7k-arabic).
|
20 |
|
21 |
-
I wanted to try ORPO and see if it will better align a biased English model like llama3 to the arabic language or
|
22 |
|
|
|
23 |
|
24 |
-
|
|
|
|
|
|
|
|
|
25 |
|
26 |
| Community | Llama-3-8B-Instruct | Arabic-ORPO-Llama-3-8B-Instrcut |
|
27 |
|----------------------------------|---------------------|----------------------------------|
|
|
|
8 |
pipeline_tag: text-generation
|
9 |
---
|
10 |
|
11 |
+
# 👳 Arabic ORPO LLAMA 3
|
12 |
<center>
|
13 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/6116d0584ef9fdfbf45dc4d9/3ns3O_bWYxKEXmozA073h.png">
|
14 |
</center>
|
15 |
|
16 |
|
17 |
+
## 👓 Story first
|
18 |
|
19 |
This model is the a finetuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) using [ORPO](https://github.com/xfactlab/orpo) on [2A2I/argilla-dpo-mix-7k-arabic](https://huggingface.co/datasets/2A2I/argilla-dpo-mix-7k-arabic).
|
20 |
|
21 |
+
I wanted to try ORPO and see if it will better align a biased English model like **llama3** to the arabic language or it will faill.
|
22 |
|
23 |
+
While the evaluations favour the base llama3 over my finetune, in practice i found my finetune was much better at spitting coherent (mostly correct) arabic text which i find interesting.
|
24 |
|
25 |
+
I would encourage everyone to try out the model from [here](https://huggingface.co/spaces/MohamedRashad/Arabic-Chatbot-Arena) and share his insights with me ^^
|
26 |
+
|
27 |
+
## 🤔 Evaluation and Results
|
28 |
+
|
29 |
+
This result was made using [lighteval](https://github.com/huggingface/lighteval) using the `community|arabic_mmlu` tasks.
|
30 |
|
31 |
| Community | Llama-3-8B-Instruct | Arabic-ORPO-Llama-3-8B-Instrcut |
|
32 |
|----------------------------------|---------------------|----------------------------------|
|