File size: 1,014 Bytes
bee637e d880245 e28a7b2 d880245 e28a7b2 d880245 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
---
license: llama2
---
<h1>Kaori-34b-v2 Model Card</h1>
This Model was Finetuned By Kaeri and Jenti.
<h3>Datasets</h3>
- Open-Platypus
- Dolphin
- OpenOrca
We trained the model with <b>100%</b> Open-Platypus data, <b>5%</b> Dolphin data and <b>10%</b> OpenOrca data and applied SFT strategy.
We did not use GSM8k samples when generating data.
Also we were careful of data contamination by similarity filtering
the training data if the data correspond to any of the following list.
<pre>
filtering_tasks = [
'cot_gsm8k',
'cot_gsm8k_ii',
'drop:2.0.0',
'winogrande:1.1.0'
'task228_arc_answer_generation_easy',
'ai2_arc/ARC-Challenge:1.0.0',
'ai2_arc/ARC-Easy:1.0.0',
'task229_arc_answer_generation_hard',
'hellaswag:1.1.0',
'task1389_hellaswag_completion'
]
</pre>
<h3>Framework:</h3>
- https://github.com/hiyouga/LLaMA-Factory
<h3>Parameters:</h3>
- Finetune_Type : LoRA
- GPUs : A100x4(80GB)
- Epochs : 3
- Batchsize : 8
|