SinanAkkoyun
commited on
Commit
•
bb6165f
1
Parent(s):
fdea479
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,62 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
+
base_model: mistralai/Mistral-7B-v0.1
|
4 |
+
datasets:
|
5 |
+
- Open-Orca/SlimOrca
|
6 |
+
- HuggingFaceH4/no_robots
|
7 |
+
- Intel/orca_dpo_pairs
|
8 |
+
- rizerphe/glaive-function-calling-v2-zephyr
|
9 |
+
- codefuse-ai/Evol-instruction-66k
|
10 |
+
library_name: transformers
|
11 |
+
pipeline_tag: text-generation
|
12 |
---
|
13 |
+
|
14 |
+
AWQ GEMM quant of `TokenBender/pic_7B_mistral_Full_v0.2`
|
15 |
+
|
16 |
+
# pic_7B_mistral_Full_v0.2
|
17 |
+
PIC_7B_Mistral (First phase)
|
18 |
+
|
19 |
+
This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)
|
20 |
+
A curated, decontaminated subset of datasets used have been mentioned in the model card.
|
21 |
+
All used datasets are public as of the time of release of this model.
|
22 |
+
|
23 |
+
Collaborate or Consult me - [Twitter](https://twitter.com/4evaBehindSOTA), [Discord](https://discord.gg/ftEM63pzs2)
|
24 |
+
|
25 |
+
*Recommended format is ChatML, Alpaca will work but take care of EOT token*
|
26 |
+
|
27 |
+
#### Chat Model Inference
|
28 |
+
|
29 |
+
|
30 |
+
## Model description
|
31 |
+
|
32 |
+
First generic model of Project PIC (Partner-in-Crime) in 7B range.
|
33 |
+
Trying a bunch of things and seeing what sticks right now.
|
34 |
+
|
35 |
+
Empathy + Coder + Instruction/json/function adherence is my game.
|
36 |
+
Finding lots of challenges and insights in this effort, patience is key.
|
37 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6398bf222da24ee95b51c8d8/Fl-pRCsC_lvnuoP734hsJ.png)
|
38 |
+
|
39 |
+
## Intended uses & limitations
|
40 |
+
|
41 |
+
Should be useful in generic capacity.
|
42 |
+
Demonstrates little bit of everything.
|
43 |
+
|
44 |
+
Basic tests in -
|
45 |
+
Roleplay: Adherence to character present.
|
46 |
+
json/function-calling: Passing
|
47 |
+
Coding: To be evaluated
|
48 |
+
|
49 |
+
|
50 |
+
## Training procedure
|
51 |
+
SFT + DPO
|
52 |
+
|
53 |
+
### Training results
|
54 |
+
Humaneval and evalplus results to be shared as well.
|
55 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6398bf222da24ee95b51c8d8/Ysat2Hi3MInSKSlUrft-i.png)
|
56 |
+
|
57 |
+
### Framework versions
|
58 |
+
|
59 |
+
- Transformers 4.35.2
|
60 |
+
- Pytorch 2.0.1
|
61 |
+
- Datasets 2.15.0
|
62 |
+
- Tokenizers 0.15.0
|