Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ pipeline_tag: text-generation
|
|
12 |
---
|
13 |
|
14 |
|
15 |
-
# MiniLLM
|
16 |
|
17 |
[paper](https://arxiv.org/abs/2306.08543) | [code](https://github.com/microsoft/LMOps/tree/main/minillm)
|
18 |
|
@@ -22,7 +22,7 @@ pipeline_tag: text-generation
|
|
22 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/624ac662102fcdff87be51b9/7hBWGZzYMJihCRQ70XoiQ.png" width="1000">
|
23 |
</p>
|
24 |
|
25 |
-
**Note**: MiniLLM requires a [SFT model]() for initilization to perform the PPO optimization.
|
26 |
|
27 |
## Evaluation
|
28 |
|
@@ -33,9 +33,9 @@ We ask GPT-4 to give scores for the generated responses of MiniLLM. The prompts
|
|
33 |
</p>
|
34 |
|
35 |
## Baseline Models
|
36 |
-
+ [SFT w/o KD]()
|
37 |
-
+ [KD]()
|
38 |
-
+ [SeqKD]()
|
39 |
|
40 |
## Citation
|
41 |
```
|
|
|
12 |
---
|
13 |
|
14 |
|
15 |
+
# MiniLLM-gpt2-760M
|
16 |
|
17 |
[paper](https://arxiv.org/abs/2306.08543) | [code](https://github.com/microsoft/LMOps/tree/main/minillm)
|
18 |
|
|
|
22 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/624ac662102fcdff87be51b9/7hBWGZzYMJihCRQ70XoiQ.png" width="1000">
|
23 |
</p>
|
24 |
|
25 |
+
**Note**: MiniLLM requires a [SFT model](https://huggingface.co/MiniLLM/init-gpt2-760M) for initilization to perform the PPO optimization.
|
26 |
|
27 |
## Evaluation
|
28 |
|
|
|
33 |
</p>
|
34 |
|
35 |
## Baseline Models
|
36 |
+
+ [SFT w/o KD](https://huggingface.co/MiniLLM/SFT-gpt2-760M)
|
37 |
+
+ [KD](https://huggingface.co/MiniLLM/KD-gpt2-760M)
|
38 |
+
+ [SeqKD](https://huggingface.co/MiniLLM/SeqKD-gpt2-760M)
|
39 |
|
40 |
## Citation
|
41 |
```
|