---
tags:
- Distillation
---
the model(student) after finetuning outperforms [410m-deduped](https://huggingface.co/EleutherAI/pythia-410m-deduped) model on wsc accuracy.

lm-eval : aloobun/distill-p-test
![image/png](https://huggingface.co/aloobun/distill-p-test/blob/main/Screenshot%20from%202023-12-07%2019-29-32.png)

lm-eval : EleutherAI/pythia-410m-deduped
![image/png](https://huggingface.co/aloobun/distill-p-test/blob/main/Screenshot%20from%202023-12-07%2019-48-36.png)