|
--- |
|
library_name: transformers |
|
license: llama2 |
|
--- |
|
|
|
# Llama-3-Smaug-8B |
|
|
|
### Built with Meta Llama 3 |
|
|
|
|
|
 |
|
|
|
This model was built using the Smaug recipe for improving performance on real world multi-turn conversations applied to |
|
[meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B). |
|
|
|
### Model Description |
|
|
|
- **Developed by:** [Abacus.AI](https://abacus.ai) |
|
- **License:** https://llama.meta.com/llama3/license/ |
|
- **Finetuned from model:** [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B). |
|
|
|
|
|
## Evaluation |
|
|
|
``` |
|
########## First turn ########## |
|
score |
|
model turn |
|
llama3-8b-smaug-2-merged-600 1 8.79375 |
|
llama3-8b-smaug-2-merged-150 1 8.71250 |
|
llama3-8b-smaug-2-merged-300 1 8.66250 |
|
base_Meta-Llama-3-8B-Instruct 1 8.53125 |
|
llama3-8b-smaug-2-merged-450 1 8.42500 |
|
########## Second turn ########## |
|
score |
|
model turn |
|
llama3-8b-smaug-2-merged-450 2 7.8125 |
|
llama3-8b-smaug-2-merged-300 2 7.7375 |
|
llama3-8b-smaug-2-merged-600 2 7.7250 |
|
llama3-8b-smaug-2-merged-150 2 7.7125 |
|
base_Meta-Llama-3-8B-Instruct 2 7.5500 |
|
########## Average ########## |
|
score |
|
model |
|
llama3-8b-smaug-2-merged-600 8.259375 |
|
llama3-8b-smaug-2-merged-150 8.212500 |
|
llama3-8b-smaug-2-merged-300 8.200000 |
|
llama3-8b-smaug-2-merged-450 8.118750 |
|
base_Meta-Llama-3-8B-Instruct 8.040625 |
|
``` |
|
|
|
| Model | First turn | Second Turn | Average | |
|
| :---- | ---------: | ----------: | ------: | |
|
| llama3-8b-smaug-2-merged-600 | **8.79** | 7.73 | **8.26** | |
|
| llama3-8b-smaug-2-merged-450 | 8.43 | **7.81** | 8.12 | |
|
| llama3-8b-smaug-2-merged-300 | 8.66 | 7.74 | 8.20 | |
|
| llama3-8b-smaug-2-merged-150 | 8.71 | 7.71 | 8.21 | |
|
| Meta-Llama-3-8B-Instruct | 8.53 | 7.55 | 8.04 | |
|
|
