|
--- |
|
library_name: transformers |
|
tags: |
|
- trismegistus |
|
- llama3 |
|
- esoteric |
|
license: llama3.2 |
|
datasets: |
|
- teknium/trismegistus-project |
|
base_model: |
|
- meta-llama/Llama-3.2-1B |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
Trismegistus for Llama 3.2 1b. Credits to teknium for dataset and original model. |
|
|
|
### Model Sources [optional] |
|
|
|
Llama 3.2 1b |
|
|
|
## Uses |
|
|
|
Use for esoteric joy. |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
May be biased as hell. |
|
|
|
- Recommendation: |
|
- Don't take it personally. |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
Run it. |
|
|
|
|
|
### Training Data |
|
|
|
#### Training Hyperparameters |
|
|
|
- lora 4bit peft |
|
|
|
#### Speeds, Sizes, Times [optional] |
|
|
|
- global_step=16905 |
|
- training_loss=1.169401215731269 |
|
- train_runtime: 21882.4747 |
|
- train_samples_per_second: 3.09 |
|
- train_steps_per_second: 0.773 |
|
- total_flos: 4.437195883099177e+17 |
|
- train_loss': 1.169401215731269 |
|
- epoch: 5.0 |
|
|
|
## Evaluation and Metrics |
|
|
|
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr| |
|
|-------------|------:|------|-----:|--------|---|-----:|---|-----:| |
|
|arc_challenge| 1|none | 0|acc |↑ |0.3345|± |0.0138| |
|
| | |none | 0|acc_norm|↑ |0.3695|± |0.0141| |
|
|arc_easy | 1|none | 0|acc |↑ |0.6044|± |0.0100| |
|
| | |none | 0|acc_norm|↑ |0.5694|± |0.0102| |
|
|boolq | 2|none | 0|acc |↑ |0.6410|± |0.0084| |
|
|hellaswag | 1|none | 0|acc |↑ |0.4400|± |0.0050| |
|
| | |none | 0|acc_norm|↑ |0.5728|± |0.0049| |
|
|openbookqa | 1|none | 0|acc |↑ |0.2260|± |0.0187| |
|
| | |none | 0|acc_norm|↑ |0.3540|± |0.0214| |
|
|piqa | 1|none | 0|acc |↑ |0.7002|± |0.0107| |
|
| | |none | 0|acc_norm|↑ |0.7024|± |0.0107| |
|
|winogrande | 1|none | 0|acc |↑ |0.5785|± |0.0139| |
|
|
|
|
|
## Environmental Impact |
|
|
|
Will steal your horse and kill your cat. |
