|
--- |
|
library_name: peft |
|
license: llama2 |
|
datasets: |
|
- vicgalle/alpaca-gpt4 |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
tags: |
|
- llama-2 |
|
- llama |
|
- instruct |
|
- instruction |
|
--- |
|
<a href="https://www.buymeacoffee.com/PulsarAI" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a> |
|
|
|
|
|
# Info |
|
|
|
This model is an adapter model trained with [**QloRA**](https://arxiv.org/abs/2305.14314) technique. |
|
|
|
* 📜 Model license: [Llama 2 Community License Agreement](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) |
|
* 🏛️ Base Model: [Llama-2-70b-hf](https://huggingface.co/meta-llama/Llama-2-70b-hf) |
|
* 🖥️ Machine: Nvidia A100 (40 GB vRAM) |
|
* 💵 Cost: $3.5 |
|
* ⌛ Training Time: 3 hour 22 minutes |
|
* 📊 Dataset Used: [vicgalle/alpaca-gpt4](https://huggingface.co/datasets/vicgalle/alpaca-gpt4) |
|
|
|
You can acces Llama-2 paper by clicking [here](https://arxiv.org/abs/2307.09288) |
|
|
|
# Evaluation Results ([Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)) |
|
|
|
| | Average | ARC (25-shot) | HellaSwag (10-shot) | MMLU (5-shot) | TruthfulQA (0-shot) | |
|
|---------|---------|---------------|---------------------|---------------|--------------------| |
|
| Scores | 67.3 | 66.38 | 84.51 | 62.75 | 55.57 | |
|
|
|
|
|
# Loss Graph |
|
|
|
 |
|
|
|
|
