YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Quantization made by Richard Erkhov.
ShortKing-1.4b-v0.1 - bnb 8bits
- Model creator: https://huggingface.co/AtAndDev/
- Original model: https://huggingface.co/AtAndDev/ShortKing-1.4b-v0.1/
Original model description:
license: cc-by-nc-4.0 datasets: - vicgalle/alpaca-gpt4 language: - en
Model Overview
Model license: cc-by-nc-4.0
This model is trained based on EleutherAI/pythia-1.4b-deduped model that is LoRA finetuned on vicgalle/alpaca-gpt4 dataset.
Prompt Template: Alpaca
<system_prompt>
### Instruction:
<user_message>
### Response:
<assistant_response>
Intended Use
THIS IS A TEST MODEL, IT IS NOT INTENDED FOR REAL APPLICATIONS BY ANY MEANS. HOWEVER, A NEW MODEL IS COMING IN THE SAME TOPIC.
This model series will be used for small but intense applications.
Training Details
This model took 2:31:23 to train in QLoRA on a single T4 GPU.
- epochs:
1 - train batch size:
12 - eval batch size:
12 - gradient accumulation steps:
1 - maximum gradient normal:
0.3 - learning rate:
2e-4 - weight decay:
0.001 - optimizer:
paged_adamw_32bit - learning rate schedule:
cosine - warmup ratio (linear):
0.03
- Downloads last month
- 12
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support