Self-Rewarding Mistral-7B-v0.3 JA
Collection
Mistral-7B-v0.3をloraでself-rewardingしたモデル
•
4 items
•
Updated
Mistral-7B-v0.3-m3-lora
- HachiML/Mistral-7B-v0.3-dpo-lora_sr_m3_lr1e-5_3epのAdapterをマージしたモデル
- This model is a fine-tuned version of HachiML/Mistral-7B-v0.3-m2-lora on following datasets.
- HachiML/self-rewarding_AIFT_MSv0.3_lora(split=AIFT_M2)
Model Details
Model Description
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- Developed by: HachiML
- Model type: Mistral-7B
- Language(s) (NLP): Japanese
- License: Apache-2.0
- Finetuned from model: HachiML/Mistral-7B-v0.3-m2-lora
- Finetuned type: DPO
- Finetuned dataset: HachiML/self-rewarding_AIFT_MSv0.3_lora(split=AIFT_M2)
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 3
Training results
Framework versions
- PEFT 0.11.1
- Transformers 4.41.0
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
- Downloads last month
- 5
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.