metadata
base_model:
- GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
license: apache-2.0
language:
- id
datasets:
- psetialana/multi_session_chat-informal_indonesian-transformed
Personalized Sahabat AI Llama 3.1 8 B
- Developed by: Pradana Setialana
This model is a fine-tuned version of GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct on psetialana/multi_session_chat-informal_indonesian-transformed dataset.
Model description
This model can be used to personalize conversations and role-play based on the persona given with the prompt
Kamu adalah sahabat user. Kamu memiliki karakter PERSONA_ASSISTANT. User memiliki karakter PERSONA_USER. Kamu berperilaku sesuai PERSONA_ASSISTANT dan menyesuaikan responmu sesuai PERSONA_USER.
PERSONA_ASSISTANT:
{assistant_persona}
PERSONA_USER:
{user_persona}
Training procedure
LoRA config
The following lora config were used during training:
- alpha: 16
- r: 16
- droput: 0
- modules: "q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-4
- optimizer: adamw_8bit