Model Card for Model ID
Model Details
Model Description
- Language(s) (NLP): Khmer
- Finetuned from model: Qwen/Qwen2-1.5B-Instruct
Usage Notebook on Kaggle
https://www.kaggle.com/code/mlao01/km-typing-assist
Run Validation Set Notebook on Kaggle
https://www.kaggle.com/code/mlao01/benchmark
Bias, Risks, and Limitations
Be cautious, we did not proof read the dataset used to tune this model.
Training Data
mlao01/km-news
Training Procedure
QLoRa for 20% of the training data:
- Rank = 128
- Learing rate = 2e-4
- Warmup = 25% of the training data
- Gradient Clipping = 5.0
Number of Parameters
trainable params: 147,718,144 || all params: 1,691,432,448 || trainable%: 8.7333
Evaluation
Last logged training loos reached ~0.5 cross entropy
- Downloads last month
- 7
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.