Instructions to use dedyirama/qwen2.5-1.5b-indonesian-qlora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use dedyirama/qwen2.5-1.5b-indonesian-qlora with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="dedyirama/qwen2.5-1.5b-indonesian-qlora") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("dedyirama/qwen2.5-1.5b-indonesian-qlora") model = AutoModelForCausalLM.from_pretrained("dedyirama/qwen2.5-1.5b-indonesian-qlora", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use dedyirama/qwen2.5-1.5b-indonesian-qlora with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dedyirama/qwen2.5-1.5b-indonesian-qlora" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dedyirama/qwen2.5-1.5b-indonesian-qlora", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/dedyirama/qwen2.5-1.5b-indonesian-qlora
- SGLang
How to use dedyirama/qwen2.5-1.5b-indonesian-qlora with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "dedyirama/qwen2.5-1.5b-indonesian-qlora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dedyirama/qwen2.5-1.5b-indonesian-qlora", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "dedyirama/qwen2.5-1.5b-indonesian-qlora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dedyirama/qwen2.5-1.5b-indonesian-qlora", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Unsloth Desktop
- Docker Model Runner
How to use dedyirama/qwen2.5-1.5b-indonesian-qlora with Docker Model Runner:
docker model run hf.co/dedyirama/qwen2.5-1.5b-indonesian-qlora
Qwen2.5-1.5B-Instruct, QLoRA fine-tuned for Indonesian instructions
This is Qwen2.5-1.5B-Instruct fine-tuned with QLoRA on about 47k Indonesian instruction/response pairs. It is the generator model in indonesian-labor-law-rag, a retrieval-augmented QA system over Indonesian labor regulations.
The model was fine-tuned for general Indonesian instruction following. It was not trained on legal text: legal knowledge in the RAG system comes from retrieved documents, not from the model weights.
Model details
| Base model | unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit, a 4-bit build of Qwen/Qwen2.5-1.5B-Instruct |
| Method | QLoRA (4-bit NF4 with double quantization); LoRA r=16, alpha=16, dropout 0, on q/k/v/o_proj and gate/up/down_proj |
| Weights | LoRA adapter merged into the base model and saved in float16 (~3.1 GB) |
| Chat format | ChatML (<|im_start|> / <|im_end|>), with no default system prompt |
| Language | Indonesian |
| License | Apache 2.0, inherited from the base model |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "dedyirama/qwen2.5-1.5b-indonesian-qlora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Jelaskan secara singkat apa itu energi terbarukan."}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=256, repetition_penalty=1.1)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
The model also loads with Unsloth's FastLanguageModel.from_pretrained(..., load_in_4bit=True), which is how the RAG notebook runs it on a T4 GPU.
Training
| Data | Ichsan2895/alpaca-gpt4-indonesian: 49,969 single-turn pairs reformatted from FreedomIntelligence/alpaca-gpt4-indonesian (CC BY-SA 4.0), split 95/5 into train (47,470) and validation (2,499) with seed 3407 |
| Formatting | Each pair mapped to a ChatML user/assistant conversation |
| Trainer | TRL SFTTrainer via Unsloth, no sequence packing, max sequence length 2,048 |
| Schedule | 800 steps (~0.13 epoch), effective batch size 8 (2 ร 4 gradient accumulation), learning rate 2e-4, linear decay, 50 warmup steps |
| Optimizer | adamw_8bit, weight decay 0.01, fp16 |
| Hardware | One NVIDIA Tesla T4 (Google Colab) |
Hyperparameter comparison
Two configurations were trained with the same schedule, and the one with the lower validation loss was published:
| Run | LoRA r / alpha | Learning rate | Trainable params | Final validation loss |
|---|---|---|---|---|
| This model | 16 / 16 | 2e-4 | 18.5M (1.18%) | 1.3726 |
| Alternative | 32 / 32 | 1e-4 | 36.9M (2.34%) | 1.3784 |
Each configuration was trained only once, and rank and learning rate were changed together, so the small gap does not show that either setting is better. Validation loss is the only metric reported: the model was not evaluated on downstream benchmarks or compared against the base model.
Intended use and limitations
- Intended use: learning and experimentation with Indonesian instruction following, and as a small generator inside a RAG pipeline that supplies the facts.
- Not for factual or legal advice on its own. At 1.5B parameters the model can hallucinate, and it sometimes repeats itself. Use retrieval and a repetition penalty.
- Limited training. The model saw only ~13% of one epoch of the dataset, so the changes from the base model are modest.
- Quantization error. The adapter was trained against 4-bit base weights and merged into a dequantized copy of them, so the float16 weights differ slightly from the original
Qwen/Qwen2.5-1.5B-Instruct. - Inherited biases. The training data is an Indonesian version of Alpaca-GPT4, whose responses were generated by GPT-4. The model may inherit artifacts and biases from GPT-4, from the dataset's conversion to Indonesian, and from the base model.
Links
- Code, notebooks, and RAG pipeline: github.com/dedyirama-id/indonesian-labor-law-rag
- Built as a final project for Dicoding Academy's Pengembangan Generative AI berbasis LLM course.
- Trained with Unsloth and TRL.
- Downloads last month
- 94
Model tree for dedyirama/qwen2.5-1.5b-indonesian-qlora
Base model
Qwen/Qwen2.5-1.5B