🏛️ VILaw-LLM-DPO-v2 — Vietnamese Legal Reasoning LLM (Continual Training & Alignment)

VILaw-LLM-DPO-v2 is an advanced, domain-specialized Large Language Model tailored for Vietnamese Legal Advisory, Statutory Citation (Điều, Khoản, Luật), and Multi-step Legal Reasoning.

Version v2 was developed via a Continual Fine-Tuning pipeline initialized directly from the DPO-aligned checkpoint of vilaw-dpo-lora:

1. Preserved DPO Alignment: Retains the anti-hallucination capabilities achieved through Direct Preference Optimization, suppressing speculative and subjective conjecture while strictly enforcing statutory citations.
2. Expanded Statutory & Conversational Knowledge: Integrates 9,674 high-quality, deduplicated, and Unicode NFC normalized instruction pairs from duyet/vietnamese-legal-instruct combined with natural conversational data from hoanghai2110/vietnamese-dataset.
3. Structured 3-Tier Output Discipline: Generates responses following a rigorous tripartite architecture: (1) Statutory Legal Grounds $\rightarrow$ (2) Deductive Legal Reasoning applied to the context $\rightarrow$ (3) Actionable Practical Recommendations.

---

📊 Benchmark Evaluation (LLM-as-Judge)

The model was evaluated against authoritative statutory ground truth across 25 comprehensive legal benchmark test cases (eval/legal_benchmark_200.json) spanning 17 Vietnamese legal branches on a 10-point scale:

1. Comparative Benchmark Overview (v1 vs. v2)

Evaluation Metric	GPT-4o-mini (Baseline)	VILaw-LLM (SFT)	VILaw-LLM (DPO v1)	VILaw-LLM-DPO-v2 🏆
Statutory Citation Quality (0-4)	2.50 / 4.0	3.20 / 4.0	3.80 / 4.0	3.90 / 4.0
Legal Reasoning Quality (0-3)	2.10 / 3.0	2.50 / 3.0	2.80 / 3.0	2.54 / 3.0
Practical Actionable Advice (0-3)	1.60 / 3.0	2.20 / 3.0	2.50 / 3.0	2.82 / 3.0
Overall Legal Score (0-10)	6.20 / 10	7.90 / 10	9.10 / 10	9.26 / 10
Benchmark Suite Size	5 test cases	5 test cases	5 test cases	25 comprehensive cases

2. Detailed Performance by Legal Domain

Legal Branch	Governing Statute	Average Score (out of 10)	Rating
Bidding & Procurement Law	Law on Bidding 2023	10.0 / 10	Outstanding
Land Law	Land Law 2024 (Latest)	10.0 / 10	Outstanding
Investment Law	Law on Investment 2020	10.0 / 10	Outstanding
Tax Administration Law	Law on Tax Administration 2019	10.0 / 10	Outstanding
Commercial Arbitration Law	Law on Commercial Arbitration 2010	10.0 / 10	Outstanding
Penal Code	Penal Code 2015 (Amended 2017)	10.0 / 10	Outstanding
Competition Law	Competition Law 2018	10.0 / 10	Outstanding
Consumer Protection Law	Law on Protection of Consumer Rights 2023	10.0 / 10	Outstanding
Civil Law	Civil Code 2015	9.5 / 10	Excellent
Enterprise / Corporate Law	Law on Enterprises 2020	9.5 / 10	Excellent
Real Estate Business Law	Law on Real Estate Business 2023	9.0 / 10	Excellent
Housing Law	Housing Law 2023	9.0 / 10	Excellent
Labor Law	Labor Code 2019	8.88 / 10	Very Good
Commercial Law	Commercial Law 2005	8.75 / 10	Very Good
Social Insurance Law	Law on Social Insurance & Labor Code 2019	8.5 / 10	Very Good
Cybersecurity Law	Law on Cybersecurity 2018	8.5 / 10	Very Good
Intellectual Property Law	Law on Intellectual Property (Amended 2022)	6.5 / 10	Good

---

🚀 Quickstart & Inference Guide

1. Using Hugging Face Transformers & PEFT

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-7B-Instruct"
adapter_model_id = "path/to/vilaw-dpo-lora-v2"  # Local adapter directory or Hugging Face Hub repo

# 1. Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# 2. Attach VILaw-LLM v2 LoRA Adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)
model.eval()

# 3. Format with ChatML Template
SYSTEM_PROMPT = (
    "Bạn là một chuyên gia tư vấn pháp luật Việt Nam am hiểu sâu sắc các quy định pháp luật. "
    "Hãy trả lời câu hỏi dựa trên các văn bản quy phạm pháp luật hiện hành, "
    "viện dẫn chính xác số Điều, Khoản, tên luật và đưa ra lập luận logic, rõ ràng."
)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "Thời hạn các thành viên phải góp vốn vào công ty TNHH hai thành viên trở lên là bao lâu theo Luật Doanh nghiệp 2020?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)

# 4. Generate Response
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.3,
        repetition_penalty=1.15
    )

response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print("=== VILAW-LLM V2 RESPONSE ===")
print(response)

2. Using Unsloth (2x Faster Inference)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="path/to/vilaw-dpo-lora-v2",
    max_seq_length=2048,
    load_in_4bit=True
)
FastLanguageModel.for_inference(model)

# Proceed with standard inference using tokenizer.apply_chat_template(...)

---

🛠️ Continual Training Specifications

• Base Architecture: Qwen/Qwen2.5-7B-Instruct (Quantized 4-bit BnB).
• Starting Checkpoint: vilaw-dpo-lora (Pre-aligned via SFT + DPO).
• Fine-Tuning Method: LoRA (Low-Rank Adaptation) with $r=16$, $\alpha=32$, $\text{dropout}=0$.
• Target Modules: All 7 linear projection layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).
• Learning Rate: $5 \times 10^{-5}$ (Conservative rate to prevent catastrophic forgetting and preserve DPO alignment).
• Effective Batch Size: 16 (Per-device $4 \times$ gradient accumulation 4).
• Compute Infrastructure: NVIDIA Tesla T4 GPU (VRAM ~12.5GB).
• Curated Dataset: 9,674 deduplicated, high-precision statutory pairs from duyet/vietnamese-legal-instruct and hoanghai2110/vietnamese-dataset.

---

🛡️ Intended Use & Legal Disclaimer

• Research & Assistant Utility: vilaw-dpo-lora-v2 is an Artificial Intelligence Research Artifact developed to assist legal researchers, legal assistants, lawyers, and citizens with initial statutory lookup and document analysis.
• Not Certified Legal Advice: Outputs generated by this model do not constitute official legal counsel, formal legal representation, or binding procedural documentation under Vietnamese law.
• Statutory Verification Mandatory: Laws, Decrees (Nghị định), and Circulars (Thông tư) in Vietnam are subject to periodic amendments. Users must verify all statutory references against the National Database of Legal Normative Documents (Cơ sở Dữ liệu Quốc gia về Văn bản Pháp luật - vbpl.vn) or consult a certified attorney prior to making significant legal decisions.

---

📄 License

This adapter model is distributed under the Apache 2.0 License, conforming with the open-source licensing terms of the Qwen model family.