📢 Domain & Email Migration NoticeFrom May 6th, 2026, Simutrade will transition to new domains as 🌐 Website: simutrade.faizath.com (formerly simutrade.app) |
🎓 Simutrade v1 CLM-RAG Gemma3 4B IT Adapter
QLoRA Adapter for Retrieval-Grounded Bilingual EN/ID Trade Assistance
🌐 Live Demo • 🎭 Behaviours • ⚡ Usage • 🏋️ Training • 🧪 Evaluation • ⚠️ Limitations • 📁 Repositories
🌟 Overview
A QLoRA adapter that turns gemma-3-4b-it into a retrieval-grounded international-trade
assistant speaking English and Bahasa Indonesia. It answers from passages you retrieve and
hand it, credits them as "my augmented trade knowledge", and — the part that took most of the
training budget — declines to invent an answer when the retrieval comes back empty or wrong.
Trained on simutrade/simutrade-rag-sft-28k
as a local, API-free replacement for a Gemini-backed RAG service.
🧩 Why It Exists
The production assistant this distills ran on Gemini with live web search. That is a per-call cost, a network dependency, and a hard blocker for offline or on-premise deployment. A 4B model with a 131 MB adapter runs on a single consumer GPU.
The interesting problem was never fluency — it was refusal discipline. A trade assistant that confabulates an Incoterm or invents a tariff figure is worse than one that says "I don't have that", because the failure is invisible to the person asking.
🎭 What It Learned
Eight behaviours, each a rule the production service is specified to have:
| Behaviour | Train examples | What it does |
|---|---|---|
grounded |
11,265 | Answers from the retrieved chunks, credited as the knowledge alias |
empty_context |
3,076 | Nothing retrieved → fixed out-of-scope opener, then a parametric answer |
multi_hop |
2,963 | Synthesises across chunks from different documents |
multi_turn |
2,345 | Follow-up turns with pronouns and ellipsis |
distractor |
1,966 | Chunks are near-misses → admits the gap instead of forcing a link |
refusal |
1,586 | Harmful query → one canonical string, verbatim |
tool_constraint |
1,058 | Recommends only the in-product capability, never a competitor |
small_talk |
773 | Greetings and connectivity tests → short, warm redirect |
empty_context and distractor together are 20% of the training data spent teaching the model
to say "I don't have that" rather than confabulate. That was deliberate.
Three strings are emitted byte-exact and are asserted in evaluation:
| Constant | Value |
|---|---|
| Refusal | Sorry, I can't help due to harmful content. |
| Out-of-scope opener | A bit out of scope, but here's what I know. |
| Knowledge alias | my augmented trade knowledge |
In Indonesian the possessive trails the noun — the model correctly produces
augmented trade knowledge saya, not a literal translation.
⚡ Usage
The prompt format is not optional. Gemma-3 has no system role, so the persona block lives
inside the first user turn, and retrieved context has its quotes stripped. simutrade_prompt.py
in this repo reproduces both exactly — it has no third-party imports.
from unsloth import FastModel
from unsloth.chat_templates import get_chat_template
from simutrade_prompt import render_user_turn
model, tokenizer = FastModel.from_pretrained(
model_name="simutrade/simutrade-v1-clm-rag-gemma3-4b-it-adapter",
max_seq_length=4096,
load_in_4bit=True,
)
tokenizer = get_chat_template(tokenizer, chat_template="gemma-3")
FastModel.for_inference(model)
chunks = retrieve("What documents do I need to export coffee to Japan?") # your retriever
messages = [{"role": "user", "content": render_user_turn(
"What documents do I need to export coffee to Japan?", chunks)}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
Pass context_texts=None to exercise the out-of-scope path. Bring your own retriever — the
adapter expects chunk texts, and is agnostic to how you found them.
On the base model
adapter_config.json records the base as unsloth/gemma-3-4b-it-bnb-4bit — the pre-quantized
4-bit build the adapter was actually trained and verified against. That is what the snippet above
loads, and it requires bitsandbytes.
QLoRA adapters are designed to transfer to the full-precision base, so loading against
google/gemma-3-4b-it with transformers + peft should also work — but that repo is gated, and
that path has not been verified for this adapter. The 13/13 result below is from the Unsloth
4-bit path only.
🏋️ Training
| Hardware | NVIDIA A100-SXM4-40GB |
| Precision | bf16, 4-bit QLoRA base |
| Epochs | 1 |
| Steps | 1,539 |
| Wall clock | 4h 47m 18s |
| Effective batch | 16 (8 × grad-accum 2) |
| Sequence length | 4,096 |
| LoRA | r=16, alpha=32, dropout 0.05 |
| Target modules | q,k,v,o,gate,up,down projections |
| Trainable params | 32,788,480 of 4,332,867,952 (0.76%) |
| Optimizer | adamw_8bit, lr 2e-4, cosine, warmup 5% |
| Train examples used | 24,624 of 25,032 |
| Metric | Value |
|---|---|
train_loss |
0.9559 |
eval_loss |
0.8417 |
Loss fell 3.358 → ~0.83. eval_loss sits below train_loss, so the model is not overfitting
at one epoch — a second epoch is defensible.
Response-only loss masking was essential. User turns average ~4,035 characters because they carry roughly five retrieved chunks; model turns average 530. Without masking, ~88% of the loss budget would go to teaching the model to regenerate its own context.
🧪 Evaluation
13/13 checks passed. Full transcript in EVAL.log.
| Check | Result |
|---|---|
| Documented safety case | ✅ byte-exact refusal |
refusal × 3 |
✅ byte-exact |
empty_context × 3 |
✅ exact opener, EN and ID |
tool_constraint × 3 |
✅ no competitor named |
grounded × 3 |
✅ knowledge alias credited |
The safety case is the adversarial query the production service names as its acceptance test. The model returns the canonical refusal with no paraphrase.
Held-out real production queries — genuine user turns excluded from training entirely — return fluent, correctly formatted answers, and Indonesian questions are answered in Indonesian.
⚠️ Limitations
- Distillation ceiling. Answers were generated by
gemma-4-31b-it; the student inherits its errors and cannot exceed it. Expect lower factual precision than the Gemini service it replaces. - No live search. There is no offline equivalent. Keep search as an external tool call around the model — do not expect it to learn one.
- Dated corpus. Source documents span 2023–2025. The model states stale figures confidently. Add a date disclaimer in your deployed prompt.
- Weak tariff grounding. World Tariff Profiles 2024 yielded only 4 usable chunks — it is almost entirely tables, which the PDF extractor handled poorly. Tariff questions are thinly grounded.
- Product-specific behaviour.
tool_constrainttrains the model to recommend one specific product and refuse to name alternatives. Correct for its origin, wrong for most other contexts. - Synthetic distribution. Question phrasing reflects a generator's idea of user queries, not measured traffic. Only 15 held-out examples are genuine.
- One epoch. Given
eval_loss < train_loss, this is likely under-trained rather than over. - 408 training examples contributed no gradient. The length filter measures tokens without special tokens while the trainer truncates with them, so examples within a few tokens of the 4,096 limit passed the filter and were then truncated — cutting the tail off the model turn and leaving every label masked. The trainer detects and drops these, so results are sound, but 1.63% of the data did no work.
📁 Files
adapter_model.safetensors 131 MB LoRA weights (r=16)
adapter_config.json PEFT config — base model, rank, target modules
simutrade_prompt.py prompt builder — persona + context, zero dependencies
chat_template.jinja Gemma-3 chat template
tokenizer.json tokenizer
tokenizer.model SentencePiece model
tokenizer_config.json tokenizer config
processor_config.json processor config
EVAL.log full evaluation transcript, 13/13
LICENSE layered licensing notice — read before redistributing
🏗️ Repository Architecture
This model is part of the wider Simutrade platform:
| Repository | GitHub | Hugging Face | Deployment | Description |
|---|---|---|---|---|
| 📱 Frontend | simutrade-app/simutrade-fe | — | simutrade.faizath.com | React + TypeScript interface with interactive dashboards, real-time maps, and AI-powered analytics. |
| 🔧 API Backend | simutrade-app/simutrade-api | — | simutrade-api.faizath.com | RESTful API handling business logic, authentication, and real-time WebSocket communications. |
| 🧠 AI & RAG Backend | simutrade-app/simutrade-ai | — | Integrated with platform | RAG retrieval over trade documents using Gemini embeddings and ChromaDB — the service this model distills. |
| 📊 Dataset | simutrade-app/simutrade-rag-sft-28k | simutrade/simutrade-rag-sft-28k | Offline / Colab | 27,839 bilingual RAG SFT examples and the 1,385-chunk grounding corpus this model trained on. |
| 🎓 Model | simutrade-app/simutrade-v1-clm-rag-gemma3-4b-it-adapter | simutrade/simutrade-v1-clm-rag-gemma3-4b-it-adapter | Offline / Colab | This model — QLoRA adapter finetuned from gemma-3-4b-it on the dataset above. |
⚖️ License
Gemma Terms of Use govern the weights — a LoRA of Gemma is a Model Derivative. The training data
additionally embeds text from copyrighted UNCTAD, DHL, World Bank, McKinsey, and WTO
publications. See LICENSE before redistributing or using commercially.
📞 Support & Contact
- Website: simutrade.faizath.com
- API Documentation: simutrade-api.faizath.com
- Email: contact@simutrade.faizath.com
- Organization: github.com/simutrade-app
Made with ❤️ by the Simutrade Team
- Downloads last month
- 13
Model tree for simutrade/simutrade-v1-clm-rag-gemma3-4b-it-adapter
Base model
google/gemma-3-4b-pt