adtc-agri-sft-gemma2-2b

This model is a fine-tune of google/gemma-2-2b-it, adapted via QLoRA (4-bit base + LoRA adapters, merged back into full weights) for agricultural advisory Q&A aimed at smallholder farmers and extension officers in Nigeria.

Part of the ADTC 2026 project. This is a baseline SFT run — direct supervised fine-tuning with no distillation — used to establish how much headroom a distillation stage could still add over the base model.

Model description

• Base model: google/gemma-2-2b-it

• Fine-tuning method: QLoRA (4-bit NF4 quantized base, bf16 compute dtype, double quantization), adapters merged into the base weights for the published checkpoint

• LoRA config: r=16, alpha=32, dropout=0.05, target modules: q_proj, down_proj, up_proj, gate_proj, k_proj, o_proj, v_proj

• Task: single-turn instruction following — given a farming scenario or question, produce a concise assessment, a practical recommendation, and (if relevant) a note on when to escalate to in-person expert help

• System prompt used at train/inference time: Gemma-2's chat template does not support a system role, so the prompt below is prepended into the first user turn rather than sent as a separate message:

  You are an offline agricultural advisory assistant for smallholder farmers and extension officers in Nigeria. Given the situation described, provide a clear assessment, a practical recommendation, and, where relevant, a note on when to seek in-person expert help. Be concise and avoid unnecessary hedging.

Training data

Combined and deduplicated (on normalized question text) from:

• kaggle_csv: 36,489 examples

• synthetic_market: 593 examples

• synthetic_fertilizer: 598 examples

• Total training examples: 35,796

• Held-out eval examples: 1,884 (5% split)

• Primary source: the Kaggle agriculture_qa_final_cleaned.csv dataset (query/response columns), supplemented with synthetically generated fertilizer-dosage and market/storage economics word problems to cover numeric-reasoning cases underrepresented in the base dataset.

Training procedure

Hyperparameters

Hyperparameter	Value
Base model	google/gemma-2-2b-it
Max sequence length	1024
Batch size (per device)	2
Gradient accumulation steps	8
Effective batch size	16
Max steps	-1
Learning rate	0.0002
LR scheduler	SchedulerType.COSINE
Precision	bf16
Seed	42
LoRA r / alpha / dropout	16 / 32 / 0.05

Training loss

Step	Training Loss	Validation Loss	Learning Rate
500	0.6988	0.6212	1.94e-04
1000	0.5976	0.5819	1.76e-04
1500	0.5669	0.5565	1.50e-04
2000	0.5401	0.5362	1.17e-04
2500	0.4900	0.5271	8.18e-05
3000	0.4463	0.5137	4.91e-05
3500	0.4337	0.5029	2.26e-05
4000	0.4283	0.4979	5.55e-06
4476	—	0.4970	—

Intended use & limitations

• Intended for offline agricultural advisory use cases (smallholder farmers, extension officers) in a Nigerian context; not validated for general-purpose assistant use or for regions/crops outside its training distribution.
• This is a baseline checkpoint for internal comparison against the un-tuned base model and, later, a distilled variant — treat outputs as advisory, not authoritative agronomic or financial guidance, and always recommend in-person expert follow-up for high-stakes decisions.
• Evaluated informally via qualitative spot-checks on held-out prompts; formal accuracy benchmarking (ARC-Easy / domain-quality via the ADTC profiler) is tracked separately from this card.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "EYEDOL/adtc-agri-sft-gemma2-2b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

# Gemma-2 has no "system" role — fold the system prompt into the user turn
messages = [
    {"role": "user", "content": "You are an offline agricultural advisory assistant for smallholder farmers and extension officers in Nigeria. Given the situation described, provide a clear assessment, a practical recommendation, and, where relevant, a note on when to seek in-person expert help. Be concise and avoid unnecessary hedging.

Your farming question here"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

Card generated automatically at training time — 2026-09-14 11:18 UTC.