shehryars715/finetuned-Qwen2.5-7B-Instruct

🌾 Agricultural Advisory LLM — Qwen2.5-7B (Pakistan)

A LoRA fine-tuned version of Qwen2.5-7B-Instruct specialized for Pakistani crop farming advisory. The model answers general crop questions and interprets field sensor data (NDVI, EVI, NDWI, temperature, humidity) to provide concise, actionable farm advisories.

---

Model Details

• Base model: Qwen/Qwen2.5-7B-Instruct (4-bit quantized via Unsloth)
• Fine-tuning method: LoRA (rank 16, alpha 16, RSLoRA enabled)
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable parameters: 40,370,176 / 7,655,986,688 (0.53%)
• Hardware: NVIDIA Tesla T4 (14.56 GB VRAM)
• Training time: ~30 minutes (1840s)
• Peak VRAM: 12.02 GB

---

Training Details

Dataset

• General Q&A: Synthetic agricultural advisories covering crops, topics, and questions relevant to Pakistani farming conditions
• Farm-specific: Sensor-based advisories using field readings (NDVI, EVI, SAVI, MSAVI, NDWI, GNDVI, temperature, humidity, etc.)
• Total examples: 4,723 mixed and shuffled records
• Packed examples: 335 (via Unsloth sequence packing, ~14 examples/chunk)

Hyperparameters

Parameter	Value
Epochs	1
Learning rate	1e-4
LR scheduler	Cosine
Warmup ratio	0.1
Batch size (per device)	2
Gradient accumulation steps	4
Effective batch size	8
Weight decay	0.05
Max grad norm	0.3
Optimizer	AdamW 8-bit
Precision	bf16
Max sequence length	2048
Packing	Enabled

Training Loss

Step	Loss
5	3.6717
10	1.7415
15	0.9481
20	0.7458
25	0.7098
30	0.7026
35	0.6614
40	0.6280

Final loss settled in the 0.62–0.66 range, indicating good generalization without overfitting.

---

Usage

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name    = "shehryars715/finetuned-Qwen2.5-7B-Instruct",
    max_seq_length= 2048,
    dtype         = None,
    load_in_4bit  = True,
)
FastLanguageModel.for_inference(model)

SYSTEM_PROMPT = (
    "You are an expert agricultural advisor specializing in Pakistani crop farming. "
    "You can answer general crop questions and also interpret field sensor data "
    "(NDVI, EVI, NDWI, temperature, humidity, etc.) to provide precise farm advisories. "
    "Answer accurately and concisely based on official recommendations and best practices. "
    "Keep answers under 3 sentences. Do not include citations, URLs, or markdown headers. "
    "Answer directly and stop."
)

def ask(crop, question, topic="General"):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": f"[Crop: {crop} | Topic: {topic}]\n{question}"},
    ]
    inputs = tokenizer.apply_chat_template(
        messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
    ).to("cuda")

    with torch.no_grad():
        out = model.generate(
            input_ids=inputs, max_new_tokens=150,
            use_cache=True, temperature=0.7, top_p=0.9,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id,
        )
    return tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True).strip()

print(ask("Maize", "How much seed is required per acre?"))
# → 20-35 kg/acre of certified hybrid or 40-60 kg of open-pollinated variety.

Farm Sensor Advisory

def ask_farm(crop, stage, sensors: dict):
    sensor_str = "\n".join(f"{k}: {v}" for k, v in sensors.items())
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": (
            f"[Crop: {crop} | Stage: {stage}]\n"
            f"Field sensor readings:\n{sensor_str}\n\n"
            f"Provide a detailed farm advisory based on these readings."
        )},
    ]
    inputs = tokenizer.apply_chat_template(
        messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
    ).to("cuda")

    with torch.no_grad():
        out = model.generate(
            input_ids=inputs, max_new_tokens=200,
            use_cache=True, temperature=0.7, top_p=0.9,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id,
        )
    return tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True).strip()

print(ask_farm("Wheat", "Tillering", {"NDVI": 0.42, "temperature_c": 22, "relative_humidity": 65}))

---

Sample Outputs

Crop	Question	Answer
Maize	Which varieties are high-yielding in Pakistan?	Check certified varieties approved by PABMA. K-34, K-72, K-108, and K-156 have shown good yield potential under proper management.
Maize	How much seed per acre?	20-35 kg/acre of certified hybrid or 40-60 kg of open-pollinated variety.
Maize	NDVI 0.42 — is that healthy?	Yes — indicates healthy vegetative growth during grain fill.
Cotton	Pesticide for whitefly?	Thiamethoxam or Spiromesifen at 10-20 ml per spray; rotate chemistries per resistance management guidelines.

---

Limitations

• Trained on synthetic data — real-world agronomic validation recommended before deployment
• Pakistan-specific; recommendations may not transfer to other regions
• Sensor advisory accuracy depends on data quality and crop stage alignment
• Not a substitute for consultation with local agricultural extension services

---

Authors

Developed for the Intelligent Agricultural Advisory (IAL) project.