Instructions to use prabhu09/KisanSLM-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use prabhu09/KisanSLM-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf prabhu09/KisanSLM-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf prabhu09/KisanSLM-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf prabhu09/KisanSLM-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf prabhu09/KisanSLM-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf prabhu09/KisanSLM-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf prabhu09/KisanSLM-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf prabhu09/KisanSLM-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf prabhu09/KisanSLM-GGUF:F16

Use Docker

docker model run hf.co/prabhu09/KisanSLM-GGUF:F16

LM Studio
Jan

vLLM

How to use prabhu09/KisanSLM-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "prabhu09/KisanSLM-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prabhu09/KisanSLM-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/prabhu09/KisanSLM-GGUF:F16

Ollama
How to use prabhu09/KisanSLM-GGUF with Ollama:
```
ollama run hf.co/prabhu09/KisanSLM-GGUF:F16
```
Unsloth Desktop

How to use prabhu09/KisanSLM-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf prabhu09/KisanSLM-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "prabhu09/KisanSLM-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use prabhu09/KisanSLM-GGUF with Docker Model Runner:
```
docker model run hf.co/prabhu09/KisanSLM-GGUF:F16
```

Lemonade

How to use prabhu09/KisanSLM-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull prabhu09/KisanSLM-GGUF:F16

Run and chat with the model

lemonade run user.KisanSLM-GGUF-F16

List all available models

lemonade list

Hermes Agent

How to use prabhu09/KisanSLM-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf prabhu09/KisanSLM-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default prabhu09/KisanSLM-GGUF:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use prabhu09/KisanSLM-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf prabhu09/KisanSLM-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "prabhu09/KisanSLM-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

KisanSLM

KisanSLM is unsloth/Qwen3.5-2B fine-tuned (QLoRA, 4-bit) to answer Indian farmers' agricultural questions in the register of a real Kisan Call Centre (KCC) / Farm Tele Advisor (FTA) advisor — short, specific, actionable advice (product / dose / timing), instead of generic AI-assistant hedging.

Training code, the full dataset pipeline, and evaluation scripts are here: github.com/PrabhudattaPatra/KisanSLM

Files

File	Description
`Qwen3.5-2B.Q4_K_M.gguf` (1.31GB)	The model. 4-bit quantized, ready for llama.cpp / Ollama / LM Studio.

Training data

~13,241 curated farmer-query / FTA-answer pairs, derived from India's Kisan Call Centre transcripts (data.gov.in, Government Open Data License – India). Built via a pipeline that pulled 400K+ raw records and cleaned, deduplicated, and quality-filtered them (rule-based + LLM-judge scoring) down to this set. Answers are kept verbatim from the real FTA response — nothing LLM-rewritten. Full methodology and stats: see the GitHub repo's DATASET_CARD.md.

Training method

Base model: unsloth/Qwen3.5-2B
Method: QLoRA (4-bit), via Unsloth Studio
Hardware: Google Colab, T4 GPU (16GB VRAM)
11,916 training examples, 662 validation
Exported to GGUF, Q4_K_M quantization

How to run (Ollama)

ollama pull hf.co/prabhu09/KisanSLM-GGUF:Q4_K_M
ollama cp hf.co/prabhu09/KisanSLM-GGUF:Q4_K_M kisanslm

The system prompt matters — set it via a Modelfile (ollama create kisanslm -f Modelfile):

FROM hf.co/prabhu09/KisanSLM-GGUF:Q4_K_M
SYSTEM """You are a Farm Tele Advisor (FTA) at India's Kisan Call Centre (KCC). Farmers call in with questions about crops, pests, diseases, soil, irrigation, weather, and government agricultural schemes. Give specific, locally-relevant, actionable advice in a direct, practical register -- the way a real KCC advisor would, not a generic AI assistant."""

ollama run kisanslm --think=false "My paddy leaves have brown spots, what should I do?"

Qwen3.5 has a built-in "thinking" reasoning phase that can loop indefinitely on this quantized 2B model. Always pass --think=false (CLI) or "think": false (API) — see the GitHub repo's FINETUNE.md for details.

Evaluation results and known limitations

Evaluated on 30 held-out test examples (never seen in training) against the real FTA answers. Full report/methodology in the GitHub repo.

Style/register transfer: works well. Answers are short, direct, product+dose+timing style — close to indistinguishable from real FTA answers on plant-protection queries, the dataset's largest category.
Factual grounding: a real limitation. On specific facts the model wasn't taught precisely enough to recall — market prices, phone numbers/contacts, exact pesticide product names — it confidently fabricates plausible-looking wrong answers rather than hedging. This is expected for a 2B model with no retrieval grounding; fixing it needs RAG against a live source, not more fine-tuning.

Practical guidance: trust this model's plant-protection / cultural practice / nutrient-management advice as a starting point (verify doses locally); do not trust prices, contact details, or exact product names without independent verification.