ajvikram/emergency-triage-4b-gguf

Emergency Triage 4B (GGUF)

A fine-tuned Qwen3-4B model specialized for 911/emergency incident triage in remote locations. Designed for on-device inference on phones, tablets, and edge hardware where connectivity is limited or unavailable.

What it does

Given a free-text emergency incident report, the model produces a structured JSON triage assessment:

{
  "severity": "CRITICAL",
  "incident_type": "MEDICAL",
  "summary": "Hiker fell 30 feet from cliff, unable to move legs...",
  "location": {
    "description": "Pacific Crest Trail, mile marker 1847",
    "accessibility": "TRAIL"
  },
  "resources_needed": ["Helicopter EMS", "Rope rescue team", "Trauma kit"],
  "priority_actions": ["Immobilize legs", "Apply thermal protection", "Request helicopter evacuation"],
  "requires_evacuation": true,
  "communication_status": "LIMITED"
}

---

How to run

1. Ollama (Mac / Linux / Windows)

The easiest way to run locally.

# Download the GGUF file
hf download ajvikram/emergency-triage-4b-gguf emergency-triage-4b-q4_k_m.gguf

# Create a Modelfile
cat > Modelfile << 'EOF'
FROM ./emergency-triage-4b-q4_k_m.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM """You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment."""

PARAMETER temperature 0.3
PARAMETER num_ctx 2048
PARAMETER stop "<|im_end|>"
EOF

# Import into Ollama
ollama create emergency-triage -f Modelfile

# Run it
ollama run emergency-triage "A hiker has collapsed on a remote trail 8 miles from the nearest road. They are unresponsive, breathing shallow. Two other hikers are present. No cell service. Temperature is 95F with high humidity."

2. llama.cpp (Mac / Linux / Windows)

For direct GGUF inference without Ollama.

# Download the GGUF
hf download ajvikram/emergency-triage-4b-gguf emergency-triage-4b-q4_k_m.gguf

# Run inference (Metal GPU on Mac, CUDA on Linux)
llama-cli \
  -m emergency-triage-4b-q4_k_m.gguf \
  -p "<|im_start|>system
You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment.<|im_end|>
<|im_start|>user
Your emergency scenario here<|im_end|>
<|im_start|>assistant" \
  -n 512 --temp 0.3 -ngl 99 --single-turn

3. iPhone / iPad (iOS)

Using LLM Farm (free) or PocketPal AI:

1. Download emergency-triage-4b-q4_k_m.gguf (2.4 GB) to your device
2. Open the app and import the GGUF file
3. Set the system prompt to:

   You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment.
   

4. Set temperature to 0.3 for consistent outputs
5. Type or speak your emergency scenario

Device requirements: iPhone 12+ or iPad Air 4+ (4GB+ RAM). The Q4_K_M quantization runs well on A14 chip and newer.

4. Android

Using ChatterUI or llama.cpp Android:

1. Download emergency-triage-4b-q4_k_m.gguf to your device
2. Open the app and load the model
3. Set system prompt as above
4. Set temperature to 0.3

Device requirements: 6GB+ RAM recommended. Works on Snapdragon 8 Gen 1+ or equivalent.

5. Python (transformers)

For the full-precision merged model:

from transformers import AutoModelForCausalLM, AutoTokenizer
import json

model = AutoModelForCausalLM.from_pretrained("ajvikram/emergency-triage-4b-gguf", subfolder="merged")
tokenizer = AutoTokenizer.from_pretrained("ajvikram/emergency-triage-4b-gguf", subfolder="merged")

messages = [
    {"role": "system", "content": "You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment."},
    {"role": "user", "content": "Your scenario here"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.3)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

6. Edge Deployment (Raspberry Pi / Jetson)

The Q4_K_M GGUF works on edge devices with 4GB+ RAM:

# On Raspberry Pi 5 (8GB) or Jetson Nano
# Build llama.cpp for your platform
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build

# Run inference (CPU-only on Pi, CUDA on Jetson)
./build/bin/llama-cli -m emergency-triage-4b-q4_k_m.gguf \
  -p "your prompt" -n 512 --temp 0.3 --single-turn

Expected speeds: ~5-10 tok/s on Pi 5, ~30+ tok/s on Jetson Orin.

---

Training details

• Method: QLoRA (r=32, alpha=64) via Unsloth
• Data: 500 examples (52 hand-crafted seeds + 448 synthetic via knowledge distillation)
• Teacher: 120B parameter model for synthetic data generation and quality judging
• Epochs: 3
• Final eval loss: 0.52
• Hardware: NVIDIA RTX A6000 (48GB), ~8 minutes training time

Quantization

Format	Size	BPW	Use case
F16 (merged)	7.6 GB	16.0	Server / fine-tuning base
Q4_K_M	2.4 GB	4.95	Mobile / edge deployment

Performance

Platform	Speed	Notes
Mac (Metal)	~84 tok/s	M1/M2/M3 Apple Silicon
NVIDIA GPU	~100+ tok/s	CUDA, depends on GPU
iPhone 15 Pro	~15-20 tok/s	A17 Pro chip
Raspberry Pi 5	~5-10 tok/s	CPU only, 8GB model

Output schema

{
  "severity": "CRITICAL | HIGH | MEDIUM | LOW",
  "incident_type": "MEDICAL | NATURAL_DISASTER | RESCUE | INFRASTRUCTURE | WILDLIFE | WEATHER | VEHICLE | OTHER",
  "summary": "1-2 sentence dispatch summary",
  "location": {
    "description": "location details",
    "accessibility": "ROAD | TRAIL | OFF_TRAIL | AIR_ONLY"
  },
  "resources_needed": ["specific resources"],
  "priority_actions": ["ordered by urgency"],
  "requires_evacuation": true/false,
  "communication_status": "FULL | LIMITED | NONE"
}

Limitations

• Phase 0 prototype trained on 500 examples — production would use 40,000+
• Schema adherence is good but not perfect with limited training data
• Optimized for English-language incident reports
• Designed for remote/rural emergency scenarios
• Not a replacement for trained dispatchers — intended as a decision support tool

License

Apache 2.0 (inherited from Qwen3-4B base model)