Instructions to use Sandy-sys/osint-argus-gemma4-qat-w4a16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sandy-sys/osint-argus-gemma4-qat-w4a16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Sandy-sys/osint-argus-gemma4-qat-w4a16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Sandy-sys/osint-argus-gemma4-qat-w4a16")
model = AutoModelForMultimodalLM.from_pretrained("Sandy-sys/osint-argus-gemma4-qat-w4a16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Sandy-sys/osint-argus-gemma4-qat-w4a16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sandy-sys/osint-argus-gemma4-qat-w4a16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sandy-sys/osint-argus-gemma4-qat-w4a16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Sandy-sys/osint-argus-gemma4-qat-w4a16

SGLang

How to use Sandy-sys/osint-argus-gemma4-qat-w4a16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sandy-sys/osint-argus-gemma4-qat-w4a16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sandy-sys/osint-argus-gemma4-qat-w4a16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sandy-sys/osint-argus-gemma4-qat-w4a16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sandy-sys/osint-argus-gemma4-qat-w4a16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Sandy-sys/osint-argus-gemma4-qat-w4a16 with Docker Model Runner:
```
docker model run hf.co/Sandy-sys/osint-argus-gemma4-qat-w4a16
```

OSINT ARGUS — Gemma 4 12B QAT · Indian-language OSINT · W4A16 tri-modal

Google's QAT (quantisation-aware-trained) Gemma 4 12B, full-parameter fine-tuned for Indian-context open-source-intelligence analysis over text, images and speech, and packed to int4 / group-32 / symmetric — the exact numerical grid it was trained on.

One self-contained repository. No adapter, no LoRA, no merge step, no second model to download. Pull this repo, point a server at it, and it runs.


Base	`google/gemma-4-12B-it-qat-q4_0-unquantized`
Architecture	`Gemma4UnifiedForConditionalGeneration`, 48-layer early-fusion trunk (10.90 B params)
Modalities	text · image (280 soft tokens/image) · audio
Quantisation	int4, group size 32, symmetric, `pack-quantized` (compressed-tensors)
Weights	8.25 GB, single `model.safetensors`
Trained parameters	all 10.90 B trunk weights + 52.4 M modality embedders
Output contract	strict 11-key OSINT JSON record

1 · Running it

huggingface-cli download Sandy-sys/osint-argus-gemma4-qat-w4a16 --local-dir argus

python -m sglang.launch_server --model-path ./argus \
  --quantization compressed-tensors \
  --mem-fraction-static 0.45 \
  --context-length 8192

That is the entire procedure. The repository already contains the packed weights, tokenizer, multimodal processor, chat template, and the config keys the server needs. Nothing to patch, merge, or fetch separately.

Verified end-to-end: pulled fresh from this repo into an empty directory, loaded with no local files, and generated correctly at 8.25 GB resident.

VRAM — read this before deploying

runtime	behaviour	VRAM
SGLang / vLLM	keeps weights packed at int4	8.25 GB ✅
Transformers	decompresses to bf16 at first inference	~25.1 GB ⚠️

The 10 GB budget holds only under a runtime with native compressed-tensors int4 kernels. Transformers is fine for a desktop sanity check and wrong for an edge box.

Under SGLang this leaves ~1.75 GB of a 10 GB allocation free, so the model co-resides with a second model on a 30 GB card (validated target: MSI Edge Expert alongside Nemotron 3 Super NVFP4).

Transformers, for a quick check on a large card

from transformers import AutoProcessor, AutoModelForCausalLM
import torch

repo = "Sandy-sys/osint-argus-gemma4-qat-w4a16"
proc = AutoProcessor.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, device_map="cuda:0")

msgs = [{"role": "user", "content": [
    {"type": "image", "url": "poster.jpg"},
    {"type": "text", "text": "Analyse this image as an OSINT record."}]}]
inp = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
                               return_dict=True, return_tensors="pt").to(0)
print(proc.decode(model.generate(**inp, max_new_tokens=512)[0][inp["input_ids"].shape[1]:],
                  skip_special_tokens=True))

Config note. vision_config.model_patch_size=48, vision_config.num_soft_tokens=280 and audio_config.output_proj_dims=640 are already backfilled in config.json. They are absent from Google's upstream QAT config and SGLang exits without them. Listed so the difference from upstream is documented — no action needed.

2 · The output contract

Every response is a single JSON object with exactly these 11 keys:

{
  "language": "hi", "script": "Deva", "code_mixed": false,
  "entities": [{"text": "...", "type": "PER|ORG|LOC|MISC"}],
  "claims": [{"text": "...", "verdict": "supported|refuted|unverified"}],
  "stance": "favor|against|neutral",
  "sentiment": "positive|negative|neutral",
  "misinformation": "verified|false|unverified",
  "hostility": {"label": "none|offensive|hateful"},
  "events": [{"type": "...", "text": "..."}],
  "summary_en": "English summary, <= 280 chars"
}

The schema_lock gate below measures how reliably the model emits a valid record under bare decoding — no constrained grammar, no JSON mode, no retries.

3 · Evaluation

All figures measured on the exact weights in this repository. Evaluation items are held out by construction: every media path used in training is excluded, article texts are matched by normalised content hash so the same story behind a different image is still excluded, and audio clips are excluded by raw-byte SHA-256.

3.1 Text — 200 items per task, bare decoding

gate	stock Gemma-4 QAT	this model
`schema_lock` — valid 11-key OSINT record	0.0	0.985
`ner` — micro-F1, 11 Indian languages	0.6163	0.7046
`sentiment` — accuracy, 13 languages	0.76	0.785
`claim` — refuted-class F1	0.8125	0.8685
`halluc_detect` — hallucination-detection F1	0.3981	0.4783
`faithfulness` — grounded-answer schema rate	0.9536	0.9669
`general` — MCQ accuracy %	59.0	53.0

3.2 Multimodal — 106–120 items per task

gate	this model	majority baseline	n
`vision_ocr` — CER ↓	0.5482	—	120	—
`vision_misinfo` — accuracy	0.5083	0.5083	120	at baseline
`vision_hostility` — accuracy	0.434	0.6321	106	below baseline
`audio_asr` — CER ↓	0.6889	—	120	—
`audio_langid` — accuracy	0.1833	0.175	120	above baseline
`audio_abuse` — accuracy	0.425	0.5417	120	below baseline

3.3 Image text by script (CER, lower is better)

script	code	CER	n
Arabic (Urdu)	`Arab`	0.2271	38
Devanagari (hi, mr, sa, ne)	`Deva`	0.6059	31
Bengali (bn, as)	`Beng`	0.6831	38
Odia	`Orya`	0.9546	13

Trained on all 12 INDICSTR12 scripts; the four above are the ones with held-out measurement.

3.4 Speech transcription by language (CER, lower is better)

language	code	CER	n
Bhojpuri	`bho`	0.069	1
Hindi	`hi`	0.2832	7
Maithili	`mai`	0.3016	2
English	`en`	0.4189	2
Marathi	`mr`	0.4717	3
Sindhi	`sd`	0.5502	1
Dogri	`doi`	0.5673	2
Bengali	`bn`	0.8173	6
Kashmiri	`ks`	0.8588	3
Punjabi	`pa`	0.8615	1
Assamese	`as`	0.8778	2
Tamil	`ta`	0.9083	2
Urdu	`ur`	0.9211	1
Telugu	`te`	0.9278	1
Malayalam	`ml`	0.9289	2
Gujarati	`gu`	0.9335	2
Odia	`or`	0.9346	2
Kannada	`kn`	0.939	3
Bodo	`brx`	0.9545	2
Konkani	`kok`	0.9583	2
Manipuri	`mni`	1.0	2

Per-language sample counts are small (n = 1–7); treat these as indicative ranking, not precise measurement.

3.5 Spoken language identification (recall)

language	code	recall
English	`en`	1.0
Hindi	`hi`	0.9231
Sanskrit	`sa`	0.5
Bengali	`bn`	0.2381

Zero recall: Assamese, Bhojpuri, Gujarati, Kannada, Konkani, Maithili, Malayalam, Manipuri, Marathi, Odia, Punjabi, Sindhi, Tamil, Telugu, Urdu. Spoken language-ID is effectively usable for Hindi and English only.

4 · Language coverage

Text OSINT — trained across Assamese, Bengali, Bodo, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu and Urdu, plus Maithili, Sanskrit, Dravidian code-mixed content and English. The text gates in §3.1 are aggregates over this mix.

Practical summary. The Hindi belt is the strongest region across all three modalities (Hindi/Bhojpuri/Maithili/Marathi speech, Devanagari reading, full text analysis). Urdu is the best-read script from images. The Dravidian languages and the north-east are well served in text but weak in speech; Odia is the weakest script for image reading.

5 · Known limitations

Stated plainly, because they matter operationally:

Media classifiers are weak. vision_hostility and audio_abuse score below their majority baselines; vision_misinfo is only marginally above baseline with poor fake-class recall. Treat the misinformation and hostility fields as weak signals on media inputs, never as decisions. They are considerably more reliable on text.
Spoken language-ID is near chance outside Hindi and English (§3.5).
Odia image text is not usable (CER 0.95).
General MCQ knowledge dropped from the stock model's 59.0 to 53.0 — the cost of heavy specialisation toward the OSINT contract.
Small evaluation samples per language in §3.4/§3.5.

These axes received five dedicated training rounds. The finding: full-parameter QAT training at learning rates that preserve text quality shifts classifier priors without creating discrimination, and rates that create discrimination damage everything else. This is a genuine frontier of the method, documented rather than papered over.

6 · Method

Quantisation-aware fine-tuning with a straight-through estimator. The forward pass computes fake_quant(W) on the deployment grid (int4, group 32, scale = amax / 7.5); the backward pass updates the bf16 master weight unchanged. The weights are therefore optimised for the exact numerical grid they are served on, rather than trained in bf16 and quantised afterwards.

Full-parameter, single artifact. All 10.90 B trunk parameters plus the 52.4 M modality embedders were trained directly. Because Gemma 4 Unified is early-fusion — 664 of its 677 tensors are a shared trunk processing text, vision and audio alike — training the trunk improves all three modalities at once. There is no adapter to load and no merge step, which also eliminates a whole class of deployment defect (a merge that shifts every quantisation scale).

Other techniques used: stratified round-robin corpus interleaving so every training window sees the full capability mix; selective layer unfreezing; classifier-token loss weighting; source-conditional field masking; and a final weight average of two sibling checkpoints of this same model (36 of 48 trunk layers were bit-identical by construction).

Selection. Checkpoints were chosen by capability gates, never by language-model loss — loss and capability moved in opposite directions three separate times during this programme.

7 · Datasets

Referenced under their respective licences; none are redistributed here. Evaluation slices were reserved before the corresponding training data was built, in every case.

Factuality, claims and hallucination RAGTruth · LLM-AggreFact · HaluEval · ClaimBuster · AVeriTeC · CLEF CheckThat! · X-Fact · Climate-FEVER · ASQA · Indic-Hallucination (as, bn, gu, hi, kn, ml, ta, te, ur)

Indian-language understanding Naamapadam NER (11 languages) · IndicSentiment (13 languages) · Dravidian CodeMix · Indic-RAG (bn, gu, hi, ta) · MMLU · Indic-MMLU · AI2-ARC

Misinformation (text) IFND · BanFakeNews-2.0 · HinFakeNews · Urdu Fake News

Vision IFND-multimodal (fake-news imagery) · INDICSTR12 full + crops (12-script Indic scene text) · Memotion / Memotion 2 · MMSoc Hateful Memes · BHM (Bengali hateful memes) · MET-Meme · Bangla document OCR · Sanskrit/Devanagari OCR · Urdu (Arabic-script) OCR

Audio ADIMA (abusive speech, 8 Indic languages) · IndicSUPERB · IndicVoices · IndicTTS · FLEURS · Assamese / Bengali / Gujarati / Hindi / Nepali speech corpora · per-language language-ID pools (16 languages)

A machine-readable manifest of every source, pinned by commit SHA, is in the companion state repository as sources.json.

8 · Provenance and reproduction

Training code, all corpora builders, the complete set of 30+ gate reports, the per-round campaign history and the full session log live in the companion repository Sandy-sys/aditi-ps18-state.

Programme: OSINT ARGUS — AI-based OSINT analysis and monitoring, Indian context.

This card was generated on 2026-08-25 directly from this artifact's own gate reports. Every number above is measured, not estimated.

Downloads last month: 18

Safetensors

Model size

12B params

Tensor type

I32

BF16

Model tree for Sandy-sys/osint-argus-gemma4-qat-w4a16

Base model

google/gemma-4-12B

Finetuned

google/gemma-4-12B-it

Finetuned

google/gemma-4-12B-it-qat-q4_0-unquantized

Quantized

(64)

this model