JaisGuard-Frozen-0.9B

Purpose and supported API

Arabic-script VALID/GIBBERISH detection based on recoverable meaning. Meaningful typos, informal language and dialects can be VALID; meaning cannot be equated with correctness, safety or whether a request can be fulfilled. Examples: كيف يمكنني تعلم اللغة العربية؟ and شنوة نعمل باش نتعلم البرمجة are meaningful requests. ضصثقفغعهخحجشس is a gibberish example. These examples illustrate intent, not a claim of perfect predictions.

This is a compact checkpoint package, not a merged backbone. Use the bundled JaisGuardClassifier API. Compatibility with AutoModelForSequenceClassification, Transformers pipelines, hosted Inference Providers or native vLLM is not claimed.

Installation and inference

uv venv --python 3.11 .venv
uv pip install --python .venv/bin/python -r requirements.txt
.venv/bin/python example.py --text 'كيف يمكنني تعلم اللغة العربية؟'
.venv/bin/python example.py --precision fp16 --text 'ضصثقفغعهخحجشس'

Run in the downloaded model directory. The base snapshot is downloaded as needed. For cached-only execution add --local-files-only. Review the base's trusted remote model code before running. From Python:

from jaisguard_model import JaisGuardClassifier
classifier = JaisGuardClassifier.from_pretrained(".", device="cuda", local_files_only=True)
print(classifier.predict("كيف يمكنني تعلم اللغة العربية؟"))
print(classifier.predict("ضصثقفغعهخحجشس", operating_point="default"))

For remote use, first snapshot_download this repository at an approved immutable revision, inspect its code, then import jaisguard_model.py from that directory. A standalone runtime plus tokenizer, adapter/head, config and integrity manifest are supplied; no training-repository checkout is required.

Architecture and selected checkpoint

Base: IFM/Jais-3-0.9B, revision e51941b37091287fee4934e3f8a799e0d3fef37f (currently redirected to inception42/Jais-3-0.9B). Hidden size 1536; final nonpadding token pooling; FP32 linear 1536→2 classifier with its original trained bias, 3,074 head parameters. Label order VALID=0, GIBBERISH=1. Right padding; raw text with special tokens, no chat prompt. Maximum 64 tokens, no truncation. Empty or unsupported-script inputs are errors, not GIBBERISH predictions.

Selected frozen v2 seed 43, epoch 5, validation loss 0.22949336469173431. Selection uses the first minimum validation loss, not test quality. Frozen backbone with trained classification head only.

Saved validation-selected threshold 0.9563547372817993; decision is p_gibberish >= threshold. The explicit default threshold is 0.5. The saved exploratory operating point targeted low VALID rejection on validation; it is not a production guarantee and is not optimized on the test benchmark.

Training data and configuration

These weights were trained on the original 1,500-row training export, with 250 validation rows. Original training contains 710 VALID and 790 GIBBERISH; validation contains 120 VALID and 130 GIBBERISH. Training has 1,367 human-reviewed and 133 audited-provisional labels. The dataset derivative has 1,498 train + 250 validation + 400 test = 2,148 rows and was not used to train these checkpoints. Two privacy-uncertain human-reviewed VALID Aya records were omitted only from that derivative. Their prior use in training remains a disclosed privacy limitation; no extraction or memorization audit is claimed. No original training texts or reviewer metadata are bundled. The HF dataset was private at preparation; authorized access may be required.

AdamW, weight decay 0.01, warmup ratio 0.1, gradient clipping 1.0. Frozen: batch 32, head LR 0.001, up to 30 epochs, patience 5. LoRA: microbatch 2, accumulation 8, adapter LR 0.0001, head LR 0.001, up to 8 epochs, patience 2. Complete portable hyperparameters and source hashes are in training_config.json and provenance.json. Original training recorded git fab3580 with uncommitted changes; a clean commit alone cannot exactly reconstruct that workspace.

Individual seed results — historical BF16, batch size 2

Previously examined, constructed 400-input benchmark: 188 VALID, 212 GIBBERISH. The following numbers belong to this checkpoint, not the three-seed average.

Operating point	Accuracy	Macro F1	GIBBERISH recall	VALID rejection
default	0.880000	0.879020	0.915094	0.159574
validation_locked	0.815000	0.812297	0.655660	0.005319

Three-seed experiment aggregates — separate from this model

Seeds 42/43/44; mean ± sample SD (ddof=1) on the same test inputs. This is variability across trained seeds, not independent test datasets.

Operating point	Accuracy	Macro F1	GIBBERISH recall	VALID rejection
default	0.879167 ± 0.003819	0.878114 ± 0.003553	0.916667 ± 0.016565	0.163121 ± 0.011073
validation_locked	0.827500 ± 0.019526	0.825631 ± 0.020333	0.685535 ± 0.036639	0.012411 ± 0.008125

Full precision metrics, confusion counts and separate scopes are in metrics.json.

Precision, validation and performance scope

Historical HF inference uses BF16 on the tested RTX 4050 when supported, with the classification head/bias always FP32. FP16 mode reproduces the historical HF adaptation: load the original model then cast the backbone to FP16. The unmerged adapter can round differently from a static merge. BF16 and FP16 are not assumed equivalent. The two requested serving checkpoints had known BF16/FP16 disagreements in the historical diagnostics.

The historical full-backbone vLLM exports passed matched FP16 HF/vLLM gates (zero default/locked mismatches, maximum/mean probability differences at most 0.02/0.005) and HTTP concurrency 1/4/8/16. Those results concern the exported pooling models, not native loading of this compact adapter/head package. No new vLLM server benchmark is claimed. See JaisGuard evidence.

Real local inference validation and its precision-specific acceptance results are recorded in validation.json. Check that file before making claims; pending or failed validation blocks publication. Tested hardware: RTX 4050 Laptop GPU, 6 GB, Python 3.11, Torch 2.14.1/CUDA 13.0, Transformers 5.20.2, PEFT 0.21.2. CPU is an explicit FP32 debugging path and is not a measured deployment or probability-parity claim.

Limitations and intended use

Experimental Arabic recoverability filtering, not a safety moderator, language identifier, factuality checker or general production-quality guarantee. Dialect coverage and synthetic corruption recipes are limited. The test benchmark was examined and informed iteration two; it is not an untouched independent final test. Test[319] is contained in test[204]. Parents for 260 transformed test requests are unavailable; 80 passthrough parents are inferable. Public-data training cannot exactly reproduce these original experiments. Confidence scores are not calibrated truth guarantees. Rejecting meaningful dialects is a known risk.

Licensing, integrity and publication status

Component scope: MIT for the project's loader and classifier components; Apache 2.0 for copied tokenizer assets and the separately downloaded base. See LICENSE and LICENSES/ATTRIBUTION.md, including upstream credits and the Apache license copy. Dataset attribution/licenses remain separate. No base weights or upstream modeling code are bundled. SHA-256 hashes are in manifest.json; they check integrity, not independent code authenticity. Publication is controlled separately; verify hashes and obtain owner approval before any upload.

Validated standalone batch-one test observations

These are new runtime observations on the same examined 400 inputs, separate from the historical batch-two quality table above. Both batch sizes reproduced their matching saved references exactly; a few reduced-precision threshold decisions differ across batch sizes. No thresholds or archived metrics changed.

Runtime	Default accuracy	Locked accuracy
BF16 batch 1	0.885000	0.815000
FP16 batch 1	0.880000	0.815000

All four real GPU parity checks (refactored/package × BF16/FP16) had maximum probability error 0.0 and zero mismatches at both thresholds. The additional package BF16 batch-two check reproduced all 400 original quality probabilities exactly. See validation.json for complete receipts.