mehedi4/SafeAge-TigerLLM-1B

SafeAge-adapted TigerLLM 1B

This repository releases the original SafeAge-trained TigerLLM baseline evaluated in SafeAge: Context-Sensitive Safety Classification for Bangla LLMs and Agentic Systems across Four Language Forms. It contains a LoRA adapter and trained binary classification head on the pinned TigerLLM-1B-it checkpoint, its tokenizer, a validation-selected threshold, loading code, and verification metadata.

This is an adapted comparison model, distinct from SafeAge Guard, whose backbone is Qwen3-1.7B. We credit TigerLLM to Nishat Raihan and Marcos Zampieri, its pretrained backbone to the upstream model developers, and LoRA to Hu et al.. The SafeAge authors' work is the dataset, task-specific supervised adaptation, and evaluation. No new architecture or adaptation algorithm is claimed.

The actual pinned TigerLLM checkpoint uses gemma3_text, as recorded in its configuration and the training code. The loader follows that configuration. The upstream repository's general model-family description mentions other architectures; that description does not change the architecture of this exact evaluated artifact.

Task and decision rule

Supply the complete raw request and relevant visible context. The model predicts allow or block under the SafeAge contextual decision policy, across Bangla, Bangla-English mixed text, Banglish, and aligned English reference text. Sensitive-topic discussion and authorized defensive tasks may be allowed; harmful facilitation and permission violations may be blocked. The labels do not measure age suitability.

The archived class names are 0 = benign (allow) and 1 = attack (block); the block class includes more than prompt injection. Predict block when the class-1 softmax probability is greater than or equal to 0.08. This threshold was chosen on validation and then frozen for test.

Loading and inference

This adapter is not a standalone full-base model. inference.py downloads the exact pinned public TigerLLM base, instantiates a two-label sequence classifier, and restores both the LoRA weights and saved score head through PEFT. Use the explicit loader rather than a text-generation pipeline. The inherited config.json architecture string is Gemma3ForCausalLM, while the adapter task is SEQ_CLS.

pip install -r requirements.txt
python inference.py --text "Explain how to protect my own account from phishing."

For a complete request stored in UTF-8:

python inference.py --text-file request.txt

With inference.py in the working directory:

from inference import load_guard, classify_texts

model = load_guard("mehedi4/SafeAge-TigerLLM-1B")
print(classify_texts(["Explain how to protect my own account from phishing."], model))

The loader follows the original run: FP32, SDPA attention, right padding, and maximum length 512 tokens. Right truncation is made explicit; the archived code used the library default without an override. No chat template is applied to the raw request. The retained chat-template file is tokenizer provenance, not part of the classification input pipeline. Padding to a multiple of 8 follows the original evaluation collator. The base classifier's temporary random-head warning can occur before PEFT restores the trained head; the unadapted base head must not be used for the reported results.

Training and data

SafeAge contains 24,508 requests from 6,127 four-form cases, partitioned into 22,096 training, 1,208 validation, and 1,204 test requests with connected groups kept within one partition. Validation and test are author-reviewed constructed cases from synthetic scenario expansions. Two authors annotated disjoint assigned subsets and repeatedly reviewed their own examples; no independent shared double-annotation agreement was measured.

Setting	Value
Base	md-nishat-008/TigerLLM-1B-it
Exact base revision	49c2001b637a84d997dbe34302fe527d70782097
Actual architecture	gemma3_text
Adaptation	LoRA plus trained binary score head
LoRA rank / alpha / dropout	16 / 32 / 0.05
LoRA targets	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable parameters	13,048,064
Optimizer / learning rate	AdamW / 0.0001
Effective batch size	32
Completed epochs / selected epoch	5 / 5
Training seed	42
Precision / maximum input length	FP32 / 512 tokens
Checkpoint selection	Highest validation macro F1 at threshold 0.5
Test operating point	Validation-selected threshold 0.08

Original held-out evaluation

On 1,204 SafeAge test requests, at threshold 0.08:

Metric	Value
Accuracy	95.10%
Macro F1	94.73%
Block precision	94.51%
Block recall	92.19%
Benign false positive rate	3.17%
Brier score	0.04758

Confusion counts: 732 true negatives, 24 false positives, 35 false negatives, and 413 true positives. Exact archived metrics, including the prespecified threshold 0.5 and per-language breakdown, are included under evaluation/. These are the original run's measurements; this release preparation did not rerun model inference. The paper's paired analysis does not establish a clear pooled difference between this adapted baseline and SafeAge Guard.

Artifact verification and limitations

All 11 recovered original checkpoint files were checked against the SHA256 manifest archived with the paper before release preparation. The published model/tokenizer files are byte-identical to those originals. The adapter SHA256 is:

761557d85f4f89af3bf378067a4ffa98e9b44443a958104fa61c9fce60a518c3

The adapter contains 364 LoRA tensors and a trained score.weight of shape [2, 1152], all FP32. The original run's saved reload check passed on eight validation examples, with a maximum probability difference of 3.01e-06. The check is archived under provenance/. Release preparation verifies identity and structure without claiming a new inference evaluation.

The reported experiment uses one seed and a constructed benchmark. Template regularities, translation style, tokenization, and truncation can influence results. No universal safety performance, independent annotation reliability, deployment validation, or full agent security is established. Applications must supply trustworthy authorization context; text claiming permission does not itself establish authorization. The fixed threshold does not guarantee the same error rates on other distributions.

License, modifications, and attribution

The pinned TigerLLM repository declares CC BY 4.0. Its original model card is retained under provenance/upstream_TigerLLM_README.md, and the CC BY 4.0 text is included. The SafeAge authors' adaptation contributions and original release materials are also provided under CC BY 4.0, to the extent the authors hold the relevant rights. The actual checkpoint is Gemma-derived: Gemma Terms of Use and its Prohibited Use Policy remain applicable to Gemma-derived materials. They are included in this repository along with the required NOTICE; this release does not replace or waive upstream terms. See LICENSE for the applicable scopes and copies.

The model modification is SafeAge-supervised LoRA adaptation plus a trained two-class classification head and validation-selected threshold. Base weights are downloaded separately and are not redistributed here. Tokenizer and configuration artifacts retain upstream provenance. This release is not endorsed by the TigerLLM authors or Google.

SafeAge authors: Md. Rehanul Bary and Mehedi Hasan Emon. Dataset: SafeAge release. TigerLLM citation: Raihan and Zampieri, ACL 2025. Historical experiment identifiers retain version strings for traceability; the dataset's display name is SafeAge. No conference acceptance or publication of the SafeAge manuscript is claimed.