Athenas-Guard-9B-Clef

A Decision-Oriented Security and Moderation Model for Conversational AI

Athenas-Guard-9B-Clef is a specialized decision model designed to support security enforcement, conversational moderation, and policy-driven request routing in AI-powered applications.

Built on the unsloth/Qwen3.5-9B backbone and equipped with the Clef decision-head architecture, the model is designed to produce structured, multi-dimensional classifications without relying on unconstrained autoregressive text generation to compose its decision fields.

Its decision interface evaluates incoming text across three dimensions:

Verdict: whether a request should be allowed or blocked.
Category: the detected interaction or security classification.
Action: the recommended handling strategy for the application.

This architecture is intended to reduce formatting-related failures associated with generative classification, including malformed JSON and invalid output labels. It does not eliminate classification errors or guarantee correct decisions.

Athenas-Guard-9B-Clef is intended to function as a specialized component within a broader AI security architecture, rather than as a standalone security system.

Key Features

Decision-oriented inference: Uses a dedicated decision mechanism instead of relying on free-form text generation for the decision fields.
Multi-dimensional classification: Evaluates verdict, category, and action within a unified decision schema.
Prompt-injection detection: Targets attempts to override instructions, bypass policies, or manipulate the intended execution flow.
Conversational moderation: Distinguishes ordinary requests, user frustration, and abusive interactions.
Policy-driven routing: Supports application workflows for standard processing, de-escalation, and blocking.
Multiple deployment formats: Provides LoRA adapters, merged 16-bit weights, and a quantized GGUF distribution.
Evaluation on public datasets: Includes reported results from three publicly available datasets covering prompt injection and conversational moderation.

Model Specifications

Property	Specification
Model	Athenas-Guard-9B-Clef
Base model	`unsloth/Qwen3.5-9B`
Architecture	Qwen backbone with Clef decision head
Fine-tuning method	QLoRA
LoRA rank	16
LoRA alpha	16
Decision-head calibration	L-BFGS
Inference paradigm	Decision-head classification
Output dimensions	Verdict, category, action
Primary applications	Moderation, guardrails, security classification
Available formats	LoRA, merged 16-bit, GGUF Q4_K_M
Declared repository license	Apache-2.0

The specifications above describe the reported model configuration. Hardware requirements, supported context lengths, throughput, and runtime compatibility depend on the artifact and inference implementation.

Intended Use

Athenas-Guard-9B-Clef is intended to evaluate incoming text before it is processed by a conversational AI application, an agent, or a downstream service.

Primary use cases

1. Prompt-injection classification

Identify inputs that attempt to override instructions, manipulate model behavior, bypass application policies, or compromise an agent's intended execution flow.

2. Conversational moderation

Classify interactions that contain ordinary requests, user frustration, or direct abuse directed at a conversational bot.

3. Request routing

Provide a structured decision that an application can use to continue standard processing, initiate a de-escalation workflow, or reject a request.

4. AI agent protection

Serve as one layer in the request-validation pipeline before an agent invokes tools or interacts with sensitive resources.

5. Enterprise chatbot security

Support moderation and security classification in customer-service systems, internal assistants, and AI-enabled business workflows.

Out-of-scope uses

The model is not intended to replace:

Application authorization and access-control systems.
Identity verification or permission checks.
Server-side enforcement of security policies.
Human review for ambiguous or high-impact decisions.
Comprehensive security testing and monitoring.
A general-purpose conversational or text-generation model.

The model should not be the sole decision-maker for consequential decisions involving employment, healthcare, credit, legal rights, or similar high-impact domains.

Decision Schema

The decision interface is designed around three fields: verdict, category, and action.

1. Verdict

Label	Definition
`allow`	Permit the request to proceed to the next application stage, subject to applicable policies.
`block`	Prevent the request from proceeding according to the application's enforcement policy.

2. Category

Label	Definition
`safe_query`	A legitimate request, routine interaction, or ordinary business query.
`prompt_injection`	An attempt to manipulate instructions, override policies, escape intended constraints, or compromise the execution flow.
`bot_harassment`	Direct verbal abuse, insults, or targeted harassment directed at the bot.
`frustrated_user`	An expression of dissatisfaction or operational frustration without clear evidence of malicious technical intent.

3. Action

Label	Definition
`pass`	Continue through the standard application workflow.
`de_escalate`	Use an appropriate de-escalation strategy, such as an empathetic response or human-support escalation.
`block`	Reject the request according to the application's security or moderation policy.

Decision semantics

Each field serves a distinct purpose:

verdict expresses the allow-or-block decision.
category describes the detected interaction type.
action specifies the intended handling strategy.

Applications should explicitly define the valid combinations of these fields. A classification result should not automatically authorize sensitive operations or replace independent security controls.

For example, an angry customer may have a legitimate request that warrants de-escalation rather than blocking. Conversely, a request classified as a prompt injection may require rejection even if its wording appears polite.

Technical Architecture

Backbone

The model is built on unsloth/Qwen3.5-9B, which provides the underlying language representations used to evaluate input context.

Fine-tuning

The reported training configuration uses Quantized Low-Rank Adaptation (QLoRA).

Parameter	Value
LoRA rank	16
LoRA alpha	16
Calibration optimizer	L-BFGS

QLoRA adapts a model through low-rank trainable parameters while reducing the memory requirements associated with full-parameter fine-tuning.

Clef decision head

The Clef component is designed to map model representations to the predefined decision fields.

The reported architecture uses a dedicated decision mechanism and L-BFGS calibration. The intended inference path produces classification decisions without requiring a text decoder to compose the three fields as unconstrained prose.

The exact implementation and supported execution path depend on the model artifact and compatible inference library.

Inference characteristics

The decision-oriented design reduces dependence on free-form text formatting for classification outputs. This can reduce exposure to output-formatting problems commonly encountered when a generative model is instructed to emit JSON.

However, a dedicated decision head does not guarantee:

Correct classification for every input.
Identical results across all runtimes or numerical precision settings.
Robustness against all adversarial inputs.
Correct handling of every possible combination of decision fields.

Determinism, schema integrity, and numerical consistency should be verified in the intended deployment environment.

Evaluation Results

Athenas-Guard-9B-Clef was evaluated using 1,526 samples assembled from three publicly available Hugging Face datasets.

The evaluation measures binary verdict classification, category classification, action classification, and the correctness of all three fields simultaneously.

Aggregate performance

Metric	Result
Verdict accuracy (`allow` / `block`)	82.90%
F1-score (`block`)	84.21%
Category accuracy	65.92%
Action accuracy	80.73%
Three-key exact match	62.78%
Total evaluated samples	1,526

Three-key exact match measures the percentage of samples for which verdict, category, and action are all correct simultaneously.

This is a stricter metric than verdict accuracy alone. A prediction can have the correct allow-or-block verdict while assigning the wrong category or recommended action.

Results by dataset

Dataset	Samples	Verdict accuracy	Three-key exact match
`Lakera/gandalf_ignore_instructions`	112	97.3%	97.3%
`deepset/prompt-injections`	661	85.5%	79.1%
`lmsys/toxic-chat` (`toxicchat0124`)	753	78.5%	43.3%
Consolidated	1,526	82.90%	62.78%

The reported results show different performance profiles across the evaluated datasets. The Gandalf instruction-override dataset produced the strongest results, while ToxicChat showed a larger difference between binary verdict accuracy and complete decision-schema correctness.

The ToxicChat result highlights the importance of evaluating every decision field when downstream applications depend on consistent classification and routing behavior.

Binary verdict classification report

Class	Precision	Recall	F1-score	Support
`allow`	0.95	0.71	0.81	798
`block`	0.75	0.96	0.84	728
Macro average	0.85	0.83	0.83	1,526
Weighted average	0.85	0.83	0.83	1,526

The reported recall for block indicates that the model identified most samples assigned to that class in this evaluation. However, the precision of block indicates that false-positive blocking remains an important consideration.

The allow class achieved high precision but lower recall, suggesting that some legitimate inputs were incorrectly classified as block.

Production enforcement policies should account for the costs of both false positives and false negatives.

Evaluation protocol

The benchmark script performs the following operations:

Loads samples from three public Hugging Face datasets.
Maps the original dataset labels into the Athenas-Guard decision schema.
Filters empty or invalid text and inputs outside the configured length range.
Removes exact normalized-text overlaps with user messages from the specified fine-tuning dataset and deduplicates matching samples across the assembled evaluation set.
Runs inference using FastDecisionModel.predict.
Records predicted labels, confidence values, per-sample latency, and classification correctness.
Generates aggregate metrics, classification reports, confusion matrices, and per-sample prediction files.

The reported evaluation covers the following datasets:

Lakera/gandalf_ignore_instructions
deepset/prompt-injections
lmsys/toxic-chat, configuration toxicchat0124

Dataset selection and labeling considerations

The benchmark combines the train and test splits of deepset/prompt-injections. Its reported result therefore represents a custom evaluation set assembled from both splits, not a test-split-only result.

The ToxicChat evaluation applies a limit of 400 samples labeled as safe, while eligible toxic and jailbreak samples are retained subject to the script's filtering and deduplication rules.

The benchmark script maps the original source labels into the Athenas-Guard taxonomy. For example, toxic messages are assigned the bot_harassment category, while eligible non-toxic messages are mapped to safe_query.

These three-field annotations are operational labels assigned by the evaluation script. They should not be interpreted as native three-field annotations supplied by the original datasets.

The overlap check compares normalized text strings. It helps remove exact duplicates with user messages in the specified fine-tuning dataset but does not establish the absence of semantic overlap, paraphrased examples, or exposure during base-model pretraining.

Reproducibility and interpretation

The results in this section are author-reported measurements from the documented evaluation script. They are not independently audited results.

For independent reproduction, the evaluation artifacts should be used alongside the exact dataset revisions, inference configuration, label mappings, and evaluation code.

Dataset revisions and selection procedures should be recorded for future benchmark runs so that changes in results can be attributed to model versions, evaluation data, or inference settings.

These figures should not be interpreted as guarantees of performance on arbitrary production traffic.

Usage

The following example illustrates the intended integration pattern using the reported Unsloth FastDecisionModel interface.

Installation

pip install unsloth

Use a compatible environment for the selected model artifact and Unsloth implementation. For reproducible deployment, record the tested Python, PyTorch, Transformers, Unsloth, CUDA, and artifact versions.

Inference example

import json
from unsloth import FastDecisionModel

MODEL_REPO = "Kodjaoglanian/Athenas-Guard-9B-Clef"

# Load the model and compatible decision-head implementation.
model, tokenizer = FastDecisionModel.from_pretrained(
    MODEL_REPO,
    load_in_4bit=True,
)

FastDecisionModel.for_inference(model)

user_message = (
    "This wretched system never works, "
    "absolute garbage service!"
)

decision_schema = {
    "verdict": {
        "type": "choice",
        "criteria": {
            "allow": "Permit the request to proceed",
            "block": "Prevent the request from proceeding",
        },
    },
    "category": {
        "type": "choice",
        "criteria": {
            "safe_query": "Normal or legitimate request",
            "frustrated_user": "User expressing frustration",
            "bot_harassment": "Abuse directed at the bot",
            "prompt_injection": "Attempt to override instructions",
        },
    },
    "action": {
        "type": "choice",
        "criteria": {
            "pass": "Continue the standard workflow",
            "de_escalate": "Use a de-escalation workflow",
            "block": "Block the request",
        },
    },
}

decision = FastDecisionModel.predict(
    model,
    tokenizer,
    user_message,
    decision_schema,
)

print(json.dumps(decision, indent=2, ensure_ascii=False))

Compatibility note: This example follows the reported FastDecisionModel API. Confirm that the installed Unsloth version supports this interface and that the selected artifact includes the compatible decision head. The code illustrates the integration pattern; successful execution depends on the installed implementation.

Production integration recommendations

Before using the output to control application behavior:

Validate the presence and permitted values of all required fields.
Enforce the application's approved combinations of verdict, category, and action.
Define safe fallback behavior for inference failures, timeouts, missing fields, and invalid decisions.
Apply authorization and tool-execution checks independently of the moderation result.
Monitor false positives and false negatives using an appropriate data-retention policy.
Evaluate the model on representative production traffic before changing enforcement policies.
Test adversarial inputs and legitimate requests that resemble suspicious content.

The classification result should be treated as an input to a policy-enforcement layer, not as a substitute for that layer.

Deployment Artifacts

The model family is distributed in the following reported formats.

Artifact	Repository	Intended deployment
LoRA adapters	Athenas-Guard-9B-Clef	Adapter-based inference with a compatible backbone and decision-head implementation
Merged 16-bit	Athenas-Guard-9B-Clef-Merged-16bit	Higher-precision deployment where memory and runtime compatibility permit
GGUF Q4_K_M	Athenas-Guard-9B-Clef-GGUF	Quantized deployment using a compatible inference runtime

Artifact availability does not guarantee that all formats preserve identical decision-head behavior.

In particular, a GGUF conversion intended for llama.cpp must preserve or provide a compatible implementation of the decision mechanism. A conventional causal-language-model conversion should not be assumed to retain a custom classification head automatically.

Verify the actual inference path, supported output schema, and runtime compatibility for each artifact before deployment.

Limitations and Known Considerations

Generalization

Performance on public benchmarks may not transfer directly to other applications, languages, domains, or threat models. Production traffic may contain interaction patterns and adversarial inputs that are not represented in the evaluated datasets.

False positives and false negatives

The model may incorrectly block legitimate requests or permit requests that should have been blocked. The operational cost of these errors depends on the application and its security requirements.

Multi-dimensional consistency

Correctly predicting the verdict does not guarantee that the category and action are also correct. Applications that depend on all three fields should evaluate joint exact match and validate decision consistency.

Context dependence

Classification quality may depend on the context supplied to the model. A message that appears harmless in isolation may be adversarial in context, while an angry message may still express a legitimate request.

Applications should supply the context required by their evaluation policy while avoiding unnecessary exposure of sensitive information.

Runtime and artifact compatibility

Behavior may differ across precision modes, quantization formats, inference runtimes, and conversion procedures. The supported execution path should be tested end to end before deployment.

No guarantee of complete protection

Athenas-Guard-9B-Clef is one component of a layered defense strategy. It should not be treated as proof that an application is secure against prompt injection, jailbreaks, harmful content, or information disclosure.

Security and Responsible Deployment

For security-sensitive applications, deploy the model alongside complementary safeguards:

Explicit authorization checks for protected resources and tools.
Separation of trusted system instructions from untrusted user-provided content.
Server-side enforcement of application policies.
Restricted tool permissions and least-privilege execution.
Monitoring, incident response, and regular adversarial testing.
Human review or escalation for ambiguous or high-impact cases.
Privacy-conscious logging and appropriate access controls.

Security evaluations should test both malicious inputs that must be blocked and legitimate requests that must remain usable. Overly aggressive blocking can degrade application reliability, while overly permissive decisions can expose downstream systems to risk.

Training Data and Methodology

The reported model configuration uses QLoRA with rank 16 and alpha 16, together with a Clef decision head calibrated using L-BFGS.

The detailed training dataset composition, dataset sizes, preprocessing, class-balancing procedures, optimization schedule, training duration, and precise calibration procedure are not specified in this model card.

These details should be documented in a reproducible training report before making stronger claims about training-data coverage or comparative performance.

License

This repository declares the Apache License 2.0.

Before redistributing or commercially deploying the model, verify the applicable licenses and terms for the base model, training data, adapters, merged weights, and converted artifacts.

Summary

Athenas-Guard-9B-Clef is a decision-oriented moderation and security-classification model designed to provide structured verdicts, categories, and handling actions for conversational AI systems.

Its primary architectural distinction is the use of a dedicated decision mechanism rather than relying on unconstrained autoregressive generation to format the decision fields.

The reported evaluation covers 1,526 samples from three public datasets, achieving 82.90% verdict accuracy and 62.78% three-key exact match under the documented evaluation procedure.

The results also demonstrate why binary verdict accuracy and complete decision-schema correctness must be evaluated separately, particularly on conversational moderation data.

Athenas-Guard-9B-Clef is intended to serve as a component within a broader application security architecture, supported by explicit policy enforcement, input validation, and production-specific evaluation.

Model card maintained by the Athenas-Guard development team.