giorebil/Agumon-v1.0

Agumon v1.0: a 2B language model as an inline web application firewall

A LoRA adapter for MiniCPM5-2B-Base that reads one raw HTTP request and decides in a single forward pass whether a web application firewall should block it or allow it. It was built for one operating point: at most 0.01% of legitimate requests blocked.

Paper: A Small Language Model as an Inline Web Application Firewall, Giovanni Liber (arXiv submission in progress).

Results

Threshold 0.998675, frozen on a separate calibration split before any test was opened. Baselines: OWASP CRS 4.20 (ModSecurity, paranoia level 1, default) and open-appsec 1.1.36 (open-source agent, bundled basic model, default), on the same requests.

Attacks blocked (higher is better)

Benchmark	Agumon v1.0	OWASP CRS	open-appsec
Own held-out test (510 sites never seen in training)	98.1%	87.4%	91.9%
CIC-IDS2017 web attacks (XSS)	96.0%	96.0%	96.0%
HttpParams	97.4%	98.3%	99.9%
CSIC 2010 (SQLi/XSS subset)	95.1%	100%	100%
WAF Comparison Project (5,000-request sample)	85.3%	91.4%	99.7%
GoTestWAF	40.3%	45.9%	92.0%

Legitimate requests blocked (lower is better)

Benchmark	Agumon v1.0	OWASP CRS	open-appsec
Own held-out test (510 sites never seen in training)	0.0045%	11.9%	15.6%
CIC-IDS2017	0.003%	17.9%	5.2%
HttpParams	0.008%	0%	50.1%
CSIC 2010	0%	0.011%	2.8%
WAF Comparison Project (5,000-request sample)	0.034%	33.0%	26.6%
GoTestWAF (141 legitimate requests)	3.5% (5)	9.2% (13)	41.1% (58)
Real production traffic (10 min, one application, ~95,000 requests, private)*	0.002%	26.3%	34.7%

* Share of all requests blocked; the sample is nearly all legitimate and is not released.

At the model's false-positive rate on its own test (0.0045%), CRS has to raise its anomaly threshold to 100 and then blocks 0.02% of attacks.

Where it is weak. Encoded and obfuscated payloads (GoTestWAF, overlong-UTF-8 and %u path traversal), payloads beyond the visible window (it reads the first 2,000 characters of the URL, 1,000 of each header and 2,500 of the body), CRLF injection, LDAP injection, and scanner probes for unusual script names. WAF-A-MoLE's semantics-preserving SQL mutations evaded it for 20 of 40 SQL injections, almost always by scores just below the threshold. It is meant to sit next to rules and rate limiting, not to replace them.

Use

pip install torch transformers peft
python waf_classify.py

from waf_classify import WafClassifier
clf = WafClassifier()
decision, p = clf.decide('GET', '/search?q=shoes', {'Host': 'shop.example', 'User-Agent': 'Mozilla/5.0'}, '')

The prompt template, truncation and chat-template call in waf_classify.py are part of the model: change them and the scores change. p is the softmax over the two answer tokens block and allow at the last prompt position.

Threshold. threshold.json holds the frozen value. Scores of legitimate traffic differ between sites; before enforcing, score a sample of your own legitimate traffic and check how many requests it would block.

Latency. One H100 SXM, one request at a time, 300 validation requests (479 tokens on average), same decisions on all 300: 10.9 ms median with vLLM 0.31 at its defaults, 8.5 ms with vLLM tuned (in-process engine, CUDA graphs up to 2,048 tokens, two decision logits only), 6.2 ms with a minimal hand-written engine (one CUDA graph per 64-token bucket). Latency grows with request length; the 99th percentile (about 20 ms) comes from the longest requests.

Training

• Base: MiniCPM5-2B-Base, revision 96a57cd572a02506b4500f54427dca24970c1bac (Apache-2.0).
• LoRA rank 8, alpha 160, all linear projections of the last 16 of 42 layers (4.78M parameters).
• One epoch over 408,806 requests, loss on the single answer token; 64 rows per step, peak learning rate 2e-4.
• Data: browsing recorded by our own crawler on public sites; pseudonymized logged-in sessions from the author's own accounts; synthetic hard legitimate requests; 1,024 real legitimate requests an earlier model had blocked; attack payloads from SecLists (MIT), PayloadsAllTheThings (MIT), OWASP CRS regression tests (Apache-2.0) and nuclei-templates (MIT), grafted into recorded requests, each with its unchanged clean twin; 67 requests from the 30-day ModSecurity dataset by G. Lucz and B. Forstner (CC BY 4.0).

Intended use and limits

For research and as one layer of a web application firewall. Not a complete WAF: it does not see request rates, sessions, responses or protocol framing, so it cannot detect brute force, credential stuffing, request smuggling, authorization abuse or business-logic attacks. Publishing the weights lets an attacker search for evasions offline; deploy it with rule-based defences and keep the threshold under your own control.

A new, substantially larger training run is in progress.

Citation

@misc{liber2026agumon,
  title  = {A Small Language Model as an Inline Web Application Firewall},
  author = {Giovanni Liber},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/giorebil/Agumon-v1.0}},
  note   = {Agumon v1.0}
}