PhishScout

Catch phishing URLs before they catch you — 131.5 KB of on-device LightGBM. No GPU, no cloud, no API calls. 35 URL features, trained on PhishTrap's auto-refreshing 20K-URL dataset.

Priorities: Quality > Size > Speed

Trained on: saidutta69/PhishTrap — auto-refreshed every 6 hours from OpenPhish, Phishing.Database, PhishStats, and Tranco

---

Model Overview

PhishScout is a tiny gradient-boosted tree model that classifies URLs as phishing or legitimate using URL structure alone — no page content, no third-party lookups, no network calls at inference time. The full model is a single 131.5 KB ONNX file that runs anywhere ONNX Runtime runs (Python, Node, browser via ONNX Runtime Web, mobile).

Property	Value
Architecture	LightGBM (gradient boosting)
Config	60 trees, max_depth 8, 31 leaves
Features	35 (URL-structural + semantic signals)
Model size	131.5 KB (ONNX, opset 15)
Training time	~0.5s on CPU
Inference latency	~9 µs/URL (median, CPU)
ONNX parity	100% (predictions match LightGBM exactly)
License	MIT

---

Use Cases

Primary — Real-time, on-device URL screening

• Browser extension — screen every URL before navigation/click, entirely locally. Zero data leaves the device; works offline; no privacy trade-offs.
• Email security clients — pre-click link inspection in desktop/mobile mail apps; flag or block suspicious links before the user opens them.
• SMS / messaging link scanners — instant check of short links from unknown senders in WhatsApp, Telegram, and SMS apps.
• QR code scanner apps — decode -> classify -> warn, before opening the URL.

Secondary — Infrastructure integration

• Email gateway & firewall plugins — lightweight first-pass filter; hand off high-probability URLs to heavier sandboxing for deep inspection.
• Proxy / DNS-layer filtering — classify requested URLs at the edge with single-digit-microsecond latency.
• SIEM / SOAR enrichment — batch-classify URLs from logs, tickets, and incident data to prioritize triage.
• Security awareness training — generate realistic phishing examples or explain why a URL is suspicious (feature diagnostics).

Tertiary — Research & data pipelines

• URL crawler triage — pre-filter the streams of phishing feeds, mirroring how the PhishTrap dataset is built.
• Phishing trend measurement — lightweight prevalence estimation across URL corpora.
• Benchmark baseline — a compact, reproducible reference model for comparing larger detectors.

---

Performance

Final model (test split, held out, seed 42)

Metric	Value
Accuracy	0.8697
Precision	0.9033
Recall	0.8254
F1	0.8626
ROC AUC	0.9369

Confusion matrix (test, 2,993 URLs): TP=1224, FP=131, FN=259, TN=1379.

Determinism & stability

• 10-seed variance on test: F1 0.8626 +/- 0.0000, AUC 0.9369 +/- 0.0000 — fully deterministic.

Robustness

Perturbation	F1
Clean	0.8626
5% extreme-value corruption	0.8080
Gaussian noise sigma=0.5	0.7097

Threshold behavior

• Default threshold 0.5 is near-optimal.
• F1-optimal threshold tuned on val: t=0.36 -> test F1 0.8529 (trades precision for recall: rec 0.869, prec 0.818).

---

Model Selection — Full Benchmark

PhishScout is the winner of a 34-config Pareto sweep plus a 12-model baseline benchmark. Every candidate: 5-fold CV on train, held-out test eval, ONNX export, size, and parity.

Baseline candidates (16 features)

model	CV F1	test F1	acc	prec	rec	AUC	size KB
gaussian_nb	0.5571	0.542	0.681	0.939	0.381	0.863	1.8
logistic_regression	0.816	0.816	0.825	0.854	0.781	0.901	0.7
decision_tree d8	0.8401	0.846	0.853	0.884	0.811	0.919	14.0
random_forest 10x6	0.8246	0.829	0.840	0.879	0.784	0.907	32.4
extra_trees 10x6	0.7953	0.801	0.812	0.844	0.761	0.880	14.9
adaboost 30x3	0.8305	0.826	0.834	0.861	0.794	0.903	33.7
gradient_boosting 30x3	0.8283	0.832	0.841	0.871	0.796	0.909	16.8
lightgbm 30x6L15	0.8415	0.846	0.854	0.890	0.805	0.921	31.9
xgboost 30x4	0.8367	0.840	0.848	0.874	0.808	0.914	28.0
svm rbf	0.8324	0.836	0.847	0.892	0.787	0.914	444.4
knn k=5	0.8306	0.833	0.841	0.865	0.805	0.897	889.4

Pareto sweep (16 vs 35 features, 34 configs)

Pareto frontier (F1 vs ONNX size):

model	feats	F1	acc	AUC	size KB
dt_d8	full	0.8446	0.8527	0.9208	11.5
dt_d8	base16	0.8456	0.8533	0.9190	14.0
xgb_30x4	full	0.8472	0.8547	0.9239	28.1
lgbm_30x6L15	full	0.8502	0.8567	0.9270	31.9
gb_50x4	full	0.8506	0.8573	0.9294	53.9
lgbm_60x8L31	base16	0.8545	0.8620	0.9285	131.5
PhishScout (lgbm_60x8L31)	full	0.8626	0.8697	0.9369	131.5

Key findings:

• Everything above 131.5 KB does worse (overfitting): lgbm_100x10L63 0.8589 @ 447 KB, rf_100x12 0.8566 @ 2388 KB, lgbm_200x12L127 0.8477 @ 1894 KB. 131.5 KB is the unambiguous elbow.
• The 35-feature set beats 16 features at every size: +0.018 F1 for the 131.5 KB LightGBM config.
• The expanded features (keywords, brand impersonation, punycode, ports, hex-IP, encoding tricks) are what unlock the quality jump — all 16 base features verified byte-exact against the stored dataset.

Why not the smaller models?

The 14 KB decision tree (F1 0.846) is the best sub-50 KB option, but PhishScout beats it statistically significantly: McNemar p=0.0002 (lgbm-only-right=115 vs tree-only-right=64), paired bootstrap 95% CI of F1 difference [+0.0084, +0.0271] — entirely positive. +0.018 F1 / +0.017 AUC for 9x the size of a file that is 0.1% of any app.

---

Features (35)

Base 16 — URL structure

Feature	Description
url_length	Total URL character count
hyphen_count	Number of hyphens
digit_count	Number of digits
subdomain_count	Number of subdomains
trusted_tld	1 if TLD is .com/.org/.net/.edu/.gov
protocol_exists	1 if http/https present
special_char_count	Special characters (@, -, _, ., etc.)
entropy	Shannon entropy of URL string
path_depth	Number of path segments
domain_length	Length of domain portion
is_domain_ip	1 if domain is an IP address
has_at_symbol	1 if @ present
has_double_slash_redirect	1 if // appears after protocol
tld_length	Length of top-level domain
query_param_count	Number of query parameters
path_length	Length of path portion

Expanded 19 — Semantic signals

Feature	Description
keyword_count	Occurrences of 28 suspicious keywords (login, verify, secure, wallet, refund, ...)
brand_in_host	1 if one of 33 well-known brands in hostname
brand_in_path	1 if a brand in the path
brand_count	Total brand mentions
has_punycode	1 if hostname uses xn-- punycode
has_nonstandard_port	1 if explicit port other than 80/443
percent_encoding_count	Count of %-escapes
has_hex_ip	1 if hex-encoded IP (0x...) present
uppercase_ratio	Fraction of uppercase characters
www_count	Occurrences of "www."
max_digit_run	Longest consecutive digit run
max_label_len	Longest hostname label
host_label_count	Number of hostname labels
has_domain_in_path	1 if a domain-like token appears in the path
query_length	Query string length
equals_count	Number of = in query
sensitive_extension	1 if ends with .exe/.scr/.apk/.zip/.jar/...
has_multiple_tlds	1 if multiple TLD-like tokens with slashes
consecutive_punct	1 if 3+ punctuation chars in a row

---

Usage

Python (ONNX Runtime)

import onnxruntime as ort
import numpy as np

sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
features = np.array([[ /* 35 features, order below */ ]], dtype=np.float32)
pred, proba = sess.run(None, {"input": features})
# pred == 1 -> phishing, pred == 0 -> legitimate

Feature order (35):

url_length, hyphen_count, digit_count, subdomain_count, protocol_exists,
special_char_count, entropy, path_depth, domain_length, is_domain_ip,
has_at_symbol, has_double_slash_redirect, tld_length, trusted_tld,
query_param_count, path_length, keyword_count, brand_in_host, brand_in_path,
brand_count, has_punycode, has_nonstandard_port, percent_encoding_count,
has_hex_ip, uppercase_ratio, www_count, max_digit_run, max_label_len,
host_label_count, has_domain_in_path, query_length, equals_count,
sensitive_extension, has_multiple_tlds, consecutive_punct

Browser (ONNX Runtime Web)

Load model.onnx into ort.InferenceSession with WebAssembly backend — runs fully in-browser, no server.

---

Verification & Reproducibility

• ONNX parity — ONNX Runtime predictions match LightGBM on 100% of 2,000 sampled test URLs.
• Deterministic — identical results across 10 seeds (LightGBM with fixed seed).
• No leakage — train/val/test splits are disjoint (verified in the dataset build); test set untouched during model selection except for the final evaluation.
• Reproducible — fixed seed 42; full training and benchmark code open source at github.com/instax-dutta/PhishScout (train_phishscout.py, benchmark_models.py, sweep_models.py, deep_benchmark.py).

---

Limitations

• URL-only signals — no page content, TLS certificate, or behavioral analysis. A URL that looks benign but hosts phishing content can be missed.
• Bare trusted-domain phishing — the hardest case for any URL-only model: phishing hosted on the actual root of a trusted domain (e.g. paypal.com/... after compromise) is largely invisible to structure-based features. 63-69% of false negatives are URLs with trusted TLDs and no subdomains.
• English-centric keyword/brand lists — the 28 keywords and 33 brands are English/Western-market oriented; detection quality degrades for other languages and smaller regional brands.
• Feed dependence — training data quality is inherited from PhishTrap's live feeds; retrain on the latest dataset snapshot for current campaigns (pipeline refreshes every 6 hours).
• No temporal adaptation on its own — the model does not adapt between retrains; novel phishing techniques appear between dataset refreshes.
• Threshold sensitivity — at default 0.5 the model favors precision (0.903); use t=0.36 for higher recall at lower precision, or calibrate per use case.

---

Training Data

Trained on PhishTrap v2 (build 20260731T085613Z): 19,954 URLs, balanced 50/50, 70/15/15 split. Phishing URLs from PyFunceble-verified feeds (Phishing.Database, OpenPhish, PhishStats); legitimate URLs from Tranco top 10K. See the dataset card for full provenance.

---

Citation

@misc{saidutta69_2026_phishscout,
  author = {Sai Dutta Abhishek Dash},
  title = {PhishScout: Tiny On-Device Phishing URL Detector},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/models/saidutta69/PhishScout}},
  note = {Trained on the auto-refreshing PhishTrap dataset}
}

---

Built on PhishTrap + Phishing.Database, OpenPhish, PhishStats, and Tranco. MIT licensed.