downlz/x-abuse-shield-model

X Abuse Shield – Toxicity Model (ORT format)

This repo hosts the on-device model used by X Abuse Shield. The extension hides or blurs abusive posts and authors on x.com. It scores each post in the browser with this model and with word lists, and it doesn't send any post text to a server.

The model is Horizon-Labs/multilingual-toxicity-small, converted to the ONNX Runtime (.ort) format. ONNX Runtime Web can then run the weights in place, which lowers memory use in the browser. The weights and outputs are unchanged.

What was changed

Source	onnx/model_quantized.onnx from the upstream repo (int8 / q8)
Conversion	onnxruntime-node with graphOptimizationLevel: 'basic' and save_model_format: 'ORT'. basic keeps only operators that the WebAssembly runtime supports.
Quantization	None beyond the upstream file. The weights are already int8.
Output	model.ort, plus config.json, tokenizer.json, tokenizer_config.json and NOTICE.md, copied or written alongside it

The model:build script in the extension repo (scripts/build-model.ts) does this conversion. You can reproduce it with npm run model:build.

Files

The files sit under the upstream model ID path, because the extension resolves the model as <base URL>/<MODEL_ID>/…:

Horizon-Labs/multilingual-toxicity-small/
├── model.ort
├── config.json
├── tokenizer.json
├── tokenizer_config.json
└── NOTICE.md

The files total about 289 MB.

Model details

• Architecture: fine-tuned from mmBERT-small, a multilingual encoder with about 141M parameters
• Task: multi-label text classification, with an independent sigmoid per label
• Labels (7): toxicity, severe_toxicity, obscene, threat, insult, identity_attack, sexual_explicit (the same labels as Detoxify)
• Languages: English plus 33 other languages
• Max sequence length: 256 tokens
• Training data (upstream): Civil Comments (about 519K English comments), with about 391K machine-translated into 33 languages

For the full training details and upstream benchmarks, see the upstream model card. It reports a mean AUC of 0.988 on the Civil Comments English test set and an AUC of 0.835 on TextDetox across 15 languages.

Usage

In X Abuse Shield (hosted model)

Build the extension so that it fetches the model from this repo instead of bundling it:

WXT_BUNDLE_MODEL=false \
WXT_MODEL_URL=https://huggingface.co/downlz/x-abuse-shield-model/resolve/main/ \
npm run zip            # Chrome
# npm run zip:firefox  # Firefox

To check that the files are reachable:

https://huggingface.co/downlz/x-abuse-shield-model/resolve/main/Horizon-Labs/multilingual-toxicity-small/config.json

Session options for in-place loading

The extension creates the ONNX Runtime Web session with these options, so the weights run directly from the .ort bytes:

const IN_PLACE_SESSION_OPTIONS = {
  extra: {
    session: {
      use_ort_model_bytes_directly: '1',
      use_ort_model_bytes_for_initializers: '1',
      disable_prepacking: '1',
    },
  },
};

use_ort_model_bytes_for_initializers requires the ONNX Runtime patch in scripts/ort-in-place-plugin.ts.

If you want standard Transformers.js or Python usage with pipeline(...), use the upstream repo directly. This repo only holds the browser-optimised .ort build.

Performance in the extension

Build	Extension size	First-use download	Memory while loaded (Chrome)
Converted .ort, bundled	~331 MB	none	~500 MB
Converted .ort, hosted here	~28 MB	289 MB	~500 MB
Original ONNX from upstream	~28 MB	289 MB	~850 MB

• Scoring takes about 28–30 ms per post.
• The model loads only while an x.com tab is open and unloads after 5 minutes idle.
• On Firefox the model runs single-threaded, because extensions there have no SharedArrayBuffer. End-to-end speed is similar.

Calibration (extension pipeline)

The extension combines the model with word lists for English, Hindi (Devanagari) and Hinglish. The results below are for the whole pipeline, not for the raw model. They come from a small labelled set of 68 posts (44 abusive and 24 clean: English, Hindi, Hinglish and other languages), using the Balanced preset:

Threshold	Value
toxicity	0.75
insult	0.70
threat	0.40
identity_attack	0.50

Preset (word lists + model)	Abusive caught	Clean flagged
Relaxed	28 / 44	1 / 24
Balanced	36 / 44	2 / 24
Strict	39 / 44	7 / 24

For Balanced, that works out to about 94.7% precision, 81.8% recall and 85.3% accuracy. The set is very small, so treat these as indicative only. The full report is in docs/CALIBRATION.md.

Limitations

• Emoji: the model barely reacts to emoji, so the extension handles emoji insults with separate rules.
• Sarcasm and implicit abuse: these are often missed. Identity terms or quoted language may be over-flagged.
• Low-resource languages: accuracy is weaker. Non-English training data was machine-translated, so slurs specific to a language are under-represented.
• Subjective labels: labels reflect the judgments of English-language annotators and may not match every community's norms.
• Not for decisions about people: don't use this model as the sole basis for decisions about people. It is intended to filter what a user sees on their own feed.

License and attribution

• Model: Apache-2.0, inherited from Horizon-Labs/multilingual-toxicity-small. The included NOTICE.md records the modification (conversion to ONNX Runtime format from onnx/model_quantized.onnx).
• Extension code: MIT. See downlz/x-abuse-shield.