PSompong/llama-hredd-adverse-text-distilbert-onnx

Llama-distilled HREDD adverse-text detector (DistilBERT ONNX student)

A compact, browser-ready text classifier that reads adverse media and allegations and predicts a human-rights or environmental harm category for supplier due-diligence screening. It is the offline student reader of the risk-screening tool - Supplier diligence · HREDD, distilled from a Llama 3.3 70B teacher and int8-quantized to run entirely in the browser, with no API key and no server.

Screening aid, not a verdict. This model helps prioritise where human due-diligence attention goes. It does not clear, terminate, or blacklist suppliers, and every High or Critical outcome is reviewed by a person. It approximates its 70B teacher; it does not beat it (see Evaluation below). Built with Llama (see Provenance and licensing below).

---

What it does

• Input: a short adverse-text snippet (news headline or allegation, English).

• Output: one of 16 classes, a trained subset of the SusTech Supplier Risk Screener's 18-category HREDD harm taxonomy:

  • biodiversity_loss
  • child_labour
  • deforestation
  • discrimination
  • fatal_incident
  • forced_labour
  • freedom_of_association
  • gbvh
  • hazardous_waste
  • none
  • osh_nonfatal
  • retaliation
  • toxic_pollution
  • trafficking
  • wage_theft
  • water_depletion

  The taxonomy also defines two band-level categories, elevated and limited, that sit outside the trained head. They are not among this model's possible outputs and can never be emitted by it.

• Tier: the risk tier (Low / Medium / High / Critical) is derived downstream by a severity rubric in the screener. The model emits a harm category, not a tier.

Usage

transformers.js (primary surface)

import { pipeline } from "@huggingface/transformers";

const detect = await pipeline(
  "text-classification",
  "PSompong/llama-hredd-adverse-text-distilbert-onnx"
);

const out = await detect(
  "Auditors documented withheld wages and confiscated passports at the plant."
);
console.log(out); // e.g. [{ label: "forced_labour", score: 0.91 }]

The repo ships an int8-quantized ONNX graph at onnx/model_quantized.onnx for in-browser inference.

Python (onnxruntime)

from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer
import onnxruntime as ort
import numpy as np
import json

repo_id = "PSompong/llama-hredd-adverse-text-distilbert-onnx"

model_path = hf_hub_download(repo_id=repo_id, filename="onnx/model_quantized.onnx")
label_map_path = hf_hub_download(repo_id=repo_id, filename="label_map.json")

tokenizer = AutoTokenizer.from_pretrained(repo_id)
session = ort.InferenceSession(model_path)

with open(label_map_path, encoding="utf-8") as f:
    labels = json.load(f)  # 16-label list, alphabetical

text = "Auditors documented withheld wages and confiscated passports at the plant."
inputs = tokenizer(text, return_tensors="np")
input_names = {i.name for i in session.get_inputs()}
onnx_inputs = {k: v for k, v in inputs.items() if k in input_names}

logits = session.run(None, onnx_inputs)[0]
predicted_id = int(np.argmax(logits, axis=-1)[0])
print(labels[predicted_id])  # e.g. "forced_labour"

The int8-quantized graph lives at onnx/model_quantized.onnx; label_map.json gives the label order used to map logits back to class names.

Model details

Field	Value
Base model	distilbert-base-uncased (about 66M params, English)
Teacher	Groq Llama 3.3 70B Instruct (llama-3.3-70b-versatile), 5-seed modal labels
Format	ONNX, int8-quantized, about 65 MB (vs about 268 MB full precision)
Task	16-class head over the 18-category HREDD harm taxonomy
Tier derivation	Downstream, via the shared severity rubric (outside this model)
Random seed	42
Training	Free Colab T4 GPU, inverse-frequency class weights, macro-F1 model selection

Training data

The model was distilled on a 961-row, three-source corpus: synthetic HREDD news snippets, 180 targeted synthetic rows (20 per category, deterministic templates, seed 42), and 54 de-identified curated real cases, each carrying a generic public descriptor. The 30-case evaluation gold set is held out of training entirely. Real company names are never used as labels.

Synthetic rows carry the verbatim data marker [SYNTHETIC NEWS — for portfolio demonstration].

Evaluation

Measured on the 30-case held-out gold set, fully offline (no API, no key). Detection means the predicted tier meets or exceeds the gold case's minimum expected tier.

Stratum	Bare baseline	This model (offline)	70B teacher
Labour detection (n=23)	1/23 (4.3%)	16/23 (69.6%)	22/23 (95.7%)
Environmental detection (n=7)	not measured	5/7 (71.4%)	7/7 (100%)
Severe-trio Critical recall (n=18)	0/18	11/18 (61.1%)	17/18 (94.4%)
Full gold (n=30)	not measured	21/30 (70.0%)	not measured

The student recovers roughly two-thirds of the distance from the bare model to the 70B teacher. Its remaining misses are one-tier-short adjacent-family confusions (for example, forced_labour read as discrimination, which is still High), not missed harms; every such case still triggers mandatory human review.

Bias, risks and coverage

• Single-teacher-family bias: all labels come from one model family (Llama 3.3); its systematic errors transfer to the student.
• English only (distilbert-base-uncased). Non-English allegations are invisible to this model.
• Coverage asymmetry: English-language adverse media under-reports exactly the opaque, low-press-freedom jurisdictions where risk can be highest; silence is not safety. The parent screener carries a dedicated opacity signal for this reason, and treats absence of evidence as an information problem, not clearance.
• Template-overfit risk from the synthetic training rows.
• Cross-language behaviour was audited separately in the Git

Limitations and out-of-scope use

This model approximates its teacher; it does not beat it. The distillation trades some accuracy for offline reproducibility (no API, no key, no server).

Out of scope:

• Supplier termination decisions.
• Worker-level scoring.
• Production HREDD screening without human review.
• Out-of-domain text: financial, medical, or legal documents.

Provenance and licensing

• Weights: a fine-tuned DistilBERT student (Apache-2.0 base model). This derivative is released under Apache-2.0.
• Built with Llama: the training labels were generated by Meta Llama 3.3 70B via Groq. Use of those labels is subject to the Llama 3.3 Community License.
• Naming: this repository was renamed from PSompong/hredd-adverse-text-distilbert-onnx to carry "Llama" at the start of the model name, as the Llama 3.3 Community License requires for models trained on Llama outputs. The old id redirects here.
• Synthetic rows are marked as demonstration data; curated real cases are de-identified.
• Portfolio and methodology artefact; not legal advice and not a certification of any supplier.

Version history

• v0.2 (2026-07-04): corpus widened to 961 rows across three sources; 16-class head; offline gains on the held-out gold set (labour 10 to 16 of 23; environmental 1 to 5 of 7; severe-trio Critical recall 8 to 11 of 18).
• v0.1 (June 2026): initial distillation.
• Card revised 2026-07-17: 16-class head correction, new Bias, risks and coverage section, rename note.

Links

• Live demo: https://supplier-risk-model.pages.dev
• AI governance page: https://supplier-risk-model.pages.dev/#/governance
• Remedy and engagement page (V2): https://supplier-risk-model.pages.dev/#/remedy

Citation

@software{sompong2026llamahredd,
  author = {Sompong, P.},
  title  = {Llama-distilled HREDD adverse-text detector (DistilBERT ONNX student)},
  year   = {2026},
  url    = {https://huggingface.co/PSompong/llama-hredd-adverse-text-distilbert-onnx}
}