Tamyiz

Reads one message sent to an Islamic app and answers 17 questions about it in about 50 ms on a CPU: is it about Islam, what does the person want, does it need the web or a scholar, and is the person at risk of harming themselves.

It runs in Sirathic, a free Islamic answering engine, before the answer model sees the message.

pip install torch transformers safetensors huggingface_hub

from huggingface_hub import hf_hub_download
import importlib.util

spec = importlib.util.spec_from_file_location(
    "tamyiz", hf_hub_download("sirathic/tamyiz", "tamyiz.py"))
tamyiz = importlib.util.module_from_spec(spec)
spec.loader.exec_module(tamyiz)

decide = tamyiz.Tamyiz.from_pretrained("sirathic/tamyiz")
result = decide("Is it allowed to pray with my shoes on?")
print(result["answers"]["ask"]["answer"], result["route"])

result["answers"] holds every question's answer and the probability of each option. result["route"] is "self_harm", "distress" or None.

What you can rely on

Use the route. A message goes to self_harm when the self-harm probability is at least 0.132, and to distress when distress is the most likely risk after that. On our safety test of 320 messages in 8 languages, the route sent 77 of 80 self-harm messages to a route (76 to self_harm, 1 to distress) and fired on none of 40 harmless ones. We picked the 0.132 threshold on that same test, so expect a little less on new messages.

Log the other answers, but don't act on them yet:

• The waswasa and harm (takfir, violence, hate) flags fire on 7 of 40 harmless messages. A single doubt about wudu reads as waswasa, and a question about gentle parenting can read as violence.
• When needs_web, needs_scholar, madhab_matters or the other routing flags say yes, they are right 24% to 56% of the time on our hand-labelled messages.

The 17 questions

Question	Options
scope	islamic, general, social
ask	ruling, explain, find_text, calculate, how_to, advice, recommend, write
topic	belief, worship, family, money, daily_life, health, character, texts, history, society
text	quran, hadith, dua, none
risk	none, self_harm, abuse, emergency, distress
harm	none, takfir, violence, hate
asker	general, minor, convert, non_muslim, student
needs_current, needs_location, needs_web, needs_community, madhab_matters, needs_scholar, waswasa, follow_up, short_ok, scholars_differ	true, false

config.json has the exact wording of every question and option.

Benchmarks

916 hand-labelled messages in Arabic, English, French, Indonesian, Turkish and Urdu. Laya is our previous model, which asked each question separately.

	Laya	Tamyiz
Accuracy over all answers	90.4%	90.0%
Calibration error (ECE, lower is better)	6.0	1.8
Share of wrong answers made at 90%+ confidence	62%	23%
Median time per message, 4 CPU threads	3,363 ms	50 ms

Rare answers found (recall of the less common options):

Question	Laya	Tamyiz
risk	74%	95%
harm	49%	88%
waswasa	74%	94%
asker	68%	98%
scope	67%	88%
needs_scholar	77%	91%
needs_web	83%	92%
madhab_matters	59%	73%

Accuracy by language: Arabic 91.9%, English 89.2%, French 91.4%, Indonesian 91.3%, Turkish 89.6%, Urdu 91.3%.

Safety test, 320 messages in 8 languages:

Messages	Result
80 self-harm	77 sent to a route
40 harmless	0 sent to a route, 7 flagged by shadow heads
40 abuse victims	39 found by the abuse head (shadow)
40 distress near-misses	38 sent to distress or self-harm

Training

About 56,000 messages in 17 languages: questions from public fatwa sites, written examples, and pairs of harmful and harmless messages that share the same words. Qwen labelled most of them. Claude Opus labelled the hard safety cases and settled disagreements. The training data is not released.

License

Apache-2.0. The base model, mmBERT-base, is MIT.