akifkulll/mayi-distilbert-a

distilbert-base-uncased fine-tuned to decide whether an AI agent's proposed tool call should be allowed (0) or denied (1). Input format A: the tool call alone.

Trained for the COE025 NLP course project (Istinye University) on the cleaned English split of auto-1b-data (cleaning and splits).

Test results (threshold 0.5)

rows	accuracy	macro-F1	false allow rate	false deny rate	AUROC
71613	0.9142	0.9142	0.0879	0.0837	0.9685

Use

from transformers import pipeline
clf = pipeline("text-classification", model="akifkulll/mayi-distilbert-a")
clf("### PROPOSED TOOL CALL\ntool: shell\nargs: rm -rf node_modules ...")

Labels were generated by AI models; expect weaker results on inputs written in a different format.