Shinigami V3
A DistilBERT-based classifier for detecting potentially malicious shell commands and tool inputs.
Model Details
- Architecture: DistilBERT
- Base model: distilbert-base-uncased
- Version: 3.0.0
- Task: Binary text classification
- Maximum sequence length: 256
- Label 0: BENIGN
- Label 1: MALICIOUS
Performance
| Metric | Result |
|---|---|
| Accuracy | 97.49% |
| Malicious precision | 98.19% |
| Malicious recall | 97.60% |
| Malicious F1 | 97.89% |
| False positive rate | 2.68% |
| False negative rate | 2.40% |
Confusion matrix:
| Actual / Predicted | Benign | Malicious |
|---|---|---|
| Benign | 435 | 12 |
| Malicious | 16 | 651 |
Dataset
Main dataset: dataset/ml-data-v4-clean.csv
Columns: command, attack_family, category, threat_level, platform, is_malicious.
Intended Use
Designed as an inference component for security systems protecting AI agents and applications that interact with external tools.
The model classifies text. It does not execute commands or independently enforce security policies.
Limitations
- Predictions are based on learned text patterns, not execution context.
- False positives and false negatives are possible.
- Confidence scores are not guarantees of safety.
- Combine predictions with deterministic policies, authorization, and least-privilege execution.
License
MIT
- Downloads last month
- 42