Instructions to use Project-Bastion/attack-technique-tagger with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Project-Bastion/attack-technique-tagger with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="Project-Bastion/attack-technique-tagger")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("Project-Bastion/attack-technique-tagger") model = AutoModelForSequenceClassification.from_pretrained("Project-Bastion/attack-technique-tagger", device_map="auto") - Transformers.js
How to use Project-Bastion/attack-technique-tagger with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('text-classification', 'Project-Bastion/attack-technique-tagger'); - Notebooks
- Google Colab
- Kaggle
attack-technique-tagger (experimental v0.1)
Tags a sentence from a threat report with the MITRE ATT&CK techniques it describes (50 techniques, multi-label). Meant as an assistant for CTI analysts and detection engineers who triage reports; it is only moderately accurate (sealed test micro F1 0.59), so use its output as suggestions to review, not as ground truth.
Try it in your browser: a static page that runs the model locally (transformers.js, ONNX). The text you paste never leaves your device.
Usage
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
repo = "Project-Bastion/attack-technique-tagger"
tok = AutoTokenizer.from_pretrained(repo); model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
sentences = ["The malware dumps credentials from LSASS memory using a Mimikatz-like routine.",
"It then creates a scheduled task to maintain persistence on the host.",
"Command and control traffic is sent over HTTPS to a hard-coded domain."] # sentences of ONE report, in order
def tag(i, threshold=0.2):
prev = sentences[i - 1] if i > 0 else ""; nxt = sentences[i + 1] if i + 1 < len(sentences) else ""
ctx = prev[-300:] + " | " + nxt[:300] # neighbouring sentences are part of the input (the model was trained this way)
enc = tok(sentences[i], ctx, truncation="longest_first", max_length=256, return_tensors="pt")
with torch.no_grad(): p = torch.sigmoid(model(**enc).logits[0])
return [(model.config.id2label[j], round(float(p[j]), 3)) for j in torch.nonzero(p >= threshold).flatten().tolist()]
for i, s in enumerate(sentences): print(s[:60], "->", tag(i))
Details that matter: input is the sentence plus its previous and next sentence from the same report (adding them improved validation F1 slightly); outputs are independent sigmoids over 50 technique ids
(see config.json; look any id up at attack.mitre.org/techniques/<ID>); the threshold 0.2 was chosen on validation data and trades precision for recall (raise it for fewer false positives).
Evaluation (sealed test)
Data: the TRAM2 annotated threat-report sentences (Center for Threat-Informed Defense, Apache-2.0): 19,178 sentences from 151 public reports, 4,070 of them labelled with 1 or more of 50 techniques. Split by report (so no report is in two splits): 105 reports train, 20 validation, 26 test (2,920 sentences, 683 with a technique). Two techniques that occur in only 1 and 5 reports were pinned to train, so test covers 48 of the 50. All model and threshold choices (about 15 training runs over 4 base models, learning rates, class weights, context and seeds) used the validation split only. The test split was scored once, on the final model.
| System | Micro F1 | Macro F1 | "has any technique" F1 |
|---|---|---|---|
| attack-technique-tagger (this model) | 0.593 | 0.537 | 0.730 |
| TF-IDF + logistic regression (same split) | 0.536 | 0.496 | 0.672 |
| nanda-rani/TTPXHunter (MIT; best existing open model I found; its confidence threshold tuned on validation) | 0.296 | 0.243 | 0.499 |
Micro precision / recall of this model on test: 0.595 / 0.590. Validation micro F1: 0.605 (three seeds: 0.597, 0.606, 0.596). TTPXHunter was trained on a different corpus (ATT&CK procedure examples), covers all 50 of these techniques among its 193 classes, and was run off the shelf; its low score reflects that domain shift as much as model quality. Averaging three seeds scored 0.612 on validation (not shipped).
Training
cisco-ai/SecureBERT2.0-base (Apache-2.0, ModernBERT architecture) fine-tuned with binary cross-entropy (positive-class weight exponent 0.5), lr 6e-5, 12 epochs, max length 256, bf16, one A10G; the epoch with the best validation micro F1 is kept.
Code: GitHub Project-Bastion/models, scripts/ttp/.
ONNX / browser
onnx/model.onnx (fp32, 599 MB) is published for transformers.js / onnxruntime. Checked against PyTorch on 200 validation sentences: onnxruntime fp32 differs by <1e-5 in probability; the in-browser code path (run in Node with transformers.js) gives identical decisions at the 0.2 threshold on all 200 sentences, with probabilities identical for 197 of them and within 0.014 on 3 long inputs (the demo works around a transformers.js quirk that drops the closing [SEP] when it truncates; a small residual on very long inputs remains, cause not fully isolated).
int8 quantization does not work for this model: four dynamic-quantization recipes kept only 91-94% of decisions, so no int8 file is published. The demo therefore downloads the full 599 MB once.
Known limits
- 50 techniques only (the TRAM2 label set); sub-technique distinctions are limited to the ones in that set; anything else is invisible to the model.
- Sentence-level and context-limited; technique labels in the training reports are themselves noisy and sometimes depend on the whole report.
- Test set is small (26 reports); numbers will move on other report styles, languages (English only) or newer malware reports. Not evaluated on non-report text such as logs or alerts.
- Moderate accuracy: expect many missed and extra techniques. Do not use it to automate decisions.
License and credits
Apache-2.0. Training data: TRAM2 data from the Center for Threat-Informed Defense TRAM project (Apache-2.0), derived from public threat reports labelled with MITRE ATT&CK techniques (ATT&CK is a trademark of The MITRE Corporation; use per the ATT&CK Terms of Use).
- Downloads last month
- 1
Model tree for Project-Bastion/attack-technique-tagger
Base model
answerdotai/ModernBERT-baseSpace using Project-Bastion/attack-technique-tagger 1
Evaluation results
- micro F1 on TRAM2 sealed test (26 held-out reportsself-reported0.593
- macro F1 on TRAM2 sealed test (26 held-out reportsself-reported0.537
- any-technique detection F1 on TRAM2 sealed test (26 held-out reportsself-reported0.730