Architecture Doc Type Classifier (PT/EN)
Tells you whether an architecture document reads like an ADR, design doc, guide, playbook, postmortem or teardown, in Portuguese or English. It is a 6-class logistic regression on top of paraphrase-multilingual-MiniLM-L12-v2, with 2,310 weights, in plain JSON.
Use it to triage a documentation folder, not to decide anything alone. Try it in the Space.
How well it works
Measured with 5-fold cross-validation, always keeping the pt and en versions of a document in the same fold.
| Metric | Value |
|---|---|
| Accuracy | 0.648 (always guessing the largest class: 0.284) |
| Macro F1 | 0.643 |
| Class | F1 | Samples |
|---|---|---|
| postmortem | 0.92 | 40 |
| guide | 0.71 | 16 |
| teardown | 0.60 | 20 |
| playbook | 0.59 | 26 |
| adr | 0.55 | 50 |
| design-doc | 0.47 | 24 |
Postmortems are easy because they have a very particular structure. ADRs and design docs get confused with each other, which matches what happens in real life: many ADRs carry a design inside. Full numbers are in metrics.json.
Limits
- Small training set: 176 samples from 88 documents.
- AI-generated training data: real documents from your company will look different, so expect lower numbers. I have not measured that.
- Not robust to out-of-scope text: it always answers one of the six classes, even for a recipe. Treat low confidence as "none of these".
- The label names themselves (
ADR,postmortem...) are masked before training and inference, so the model learns from the content and not from the document announcing its type.
Usage
from huggingface_hub import hf_hub_download
import importlib.util
path = hf_hub_download("fernandofa42/architecture-doc-type-classifier", "inference.py")
spec = importlib.util.spec_from_file_location("inference", path)
inference = importlib.util.module_from_spec(spec)
spec.loader.exec_module(inference)
clf = inference.load()
print(clf.predict("Tivemos 40 minutos de indisponibilidade no PIX. Linha do tempo, causa raiz e ações corretivas..."))
Built by Fernando Azevedo, Specialist IT Architect focused on AI and AWS.