Veyra-100M: Fast Encoder-Based Topic Classification

Veyra-100M is a compact, encoder-only text classification model built by Dl26. It is designed for fast English topic classification using a BERT-style bidirectional Transformer encoder. Instead of generating answers autoregressively, it reads the input text in a single encoder pass and returns a class score over a fixed topic taxonomy.

The released checkpoint, Dl26/Veyra-100M, is a 100M-parameter-class classifier trained from random initialization. It uses the standard Transformers bert model type, so it loads with AutoModelForSequenceClassification and does not require trust_remote_code=True.

Veyra-100M is intended for practical classification workflows such as document routing, topic labeling, metadata enrichment, dataset filtering, and lightweight encoder research.

Why this model

• Compact BERT-style encoder for fast text classification
• Standard Hugging Face Transformers compatibility
• No custom architecture files or remote code requirement
• 14-way topic taxonomy based on DBpedia-style categories
• Useful for document routing, triage, and automatic labeling
• Trained from scratch with AdamW rather than adapted from a pretrained encoder
• Designed to run efficiently on CPU, GPU, and batch inference pipelines

Model details

Property	Value
Model name	Veyra-100M
Developer	Dl26
Model type	Encoder-only sequence classifier
Transformers model type	bert
Architecture	BertForSequenceClassification
Parameters	101,515,758
Hidden size	736
Layers	12
Attention heads	16
Intermediate size	2,944
Max positions	512
Vocabulary size	30,522
Number of labels	14
Training objective	Supervised single-label classification
License	Apache 2.0

Supported labels

Veyra-100M predicts one label from the following fixed topic set:

Label	Typical meaning
Company	Businesses, organizations, commercial entities
EducationalInstitution	Schools, universities, colleges, academic institutions
Artist	Musicians, painters, writers, performers, creative people
Athlete	Sports players and athletic figures
OfficeHolder	Political figures and public officials
MeanOfTransportation	Vehicles, ships, aircraft, trains, transport systems
Building	Buildings, structures, monuments, architectural locations
NaturalPlace	Mountains, rivers, islands, parks, natural locations
Village	Villages, towns, municipalities, local settlements
Animal	Animal species and animal-related entries
Plant	Plant species and plant-related entries
Album	Music albums and recorded releases
Film	Films, movies, cinematic works
WrittenWork	Books, written publications, literary works

Installation

pip install -U transformers torch accelerate

For CPU-only inference, accelerate is optional:

pip install -U transformers torch

Quick start

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-100M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Apple announced a new processor for its laptop computers."

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=128,
)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_id = int(logits.argmax(dim=-1))
label = model.config.id2label[predicted_id]
print(label)

GPU inference

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-100M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

texts = [
    "The athlete scored two goals in the final match.",
    "The album includes twelve songs recorded in London.",
    "The village is located near a river and surrounded by hills.",
]

inputs = tokenizer(
    texts,
    return_tensors="pt",
    truncation=True,
    padding=True,
    max_length=128,
).to(model.device)

with torch.no_grad():
    logits = model(**inputs).logits

predictions = logits.argmax(dim=-1).tolist()
for text, label_id in zip(texts, predictions):
    print(model.config.id2label[label_id], "-", text)

Batch inference

For high-throughput classification, batch inputs together and keep a consistent max_length.

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-100M"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).to(device)
model.eval()

def classify_batch(texts, batch_size=64):
    results = []
    for start in range(0, len(texts), batch_size):
        batch = texts[start:start + batch_size]
        inputs = tokenizer(
            batch,
            return_tensors="pt",
            truncation=True,
            padding=True,
            max_length=128,
        ).to(device)
        with torch.no_grad():
            logits = model(**inputs).logits
        probs = logits.softmax(dim=-1)
        label_ids = probs.argmax(dim=-1).tolist()
        scores = probs.max(dim=-1).values.tolist()
        for text, label_id, score in zip(batch, label_ids, scores):
            results.append({
                "text": text,
                "label": model.config.id2label[label_id],
                "score": float(score),
            })
    return results

Interpreting outputs

Veyra-100M is a single-label classifier. For each input text, the model returns logits over the 14 supported labels. The highest-scoring label is usually treated as the predicted class.

Use confidence scores as a routing signal, not as a perfect measure of correctness. Low-confidence predictions are good candidates for fallback handling, manual review, or a broader classification model.

Recommended practices:

• Use title plus body text when available.
• Keep inputs descriptive rather than extremely short.
• Use max_length=128 for fast classification.
• Increase to max_length=256 or 512 when documents need more context.
• Calibrate confidence thresholds on your own validation set.

Evaluation highlights

The downloaded checkpoint was tested after training on DBpedia-style topic classification.

Evaluation	Result
Held-out 5K evaluation sample	98.4% accuracy
Separate 2K test script sample	96.2% accuracy
Model load path	AutoModelForSequenceClassification
Remote code required	No

These numbers are useful as sanity checks for the released checkpoint, but users should evaluate the model on their own data before deployment.

Input formatting

The model works best with natural, descriptive English text.

Good examples:

Apple announced a new processor for its laptop computers.
The athlete scored two goals in the final match.
The album includes twelve songs recorded in London.

For records with separate title and body fields, combine them:

text = f"{title}. {description}"

Very short or generic inputs may be ambiguous. For example, a sentence such as “The film won several awards” may not contain enough detail for stable classification.

Intended use

Veyra-100M is intended for:

• topic classification
• document routing
• metadata enrichment
• dataset filtering
• search indexing labels
• moderation queue triage by broad topic
• lightweight encoder benchmarking
• educational experiments with from-scratch classifiers

Out-of-scope use

Veyra-100M is not intended for:

• open-ended text generation
• semantic embedding search
• safety moderation as a policy model
• legal, medical, financial, or identity-sensitive decision making
• reliable classification outside the supported label taxonomy without additional validation

Limitations

• The model supports a fixed 14-label taxonomy.
• It is trained for English text and may be unreliable on other languages.
• It can misclassify short, vague, adversarial, or out-of-domain inputs.
• It is not a general-purpose reasoning model.
• It is not a replacement for human review in high-impact workflows.
• Confidence scores may require calibration for production systems.

Citation

@misc{dl26_2026_veyra_100m,
  title        = {Veyra-100M: Fast Encoder-Based Topic Classification},
  author       = {Dl26},
  year         = {2026},
  url          = {https://huggingface.co/Dl26/Veyra-100M}
}