LinkjeBERT

A Dutch language model that predicts where a human editor would place a hyperlink in plain text. Given a passage, LinkjeBERT returns per-token confidence scores indicating how likely each word is to be part of a link anchor.

Developed by DEJAN AI.

Read the full write-up: LinkjeBERT: A Dutch Language Model for Link Prediction

How It Works

LinkjeBERT is a binary token classifier fine-tuned on microsoft/mdeberta-v3-base. Every token is classified as either O (not a link) or LINK (part of a link anchor). The model outputs probabilities rather than hard labels, enabling heatmap-style visualization and tuneable thresholds for production pipelines.

Key Innovation: Markdown-Aware Training

Unlike its English predecessor LinkBERT, LinkjeBERT is trained on structure-preserving Markdown rather than flat text. The training data retains headings (#, ##), bold (**), italics (*), lists (-), and blockquotes (>). This gives the model the same structural context a human editor uses when deciding where a link belongs.

Training Details

Parameter	Value
Base model	microsoft/mdeberta-v3-base
Task	Binary token classification (O / LINK)
Loss function	Focal loss (γ=2.0)
Training tokens	200M
Training sources	5 Dutch editorial domains (news, tech, science, business, lifestyle)
Learning rate	2e-5 with linear warmup (10%)
Batch size	32 (16 × 2 gradient accumulation)
Epochs	10 (early stopping, patience 3, monitoring F1)
Precision	bf16
Max sequence length	512
Hardware	NVIDIA RTX 4090

Usage

from transformers import AutoModelForTokenClassification, AutoTokenizer
import torch

model_id = "dejanseo/LinkjeBERT"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)
model.eval()

text = "De minister liet weten dat het plan doorgaat."
inputs = tokenizer(text, return_tensors="pt", return_offsets_mapping=True)
offsets = inputs.pop("offset_mapping")[0]

with torch.no_grad():
    logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)[0, :, 1]

for i, (start, end) in enumerate(offsets):
    if start == end:
        continue
    word = text[start:end]
    p = probs[i].item()
    if p > 0.1:
        print(f"{word:20s} {p:.1%}")

Long Document Inference

For documents exceeding 512 tokens, use a sliding window with overlap:

• MAX_LENGTH = 512
• DOC_STRIDE = 128
• Aggregate overlapping token probabilities with np.maximum

This ensures consistent predictions across article-length text.

Input Format

Feed the model Markdown-formatted plain text with links stripped. The model was trained on editorial content where existing <a> tags were removed, so it predicts link placement from context alone. Preserve structural markers (#, **, -, >) as the model relies on them.

Labels

ID	Label
0	O
1	LINK

Intended Use

• Anchor text suggestion for internal linking
• Evaluating naturalness of existing link placements
• Link placement guidance for content writers
• Detection of unnatural or spammy link patterns

Citation

@misc{linkjebert2026,
  title={LinkjeBERT: A Dutch Language Model for Link Prediction},
  author={DEJAN AI},
  year={2026},
  url={https://dejan.ai/blog/linkjebert/}
}