Haruka-Tagger-Large (EVA02-Large-448, 98.8k Tags, 18-Query Slot-Sigmoid Tree)

The 421M-parameter (304M eva02_large_patch14_448 backbone) big sister to Haruka-Tagger, co-trained end-to-end across 9.25M Danbooru images and 98,782 tags (70.14%+ Val F1).

Compute (1 Node of 8x H100) graciously sponsored by alpindale from dphn.ai.

---

Overview

Haruka-Tagger-Large scales up the Haruka-Tagger architecture from eva02_base (86M backbone, 768-d, 12 blocks) to eva02_large_patch14_448.mim_m38m_ft_in22k_in1k (304M backbone, 1024-d, 24 blocks) while keeping the 2-Level 18-Query Slot-Sigmoid Attention Tree (MHAttnPool, 18 × 64 = 1,152-d) and 98,782-tag classifier head:

Category	Vocabulary Count	Recommended Threshold
Artist (artist)	38,447	0.25 (or 0.15 high-recall)
Character (character)	31,471	0.35 (or 0.25 for rare/crossover chars)
General (general)	21,748	0.35 (or 0.45 high-precision)
Copyright (copyright)	7,091	0.35
Year / Era (year)	21	0.25
Rating (rating)	4	argmax / 0.50
Total Vocabulary	98,782	—

Files in This Repository

• haruka-tagger.safetensors (1.6 GB) — Full PyTorch weights (backbone.*, head_pool.*, head.*).
• haruka-tagger.onnx (1.6 GB) — Standalone exported ONNX graph (input: images (B, 3, 448, 448) -> output: logits (B, 98782)).
• haruka-tagger_metadata.json (13 MB) — Complete tag index mapping (idx_to_tag, tag_to_category, idx_to_init_logit_bias, idx_to_irfs_factor) and model configuration.
• haruka_infer.py — Self-contained CLI inference script, 2-level query tree inspector, and per-tag spatial attribution heatmap visualizer (python haruka_infer.py image.jpg --save-vis tree_vis.jpg).

---

Standalone Evaluation on Unseen 2026 Danbooru Uploads (70.47% Val F1)

Evaluated on 1,000 unseen post-cutoff 2026 Danbooru uploads (new-dan) across all 98,782 native tags:

Category / Slice	Active Tags in Eval / Total Vocab	mAP (AUPRC)	Micro-F1	Per-Tag Macro-F1	Median Tag F1
General (Full Native Vocab)	4,143 / 21,748	59.33%	64.69%	61.52%	66.67%
Character (Full Native Vocab)	745 / 31,471	95.32%	87.96%	92.89%	100.00%
Copyright (Full Native Vocab)	295 / 7,091	85.63%	80.97%	84.98%	100.00%
Artist (Full Native Vocab)	221 / 38,447	92.96%	87.62%	87.26%	100.00%
Rating	4 / 4	72.22%	76.52%	78.02%	77.76%
10k-Core Anchor Subset	7,778 Gen / 2,649 Char	62.72% Gen / 99.02% Char	67.91% Gen	64.80% Gen / 98.42% Char	—

General Tag Performance by Training Support Bucket

Support Bucket	Active Tag Count in Eval	mAP (AUPRC)	Per-Tag Macro-F1	Median Tag F1
Head (>= 10,000 training imgs)	2,142	59.04%	62.04%	66.67%
Common (1,000 – 9,999 training imgs)	1,549	60.46%	62.61%	66.67%
Uncommon (100 – 999 training imgs)	452	56.80%	55.34%	66.67%

---

Quickstart Usage

Option 1: Included CLI + Tree Inspector & Spatial Heatmap Visualizer (haruka_infer.py)

python haruka_infer.py path/to/image.jpg --save-vis tree_vis.jpg

---

Option 2: Standalone PyTorch + safetensors

import json
import numpy as np
import timm
import torch
import torch.nn as nn
from PIL import Image
from safetensors.torch import load_file

IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32).reshape(3, 1, 1)
IMAGENET_STD = np.array([0.229, 0.224, 0.225], dtype=np.float32).reshape(3, 1, 1)


def preprocess_image(img: Image.Image, target_size: int = 448) -> torch.Tensor:
    """Alpha-composites onto white (255, 255, 255), letterbox-resizes (BICUBIC) to 448x448, and normalizes."""
    if img.mode != "RGB":
        img = img.convert("RGBA")
        bg = Image.new("RGBA", img.size, (255, 255, 255, 255))
        bg.alpha_composite(img)
        img = bg.convert("RGB")

    w, h = img.size
    if w != target_size or h != target_size:
        scale = min(target_size / w, target_size / h)
        nw, nh = max(1, int(w * scale)), max(1, int(h * scale))
        img = img.resize((nw, nh), Image.Resampling.BICUBIC)
        canvas = Image.new("RGB", (target_size, target_size), (255, 255, 255))
        canvas.paste(img, ((target_size - nw) // 2, (target_size - nh) // 2))
        img = canvas

    arr = np.asarray(img, dtype=np.float32).transpose(2, 0, 1) / 255.0
    arr = (arr - IMAGENET_MEAN) / IMAGENET_STD
    return torch.from_numpy(arr).unsqueeze(0)


class MHAttnPool(nn.Module):
    def __init__(self, dim: int = 1024, num_heads: int = 16, num_queries: int = 18, leaf_dim: int = 64, pool_mode: str = "slot_sigmoid"):
        super().__init__()
        self.num_heads = num_heads
        self.head_dim = dim // num_heads
        self.num_queries = num_queries
        self.pool_mode = pool_mode
        self.norm = nn.LayerNorm(dim)
        self.kv = nn.Linear(dim, dim * 2)
        self.num_coarse = min(6, num_queries)
        self.branches = num_queries // self.num_coarse
        self.out_dim = num_queries * leaf_dim
        self.q_coarse = nn.Parameter(torch.zeros(1, self.num_coarse, dim))
        self.trunk_norm = nn.LayerNorm(dim)
        self.q_child = nn.Parameter(torch.zeros(1, self.num_coarse, self.branches, dim))
        self.child_q_norm = nn.LayerNorm(dim)
        self.leaf_norm = nn.LayerNorm(dim)
        if pool_mode == "slot_sigmoid":
            self.q_coarse_bias = nn.Parameter(torch.full((1, 1, self.num_coarse, 1), -2.0))
            self.q_child_bias = nn.Parameter(torch.full((1, 1, num_queries, 1), -2.0))
        self.leaf_proj_weight = nn.Parameter(torch.empty(num_queries, dim, leaf_dim))
        self.leaf_proj_bias = nn.Parameter(torch.zeros(num_queries, leaf_dim))

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        B, N, D = x.shape
        x = self.norm(x)
        k, v = self.kv(x).chunk(2, dim=-1)
        k = k.view(B, N, self.num_heads, self.head_dim).transpose(1, 2)
        v = v.view(B, N, self.num_heads, self.head_dim).transpose(1, 2)
        scale = self.head_dim ** -0.5

        q_c = self.q_coarse.expand(B, self.num_coarse, D).view(B, self.num_coarse, self.num_heads, self.head_dim).transpose(1, 2)
        logits_c = (q_c @ k.transpose(-2, -1)) * scale
        attn_c = (
            torch.softmax(logits_c, dim=-2) * torch.sigmoid(logits_c + self.q_coarse_bias)
            if self.pool_mode == "slot_sigmoid"
            else torch.softmax(logits_c, dim=-1)
        )
        z_coarse = self.trunk_norm((attn_c @ v).transpose(1, 2).contiguous().view(B, self.num_coarse, D))

        q_leaf = self.child_q_norm((z_coarse.unsqueeze(2) + self.q_child).view(B, self.num_queries, D))
        q_l = q_leaf.view(B, self.num_queries, self.num_heads, self.head_dim).transpose(1, 2)
        logits_l = (q_l @ k.transpose(-2, -1)) * scale
        attn_l = (
            torch.softmax(logits_l, dim=-2) * torch.sigmoid(logits_l + self.q_child_bias)
            if self.pool_mode == "slot_sigmoid"
            else torch.softmax(logits_l, dim=-1)
        )
        z_child = (attn_l @ v).transpose(1, 2).contiguous().view(B, self.num_coarse, self.branches, D)

        z_tree = self.leaf_norm((z_coarse.unsqueeze(2) + z_child).view(B, self.num_queries, D))
        if self.pool_mode == "slot_sigmoid":
            mass_q = attn_l.sum(dim=-1).mean(dim=1).unsqueeze(-1)
            z_tree = z_tree * (mass_q / (mass_q + 4.0))

        leaf_vecs = torch.einsum("bqd,qdf->bqf", z_tree, self.leaf_proj_weight) + self.leaf_proj_bias
        return leaf_vecs.reshape(B, self.out_dim)


class HarukaStandalone(nn.Module):
    def __init__(self, meta: dict):
        super().__init__()
        self.backbone = timm.create_model(meta["backbone"], pretrained=False, num_classes=0, global_pool="")
        self.head_pool = MHAttnPool(
            dim=self.backbone.num_features,
            num_queries=meta.get("num_queries", 18),
            pool_mode=meta.get("pool_mode", "slot_sigmoid"),
        )
        self.head = nn.Linear(self.head_pool.out_dim, meta["total_tags"])

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        feats = self.backbone.forward_features(x)
        return self.head(self.head_pool(feats))


# Load metadata & weights
with open("haruka-tagger_metadata.json", "r", encoding="utf-8") as f:
    meta = json.load(f)

device = "cuda" if torch.cuda.is_available() else "cpu"
model = HarukaStandalone(meta)
model.load_state_dict(load_file("haruka-tagger.safetensors"), strict=True)
model.to(device).eval()

# Run inference on an image
x = preprocess_image(Image.open("example.jpg"), target_size=meta["img_size"]).to(device)
with torch.no_grad(), torch.autocast(device_type="cuda", dtype=torch.bfloat16, enabled=(device == "cuda")):
    probs = torch.sigmoid(model(x).float())[0].cpu().numpy()

thresholds = {"character": 0.35, "copyright": 0.35, "artist": 0.25, "general": 0.35, "rating": 0.50, "year": 0.25}
idx_to_tag = meta["idx_to_tag"]
tag_to_cat = meta["tag_to_category"]

results = {c: [] for c in thresholds}
for idx in np.where(probs >= 0.15)[0]:
    tag = idx_to_tag[str(int(idx))]
    cat = tag_to_cat.get(tag, "general")
    score = float(probs[idx])
    if score >= thresholds.get(cat, 0.35):
        results[cat].append((tag, round(score, 4)))

for cat, items in results.items():
    items.sort(key=lambda x: -x[1])
    print(f"{cat.upper():<10}:", ", ".join(f"{t} ({s:.0%})" for t, s in items[:15]))

---

Option 3: ONNX Runtime (haruka-tagger.onnx)

import json
import numpy as np
import onnxruntime as ort
from PIL import Image

with open("haruka-tagger_metadata.json", "r", encoding="utf-8") as f:
    meta = json.load(f)

sess = ort.InferenceSession("haruka-tagger.onnx", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
x_np = preprocess_image(Image.open("example.jpg"), target_size=meta["img_size"]).numpy()

logits = sess.run(["logits"], {"images": x_np})[0][0]
probs = 1.0 / (1.0 + np.exp(-logits))