KaraKaraWitch/haruka-tagger-alpha

Haruka-Tagger (Alpha / Stage-2-v3)

What's this?

Just another Danbooru anime image tagger — built on EVA02-Base (448×448) + Multi-Head Attention Pooling (MHAttnPool) and trained across 9.25M deduplicated Danbooru posts (up to 2026) covering 98,726 tags.

What happened to haruka-tagger?

Mainly a bunch of misplaced hopes about being actually good out of the gate when there were still real training/calibration issues to iron out (clamped prior biases fighting IRFS oversampling, single-query global pooling bottlenecks, and failed low-rank logit calibration experiments).

Rather than pretending it's a finished v1.0, this release (stg-2-v3) is Haruka-Tagger Alpha: the cleaned-up, properly post-trained single-query baseline while the next-gen multi-query head is in the oven.

Does this still work?

Yes! You can mostly trust the results now (especially on single-character / standard compositions, including 2025–2026 characters and franchises).

However, please keep these known Alpha caveats in mind:

1. Year/Era tags (era_2023_2025, etc.) are by design not meant to be exact: They are overlapping sliding-window era buckets to give rough stylistic time-period signal, not exact upload timestamps.
2. Artists might be wrong (especially on rare artists or fanart):
   • Because Stage-1 used Inverse Repeat Factor Sampling (IRFS) with a clamped bias floor (-4.5), some rare artists (50–300 posts) were over-boosted relative to prolific artists, and strong franchise/character features can occasionally bleed into artist predictions.
   • Use a lower threshold for artists (~0.20–0.25) and treat artist predictions as "style candidates" rather than ground truth.
3. Multi-character / Group banners (6+girls) & Chimera tags:
   • The Alpha model uses a single global query (1 × 768) in MHAttnPool. On crowded group images or promotional banners with lots of text, traits from multiple characters (e.g., Girl A's cat_ears + Girl B's blonde_hair + Girl C's yellow_eyes) can get averaged together and trigger a lookalike "chimera" character (coconut_(nekopara)), or drown out smaller background characters.
4. General tags vs. Character/Copyright:
   • Character (70.2% Val Micro-F1 @ 0.50, 84.0% Precision) and Copyright (66.5% Val Micro-F1 @ 0.50) are strong, while hyper-specific general tags can still be noisy due to incomplete Danbooru tagging.

---

Model Details & Vocabulary (98,726 Tags)

Property	Value
Backbone	eva02_base_patch14_448.mim_in22k_ft_in22k_in1k (86.3M params)
Pooling + Head	MHAttnPool (1.8M params) + Linear(768, 98726) (75.9M params)
Total Parameters	164M (~656 MB in FP32 .safetensors / .onnx)
Input Resolution	448 × 448 (Aspect-ratio preserving BICUBIC resize + white 255 pad)
Normalization	ImageNet (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
Dataset Cutoff	Danbooru 2026 (~9.25M wHash-deduplicated images)

Tag Breakdown by Category (Minimum Image Floors)

Category	Tag Count	Dataset Floor	Recommended Threshold	Val Micro-F1 (@0.50)	Val Macro AUROC
Artist	38,420	>= 50 posts	0.20 – 0.25	45.0% (at 0.35)	99.34%
Character	31,454	>= 45 posts	0.30 – 0.40	70.19% (84.0% P)	99.85%
General	21,743	>= 100 posts	0.35	59.15%	97.71%
Copyright	7,084	>= 50 posts	0.35	66.48% (77.1% P)	99.28%
Year / Era	21	Sliding 3-yr	Top-1	60.05%	89.84%
Rating	4	g, s, q, e	Top-1	81.08%	95.43%
Total	98,726	—	0.35 (overall)	60.67%	99.06%

---

Quickstart (ONNX Runtime)

haruka-tagger.onnx is a single, self-contained Opset 18 ONNX file (no external .data file required).

import json
import numpy as np
import onnxruntime as ort
from PIL import Image

# 1. Load metadata & ONNX session
with open("haruka-tagger_metadata.json", "r", encoding="utf-8") as f:
    meta = json.load(f)

idx_to_tag = meta["idx_to_tag"]
tag_to_cat = meta["tag_to_category"]

sess = ort.InferenceSession(
    "haruka-tagger.onnx",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)

# 2. Preprocess image (aspect-ratio preserving resize + white pad to 448x448)
def preprocess(img_path: str, size: int = 448) -> np.ndarray:
    img = Image.open(img_path).convert("RGBA")
    bg = Image.new("RGBA", img.size, (255, 255, 255, 255))
    bg.alpha_composite(img)
    img = bg.convert("RGB")

    w, h = img.size
    scale = min(size / w, size / h)
    nw, nh = max(1, int(w * scale)), max(1, int(h * scale))
    img = img.resize((nw, nh), Image.Resampling.BICUBIC)

    canvas = Image.new("RGB", (size, size), (255, 255, 255))
    canvas.paste(img, ((size - nw) // 2, (size - nh) // 2))

    arr = np.asarray(canvas, dtype=np.float32) / 255.0
    mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
    std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
    arr = (arr - mean) / std
    return arr.transpose(2, 0, 1)[None, ...]  # (1, 3, 448, 448)

# 3. Run inference
x = preprocess("sample.jpg")
logits = sess.run(["logits"], {"input": x})[0][0]
probs = 1.0 / (1.0 + np.exp(-logits.astype(np.float64)))

thresholds = {"general": 0.35, "character": 0.30, "copyright": 0.35, "artist": 0.20}
for idx_str, tag in idx_to_tag.items():
    p = float(probs[int(idx_str)])
    cat = tag_to_cat[tag]
    if cat in thresholds and p >= thresholds[cat]:
        print(f"[{cat:<9}] {tag}: {p:.3f}")