BattleGhost/anime-quartet-preference-ranker

Anime Quartet Preference Ranker

A small pairwise preference head for ranking four related anime-style images from best to worst. It is intended for candidate selection after image generation, not as a general-purpose image quality model.

The release contains the learned linear ranking head, four lightweight artifact probes, configuration, and a minimal inference script. It does not contain training images, evaluation images, captions, identities, filenames, prompts, or production-specific rules.

Intended use

Give the model four variations of the same subject and scene. It returns a score and ordering for those four candidates. The model is most useful as:

• one signal in a larger selection pipeline;
• a starting point for personal preference learning;
• a candidate generator where a human reviews the top one or two results.

This model is not designed to compare unrelated images, identify characters, moderate content, or guarantee anatomical correctness.

Architecture

This is feature fusion followed by one linear pairwise-logistic head, not majority voting among independent rankers.

For each image, the input is built from:

1. WD SwinV2 Tagger v3: the full image and four overlapping corner crops. Each 1,024-dimensional pooled vector is L2-normalized and the five views are concatenated (5,120 values).
2. PixAI Tagger v1.0: one L2-normalized 1,024-dimensional pooled vector.
3. Four logistic artifact signals: hand structure, human anatomy, element overlap, and object morphology.
4. One aesthetic scalar: the expected class (0–3) from the Danbooru aesthetic classifier over the WD global-image feature.

The first 6,148 values are jointly L2-normalized per image and centered within each group of four. The aesthetic scalar is centered separately within the group. The resulting 6,149-dimensional vector is scored by the published head.

The artifact probes were trained on an MIT-licensed subset of MagicData340K. They are soft features, not calibrated probabilities or hard rejection gates, and may capture style or composition confounders.

Evaluation

Evaluation used 435 private groups of four generated images, with a single human top-1 choice per group. Five-fold cross-validation held entire subjects out of training, so images of an evaluated subject never appeared in its training fold.

Metric	Model	Random ordering
Top-1 exact match	167 / 435 = 38.4%	25.0%
Recall@2	286 / 435 = 65.7%	50.0%
Recall@3	380 / 435 = 87.4%	75.0%
Mean reciprocal rank	0.624	0.521

Human-winner rank counts were 167 / 119 / 94 / 55 for ranks 1 through 4.

These are strict exact-match metrics against one person's single selected winner. They are not the percentage of outputs that a person would consider acceptable: several candidates in a group can be effectively equivalent. The private images and annotations are not redistributed.

In a separate private end-to-end workflow, additional identity and within-set consistency checks improved approval rates of automatic selections to approximately 58%–90% on new sets. Those numbers are not metrics for this released head, and part of that rule system was tuned on one set, so the range is optimistic. Those production-specific rules are not included here.

Quick start

pip install -r requirements.txt
python inference.py image_a.png image_b.png image_c.png image_d.png

The first run downloads the three frozen feature models. GPU inference is strongly recommended; PixAI Tagger at 1008 px is the largest component. --device cpu is supported but slow.

The script prints JSON ordered from best to worst. Scores are meaningful only relative to the other images in the same quartet.

Limitations

• Preferences come from one annotator and should not be interpreted as universal quality.
• Training and evaluation images were private generated-image groups and included adult content. No images are included in this release.
• The model has weak absolute Top-1 performance. Recall@2 is the more practical metric when a second-stage reviewer is available.
• It does not receive reference images or character names, so visual identity and canonical hair or clothing are not guaranteed.
• Artifact probes were trained out of domain on a mixed-style external dataset and can be confounded by subject, framing, or style.
• Group centering means the score of an image changes when the other three candidates change.
• The model should not be used for safety moderation, medical interpretation, biometric decisions, or judgments about people.

Dependencies and licenses

• SmilingWolf/wd-swinv2-tagger-v3 — Apache-2.0.
• pixai-labs/pixai-tagger-v1.0 — Apache-2.0.
• aipracticecafe/danbooru-aes-cls — MIT.
• wj-inf/MagicData340k — MIT; used only for the four artifact probes.

This repository's original code and weights are released under Apache-2.0. Downloaded dependencies remain subject to their own licenses and model cards.