MarioTheKnight/booru-lighting-tagger

Booru Lighting Tagger v1.0.2

Booru Lighting Tagger adds lighting tags to anime-style illustrations: where the light comes from, how contrasted and colored it is, which light effects and shadows show, and what lights the scene. It outputs 39 booru-style tags (backlighting, rim_light, chiaroscuro, caustics, window_shadow, candlelight…) and a tonal key (dark, dim_lighting, high_key).

Booru tags describe what is in an image much better than how it is lit. Most images with a strong backlight or a dramatic chiaroscuro are not tagged as such, so generative models trained on these captions have no reliable handle on lighting. This tagger is meant to complete the captions of image datasets, so that lighting becomes something you can prompt for.

• Base model: WD EVA02-Large Tagger v3 by SmilingWolf. 16 of its 24 transformer blocks were fine-tuned, under a new lighting head.
• Teacher: Claude Opus 5.5 rated about 19,500 images from 0 to 10 on each label; the tagger learned these ratings.
• Checked against a human: on blind human answers, the tagger ranks images better than its teacher (mean AP 0.87 against 0.78 on the 28 labels where both were measured). 29 of the 39 labels reach 70 % precision and 30 % recall.
• License: Apache 2.0, like the base model.

The project report, with the goal, method, experiments, results and limitations, is in REPORT.md.

Quick start

pip install -U huggingface_hub   # provides the hf command
hf download MarioTheKnight/booru-lighting-tagger --local-dir booru-lighting-tagger
pip install -r booru-lighting-tagger/requirements.txt
python booru-lighting-tagger/inference.py my_image.png

my_image.png: backlighting, sunset, warm_lighting, light_particles

At the first run, the base model (1.2 GB) is downloaded from the Hub and cached. If you need a specific CUDA build of PyTorch, install it first (pytorch.org).

Tag a dataset

# Add the missing lighting tags to each image's .txt caption (the file is created if needed)
python booru-lighting-tagger/inference.py dataset/ --recursive --append-to-captions

# Same, with spaces instead of underscores ("rim light")
python booru-lighting-tagger/inference.py dataset/ -r --append-to-captions --spaces

# Keep every score, to apply your own thresholds
python booru-lighting-tagger/inference.py dataset/ -r --jsonl lighting.jsonl

Option	Effect
-r, --recursive	Look into subfolders.
--append-to-captions [EXT]	Add the missing tags at the end of <image name>.txt (or EXT), comma-separated. Existing tags are kept and never duplicated (rim_light and rim light count as the same tag).
--write-captions [EXT]	Write the lighting tags alone to <image name>.lighting.txt (or EXT), replacing that file.
--jsonl FILE	Write every result to a JSON Lines file: all scores, tonal key probabilities, tags.
--spaces	Write rim light instead of rim_light.
--preset recall	More tags: about 1.5 times as many, a little less precise (see Threshold presets).
--validated-only	Emit only the 29 validated labels (see Labels).
--no-tonal-key	Do not emit dark, dim_lighting or high_key.
--seeds N	Use only the first N of the 3 seeds: faster, slightly less accurate.
--repo ID, --revision v1.0.2	Download the model from the Hub instead of reading it next to the script.
--device, --precision, --batch-size, --workers	Hardware settings (--help for details).

Formats read: PNG, JPEG, WebP, BMP, GIF (first frame), TIFF. An unreadable file is reported and skipped; the run goes on and exits with code 2.

Python

import sys
from huggingface_hub import snapshot_download

model_dir = snapshot_download("MarioTheKnight/booru-lighting-tagger", revision="v1.0.2")
sys.path.insert(0, model_dir)
from inference import LightingTagger

tagger = LightingTagger(model_dir)              # device="auto", seeds=None (all 3), preset="default"
for result in tagger.tag_files(["a.png", "b.jpg"], batch_size=8):
    print(result["file"], result["tags"])

from PIL import Image
result = tagger.tag_images([Image.open("a.png")])[0]
print(result["scores"]["rim_light"])            # 0 to 10
result = tagger.tag_images([Image.open("a.png")], preset="recall")[0]   # more tags, for this call only

Output

One result per image (abridged):

{
  "file": "a.png",
  "tags": ["backlighting", "sunset", "warm_lighting", "dim_lighting"],
  "scores": {"backlighting": 7.84, "rim_light": 3.12, "sidelighting": 0.02, "...": "..."},
  "tonal_key": "dim_lighting",
  "tonal_key_probabilities": {"dark": 0.01, "dim_lighting": 0.86, "normal": 0.13, "high_key": 0.0},
  "error": null
}

• scores: for each of the 39 labels, the rating the teacher would most likely give, from 0 (absent) to 10 (obvious and dominant). It is not a probability. A score of 5 means "hesitant", 8 "clearly present". The full scale is in REPORT.md.
• tags: the labels whose score reaches their threshold, highest score first, then the tonal key unless it is normal. By default, each threshold makes the tagger emit as many tags as the teacher did on the validation images. Thresholds are listed in tagger_config.json.

Threshold presets

The teacher is cautious: when a tag is emitted it is usually right, but many images that deserve it are missed. The recall preset (--preset recall, or preset="recall" in Python) lowers the thresholds to emit about 1.5 times as many tags as the teacher on the validation images. Four labels keep their default threshold, because their precision collapses otherwise: golden_hour, diffraction_spikes, glowing and night. A recall threshold is never higher than the default one: the recall preset only adds tags.

Preset	Mean precision	Mean recall
default	87 %	56 %
recall	78 to 83 %	67 %

Means over the 39 labels, on the human tests. The four exceptions were chosen on these tests, so the recall preset's figures on them are a little optimistic: 83 % precision on the tests, 78 % in cross-validation. Per-label figures are in REPORT.md. For finer control, keep every score with --jsonl and apply your own thresholds.

Speed and memory

	3 seeds (default)	1 seed
GPU, RTX PRO 6000 Blackwell, batches of 16	40 images/s; 3.8 GB of VRAM, 4.6 GB at peak	89 images/s; 1.3 GB, 2.1 GB at peak
CPU, 16 threads	about 3 s per image	not measured

On a GPU, the tagger runs in bfloat16 as in training (float16 on GPUs without bfloat16). On a CPU it runs in float32 and gives the same tags (largest score gap 0.05 on a check set). The thresholds were set for the average of the 3 seeds; with fewer seeds they still apply, a little less precisely (seed 0 alone: mean validation AP 0.758 against 0.766).

Labels

Family	Tag	What it means	Status	Precision	Recall	Human positives
Direction	backlighting	Main light behind the subject, facing the viewer	validated	91 %	62 %	32
Direction	rim_light	Bright edge outlining the subject	low recall	80 %	14 %	28
Direction	sidelighting	Light from one side, the other half in marked shadow	low precision	60 %	39 %	23
Direction	overlighting	Light from above, with marked downward shadows	low recall	86 %	12 %	50
Direction	underlighting	Light from below (lamp, fire, screen)	validated	89 %	55 %	31
Direction	spotlight	A targeted beam isolates the subject	validated	79 %	92 %	12
Direction	crack_of_light	Light entering a dark space through a gap	validated	100 %	58 %	12
Contrast	high_contrast	Deep blacks against bright highlights, few midtones	validated	80 %	59 %	27
Contrast	low_contrast	Compressed tones: soft, hazy, washed-out	low recall	100 %	21 %	14
Contrast	chiaroscuro	Light and shadow structure the whole composition	validated	100 %	57 %	35
Contrast	silhouette	Subject reduced to a dark outline against light	validated	86 %	33 %	18
Quality	dramatic_lighting	Light staged for mood, like a film shot	validated	77 %	64 %	42
Quality	volumetric_lighting	Light visible in the air: mist, dust, beams	low recall	100 %	12 %	48
Color	warm_lighting	Clearly warm light tinting the subject	validated	100 %	59 %	17
Color	cool_lighting	Clearly cool light tinting the subject	low recall	100 %	11 %	18
Color	golden_hour	Low golden sun after sunrise or before sunset	low precision	64 %	86 %	21
Color	red_light	Saturated red light tinting the scene	validated	87 %	74 %	27
Color	orange_light	Saturated orange light tinting the scene	low recall	100 %	29 %	21
Color	green_light	Saturated green light tinting the scene	validated	77 %	77 %	13
Color	purple_light	Saturated purple light tinting the scene	validated	85 %	55 %	20
Effects	light_particles	Shining particles floating in the air	validated	82 %	49 %	37
Effects	lens_flare	Lens artifact: streaks, halos, aligned circles	validated	78 %	50 %	14
Effects	diffraction_spikes	Star-shaped spikes around a bright source	validated	100 %	54 %	13
Effects	caustics	Rippling light patterns cast by water or glass	validated	100 %	70 %	23
Effects	refraction	Light or image bent through a transparent medium	validated	86 %	52 %	48
Effects	subsurface_scattering	Light glowing through skin, leaves, wax	validated (barely)	75 %	30 %	10
Effects	overexposure	Highlights blown out to pure white	does not work	–	0 %	24
Shadows	shadow	Strong or striking cast shadows	validated	83 %	45 %	11
Shadows	window_shadow	Light and shadow pattern from a window or blinds	validated	89 %	78 %	32
Shadows	dappled_sunlight	Patches of light through foliage	validated	95 %	90 %	21
Shadows	partially_shaded_face	Face split into a lit half and a shaded half	validated	92 %	46 %	24
Sources	sunlight	Sunlight emphasized	low precision	67 %	54 %	26
Sources	moonlight	Pale moonlight at night	validated	84 %	94 %	17
Sources	candlelight	Scene lit mostly by candles	validated	78 %	95 %	19
Sources	fire_light	Scene lit by the orange glow of a fire	validated	100 %	77 %	26
Sources	screen_light	A visible screen lights a character	validated	91 %	87 %	23
Sources	glowing	An object or character emits light	validated	81 %	74 %	23
Sources	night	Night scene	validated	91 %	71 %	14
Sources	sunset	Sun near the horizon, orange or red sky	validated	100 %	92 %	37

Precision and recall are measured at the tagger's default thresholds against blind answers from one human annotator, on 1,462 test images never seen in training. The test sets are small: the figures of a single label can move by 10 to 20 points.

• validated (29 labels): at least 70 % precision and 30 % recall, on at least 10 human positives. subsurface_scattering passes with exactly 10 positives and 30 % recall: treat it as fragile.
• low recall (6): the tag is usually right when emitted, but it misses many cases. The recall preset helps some of them (rim_light, cool_lighting, orange_light).
• low precision (3): the tag is emitted too often.
• does not work (1): overexposure is never emitted at its threshold, and its ranking is weak (AP 0.52, against 0.28 for random scores on the same questions). It is kept in the outputs for compatibility; do not rely on it.

In tagger_config.json, overexposure has the status low_recall, like the six labels above.

Version 1.0.1 added 535 blind questions on the 11 labels that had fewer than 10 human positives in v1.0.0. The images were drawn by booru tag without looking at any model's scores, with one question per image.

The tonal key (dark, dim_lighting, normal, high_key) is a single choice per image. It agrees with the teacher on 90 % of the validation images, but has no human reference.

The exact definitions, as given to the teacher, are in tagger_config.json. They share a few rules: the light is judged, not the drawing style (cel-shading, inking or an overall color grading do not count); an artificial source (screen, candles, fire) or its glow must be visible; the sun and the moon may be off frame.

Not in this version: 13 labels of the original taxonomy, which the teacher could not rate reliably: soft_lighting, hard_lighting, bloom, complex_lighting, colored_lights, blue_light, yellow_light, pink_light, vignetting, light_rays, colored_shadow, shaded_face, shade.

Tag names follow Danbooru where the tag exists there, otherwise e621.

How it was made

1. Taxonomy. 52 lighting labels in 7 families, plus the tonal key, each with a written definition.
2. Teacher. Claude Opus 5.5 rated each image from 0 to 10 on every label, following a fixed guideline (v2.1). The guideline was revised after discussing its disagreements with a human.
3. Human calibration. One annotator answered yes or no, blind and one label at a time, on test images. These answers set a threshold per label on the teacher's ratings, and led to dropping the 13 labels the teacher could not rate reliably.
4. Training data. 19,454 images from Danbooru, Gelbooru, e621, yande.re and Konachan, rated by the teacher (about 21 M tokens). Half were drawn at random, half by booru tag to enrich the rare labels. General and sensitive ratings only, without AI-generated images or comic pages. The split was 17,508 images for training and 1,946 for validation, with near-duplicates kept on the same side.
5. Student. WD EVA02-Large Tagger v3 with its last 16 blocks unfrozen (202 M trainable parameters). For each label, the head predicts P(rating ≥ t) for t = 1 to 10, and the score is their sum. 8 epochs, AdamW, horizontal flips only, since color or crop augmentations would change the lighting. Three seeds, whose scores are averaged.
6. Thresholds. For each label, the threshold that gives as many tags as the teacher on the validation images.

Details, experiments and ablations: REPORT.md. An example training script, for reference, shows the method: training/README.md.

Results

Model	Validation: mean AP against the teacher (39 labels)	Tests: mean AP against the human (28 labels)	Validated labels (of 28)
Random	0.04	0.46	0
WD tagger v3, matching Danbooru tags	0.56 (18 labels)	0.77 (16 labels)	11
Linear heads on the frozen WD tagger	0.59	0.82	21
Fine-tuned, 4 blocks	0.68	0.85	23
This model (16 blocks, 3 seeds)	0.77	0.87	23
Teacher (Claude Opus 5.5)	0.81 (second rating run)	0.78	24

This table uses the v1.0.0 test sets, where the teacher was measured too. With the questions added in v1.0.1, all 39 labels are measured: mean AP 0.85, and 29 validated labels. The teacher did not rate these new questions.

The tagger beats its teacher on the human tests partly because it learns the teacher's average rating, which smooths out the teacher's run-to-run noise. The v1.0.0 test questions were also selected from the teacher's ratings. See the report for the caveats.

Limitations

• Domain. Trained on anime, manga, furry and game-style illustrations from image boards. Photos, 3D renders, comic pages and AI-generated images were not in the training data and were not tested.
• Ratings. Only general and sensitive images were used. Behavior on explicit images was not measured.
• Recall over precision. The default thresholds copy the teacher's tagging rate, which is cautious: when a tag is emitted it is usually right, but many images that deserve it are missed. Use --preset recall, or --jsonl and your own thresholds, if you need more recall.
• It imitates a teacher. The tagger reproduces the judgment of Claude Opus 5.5 under one guideline, including its blind spots. The human reference comes from a single annotator.
• Weak spots. Light direction is the weakest family (validation AP 0.62). Ten labels missed the bar: rim_light, overlighting, low_contrast, volumetric_lighting, cool_lighting and orange_light have a low recall; sidelighting, golden_hour and sunlight have a low precision; overexposure does not work.
• Resolution. Images are padded to a square and seen at 448 × 448 px. Fine effects (small caustics, thin rims) can be lost, especially on very tall or wide images.
• Cost. Three EVA02-Large copies make it about three times slower than a single tagger; --seeds 1 trades a little accuracy for speed.

Intended use

• Completing the captions of image datasets before training or fine-tuning generative models.
• Searching, filtering or describing a collection of illustrations by lighting.
• Pre-sorting images before a more expensive annotation. On random validation images, sending the teacher only the 43 % of images that score high on at least one label was enough to recover 90 % of its tags on every label.

It is not meant to judge artistic quality, and it was not designed for photographs.

Versioning

Versions are MAJOR.MINOR.PATCH. MAJOR changes the labels or their meaning, MINOR brings new weights for the same labels, and PATCH only touches thresholds, scripts or documentation. Each version is a tag of this repository; pin one with --revision v1.0.2 or revision="v1.0.2". History: CHANGELOG.md.

License and attribution

Apache License 2.0 (LICENSE), for the weights, the code and the documentation.

This model is a modified version of WD EVA02-Large Tagger v3 by SmilingWolf, released under the same license. The changes are listed in NOTICE. The weight files contain only the fine-tuned parameters; the rest is loaded from the original repository, at the revision used for training.

The training labels were produced with Claude Opus 5.5 by Anthropic. The training images belong to their artists and are not distributed here.

Citation

@misc{booru_lighting_tagger_2026,
  title        = {Booru Lighting Tagger: lighting tags for anime-style illustrations},
  author       = {MarioTheKnight},
  year         = {2026},
  version      = {1.0.2},
  howpublished = {\url{https://huggingface.co/MarioTheKnight/booru-lighting-tagger}}
}