rukai2574/uw_nriqa

uw-benchmark

Unified underwater image quality assessment utilities.

This package provides one Python API and CLI for common underwater image evaluation workflows:

• No-reference underwater metrics: uciqe, uiqm, niqe, brisque.
• Deep no-reference adapters used in this lab: tuda, atuiqp, rsuia, uranker, uwiqf, nuiq.
• Experimental local NUIQ port: nuiq_local keeps the existing network-backed nuiq adapter untouched and uses the packaged SVM-rank model.
• Full-reference auxiliary metrics: psnr, ssim, mse, rmse, mae, lpips.
• Downstream enhancement benchmark: uw-seg-benchmark copies fixed underwater detection/segmentation splits, lets an enhancement method process the copied images, then reports Ultralytics box/mask metrics.

The pure/image-processing metrics are self-contained and are the default. Deep metrics remain in-process. Their minimal inference source code and checkpoints/configs are vendored in the Hugging Face Git/Xet repository, so a normal install from Hugging Face is intended to be self-contained for tuda, atuiqp, rsuia, uranker, and uwiqf. UW_NRIQA_ROOT is still supported as an override when you want to run against an external checkout of the original metric projects.

Weights are resolved in this order:

1. UW_NRIQA_ROOT/<relative-weight-path>
2. UW_NRIQA_MODEL_ROOT/<relative-weight-path>
3. packaged files under uw_nriqa/weights/<relative-weight-path>
4. Hugging Face Hub via UW_NRIQA_HF_REPO

The default repository is the Hugging Face Git/Xet repo rukai2574/uw_nriqa, where code and metric weights are stored together. The GitHub remote is a code-only mirror because several weights are larger than GitHub's 100 MB single-file limit. See docs/WEIGHTS.md for the Git/Xet weight workflow.

Environment Requirements

• Python >=3.10; Python 3.12 is supported for the packaged all install.
• For deep metrics, install with uw-benchmark[all]. This allows PyTorch >=2.2,<3 and torchvision >=0.17,<1; install a CUDA wheel that matches your GPU before installing the package when needed. Blackwell GPUs require a PyTorch build with CUDA 12.8 or newer, such as PyTorch 2.7+ CUDA 12.8 wheels. transformers is constrained to <5.
• A CUDA GPU is recommended for uwiqf, tuda, atuiqp, and uranker. CPU can run some adapters, but UWIQF is large and is intended for GPU use.
• Use Hugging Face Git/Xet or Git LFS when installing from the HF repo: set UV_GIT_LFS=1 for uv pip install or compatible pip frontends.
• The full uw-benchmark[all] install includes vendored model weights and needs several GB of disk space. UWIQF alone packages the DINOv3 backbone weight and the IQA checkpoint.

Install

Default install from Hugging Face. This pulls both code and vendored weights:

UV_GIT_LFS=1 uv pip install "uw-benchmark[all] @ git+https://huggingface.co/rukai2574/uw_nriqa"

If the Hugging Face repo requires authentication, use your HF SSH key:

UV_GIT_LFS=1 uv pip install "uw-benchmark[all] @ git+ssh://git@hf.co/rukai2574/uw_nriqa"

Install only the lightweight core from Hugging Face:

UV_GIT_LFS=1 uv pip install "git+https://huggingface.co/rukai2574/uw_nriqa"

Install with the downstream segmentation benchmark:

UV_GIT_LFS=1 uv pip install \
  "uw-benchmark[segmentation] @ git+https://huggingface.co/rukai2574/uw_nriqa"

GitHub mirror, code only:

pip install "git+https://github.com/lz59970062/uw_nriqa.git"

For local development:

pip install -e .

Usage

Evaluate one image with UWIQF using only packaged vendor code and weights:

from uw_nriqa import NRIQAEvaluator

img = "Augmented_Datasets/_UIEB/challenging-60/100001.png"
scores = NRIQAEvaluator(["uwiqf"], root="/tmp/uw_nriqa_no_external_root", cuda=True).predict_image(
    img,
    raise_errors=True,
)
print(scores)

Expected reference output for the current packaged UWIQF checkpoint is close to:

{"uwiqf": -0.7794}

Run several no-reference metrics on one image:

from uw_nriqa import NRIQAEvaluator

evaluator = NRIQAEvaluator(["uciqe", "uiqm", "niqe", "uwiqf"], cuda=True)
scores = evaluator.predict_image("enhanced.png", raise_errors=True)
print(scores)

Evaluate a folder from Python:

from uw_nriqa import evaluate_folder

scores = evaluate_folder(
    "results/challenging-60",
    methods=["uciqe", "uiqm", "uwiqf"],
    cuda=True,
)
print(scores)

Tensor input for online training/reward loops:

import torch
from uw_nriqa import NRIQAEvaluator, score_tensor

# BCHW, RGB, float values in [0, 1]. The tensor is detached internally.
images = torch.rand(8, 3, 256, 256)

# Returns a CPU float32 tensor with columns in method order.
scores = score_tensor(images, methods=["uranker", "uciqe", "uiqm", "niqe", "nuiq_local"], cuda=True)
print(scores.shape)  # [8, 5]

# Reuse loaded models through the evaluator API.
evaluator = NRIQAEvaluator(["uranker", "uciqe", "uiqm", "niqe", "nuiq_local"], cuda=True, preload=True)
per_method = evaluator.predict_tensor(images, raise_errors=True)
print(per_method["uranker"][0], per_method["niqe"][0])

Tensor direct mode currently supports uranker, uciqe, uiqm, niqe, and experimental nuiq_local. Inputs are detached, clamped to [0, 1], and rounded to the same uint8 grid used by the image-path evaluator. nuiq_local defaults to the service-compatible raw SVM-rank score. Use calculate_nuiq_local(..., normalize=True, group_ids=...) when you need the original MATLAB demo's group-wise mapminmax ranking stage. rsuia is intentionally excluded from the tensor reward path because its current implementation still depends on the legacy TensorFlow/libsvm adapter. The metric models are run under inference/no-grad behavior and are not part of the caller's autograd graph.

rsuia does use a batched evaluator path for folder scoring: images are stacked into a single Keras batch and the original RBF-SVR head is evaluated with a vectorized NumPy implementation. This accelerates evaluation, but it is still not a pure tensor reward path.

Full-reference metrics:

from uw_nriqa import ReferenceEvaluator

evaluator = ReferenceEvaluator(["psnr", "ssim", "lpips"])
scores = evaluator.predict_pair("enhanced.png", "reference.png")
print(scores)

Direct functions:

from uw_nriqa import calculate_psnr, calculate_ssim, calculate_uciqe, calculate_uiqm

print(calculate_uciqe("enhanced.png"))
print(calculate_uiqm("enhanced.png"))
print(calculate_psnr("enhanced.png", "reference.png"))
print(calculate_ssim("enhanced.png", "reference.png"))

Verify installed UWIQF weights are real payloads, not LFS pointer files:

from uw_nriqa.model_store import is_lfs_pointer, resolve_asset

p = resolve_asset(
    "UWIQF/checkpoints_ablation_attention_plus_worstk_randompool_sc_varlossch16_localdefect/"
    "checkpoint_epoch_19/model_state.pt",
    local_root="/tmp/uw_nriqa_no_external_root",
)
print(p, p.stat().st_size, is_lfs_pointer(p))

CLI

No-reference folder evaluation:

uw-benchmark results/challenging-60 --methods uciqe uiqm niqe --output nriqa.csv

Full-reference folder evaluation, matching files by filename:

uw-benchmark results/challenging-60 \
  --reference-folder references/challenging-60 \
  --reference-methods psnr ssim mse mae lpips \
  --output fr_iqa.csv

Grouped folder summaries:

uw-benchmark-by-folder all_results --methods uciqe uiqm tuda atuiqp rsuia uranker uwiqf

Downstream underwater enhancement benchmark:

uw-seg-benchmark prepare \
  --dataset liaci_seg \
  --destination /path/to/work/liaci_seg_eval

# Run your enhancement method on:
# /path/to/work/liaci_seg_eval/images/val

uw-seg-benchmark eval \
  --weight pretrained_liaci_yolov8n_seg \
  --dataset liaci_seg \
  --prepared /path/to/work/liaci_seg_eval \
  --device 0

This downstream task requires filenames, image count, labels, and image geometry to stay aligned. See docs/SEGMENTATION_BENCHMARK.md for the full copy-enhance-evaluate workflow, dataset resolutions, preprocessing rules, and metric interpretation.

Notes

• lpips, niqe, and brisque require optional packages (lpips/pyiqa).
• tuda, atuiqp, uranker, and uwiqf require PyTorch/torchvision/timm.
• uwiqf also uses accelerate, transformers, requests, and tqdm.
• rsuia requires TensorFlow and libsvm-official.
• nuiq remains the existing service-backed adapter. Use nuiq_local for the experimental in-process MATLAB port and tensor batch path.
• The deep adapters do not call subprocesses; models are loaded once and reused, so they can be integrated into training or evaluation pipelines.
• The default CLI only runs uciqe uiqm. Deep metrics must be explicitly requested.