Learned Lightweight Smartphone ISP with Unpaired Data
Authors: name: Andrei Arhire, name: Radu Timofte

The Image Signal Processor (ISP) is a fundamental component in modern
smartphone cameras responsible for conversion of RAW sensor image data to RGB
images with a strong focus on perceptual quality. Recent work highlights the
potential of deep learning approaches and their ability to capture details with
a quality increasingly close to that of professional cameras. A difficult and
costly step when developing a learned ISP is the acquisition of pixel-wise
aligned paired data that maps the raw captured by a smartphone camera sensor to
high-quality reference images. In this work, we address this challenge by
proposing a novel training method for a learnable ISP that eliminates the need
for direct correspondences between raw images and ground-truth data with
matching content. Our unpaired approach employs a multi-term loss function
guided by adversarial training with multiple discriminators processing feature
maps from pre-trained networks to maintain content structure while learning
color and texture characteristics from the target RGB dataset. Using
lightweight neural network architectures suitable for mobile devices as
backbones, we evaluated our method on the Zurich RAW to RGB and Fujifilm
UltraISP datasets. Compared to paired training methods, our unpaired learning
strategy shows strong potential and achieves high fidelity across multiple
evaluation metrics. The code and pre-trained models are available at
https://github.com/AndreiiArhire/Learned-Lightweight-Smartphone-ISP-with-Unpaired-Data .