Introduction
Image processing pipelines (ISPs) are widely used by photographers and artists to retouch images to match their desired appearance. Existing pipelines like Adobe Lightroom and Darktable allow users to interactively tweak meaningful sliders such as exposure, white balance, and contrast, which control the pipeline to perform a series of non-destructive edits to the input image. Though users can manually tune the slider parameters, it is laborious and time-consuming even for experienced experts. To this end, automatic photo finishing algorithms have been introduced to automate the process and have drawn growing attention in the community hu2018exposure .

In this work, we aim to design an automatic tuning algorithm for black-box non-differentiable image processing pipelines. Although some preliminary research tries to propose fully differentiable image processing pipelines hu2018exposure ; gharbi2016deep or approximate the existing pipelines using neural networks tseng2022neural , they only support a limited set of image processing operations. On the other side, most commercial image processing pipelines, like Adobe Lightroom, are still black-box and non-differentiable, with a set of tunable parameters exposed to users. Under this setup, the goal of pipeline tuning is to automatically find the set of optimal parameters to achieve a desired image appearance, named the tuning target. More specifically, in this work, we study two different tuning targets. One is a target image with the same content as the input, but with a different rendering style, and we call it photo finishing tuning . The other is a target style image with different content, and the algorithm is to render the target in a similar way as the style target, and we call that photo stylization tuning .

One previous solution for photo finishing tuning is to use zeroth-order or first-order optimization. However, both of them are either time-consuming or limited to a small set of parameters. Zeroth-order optimizations hansen2006cma ; nishimura2018automatic ; mosleh2020hardware ; chen2017zoo are gradient-free searching methods and thus are normally very slow when the search space increases.
First-order optimizations tseng2019hyperparameter ; yu2021reconfigisp ; tseng2022neural accelerate the searching process using gradient descent, but they either require the processing pipeline itself to be differentiable, or a neural proxy is pre-trained to find a differentiable proxy of the original pipeline. For a complex commercial imaging pipeline, like cellphone camera pipelines, this proxy may not fully reproduce original pipelines tseng2019hyperparameter . Neural photo-finishing tseng2022neural improves the proxy accuracy by breaking a complex pipeline into small modules, but this does not apply to black-box or dynamically reconfigurable pipelines yu2021reconfigisp . Considering all these limitations, this brings a challenging question: Is there an efficient parameter-searching algorithm that is applicable to non-differentiable finishing pipelines?

To solve this challenge, we propose a novel goal-conditioned reinforcement learning (RL) approach dedicated to photo finishing tuning. At each RL iteration, the policy network takes the tuning target and the currently tuned image as input, and finds a better set of parameters that makes the finishing results closer to the target. This RL-based searching algorithm has several advantages over traditional optimization methods. Compared with the zeroth-order solution, the RL policy can more accurately predict the potential searching direction, while zeroth-order searching can only rely on less effective tries. As a result, RL searching is much more efficient. As shown in Fig. 1 , the RL-based solution (top row) only takes about 10 iterations to reach a similar PSNR as the 500-iteration output of a zeroth-order algorithm (bottom row). Also, compared with first-order optimization, RL-based tuning directly optimizes the non-differentiable image processing pipeline without a differential proxy. Therefore, it is not limited by the variety and complexity of image processing operations and pipelines, achieving much better tuning results. As shown in Fig. 1 , RL-based tuning reaches 38.92dB (top row), while the first-order solution only obtains 18.69dB (middle row).

To train an efficient RL policy for tuning, we also propose a novel state representation dedicated to this task. The state representation should model the relationship between the photo editing space and our policy. To achieve that, our state representation consists of three key components: a CNN-based feature representation to encode global and local features, a photo statistics representation to match the photographic statistics between the input and the goal, and an embedding of historical actions. These representations better fit the RL policy into our task and guide the policy to generate the next set of parameters effectively. Lastly, we also design reward functions for both photo finishing tuning and photo stylization tuning, enabling our framework to tune photos to different targets.

We validate the effectiveness of our framework with extensive experiments on both photo finishing tuning and photo stylization tuning. Our RL-based framework significantly outperforms previous methods in both tasks in terms of both efficiency and image quality. Experimental results demonstrate that our goal-conditioned policy is an efficient photo finishing tuner capable of performing fine-grained control on image processing pipeline parameters to achieve various goals.