Photo Finishing State Representation
State presentation is also critical for the success of the proposed RL policy. This is particularly important in challenging scenarios where the policy must tune parameters for unseen goals of any style.
Our experiment in Sec. 4.3 also shows that a simple concatenation of the input and goal images yields a sub-optimal result. Therefore, we design a comprehensive photo finishing state representation to extract features from observations that are critical for photo finishing tuning.

Specifically, our representation consists of three components: a CNN-based dual-path feature representation to encode both global and local features, a photo statistics representation to match the traditional photographic statistics between input and goal images, and an embedding of historical actions. Details of each component are described below.

Dual-path feature representation. In our dual-path feature representation, we seek to extract both global and local features from both input and goal images, inspired by gharbi2017deep . This is because the tuning task requires not only global image characteristics such as overall color, tone, average intensity, and scene category, but also local features associated with texture, highlight, and shadow.
As shown in Fig. 2 , the architecture begins with a stride-2 convolutional encoder to reduce spatial resolution and extract initial low-level features. It then splits into two paths: a local path and a global path. The local path \{L^{i}\}_{i=1...N_{L}} includes two stride-1 convolutional layers, preserving spatial resolution to extract local features. The global path \{G^{i}\}_{i=1...N_{G}} has one stride-2 convolutional layer and three fully-connected layers, providing a global scene summary vector. This global feature encapsulates essential global image characteristics such as overall color and tone, as well as a global notion of scene category (light condition or indoor/outdoor).
We fuse the global and local features by adding the global feature at each x,y spatial location of the local feature: F^{D}_{x,y}=\sigma(G^{N_{G}}+L^{N_{L}}_{x,y}) , where \sigma is the ReLU activation. This results in a dual-path feature representation F^{D} .

Photo statistics representation. Simply relying on a CNN-based policy may lead to unsatisfactory results with input and goal images outside of training distribution. To better represent invariant features across diverse styles and content of both input and goal images, we introduce a photo statistics representation, which matches traditional image statistics such as a histogram between input and goal.
Global photo statistics, such as histograms, are critical in global image processing operations, such as exposure control onzon2021neural , highlight, and shadow. Since they cannot be well represented by conventional convolutional neural networks due to limited receptive fields tseng2022neural , we propose to pre-compute these statistics and concatenate them into our state representation. Specifically, we compute histograms H_{rgb} on the RGB channels of input and goal images and map these to a fixed dimension using a linear layer H^{\prime}=\text{Linear}((H_{rgb}(I_{t});H_{rgb}(I_{g})) . This feature is then concatenated with the luminance, median, contrast, and saturation of both input and goal images to form the photo statistics representation F^{S} .

Policy network. At last, we combine the dual-path feature representation and photo statistics representation with a historical action embedding F^{H}_{t}=\text{Linear}(a_{1:t};\ell_{1:t}) , where \ell_{t} is the \ell_{2} -distance of image I_{t} and goal image I_{g} . As shown in Fig. 2 , the input of policy network can be formulated as s_{t}=\text{Concat}(F^{D}_{t};F^{S}_{t};F^{H}_{t}) . The policy network is a multi-layer perceptron network (MLP) that maps from the current state and the goal representation to the next action to take. We choose deterministic policy to directly output continuous action a_{t}=\mu_{\theta}(o_{t},g)=\mu_{\theta}(s_{t}) . We use the same architecture to estimate the value function for RL updates.