Neural Film Grain Rendering

Gwilherm Lesné, Yann Gousseau, Saïd Ladjal, Alasdair Newson

► To cite this version:

Gwilherm Lesné, Yann Gousseau, Saïd Ladjal, Alasdair Newson. Neural Film Grain Rendering. 2024. [hal-04667141](https://hal.science/hal-04667141v1)

HAL Id: hal-04667141

https://hal.science/hal-04667141v1

Preprint submitted on 2 Aug 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
NEURAL FILM GRAIN RENDERING

Gwilherm Lesné

LTCI
Télécom Paris
Institut Polytechnique de Paris, France
gwilherm.lesne@telecom-paris.fr

Yann Gousseau *

LTCI
Télécom Paris
Institut Polytechnique de Paris, France
yann.gousseau@telecom-paris.fr

Saïd Ladjal *

LTCI
Télécom Paris
Institut Polytechnique de Paris, France
said.ladjal@telecom-paris.fr

Alasdair Newson *

ISIR
Sorbonne Université
France
anewson@isir.upmc.fr

<figure><img src="image_2.png" /><figcaption>Figure 1: Example of grain rendering. From a clean image (left), our proposed method can render grain with arbitrary sizes (right).</figcaption></figure>

ABSTRACT

Film grain refers to the specific texture of film-acquired images, due to the physical nature of photographic film. Being a visual signature of such images, there is a strong interest in the film-industry for the rendering of these textures for digital images. Some previous works are able to closely mimic the physics of films and produce high quality results, but are computationally expensive. We propose a method based on a lightweight neural network and a texture aware loss function, achieving realistic results with very low complexity, even for large grains and high resolutions. We evaluate our algorithm both quantitatively and qualitatively with respect to previous work.

**Keywords** Film Grain · Style Transfer · Image Processing

<footnote>*Equal contribution</footnote>
# 1 Introduction

In the streaming, photographic and film industries, there is a great demand for efficient, high quality film grain reconstruction/synthesis algorithms. In the case of video streaming content, film grain is often lost due to compression, and thus must be recreated on the viewer's side. Furthermore, for professional photographers and film makers alike, obtaining the authentic feeling of film grain is crucial to the final artistic value of images. In the case of the recent "Dune" movie, the director wanted film grain so much that real film grain was actually scanned and added to the film a posteriori [17]. Other examples include "Sex Education" or "Bodies", two Netflix series using film grain.

Therefore, an algorithm to synthesize grain with high quality is of wide interest. Furthermore, since many of the target applications are videos, the algorithm must be computationally efficient. In this paper, we propose a method based on a neural network permitting an efficient synthesis of realistic film grain. Previous work can be separated into two categories, model-based and learning-based. In the first case, models are either too simplistic to generate the quality of film grain which is required for high resolutions or the algorithms are too computationally expensive to be feasible in real-world applications. In the case of learning-based approaches, of which there are few examples in the literature, the algorithms lack stochasticity, in the sense that they are not able to produce random realisations of a given film grain type. This is not adapted to movies, for which the grain should be different in each frame. Our method aims to address these issues. For this, we choose a small network architecture (with about 10k parameters) which takes the physical grain parameters as an input and a noise map. We employ a loss function inspired by the style transfer literature to train our network.

A case of particular interest is high resolution images with large grain sizes. Indeed, when film grain is smaller, and the resolution of the image is not large enough to view the grain's details, film grain resembles additive white noise. However, in the case of a knowledgeable amateur or a professional photograph, such a synthesis is not sufficient. In such cases, large grains with high resolution, it is crucial to use methods that go beyond simple additive noise models.

Our paper is structured as follows. We will first introduce related works, both those concerning film grain synthesis and some works for neural style transfer. We then present the proposed approach, describing the architecture and the training process inspired from style transfer techniques. Finally, we show qualitative and quantitative results illustrating the improvements introduced by our approach.

To summarize, our contributions are:

* a light neural network which can generate random examples of film grain, including physical parameters;

* the possibility to handle a continuum of different grain sizes with one model, including large grain sizes;

* high quality synthesis in high-resolution images;

* fast generation times (1500× faster than the reference) due to our lightweight network (~ 10k parameters);

We argue that a lightweight neural network is well suited for fast and realistic film grain synthesis. Indeed, film grain is a *local* image property (corresponding to small-range spatial correlations), so that a large neural network with many parameters is not required. On the other hand, since physically realistic approaches are extremely slow [19], it makes sense to learn how to render film grain in a training process. Thus, a lightweight neural network is a sensible tool for our purpose. Our method aims to learn the texture of the Boolean model, which is the most physically realistic model to date, but is computationally expensive. For this, we create a grain synthesis database using the work of Newson et al. [19], whose model is at the heart of most state-of-the art methods, and learn the grain on this database. However, our approach is several order of magnitudes faster than that of Newson et al..

# 2 Related work

In the following section, we will discuss previous works on both film grain rendering and style transfer, as we want to achieve the former using an approach inspired by the latter. Note that a separate literature on film grain removal/filtering exists, however this is not the goal of our method, so we do not discuss them here.

## 2.1 Film grain rendering

Film grain is a well documented phenomenon related to the acquisition on silver films [12] [30] [21]. During acquisition, a raw silver film (that is a gelatin layer on which transparent silver-halide crystals are spread) is used to capture light emitted in a visual scene. When exposed to light, these crystals may form silver specks with a probability following a Poisson distribution which parameter depends on the luminosity. Later on, during the development process, these silver specks are turned into opaque silver grains which form the image.
<figure><img src="image_3.png" /><figcaption>Figure 2: Our film grain rendering neural network architecture. $\delta$ corresponds to the grain size in pixel units.</figcaption></figure>

To simulate such behaviour, several methods have been proposed. An approach commonly used in image-processing softwares consists in adding to the digital image a pre-scanned film-grain image [9] [11]. However, this requires the software to have such images for each required type of grain. Another popular way to synthesize film grain is to approximate it as an additive locally-correlated noise which is obtained by filtering some white noise. If we call, respectively, $\tilde{u}$ and $u$ the grain-simulated and grainless images, we have $\tilde{u} \approx u + h * n$ where $n \sim N(0, \sigma^2)$, and $h$ is some convolution filter whose size is related to that of the grain. Stephenson and Saunders [26] proposed such an approach by using a Gaussian kernel blurring white noise in the Fourier domain and adding the simulated grain to the digital image through tone mapping. Both Yan et al. [31] and Moldovan et al. [18] also use the additive approach, only changing the white noise to a signal-dependent one. Similarly, in a more recent work, Zhang et al. [32] also use this framework and try to directly learn the noise kernel so that the resulting grain matches the theoretical second order statistics in the case of locally constant inputs. Even though these methods are quite efficient computationally and show satisfying results for low-resolution images, they are relatively rough approximations which are only effective for a limited range of grain size due to the strong assumptions made (i.e. the grains are completely described by their second order statistics).

Auto-regressive methods were also introduced [22] [7] [20]. However, such methods have their own shortcomings like the bias introduced by only taking into account the upper-left neighborhood of the rendered pixel or the high dependence on the first pixel rendered.

Another film grain model, based on the Boolean model from the stochastic geometry literature, was first investigated by Bayer et al. [5] to evaluate the connection between granularity and density of film grains. Subsequently, Newson et al. [19] used this model for the purpose of film grain synthesis. First, a continuous Boolean model is sampled with density depending on the input luminosity. Then, the grainy image pixel values are estimated by applying a filter, representing the optical acquisition system, to the Boolean model. While this method is the closest to the physics of film grain, it is extremely slow, around 45 minutes/50 seconds for 1024 × 1024 images using a Nvidia T600/P100.

More recently, two deep learning approaches have been published by Ameur et al. [3][2] with good visual quality. They first proposed a U-Net which takes as input both the grainless image and the grain size and outputs a grainy version. This output is later fed to a PatchGAN and the whole architecture is trained following the conditional GAN framework. This model is trained on a custom dataset, namely FilmGrain [3], generated using the method of Newson et al. on several natural image databases. Although their method gives realistic results and manages different grain sizes, their approach uses a large (~ 150K) amount of data and is deterministic (each run produces the same grain), which is not satisfactory and not applicable to movies. Following up this work, Ameur et al. proposed a second approach [2] using another generative architecture with an additional network whose role is to encode the grain parameters from a grainy image into a "style" vector, which is then fed to the generator. This method was trained on a new dataset, FilmGrainStyle740k [2], which contains more artistic grain parameters. However, while improving the performances, this method is affected by similar drawbacks. Firstly, while their method can take as input a random style vector, this does not allow for several random realisations of the same film grain style. Furthermore, the method still requires a large amount of data to be trained.
## 2.2 Style Transfer and texture synthesis

Style Transfer is a domain which is closely related to texture synthesis. Given an input image $u$, the goal is to impose a particular *style* of another image $u_s$, while maintaining the semantic/geometric content of $u$. The notion of style is in fact very similar to the visual appearance of textures.

Gatys *et al.* [10] were the first to consider deep neural features for characterizing textures. The core idea is to describe textures with some statistics of VGG19 [25] activations. Following this work, Ulyanov *et al.* [28] and Johnson *et al.* [14] proposed feed-forward networks to transfer a given style to any image. Subsequently, the Johnson *et al.*'s approach has been extended to any style through the introduction of Adaptive Instance Normalization (AdaIN) blocks [13].

# 3 Our method

Our work aims at overcoming the shortcomings from previous film grain rendering methods, in particular, the limited grain size range, long execution time and the deterministic nature of some algorithms. For this, we take advantage of efficient style transfer techniques, in particular the work of Huang and Belongie [13]. Contrary to the classical setting of style transfer, however, film grain cannot be summed up as a single texture, due to the fact that the physical parameters (e.g. the grain size) greatly influence the visual result, and must therefore be integrated into the model as a parameter. In the following section, we will first describe our architecture and then its training process.

## 3.1 Architecture

Our model takes 3 inputs:

* $u \in [0, 1]^{N \times N}$ the grain-free image on which we want to render film grain.

* $n \sim N(0, I_N)$ a white noise map. We use it to introduce stochasticity into our network in order to avoid getting the same output twice if we take the same input image.

* $\delta \in \mathbb{R}$, which corresponds to the grain size in pixel units. Indeed, our primary goal is to carry out film grain rendering at any grain size. Hence, we need to provide this information to our model.

We first concatenate the grain-free image $u$ and the white Gaussian noise map $n$ across the channel dimension, and pass it through a first convolutional layer. Then, our architecture is composed of 2 residual blocks made of convolutional layers, followed by AdaIN modules. As Huang *et al.* [13] showed, style can be encoded as the mean and variance of hidden feature maps. Since film grain can be viewed as a different texture/style for different grain sizes $\delta$, it makes sense to control the graininess of the output image by re-normalizing feature maps via AdaIN layers. More precisely, let $M_k^l$ be the k-th feature map of our network at the depth $l$. The AdaIN module in our network applies the following:

$$ \hat{M}_k^l = \Phi_k^l(\delta) \times \frac{(M_k^l - \mu(M_k^l))}{\sigma(M_k^l)} + \Psi_k^l(\delta) \quad (1) $$

where $\Psi_k^l$ and $\Phi_k^l$ are learnable linear transforms while $\mu$ and $\sigma$ denote the mean and standard deviation operators.

Finally, we use a last convolutional layer to reduce the number of channels to 1 and a tanh activation followed by a re-normalisation operation in order to get the output in the $[0, 1]$ image range. Note that contrary to Zhang *et al.*, we do not use a clipping operation to constrain our values in $[0, 1]$. The importance of this operation in their work is discussed in supplementary materials. A summary of the architecture is displayed on Figure 2. The size of our network, 10 000 parameters, is also discussed in supplementary materials.

We note that, while our work is inspired by arbitrary style transfer approaches [13], there are significant differences in our method. Firstly, standard style transfer networks are far too large for film grain, where we wish for a lightweight network; the less parameters, the faster our method will be. Furthermore, the auto-encoder architecture of their network leads to a very large receptive field, thus inducing long-range dependencies in the image. This behavior is the opposite of what we require for film grain rendering. Indeed, grain texture is a *local* property, corresponding to local correlations. Thus, the auto-encoder architecture is not relevant. Finally, [13] directly use the VGG activations of the style image as input to adjust the style, while we directly take into account physical parameters to control it.

To further induce local correlations, we add skip connections, ensuring that the network can take into account the input grayscale value at any layer of the network. We only use convolution and AdaIN layers, allowing our network to process images of any size. Thus, we do not need to adapt the architecture or retrain it for input images of different resolution, since the model is scale agnostic.
## 3.2 Training

In order to train our model, we require a set of film grain examples from which to learn. In the absence of a large database of scanned film samples, we use the method of Newson et al. [19], which is grounded in the physics of the grain phenomenon, to create the grainy image database, in way similar to [3]. This method is physically realistic, but also very slow. Thus, in this respect, our approach can be seen as an acceleration of the work of [19], which we consider as a reference. However, our model is completely generic, and can adapt to other types of grain if necessary (we would just need to retrain the network). We train our model on a custom database with 4000 triplets containing a grain-free image u, its grainy version $u_s$ and the matching grain size $\delta_s$.

### 3.2.1 Grain database creation : natural and synthetic images

We choose $N = 200$ grain-free images u, which are taken from two sources. The first 100 of these are natural images taken from BSD500 [4]. Contrary to previous work, we also decide to include synthetic images in the database, more precisely dead-leaves images, inspired by the recent work of Achddou et al. [1]. Indeed, we found that natural images often do not provide enough constant regions to learn the grain texture efficiently. In very textured images, it can be difficult to distinguish between natural texture and film grain. Furthermore, HR images may contain more constant regions, making it important to synthesize grain in these regions.

The utility of this data augmentation is confirmed by an ablation study on the database in the supplementary materials. To build our database, for each image, we compute 20 output images $u_s$ with grain. Except for the grain sizes and the number of Monte Carlo samples (fixed at $10^5$) all parameters are left with the default values of [19]. The grain sizes $\delta_s$ are sampled from a set of 32 values uniformly distributed in [0.025, 0.8]. The benefit of this approach is that one network is sufficient to render any grain size.

### 3.2.2 Training loss

Given u, let $u_s$ be the grainy version of u and $\delta_s$ its corresponding grain size. Let $F_i, i \in [1; 4]$ be the outputs of a pretrained VGG-19 network at several fixed depths in the network (at ReLU $i_{-1}$, $i \in [1; 4]$) and G our grain rendering network taking as input the grain-free image u and the grain size $\delta_s$. The network G also takes as input a Gaussian white noise map n. This input is not shown in the equations for simplicity.

To perform training we use the classical loss of AdaIN [13] for style transfer, consisting of two terms:

* A content loss which consists in the L2 distance between VGG19 activations of the synthesised image and those of the grainy reference:

$$L_{\text{content}} = \|F_4(u_s) - F_4(G(u, \delta_s))\|_2^2 \quad (2)$$

* A style term, where the VGG19 feature maps' moments of first and second order must match:

$$L_{\text{style}} = \sum_{i=1}^{4} \frac{1}{2^{i-1}} \left( \|\mu(F_i(u_s)) - \mu(F_i(G(u, \delta_s)))\|_2^2 + \|\sigma(F_i(u_s)) - \sigma(F_i(G(u, \delta_s)))\|_2^2 \right) \quad (3)$$

where $\mu$ and $\sigma$ are respectively the mean and the standard deviation of features.

Note that for the style loss, contrary to [13], we weight the different terms such that the shallower feature maps' moments have more importance. This modification is motivated by the fact that film grain is a local texture, hence, we prioritize the corresponding texture descriptors. For more details, an ablation study on the layers to take into account is provided in supplementary materials.

The total loss is the sum of these two terms with a weight $\lambda = 1$ controlling the trade-off between content and style:

$$L_{\text{total}} = L_{\text{style}} + \lambda \times L_{\text{content}} \quad (4)$$

We train our model with a batch size of 16 during 500 epochs, using an Adam optimizer with a learning rate of $10^{-3}$.

## 4 Results

In this section, we will discuss our results both qualitatively and quantitatively, showing that our approach overcomes principal limitations of competing methods. More precisely, we propose to compare our approach to Newson et al. [19],
<figure><img src="image_4.png" /><figcaption>Figure 3: Comparison between our method and the baseline (Newson *et al.*). The grain size used is respectively *δ* = 0.1 and *δ* = 0.2 for the up and bottom images.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Example of colored grain rendering with *δ* = 0.35.</figcaption></figure>

Zhang *et al.* [32] and Ameur *et al.* [3] when possible, that is, when the grain size is within the range they consider. There is unfortunately no code nor weights available for the work of Ameur *et al.* [2], so we could not compare to their approach.

**4.1 Qualitative evaluation**

First, we recall that all compared methods (including ours) are based on the Boolean model from Newson *et al.*. Thus, we will refer to it as the *baseline*.

Visually, our results are indeed very similar to it. A visual comparison between these two can be seen in Figure 3.
<figure><img src="image_6.png" /><figcaption>Figure 5: Three different outputs obtained from the same image ($\delta = 0.1$), illustrating that the added grain is different each time.</figcaption></figure>

<table><caption>Table 1: Comparison of grain statistics between our method, the baseline and Zhang *et al.*. The results are L1 norms between each model and the theoretical values of the statistics.</caption><thead><tr><th>Model<br>Scale</th><th>mean (↓)<br>10<sup>-3</sup></th><th>var (↓)<br>10<sup>-4</sup></th><th>covar (↓)<br>10<sup>-4</sup></th></tr></thead><tbody><tr><td><em>Newson et al.</em></td><td>3.335 ± 0.063</td><td>4.47 ± 0.11</td><td>0.18 ± 0.006</td></tr><tr><td><em>Zhang et al.</em></td><td><strong>4.269 ± 0.075</strong></td><td>9.65 ± 0.15</td><td>0.32 ± 0.007</td></tr><tr><td><em>Ours</em></td><td>6.665 ± 0.065</td><td><strong>4.75 ± 0.10</strong></td><td><strong>0.20 ± 0.007</strong></td></tr></tbody></table>

We give another comparison with all the considered methods in Figure 6. We observe that Zhang *et al.* yields sharper results than the ideal result from Newson *et al.*. This is a direct consequence of the approximation they use, which is not in agreement with the physics of scanned silver halide films. Indeed, their method boils down to the addition of a linearly filtered noise to the image. Edges therefore remain sharp, which is not expected with the realistic physical model. The method of Ameur *et al.* gives overall satisfying visual results, but it can also produce strong artefacts, as can be seen in the first row, right column of Figure 6 (see supplementary materials for further examples).

We also provide a qualitative comparison for large grain sizes, applied to high-resolution images in Figure 7. As discussed in the introduction, satisfactorily dealing with such images is one of our main objectives, since filmmakers and photographers are interested in such HR images. We notice that for this setting, our method is still very similar to the baseline. In comparison, the method of Zhang *et al.* shows significant differences when the image is zoomed in. Indeed, by looking at the middle row of Figure 7, a zoom on the pupil of the input eye image, we can clearly see the circular grains appearing for the baseline and our approach, while Zhang *et al.* gives quite a different texture. This behavior is in agreement with the claim of the authors that their approach is only relevant for grain sizes $\delta \le 0.2$. We also recall that Ameur *et al.* is not compared for this rendering task since their provided code only accepts 5 small grain sizes: 0.01, 0.025, 0.05, 0.075 and 0.1.

Our method can also be employed to render color film grain. Since the film-based colored image acquisition consists in using superposed layers of silver film with different sensitivity to light, we can assume that each color channel is independent. Hence, we only need to apply our method to each channel separately, giving the results shown on Figure 1, 4, 6 and 7.

A final key property of our method is its stochasticity. Indeed, we want to be able to generate different samples of grain for the same image. This behavior is very important in movie production since actual film grain gives different realizations for each frame. Figure 5 shows the stochasticity of our method: our model has successfully used the input white noise as a source of randomness.
<table><caption>Table 2: Comparison of perceptual metrics between our method, the baseline and Zhang et al.. Values quantify the distance to Newson's model. The row with "Newson" shows the inner variability of their (stochastic) method.</caption><thead><tr><th>Model</th><th>DISTS<br>(↓)</th><th>SSIM<br>(↑)</th><th>LPIPS<br>(↓)</th><th>SIFID<br>(↓)</th><th>Gatys<br>(↓)</th><th>JSD-NSS<br>(↓)</th></tr></thead><tbody><tr><td>Scale</td><td>10<sup>-2</sup></td><td>10<sup>-1</sup></td><td>10<sup>-1</sup></td><td>10<sup>-1</sup></td><td>10<sup>-6</sup></td><td>10<sup>-4</sup></td></tr><tr><td>Newson</td><td>8.41</td><td>3.37</td><td>1.18</td><td>1.29</td><td>0.11</td><td>0.02</td></tr><tr><td>Zhang</td><td>15.97</td><td>3.08</td><td>1.87</td><td>26.83</td><td>50.03</td><td>60.59</td></tr><tr><td>Ours</td><td><b>9.66</b></td><td><b>3.30</b></td><td><b>1.20</b></td><td><b>2.13</b></td><td><b>0.20</b></td><td><b>0.32</b></td></tr></tbody></table>

## 4.2 Quantitative evaluation

We propose here several metrics to evaluate our approach. Firstly, we compare the statistics of our grainy images with those of the baseline and of Zhang et al.. Furthermore, we employ perceptual metrics commonly used in evaluation of image similarity and quality. The statistical evaluation can be seen in Table 1, while the perceptual metrics are shown in Table 2 and Table 3. As we wish to evaluate these statistics on a wider range of grain sizes than is possible with Ameur et al.'s implementation, we only compare the statistics with Zhang et al.. Afterwards, we provide a complete comparison of perceptual metrics including Ameur et al..

### 4.2.1 Theoretical statistics

We propose to compare our algorithm's outputs statistics with the theoretical values of the Boolean model, which our method is linked to, since the images in our database are generated with the work of Newson et al. We evaluate our method under the assumption of the Boolean model with constant grain radius and locally constant images. Indeed, in this case, the theoretical values of mean, variance and covariance can be explicitly calculated as noted by Zhang et al. [32]. Let $u[p]$ be the input image value at pixel position $p$, and $\tilde{u}$ the grainy image. We have:

$$ \mathbb{E}(\tilde{u}[p]) = u[p] \qquad (5) $$

$$ Var(\tilde{u}[p]) = \frac{1}{2\sigma^2} \int_0^{2\delta} e^{-\frac{x^2}{4\sigma^2}} (1 - u[p])^2 (e^{f(p,x,\delta)} - 1) \, x \, dx \qquad (6) $$

where

$$ \begin{gathered} f(p, x, \delta) = \frac{1}{\pi} \log \left( \frac{1}{1 - u[p]} \right) (\alpha - \sin(\alpha)) \qquad (7) \\ \text{with } \alpha = 2 \cos^{-1} \left( \frac{x}{2\delta} \right) \end{gathered} $$

$$ \begin{gathered} Cov(\tilde{u}[p], \tilde{u}[p + \Delta_p]) = \\ \frac{e^{-\frac{||\Delta_p||^2}{4\sigma^2}}}{4\pi\sigma^2} \int_{x=0}^{2\delta} \int_{y=0}^{2\pi} e^{-\frac{x^2 - 2x[\cos y, \sin y] \cdot \Delta}{4\sigma^2}} (1 - u[p])^2 (e^{f(p,x,\delta)} - 1) \, x \, dxdy \qquad (8) \end{gathered} $$

These formulas are directly derived from the ones given by Zhang et al.. The goal of this statistical evaluation is to make sure that the basic texture properties of the database examples are maintained by our method. In the case of non-constant images, such formula cannot be derived, and we will rely on perceptual metrics.

Since the previous formulas suppose a locally constant gray-level, we evaluate these statistics on a piecewise constant image made of 64x64 constant patches, each having a unique value in the range [0, 255]. We define our metric as the L1 distance between the theoretical values of the mean (Eq. (5)), variance (Eq. (7)) and covariance (Eq. (8)) of each patch, and the empirical values.

We apply this process 60 times for every grain size $\delta$ and take the average of the L1 distance between the theoretical and empirical values. These results are shown in Table 1. The covariance is computed for a 7 × 7-square neighborhood.

We provide further plots showing the metrics for each grain size taken individually in the supplementary materials. We do not compare those statistics with Ameur et al. since it is not able to generate grain for constant areas as pictured in
<table><caption>Table 3: Comparison of perceptual metrics between all methods on 4 grain sizes that Ameur *et al.* allow (0.025, 0.05, 0.075 and 0.1).</caption><thead><tr><th>Model</th><th>DISTS<br>(↓)<br>10<sup>-1</sup></th><th>SSIM<br>(↑)<br>10<sup>-1</sup></th><th>LPIPS<br>(↓)<br>10<sup>-2</sup></th><th>SIFID<br>(↓)<br>10<sup>-1</sup></th><th>Gatys<br>(↓)<br>10<sup>-7</sup></th><th>JSD-NSS<br>(↓)<br>10<sup>-6</sup></th></tr></thead><tbody><tr><td><em>Newson</em></td><td>0.35</td><td>8.65</td><td>2.19</td><td>0.30</td><td>0.12</td><td>0.25</td></tr><tr><td><em>Zhang</em></td><td>1.29</td><td>7.49</td><td>14.84</td><td>59.55</td><td>543.52</td><td>3946</td></tr><tr><td><em>Ameur</em></td><td>1.15</td><td>7.74</td><td>6.74</td><td>12.01</td><td>30.95</td><td>7.49</td></tr><tr><td><em>Ours</em></td><td><strong>0.55</strong></td><td><strong>8.54</strong></td><td><strong>2.92</strong></td><td><strong>1.72</strong></td><td><strong>0.85</strong></td><td><strong>1.72</strong></td></tr></tbody></table>

the first row of Figure 6. We show, for all their handled grain size, the exact outputs of their method applied to our test image in supplementary materials.

As we can see, all the evaluated approaches are quite close to the theoretical values of the statistics. This is expected for Zhang *et al.* since they exactly reproduce second order statistics of the Boolean model into a gaussian field. However, as we will see further in section 4.2.2, their method is not able to reproduce more complex features captured by the perceptual metric. We notice that our mean error is slightly larger than the competing methods. We believe this is due to our training loss. Since we train ours with a perceptual loss instead of the MSE, we obtain results with less close pixel-to-pixel correspondence. Nevertheless, we note that this error corresponds to less than 2 gray levels. Even though the previous quantitative evaluation gives insights about the fidelity of the generated grain, such statistics may not necessarily be aligned with human perception. For example, taking a white noise with a modulated variance would result in very good performances for the mean and variance, while not corresponding at all to a realistic looking film grain. Therefore, we also evaluate our model on several perceptual metrics.

## 4.2.2 Perceptual metrics

We now evaluate our results with perceptual metrics which are designed to reflect the human visual sensitivity. We used SSIM [29], DISTS [8], LPIPS [33], SIFID [24], the Gatys loss [10] and JSD-NSS [6], which are commonly used in the texture or image quality evaluation literature. Before discussing the results, we first explain what these metrics attempt to quantify.

* SSIM: standard metric to quantify the structural similarity between two images using first and second order moments;

* DISTS: highly inspired from SSIM as it evaluates structure and texture similarity but using neural network activations;

* LPIPS: evaluates the perceptual content similarity between two images by computing the L2 norm between their activations of a pretrained network. This metric corresponds to our content term $L_{content}$ when using VGG19. To avoid any bias, we use this metric on AlexNet [15].

* SIFID: deviation between the distribution of patchwise deep-features from the rendered image and the grainy reference.

* Gatys loss: computes the L2 norm between the Gram matrices of the VGG19 network activations for the rendered image and the reference texture. This measures the texture similarity between two images. Note that it does not correspond to our style term $L_{style}$ since we use the mean and variance of activations instead of Gram matrices as descriptors.

* JSD-NSS: This metric, used in [3], first computes correlations on locally standardized images and further compares them by taking the Jensen-Shannon divergence [16] of their histograms.

We compute these metrics on the reference $u_s$ image and the grainy output $\tilde{u}$. Again, we quantify the methods for different grain sizes, using as input 60 natural images from Country211 [23] dataset. To obtain our final results, we average the values across all the grain sizes $\delta$ and images. The results given in Table 2 are those with every grain size, while Table 3 shows the results for the 4 grain sizes [3] can generate. The fact that the distance between Newson *et al.*'s $\tilde{u}$ and $u_s$ is not perfect (0 or 1 depending on the metric) is due to the variability of their method. Still, it is expected that Newson *et al.* performs the best, their value reflecting the inner method variability. Again, for further details, we also provide in supplementary materials the metric values for each individual grain size.
<table><caption>Table 4: Comparison of speed for different methods. The quantity are evaluated on a Nvidia P100.</caption><thead><tr><th rowspan="2">Model</th><th colspan="3">Execution time (s) (↓)</th></tr><tr><th>256 × 256</th><th>1024 × 1024</th><th>2048 × 2048</th></tr></thead><tbody><tr><td>Newson et al.</td><td>5.492 ± 0.869</td><td>50.406 ± 0.162</td><td>198.381 ± 0.547</td></tr><tr><td>Zhang et al.</td><td><strong>0.001 ± 0</strong></td><td><strong>0.009 ± 0.001</strong></td><td><strong>0.037 ± 0.000</strong></td></tr><tr><td>Ameur et al.</td><td>1.290 ± 0.008</td><td>3.392 ± 0.003</td><td>9.130 ± 0.031</td></tr><tr><td>Ours</td><td>0.011 ± 0.002</td><td>0.033 ± 0.000</td><td>0.133 ± 0.002</td></tr></tbody></table>

As we can see, our perceptual metrics are quite close to the baseline shown at the row “Newson et al.”. Since it is our target, reflecting physical film grain, we can at most expect our method to give equivalent results, which is the case here. The remaining gap between theirs and ours can be explained by the fact that our network tends to have difficulties in learning the exact mean value of the image (see Table 1), hence our metric values are slightly biased, explaining the difference. However our proposed method is still the second best performing and moreover, it is much faster due to the use of a light neural network, as shown in Table 4. We recall that the execution time is a key concern for industrial applications.

We note that the method of Zhang et al. [32] is also fast. However, we notice the large difference in terms of results for the SIFID, Gatys and JDS-NSS metrics. This behaviour can be explained by the approximation which they use of the Boolean model. While this applies to small grain sizes (as noted by the authors), it is not sufficient for larger grains, which interest us. The consequence of this is that their method produces sharper results than those of the baseline Boolean model which is reflected in the perceptual metrics, since they are sensitive to such differences.

Finally, an important detail in the method of Zhang et al. [32] is that the output image is clipped to [0,1]. This is problematic, since in the case of large grain sizes, it greatly modifies the histogram of the output image, as can be seen in supplementary material. Indeed, their simplifying hypothesis (additive filtered noise) means that the gray-levels can exceed 1. Thus, clipping modifies their original model.

Finally, while the results of Ameur et al. are satisfactory, we perform slightly better, showing that our approach, in spite of its much lighter network, produces good results.

### 4.2.3 Execution time

To verify our performances compared to the competing methods, we provide in table 4 the execution times. We recall that for every method, grain size does not influence execution time. It is clear that our method is several order of magnitudes faster than the baseline approach of Newson et al.. The only method which is faster is that of Zhang et al., however, its approximation of the Boolean model is not good enough for the large grain, high resolution situations which interest us. We also note that our approach is much faster compared to the one from Ameur et al.. This is explained by the frugality of our network, which is made of less than 10 000 parameters compared to 32 447 000 for theirs. As an other performance metric, we note that our network needs $10^4$ MACs/pixel to render a grainy image.

## 4.3 Grain size estimation

For video coding, generating grainy images is only one part of the complete pipeline. Indeed, it is also necessary to estimate the grain parameters before coding the video in order to produce similar images. We recall that video coding is a process that tends to cut high frequencies, and therefore delete the grainy appearance of images. To this end, we show that estimating the grain size from a grainy image is indeed possible. For that, we build a classical CNN made of 3 convolutional blocks and 2 dense layers and we train it on the same type of dataset as presented before, using grainy images as input and grain size as label, giving a L1 error of 0.009. For further information, please look at the supplementary materials.

# 5 Conclusion and Future work

We have presented a film grain rendering method based on a neural approach grounded in the style transfer literature. Our architecture is lightweight, leading to very fast rendering times, which is crucial for real-world applications. The model is trained on a database of images obtained from a physically-based method, from which it inherits its realistic
results, without the associated computational burden. It allows for rendering images depending on the physical grain size, which is important for artistic control of the result.

We noted that the video content streaming industry requires grain added to videos on the viewer's end, to this end, a first possible extension of this work is its inclusion into a complete video coding pipeline, which is of importance for this industry. Furthermore, as noted in [19], several grain shapes could be taken into account, increasing the number of hyper-parameters to learn. In our case, a direct way to handle such a parameter would be to concatenate it to the grain size as input of the AdaIN layers. Another possible extension is to adapt our approach for the purposes of dithering, to accommodate greater compression rates while maintaining high perceptual quality. Finally, a further perspective would be to adapt the method to deal with RAW images, using them as maps of captured photons. However, doing so would require to also modify the approach of Newson et al. in order to be faithful to the film acquisition process.

## Acknowledgements

This work was granted access to the HPC resources of IDRIS under the allocation 20XX-AD011013749R1 made by GENCI.

We acknowledge the support of the French Agence Nationale de la Recherche (ANR) under reference ANR-21-CE23-0024 IDEGEN and ANR-19-CE40-005 MISTIC.

## References

[1] R. Achddou, Y. Gousseau, and S. Ladjal. Fully synthetic training for image restoration tasks. *Comput. Vis. Image Underst.*, 233(C), aug 2023.

[2] Z. Ameur, C.-H. Demarty, O. Le Meur, D. Ménard, and E. François. Style-based film grain analysis and synthesis. MMSys '23, page 229–238, New York, NY, USA, 2023. Association for Computing Machinery.

[3] Z. Ameur, W. Hamidouche, E. François, M. Radosavljević, D. Menard, and C.-H. Demarty. Deep-based film grain removal and synthesis, 2022.

[4] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik. Contour detection and hierarchical image segmentation. *IEEE Trans. Pattern Anal. Mach. Intell.*, 33(5):898–916, May 2011.

[5] B. E. Bayer. Relation between granularity and density for a random-dot model. *J. Opt. Soc. Am.*, 54(12):1485–1490, Dec 1964.

[6] L.-H. Chen, C. G. Bampis, Z. Li, and A. C. Bovik. Learning to distort images using generative adversarial networks. *IEEE Signal Processing Letters*, 27:2144–2148, 2020.

[7] J. Dai, O. C. Au, C. Pang, W. Yang, and F. Zou. Film grain noise removal and synthesis in video coding. In *2010 IEEE International Conference on Acoustics, Speech and Signal Processing*, pages 890–893, 2010.

[8] K. Ding, K. Ma, S. Wang, and E. P. Simoncelli. Image quality assessment: Unifying structure and texture similarity. CoRR, abs/2004.07728, 2020.

[9] DxO. Dxo, 2021.

[10] L. A. Gatys, A. S. Ecker, and M. Bethge. Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks. CoRR, abs/1505.07376, 2015.

[11] GrubbaSoftware. Truegrain 2, 2022.

[12] R. W. Gurney and N. F. Mott. The theory of the photolysis of silver bromide and the photographic latent image. *Proceedings of the Royal Society of London. Series A - Mathematical and Physical Sciences*, 164(917):151–167, 1938.

[13] X. Huang and S. J. Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. CoRR, abs/1703.06868, 2017.

[14] J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. CoRR, abs/1603.08155, 2016.

[15] A. Krizhevsky. One weird trick for parallelizing convolutional neural networks. ArXiv, abs/1404.5997, 2014.

[16] J. Lin. Divergence measures based on the shannon entropy. *IEEE Transactions on Information Theory*, 37(1):145–151, 1991.

[17] Y. Mendelovich. Dune was shot on alexa 1f, transferred to 35mm film, then scanned back to digital, 2021.
[18] T. M. Moldovan, S. Roth, and M. J. Black. Denoising archival films using a learned bayesian model. In *2006 International Conference on Image Processing*, pages 2641–2644, 2006.

[19] A. Newson, J. Delon, and B. Galerne. A Stochastic Film Grain Model for Resolution-Independent Rendering. In *Computer Graphics Forum*, 2017. Pre-publication in Computer Graphics Forum.

[20] A. Norkin and N. Birkbeck. Film grain synthesis for av1 video codec. In *2018 Data Compression Conference*, pages 3–12, 2018.

[21] P. Nutting. XXXII. on the absorption of light in heterogeneous media. *The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science*, 26(153):423–426, 1913.

[22] B. T. Oh, S. Lei, and C.-C. J. Kuo. Advanced film grain noise extraction and synthesis for high-definition video coding. *IEEE Transactions on Circuits and Systems for Video Technology*, 19:1717–1729, 2009.

[23] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. In *International Conference on Machine Learning*, 2021.

[24] T. R. Shaham, T. Dekel, and T. Michaeli. Singan: Learning a generative model from a single natural image. CoRR, abs/1905.01164, 2019.

[25] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.

[26] I. Stephenson and A. Saunders. Simulating Film Grain using the Noise-Power Spectrum. In I. S. Lim and D. Duce, editors, *Theory and Practice of Computer Graphics*. The Eurographics Association, 2007.

[27] R. Timofte, R. Rothe, and L. V. Gool. Seven ways to improve example-based single image super resolution. *2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 1865–1873, 2015.

[28] D. Ulyanov, V. Lebedev, A. Vedaldi, and V. S. Lempitsky. Texture networks: Feed-forward synthesis of textures and stylized images. CoRR, abs/1603.03417, 2016.

[29] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. Image quality assessment: from error visibility to structural similarity. *IEEE Transactions on Image Processing*, 13(4):600–612, 2004.

[30] G. Wernicke. Silver-halide recording materials for holography and their processing. *Zeitschrift für Physikalische Chemie*, 187(2):322–323, 1994.

[31] J. C. K. Yan. Statistical methods for film grain noise removal and generation. Master's thesis, University of Toronto, 1997.

[32] K. Zhang, J. Wang, D. Tian, and T. N. Pappas. Film grain rendering and parameter estimation. *ACM Trans. Graph.*, 42(4), jul 2023.

[33] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. CoRR, abs/1801.03924, 2018.
<figure><img src="image_7.png" /><figcaption>Figure 6: More qualitative results for grain rendering. For each row, $\delta$ are the following: 0.1; 0.2; 0.3; 0.1 and 0.05. Please zoom on the pdf for best grain viewing. Note that for grain sizes $\delta > 0.1$, Ameur et al. does not render film grain. Also, for flat regions, artefacts appear, as can be seen in the example on the top right (first row). For results of Zhang et al., images are much sharper than baseline (eg second row, right column, and last row, fourth column), due to their approximation of the Boolean model.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Figure 7: More qualitative results for large grain sizes. First row is $\delta = 0.8$ while last row is $\delta = 0.6$. The intermediate row is a zoom of the first row images onto the pupil zone. Please zoom on the pdf for best grain viewing. In particular, we notice that in the zoom on the pupil (second row), Zhang et al. are not able to correctly synthesise a Boolean model, due to their approximation by an additive model. This can be seen in the texture they produce, which does not resemble small grains, as does the baseline and our method.</figcaption></figure>
## A Loss ablation study

We provide here some insights about the importance of deep layers in the texture loss we use. Indeed, since film grain is a local texture, we could think of only taking the shallow layers of VGG19 as texture descriptors while dropping the deeper ones. Contrary to this idea, we show here that the deep layers still have some importance to reconstruct the grain. Indeed, when taking deeper layers, we show in Figure 9 and Figure 10 that we obtain better performances. We detail which layers are taken into account for each version of the style loss in Table 5. Moreover, we display in Figure 8 some artefacts that are introduced when only taking the 2 shallowest layers of our texture loss term.

<table><caption>Table 5: Table indicating which layers are taken into account for each version of our style loss</caption><thead><tr><th>VGG19 layer</th><th>2 layers</th><th>3 layers</th><th>4 layers</th></tr></thead><tbody><tr><td>ReLU1_1</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>ReLU2_1</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>ReLU3_1</td><td></td><td>✓</td><td>✓</td></tr><tr><td>ReLU4_1</td><td></td><td></td><td>✓</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>Figure 8: Image grained by our network with a 2-layers loss and $\delta = 0.05$. We can see that some artefacts appear in the rendered image.</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>Figure 9: Metric plots for every version of our loss.</figcaption></figure>
<figure><img src="image_11.png" /><figcaption>Figure 10: Metric plots for every version of our loss.</figcaption></figure>
## B Ablation study on networks architecture

To ensure that the network used could not be made smaller and still achieve satisfactory film grain rendering, we provide here an ablation study on the choices of architecture of our network. More precisely, we propose 2 variants of the chosen architecture, a bigger one and a smaller one. All the compared architectures are shown in Figure 11 and described in Table 6. We remind that the chosen one corresponds to the medium architecture. As it can be seen in Figure 12 and Figure 13, while the metrics are highly worsened by taking the smaller network, there is no gain when using the bigger one. Hence, for the sake of computational efficiency, the medium architecture is the one we kept.

<table><caption>Table 6: Description of all tested architectures. All convolutions are using padding='same' and stride=1.</caption><thead><tr><th>Small</th><th>Medium</th><th>Big</th></tr></thead><tbody><tr><td></td><td>3x3 (2x16) Conv<br>LeakyReLU<br>3x3 (16x16) Conv<br>LeakyReLU<br>Instance Normalisation<br>3x3 (16x16) Conv<br>LeakyReLU<br>AdaIN</td><td></td></tr><tr><td></td><td></td><td>3x3 (16x32) Conv<br>3x3 (16x16) Conv<br>LeakyReLU<br>Instance Norm<br>3x3 (16x16) Conv<br>LeakyReLU<br>AdaIN<br>3x3 (32x16) Conv</td></tr><tr><td></td><td></td><td></td></tr><tr><td></td><td>3x3 (16x16) Conv<br>LeakyReLU<br>Instance Normalisation<br>3x3 (16x16) Conv<br>LeakyReLU<br>AdaIN</td><td></td></tr><tr><td></td><td></td><td>3x3 (16x1) Conv + Tanh</td></tr></tbody></table>
<figure><img src="image_12.png" /><figcaption>Figure 11: Schemes representing the tested architectures. Top is 'Big', mid is 'Medium' and bottom is 'Small'. For more details, see Table 6</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>Figure 12: Metric plots for every version of our architecture.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Figure 13: Metric plots for every version of our architecture.</figcaption></figure>
C Ablation study on the dataset used

We provide here an ablation study on the dataset used to train our network. We propose three variants:

* A dataset fully made of natural images from BSDS500 [4]

* A dataset fully made of synthetic dead-leaves images. We provide some samples of such data on figure 14

* The hybrid dataset presented in the paper, half made of data from BSDS500, half made from synthetic dead-leaves images

For the three variants, we took 4000 grainy images to train our network. As we can see on the metrics graphs 15 and 16, the hybrid dataset called 'Mixed' performs the best out of the three variants. This behaviour is expected since training only on synthetic data suffers from domain shift, that is, the synthetic data doesn't represent well natural data which is more complex and textured. On the other hand, adding synthetic data to the natural images can help the network to learn more easily the expected behavior on constant regions.

<figure><img src="image_15.png" /><figcaption>Figure 14: Example of synthetic images on which our network is trained.</figcaption></figure>
<figure><img src="image_16.png" /><figcaption>Figure 15: Metric plots for different datasets used.</figcaption></figure>
<figure><img src="image_17.png" /><figcaption>Figure 16: Metric plots for different datasets used.</figcaption></figure>
D Detailed quantitative results

We plot below the metric values for each grain size taken individually. We recall that those metrics are calculated
over 60 natural images (from Country211 [23] dataset) unseen during training. As pointed out in the paper, Ameur *et al.*
is only evaluated on grain sizes 0.025, 0.05, 0.075 and 0.1 since their provided code doesn't handle higher values.
However, we acknowledge that their method could be adapted for larger grains.

<figure><img src="image_18.png" /><figcaption>Figure 17: Metric plots for competing methods and our proposed approach.</figcaption></figure>
<figure><img src="image_19.png" /><figcaption>Figure 18: Metric plots for competing methods and our proposed approach.</figcaption></figure>
E **Quantitative results on High-Resolution images**

We give here the results of our quantitative evaluation performed on high-resolution images from the L20 dataset [27]. As for the previous section, the metric values are plotted for each grain size taken individually. We couldn't compute values for Ameur *et al.* since the memory required was too large for our available GPUs (3 Nvidia P100). As we can see, the relative performance of the compared methods remain the same as for smaller resolution images, which is logical since the effect of the grain is local.

<figure><img src="image_20.png" /><figcaption>Figure 19: Metric plots for competing methods and our approach.</figcaption></figure>
## F Grain size estimation

We provide here more details on the grain estimation network we presented in section 4.3 of our paper. We display on Figure 20 the L1 error of our estimation network for each grain size taken individually. We recall that our mean L1 error across all grain size is 0.009. While not being at the core of the presented work, the simple estimation method we provide show that estimating the grain size from an image is a feasible task. This step could be added to a compression chain in the case of video coding, as addressed in [3] who proposes to filter the grain out of an image. Figure 21 illustrates a possible pipeline. For the sake of reproducibility, we give the exact architecture of our estimation network in Figure 22. We highlight the fact that we simply took the simplest possible model as a proof of concept.

<figure><img src="image_21.png" /><figcaption>Figure 20: Accuracy of our grain size estimation network for each size in our dataset. The Y axis corresponds to the L1 error and the X axis to the grain size on which the error is computed.</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>Figure 21: Film grain preserving video coding pipeline</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>Figure 22: Architecture trained to estimate the grain size from grainy images.</figcaption></figure>
G Importance of the clipping in the method from [32]

We show here on Figure 24 3 histograms of a rendered image with the methods proposed by Zhang et al., Newson et al. and ours. The grain size chosen is 0.8 and the input image was a constant 1024 × 1024 image of value 0.5. We can clearly see the saturation of pixel values due to the clipping operation at the end of the algorithm while it does not apply to the baseline nor our approach. On Figure 23, we show the output from a constant 1024 × 1024 image of value 0.5, using the method from [32], rendered with $\delta = 0.8$ and an alternative output obtained with a rescaling between 0 and 1 instead of the clipping proposed by the authors. We believe this study helps to better understand how much the clipping operation has an influence on the results for large grains.

<figure><img src="image_24.png" /><figcaption>Figure 23: Comparison between clipping (left) and rescaling as the last operation for the method of [32].</figcaption></figure>

<figure><img src="image_25.png" /><figcaption>Figure 24: Histogram of the output of Zhang et al.'s method compared to the baseline and our approach. We clearly see the saturation on the output image due to clipping at 0 and 1. All methods are applied to a 1024 × 1024 image of value 0.5 with grain size $\delta = 0.8$.</figcaption></figure>
## H Further methodological comparison with the method from [3]

As described previously, [3] proposes a deep learning approach with significant differences from ours. Firstly, their method needs more data to support less grain sizes with a bigger architecture. Indeed, their network is trained on 37 times more data than us for generating 5 grain sizes. Secondly, they rely on a L1 norm combined with an adversarial loss to train a PatchGAN architecture, instead of the texture loss and the AdaIn normalization that we use. Finally, they do not introduce stochasticity in their method, giving the same output for two forward passes on the same input image. We also show on Figure 25 adversarial examples of their method, using a grain size $δ = 0.1$. Indeed, their approach tends to create artefacts on flat regions. This behavior is especially visible on flat images. On Figure 26 we show the outputs of their method for the image we used to compute the grain statistics. As we can see, the rendered images doesn't look like grainy images. Although sometimes producing artefacts on flat regions, it is important to note that it doesn't seem to have an influence on the metrics we computed. An explanation for this could be that no artefacts were produced on the evaluation dataset (which comes from Country 211 [23]).

<figure><img src="image_26.png" /><figcaption>Figure 25: Adversarial examples of [3]</figcaption></figure>
<figure><img src="image_27.png" /><figcaption>Figure 26: Outputs of [3] for statistics computing. As we can see, there is no grain which is rendered. Please zoom in for better viewing</figcaption></figure>
# I More qualitative comparisons

We provide here more qualitative results. In particular, we show other comparisons between our approach, Newson *et al.* and Zhang *et al.* on Figure 27. As we see, the Zhang *et al.* approach gives sharper images and grains are not distinguishable for large sizes. Other examples of colored grain rendering are given on Figure 30 and Figure 31.

<figure><img src="image_28.png" /><figcaption>Figure 27: Comparison between our method, the baseline (Newson *et al.*.) and Zhang *et al.* [32]. The input is a natural image of squirrel fur. As can be seen, the work of Zhang *et al.* is visually further from the model which it is based on (Newson *et al.*.) than our method. Ours faithfully produces film grain at large grain sizes.</figcaption></figure>
<figure><img src="image_29.png" /><figcaption>Figure 28: Comparison between our method, the baseline (Newson *et al.*.) and Zhang *et al.* [32]. The input is a constant image of value 0.5. As can be seen, as the grain size increases, the work of Zhang *et al.* becomes visually further away from the model it is based on (Newson *et al.*). Our method, however, faithfully produces film grain at large grain sizes. Please zoom for details.</figcaption></figure>

We also show that our method is capable of rendering at any grain size in the interval [0.025, 0.8]. For that, we plot on Figure 29 some grained images rendered with grain sizes our model was never trained on. As we can see, the results are of the same quality as the ones shown for known grains (Figures 28) proving our claim.
<figure><img src="image_30.png" /><figcaption>Figure 29: Grainy examples for grain size which our method did not see during the learning process.</figcaption></figure>
<figure><img src="image_31.png" /><figcaption>Figure 30: More qualitative results for colored grain rendering.</figcaption></figure>
<figure><img src="image_32.png" /><figcaption>Figure 31: A qualitative result for a grain size $\delta = 0.4$.</figcaption></figure>