Patent Description:
Image denoising aims to estimate the underlying clean image from its noisy observation. Denoising is an important step in many digital imaging and computer vision systems. <FIG> shows how the presence of noise can affect image quality. <FIG> shows the improvement of the image quality achieved by applying an image denoising technique to the noisy image of <FIG> (from <NPL>).

Camera sensors output RAW data in a linear color space, where pixel measurements are proportional to the number of photoelectrons collected. The primary sources of noise are shot noise, a Poisson process with variance equal to the signal level, and read noise, an approximately Gaussian process caused by a variety of sensor readout effects. These effects are well-modelled by a signal-dependent Gaussian distribution: <MAT> where xp is a noisy measurement of the true intensity yp at pixel p. The noise parameters σr and σs are fixed for each image but can vary from image to image as sensor gain (ISO) changes.

However, the noise in real images originates from various sources (for example, dark current noise and thermal noise) and is much more sophisticated. Although the noise in RAW sensor data is relatively well understood, in the RGB domain, the post-processing performed between capture and display (such as demosaicking, sharpening, tone mapping and compression) makes the noise model more complex, which makes the image denoising task more challenging.

For example, by taking the in-camera image processing pipeline into account, the channelindependent noise assumption may not hold true. In general, a realistic noise model as well as the in-camera image processing pipeline are important aspects in training CNN-based denoising methods for real photographs.

Traditional single-image denoising algorithms often analytically model properties of images and the noise they are designed to remove. Many methods have been developed using different mathematical tools and models, including partial differential equations, sparse coding and low-rank approximation. Most of these methods rely on very limited human knowledge or assumptions about the image prior, limiting their capability in recovering complex image structures.

Modern denoising methods often employ neural networks to learn a mapping from noisy images to noise-free images. Deep learning is capable of representing complex properties of images and noise, but training these models requires large paired datasets. As a result, most learning-based denoising techniques rely on synthetic training data.

By stacking convolution, batch normalization, ReLU layers and adopting the idea of residual learning, the DnCNN approach, as described in <NPL>, achieves a much higher PSNR index than conventional state-of-the-art approaches. Some complex networks, such as that described in <NPL>, have also been proposed.

The Generative Adversarial network (GAN) approach to denoising comprises a generator and a discriminator module commonly optimized with alternating gradient descent methods. The generator samples z from a prior distribution pz, such as a uniform distribution, and tries to model the target distribution pd. The discriminator D aims to distinguish between the samples generated from the model and the target (ground-truth) distributions.

Conditional GAN (cGAN), as described in <NPL>, extends the formulation by providing the generator with additional labels. The generator G typically takes the form of an encoder-decoder network where the encoder projects the label into a low-dimensional latent subspace and the decoder performs the opposite mapping, i.e. from low-dimensional to high-dimensional subspace. If s denotes the conditioning label and y denotes a sample from the target distribution, the adversarial loss is expressed as: <MAT> by solving the following min-max problem: <MAT> where wG, wD denote the parameters of the generator and the discriminator respectively. To simplify the notation, the dependencies on the parameters and the noise z have been omitted in the description below.

A recent method extending the conditional GAN approach is the Robust Conditional GAN (RoCGAN), as described in <NPL>. <FIG> schematically shows a conventional generator according to the RoCGAN approach. In this approach, the generator is augmented with an unsupervised pathway to encourage the outputs of the generator to span the target manifold even in the presence of large amounts of noise. The first pathway <NUM>, referred to as the reg pathway, performs a similar regression (denoising) as its counterpart in cGAN. It accepts a sample from the source domain (noisy image) and maps it to the target domain (clean image). The additional AE pathway <NUM> works as an autoencoder in the target domain.

In RoCGAN, the AE pathway contributes the following loss term: <MAT> where fd denotes a divergence metric (ℓ<NUM> loss), the superscript 'AE' abbreviates modules of the AE pathway, 'G' modules of the reg pathway and G(AE)(y(n)) = d(AE)(e(AE)(y(n))) is the output of the AE pathway.

Despite sharing the weights of the encoders, RoCGAN forces the latent representations of the two pathways to span the same space. To further reduce the distance of the two representations in the latent space, a latent loss term <IMG> is used. This term minimizes the distance between the encoders' outputs, i.e. the two representations are spatially close (in the subspace spanned by the encoders).

The feature matching loss enables the network to match the data and the model's distribution faster. The intuition is that to match the high-dimensional distribution of the data with reg pathway, their projections in lower-dimensional spaces are encouraged to be similar.

The feature matching loss is given by: <MAT> where π() extracts the features from the penultimate layer of the discriminator.

Skip connections can enable deeper layers to capture more abstract representations without the need to memorize all of the information. The lower-level representations.

By defining G(s(n)) = d(G)(e(G)(s(n))) as the output of the reg pathway, the final loss function of RoCGAN combines the loss terms of the original cGAN with the additional three terms for the AE pathway: <MAT> where λc, λπ, λae, λl and λd are hyper-parameters to balance the loss terms.

The AE pathway is an unsupervised learning method whose hidden layers contain representations of the input data for compressing (and decompressing) the data while losing as little information as possible. However, even in the presence of skip connections, the AE pathway is not capable of reconstructing all natural scenes and patterns. In other words, the use of one autoencoder to define a nonlinear manifold which can accurately reconstruct image patterns from a variety of real complex objects/scenes is not realistic. As a result, previous methods such as RoCGAN very often hallucinate complex image structures by introducing severe blurry effects or unnatural image patterns/artifacts.

The heavy computation and memory footprint of these methods also hinders their application on hardware constrained devices, such as smartphones or consumer electronic products. In addition, these methods try to exploit image priors to better model the clean image; something which is a very complex problem given the variety of all natural image patterns.

While the prevalence of smartphones makes them a convenient device for photography, their images are typically degraded by higher levels of noise due to the smaller sensors and lenses found in their cameras. This problem has heightened the need for progress in image denoising, particularly in the context of smartphone imagery.

It is therefore desirable to develop an approach to image denoising that overcomes these problems.

Further, <CIT> refers to a detail preserving image denoising method providing a detail preserving convolutional neural network (DRCNN) denoising model for a fuzzy visual effect and an artifact phenomenon of a denoised image caused by a large amount of missing detail information. The method comprises the following steps: step <NUM>, analyzing a minimization problem and constructing a denoising mathematical model; <NUM>, building a DRCNN generation module (GM) and a detail maintenance module (DRM); <NUM>, learning the noise of the image by using the GM, and subtracting the noise from the noise image to obtain an intermediate feature map (IFM); step <NUM>, learning detail information lost by the IFM by using the DRM, and adding the detail information and the IFM to obtain a denoised image; and step <NUM>, performing comparative analysis on an experimental result and a current advanced image denoising method.

<CIT> discloses a method for denoising captured astronomical images, wherein a dark noise database comprising dark noise images calibrated specific to the image capturing apparatus for the purpose of dark noise subtraction is employed. Dark noise libraries allow noise reduction algorithms to be implemented without the need to capture dark frames (i.e. dark frames are images with the similar or same exposure settings as the intended image, without exposing the imaging sensor to light) for noise reduction purposes after every image is captured.

The above mentioned problem is solved by the subject matter of the independent claims.

Further implementation forms are provided in the dependent claims. According to a first aspect there is provided an apparatus for denoising an image, as defined in appended claim <NUM>.

The apparatus may comprise an imaging device in which the image sensor is included, and the apparatus may be configured to generate the input image using the imaging device and to provide information indicative of the noise behaviour of the image sensor as input to the model. Providing further information as input to the model, such as the parameters of the noise model for the sensor, may result in improved image quality.

The noise pattern may be non-Gaussian. This may result in improved quality in the denoised image and accommodate more complex noise models which may be needed due to post-processing performed between capture and display, in processes such as demosaicking, sharpening, tone mapping and compression.

Described herein is a method for image denoising based on explicitly understanding the structure of the noise added by an image sensor to the images captured by the sensor. The method directly reconstructs the image noise. Using this approach, meaningful image structures may be better retained through the denoising process, which may enhance the image quality.

The goal of the method is to perform image denoising using reconstructed image noise that spans the target image signal-dependent noise manifold. The input to the image processor may comprise RAW image data or RGB image data. The image processor comprises a generator and a discriminator module each comprising a convolutional neural network (CNN), which may be optimized with alternating gradient descent methods. The generator samples from a prior distribution (for example, a uniform distribution), and aims to model the target distribution. The discriminator aims to distinguish between the samples generated from the model and the target (ground-truth) distributions.

A preferred embodiment of the CNN design of the generator is schematically illustrated in <FIG> schematically illustrates the inputs and outputs of the pathways of the generator during training of the model. An adversarial neural network is used based on an encoder-decoder generator with two pathway modules and shared decoder parameters. As illustrated schematically in <FIG>, the generator typically takes the form of an encoder-decoder network where the encoder projects the label into a low-dimensional latent subspace and the decoder performs the opposite mapping, i.e. from low-dimensional to high-dimensional subspace.

The generator comprises a first pathway <NUM> (herein referred to as the reg pathway <NUM>) and a second pathway <NUM> (herein referred to as the AE pathway <NUM>). Both the reg pathway <NUM> and AE pathway <NUM> are based on deep learning and may, for example, apply a CNN to process the image. A CNN learns a collection of filters, which are applied to the image through convolution. The convolution is designed to be spatially invariant, meaning the convolution has the same effect when applied to any location in the image.

Noising is a challenging process to be reversed by the few convolutional layers of the encoder in the reg pathway, especially in the object-independent scenario. To that end, a backbone network, shown at <NUM>, may be used prior to the reg pathway to extract complex feature representations useful to preserve later on the low and high image frequencies. The backbone network <NUM> may be a residual network (ResNet), as described in <NPL>, created by stacking building blocks as schematically depicted in <FIG>. Therefore, the input to the reg pathway <NUM> may be features extracted from the image to be denoised (for example, in the form of a tensor) rather than the image itself.

During training, the network learns the convolutional filter weights. This can be done using a plurality of training pairs comprising a reference clean RGB input image y, <NUM>, which is used as a ground truth (GT) image (i.e. a training image), and a noisy RGB input image s = y + v, <NUM>, where v is the real (GT) residual image <NUM>, which is a noise signature of the image. The noise signatures v, <NUM>, may be generated by a noise model and applied to the respective ground truth input image y, <NUM>, to give the noisy input image s, <NUM>. Initially, the convolutional filters are set to random values. The noisy RGB input image s, <NUM>, is input into the network, and the network regresses a denoised output image <NUM> (i.e. the predicted clean image). The difference between the regressed denoised output image <NUM> and the clean GT image <NUM> forms an error, which is then back-propagated through the network from the output to the input though gradients. The weights of the network are then updated to reduce the error. The training process iterates using a large collection of training images until the network weights converge.

During training, the AE pathway <NUM> learns how to reconstruct the image noise (the residual image). The second pathway <NUM> receives the ground truth residual image (noise signature) v, <NUM>, and the clean ground truth image y, <NUM>, and outputs a reconstructed ground truth residual image, G'(s), <NUM>. The input to the AE pathway <NUM> is the GT residual image (i.e. the noise signature) v , <NUM>, concatenated with the clean ground truth input image y, <NUM> (i.e. the input is v⊙y). In that way, the task of the AE pathway <NUM> is not to learn the underlying structure of a huge variety of complex image patterns, but to learn how image structures are affected by the presence of structured noise. By sharing the weights of their decoders, the generator adopts the residual learning strategy to remove from the noisy observation that image information which spans the image noise manifold.

As described above, the reg pathway <NUM> takes the noisy RGB image s, <NUM>, as input which is then processed by a backbone ResNet model <NUM> followed by a Unet, as described in <NPL>. The output of the pathway <NUM> is the predicted residual image (noise) G'(s) (i.e. an estimate of the noise in the image) which is then removed from the noisy input image <NUM> to obtain the denoised output image <NUM>, which is a predicted clean RGB image.

The reg pathway <NUM> implicitly removes the latent clean image with the operations in the hidden layers. Because of this, the unsupervised AE pathway <NUM> works as an autoencoder in the domain of the real residual image <NUM> (the noise signature of the training image), v = s - y.

Once the network is trained, during inference only the reg pathway <NUM> is applied to a noisy RGB input image to produce its denoised clean version. At inference, the first pathway can predict residual noise in an input image captured by an imaging device and subtracts the predicted residual noise from the input image to obtain the denoised output image.

The first pathway <NUM> therefore acts as generator, based on residual learning, and performs regression. The generator is augmented with the second pathway <NUM> which, during training, promotes the generator to remove from the noisy input image the residual noise which spans the target image signal-dependent noise manifold.

The present method directly reconstructs the image noise. Rather than directly outputting a denoised image, the first pathway <NUM> in the method described herein is designed to predict the ground-truth residual image G'(s), i.e., a first noise estimate which is the difference between the noisy observation s and the clean (ground-truth) image y.

In a further embodiment, this residual learning can benefit from any conditional information related to the specific image sensor type and/or the noise characteristics of the sensor, c, schematically shown at <NUM> in <FIG>. During training, this can be explicitly given to the generator in concatenation (denoted as ⊙) with s to get its output G'(s⊙c). In other words, the proposed conditional reg pathway <NUM> implicitly removes the latent clean image with the operations in the hidden layers. Because of this, the unsupervised AE pathway <NUM> works as a conditional autoencoder in the domain of v. The input to this pathway is v in concatenation with y and c. In this way, by explicitly giving y as additional input, the task of the AE pathway <NUM> is not to learn the underlying structure of a huge variety of complex image patterns, but to learn how image structures are affected by the presence of structured noise. By sharing the weights of their decoders, the generator adopts the residual learning strategy to remove from the noisy observation that image information which spans the image noise manifold.

In the case of image denoising in RGB domain, the images are <NUM>-channel based tensors. On the other hand, in RAW domain, each pixel in a conventional image sensor (linear Bayer sensor readings) is covered by a single red, green, or blue color filter, arranged in a Bayer pattern, such as R-G-G-B. Also, the information that c represents varies. In the case that the noise model of the image sensor of the imaging device is known, c could contain the two noise parameters σr and σs (both same for each pixel). In the case of more than one imaging device sensor, c may also contain one hot vector per pixel defining the imaging device ID (for example, camera ID) used to take each picture.

The model is configured to form estimates of noise patterns that are characteristic to each of a plurality of types of image sensors. The apparatus implementing the model is configured to receive an indication of the specific image sensor type of an image sensor that captured the input image, and the apparatus provides that indication as an input to the model. The model generates the estimate of the noise pattern in dependence on the indication. The model may further be provided with information indicative of the noise behaviour of the image sensor as input. This may assist the model to form estimates of noise patterns that are characteristic to specific types of image sensors.

<FIG> schematically illustrates an example of the use of more than one AE pathway <NUM>, <NUM>, in the generator which can be used in the case of multi-source image noise. Reg pathways are shown at <NUM> and <NUM>. Therefore, the generator of the image processor may comprise at least one further pathway having the same structure as the primary AE pathway. In this case, each AE pathway <NUM>, <NUM> can be responsible for removing noise information that comes from a specific noise source.

An example of the discriminator <NUM> of the image processor is schematically illustrated in <FIG>. The discriminator <NUM> accepts the predicted clean image, s - G'(s) (or s - G'(s⊙c)), <NUM>, along with y, <NUM>, as input.

In the method described herein, the content loss comprises two terms that compute the perpixel difference between the predicted clean image, and the clean (ground-truth) image. The two terms are i) the ℓ<NUM> loss between the ground-truth image and the output of the generator, ii) the ℓ<NUM> of their gradients, mathematically expressed as: <MAT>.

The loss term for the unsupervised (AE) module is: <MAT> where G'(AE)(v(n) ⊙ y(n) ⊙ c(n)) = d(AE)(e(AE)(v(n) ⊙ y(n) ⊙ c(n))) is the output of the AE pathway and fd represents an ℓ<NUM> loss due to the auto-encode in the domain of the residual image.

The feature matching loss is given by: <MAT>.

The final loss function is given by: <MAT> where λc, λcg, λπ, λae, λl, and λd are hyper-parameters to balance the loss terms.

In the examples described herein, the reg pathways <NUM>, <NUM>, <NUM> and the AE pathways <NUM>, <NUM>, <NUM> comprise "fully convolutional" networks.

<FIG> schematically illustrates an example of the Unet architecture which can be used in the approach described herein. The Unet uses an encoder-decoder architecture with two lateral Unet style skip connections. The encoder part of the network is shown generally at <NUM>, the decoder at <NUM> and the skip connections are shown at <NUM>. These skip connections <NUM>, which connect the intermediate layers of the encoder with the corresponding intermediate layers of the decoder <NUM>, enforce the network to learn the residual between the features corresponding to the predicted image noise and the actual image noise. This has the impact of faster convergence as empirically detected. The AE pathway takes as input the real (groundtruth) residual image which is then processed by a Unet similar to that one in the reg pathway. The output is the reconstruction of the real residual image.

The encoder part of the network, shown generally at <NUM>, processes the noisy RGB input with six consecutive layers. Each layer applies to its input a strided convolutional with 3x3 convolutional filters (together with a ReLU activation function and batch normalization). The strided convolution increases the number of filters (i.e. channels) by a factor of two while at the same time it reduces the spatial image resolution by a factor of two (i.e. from H, W, C to H/<NUM>, W/<NUM>, C). The image is processed at multiple scales and the network adapts to different frequency content. This produces output channels that capture features inherent in the data and relevant to the RGB image denoising task.

The decoder part of the network, shown generally at <NUM>, processes the output of the encoder with five consecutive layers of a Transposed Convolution operation with 3x3 convolutional filters (together with a ReLU activation function and batch normalization). The Transposed Convolution is an upsampling layer which increases the spatial resolution by a factor of two in each dimension (width and height) and decreases the number of filters by a factor of two.

The skip connections, schematically shown at <NUM> in <FIG>, may enable deeper layers to capture more abstract representations without the need to memorize all of the information. The lower level representations are propagated directly to the decoder through the shortcut. In the case of layers with a Unet style skip connection, the input to each of these decoder layers is a concatenation of i) the high resolution features from the encoding part related to the same spatial resolution and ii) the output of the previous decoding layer (i.e. spatially upsampled features). The subsequent convolution learns to assemble a more precise output based on the concatenated input. The input to each of the rest of the decoder layers is only the output of the previous decoding layer.

<FIG> represents a workflow of a method <NUM> for training a model to perform noise reduction on images. At step <NUM>, the method comprises receiving a plurality of training images <NUM>. At step <NUM>, the method comprises receiving a plurality of noise signatures <NUM>. Then, for each of the plurality of training images <NUM> the method comprises performing the following steps. At step <NUM> the method comprises selecting one of the plurality of noise signatures <NUM> and applying that noise signature <NUM> to the training image <NUM> to form a noisy input image <NUM>. At step <NUM>, the method comprises forming a first noise estimate in the noisy input image <NUM> by implementing a candidate version of the model on the noisy input image <NUM> and forming an estimate <NUM> of the respective training image <NUM> by subtracting the first noise estimate from the noisy input image <NUM>. At step <NUM>, the method comprises forming a second noise estimate <NUM> by implementing the candidate version of the model on the respective training image <NUM> and the selected noise signature <NUM>. At step <NUM>, the method comprises adapting the candidate version of the model in dependence on (a) a difference between the respective training image <NUM> and the estimate <NUM> of the respective training image and (b) a difference between the second noise estimate <NUM> and the selected noise signature <NUM>.

<FIG> schematically illustrates an example of an apparatus <NUM> including an imaging device 901a that can implement the method <NUM> described above. According to some embodiments, the imaging device 901a may be a camera. The imaging device 901a is connected to a communications network. The imaging device 901a comprises an image sensor <NUM>. The imaging device 901a also comprises a memory <NUM>, a processor <NUM> and a transceiver <NUM>. The memory stores in non-transient form code that can be run by the processor <NUM>. In some implementations, that code may include a data-driven model as described above. The model may include code that is directly executable by the processor and/or parameters such as neural network weightings which are not directly executable instructions but serve to configure other executable code that is stored in the memory <NUM>. The transceiver <NUM> may be capable of transmitting and receiving data over either or both of wired and wireless communication channels. For example, it may support Ethernet, IEEE <NUM>. 11B and/or a cellular protocol such as <NUM> or <NUM>.

Such an imaging device 901a typically includes some onboard processing capability. This could be provided by the processor <NUM>. The processor <NUM> could also be used for the essential functions of the imaging device 901a.

The transceiver <NUM> is capable of communicating over a network with other entities <NUM>, <NUM>. Those entities may be physically remote from the imaging device901a. The network may be a publicly accessible network such as the internet. The entities <NUM>, <NUM> may be based in the cloud <NUM>. Entity <NUM> is a computing entity. Entity <NUM> is a command and control entity. These entities are logical entities. In practice they may each be provided by one or more physical devices such as servers and datastores, and the functions of two or more of the entities may be provided by a single physical device. Each physical device implementing an entity comprises a processor <NUM> and a memory. The devices may also comprise a transceiver for transmitting and receiving data to and from the transceiver <NUM> of the imaging device 901a. The memory stores in a non-transient way code that is executable by the processor <NUM> to implement the respective entity in the manner described herein.

The command and control entity <NUM> may train the model. This is typically a computationally intensive task, even though the resulting model may be efficiently described, so it may be efficient for the development of the model to be performed in the cloud <NUM>, where it can be anticipated that significant energy and computing resource is available. It can be anticipated that this is more efficient than forming such a model at a typical imaging device.

In one implementation, once the model has been developed in the cloud <NUM>, the command and control entity can automatically form a corresponding model and cause it to be transmitted to the relevant imaging device 901a. In this example, denoising is performed at the imaging device 901a by processor <NUM>.

In another possible implementation, an image may be captured by the image sensor <NUM> and the image data may be sent by the transceiver <NUM> to the cloud <NUM> for processing. The resulting target image could then be sent back to the imaging device 901a, as shown at <NUM> in <FIG>.

Therefore, the method may be deployed in multiple ways; for example in the cloud <NUM>, on the device, or alternatively in dedicated hardware. As indicated above, the cloud facility could perform training to develop new models or refine existing ones. Depending on the compute capability near to the data corpus, the training could either be undertaken close to the source data, or could be undertaken in the cloud <NUM>, e.g. using an inference engine.

At inference, the imaging device 901a may implement, on the input image, the trained model so as to form a noise estimate in the image and to subtract the noise estimate from the input image to form a reduced-noise output image. At inference, the imaging device 901a may perform a method comprising the following steps: receiving an input image (i.e. a noisy image); implementing a trained artificial intelligence model to form an estimate of a noise pattern in the input image; and forming an output image by subtracting the estimate of the noise pattern from the input image, wherein the model is configured to form the estimate of the noise pattern such that the estimate of the noise pattern is representative of a noise pattern that is characteristic to a specific image sensor type.

The methods described herein have many advantages.

For the AE pathway <NUM>, the reconstruction of the true image noise is an easier task compared to the reconstruction of the clean image and only required consideration of the structure of the residual image. This property renders the method an object-independent image denoiser and helps the denoiser to avoid image over-smoothing, which is very important for any image denoiser.

The unsupervised AE pathway <NUM> enables the utilization of all the samples in the domain of the residual image even in the absence of a corresponding noisy input samples. For example, in the case of a well-defined image noise source, a large amount of different residual image realizations could be generated and used to train that pathway.

Using the present method, it is easier to adapt an existing trained model to a new imaging device sensor (domain transfer). To do so, the AE pathway <NUM> can be retrained while the reg pathway <NUM> needs only to be fine-tuned using a small number of paired training samples obtained using the new sensor.

There are drawbacks of the previous RoCGAN method related to the use (or not) of skip connections. In the absence of skip connections, RoCGAN performs well only in the case of object-depended image denoising (i.e. face denoising). This restricts its use, since the need of having different models for different objects makes it not suitable for digital devices with limited resources, such as smartphones, where the run-time performance is of importance.

Therefore, a device implementing the method can learn how to transform noisy images by only looking at the structure of the residual image. In this way, the task of image denoising is simplified. Furthermore, explicitly learning a low dimensional manifold for a noise source gives the ability not only to better remove that noise from the image but, in the case of many different kind of noise sources, it is possible to combine all of them in the same pipeline. The device can form an estimate of a noise pattern that is representative to a specific image sensor type and can effectively remove noise where the noise pattern is non-Gaussian.

When the original mapping is more like an identity mapping, the residual mapping may be optimized more easily. Note that the ground-truth clean image is much more like the noisy one than the output of the reg pathway (especially when the noise level is low). Thus, the original mapping would be closer to an identity mapping than the output of the reg pathway, and the residual learning formulation is more suitable for image denoising.

<FIG> show an example of denoising results and a comparison with the result from DnCNN, as described in <NPL>. <FIG> shows the noisy RGB input image, <FIG> shows the denoised output image given by the DnCNN method, <FIG> shows the denoised output image given by the method described herein and <FIG> shows the clean ground truth image. It can be seen that in this implementation, the present method preserves the high frequency image details.

Claim 1:
An apparatus (<NUM>) for denoising an image, the apparatus (<NUM>) having a processor (<NUM>) and an imaging device (901a) having an image sensor (<NUM>), the image sensor (<NUM>) capturing an input image, the processor (<NUM>) being configured to:
• receive the input image,
• implement a trained artificial intelligence model to form an estimate of a first noise pattern in the input image and form an output image by subtracting the estimate of the first noise pattern from the input image, the model being configured to:
∘ form estimates of noise patterns in the input image, wherein each noise pattern among the noise patterns in the input image is characteristic to a different type of image sensor among a plurality of types of image sensors (<NUM>), the plurality of types of image sensors (<NUM>) comprising a specific image sensor type, and
∘ form the estimate of the first noise pattern characteristic to the specific image sensor type by means of a projection onto a trained noise manifold; and
the apparatus (<NUM>) is further configured to:
• receive an indication of the specific image sensor type of the image sensor (<NUM>) that captured the input image, and
• provide that indication as an input to the model, and the model is configured to generate the estimate of the first noise pattern in dependence on the indication.