Original: I MPROVED S ELF -S UPERVISED D EEP I MAGE D ENOISING
Revised: I MPROVED S ELF -S UPERVISED D EEP I MAGE D ENOISING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We describe techniques for training high-quality image denoising models that require only single instances of corrupted images as training data.
Revised: We describe techniques for training high-quality image denoising models that require only single instances of corrupted images as training data.

Original: Inspired by a recent technique that removes the need for supervision through image pairs by employing networks with a “blind spot” in the receptive ﬁeld, we address two of its shortcomings: inefﬁcient training and poor ﬁnal denoising performance.
Revised: Inspired by a recent technique that removes the need for supervision through image pairs by employing networks with a “blind spot” in the receptive ﬁeld, we address two of its shortcomings: inefﬁcient training and poor ﬁnal denoising performance.

Original: This is achieved through a novel blind-spot convolutional network architecture that allows efﬁcient self-supervised training, as well as application of Bayesian distribution prediction on output colors.
Revised: This is achieved through a novel blind-spot convolutional network architecture that allows efﬁcient self-supervised training, as well as application of Bayesian distribution prediction on output colors.

Original: Together, they bring the self-supervised model on par with fully supervised deep learning techniques in terms of both quality and training speed in the case of i.i.d.
Revised: Together, they bring the self-supervised model on par with fully supervised deep learning techniques in terms of both quality and training speed in the case of i.i.d.

Original: Gaussian noise.
Revised: Gaussian noise.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Denoising, the removal of noise from images, is a major application of deep learning.
Revised: Denoising, the removal of noise from images, is a major application of deep learning.

Original: Several architectures have been proposed for general-purpose image restoration tasks, e.g., U-Nets (Ronneberger et al., 2015), hierarchical residual networks (Mao et al., 2016), and residual dense networks (Zhang et al., 2018).
Revised: Several architectures have been proposed for general-purpose image restoration tasks, e.g., U-Nets (Ronneberger et al., 2015), hierarchical residual networks (Mao et al., 2016), and residual dense networks (Zhang et al., 2018).

Original: Traditionally, the models are trained in a supervised fashion with corrupted images as inputs and clean images as targets, so that the network learns to remove the corruption.
Revised: Traditionally, the models are trained in a supervised fashion with corrupted images as inputs and clean images as targets, so that the network learns to remove the corruption.

Original: Lehtinen et al.
Revised: Lehtinen et al.

Original: (2018) introduced N OISE 2N OISE training, where pairs of corrupted images are used as training data.
Revised: (2018) introduced N OISE 2N OISE training, where pairs of corrupted images are used as training data.

Original: They observe that when certain statistical conditions are met, a network faced with the impossible task of mapping corrupted images to corrupted images learns, loosely speaking, to output the “average” image.
Revised: They observe that when certain statistical conditions are met, a network faced with the impossible task of mapping corrupted images to corrupted images learns, loosely speaking, to output the “average” image.

Original: For a large class of image corruptions, the clean image is a simple per-pixel statistic — such as mean, median, or mode — over the stochastic corruption process, and hence the restoration model can be supervised using corrupted data by choosing the appropriate loss function to recover the statistic of interest.
Revised: For a large class of image corruptions, the clean image is a simple per-pixel statistic — such as mean, median, or mode — over the stochastic corruption process, and hence the restoration model can be supervised using corrupted data by choosing the appropriate loss function to recover the statistic of interest.

Original: While removing the need for clean training images, N OISE 2N OISE training still requires at least two independent realizations of the corruption for each training image.
Revised: While removing the need for clean training images, N OISE 2N OISE training still requires at least two independent realizations of the corruption for each training image.

Original: While this eases data collection signiﬁcantly compared to noisy-clean pairs, large collections of (single) poor images are still much more widespread.
Revised: While this eases data collection signiﬁcantly compared to noisy-clean pairs, large collections of (single) poor images are still much more widespread.

Original: This motivates investigation of self-supervised training: how much can we learn from just looking at bad data?
Revised: This motivates investigation of self-supervised training: how much can we learn from just looking at bad data?

Original: While foregoing supervision would lead to the expectation of some regression in performance, can we make up for it by making stronger assumptions about the corruption process?
Revised: While foregoing supervision would lead to the expectation of some regression in performance, can we make up for it by making stronger assumptions about the corruption process?

Original: In this paper, we show that under the assumption of additive Gaussian noise that is i.i.d.
Revised: In this paper, we show that under the assumption of additive Gaussian noise that is i.i.d.

Original: between pixels, no concessions in denoising performance are necessary.
Revised: between pixels, no concessions in denoising performance are necessary.

Original: We draw inspiration from the recent N OISE 2V OID (N2V) training technique of Krull et al.
Revised: We draw inspiration from the recent N OISE 2V OID (N2V) training technique of Krull et al.

Original: The algorithm needs no image pairs, and uses just individual noisy images as training data, assuming that the corruption is zero-mean and independent between pixels.
Revised: The algorithm needs no image pairs, and uses just individual noisy images as training data, assuming that the corruption is zero-mean and independent between pixels.

Original: The method is based on blind-spot networks where the receptive ﬁeld of the network does not include the center pixel.
Revised: The method is based on blind-spot networks where the receptive ﬁeld of the network does not include the center pixel.

Original: This allows using the same noisy image as both training input and training target — because the network cannot see the correct answer, using the same image as target is equivalent to using a different noisy realization.
Revised: This allows using the same noisy image as both training input and training target — because the network cannot see the correct answer, using the same image as target is equivalent to using a different noisy realization.

Original: This approach is self-supervised in the sense that the surrounding context is used to predict the value of the output pixel without a separate reference image (Doersch et al., 2015).
Revised: This approach is self-supervised in the sense that the surrounding context is used to predict the value of the output pixel without a separate reference image (Doersch et al., 2015).

Original: The networks used by Krull et al.
Revised: The networks used by Krull et al.

Original: (2018) do not have a blind spot by design, but are trained to ignore the center pixel using a masking scheme where only a few output pixels can contribute to the loss function, reducing training efﬁciency considerably.
Revised: (2018) do not have a blind spot by design, but are trained to ignore the center pixel using a masking scheme where only a few output pixels can contribute to the loss function, reducing training efﬁciency considerably.

Original: We remedy this with a novel architecture that allows efﬁcient training without masking.
Revised: We remedy this with a novel architecture that allows efﬁcient training without masking.

Original: Furthermore, the existence of the blind spot leads to poor denoising quality.
Revised: Furthermore, the existence of the blind spot leads to poor denoising quality.

Original: We derive a scheme for combining the network output with data in the blind spot, bringing the denoising quality on par with conventionally trained networks.
Revised: We derive a scheme for combining the network output with data in the blind spot, bringing the denoising quality on par with conventionally trained networks.

Original: [Figure]
Revised: [Figure]

Original:  C ONVOLUTIONAL BLIND - SPOT NETWORK ARCHITECTURES
Revised:  C ONVOLUTIONAL BLIND - SPOT NETWORK ARCHITECTURES

Original: Our convolutional blind-spot networks are designed by combining multiple branches that each have their receptive ﬁeld restricted to a half-plane (Figure 1) that does not contain the center pixel.
Revised: Our convolutional blind-spot networks are designed by combining multiple branches that each have their receptive ﬁeld restricted to a half-plane (Figure 1) that does not contain the center pixel.

Original: The principle of limiting the receptive ﬁeld has been used in PixelCNN (van den Oord et al., 2016a) image synthesis networks, where only pixels synthesized before the current pixel are allowed in the receptive ﬁeld.
Revised: The principle of limiting the receptive ﬁeld has been used in PixelCNN (van den Oord et al., 2016a) image synthesis networks, where only pixels synthesized before the current pixel are allowed in the receptive ﬁeld.

Original: We combine the four branches with a series of 1 × 1 convolutions to obtain a receptive ﬁeld that can extend arbitrarily far in every direction but does not contain the center pixel.
Revised: We combine the four branches with a series of 1 × 1 convolutions to obtain a receptive ﬁeld that can extend arbitrarily far in every direction but does not contain the center pixel.

Original: In order to transform a restoration network into one with a restricted receptive ﬁeld, we modify each individual layer so that its receptive ﬁeld is fully contained within one half-plane, including the center row/column.
Revised: In order to transform a restoration network into one with a restricted receptive ﬁeld, we modify each individual layer so that its receptive ﬁeld is fully contained within one half-plane, including the center row/column.

Original: The receptive ﬁeld of the resulting network includes the center pixel, so we offset the feature maps by one pixel before combining them.
Revised: The receptive ﬁeld of the resulting network includes the center pixel, so we offset the feature maps by one pixel before combining them.

Original: Layers that do not extend the receptive ﬁeld, e.g., concatenation, summation, 1 × 1 convolution, etc., can be used without modiﬁcations.
Revised: Layers that do not extend the receptive ﬁeld, e.g., concatenation, summation, 1 × 1 convolution, etc., can be used without modiﬁcations.

Original: Convolution layers.
Revised: Convolution layers.

Original: To restrict the receptive ﬁeld of a zero-padding convolution layer to extend only, say, upwards, the easiest solution is to offset the feature maps downwards when performing the convolution operation.
Revised: To restrict the receptive ﬁeld of a zero-padding convolution layer to extend only, say, upwards, the easiest solution is to offset the feature maps downwards when performing the convolution operation.

Original: For an h × w kernel size, a downwards offset of k = (cid:98) h/ 2 (cid:99) pixels is equivalent to using a kernel that is shifted upwards so that all weights below the center line are zero.
Revised: For an h × w kernel size, a downwards offset of k = (cid:98) h/ 2 (cid:99) pixels is equivalent to using a kernel that is shifted upwards so that all weights below the center line are zero.

Original: Speciﬁcally, we ﬁrst append k rows of zeros to the top of input tensor, then perform the convolution, and ﬁnally crop out the k bottom rows of the output.
Revised: Speciﬁcally, we ﬁrst append k rows of zeros to the top of input tensor, then perform the convolution, and ﬁnally crop out the k bottom rows of the output.

Original: Downsampling and upsampling layers.
Revised: Downsampling and upsampling layers.

Original: Many image restoration networks involve downsampling and upsampling layers, and by default, these extend the receptive ﬁeld in all directions.
Revised: Many image restoration networks involve downsampling and upsampling layers, and by default, these extend the receptive ﬁeld in all directions.

Original: Consider, e.g., a bilinear 2 × 2 downsampling step followed immediately by a nearest-neighbor 2 × 2 upsampling step.
Revised: Consider, e.g., a bilinear 2 × 2 downsampling step followed immediately by a nearest-neighbor 2 × 2 upsampling step.

Original: The contents of every 2 × 2 pixel block in the output now correspond to the average of this block in the input, i.e., information has been transferred in every direction within the block.
Revised: The contents of every 2 × 2 pixel block in the output now correspond to the average of this block in the input, i.e., information has been transferred in every direction within the block.

Original: We ﬁx this problem by again applying an offset to the data.
Revised: We ﬁx this problem by again applying an offset to the data.

Original: It is sufﬁcient to restrict the receptive ﬁeld for the pair of downsampling and upsampling layers, which means that only one of the layers needs to be modiﬁed, and we have chosen to attach the offsets to the downsampling layers.
Revised: It is sufﬁcient to restrict the receptive ﬁeld for the pair of downsampling and upsampling layers, which means that only one of the layers needs to be modiﬁed, and we have chosen to attach the offsets to the downsampling layers.

Original: For a 2 ×bilinear downsampling layer, we can restrict the receptive ﬁeld to extend upwards only by padding the input tensor with one row of zeros at top and cropping out the bottom row before performing the actual downsampling operation.
Revised: For a 2 ×bilinear downsampling layer, we can restrict the receptive ﬁeld to extend upwards only by padding the input tensor with one row of zeros at top and cropping out the bottom row before performing the actual downsampling operation.

Original:  B AYESIAN TRAINING AND MAP DENOISING
Revised:  B AYESIAN TRAINING AND MAP DENOISING

Original: In their basic form, blind-spot networks suffer from the inability to utilize the data at the center pixel at test time; yet, clearly, the observed value carries information about the underlying clean signal.
Revised: In their basic form, blind-spot networks suffer from the inability to utilize the data at the center pixel at test time; yet, clearly, the observed value carries information about the underlying clean signal.

Original: For training it is mandatory to disconnect the information ﬂow from pixel position to itself, but there is no such restriction when using the network to restore novel images after it has been trained.
Revised: For training it is mandatory to disconnect the information ﬂow from pixel position to itself, but there is no such restriction when using the network to restore novel images after it has been trained.

Original: We capitalize on this by training the network to predict, based on the context, a distribution of values instead of a single mean prediction, and applying maximum a posteriori estimation at test time.
Revised: We capitalize on this by training the network to predict, based on the context, a distribution of values instead of a single mean prediction, and applying maximum a posteriori estimation at test time.

Original: In Bayesian training (Nix & Weigend, 1994; Le et al., 2005; Kendall & Gal, 2017), the network predicts output distributions using a negative log-likelihood loss function.
Revised: In Bayesian training (Nix & Weigend, 1994; Le et al., 2005; Kendall & Gal, 2017), the network predicts output distributions using a negative log-likelihood loss function.

Original: We model the data using multivariate Gaussian distributions: For images with c color components, we have the denoising network output a vector of means µ y and a covariance matrix Σ y for each pixel.
Revised: We model the data using multivariate Gaussian distributions: For images with c color components, we have the denoising network output a vector of means µ y and a covariance matrix Σ y for each pixel.

Original: For convenience, we parameterize the c × c inverse per-pixel covariance matrix as Σ − 1 y =
Revised: For convenience, we parameterize the c × c inverse per-pixel covariance matrix as Σ − 1 y =

Original: A y T
Revised: A y T

Original: A y , where A y is an upper triangular matrix.
Revised: A y , where A y is an upper triangular matrix.

Original: This choice ensures that Σ y is positive semideﬁnite with non-negative diagonal entries, as required for a covariance matrix.
Revised: This choice ensures that Σ y is positive semideﬁnite with non-negative diagonal entries, as required for a covariance matrix.

Original: For RGB images, the network thus outputs a total of nine values per pixel: the three-component mean µ y and the six nonzero elements of A y .
Revised: For RGB images, the network thus outputs a total of nine values per pixel: the three-component mean µ y and the six nonzero elements of A y .

Original: Let f ( y ; µ y , Σ y ) denote the probability density of a multivariate Gaussian distribution N ( µ y , Σ y ) at target pixel color y , i.e., exp[ − 12 ( y − µ y ) T Σ − 1 y
Revised: Let f ( y ; µ y , Σ y ) denote the probability density of a multivariate Gaussian distribution N ( µ y , Σ y ) at target pixel color y , i.e., exp[ − 12 ( y − µ y ) T Σ − 1 y

Original: ( y − µ y )] / (cid:112) (2 π ) c | Σ y | .
Revised: ( y − µ y )] / (cid:112) (2 π ) c | Σ y | .

Original: Under our parameterization, the corresponding negative log-likelihood loss to optimize during training is
Revised: Under our parameterization, the corresponding negative log-likelihood loss to optimize during training is

Original: [Equation]
Revised: [Equation]

Original: where C is a constant term that can be discarded.
Revised: where C is a constant term that can be discarded.

Original: Because A y is a triangular matrix, its determinant | A y | is the product of its diagonal elements.
Revised: Because A y is a triangular matrix, its determinant | A y | is the product of its diagonal elements.

Original: To avoid numerical issues, we clamp this determinant to a small positive epsilon ( (cid:15) =
Revised: To avoid numerical issues, we clamp this determinant to a small positive epsilon ( (cid:15) =

Original: 10 − 8 ) so that the logarithm is always well-deﬁned.
Revised: 10 − 8 ) so that the logarithm is always well-deﬁned.

Original: We assume that all of our images are corrupted by additive uniform Gaussian noise N (0 , σ 2 I ) with a known standard deviation σ .
Revised: We assume that all of our images are corrupted by additive uniform Gaussian noise N (0 , σ 2 I ) with a known standard deviation σ .

Original: Using noisy targets means that there is a “baseline” level of noise in the network output distributions that we must discount.
Revised: Using noisy targets means that there is a “baseline” level of noise in the network output distributions that we must discount.

Original: Thanks to the blind spot, the network output is independent of the noise in the center pixel, so their (co-)variances are additive.
Revised: Thanks to the blind spot, the network output is independent of the noise in the center pixel, so their (co-)variances are additive.

Original: We can therefore calculate Σ p =
Revised: We can therefore calculate Σ p =

Original: Σ y − σ 2 I to determine the actual uncertainty Σ p of the network.
Revised: Σ y − σ 2 I to determine the actual uncertainty Σ p of the network.

Original: To avoid negative variances due to approximation errors, the diagonal elements of Σ p can be clamped to zero.
Revised: To avoid negative variances due to approximation errors, the diagonal elements of Σ p can be clamped to zero.

Original: Let us now derive our maximum a posteriori (MAP) denoising procedure.
Revised: Let us now derive our maximum a posteriori (MAP) denoising procedure.

Original: For each pixel, our goal is to ﬁnd the most likely clean value ˆ x given our knowledge of the noisy value ˜ x and the output distribution predicted by the network based on the blind-spot neighborhood Ω .
Revised: For each pixel, our goal is to ﬁnd the most likely clean value ˆ x given our knowledge of the noisy value ˜ x and the output distribution predicted by the network based on the blind-spot neighborhood Ω .

Original: It follows that
Revised: It follows that

Original: [Equation]
Revised: [Equation]

Original: where we have ﬁrst applied Bayes’ theorem to obtain the MAP objective P ( ˜ x | x ) P ( x | Ω) , and then expressed the associated probabilities as pdfs of Gaussian distributions.
Revised: where we have ﬁrst applied Bayes’ theorem to obtain the MAP objective P ( ˜ x | x ) P ( x | Ω) , and then expressed the associated probabilities as pdfs of Gaussian distributions.

Original: In the ﬁrst term we have exploited the symmetry of the Gaussian distribution, and as the prior term P ( x | Ω) we use the prediction of the network with the baseline uncertainty removed.
Revised: In the ﬁrst term we have exploited the symmetry of the Gaussian distribution, and as the prior term P ( x | Ω) we use the prediction of the network with the baseline uncertainty removed.

Original: Following Bromiley (2003), the mean, and consequently the argmax , of this product of two Gaussian distributions is
Revised: Following Bromiley (2003), the mean, and consequently the argmax , of this product of two Gaussian distributions is

Original: [Equation]
Revised: [Equation]

Original:  R ESULTS AND C ONCLUSIONS
Revised:  R ESULTS AND C ONCLUSIONS

Original: For the baseline experiments, as well as for the backbone of our blind-spot networks, we use the same U-Net (Ronneberger et al., 2015) architecture as Lehtinen et al.
Revised: For the baseline experiments, as well as for the backbone of our blind-spot networks, we use the same U-Net (Ronneberger et al., 2015) architecture as Lehtinen et al.

Original: (2018), see their appendix for details.
Revised: (2018), see their appendix for details.

Original: The only differences are that we have layers DEC CONV 1 A and DEC CONV 1 B output 96 feature maps like the other convolution layers at the decoder stage, and layer DEC CONV 1 C is removed.
Revised: The only differences are that we have layers DEC CONV 1 A and DEC CONV 1 B output 96 feature maps like the other convolution layers at the decoder stage, and layer DEC CONV 1 C is removed.

Original: After combining the four receptive ﬁeld restricted branches, we thus have 384 feature maps.
Revised: After combining the four receptive ﬁeld restricted branches, we thus have 384 feature maps.

Original: These are fed into three successive 1 × 1 convolutions with 384, 96, and n output channels, respectively, where n is the number of output components for the network.
Revised: These are fed into three successive 1 × 1 convolutions with 384, 96, and n output channels, respectively, where n is the number of output components for the network.

Original: All convolution layers except the last 1 × 1 convolution use leaky ReLU with α = 0 .
Revised: All convolution layers except the last 1 × 1 convolution use leaky ReLU with α = 0 .

Original: 1 (Maas et al., 2013).
Revised: 1 (Maas et al., 2013).

Original: All networks were trained using Adam with default parameters (Kingma & Ba, 2015), learning rate λ = 0 .
Revised: All networks were trained using Adam with default parameters (Kingma & Ba, 2015), learning rate λ = 0 .

Original: 0003 , and minibatch size of 4.
Revised: 0003 , and minibatch size of 4.

Original: As training data, we used random 256 × 256 crops from the 50K images in the ILSVRC 2012 (Imagenet) validation set.
Revised: As training data, we used random 256 × 256 crops from the 50K images in the ILSVRC 2012 (Imagenet) validation set.

Original: The training continued until 1.2M images were shown to the network.
Revised: The training continued until 1.2M images were shown to the network.

Original: All training and test images were corrupted with Gaussian noise, σ = 25 .
Revised: All training and test images were corrupted with Gaussian noise, σ = 25 .

Original: Table 1 shows the denoising quality in dB for the four test datasets used.
Revised: Table 1 shows the denoising quality in dB for the four test datasets used.

Original: From the BSD300 dataset we use the 100 validation images only.
Revised: From the BSD300 dataset we use the 100 validation images only.

Original: Similar to Krull et al.
Revised: Similar to Krull et al.

Original: (2018), we use the grayscale version of the BSD68 dataset — for this case we train a single-channel ( c = 1 ) denoiser using only the luminance channel of the training images.
Revised: (2018), we use the grayscale version of the BSD68 dataset — for this case we train a single-channel ( c = 1 ) denoiser using only the luminance channel of the training images.

Original: All our blind-spot noise-to-noise networks use the convolutional architecture (Section 2) and are trained without masking.
Revised: All our blind-spot noise-to-noise networks use the convolutional architecture (Section 2) and are trained without masking.

Original: In BSD68 our simpliﬁed L variant closely matches the original N OISE 2V OID training, suggesting that our network with an architecturally enforced blind spot is approximately as capable as the masking-based network trained by Krull et al.
Revised: In BSD68 our simpliﬁed L variant closely matches the original N OISE 2V OID training, suggesting that our network with an architecturally enforced blind spot is approximately as capable as the masking-based network trained by Krull et al.

Original: We see that the denoising quality of our Full setup (Section 3) is on par with baseline results of N2N and N2C, and clearly surpasses standard blind-spot denoising ( L2 ) that does not exploit the information in the blind spot.
Revised: We see that the denoising quality of our Full setup (Section 3) is on par with baseline results of N2N and N2C, and clearly surpasses standard blind-spot denoising ( L2 ) that does not exploit the information in the blind spot.

Original: Doing the estimation separately for each color
Revised: Doing the estimation separately for each color

Original: [Table]
Revised: [Table]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: channel ( Per-comp. ) performs signiﬁcantly worse, except in the grayscale BSD68 dataset where it is equivalent to the Full method.
Revised: ( Per-comp. ) performs signiﬁcantly worse, except in the grayscale BSD68 dataset where it is equivalent to the Full method.

Original: Figure 2 shows example denoising results.
Revised: Figure 2 shows example denoising results.

Original: Our Full setup produces images that are virtually identical to the N2N baseline both visually and in terms of PSNR.
Revised: Our Full setup produces images that are virtually identical to the N2N baseline both visually and in terms of PSNR.

Original: The ablated Per-comp.
Revised: The ablated Per-comp.

Original: setup tends to produce color artifacts, demonstrating the shortcomings of the simpler per-component univariate model.
Revised: setup tends to produce color artifacts, demonstrating the shortcomings of the simpler per-component univariate model.

Original: Finally, the L2 variant that ignores the center pixel during denoising produces visible checkerboard patterns, some of which can also be seen in the result images of Krull et al.
Revised: Finally, the L2 variant that ignores the center pixel during denoising produces visible checkerboard patterns, some of which can also be seen in the result images of Krull et al.

Original: Conclusions.
Revised: Conclusions.

Original: We have shown that self-supervised training — looking at noisy images only, without the beneﬁt of seeing the same image under different noise realizations — is sufﬁcient for learning deep denoising models on par with those that make use of another realization as a training target, be it clean or corrupted.
Revised: We have shown that self-supervised training — looking at noisy images only, without the beneﬁt of seeing the same image under different noise realizations — is sufﬁcient for learning deep denoising models on par with those that make use of another realization as a training target, be it clean or corrupted.

Original: Currently this comes at the cost of assuming pixel-wise independent noise with a known analytic likelihood model.
Revised: Currently this comes at the cost of assuming pixel-wise independent noise with a known analytic likelihood model.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: PixelCNNs (van den Oord et al., 2016b;a; Salimans et al., 2017) generate novel images in a scanline order, one pixel at a time, by conditioning the possible pixel colors using all previous, already generated pixels.
Revised: PixelCNNs (van den Oord et al., 2016b;a; Salimans et al., 2017) generate novel images in a scanline order, one pixel at a time, by conditioning the possible pixel colors using all previous, already generated pixels.

Original: The training uses masked convolutions that prevent looking at pixels that would not have been generated yet — one good implementation of masking (van den Oord et al., 2016a) combines a vertical half-space (previous scanlines) with a horizontal line (current scanline).
Revised: The training uses masked convolutions that prevent looking at pixels that would not have been generated yet — one good implementation of masking (van den Oord et al., 2016a) combines a vertical half-space (previous scanlines) with a horizontal line (current scanline).

Original: In our application we use four half-spaces to exclude the center pixel only.
Revised: In our application we use four half-spaces to exclude the center pixel only.

Original: Regrettably the term “blind spot” has a slightly different meaning in PixelCNNs:
Revised: Regrettably the term “blind spot” has a slightly different meaning in PixelCNNs:

Original: van den Oord et al.
Revised: van den Oord et al.

Original: (2016a) uses it to denote valid input pixels that the network in question fails to see due to poor design, whereas we follow the naming convention of Krull et al.
Revised: (2016a) uses it to denote valid input pixels that the network in question fails to see due to poor design, whereas we follow the naming convention of Krull et al.

Original: (2018) so that a blind spot is always intentional.
Revised: (2018) so that a blind spot is always intentional.

Original: Applying Bayesian statistics to denoising has a long history.
Revised: Applying Bayesian statistics to denoising has a long history.

Original: Non-local means (Buades et al., 2005),
Revised: Non-local means (Buades et al., 2005),

Original: BM3D (Dabov et al., 2007), and WNNM (Gu et al., 2014) identify a group of similar pixel neighborhoods and estimate the center pixel’s color from those.
Revised: BM3D (Dabov et al., 2007), and WNNM (Gu et al., 2014) identify a group of similar pixel neighborhoods and estimate the center pixel’s color from those.

Original: This is conceptually similar to our solution, which uses a convolutional network to represent the mapping from neighborhoods to the distilled outputs.
Revised: This is conceptually similar to our solution, which uses a convolutional network to represent the mapping from neighborhoods to the distilled outputs.

Original: Both approaches need only the noisy images, but while the explicit block-based methods determine a small number of neighborhoods from the input image alone, our blind-spot training can implicitly identify and regress an arbitrarily large number of neighborhoods from a collection of noisy training data.
Revised: Both approaches need only the noisy images, but while the explicit block-based methods determine a small number of neighborhoods from the input image alone, our blind-spot training can implicitly identify and regress an arbitrarily large number of neighborhoods from a collection of noisy training data.