Planckian Jitter: countering the color-crippling effects of color jitter on self-supervised training

Abstract: Several recent works on self-supervised learning are trained by mapping different augmentations of the same image to the same feature representation. The data augmentations used are of crucial importance to the quality of learned feature representations. In this paper, we analyze how the color jitter traditionally used in data augmentation negatively impacts the quality of the color features in learned feature representations. To address this problem, we propose a more realistic, physics-based color data augmentation – which we call Planckian Jitter – that creates realistic variations in chromaticity and produces a model robust to illumination changes that can be commonly observed in real life, while maintaining the ability to discriminate image content based on color information. Experiments confirm that such a representation is complementary to the representations learned with the currently-used color jitter augmentation and that a simple concatenation leads to significant performance gains on a wide range of downstream datasets. In addition, we present a color sensitivity analysis that documents the impact of different training methods on model neurons and shows that the performance of the learned features is robust with respect to illuminant variations.

Reviews and author responses:

 Thank you again for this discussion and for appreciating our experimental evaluation. We address your concerns below:

>**Q3.1** I think the author may have misunderstood my question. My concern is that the experiment is misleading--it seems to suggest that the illuminant change proposed here is accurate and causes the model trained with it to be stable across illuminant variation. However, if the train and test are being perturbed by the same illumination, it does not matter if the model is correct. The training will allow the resulting model to be stable, regardless of whether the illumination change model is correct. This is why I suggest the author to evaluate using real dataset. While I agree that there is no large dataset that will allow the author to perform this exact experiment, varying temperature of the black body, there are several multi-illumination dataset out there. Murmann et al's "A Multi-Illumination Dataset of Indoor Object Appearance" is an example (1000 scenes, 25 "illuminations" each).

 We agree that confirming results on a real dataset with illuminant changes and object classes would be better. However, we have not found such a dataset. The proposed dataset has no illuminant variation but only the illuminant position is changing. We will also adapt the text to better reflect that the tested augmentations resemble those of the training and will change line 229 ‘Planckian Jitter obtains a remarkably stable performance from around 4000-14000K,’ to ‘The results confirm that Planckian Jitter encourages learning invariance to the augmentations (from 4000-14000K) it has been exposed to during training.

We would like to stress that Figure 3 does show that the CJ representation is not robust to illumination changes and shows a significant drop in performance on the sites of the graph.


>**Q3.3**  In my opinion 32x32 images would barely have enough structure for the model to learn "shape" representation, and it is too contrived to draw meaningful conclusion for the reader. I understand that computing resource can be limited, but that should lead to different prioritization of the experiment. For example, the authors could choose to focus their augmentation model to just CJ and PJ, rather than adding H&S, B&C, and so on into the mix.

The results on 32x32 pixel images in our opinion already demonstrate that CJ learns a suboptimal color representation, since PJ obtains significantly better performance on the Flower dataset. We agree that rescaling hurts shape/texture representations more than color representations. This is also shown in our additional experiments done for R2 (see Q2.1 where we show that CJ performance drops much more than PJ performance when lowering the resolution)

As far as larger images are concerned, we are not sure what result the reviewer would like us to add (for the 224x224 images). In Table 3 we first show CJ, PJ alone and then their combination trained on ImageNet (with 224x224 pixel images) and then evaluated on downstream tasks with 224x224 pixel images. These results confirm that also for higher resolution (224x224) the combination of [CJ,PJ] obtains the best results on all four datasets.

It is true that the performance gain is smaller than for 32x32, but especially for CUB-200, T1K and VegFru the gain with respect to [CJ,CJ] is still considerable (in all cases over 4%). Note that also in Table 5 of the Supplementary Material we show results on TinyImageNet (with 64x64 pixel images) and our results again show that the combination of [CJ,PJ] obtains superior results.
 

 I have looked at the other review and carefully read the rebuttal.

Q3.1: I think the author may have misunderstood my question. My concern is that the experiment is misleading--it seems to suggest that the illuminant change proposed here is accurate and causes the model trained with it to be stable across illuminant variation. However, if the train and test are being perturbed by the same illumination, it does not matter if the model is correct. The training will allow the resulting model to be stable, regardless of whether the illumination change model is correct. This is why I suggest the author to evaluate using real dataset. While I agree that there is no large dataset that will allow the author to perform this exact experiment, varying temperature of the black body, there are several multi-illumination dataset out there. Murmann et al's "A Multi-Illumination Dataset of Indoor Object Appearance" is an example (1000 scenes, 25 "illuminations" each).

Q3.2: I think the author response is fair, and it is true that the most interesting part about this work is not necessarily the performance of PJ. The combination of CJ and PJ tend to lead to higher performance than either alone (Table 3). However, it is unclear if this conclusion would hold in the real world (see comment below).

Q3.3: In my opinion 32x32 images would barely have enough structure for the model to learn "shape" representation, and it is too contrived to draw meaningful conclusion for the reader. I understand that computing resource can be limited, but that should lead to different prioritization of the experiment. For example, the authors could choose to focus their augmentation model to just CJ and PJ, rather than adding H&S, B&C, and so on into the mix. 

Q3.4: The author is correct. In the simple scaling model, gamma correction only amounts to different scaling parameter. However, I will say that due to the compression at low range, and quantization to uint8, much of the dynamic range is loss and this affect the ability to do proper white balance. The datasets used in this experiment does not seem to contain a lot of night scenes, so this problem is alleviated. However, this is a crucial fact that should be added to the paper.

I will also mention that I had missed in the first review that the author has done a very comprehensive set of experiments, as other reviewers have noted. With this insight, I have upgraded the rating for this paper to 4.

 >**Q3.4** The color jittering model missed one crucial fact about the sRGB space, which is that it is gamma-encoded. Gamma compression is lossy, and resulted in far inferior color correction result compared to images captured in RAW. Nonetheless, I do not see the mentioning of gamma correction prior to color correction (and then reapplication to get back into the original color space)

One way to account for gamma correction is, as the reviewer suggests, to linearize the sRGB image, apply a RAW-like illuminant, and then delinearize the resulting image (restoring gamma correction). An alternative approach, which is used in our method, is to apply gamma correction to the RAW-like illuminant and then apply the gamma-corrected illuminant directly to the sRGB image.

The two approaches are mathematically equivalent, as shown in the following:

Let $[a,b,c]$ indicate a RAW-like illuminant, $[R,G,B]$ an sRGB image pixel, $\gamma$ a delinearization value (gamma), and * a pointwise multiplication.

Then:

$([a,b,c]*[R,G,B]^\gamma)^{(1/\gamma)}= ([a,b,c]^{(1/\gamma)} * [R,G,B])$

The confusion arises from the fact that we did not explicitly state in our manuscript that the RAW-like illuminant is gamma-corrected $([a,b,c]^{1/\gamma})$. This was in fact "hidden" in the lab-to-sRGB conversion described at the end of page 4, line 149. We apologize for the misunderstanding and will make this explicit in the revised manuscript.


 Thank you for your review. We address your concerns (weakness) below (in two separate comments).

>**Q3.1** Fig 3 shows a more stable result, but it seems that the testing dataset also undergoes the simulated jittering rather than a real captures of different illumination. Such artificial manipulation of the test set is going to yield the same result regardless of how poorly the jittering performs.

The experiment behind Figure 3 is driven by two motivations. One, as mentioned by the reviewer, is to verify that we obtain invariance to the transformation applied during training. This is indeed verified. We agree that this experiment would offer additional insights if performed on real data, however this would require a dataset labeled for both illuminant estimation and image classification. Unfortunately, we were not able to find such a dataset. The second motivation is to characterize the degradation of different non-Planckian training modalities. How do these non-Planckian curves vary under synthetic illuminant changes (this is often used as a  predictor for variations under real illuminant). This, in fact, shows different test curves, as can be seen in the figure 3. We will make this double motivation more clear in the revised manuscript

>**Q3.2**  Planckian jitter do not always perform the best, and can underperform by a significant margin (see Table 1).

Figure 1 (left) in the paper shows how the standard color jitter transforms an example image. In this paper, we argue that these unrealistic variations (easily verified by the human eye) impede the learning of good color representations. Importantly, this does allow it to learn a good shape/texture representation (since it cannot rely on color to solve the matching problem, see also Figure 5  in [4]). Our proposed color augmentation, in Figure 1 (right), yields more realistic color variations which resemble those caused by changing illuminants in the real world. However, since the color augmentations are less drastic, the shape/texture representation is not as strong as the one learned with standard color jitter. Combining both representations is therefore required to get excellent color/shape/texture features, as is confirmed in our experiments on several datasets.

Our aim is not for Planckian Jitter to be better than Color Jitter (a method used by hundreds of papers). Instead, we argue that the color representation of standard Color Jitter is inferior and that when combined with Planckian Jitter (with superior color representation) results should improve. We show that in general this is the case, and more notably on datasets where color plays an important role -- the combined results [CJ,PJ] consistently outperform [CJ,CJ] and often by a large margin.

More in detail about the results in Table 1. We argue that the low performance of Planckian data augmentation on the CIFAR-100 dataset is due to the intrinsic nature of the task – i.e. one in which chromatic variations are irrelevant. For example, the "car" class in CIFAR-100 has a wide variability of car body colors, and thus color information is not useful for correctly identifying the class. Conversely, each flower/bird/vegetable has one characteristic “reflectance” color, and any observed color variation is primarily due to the illuminant. On CIFAR100 shape/texture is of more importance than color, and consequently the results of CJ are much better than those of PJ. However, when combined [CJ, PJ] obtains good results on CIFAR100 and excellent results on Flowers-102.

>**Q3.3** The motivation for resizing images to 32x32 is unclear. In most practical application, such small patch sizes are unlikely to contain enough information to be useful. More investigation is needed on more realistic patch size of 200+pixel. In the present study, only Table 3 shows this result, and the performance of PJ is mixed on different dataset

We structured our experiments so that the explorative part of our research can be conducted in an agile manner by exploiting low-resolution images (computationally, even on 32x32 images the training of self-supervised representations is very demanding). The most significant configurations are then transferred to a higher-resolution setup, with the goal of either confirming or refuting the initial hypotheses. This will be made more explicit in the revised version.

32x32 pixel images are admittedly small for some applications where shape and texture are particularly important, but they contain significant information for color-sensitive applications which are the focus of this manuscript. The results of PJ alone in Table 3 are indeed mixed, but the joint combination LSC [CJ,PJ] yields consistent improvement over any individual form of jittering across all analyzed datasets. This, in our opinion, demonstrates the value of Planckian Jitter in practice




 > **Q2.2** Although the LSC combination is a good start to fuse both types of color augmentation, what about a way to make it more class-specific? In other words, for even a coarse dichotomy such as natural vs man-made images, apply some complementary weighting/distribution of Planckian/default color jitter to get the best of both worlds?

We would like to thank the reviewer for suggesting this interesting approach for a more refined combination of different types of data augmentation. A data-driven optimization of the ideal weight for each component as a function of a macro-level classification will definitely be considered as future developments. We did try training the network simultaneously with both Planckian jitter and Color jitter (switching between these at each minibatch). In this case, we set only part of the latent space to be invariant with respect to Planckian Jitter and another partially overlapping part to be invariant with respect to Color jitter. The advantage of this method was that a single network could be used at inference time. However, results were not outperforming our current naive concatenation method. We are planning to investigate this in more detail in future work. 


 Thank you for your review. Also thank you for endorsing the empirical results and making good points for discussion.
We address your concerns below (in two separate comments).

> **Q2.1** Although the LSC combined augmentation results show that Planckian and random jitter are somewhat complementary, the higher-resolution, 224x244, results in Table 3 do cause the reader to question whether the low resolution case is somehow special with respect to color augmentation. Here Planckian jitter can actually be worse for the target data set of Flowers-102 which isn't fully explained why this happens. Although the idea seems ok for CUB-200 and better for T1K+, again for the natural image data set of VegFru, performance is worse or similar than default random color jitter. The main explanation is a one sentence claim that for higher resolution shape is very representative and color may add little discrimination, but more investigation here is necessary. For example, perhaps higher resolution is actually causing texture to be more discriminative, rather than shape? Intuitively, shape seems reasonably preserved even at lower resolution, unless this is correlated with the actual self-supervised architecture of hyperparameters.
> As mentioned above, the main question for the reader is the somewhat counter-intuitive results in Table 3, for the same data sets but with higher resolution at 224x224. One would expect a similar trend as the low-resolution data, but the numbers show otherwise. Also, the explanation on better shape discrimination on higher resolution bears more analysis and detail, e.g., is it more about texture than shape at higher resolution?

The reviewer is correct in pointing out this phenomenon. We agree with their analysis that this is probably due to more access to texture information. Our paper focuses on color representations, and the non color-part of the representation consists of shape and texture. Sometimes we differentiate these three factors (‘color’, ‘shape’, ‘texture’, e.g., lines 27, 165) but often we use the term ‘shape’ to refer to the non-color part, being both shape and texture (e.g., lines 43, 53, 120, 292, etc). We agree that this leads to some inaccuracies, especially in the lines cited by the reviewer, where shape is indeed less high-frequency and texture would be mostly high-frequency. We will carefully revise the text and will explicitly include reference to ‘texture’ where this term is more accurate.

As an additional experiment, we took the representations and classifiers learned at high-resolution (224x224) and investigated their sensitivity to high-frequency information in images. At inference time, we down-sample (down-sample resolution is given in Table) and then up-sample all images. In this way, we can compare the dependence on high-resolution information of different methods. Note, that here we do not retrain the classifier but use the one trained at 224x224. The results clearly show that CJ suffers more from down-sampling than PJ. For the Flowers datasets, the results of CJ at resolution 224 are better than PJ. However, when we down-sample to 64, the results change and results for PJ are already significantly better than CJ. As suggested by the reviewer, we think the texture information (important for CJ) is removed and this hurts performance. For PJ, which is more dependent on color information, down-sampling hurts results less (note PJ is also using texture, shape but to a lesser degree, so results still deteriorate for smaller resolutions). 

**Table 1:**  _Classification accuracy as a function of down-sampling size on Flowers. Results confirm that PJ is less sensitive to down-sampling than CJ._

| Method   |    32 |    64   |   128 |   224 |
|----------|-------|-------|-------|-------|
| CJ       |  7.74 | 49.23 | 91.43 | 91.90 |
| CJ+PJ    | 16.30 | 66.43 | 93.01 | 93.45 |
| PJ       | 23.11 | 70.13 | 89.04 | 89.22 |

To verify that CJ uses less color information than PJ, we did a simple experiment where at inference time we changed the input images from sRGB to gray-scale images. These results are provided in Table 2. These results clearly show that PJ is much more dependent on color than CJ. PJ has a drop of over 67.6% whereas CJ only drops 3.2 percentage points. 

We will incorporate this analysis in any final version of this paper. 

**Table 2:** _Methods evaluated with color and gray images on Flowers dataset._

| Method   | Color   |   Accuracy |
|----------|---------|------------|
| CJ       | COLOR   |      92.73 |
| CJ       | GS      |      89.51 |
| PJ       | COLOR   |      88.97 |
| PJ       | GS      |      21.38 |

 We would like to thank the reviewer for the appreciation of our manuscript and for the interesting points of discussion.

>Questions:  
>Are there other color distributions, or ways of obtaining color distributions, that might be better than the Planckian illuminants? There are illumination situations, such as light filtering through trees, that are non-Planckian but still non-zero in terms of likelihoods.

Considering a wider distribution of illuminants could be a good alternative, possibly referring to real-world data or to synthetic datasets, as rendering engines are getting ever more realistic. As a general observation, we add that accounting for non-uniform illuminant distributions might yield diminishing returns. If a given illuminant is known to be less likely to occur naturally, we might want to sample it less frequently. But then its sparse appearance during data augmentation might turn out to have a small impact on the learned features. Nonetheless, we consider this an interesting direction to evaluate for future research.

>Limitations:  
>There is a difference between color augmentation of materials and color augmentation of illumination. Illumination color is one color augmentation that can be done easily through a color rotation of the pixels. While on a jpg-compressed/color-enhanced image the effect is not perfect, it's close enough that the augmentation can allow the network to learn color-based rules that follow a more typical distribution of colors. Regardless of whether color is a useful feature for an object, this type of augmentation matches what is likely to occur in the real world.  
>Color augmentation for materials, on the other hand, is challenging to do properly as an augmentation. First, it is class-based, as every class will have a different material color distribution. Second, it is not appropriate to change the colors of the entire image in order to modify the material color. The uniformly random color augmentation seems like it is intended to try and cover both types of color changes (illumination and material).  
>It would be nice to have some discussion of this in the paper, though not necessary as it is not directly relevant to the author's contribution but an an additional limitation of the standard color augmentation methods. But it would give a little more context to the issue in the introduction.

Thank you for suggesting this additional limitation of default color jittering. We agree that the default color jitter can be interpreted as applying material and illuminant color changes. We also agree that uniformly applying the same color augmentation to entire patches would not yield realistic material color changes, but would require knowledge of object boundaries. In contrast, Planckian Jitter is more realistic in practice because it is based on a physical model of illumination, and it is in fact a transformation uniformly applied over all pixels in the image. We will incorporate this discussion in any final version of the paper.

 The paper examines the color data augmentation used to train many deep networks for object classification tasks. It proposes that the existing color augmentation method does not permit the network to make effective use of color information when appropriate. They propose using a color augmentation that follows Planckian illuminants, which are physically-realistic. The paper shows that using the combination of a network trained using the standard color jitter and a network using the Plankcian jitter perform significantly better on classification tasks that can take advantage of color information. The ablation study and analysis of the results support the hypothesis that a physics-based color jitter enables the network to more effectively use color information. However, there is no direct analysis of the network's learned features/structure to confirm that. The author's assume that color is a useful feature on the flower data set--but is not necessarily a useful feature on other data sets--and use that assumption to make the claim. It is clear that the Planckian color jitter is responsible for the bump in performance on the flower data set, and the assumption is reasonable.

It's important to look at data augmentation from a physics-based perspective, because uniformly randomized data augmentation ignores the fact that illumination does have typical distributions. The existing data sets usually do not sample extensively from illumination distributions, especially given that most of the images are have white-balancing applied by default.

It would be nice to see more extensive use of physics-based principles, however, since almost all standard data sets use jpg-compressed and color enhanced images (designed for human viewing), most of the physical rules that govern illumination and materials are lost.  The Plackian jitter at least replaces the uniformly random color data augmentation with expected results on a data set where color is likely to play a strong role in classification. Other examples of data sets where that might be the case are the bird and animal identification data sets. Are there other color distributions, or ways of obtaining color distributions, that might be better than the Planckian illuminants?  There are illumination situations, such as light filtering through trees, that are non-Planckian but still non-zero in terms of likelihoods. There is a difference between color augmentation of materials and color augmentation of illumination. Illumination color is one color augmentation that can be done easily through a color rotation of the pixels. While on a jpg-compressed/color-enhanced image the effect is not perfect, it's close enough that the augmentation can allow the network to learn color-based rules that follow a more typical distribution of colors.  Regardless of whether color is a useful feature for an object, this type of augmentation matches what is likely to occur in the real world.

Color augmentation for materials, on the other hand, is challenging to do properly as an augmentation. First, it is class-based, as every class will have a different material color distribution. Second, it is not appropriate to change the colors of the entire image in order to modify the material color. The uniformly random color augmentation seems like it is intended to try and cover both types of color changes (illumination and material).

It would be nice to have some discussion of this in the paper, though not necessary as it is not directly relevant to the author's contribution but an an additional limitation of the standard color augmentation methods.  But it would give a little more context to the issue in the introduction.

 This paper adds to the data augmentation transforms in the literature by proposing a more physics-based approach to generate more color-realistic augmentations compared to the standard more random color jitter.  This is well-demonstrated in Figure 1 for natural image classes such as flowers where although some classes may have strong color variations, in general the variations are more realistic when due to illumination, compared to man-made classes such as cars.  The approach is motivated by and follows a standard black-body radiation model, which again as shown in Figure 1 constrains the color jitter to be more realistic.

The experiments are performed in a self-supervised fashion as that nicely focuses the weight on data augmentation.  Both natural and man-made data sets are tested such as Flowers-102, CUB-200, and CIFAR.  Overall the results summarized in Figure 3, and Tables 1, 2, 3 show the improvements in downstream classification with Planckian jitter on natural image classes and the potential for combining Planckian and random color jitter in general. The paper is very direct in the exact problem and contribution, namely more realistic data color augmentation in a self-supervised setting, especially for natural image classes such as birds and flowers.  For the targeted data sets such as Flowers-102, at low resolution 32x32, downstream classification can be improved by 5%.  Also by combining Planckian jitter with default random color jitter, similar or even larger improvements are shown more generally across data sets such as CIFAR and T1K+.  Overall the paper keeps it simple and clear for the reader and the overall idea is sound and the experiments very directly justify the basic idea of more realistic color augmentation for data sets with natural images.

Figure 3 analyzing the color sensitivity of the results and the general results showing where for man-made imagery data sets, Planckian Jitter can be worse is useful to point out the limitations and details of the approach.  Finally it's an added bonus to verify that this basic general idea is applicable to multiple self-supervised models as shown in Table 4.  Although this isn't particularly surprising, it does add to the completeness of the experiments to support the basic idea in the paper.

Although the LSC combined augmentation results show that Planckian and random jitter are somewhat complementary, the higher-resolution, 224x244, results in Table 3 do cause the reader to question whether the low resolution case is somehow special with respect to color augmentation.  Here Planckian jitter can actually be worse for the target data set of Flowers-102 which isn't fully explained why this happens.  Although the idea seems ok for CUB-200 and better for T1K+, again for the natural image data set of VegFru, performance is worse or similar than default random color jitter.  The main explanation is a one sentence claim that for higher resolution shape is very representative and color may add little discrimination, but more investigation here is necessary.  For example, perhaps higher resolution is actually causing texture to be more discriminative, rather than shape?  Intuitively, shape seems reasonably preserved even at lower resolution, unless this is correlated with the actual self-supervised architecture of hyperparameters. As mentioned above, the main question for the reader is the somewhat counter-intuitive results in Table 3, for the same data sets but with higher resolution at 224x224.  One would expect a similar trend as the low-resolution data, but the numbers show otherwise.  Also the explanation on better shape discrimination on higher resolution bears more analysis and detail, e.g., is it more about texture than shape at higher resolution?

Although the LSC combination is a good start to fuse both types of color augmentation, what about a way to make it more class-specific?  In other words, for even a coarse dichotomy such as natural vs man-made images, apply some complementary weighting/distribution of Planckian/default color jitter to get the best of both worlds? The paper has an explicit section on limitations regarding the overhead with the combined color augmentation approach and also mentions the incomplete investigation into how much and in what detail the proposed Planckian jitter improves color realism why discouraging shape discrimination.

 This paper proposes to use a realistic white balancing as a way to augment datasets for self-supervised learning approach. The balancing was based on Planckian blackbody radiation. The author used this to train a siamese model in conjunction with standard color jittering that involves hue and saturation perturbation, and showed that the combination of the two color jittering method works best. Furthermore, the planckian jittering seems to lead to better performance on tasks where color is an important features such as flower classification task. Strength:
- Realistic image processing is complicated and is often skipped in ML literature. The color model used in this paper is largely correct and well-grounded in the color-conversion theory

Weakness:
- My main concern for this paper is that I am remain unconvinced of the benefit of the planckian jittering as presented:
  - Fig 3 shows a more stable result, but it seems that the testing dataset also undergoes the simulated jittering rather than a real captures of different illumination. Such artificial manipulation of the test set is going to yield the same result regardless of how poorly the jittering performs.
  - Planckian jitter do not always perform the best, and can underperform by a significant margin (see Table 1).
  - The motivation for resizing images to 32x32 is unclear. In most practical application, such small patch sizes are unlikely to contain enough information to be useful. More investigation is needed on more realistic patch size of 200+pixel. In the present study, only Table 3 shows this result, and the performance of PJ is mixed on different dataset.
- The color jittering model missed one crucial fact about the sRGB space, which is that it is gamma-encoded. Gamma compression is lossy, and resulted in far inferior color correction result compared to images captured in RAW. Nonetheless, I do not see the mentioning of gamma correction prior to color correction (and then reapplication to get back into the original color space). Please see my weakness section. I would like to see the response to my weakness section. particularly around the benefits of PJ, which is my main concern for this paper. Not quite. The author mentioned that their PJ reduces the quality of shape representation, but I do not see a convincing evidence of this in the text. No harm to the society is anticipated for the current manuscript.

Meta-review: Training representations in computer vision typically requires systematic augmentations of the input training set such as crops, reflections, translations including color jitter. As illumination invariance tends to be a desired property in visual object detection tasks, design of specific color augmentations is of great interest. In this context, this paper proposes to improve 
the arguably inferior color representation of standard Color Jitter by proposing 
a realistic and convincing physics-based color representation schema, – called Planckian Jitter -- and provides some experimental support. 

The discussion about the work concentrated around the overall effectiveness and experimental validation. The reviewers have mixed conclusions about the work. In light of the mixed opinions,
I tend to agree with the concerns raised by reviewer W5kb, who suggests to 

-- Repeat experiments with higher resolution as 32 x 32 seems to be rather small for a conclusion of effectiveness.

-- Including some obvious baselines (e.g., no augmentation)

-- Clarify the take home message and conclusions from ablations.

I feel that the paper would benefit from a further iteration and that the current manuscript is not ready yet for publication.


Decision: Reject