Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work proposed a texture synthesis framework using the rectified wavelet coefficients. The paper claims that the proposed method cand achieve similar quality with the VGG feature based method (Gatys et al. 22015) and random filter based method (RF, Ustyuzhaninov et al 2017) and gets better quality than PS (Portilla & Simoncelli 2000, while requiring less number of statistics than RF. 
main_review: This paper provides a detailed review and explanation of wavelet and how to use the rectified wavelet coefficients for covariance statistics matching based texture synthesis. The motivation also seems to be clear: trying to use less number of statistics to achieve satisfying quality. However, I have several main concerns regarding to the comparisons with recent papers and the types of texture images this method can handle.

1. as the main motivation is trying to achieve good quality with as few number of statistics as possible, from figure 3 and figure 4, the quality of “VGG” seems to be better than the proposed method and also uses less number of statistics (much less for color textures with 177k vs. 320 and similar for gray texture 177k vs. 142k)

2. After Gatys et al. 2015 and Ustyuzhaninov et al. 2017, there are more recent methods on texture synthesis which have achieved many impressive results. For example:
    Deep correlations for texture synthesis.  Sendik & Cohen-or et al. 2017.
    Texture Synthesis Through Convolutional Neural Networks and Spectrum Constraints. 2016. 
    Non-Stationary Texture Synthesis by Adversarial Expansion. Zhou et al. 2018. 
    and many others.
  This submission did not cite or compare with these approaches (and other more recent methods).
 
3. The submission and supplementary file provide very limited number of results. The images are mostly stationary images. More examples of structure images and non-stationary images are needed to validate the proposed method. 

4. One main application of texture synthesis to generate a larger image output given a smaller input patch. It seems that the proposed can not generate a larger image output. 

summary_of_the_review: Although the motivation of the proposed framework seems to be pretty specify and properly positioned, the experiments does not seem to validate the superiority over existing methods. The results are quite limited. It also missed the citation and comparisons with many recent works.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper provides an attempt to characterize wavelet-based texture synthesis models contingent on the number of parameters and the nature of the parametrization. Proper baselines are added with respect to other relevant texture synthesis models.
main_review: This is a paper that has a great idea going on with regards to qualitatively estimating the minimum number of constraints in a parametric texture model that revolves around wavelets. My main motivation for marginally rejecting this paper is that it comes up short in terms of the types of qualitative insights it can deliver, not capitalizing enough in the rich mathematical motivation for a wavelet-constrained model. Nevertheless, I think I am lenient on increasing my score if authors address some of the points I mention below and I’m also genuinely curious on what the other reviewers thoughts are.

* Not enough highlight on the number of parameters of the wavelet based model. This should be emphasized more in Table 1, or in an extra table as it’s superficially mentioned at the end of section 4.1. The exploration of how the number of parameters affects the quality of the synthesized stimuli is a pivotal part of this paper that has not been highlighted enough (the results are there, but the presentation needs to improve).
* Adding more examples of model failures, and singular successes. The figures in 2,3, and 4 are too basic and standard. I think they should be resumed in another separate figure  (perhaps just using figure 4 or a variation of this), and there should be *other new* figures that highlight how the models perform for very specific images. What would the synthesized textures look like for a scene? An abject? Julesz conjecture counter-examples? And does the quality change as the number of parameters is varied (Alpha S,I,L,C) Many of the previously mentioned images that do not provide a stationary signal can magnify the differences in terms of how each model computes its perceptual constraints. 
* Linking to the previous bullet point: Why does each model fail? What texture constraint has varied, such that the model all of a sudden breaks down or succeeds for some of these other images that are different from the canonical textures shown in the texture synthesis literature.

Overall, the paper is well written, it provides a good mathematical overview of wavelets and also cites the relevant literature on this subject. I would also suggest authors to look into citing this paper as well that provides some connections of texture synthesis to neuroscience:

* Texture Interpolation for Probing Visual Perception. Vacher et al. NeurIPS 2020.

Once again, I am looking forward to reading the author's rebuttal as I am eager to increase my score as this paper is already heading in the right direction.


**Post-Rebuttal Notes**: I wanted to increase my score from 5 to 7, but since there is no 7 option I have decided to sin on the generous side and give this paper an 8. [I apologize to authors and/or reviewers if they have seen me toggle back and forth between 6 and 8. My overall vote is that this paper be accepted as a poster.]


summary_of_the_review: A solid first step, but more qualitative insights are needed given to make this paper shine as much as it should. Weak Reject with a possibility of increasing my score during rebuttal to accept.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a new image representation model based on wavelets and non-linear rectifiers that allows to synthesize complex geometric textures with a better visual quality than previous wavelet-based models. 
The main interest of the paper is the usage of the mathematical results from Mallat et al 2020 and Zhang and Mallat 2021 on wavelet phase harmonics to image texture synthesis. 

They also show that the PS model (Portilla & Simoncelli 2000) is a particular case of the wavelet phase harmonics-based (WPH) model. Both methods underline the importance of statistical dependencies between wavelet coefficients across scales. The specific choice of the parameters available for WPH is important to balance between reproducing good structural information and memorizing patterns from the observation.
 
main_review: **Positive points**: The results presented in the paper are very interesting because they shed light on what exactly are CNNs learning on images, since the presented model can produce similar results to state of the art CNNs with a much smaller number of parameters and a fixed filter structure (wavelets). This is, obviously very important given the usage of CNNs in the literature and the little current comprehension there is on the information they encode and it is relevant for image comprehension. In this sense, this paper shows that the rectifier wavelet operator is 

**Negative points**: The relationship with patch-based methods is not discussed at all. Even thought the proposed algorithm lays within the "statistics-based methods", in the numerical results it should be compared to other family of methods such as exemplary-based models (see Raad  et al 2018 for examples) 
summary_of_the_review: The paper presents the results on texture synthesis using a new image representation model based on wavelet phase harmonics. The results are interesting and show that these representations are powerful enough to encode complex image features. What this paper presents starts clarifying how CNNs are encoding image comprehension
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a novel image texture model that is a hybrid of the traditional wavelet constructions, and the more modern machine learning constructions (incorporating only ReLU nonlinearities on the wavelet basis), proves that this class of models is actually akin to phase harmonic functions and that these are sufficient for capturing texture statistics, and claims to show competitive performance with state of the art with significantly fewer parameters.
main_review: Proving the relationship between the different sets of texture models, with the additional proof that ties ReLU's and phase harmonics together is nice work and a strong contribution.  Unfortunately, much of the rest of the paper is lacking and detracts from the overall effect.

Main criticisms:
1.  This paper is not suitable for ICLR as nothing within the paper is learned from data, these are hand constructed (and well though through) representations that are similar to those found in learned representations but are not here learned.  This is not a problem per se, it simply doesn't fit with the conference themes and as such may not be found useful by most of the conference attendees.

2. The paper spends an inordinate amount of space laying out and explaining concepts and algorithms which are not novel contributions of this paper and would be better served with passing references for the reader to follow up with.  The novel contributions of the paper do not begin until the bottom of page 4.

3. The examples shown reveal that all of the Alpha models (including AlphaI and AlphaC the chosen models for analysis in this paper) are memorizing patches of the image (any image that has the portilla simoncelli badge in the bottom left corner has this badge completely recreated somewhere else in the image as completely readable text), which a proper texture model would not do unless it was simply memorizing the image patches.  This is especially apparent in the color images which have more of examples of the name badge in the corner.  

4. The authors state parameterizations of comparison models on page 8 that are at least partially incorrect (here stated that portilla simoncelli has ~7000 parameters when in reality it has 701 parameters, an order of magnitude difference).  A careful inspection of the others may find similar magnitude errors.

summary_of_the_review: For these reasons above I am inclined to reject this paper for ICLR, and suggest the authors find a more suitable conference for this work, as well as reanalyzing some of the results and rethinking the overall presentation of the work.
review_title: null

Meta-review: This paper proposes a new wavelet-based model to represent textures. The model incorporates a wide range of statistics, by computing covariances between rectified wavelets coefficients, at different scales, phases and positions. The model can synthesize textures that have a similar quality to state-of-the-art texture models using CNN structure. Qualitative results are shown to demonstrate the effectiveness of the model.  The paper studies an important problem in computer vision and neuroscience, which is texture modeling. However, many important related works are missing. After rebuttal, three of four reviewers champion accepting the work because the proposed wavelet-based texture model, which produces competitive synthesis with much less parameters than the CNN-based model, will be beneficial to the fields of computer vision and neuroscience. One reviewer has critical comments on this paper because the paper lacks a comparison again more recent works both quantitatively and qualitatively. However, during rebuttal, the authors expressed their disagreement with it and pointed out that the goal of the paper is to bridge the gap between the classical work of Portilla and Simoncelli (2000), and the CNN-based models and to find what statistics are needed to describe the geometric structures in natural textures. Their discussion didn't reach an agreement after rebuttal. After an internal discussion, AC recommends accepting the paper but urges the authors to improve their paper by taking into account all the suggestions from reviewers, especially include the discussion or comparison with those related works mentioned in the rebuttal.
Decision: Accept (Poster)