Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a new "global" pooling approach that aims at preserving detailed/fine-grained information from the representation. The proposed approach is based on Fractal Encoding that are implemented such that the overall representation is still trainable. The proposed approach is tested on the task texture representation.
main_review: Generally, the paper addresses an interesting downside of using Global Average Pooling (GAP) in CNNs, however this downside is only specific to certain situations IMHO, so I believe that the proposed solution is not broadly applicable. The following points highlight some more specific pros and cons.

Pros:
+ The paper is clear and well written. 
+ The proposed solution based on Fractal Encoding is well motivated and technically sound.
+ Making sensitive hyper-parameters, such as quantization intervals, trainable is a good approach over using some ad-hoc solution.

Cons:
-  Generally the empirical results do not seem to speak strongly in favour of the proposed approach. In particular, Table 1 is either below state-of-the-art or shows that the proposed approach is only marginally better (1% or less).
- The discussion regarding the sub-optimal performance on DTD (LINES 216-221),  further highlights the niche application of the proposed approach. That being said, I wonder how/if the proposed approach would benefit dynamic textures (e.g. [a])? have the authors considered this? 
- In terms of the method itself, the proposed approach should in fact replace GAP and I don't see a striking complementarity between the two. This comment is further confirmed in Table 2, which shows that GAP only yields a marginal contribution. Can the authors further comment on that?

[a] I. Hadji and R. P. Wildes,  "A New Large Scale Dynamic Texture Dataset
with Application to ConvNet Understanding", ECCV 2018
limitations_and_societal_impact: While the authors provided different ablations, they did not explicitly discuss the limitations of their approach. It would be useful to dedicate a small subsection to this as this would benefit progress on this domain.
Negative societal impact is not applicable in my opinion.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose fractal encoding (FE) module to encode spatial distribution of convolutional features for texture representation. The effectiveness of FE is shown with experiments about texture classification and retrieval.
main_review: The proposed fractal encoding module is simple yet efficient (e.g., parameter number, Flops, and running time). It has trainable parameters for data adaptivity and also can be inserted into existing CNN structures.
The main results on texture classification and retrieval demonstrate the effectiveness of the proposed method.
The ablation study with visual results well shows the generalization across scales.
The paper writing and organization are good and easy for readers to follow.
Would the authors provide the code of the method?

After rebuttal

Thanks for providing the feedback. I also read comments from other reviewers and like to keep my original rating.
limitations_and_societal_impact: yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose complementing global average pooling in convolutional networks with another pooling method that aims at better representation of texture. This pooling method builds on fractal geometry and is implemented by a fractal encoding module. The module first locally estimates fractal dimension, the estimates are binned and inside each bin the fractal dimension estimation is applied again. This results in pooling of one input channel into a fixed-sized vector. The operations, such as binning and counting, are adapted to make the module differentiable. The experiments show that both types of pooling modules are required for good performance.
main_review: The proposed method is new and provides improved performance for texture classification and retrieval. The method is relatively clearly explained, but there is definitely room for improvement, see detailed suggestions and questions below.

The question is whether the FE module has any application beyond texture-related tasks as it is known that convolutional networks, even without this module, focus too much on texture cues. 
limitations_and_societal_impact: Suggestions:
- Consider showing & exploring the errors/failures of the proposed method.
- Fig. 2 would benefit from a larger font ("Learnable Kernels", "Output Density Map")
- Please check if the leading term in the numerator and in denominator of (8) should be multiplied by Z.
- Explain how where the parameters of the module selected (line 196).
- Add the input image size used for Table 4.
- Fig. 4 - labels for (b) and (c) seem to be flipped, label for (d) should probably read "TE Feature".
- Fig. 4 - would it be possible to add one more panel for the baseline feature? And to all three "Feature" panels add the 384x384 series.
- Fig. 4 - please enlarge the font in the "Feature" panels.
- Fig. 5c - specify if this is FENet-18 or FENet-50.
- Fig. 4 shows DeepTEN for comparison but Fig. 5 shows DEPNet for comparison. Consider using a consistent set of baselines.

Questions:
- What are the differences between the t-SNE visualization in Fig. 5 and SFig. 4?
- Why are DeepTEN and DEPNet selected as the most common baselines when Table 1 shows that their results are relatively poor? DSRNet would seem to make more sense.
- Any comments on why is FENet clearly slower than DEPNet even if FENet requires less FLOPS than DEPNet?

Typos:
- L25: output -> outputs
- L103: point -> points / term -> terms
review_title: null

Meta-review: The paper addresses the important problem of texture representation in neural network models. The approach is interesting and well received by the reviewers, and I personally find that many industrial applications could benefit from it.

Decision: Accept (Poster)