Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a novel approach to incorporate into SVMs several properties seen in CNNs. In a specific way, authors introduce domain knowledge of invariances (i.e. image recognition), maximum similarity on a group of transformation (kernel based), incorporating local features at multiple spatial scales, as well as composition through the architecture of multiple layers. This characteristics are shown in a set of experiments wit the polynomial kernel and the L-TI-RI-SVM (and variants), CNN and ResNet applied into MNIST, EMNIST and translated MNIST.
strengths_and_weaknesses: Strengths: The core idea of the work is original in the sense of incorporating main characteristics involved in CNNs, into the construction of SVMs. Taking advantage of kernels, authors established the conditions for a symmetric kernel that is positive-definite transformation-invariant (polynomial kernel). The locality at multiple special spaces is introduced as well in the kernel with these characteristics. Figure  2 explains the advantages of the L-TI-SVM and L-RI-SVM in the T-MNIST and R-MINST

Weaknesses: the contribution on the is, at this stage, involved in problems related to image recognition, involved with the structure of pixels (and all the inner related characteristics). It is important to remark that authors use a very well-known dataset; MNIST and some variants. It is also important to consider that other datasets are also used as benchmarks, tackling few shot learning in images. It should be nice to offer a github link in order to reproduce the results.

questions: •	The manuscript section organization is missing from the introduction, it should be included.
•	Is it possible to consider to show results in table 2 in a visual manneer (i.e. figure 2)?
•	Is it possible to extend the work on future directions as, per example, other kernels?

limitations: The authors address the problem and found that polynomial kernels, under certain conditions can address the problem in image recognition. It will be nice to have the code in order to reproduce results and the corresponding architectures.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a novel formulation of SVM to tackle the problem of learning in scarce data regime. it is inspired by the properties of CNNs, namely translation invariance and locality at different scales.
strengths_and_weaknesses: The success of CNNs is related to the availability of large-scale data and the development of hardware. Moreover, the properties of CNNs such as compositionality, translation & rotation invariance,  locality, and multi-scale representations have made CNNs successful in a large scope of applications. However, CNN's are data hungrier. In this paper, the authors tackle the problem of learning in a small data regime by reformulating SVMs to take into consideration the translation invariance, locality, and compositional structure properties of CNNs. 
One of the strengths of the paper is in the modeling of the proposed SVM and the mathematical validations,  including the verification of the satisfactions of the positive-semi definiteness of the proposed kernel.  The claims are well supported and the mathematical variables are well defined which makes the paper easy to follow.

The methodological part includes two theorems and three lemmas that are validated empirically considering different experimental protocols. However, the experimental part could have included other applications (video, speech, physical data) where the data are rare and tasks (regression, etc) to show the versatility of the methods.
questions: 1/ What is the training and inference time of the proposed method? it could be interesting to add a column mentioning the training time & inference time and the number of parameters when comparing with related works and DL methods.

2/ For the sake of comparison and a better understanding of the formulation of the proposed method, could you make some visualizations of the learned kernel as in CNNs to show the learned filters? (figure 3 in ImageNet Classification with Deep Convolutional
Neural Networks https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf)
limitations: This work doesn't have any negative societal impact.
The authors didn't mention the limitations of their works.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: ** I thank the reviewers for their answers **

The paper analysis theoretically the best fit kernel under transformational invariance. Such kernel is not necessarily definite positive. The authors prove some bounds for which it is with hight probability as long as the sample size is small enough.

The authors also incorporate into the kernel the knowledge of locality.

This strategy is inspired from the success of deep convolutional neural networks.

To my understanding, the paper focuses on small samples mainly as a limitation rather than as a specific objective (this is not a critic).

The experiments confirme the benefit of the method on small samples.
strengths_and_weaknesses: - Strengths
The paper provides some theoretical and practical elements about the use of such kernels for small samples. They show that even though such kernels are not always positive definite, they can be as such on small samples.

- Weaknesses
The paper is a succession of many elements that I found some time difficult to ingest.
Experiments do show std which might be large on small samples.
questions: I understand that in the experiments L is locality, TI is translational invariance and RI rotational invariance but I think that it would improve readability to introduces the notations.

Could you provide an empirical curve of probability to be positive definite given the sample size, maybe on a syntetic dataset ?

Could you provide std on your experiments. Small samples might be linked to large variance in the experiments.
limitations: Some limitations are given in the core of the paper but not specifically addressed.
review_title: null

Meta-review: metareview: All reviews are quite positive. 

Accept.
confidence: Certain
Decision: Accept