Group-Connected Multilayer Perceptron Networks

Abstract: Despite the success of deep learning in domains such as image, voice, and graphs, there has been little progress in deep representation learning for domains without a known structure between features. For instance, a tabular dataset of different demographic and clinical factors where the feature interactions are not given as a prior. In this paper, we propose Group-Connected Multilayer Perceptron (GMLP) networks to enable deep representation learning in these domains. GMLP is based on the idea of learning expressive feature combinations (groups) and exploiting them to reduce the network complexity by defining local group-wise operations. During the training phase, GMLP learns a sparse feature grouping matrix using temperature annealing softmax with an added entropy loss term to encourage the sparsity. Furthermore, an architecture is suggested which resembles binary trees, where group-wise operations are followed by pooling operations to combine information; reducing the number of groups as the network grows in depth. To evaluate the proposed method, we conducted experiments on different real-world datasets covering various application areas. Additionally, we provide visualizations on MNIST and synthesized data. According to the results, GMLP is able to successfully learn and exploit expressive feature combinations and achieve state-of-the-art classification performance on different datasets.

Reviews and author responses:

################################
Pro:
$\bullet$ It is an exciting idea to learn the relations in among feature dimensions, group them, and apply the operations within groups to train an MLP.  The experiments on various datasets, and important ablations such as the effect of number and size of groups, types of pooling were done.

################################
Cons:
$\bullet$ $\mathtt{Group-Pool}$ operation applies pooling inside feature group. Looking into Table 6 of the Appendix, mean and max-pooling were used in different datasets.  The ablation study is done on CIFAR-10 (Fig. 4). The assumption is always to have pairs of groups and apply pooling to consecutive group members. It contradicts the initial idea. Why didn't you apply a routing layer to learn which layers to pool together?

$\bullet$ *Ablation studies on image dataset:* Among many datasets experimented on; it is difficult to understand why all ablation studies are conducted on CIFAR-10. The paper's main motivation is to propose a method that works on domains beyond image, voice, and graphs. However, all ablations are done on an image dataset naturally that has spatial relations. 

$\bullet$ *Incremental performance gain:* Looking into the performance comparison with vanilla MLP and other methods (SNN, SET, and FGR), the only considerable improvement from the MLP baseline is on the CIFAR-10 dataset. The results on all other datasets, accuracy, and area under curve scores are incremental. How many trials did you conduct the same experiments (the stdev values inside brackets of Table 2)? Did you run any statistical significance test between the trials of GMLP and MLP? 

MLP baselines can be reported from an architecture matching with the number of parameters in GMLP. In Table 2, the number of parameters should be written or standardized in each method.

$\bullet$ *What are "MIT-BIH datasets"?* They are not mentioned, referred to in the text, but given in Fig. 2 (b)

$\bullet$ *Finding optimal numbers of m and k:*  Finding the optimal number of groups and group sizes can introduce a large computational overhead. Fig. 5 and 6 show the performance margin could be 10-15%. Did you compare the optimal numbers of $m$ and $k$ between the datasets with a similar feature dimension? In other words, are the optimal values calculated in one dataset generalizable to other datasets to eliminate the necessity to train many models to find a good value.

$\bullet$ How does the group routing evolve during training? In a learned model, what is the relation between inter-group and intra-group feature correlations? (Are the members of the same group more relevant to each other in vice versa?)

################################
Minor:
$\bullet$ *As a term, the use of "group":* Use of group as a term causes confusion in the reader. In the introduction, it may be useful to include that the expression is not related to group ina mathematical sense, and only represents a subset of feature dimensions.


Regarding Appendix K, as requested by the reviewer, we added more details about our implementation of the concrete relaxation (see the first paragraph of Appendix K):

“For our implementation, we use "RelaxedOneHotCategorical" class from the PyTorch library that is based on an implementation of the concrete distribution as suggested by Maddison et al. Here, we used a similar temperature annealing schedule and took new samples from the distribution at every forward path computation.”

Regarding the softmax layer, we did not explicitly threshold the routing matrix in our implementation. However, we investigated the routing matrix and it does converge to a binary matrix (with errors in the order of floating-point rounding errors). Note that we anneal the temperature parameter to near-zero values and encourage sparsity using an entropy loss term.

For instance, the routing matrix for our final CIFAR model has logit vectors in which the selected features are about 3000 times larger than other elements. In a very low softmax temperature as we have here, it translates to values that fall below the rounding threshold and are essentially binary.


Thank you, we appreciate your time and the constructive feedback.

Regarding Appendix F, as you suggested we added a brief discussion to the revised paper:

“We hypothesize that the B-tree architecture outperforms other alternatives due to two factors: $(i)$ In a B-tree, each pooling operation only merges information from two groups whereas for larger branching factor where multiple intermediate representations are being combined this pooling operation may result in larger information loss. $(ii)$ Using larger branching factors, results in an exponentially faster merging of the groups; therefore, limiting us to a much shallower network assuming the same number of groups.”

Regarding the idea of using varying branching factors, while it might be an interesting direction to investigate, we think that the B-tree architecture is considerably simpler to implement and optimize. Note that while using a constant branching factor may look too restrictive, using a larger number of groups and more layers would result in a wider and deeper model providing the desired complexity to the network. We would like to use the analogy in the comparison of the VGG architecture versus AlexNet. In VGG, smaller fixed-size kernels are used in a deeper network compared to the hand-tuned architecture and kernel sizes of AlexNet. However, the deeper and more modular VGG network outperforms the other approach.


In the added Appendix K, the authors may present their detailed implementation of the concrete relaxation method. It was surprising that it performs significantly worse than the softmax implementation on the diabetes data. 

Also, at testing, did the authors sample/threshold the group feature layers to actually only use the subsets of features for their softmax implementation? Were the presented results based on that or simply just taking the learned softmax masks? 

This reviewer truly appreciates the authors' effort to address all the questions. 

Regarding the comparison with binary-tree, 4-way-, and 8-way-trees in the added Appendix F, it would be nice if the authors can discuss the potential indications why binary-tree appears to perform the best among these structures with the diabetes dataset. 

Any possible solution to have varying branching factors as typically the size of feature groups can vary in real-world data? 

Thank you for updating your review. We are happy to hear that you found the revisions satisfactory. 

Regarding the RF method, the hyperparameters we considered were the number of ensemble trees, the tree criterion, and the maximum depth of trees. In our initial experiments, we found that the Gini criterion tends to work very similar or slightly better than the entropy measure. Also, for our experiments, using an ensemble of 1000 trees was more than sufficient to train stable and powerful RF models. However, we found that the maximum depth of trees is an important hyperparameter to be adjusted for each experiment. As this hyperparameter search only involved one variable, we used a simple brute-force search to find the optimal values.

Regarding the computation time for RF, we understand the reviewer’s suggestion that it would be interesting to have a comparison of compute efficiencies. However, we were hesitant to provide the corresponding compute times or wall clock times for Table 2. In our comparisons, we considered very different methods ranging from standard MLPs, sparse networks, and evolutionary architectures to non-neural random forest classifiers. We used conservative training schedules and adjusted hyperparameters for the best classification accuracy and not for the best computational efficiency. We think it might be unfair to compare wall clock times under this setup as we never optimized the architectures for their best compute efficiency.


I thank the authors for the strong review they provided. I have modified my review (and my score) accordingly. In particular, I agree that point (i) is incorrect on my part. The inductive bias is larger than I assumed in my review. As for the experiments, I wonder why (a) the RF is not fine-tuned as the other techniques; (b) the computational time of the RF is also not compared (eg, wrt wall clock time). These are minor points compared to my original review, so I increased my score from 5 to 7.

The paper describes an MLP architectures for problems in which the features do not have a known structure (eg, tabular data). A "differentiable routing matrix" partitions the data into K blocks. Then, standard MLPs are applied to each block and the results are recursively aggregated by moving forward in the model.

On the plus side, the paper is well written, the topic is significant (as evidenced by any Kaggle competition on tabular data), and the model is simple enough to be understandable even on a quick reading.

However, I do have a few concerns that put the paper on a borderline situation.
[Addendum after review: most of the concerns have been addressed by the authors with a large set of additional experiments.]

1. The paper is based on the assumption that a "split-then-combine" prior is sufficiently flexible to handle most real-world cases. However, this is kind of strongly limiting, eg, each feature can only contribute to a single "high-level concept" in the network. I don't feel that the paper does a strong job in justifying such an inductive bias.
[Addendum after review: As the authors point out, this is only partially true, as each feature can be selected for multiple groups.]

2. In the experimental part, I don't understand the motivation for using images. I can't imagine a single case in which one is given raw sensory data with absolutely no structure on it.

3. Concerning the other datasets (the tabular ones), the authors report a decent improvement compared to an MLP, with a significant decrease in complexity. However, I do not understand the rationale for excluding random forests (or any non-neural baselines) from this comparison. A random forest would be the default algorithm in these situations (both in terms of accuracy and speed), and making MLPs competitive is one of the motivating factors in the paper itself.
[Addendum after review: the authors have added a random forest to the experiments. However, this is the only method whose parameters are not fine-tuned (which might be a smaller problem for random forests). No time comparison is given, and some accuracy deviations appear well within the standard deviations. Still, this is a very good addition to the paper.]

4. Is interpretability a concern here? Inspecting the routing matrix should reveal information on the feature grouping. I feel this can provide a strong boost to the paper.

Summarizing, point (3) is the most important. Showing that the proposed Group MLP is superior to a classical alternative should be the critical aim of the comparisons.
[Addendum after review: This point has been partially addressed.]

Thank you for reviewing the manuscript and helpful comments. Please find a point-to-point response to your comments in the following.

----------------------------------------------------------
 
**Comment: Group−Pool operation applies pooling inside feature group. Looking into Table 6 of the Appendix, mean and max-pooling were used in different datasets. The ablation study is done on CIFAR-10 (Fig. 4). The assumption is always to have pairs of groups and apply pooling to consecutive group members. It contradicts the initial idea. Why didn't you apply a routing layer to learn which layers to pool together?

**Response: To answer your comment, we would like to mention three points to justify the use of the fixed group pooling layers:

A) The memory complexity of the routing layer does not permit using that layer multiple times for each layer. Note that during the training, the routing matrix has not converged to a sparse matrix, and therefore we need to keep and update a dense routing matrix with a relatively large size.

B) The gradient behavior of the routing matrix is not appropriate for backpropagation. More specifically, as the routing matrix converges to a sparse matrix, the back propagated gradients from the routing layer vanish significantly making the network training very difficult. In the suggested GMLP architecture, the routing matrix is at the very end of the gradient chain and hence does not impact any other layer.

C) We would like to emphasize that while the GMLP is structured as a fixed B-tree, the network still has the flexibility to map certain features to groups that are merged later as consecutive groups. In other words, ideally, the learned feature groupings would map features to groups that the network can leverage their high-level interaction further in the network. In addition to this please note that we allow features to appear in multiple groups enabling even more modeling flexibility.

----------------------------------------------------------

**Comment: Ablation studies on image dataset: Among many datasets experimented on; it is difficult to understand why all ablation studies are conducted on CIFAR-10. The paper's main motivation is to propose a method that works on domains beyond image, voice, and graphs. However, all ablations are done on an image dataset naturally that has spatial relations.

**Response: To address your comment, in the revised paper, we conducted ablation studies using the Diabetes dataset. See Appendix E: “Additional Ablation Studies”. Based on the experimental results in Fig. 11, 12, 13, 14, the conclusions made using CIFAR-10 still hold for non-image data.


----------------------------------------------------------

**Comment: Incremental performance gain: Looking into the performance comparison with vanilla MLP and other methods (SNN, SET, and FGR), the only considerable improvement from the MLP baseline is on the CIFAR-10 dataset. The results on all other datasets, accuracy, and area under curve scores are incremental. How many trials did you conduct the same experiments (the stdev values inside brackets of Table 2)? Did you run any statistical significance test between the trials of GMLP and MLP? 
MLP baselines can be reported from an architecture matching with the number of parameters in GMLP. In Table 2, the number of parameters should be written or standardized in each method.

**Response: Regarding your concern about the statistical significance of the results, as explained in the last paragraph of Section 4.1, we conducted each experiment 8 times and reported mean and standard deviation values. Note that for many of our datasets, even a small percentage of improvement over baselines is considered significant compared to other works in the literature.

Regarding our experimental method to standardize comparisons across different models, we conducted an extensive hyperparameter search for each model (see Appendix A) to find the best configuration for each model. In Table 2, we compare the best performing configuration for each model. We believe that this approach is more fair than enforcing a standardized number of parameters because we consider very different methods that are not necessarily optimized for minimizing the number of parameters. We provided exact architectures for each method and each dataset in Appendix B.

----------------------------------------------------------


**Comment: What are "MIT-BIH datasets"? They are not mentioned, referred to in the text, but given in Fig. 2 (b)


**Response: Thank you for pointing out the missing information. We added the MIT-BIH dataset to the text and included the details as well as a link to the data in the revised Table 1.

----------------------------------------------------------

**Comment: Finding optimal numbers of m and k: Finding the optimal number of groups and group sizes can introduce a large computational overhead. Fig. 5 and 6 show the performance margin could be 10-15%. Did you compare the optimal numbers of m and 
k between the datasets with a similar feature dimension? In other words, are the optimal values calculated in one dataset generalizable to other datasets to eliminate the necessity to train many models to find a good value.

**Response: In our experiments, we searched for the optimal m and k hyperparameters for each dataset (see Appendix A). In general, and as we see from the experimental results, the optimal m and k values are highly dependent on the dataset. There is no exact relationship between the number of features and optimal m and k values. However, as a rule of thumb, we found that the number of features (d) usually has the same order as m*k. 

Regarding your concern about the computational overhead, in our experiments, we were able to find appropriate hyperparameters for our largest datasets in less than 72 hours on a mid-range machine.

----------------------------------------------------------

**Comment: How does the group routing evolve during training? In a learned model, what is the relation between inter-group and intra-group feature correlations? (Are the members of the same group more relevant to each other in vice versa?)

**Response: To address your comment, as suggested, we conducted experiments to analyze inter-group and intra-group feature correlations. See Appendix J: “Analysis of Intra-Group and Inter-Group Correlations” in the revised paper. Based on the results in Fig. 22 and 23, we did not find any significant difference in the correlation values for features within each group and features between different groups. We believe that this is an expected result as the objective function is based on a classification loss and does not enforce any correlation among the learned feature groups. Note that often a level of inter-group and intra-group redundancy improves the robustness of the trained models.

In addition to this analysis, in Appendix G, we provided a method to translate feature groups to a graph representation and then used graph visualization and clustering techniques to visually demonstrate feature selection and coappearance in GMLP networks. Based on the results, feature groups appear as clusters in the graph representation and their connection with other groups is highly dependent on the dataset and the specific group itself. In general, we observed patterns such as features that appear multiple times in many different groups and features that only appear in certain groups (see Appendix G and Fig. 18).

----------------------------------------------------------

**Comment: As a term, the use of "group": Use of group as a term causes confusion in the reader. In the introduction, it may be useful to include that the expression is not related to group in a mathematical sense, and only represents a subset of feature dimensions.

**Response: Thank you for suggesting this. We added a footnote in the introduction section to clarify this for our readers:
“In this paper, the expression "group" is not related to the group in a mathematical sense, and it only represents a subset of features.”


Thank you for reviewing the manuscript and helpful comments. Please find a point-to-point response to your comments in the following.

----------------------------------------------------------

**Comment: The authors claimed that if we consider the groups as leaves, this method then becomes growing a binary tree from the leaf to the root. The extensive experiment results showed the effect of GMLP hyper-parameter choices, e.g. number of groups (number of nodes), width of each group (size of nodes) and type of pooling layer (way to built parent nodes). But in terms of a tree, it would be interesting to have some experiments to show the effect of the way combining feature groups. 

**Response: To address your comment, as suggested, we conducted additional experiments using different tree architectures by changing the tree branching factors. See Appendix F “Experiments using Alternative Tree Structures” in the revised paper. 
Based on the presented results, the suggested B-tree architecture appears to be providing better results compared to other alternatives.

----------------------------------------------------------

**Comment: The experiments on the simulated Bayesian network dataset supported the claim that this architecture can utilize the fact that some of the features are not related and do not need to interact with each other. However, the architecture the authors used corresponds to the model that generated data, which is almost impossible in many real life problems. It can be helpful to have some results on simulated data with mismatched architectures from the model to help better understand the performance.

**Response: As suggested by the reviewer, in the revised version, we provided a comparison on the synthesized dataset for mismatched architectures. See the revised Appendix D (Fig. 10 and its explanation).

In summary, we experimented with different m and k values that are lower and higher than the matched case of m=2,k=4. Based on the results, for this dataset, there is a level of mismatch for using smaller m and k values that degrades the results significantly, while higher capacity mismatched models provide similar accuracies as the matched case.

----------------------------------------------------------

**Comment: One of the most important ideas in this paper is limiting the group-wise interactions. The size and number of groups the experiments chose would lead to many overlapped groups and many features chosen multiple times. It would be nice to have some analyses on the chosen groups and selected features, e.g. the existence of a set of features that always come into one group, and/or comparison of derived groups by GMLP and randomly chosen groups.

**Response: To address your comment, we added a new appendix to the revised paper: Appendix G: “Analysis of the Selected Feature Groups”.

In summary, in Appendix G, we provided a method to translate feature groups to a graph representation and then used graph visualization and clustering techniques to visually demonstrate feature selection and coappearance in GMLP networks. Based on the results, feature groups appear as clusters in the graph representation, and their connection with other groups is highly dependent on the dataset and the specific group itself. In general, we observed patterns such as features that appear multiple times in many different groups and features that only appear in certain groups (see Appendix G and Fig. 18).

----------------------------------------------------------


**Comment: A more detailed explanation of the dataset would be needed. For example, the authors used MIT-BIH dataset to compare the accuracy of GMLP and MLP with different sizes without introducing the dataset.

**Response: Thank you for pointing out the missing information. We added the MIT-BIH dataset to the text and included the details as well as a link to the data in the revised Table 1.

----------------------------------------------------------

**Comment: If the complexity analysis of GMLP, equation (7), is only for inference, please also include the training complexity. Another concern is that the results suggest that the number of feature groups may need to be quite large (compared to the number of original features). In addition to figure 2, please provide the analysis of model size in addition to complexity analysis. 

**Response: To address your comment, we provided the training time memory and compute complexity as well as a discussion on the model size in Appendix I of the revised version. Regarding your concern about model size or memory use, we were able to conduct all the experiments in this paper on a mid-range GPU with 11GB memory.

----------------------------------------------------------

**Comment: GMLP selects features by using soft-max of a km×d matrix. The authors may want to investigate reparametrization tricks to solve similar problems, including concrete relaxation in the following references: 
C. J. Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. In ICLR, 2017.
Muhammed Fatih Balin, Abubakar Abid, and James Y. Zou. Concrete autoencoders: Differentiable feature selection and reconstruction. In ICML, 2019.

**Response: As suggested, we conducted experiments using Group-Select layers that are implemented using the concrete relaxation method. See Appendix K: “Comparison of the Softmax and Concrete Relaxations” of the revised paper.

In summary, we did not observe any improvement using the concrete distribution. We hypothesize that as the major use case of the concrete distribution is in variational methods and it involves random sampling, it might be injecting a level of noise and variation that is not necessarily helpful for learning the feature groups.


Thank you for reviewing the manuscript and helpful comments. Please find a point-to-point response to your comments in the following.

----------------------------------------------------------

**Comment: The paper is based on the assumption that a "split-then-combine" prior is sufficiently flexible to handle most real-world cases. However, this is kind of strongly limiting, eg, each feature can only contribute to a single "high-level concept" in the network. I don't feel that the paper does a strong job in justifying such an inductive bias.

**Response: In the proposed method each feature can contribute to multiple concepts as there are no restrictions on features being selected multiple times in different groups. Note that in Section 3.3, the routing matrix in the group-select layer has the flexibility to include each feature in different  groups as the rows in the \psi matrix are trained independently. As such, the proposed method is not as restrictive as a “split-then-combine” inductive bias. 

To address your comment and to clarify this for our readers, we added the following explanation in the revised paper (see Section 3.3 after eq.1):
“Note that this formulation allows each feature to contribute to multiple groups as it allows features to be selected multiple times in different groups.”

----------------------------------------------------------

**Comment: In the experimental part, I don't understand the motivation for using images. I can't imagine a single case in which one is given raw sensory data with absolutely no structure on it.

**Response: We agree with you that image datasets are not  the best use case for the suggested method. We only included the CIFAR10 results as this commonly well-known benchmark used in the literature enabling further comparisons with other SOTA methods. In our experiments, in addition to one image dataset, we used 7 real-world non-image datasets to demonstrate the effectiveness of the proposed method.

To address your comment, in the revised version, we added a new appendix (Appendix E) and used the Diabetes dataset to re-evaluate the ablation studies for a non-image dataset. Further, in the revised manuscript, we conducted all additional experiments on non-image datasets to conform with your comment, and provide our readers with a better view of the proposed method and the problem it intends to solve.

----------------------------------------------------------

**Comment: Concerning the other datasets (the tabular ones), the authors report a decent improvement compared to an MLP, with a significant decrease in complexity. However, I do not understand the rationale for excluding random forests (or any non-neural baselines) from this comparison. A random forest would be the default algorithm in these situations (both in terms of accuracy and speed), and making MLPs competitive is one of the motivating factors in the paper itself.

**Response: To address your comment, as suggested, we included random forest classification (RFC) results in the revised Table 2. In our RFC implementation, we used ensembles of 1000 trees trained with the gini criterion and adjusted the maximum tree depth for the best accuracy. Based on the results, GMLP outperforms the random forest baseline on  all of the datasets.

----------------------------------------------------------

**Comment: Is interpretability a concern here? Inspecting the routing matrix should reveal information on the feature grouping. I feel this can provide a strong boost to the paper.

**Response: Thank you for suggesting this. We do not believe that interpretability is a main concern of this paper as the suggested objective function does not focus on interpretability. However, we believe that studying interpretability for GMLP architectures would be a very interesting subject for future studies.

In the revised paper, we added a new appendix (see Appendix G) to use graph visualizations to demonstrate feature groups and their relationship for GMLP networks trained on three different datasets. Graph representation of feature groups can be a potential avenue for further analysis of group interactions.


We thank the reviewers for the constructive comments and suggestions. We believe that the suggested revisions enhanced the scientific quality of the manuscript significantly.

I addition to making revisions throughout the paper as suggested by the reviewers, in the revised version, we included 6 new appendices:

-    Appendix E: re-evaluates the ablation studies using the Diabetes dataset.
-    Appendix F: compares the suggested binary tree (B-tree) GMLP architecture with other alternatives using tree structures with different branching factors.
-    Appendix G: uses graph clustering and visualization to study the interaction between feature groups in GMLP networks.
-    Appendix I: provides an analysis of the training time compute and memory complexity.
-    Appendix J: provides experiments to compare inter-group and intra-group feature correlations.
-    Appendix K: provides a comparison of training GMLP networks using the suggested softmax relaxation and the alternative concrete distribution.


The authors proposed a neural network architecture, Group-Connected Multilayer Perceptron (GMLP), which automatically groups the input features and extracts high level representations according to the groups. This paper focuses on classification problems.

The architecture can be decomposed to three stages. The first stage is to automatically group the input features by multiplying soft-max of a routing matrix. At the second stage, a locally fully connected layer with the corresponding activation functions is used for each group, and a pooling layer merges two groups to a new group. At the final stage, all the groups would be concatenated and input to a fully connect layer to get the final output. 

The experiments showed that GMLP outperforms vanilla MLP, SNN, SET, FGR on seven real-world classification datasets in different domains. GMLP ensures higher accuracy with lower complexity compared to vanilla MLPs.

The authors claimed that if we consider the groups as leaves, this method then becomes growing a binary tree from the leaf to the root. The extensive experiment results showed the effect of GMLP hyper-parameter choices, e.g. number of groups (number of nodes), width of each group (size of nodes) and type of pooling layer (way to built parent nodes). But in terms of a tree, it would be interesting to have some experiments to show the effect of the way combining feature groups. 

The experiments on the simulated Bayesian network dataset supported the claim that this architecture can utilize the fact that some of the features are not related and do not need to interact with each other. However, the architecture the authors used corresponds to the model that generated data, which is almost impossible in many real life problems. It can be helpful to have some results on simulated data with mismatched architectures from the model to help better understand the performance. 

One of the most important ideas in this paper is limiting the group-wise interactions. The size and number of groups the experiments chose would lead to many overlapped groups and many features chosen multiple times. It would be nice to have some analyses on the chosen groups and selected features, e.g. the existence of a set of features that always come into one group, and/or comparison of derived groups by GMLP and randomly chosen groups.

A more detailed explanation of the dataset would be needed. For example, the authors used MIT-BIH dataset to compare the accuracy of GMLP and MLP with different sizes without introducing the dataset.

If the complexity analysis of GMLP, equation (7), is only for inference, please also include the training complexity. Another concern is that the results suggest that the number of feature groups may need to be quite large (compared to the number of original features). In addition to figure 2, please provide the analysis of model size in addition to complexity analysis. 

GMLP selects features by using soft-max of a $km \times d$ matrix. The authors may want to investigate reparametrization tricks to solve similar problems, including concrete relaxation in the following references: 

C. J. Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. In ICLR, 2017.

Muhammed Fatih Balin, Abubakar Abid, and James Y. Zou. Concrete autoencoders: Differentiable feature selection and reconstruction. In ICML, 2019.


Meta-review: The paper proposes an MLP based approach for data without known structure (such as tabular data). At first, the data are partitioned into K blocks in a differentiable way, then the standard MLP is applied to each block. The results are then aggregated recursively to produce the final output. 

Pros:
1. Handling less structured data is surely an important problem in machine learning and is much less explored. 
2. The paper is well written, easily understandable even with a fast browsing. 
3. The experimental results show some improvement. 

Cons:
1. The approach is somewhat trivial, and the framework could be improved, see, e.g. Reviewers #3&#4. 
2. By the structure of the approach and the type of target data, a more reasonable comparison is with random forest (echoing Reviewer #1), which the authors added during rebuttal, rather than MLP etc. Maybe should even compare with deep random forest. Although the comparison with MLP etc. is quite favorable, the advantage over random forest is somewhat marginal (except on HAPT, which is a imagery data set and random forest may not be good at; also echoing Reviewers #1&#4's comment on why using imagery data, which do not fit the theme of the paper). Reviewers #3&#4 also had some concerns with the experiments. Reviewer #4 confirmed in the confidential comment that the performance improvement is incremental.

Although the rebuttal seemed to be successful, thus both Reviewers #1 and #4 raised their scores, the average score is still at the borderline. Due to the limited acceptance rate, the area chair has to reject the paper.

Decision: withdrawn-rejected-submissions