On a Built-in Conflict between Deep Learning and Systematic Generalization
Authors: Yuanpeng Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: out-of-distribution generalization
systematic generalization
compositional generalization

Abstract: Out-of-distribution or systematic generalization is a desirable property that most deep learning algorithms lack. In this paper, we hypothesize that internal function sharing is one of the reasons to weaken systematic generalization in deep learning for classification tasks. Under equivalent prediction, a model partitions an input space into multiple parts separated by boundaries. The function sharing prefers to reuse boundaries, leading to fewer parts for new outputs, which conflicts with systematic generalization. We show such phenomena in standard deep learning models, such as fully connected, convolutional, residual networks, LSTMs, and (Vision) Transformers. We hope this study provides novel insights and forms a basis for new research directions to improve systematic generalization.

Under review as a conference paper at ICLR 2023
ON A BUILT-IN CONFLICT BETWEEN DEEP LEARNING AND SYSTEMATIC GENERALIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
nut-of-distribution or systematic generalization is desirable property that most deep learning algorithms lack. In this paper, we hypothesize that internal function sharing is one of the reasons to weaken systematic generalization in deep learning for classification tasks. Under equivalent prediction, model partitions input space into multiple parts separated by boundaries. The function sharing prefers reuse boundaries, leading fewer parts for new outputs, which conflicts with systematic generalization. We show such phenomena in standard deep learning models, such fully connected, convolutional, residual networks, LSTMs, and (Vision) Transformers. We hope this study provides novel insights and forms a basis for new research directions to improve systematic generalization. Source codes are available in the supplementary material.
INTRODUCTION
A fundamental property of artificial intelligence is gen- eralization, where trained model appropriately pro- cesses unscen test samples. Many problems adopt the Li.d, assumption. On the other hand, out-of-distribution (o.o.d.) systematic generalization (Fodor & Pylyshyn 1988 Lake & Baroni, 2018) requires that training and test distributions disjoint so that test samples have zero probability in training distribution. Systematic gen- eralization is crucial for human learning and supports efficient data use and creativity. Therefore, machines are encouraged acquire such generalization ability achieve -imann-like intelligence.
Accuracy (%) for systematic generalization
small
large
Degree of function sharing
Systematic generalization usually requires that sam- ple has multiple explanatory factors of variation (Bengio jet al., 2013), and the generalization is enabled by pro- ducing unseen combination of seen factor values. For example, models trained on blue rectangles and green triangles predict blue triangles. We adopt factors mainly in designing experiments and developing intuitions. helps experiments because outputs only related to function sharing between factors (Section B). So we limit claim the cases for recombination of factors.
Figure 1: A simplified plot for result in the experiment section. As the degree of function sharing increases (more sharing layers), the accuracy of the test dataset generalization capacity decreases cordingly. It indicates that function shar- ing in deep learning conflicts with sys- tematic generalization. Please refer to Section 3 and Figure 5a for details.
One stream of artificial intelligence is Connectionism (Feldman & Ballard 1982 Rumelhart et al., 1986), which many simple neuron-like units richly interconnected and processed in parallel. It was criticized that Connectionist models do not support systematic generalization well (Fodor & Pylyshyn 1988; Marcus [998). Deep learning (LeCun et al. 2015) originates from Connectionism,, and various techniques have enabled multiple-layer modelings and improved performance on i.i.d. problems recent years. Also, specific algorithms have been proposed to equip deep learning with systematic generalization ability (Russin et al.] 2019 Lake 2019). However, less discussion has been made on why standard deep learning models do not achieve systematic generalization.

---

Under review as a conference paper at ICLR 2023
+/+
+/+
+
+
+
+
+/+
+
+
+
+
%
(a) DL preferred model (b) DL umpreferred model
(c) First function
(d) Second function
Figure 2: Intuitions for the model preference of deep learning (DL). Each figure an input space containing three sets of training samples with outputs of +/+, -/-, and -/+, respectively. Models have two decision boundaries (cyan and magenta). The orange dot is test sample with a new ground- truth factor combination +/-. We discuss that the training process prefers the first case (a) over the second one (b) because it likes to share or reuse functions. Suppose the first function (c) is learned, then the process tends to reuse it by learning simple function (d) and combining them instead of learning the complicated magenta function in (b) from scratch. So the function between +/+ and -/- shared. Few inputs mapped the new output, and systematic generalization is not achieved.
This paper addresses the above question by looking into a built-in characteristic of deep learning. A node (or set of nodes) is func- tion of the network input. By function shar- ing, mean nodes in one layer share inputs from nodes in the previous layer. It may also be called activation sharing feature sharing. We hypothesize that function sharing is one of the reasons to prevent systematic generaliza- tion in deep learning. Under equivalent predic- tion, classification network partitions an input space into multiple parts separated by bound- aries. Function sharing prefers to reuse bound- aries and avoid redundant ones for training pre- dictions. leads to fewer parts for new outputs and weakens systematic generalization. The nearest neighbor classifier is an analogy for this effect because it predicts training sample out- put for any test sample, so it does not have part for new outputs. We also discuss that func- tion sharing belongs naturally to deep learning (Section 4).
First oulput
Second output
Result
Figure 3: A visualized example with a two- dimensional input and two binary outputs. The model is a multiple-layer fully connected neural network. The three figures contain the predictions for the first and the second outputs and classifi- cation results for the model. Each figure is the input space. The dots are training samples whose colors are ground-truth labels. For output figures, the background colors are model predictions. We set the new combination of labels (blue-blue) as the correct pair of outputs. For the result figure, the background is orange if both predictions are wrong, white if one is correct, and blue if both are correct. The result figure does not have a blue background, indicating that the model does not predict the new label combination for any input.
Figure 2 has an intuitive example explaining why the conflict happens. A test sample (in or- ange) equals a set of training samples (+/4) on the first output. Then they also equal on the second output if the function is reused (see caption). Therefore, they are equal in all the outputs, which conflicts with systematic generalization. More generally, the two boundaries jointly partition an input space into multiple parts, and deep learning prefers (a) because it has fewer parts than (b). Figure has visualized example. It is similar to the example Figure 2a The two functions are shared the top-right region, and few inputs are pre- dicted the new combination. Figure I has simplified plot for result in the experiment section. As the degree of function sharing increases (more sharing layers), the accuracy of the test dataset or generalization capacity decreases accordingly. It supports that function sharing weakens systematic generalization. Please refer to Section 3 for more details.
This paper contributes to uncovering a built-in conflict between deep learning and systematic gener- alization. We hope this study provides novel insights, forms a basis for new research directions, and helps improve machine intelligence to the human level.
2

---

Under review as a conference paper at ICLR 2023
2 A BUILT-IN CONFLICT
We hypothesize a built-in confiict between the function sharing in deep learning and systematic generalization. We cover the definition, assumptions, and derivations of the conflict.
2.1 SYSTEMATIC GENERALIZZATION
We have input set X and output set Y. is decided by K factors Yi....,Yy, The training set Duain has input Xirain and output Virain- The test set Diea has input Xue and output Yea- In systematic generalization, Iran and Yeu are disjoint. However, the values for each factor i are included in the training output. A model f maps input X the prediction of output f(X). model enables systematic generalization if it correctly predicts the ground-trutt test outputs.
Definition 1 (Systematic generalization). The dataset requires that any test label is not training label, but each factor of a test label is seen in training label.
((r,y) y) E Diess : y E Yiain: function enables systematic generalization (xr y) Diesr : = fxx
Vi = 1==,.... -,...,,K
1i,...,,3500 K, E Ymain : 1/4 = yi
When the model is well trained, we assume it correctly predicts training samples. Assumption (Correct training prediction). V(x, y) Duvin y f(x).
2.2 FUNCTION SHARING
In Figure 2 deep learning prefers the model in (a) over the model in (b). We denote the models as and 9 (mapping from input to output), respectively. We see that g has one more region for the new factor combination between the magenta and cyan boundaries. This region exists because the magenta boundary in g splits the +/+ region in f into two parts. means the partition under g refines that under S, but not vice versa.
We assume property of function sharing. For general models f and g, deep learning prefers f over g more equally if the partition under g is refinement of that under f. Equivalently, if two inputs have an identical prediction in g. their predictions are still equal in f. means an equal prediction in g implies that in
Assumption (Function sharing). Deep learning prefers f over more or equally if
Viaa Tih E X g(xa) = g(5b) = f(xx) = f(x))
Note that it is bias in the learning process. In we need I be strictly more preferred over g. we can additionally show that identical prediction in does not imply that in
We consider what causes the function sharing. Intuitively, the preference comes from greedy opti- mization and the task requirement to learn a complicated model. A model learns to split or partition inputs by different predictions. Some splits (can be factor) might be easier to learn than others. For example, learning to split inputs by color is more straightforward than by shape. Then the greedy optimization learns quickly and reuses its intermediate functions to learn other splits. It is not likely that multiple splits (or factors) are learned equally fast during the whole training process.
The mechanism of reusing funetion is similar auxiliary tasks. The parameters are updated to address complicated main task while quickly learning and keeping the prediction ability for more straightforward auxiliary tasks. A more common but less similar example is pre-training. Pre- trained modules, such as image feature extractors word embeddings, share the internal functions with the main target tasks. However, the pre- vee-training and the main task training do not happen simultaneously. We will discuss more insights in Section 4 and Appendix
2.3 THE CONFLICT
We derive propositions for proving the theorem and explaining the reasons for the phenomena. In practice, the assumptions may not hold strictly, and the effects come more from the soft biases of the conclusions. The proofs are in Appendix
Assumption leads model to predict training outputs for any input.
3

---

Under review as a conference paper at ICLR 2023
Proposition 1 (Seen prediction). From Assumption
Vx X f(x) of fX
Informally, suppose there is a non-empty set of all inputs that a function f does not map to training outputs / miin). that case, we can design another function f' that predicts a training output for these inputs and keeps predictions for other inputs. Then both f and f" equivalently distinguish training outputs. With Assumption 2 f is preferred, hence Yz E X : f(z) E f(Xmin). It does not indicating that the phenomena may happen before a model is well trained.
apply Assumption D
If a model performs well for the training set, predicts training ground-truth output for any input.
Proposition 2 (Seen label). From Assumption and Preposition Vx E X f(x) E Yaai--
says that the prediction is not any new output. It stronger argument than avoiding partic- ular new output for each input in systematic generalization. It explains that prediction is incorrect
because any new output is resisted. We evaluate it in Section [
We then look at the conflict. For any test sample (I, y). the definition of systematic generalization requires that the output y not a training output Viain However, this contradicts Proposition
Theorem (The conflict). From Definition I and Proposition 2 ((z, y) E Pien : y + f(x).
We covered systematic generalization definition and function sharing assumption. We then derived propositions and a theorem of the conflict.
3 EXPERIMENTS
We run experiments to show that function sharing reduces the ability of systematic generalization in deep learning. We not only compare sharing or not but also adjust the degree of sharing. We focus on the cases where new outputs unseen combinations of seen factor values. We cover different standard deep neural network models. The details of networks and experiments can be found in Appendix D. More experiments with natural inputs in Appendix [ The results of zero-shot learning datasets are in Appendix [ where factors are mainly related to the input locality. We look the experiment settings and results.
3.1 SETTINGS
Data preparation We construct a dataset from two ten-class classification datasets. The training data are generated from the original training dataset. We first chose output y and chose x based on it. y/1 is chosen from all possible labels. 3/2 is chosen from five classes {y1. y1 + 1...., 8/1 + 4] (we use modular for labels). The test data are generated from the original test dataset. y1 is chosen in the same way as in training. 1/2 is chosen from the other classes {y +5, 81 + + +6..... 31 +9). this design, training and test label combinations are mutually exclusive, but test labels for each output factor are seen in training. Any factor label appears evenly in training and test combinations. æ1 and 22 are chosen conditioned on their labels /ı, V2, respectively, and merged the input The original datasets and input merge methods vary for each experiment. All the choices follow uniform distributions.
Architecture To evaluate the influence of function sharing on new outputs, we change the function sharing ability while keeping other properties stable for a deep learning model. So we modify function sharing related to new outputs. Since a new output appears only new factor combination in our setting, we adjust the sharings between output factors. It does not need to remove all sharings and avoid difficulties in model design. We choose layer and duplicate the following layers, keeping the number of all hidden nodes each layer if feasible (Figure 4). We call the former part shared network and the latter parts individual networks. Each individual network predicts one output factor, so the output is disentangled. We will discuss entangled outputs in Section 4 We keep the depth of the whole network and change the depth of the shared network and individual networks. Note that only sharing the input layer indicates learning two separate models.
We use accuracy as the metric. A sample prediction is correct if all the out- puts are correct. We have three types of accuracy. The first is the regular evaluation of test data
Evaluation metrics
4

---

Under review as a conference paper at ICLR 2023
//1
J/2
Figure 4: Deep model architecture. is input, and [(x) ÿ1 02 outputs. We duplicate the layers after layer h. The shared network is to h, and the individual networks are after (their depths are all two here). We change the shared network depth while maintaining the entire network depth so that the function sharing ability is adjusted while other network properties are stable.
for systematic generalization (a: Test Sample Accuracy) corresponding to Theorem We also consider set of inputs mapped unseen output combinations corresponding to Proposition We evaluate whether the test samples predict one of the unseen output factor combinations (b: Test Set Accuracy). However, test samples only subset of input space. If model learns different set of factors, the expected inputs may not be those of test samples. So we also evaluate systematic generalization model property for any valid input. We randomly draw inputs from the whole input space (c: Random Set Accuracy] D
We run each experiment five times and plot the mean and The result numbers are in Table (Appendix D.3).
the standard deviation (Figure (
((ra))-((..))00fff v)]
b: P( (0 (x( Yee)) E E xx(0 E YiEN)
3.2 RESULTS
Fully Connected Network We use an eight-layer fully connected neural network with a flattened image input. We use the Fashion dataset (Xiao et al. 2017) and the MNIST dataset (LeCun et al., 1998). The datasets are uncomplicated avoid the training data under-fitting for a fully connected neural network. We merge the two inputs by averaging values at each input node.
Convolutional Network We use a convolutional neural network with six convolutional layers and two fully connected layers. We use the CIFAR-10 dataset (Krizhevsky) (009) and the Fashion dataset (Xiao et al.) 2017). We scale the input sizes to that of the larger one and broadcast gray images colored ones. We merge the inputs by averaging at each node. We use the Fashion dataset as one factor because the average of two colored images can cause training data under-fitting for convolutional neural networks.
We use ResNet50 (He et al. 2016). which has five stages, each treated layer while changing the shared network depth. It has the dataset setting in the CNN experiment.
Residual Network
Vision Transformer We use Vision Transformer (Dosovitskiy et al. 202T) with one fully con- nected layer for each patch, five attention layers, and two fully connected layers. We treat the patches as one layer. It has the dataset setting in the CNN experiment.
LSTM A recurrent network has the same parameters for each layer, does not support learning different individual networks. Instead, we treat an LSTM layer. We use stacked LSTM models with an embedding layer, five bidirectional L.STM layers, and two fully connected layers. We use the Reuters dataset (Joachims) 1998) for both the first and the second datasets. We filter samples by a maximum input length of 200 and use the most frequent ten classes of samples. We merge inputs by concatenating two input texts. The inputs have different lengths because the text lengths vary.
We also run experiments for one-layer LSTM, which compares sharing or not sharing all layers. The results indicate that the shared network has less generalization than the individual network (Table
8(-) is the statement and 0 otherwise. U is the uniform distribution of valid inputs.
5

---

Under review as a conference paper at ICLR 2023
Transformer We Transformer (Vaswani al. 2017). Since it is classification problem, only use the encoder. has one embedding layer, five hidden layers, and two fully connected layers. We use the same dataset setting the LSTM experiment.
Summary results Figure 5 shows that, for each evaluation, the accuracy on the left end (not sharing any hidden layer) is higher than that the right end (sharing all hidden layers), and it generally decreases as the shared network depth increases. The results indicate that the function sharing weakens systematic generalization.
50
Test Sample Acc.
80
40
Test Set Acc.
Accuracy (%)
Rnd. Set Acc.
Accuracy (%)
60
30
20
40
10
20
1-6
4-3
5-2
6-1
0-7
1-6
2-5
3-4
5-2
6-1
7-0
Shared-Individual Network Depths
Shared-Individual Network Depths
(a) Fully connected network
(h) Convolutional network
100
80
Accuracy (%)
Accuracy (%)
60
60
40
40
20
20
8-7
0-5
144
2-3
3-2
4-1
5-0
1-6
2-5
3-4
4-3
5-2
6-1
7-0
Shared-Individual Network Depths
Shared-Individual Network Depths
(c) Residual Network
(d) Vision Transformer
40
50
Accuracy (%)
30
Accuracy (%)
40
20
20
10
10
0-7
1-6
2-5
3-4 4-3 5-2
6-1
7-0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
Shared-Individual Network Depths
Shared-Individuall Network Depths
(e) LSTM
(f) Transformer
Figure 5: Results of three types of systematic generalization accuracy with shared and individual (sub)network depths. As the shared network depth increases, the accuracy generally decreases, indicating that function sharing reduces systematic generalization ability. Test Set Accuracy and Random Set Accuracy are the ratios of unseen factor combinations in prediction. Test Sample Accuracy and Test Set Accuracy are computed with the test data, and Random Set Accuracy is with randomly generated inputs.

---

Under review as a conference paper at ICLR 2023
4 DISCUSSIONS
4.1 ENTANGLED OUTPUT
Though the theory does not require disentangled output, the experiments use split the network into individual ones. We discuss that entangled output prediction is not easier than disentangled one. Hence the experiment conclusions will likely extend to entangled outputs. (1) Disentangled output is particular and usually less complicated case of entangled output. (2) Entangled output can be seen as another shared layer, and the experiments show that increasing shared layers reduces systematic generalization ability, entangled outputs are also likely to suffer from the problem. (3) We can see each output node of an entangled output as a factor, so it becomes disentangled output. The generalization is even more difficult if value is unseen for output node.
4.2 BEYOND FACTOR RECOMBINATION
The definition of systematic generalization (Definition requires that each test label factor is seen in a training label. However, it is not directly used the derivations, and the conclusions may apply to more general o.o.d. problems beyond recombining factors. A new output may correspond to an unseen activation for an output node, e.g., a new class a classification problem. In such settings, it is sometimes discussed that the bias parameter of the output is reason to avoid the new value prediction because it does not have any training signal to increase its value. This work provides another reason for not predicting a new value.
4.3
WHY DOES FUNCTION SHARING HAPPEN IN DEEP LEARNING?
We discuss that function sharing can be caused by characteristics of deep learning: deep architec- ture, shareable network, and greedy optimization. Deep architecture and shareable networks make possible for factors share elaborated functions, and greedy optimization encourages the sharing. Deep architecture and greedy search are necessary for deep learning. Deep learning uses deep ar- chitecture to fit complicated non-linear functions. Deep learning has large and complex parameter space. To search in it, we need some prioritization, which leads to greedy search. The share- able network is widely used in standard deep learning models and works well for i..d. problems. However, it is less critical compared to the other ones.
4.4 POTENTIAL SOLUTIONS
We consider possible solutions to avoid function sharing and achieve systematic generalization. From the above discussion, we look at shareable networks. We consider the recombination of factors and focus on sharing between factors. Then, one potential solution uses individual networks for output factors, similar to the experiment setup. We discuss how to design networks when the input or the output is entangled. If the output is entangled, we can design an architecture where each individual network only changes one factor the output. For example, one individual network changes color, and another changes shape. If the input is entangled, we need to extract factors from to feed the individual networks. contains two questions: how to avoid spurious influence from other factors and keep it working in test distribution. We can bottleneck representations for the first one and divide the input into units invariant in training and test for the other, e.g., words or objects.
5 RELATED WORK
Systematic generalization and deep learning Systematic generalization Fodor & Pylyshyn
1988 Lake & Baroni 2018 Bahdanau al. 2019) is considered the "Great Move" of evolu-
tion, caused by the need to process an increasing amount and diversity of environmental informa- tion (Newell 1990). Cognitive scientists see as central for an organism to view the world (Gallistel
& King] 2011). Studies indicate it is related to the prefrontal cortex (Robin & Holyoak 1995). It
was discussed that commonsense is critical (Mcearthy 1959 Lenat et al. 1986) for systematic gen- eralization, and recent works aim to find general prior knowledge (Goyal & Bengio
2020). e.g.,
It is also called compositional generalization in other literature.

---

Under review as a conference paper at ICLR 2023
Niklas- son & van Gelder, 1994). and types of tests were summarized (Hupkes et al. 2020 We focus on the primary case with unseen combination of seen factor values.
Consciousness Prior (Bengio 2017).
Levels of systematicity were defined (Hadley
1992
A closely related field is causal learning, rooted in the hughteenth-cettueg (Hume 2003 and classical
fields of AI (Pearl ).03). It was mainly explored from statistical perspectives (Pearl 2009 Peters
jtt al.
2016; Green et al.
1999; Pearl 2018) with do-calculus (Pearl 1995: 2009) and interven-
tions [Peters et al., 2016). The causation forms Independent Causal Mechanisms (ICMs) (Peters
et al. 2017 Schölkopp et al. 202T). Systematic generalization is the counterfactual when the joint
input distribution intervened to have new values with zero probability training (covariate shift).
This work indicates that standard neural networks do not prefer to learn ICMs.
Parallel Distributed Processing (PDP) models (Rumelhart al., 1986) use Connectionist models with distributed representations, which describe an object in terms of set of factors. Though they have the potential to combine the factors to create unseen object representations (Hinton 1990), it was criticized that they do not address systematic generalization in general (Fodor & Pylyshyn 1988 Marcus| 1998). Deep learning is a recent PDP model with many achievements (LeCun et al., 2015 He et al.. 2016). It was studied that deep neural networks use the composition of functions to achieve high performance (Montufar et al., 2014). The improvements in i.idd. problems encourage to equip deep learning with systematic generalization.
Recent directions In addition to architecture design (Russin et al. 2019. Andreas et al. 2016) and data augmentation (Andreas) 2020; Akyürek et al.) [021]Jia Jia & Liang] 2016), the main perspec- tives for systematic generalization approaches include disentangled representation learning, atten- tion mechanism, and meta- letanle...nng.
Disentangled representation (Bengio et al. 2013) is learned in unsupervised manners. Early meth- ods learn the representation from statistical independence (Higgins et al., [017]] [ocatello et al. 2019). Later, the definition of disentangled representation was proposed with symmetry transfor- mation (Higgins et al.] 2018)) It leads to Symmetry-based Disentangled Representation Learn- ing (Caselles- Dupré et al. 2019 Painter et al. 2020 Pfau et al. 2020). A disentangled repre- sentation learning model can be used as a feature extractor for other systematic generalization tasks.
Attention mechanisms are widely used in neural networks (Bahdanau et al. 2015). Transform- ers /Vaswani et al., (017) are modern neural network architectures with self celf-attention. Recurrent Independent Mechanisms (Goyal et al. (2021b) use attention and the name of the incoming nodes for variable binding. Global workspace (Goyal et al. 2021a) improves them by using limited-capacio global communication to enable the exchangeability of knowledge. Diserete-valle valued communication bottleneck (Liu et al. 202T) further enhances systematic generalization ability.
Meta-learning Lake 2019) usually designs series of training tasks for learning meta-learner and uses it in target task. Each task has training and test data, where test data requires systematic gen- cralization from training data. When ICMs are available, they can be used to generate meta-learning tasks (Schölkopf et al. 2021). Meta-reinforcement learning was used for causal reasoning (Dasgupta et al. 2019) Meta- can also capture the adaptation speed to discover causal relations (Ben- gio et al.) 2020
Ke et al., 2019).
Deep learning is frsw-ggowing field, and many efforts focus on designing architectures and algo- rithms to improve its performance. However, it is less discussed why standard deep learning models do not achieve systematic generalization. This paper looks into a built-in conflict.
CONCLUSION
This paper investigates a built-in conflict between deep learning and systematic generalization. explains one of the reasons why standard neural networks seldom achieve systematic generalization. We hypothesize that the conflict caused by sharing internal functions, and experiments support it. A model partitions an input space into multiple parts separated by boundaries. The function sharing tends to reuse the boundaries, leading to fewer parts for new outputs, which conflicts with systematic generalization. The phenomena are shown in different standard deep neural networks. We hope this finding provides a new understanding of systematic generalization mechanisms in deep learning and helps to improve machine learning algorithms for higher level of artificial intelligence.
8

---

Under review as a conference paper at ICLR 2023
REFERENCES
Ekin Akyürek, Afra Feyza Akyürek, and Jacob Andreas. Learning to recombine and resample data for pennotiitioaallennnaaiiiaai In International Conference on Learning Representations, 2021.
IMMnSsugg.
Jacob Andreas. Good-enough compositional data augmentation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7556-7566, Online, July 2020. Association for Computational Linguistics. doi: 10. 1/211/////2222.a////ll (al22000.ccllaaiii acl-mamm.mmmmm67 URL
https:
//www. www.wb.ww.bb......... orq http://aaatoooooy///00...eeee acl 676
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 39
39-49,94 48, 2016.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning align and translate. In ICLR 2015: International Conference Learning Represen- tations 2015, 2015.
Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. Systematic generalization: What is required and can be learned? In International Conference Learning Representations, 2019. URL / openreview..eeeeeeeeee
net fort//foum? 2 wid=ikeeekAke9
Yoshua Bengio. The consciousness prior. arXiv preprini arXiv: 2017.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on paltern analysis and machine intelligence, 1788)11798-1828, 2013.
Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Nan Rosemary Ke, Sebastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, and Christopher Pal. A meta-transfer objective for learning to disentan- gle causal mechanisms. International Conference on Learning Representations, 2020. URL
https:://pperevaa
Hugo Duselles-Dupré, Michael Garcia Ortiz, and David Filliat. Symmetry-based dis- entangled representation learning requires interaction with environments. In H. Wal- lach, H. Larochelle, A. Beygelzimer, d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL btttps://proceeddngg www.eediiinnnneeesisseeeee. neurips. cc//papprr/219//i1leee 36e
Paper pdff
Ishita Dasgupta, Jane Wang, Silvia Chiappa, Jovana Mitrovic, Pedro Ortega, David Raposo, Edward Hughes, Peter Battaglia, Matthew Botvinick, and Zeb Kurth-Nelson. Causal reasoning from meta-reinforceemnnt learning. arXiv preprint arXiv: 990100862, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogni- tion at scale. In International Conference on Learning Representations, 2021. URL https: //openrevien ew. net./ forum? d=Y cbFdNTTy
Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing objects by their attributes. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pp. 1778-1785, 2009.
Jerome A. Feldman and Dana H. Ballard. Connectionist models and their properties. Cognitive Science, 6(3):205-2544 1982.
Jerry A Fodor and Zenon W Pylyshyn. Connectionism and cognitive architecture: critical analy- sis. Cognition, 28(1-2):3-71, 1988.
Charles Gallistel and Adam Philip King. Memory and the computational brain: Why cognitive science will transform neuroscience. John Wiley & Sons, 2011.

---

Under review as a conference paper at ICLR 2023
Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level cognition. arXiv preprint arXiv:2011.1509) 2020.
Anirudh Goyal, Aniket Didolkar, Alex Lamb, Kartikeya Badola, Nan Rosemary Ke, Nasim Ra- haman, Jonathan Binas, Charles Blundell, Michael Mozer, and Yoshua Bengio. Coordination among neural modules through a shared global workspace, 2021a.
Anirudh Goyal, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Ben- gio, and Bernhard Schölkopf. Recurrent independent mechanisms. In International Confer- ence on Learning Representations, 2021b. URL http::://ppeeeeeeeew.eeette wtwprieewneefoorureee [nIcmdEBUxy-
Sander Greenland, Judea Pearl, James M Robins, et Causal diagrams for epidemiologic research. Epidemiology, 10:37-48, 1999.
R. F. Hadley. Compositionality and systematicity in connectionist language learning. In Conference of the Cognitive Science Society, 1992.
K. He, X. Zhang, Ren, and J. Sun. Deep residual learning for image recognition. CVPR, 2016.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 3-vae: Learning basic visual concepts with a con- strained variational framework. In International Conference on Learning Representations (ICLR), 2017.
Irina Higgins, David Amos, David Pfau, Sébastien Racanière, Loic Matthey, Danilo J. Rezende, and Alexander Lerchner. Towards a definition of disentangled representations. CoRR, Abs/1812.0222 2018.
02230
Geoffrey E. Hinton. Mapping part-whole hierarchies into connectionist networks. Artificial Intelli gence, 46(1):47-75, 1990.
David Hume. A treatise of human nature. Courier Corporation, 2003.
Dieuwke Hupkes, Verna Dankers. Mathijs Mul, and Elia Bruni. Compositionality decomposed: How do neural networks generalise? (extended abstract). In Christian Bessiere (ed.). Proceedings
of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCA/-20, pp. 5065-
5069. International Joint Conferences on Artificial Intelligence Organization, 2020. doi:
(1963//jcai.222//708 URL httpps://dii.. http::///oooooo//////// 249 04563//jcaa..22222778 Journal track.
2020/708
Robin Jia and Percy Liang. Data recombination for neural semantic parsing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 12-22, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.
18653/v1/P16- 1002 htR.:ttp:: a anthol hatp::/::///caaahhclolloo/o/ www.t/hoooo....r//1111-11111
Thorsten Joachims. Text categorization with support vector machines: Learning with many relevant features. In Proceedings of the 10th European Conference on Machine Learning, ECML'98, pp. 137-142, Berlin, Heidelberg, 1998. Springer ISBN 3540644172. doi: 10.1007/BF60026663.
Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Bernhard Schölkopf, Michael C Mozer, Chris Pal, and Yoshua Bengio. Learning neural causal models from unknown interventions. arXiv preprint arXiv: 1910.01055 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sequennce-o--seqeenc recurrent networks. In International Conference Machine Learning, pp. 2873-2882, 2018.
Brenden M Lake. Compositional generalization through meta sequence-to-seqqeece to-sequence learning. In Advances in Neural Information Processing Systems, pp. 9788 198889999,, 2019.
10

---

Under review as a conference paper at ICLR 2023
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):222 22222224,,,, 1998.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521((553):436, 2015.
DB Lenat, M. Prakash, and M. Shepherd. Cyc:
Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks. Ai Magazine, 6(4):65-85, 1986.
Dianbo Liu, Alex M Lamb, Kenji Kawaguchi, Anirudh Goyal ALIAS PARTH GOYAL, Chen Sun, Michael C Mozer, and Yoshua Bengio. Discrete-valued neural communication. In M. Ranzato, A. Beygelzimer. Y. Dauphin, P.S. Liang and Wortman Vaughan (eds.), Ad- vances in Neural Information Processing Systems, volume 34, pp. 2109-2121. Curran Asso- ciates, Inc., 2021. URL https: https::/prroceeiineee new..nnuurii ips occ/paper/222//E1le .ddf
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference machine learning, pp. 4114-4124. PMLR, 2019.
Gary Marcus. Rethinking eliminative connectionism. Cognitive psychology, 37(3):243-2333 1998.
J. Mccarthy. Program with common sense. Teddington Conference on the Mechanization of Thought Processes, 1959.
Guido F Montufar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. On the number of linear regions deep neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 27, Cur- ran Associates, Inc., 2014. URL https: :ttttts:/pproeeeee h/tpocee///oooccingg.....
#1l1//00020500555000111001111112 hewp:eceeeddiinnnnii.sr.... nwwrips. cttpp///p..c/ppppe//2.//////
pdf
Allen Newell. Unified Theories of Cognition. Harvard University Press, USA, 1990. ISBN 0674920996.
Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNLP),, pp. 188-197, Hong Kong, China, November 2019.
Association for Computational Linguistics. doi: 1118653/11/D19-100 URL https://
aclanthology 001.://-10//D11-111110000
Lars Niklasson and Tim van Gelder. Can connectionist models exhibit non-classical structure sen- sitivity? In PROCEEDINGS OF THE SIXTEENTH ANNUAL CONFERENCE OF THE COGNI- TIVE SCIENCE SOCIETY. ERLBAUM, pp. 664-669, 1994.
Matthew Painter, Adam Prugel-Bennett, and Jonathon Hare. Linear disentangled representations and unsupervised action estimation. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Bal- can, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 13297-13307. Curran Associates, Inc., 2020. URL BtL. neurips............i........
Genevieve Patterson and James Hays. Sun attribute database: Discovering, annotating, and recog- nizing scene attributes. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2751-2758, 2012.
Judea Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669-688 1995.
Judea Pearl. CAUSALITY: MODELS, REASONING, AND INFERENCE. Cambridge university press, 2003.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl. Does obesity shorten life? or is it the soda? on non-manipulable causes. Journal of Causal Inference, 6(2), 2018
11

---

Under review as a conference paper at ICLR 2023
Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference using invariant predic- tion: identification and confidence intervals. Journal of the Royal Statistical Society. Series (Statistical Methodology), pp. 947-1012, 2016.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of Causal Inference: Founda- lions and Learning Algorithms. MIT Press, 2017.
David Pfau, Irina Higgins, Alex Botev, and Sébastien Racanière. Disentangling by subspace diffusion. In Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 17403-17415. Curran As- sociates, Inc., 2020. URL http:ttttss//ptoeet/teeeeooooete neurips wwtpapper/222//ile..... pdf
N. Robin and K. J. Holyoak. Relational complexity and the functions of the prefrontal cortex. Cognitive Neurosciences, 1995.
David E. Rumelhart, James L. McClelland, and Au. Parallel distributed processing: Explorations in the microstructure of cognition: Foundations, 1986.
Jake Russin, Jason Jo, Randall C O'Reilly, and Yoshua Bengio. Compositional generalization in deep seq2seq model by separating syntax and semantics, arXiv preprint arXiv: (0t.:::199409900000 2019.
Bernhard Schölkopf. Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 10965):612-634, 2021.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 1955911999- 2014. URL http: http:///mmmmr.rrr///e/ wtt..//..//////////....
Bri Privaataaval 1 14a.ht.
Tristan Sylvain, Linda Petrini, and Devon Hjelm. Locality and compositionality in zero-shot learning. In International Conference on Learning Representations, 2020. URL https:
//openrevi wwt./foruue d=Hye_VONKwr
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems, pp. 5998-6008, 2017.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset, 2011.
Yongqin Xian, Christoph H. Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning-a com- prehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4 (1))222551222 222-2225, 2019.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mmin a novel image dataset for benchmark- ing machine learning algorithms. In arXiv: 7000.07777 2017.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond em- pirical risk minimization. In International Conference on Learning Representations, 2018. URL
.nt/ wttp://wwww.eeetoouuumeeeee peet/forumiiddr1Ddy
Rb
Xingxuan Zhang, Yue He, Renzhe Xu, Han Yu, Zheyan Shen, and Peng Cui. better benchmarking for domain generalization, 2022.
Nico+:: Towards
12

---

Under review as a conference paper at ICLR 2023
A MORE EXPERIMENTS
We run experiments with natural inputs. For image data, we use NICO+ dataset (Zhang et al, 2022). We use five foregrounds the first output label and five backgrounds as the second. For text data, we use Amazon reviews (Ni et al,, 2019). We use five categories as the first output label and five ratings the second. Either dataset has two outputs, each containing five possible classes. There are 25 class combinations, and we separate them into 15 training combinations and 10 test ones in a similar way as the experiment section. For fully connected networks, we use the Fashion dataset (ten classes) and render with ten colors. Please refer to Figure and Table [ for examples.
For the image dataset, we aggregated foregrounds into five abstract classes, e.g., mammal and ve- hicle. It better uses the limited data with annotations on combined labels. We use 72,176 image samples. For text data, we randomly select 100,000 samples for each category. with length limit of 100 tokens. Data with training combinations are randomly split into training and i.i.d. generalization data with a ratio 9:1.
The results are in Figure 7 Similar to the results in the experiment section, the test accuracies de- crease as there are more shared layers. Also, both the training accuracy and the i.i.d. generalization accuracy do not decrease much as the test accuracies.
We run ablations. Figure S shows the results with different layer widths. Figure D shows the results with dropout (Srivastava al. 2014) and mixup (Zhang et al. 2018). All the results have similar effects as the original experiment.
(a) cat, grass
(b) cat, rock
(c) bus, grass
(d) bus, rock
Figure Examples of image data.
Category
Rating
Review
Book
5
quick read from the most excellent author. fun It had tears, some label covering defect and also wrinkled pages.
Book
1
Electronics
5
Works perfectly, better than the original cable.
Electronics
DID NOT fit as described to accommodate the TV size!
Table Examples of text data.
B PROOFS
Proposition 1
(Seen prediction). From Assumption Vo X f(x)= f(Xmain)
Proof. We are going prove that for function g with at least one input mapped to new output, there exists a more preferred function I with all inputs mapped to seen outputs. Therefore, if function does not have other functions more preferred over it, the function follows the proposition.
Given a function g, we construct a function f. We pick a E Xuain- Vo e X:
g(x) E g(Xuain)
f(x) g(x),
O.W. f(x) f(x').
Then, Vsa: Tb E X: ((x.)) E ((xrain):
g(xaa) g(xb) f(x.) = g(Fa) = g(xb) f(xb)
O.W.: g(xa) g(xb) ((xb) g(Xunin)) f(x.) f(x') f(x))
13

---

Under review as a conference paper at ICLR 2023
100
80
Accuracy (%)
Accuracy (%)
80
60
Sample Acc.
LJD. Sample Acc.
60
Test Sample Acc.
40
Test Set Acc.
Rnd.
40
20
20
0
0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
0-7
1-6
2-5
34
4-3
5-2
6-1
7-0
Chared-Individual Network Depths
Shared Individual Network Depths
(a) Fully connected network
(h) Convolutional network
100
100
80
80
Accuracy (%)
Accuracy (%)
60
60
40
40
20
20
0
0-5
1-4
2-3
3-2
4-1
5-0
0-7
1-6
2-5
34
4-3
5-2
6-1
7-0
Shared-Individual Network Depths
Shared-Individual Network Depths
(c) Residual Network
(d) Vision Transformer
80
80
Accuracy (%)
60
Accuracy (%)
60
40
40
20
20
0
0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
Shared-Indivion Network Depths
Shared-Individee Network Depths
(c) LSTM
(f) Transformer
Figure 7: Results of three types of systematic generalization accuracy with shared and individual nsub)network depths. Original images from the NICO dataset and texts from Amazon reviews used. Similar to Fignure as the shared network depth increases, the accuracy generally decreases, indicating that function sharing reduces systematic generalization ability. Both the training and the i.i.d. generalization accuracy do not decrease as much as test accuracies.
In both cases,
g(x.a) = g(xb) = f(xa) f(x))
On the other hand,
3r E Xust g(x) & g(Xuain) = g(x) F g(2') E g(Xuaan). f(x) = f(x').
So (xx = f(x') does not imply g(x) g(x'). With Assumption 2 f preferred over g.
Also, Vx E X:
g(x) g(Xinnn): Br" E X g(x)= g(x") f(x) f(x") s(xmin)
o.w. f(x) f(x') f(Xrran)
14

---

Under review as a conference paper at ICLR 2023
80
80
Accuracy (%)
Accuracy (%)
60
Train Sample Ace.
60
Sample
Sumple
40
Test Sent Ant.
40
Rnd. Swt Acc.
20
20
0
0-7
1-6
2-5
3-4 4-3 5-2
6-1
7-0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
-hared-Individual Network Depths
Shared-Individual Network Depths
(a) 64
(b) 128
100
80
80
Accuracy (%)
Accuracy (%)
60
40
20
20
0
0-7
1-6
2-5
3-4
5-2 6-1
1-6
2-5
3-4
4-3
5-2 6-1 7-0
Shared-Individual Network Depths
Shared-Individual Network Depths
(c) 512
(d) 1024
Figure 8: Layer width ablation results of three types of systematic generalization accuracy with shared and individual (sub)network depths. Similar to Fignure 77p (width 256), as the shared network depth increases, the accuracy generally decreases, indicating that function sharing reduces systematic generalization ability. Both the training and the i.i.d. generalization accuracy do not decrease much as test accuracies.
80
80
Accuracy (%))
Accuracy
60
60
40
40
20
20
0
0-7
1-6
2-5
34
4-3
5-2
6-1 7-0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
Shared-Individunn Network Depths
Shared-Individual Network Depths
(a) Dropout
(b) Mixup
Figure 9: Dropout and mixup ablation results of three types of systematic generalization accuracy with shared and individual (sub)network depths. Similar to Fignure 77,, as the shared network depth increases, the accuracy generally decreases, indicating that function sharing reduces systematic gen- cralization ability. Both the training and the i.i.d. generalization accuracy do not decrease as much test accuracies.
Therefore, Vr E X f(x) E f(Nuran)
L
Proposition 2 (Seen label). From Assumption I and Proposition
Vr X
f(x) Ymain-
15

---

Under review as a conference paper at ICLR 2023
Proof. From Assumption 1 (Xurrn)) Yuain From Proposition Ye X f(x) E (XXnai) Yirain
Theorem (The conflict). From Definition 1 and Proposition 2 (a, Dien y + f(x).
Proof. (x, y) E Diesi g Yuain (Definition D Xies X = f(x) E Yuain (Proposition) 2 Therefore y # f(x).
C A CONJECTURE ON WHY THE FUNCTION SHARING HAPPENS
The neural network training process is complicated, it is hard to describe what happens. Instead, we have conjecture.
For large network, when a boundary is learned for the first time, it separates the problem into two sub-problems, and their learning processes do not influence each other for the rest of the training.
We first explain the idea with an analogy of binary decision tree (or hierarchical classification). We then define a boundary and discuss its properties.
Binary Decision tree We consider such binary decision tree that at each decision node, label set is separated into two parts, each for sub-tree. For example, if there are 10 classes, the root node may divide them into 6 and classes. Then the root node of the first sub-tree divides the lasses into two sets of 3 classes. In this decision tree, node separates input space into two parts with disjoint output labels, and each part learned separately.
Such decision tree does not predict new outputs because all leaf nodes predict seen outputs. We discuss neural network training process similar to creating such decision tree from some aspects. A decision tree node is boundary in a neural network.
We consider a problem P (X, with input space X, input set X, output set Y. We have ground truth mapping f X + Y and learned mapping f : Y. We define a boundary as follows.
Boundary
Definition 2 (Boundary). Suppose we have binary partition (Na: of an input space X. Na. No. are non-empty.
X, = X n Xa, Ya J(Xa) Y, = ((x)). It is a boundary if the output sets are disjoint; for each part, all inputs map to its oulput set.
X, X,
Y,UY= f(Xx) = Ya. f(x))= = =-
A boundary separates a problem P = X,Y to two sub-problems P. = (Na, aa Ya) and P. (No, Xx, Yo). We assume that when the boundary learned for the first time, the learning processes of Pa and Ps, do not influence each other for the rest of the training.
Assumption (Separate sub-problems). When a network large enough, learning one sub-problem does not affect the prediction of another sub nub-problem.
With this assumption, for large network, a boundary separates the original problem into two prob- lems whose training processes do not influence each other. The assumption applies each sub- problem. When a problem has only one label, all the inputs are mapped to the label. So the model learned not predict an unseen output. This learning process is similar to that of decision tree.
D EXPERIMENT DETAILS
D.1 VISUALIZATION SETTINGS
The model is a fully connected neural network with two input and two output nodes. It has six hidden layers with ReLU activations, and each hidden layer has eight nodes. We use mini-batch
16

---

Under review as a conference paper at ICLR 2023
size of 10 with a learning rate of 0.01. We iterate until the model prediction becomes stable. Please see the original work of deep playground for more information. We use six Intel()) Core(TM) i5- 15-8400 2.80GHz CPUs, and the asset has public license.
D.2 EXPERIMENT SETTINGS
We use GeForce GTX 1080 or GeForce GTX 1050 Ti GPU for single GPU experiments. We use TensorFlow for implementation. The assets have public license.
Each input element is linearly scaled to [-0.5, 0.5] for image input. We also uniformly sample from this interval for random image input. We select two sentence lengths uniformly from valid integers (one to maximum length) and then generate each word uniformly from the vocabulary for random text input.
Fully Connected Network The input shape is x 28, flattened to vector. There are seven fully connected layers. Each of them has 512 hidden nodes and ReLU activation. The output has ten nodes and Softmax activation. We use cross-entropy loss and Adam optimizer with learning rate of 0.001. The batch size is 512, and we train 2,000 iterations. Each evaluation uses 10,000 samples.
Convolutional Network The input shape is 32 32 x 3. There seven convolutional layers. Each of them has 3 x 3 kernel size with 64 channels. Then the layer is flattened. We have a fully connected layer with 128 nodes and ReI.U activation. The output layer has nodes with Softmax activation. We use cross-entropy loss and Adam optimizer with a learning rate of 0.001. The hatch size is 512, and we train 5,000 iterations. Each evaluation uses 10,000 samples.
Residual Network The input is the same CNN. The model is the standard ResNet50 implemen- tation. The hidden groups are treated as one layer, so there are five hidden layers. The hidden layer size 64. The output layer has ten nodes with Softmax activation. We use cross-entropy loss and Adam optimizer with learning rate of 0,001. The batch size is 512, and we train 10,000 iterations. Each evaluation uses 10,000 samples.
Vision Transformer The input the same CNN. The model the standard Vision Transformer implementation with seven hidden layers. The hidden layer size is 64. The output layer has ten nodes with Softmax activation. We use cross-entropy loss and Adam optimizer with learning rate of 0.001. The batch size is 512, and we train 10,000 iterations. Each evaluation uses 10,000 samples.
LSTM The vocabulary size is 30,977, including start symbol and padding symbol. The input length 200. The embedding size is 64. There are seven stacked bidirectional LSTM layers, and each has 32 hidden nodes for each direction. Then the output is flattened. The output layer is fully-connected layer with ten output nodes and Softmax activation. We use cross-entropy loss and Adam optimizer with a learning rate of 0.001. The batch size is 64, and train 1,000 iterations. Each evaluation uses 10,000 samples.
Transformer The input is the same as that of LSTM. The embedding size is 64. There are seven hidden groups. The hidden layer size is 64. The output is flattened. The output layer fully- connected layer with ten output nodes and Softmax activation. We use cross-entropy loss and Adam optimizer with learning rate of 0.001. The batch size is 64, and we train 2,000 iterations. Each evaluation uses 10,000 samples.
D.3 EXPERIMENT RESULTS
We numerically compare the individual (left ends in result figures, 0-max) and shared (right ends, max-0) networks in Table LSTM is the stacked LSTM, and LSTM-1 has only one LSTM layer. It shows that shared network has lower scores than individual network on the three types of accuracy. So it indicates that function sharing avoids systematic generalization.
17

---

Under review as a conference paper at ICLR 2023
Table 2: Accuracy (mean std %). Experiment results.
Test Sample Accuracy Shared
Test Set Accuracy
Random Set Accuracy
Individual
Individual
Shared
Individual
Shared
DNN
41.6 1.0
9.2 - 0.7
47.1 = 1.4
10.2 + 0.6
36,9 + 0.8
7.1 + 0.6
CNN
41.6
21.3 1: 0.8
61.3 +: 1.0
30.6 1 1.0
81.4 4.0
41.4 1 13.7
ResNet
8.3 + 1.6
59.4 + 1.4
11.8 199
74.1 1
17.9 1 8.5
VIT
29.3 0.9
3.8 0.4
40.2 0.9
5.6
79.2 3.0
27.9 + 6.1
LSTM
26.0 1.5
2.2
41.1 1.2 1.2
29.2
16.0
5.0 + 2.7
LSTM-1
28.6 2.2
26.0 + 1.9
40.1 2.0
34.8 2.1
15.1
8.0 + 2.1
Transformer
28.4 2.5
22.1 + 2.1
48.6
33,9
45.1
16.4 + 8.4
E ZERO-SHOT LEARNING DATASETS
We look at the results for Zero-shot learning. We use aPY (Farhadi et al. 2009), AwA2 (Xian et al., 2019), CUB (Wah et al., 2011), and SUN (Patterson & Hays] (202)) datasets. In these datasets, factors are mainly related to the input locality (Sylvain et al. (020). We use the pre-extracted input features for aPY and AwA and image input for CUB and SUN. We construct output labels from attributes. Each attribute is binary number, 1 for existing in sample and 0 otherwise. We select six attributes with the most balanced average number (close to 0.5) all data. The first, third, and fifth attributes
used for the first output, and the others for the second output. Each output has eight classes yielded from all the combinations of three binary attributes. For aPY, samples may share the same image, so we construct training data from the original training and test data from the original test data. For other datasets, we split all data to disjoint training and test data.
We use eight-layer CNN model, the for aPY and AwA and 256 for CUB and SUN. Other settings are the same as that in the experiment section. The result is shown in Figure TO and Table Similar to the previous experiments, the depths of shared and individual networks reduce systematic generalization capability.
as in the experiment section. The batch size is 512
Table 3: Accuracy (mean + std %). Experiment results for Zero-Shot Learning.
Test Sample Accuracy Individual
Test Set Accuracy
Random Set Accuracy
Shared
Individual
Shared
Individual
Shared
aPY AwA2
2.5 0.3
0.6 0.2
22.4
5.9 1.2
36.6 E 30.6
6.8 + 2.8
0.6 + 0.1
3.0
9.9
45.2 + 3.8
18.5 + 0.3
CUB
2.5 2.5+22
1.2 0.2
39.7
22.2
38.8 + 2.6
21.9 + 3.7
SUN
1.9 0.1
1.1 0.3
15.3 + 1.0
8.5
35.9 + 11.7
8.3 + 4.2
F
MORE DISCUSSIONS
F.1 TRAINING PROCESS
We discussed that sharing boundaries reduces the number of partitions and shrinks the area for new outputs (Proposition [. We run experiments to find when this happens during training, We sample 10,000 inputs from data, and if an output combination has at least 50 samples, we regard it as a new output (o.o.d.) partition. We plot the number of o.o.d. partitions and the test sample ratio in the o.o.d. partitions for shared and individual networks in Figure [T] The experiment settings follow DNN and CNN settings in the experiment section. The results show that the differences start to happen in early training.
F.2 EQUALLY DIFFICULT FACTORS
We look at the results when two inputs are equally hard to learn. We use two CIFAR-10 datasets for both the first and the second datasets. Since the average of two colored images can cause training data under-fitting, fitting, six. The result is shown in Figure [2] and Table 4 Similar to the previous experiments, when the difficulties equal, the depth of the shared network weakens systematic generalization.
merge the inputs by concatenating them by channel. It means the input channel
18

---

Under review as a conference paper at ICLR 2023
50
Test Sample Acc.
60
Test Set Acc.
40
Accuracy (%)
Rnd. Set Acc.
Accuracy (%)
30
20
20
10
0-7
1-6
2-5
344
4-3
5-2
6-1
7-0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
Shared-Individual Network Depths
Shared-Individual Network Depths
(a) aPY
(b) AwA2
40
40
Accuracy (%)
Accuracy (%)
30
30
20
10
10
0-7
1-6
2-5
3-4
4-3
5-2
7-0
0-7
1-6
2-5
3-4
4-3
6-1
7-0
Shared-Individual Network Depths
Shared-Indiviidull Network Depths
(c) CUB
(d) SUN
Figure 10: Experiment results for Zero-Shot Learning with CNN models. The results similar those in the experiment section.
Table 4: Accuracy (mean + std %). Experiment results for equally difficult factors.
Test Sample Accuracy Individual
Test Set Accuracy Individual
Random Set Accuracy
Shared
Shared
Individual
Shared
DNN
24.6 - 0.9 52.5 + 1.5
4.4 + 0.5 39.6 1.2
33.3 + 1.3
6.4 + 0.6
31.2 2.5
5.4 1: 0.6 20.5 4.7
CNN
60.3 1.6
45.2 4 1.5
32.4 4.2
F.3 LABEL COMBINATIONS
We also test other training label distribution types. We design tile and one-shot combinations. In tile, a label combination is for training when Y1 < or Y2 5. similar to the split for illustrative example. In One-shot, label combination is for training when Yı < 9 Y2 < 1. In such case, only (9, 0) contains Y2 = 0. It is similar to one-shot learning. The results for the fully connected neural network in Figure 13 and Table 5 It similar the results in the experiment section.
Table 5: Accuracy (mean std %). Experiment results for other label combinations.
Test Sample Accuracy Individual
Test Set Accuracy Individual
Random Set Accuracy
Shared
Shared
Individual
Shared
Tile
33.0 3.9
8.3 1.1
36.5 4.6
9.2 + 1.1
9.7 + 2.5 1.5 0.1
One-shot
20.9 + 13.3
1.4 + 0.8
23.1 + 14.9
1.5 0.8
0.0 + 0.0
0.0 + 0.0
19

---

Under review as a conference paper at ICLR 2023
40
40
Partitions
30
Ratio (%)
Individual
20
Shared
20
0
0
500
1000 Steps
1500
2000
500
1000
1500
2000
Steps
(a) DNN o.o.d. partition number
(b) DNN o.o.d. test sample ratio
50
60
40
Partitions
30
Ratio (%)
40
20
10
0
1200
2400
3600
4800
0
1200
2400
3600
4800
Steps
Steps
(c) CNN o.o.d. partition number
(d) CNN o.o.d. test sample ratio
Figure 11: Training process. The o.o.d. partition number and the o.o.d. test sample ratio for shared and individual networks.
Test Sample Acc.
60
30
Test Set Acc.
Accuracy (%)
Rnd. Set Acc.
Accuracy (%)
50
20
40
30
10
0-7
1-6
2-5
3-4
43
5-2
6-1
7-0
0-7
1-6
2-
3-4
4-3
5-2
6-1
7-0
Shared-Individuu Network Depths
Shared-Individdal Layer Depths
(a) DNN
(b) CNN
Figure 12: Experiment results for equally difficult factors. The results similar those in the experiment section.
20

---

Under review as a conference paper at ICLR 2023
40
Test Sample Acc.
Test Set Acc.
30
Accuracy (%)
30
Rnd. Set Acc.
Accuracy (%)
20
20
10
10
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
0-7
1-6
2-5
3-4
4-3
5-2
6-1
7-0
Network Depths
Network Depths
(a) Tile
(b) One-shot
Figure 13: Experiment results for other label combinations. The results are similar to those in the experiment section.
21


Reviews:
[{"review_id": "o7YUn2fisf", "reviewer": "Reviewer_hir1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates systematic generalization of multi-label classifications in settings where the input space is shared between the different labels. The authors suggest that deep learning models are biased towards feature reuse, which conflicts with systematic generalization to new combinations of known classes. They explore this hypothesis with theoretical analyses under toy assumptions, and empirical experiments across a variety of architectures. Generally, they find that more sharing of features reduces generalization accuracy in their settings.", "summary_of_the_review": "See comment above for my post-response update. I am updating my score in accordance with the improvement in the paper, but I am concerned that key issues still remain unresolved, and not discussed with enough nuance.\n\nOriginal review\n-------------------\n\nIf this paper were completely rewritten—to describe the experiments as identifying a particular class of problems in which DL architectures seem not to generalize systematically due to feature-sharing rather than a built-in conflict—I believe it could be an acceptable paper in some venue. If, in addition to that, the authors were to identify real-world, non-adversarial datasets where their observations bear out, and perform some of the architecture/training experiments suggested above, I would consider it a strong paper for NeurIPS. As it is, I think it is misleading.", "strength_and_weaknesses": "Strengths:\n* The questions posed are interesting.\n* I appreciate the breadth of architectures considered, especially sharing different numbers of layers.\n\n\nWeaknesses:\n\nThe way the paper is presented fundamentally ignores the No Free Lunch theorem [e.g. Adam, 2019]. There is no system that can generalize perfectly on every task and training dataset—there cannot be a conflict between an architecture class and systematic generalization writ large. We have to ask the question of how the inductive biases of the model class fit the class of tasks we are interested in solving.\n* DL researchers are well aware of the feature sharing bias—it forms the basis of auxiliary task training and/or pretraining methods, as the authors note. The reason such methods tend to improve generalization is because sharing features is useful on real-world datasets. For example, the input features learned solve masked language modeling tasks empirically improve systematic generalization performance substantially even on tasks like SCAN and CFQ [Furrer et al., 2020]. There are even theoretical accounts of why feature sharing can improve generalization in the presence of noise [e.g. Lampinen et al, 2019]. \n* The datasets used in the paper are therefore cleverly created to make feature sharing an actively harmful strategy. But to do so, the authors rely on essentially adversarial dataset design, where they combine input stimuli in very unnatural ways (averaging images, or concatenating completely unrelated pieces of text), and then enforce extremely strong correlations between these inputs at train time, which are completely reversed at test time. There is no reason given to think that this process has anything to do with any real-world data generating process.\n* Therefore, I would challenge the authors to demonstrate *real-world tasks and datasets*, not artificially, adversarially created ones, in which their observations apply.\n* Otherwise, it seems to me that feature sharing is a *feature, not a bug* of deep learning. Nobody has ever claimed that deep learning is capable of generalizing systematically in every task anyone can come up with—that would violate the NFL theorem. But I’d argue that the DL family is empirically the most successful system for generalizing on real world datasets, in part because of feature sharing.\n\nArchitectures and training paradigms:\n* “We choose a layer and duplicate the following layers, keeping the number of all hidden nodes in each layer if feasible” — it is not clear to me whether this means that each “branch” of the architecture has the same number of nodes as before, or half the number of nodes. If the former, the number of parameters will be larger in networks that split earlier, thus confounding the comparison (since overparameterized models tend to generalize better). \n* More generally, it would be interesting to see the impact of parameterization on these effects—one might expect somewhat less feature sharing in wider networks, for instance, though it’s unclear how strong the effect would be.\n* And it would be interesting to see the effect of methods like dropout [Srivastava et al., 2014] or mixup [Zhang et al., 2017] which are known to improve generalization.\n\n\nReferences\n------------\n\nAdam, Stavros P., et al. \"No free lunch theorem: A review.\" Approximation and optimization (2019): 57-82.\n\n\nFurrer, D., van Zee, M., Scales, N., & Schärli, N. (2020). Compositional generalization in semantic parsing: Pre-training vs. specialized architectures. arXiv preprint arXiv:2007.08970.\n\nLampinen, A. K., & Ganguli, S. (2019). An analytic theory of generalization dynamics and transfer learning in deep linear networks. In International Conference on Learning Representations.\n\nSrivastava, Nitish, et al. \"Dropout: a simple way to prevent neural networks from overfitting.\" The journal of machine learning research 15.1 (2014): 1929-1958.\n\nZhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "kQxQWHww5Q", "reviewer": "Reviewer_e3dk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates systematic generalization in deep neural networks. Systematic generalization here refers to the ability of an algorithm to produce outputs that were not observed during training time. A potential reason for this is postulated: the lack of systematic generalization in deep neural networks is due to function sharing, i.e. that each layer in the network uses a common representation from the previous layer. Experiments show that networks with fewer shared intermediate layers exhibit a greater degree systematic generalization than those with more shared layers.", "summary_of_the_review": "After rebuttal: The paper has been improved by the inclusion of real-world experiments. I would echo the other reviewers in suggesting that these move to the main paper. As pointed out by Reviewer hir1, a more complete discussion about when function sharing is likely to be helpful seems necessary. Expanding on why function sharing occurs as started in Appendix C would also be useful. I have updated my score accordingly. \n\n---\n\nOverall, there are some potentially interesting ideas in this paper, but the clarity, unsupported claims, and incompleteness of the experiments are somewhat significant issues. The contribution of this paper is on the more incremental side and consists primarily of showing that sharing fewer layers can improve systematic generalization. However, it is not clear how to take these insights and apply them to solve out-of-distribution detection on real-world problems.", "strength_and_weaknesses": "Strengths\n- The broad aim of investigating stronger forms of generalization that move beyond the i.i.d. case is interesting.\n- The experimental observation that having fewer shared layers leads to better systematic generalization holds across multiple diverse architectures and datasets. \n- As far as I know, investigating function sharing as a reason for lack of systematic generalization is a novel approach.\n\nWeaknesses\n- The paper is not written very clearly. Sections that are difficult to understand include: the mathematical notation (e.g. writing that $f$ is a \"model\" but not explaining that this is simply a mapping from the input space to output space), the experiment section (particularly how the labels were generated and what the different evaluation metrics mean), and the discussion section.\n- Some intuitions are claimed but not supported by adequate evidence: that deep neural networks prefer to learn a simple function and combine with previously learned functions, and that neural networks greedily learn functions in order of simpler to more complex.\n- The experiments are not complete. Some relevant but missing pieces of information include training accuracy and computation time for the various levels of sharing. On a related note, the motivation for including the test set and random set accuracy metrics is unclear.\n- It is not clear how the knowledge introduced by this paper can be effectively used to improve systematic generalization. Training a multitude of independent networks, one for each underlying factor, does not seem like a practical course of action due to storage and computation constraints.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jf9EMUVAZIZ", "reviewer": "Reviewer_4tfA", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper hypothesizes that systematic generalization is fundamentally at odds with the tendency of deep learning models to share sub-modules. It provides both a theoretical description and derivation of the issue as well as supporting experiments with several deep learning architectures.", "summary_of_the_review": "This paper introduces an interesting and probably novel idea but it does not a good job at developing it. I believe it does not meet the ICLR bar.", "strength_and_weaknesses": "Strengths: the hypothesis advanced by this paper is new and interesting; it deserves to be developed further.\n\nWeaknesses:\n- The paper is not very well written; see below for some examples where the writing should be improved.\n- The propositions and theorem are trivial.\n- The vision experiments seem quite unnatural, since images from different datasets are averaged (see Section 3.2). Also the text experiments are not particularly natural (they concatenate two unrelated sequences and ask to predict both labels).\n\nA general question that might be worth addressing: the shape of the regions is not taken into account? For instance, to prove Proposition 1 (using Assumption 2) you can simply merge regions, but it might be unnatural to merge them, given the geometry.\n\nExamples of unclear parts in Section 2.1:\n- \"$Y$ contains $K$ factors $Y_1$, . . . , $Y_K$\" is not clear. Does it mean that $Y = Y_1 \\times \\dots \\times Y_K$? But this would be at odds with the (unclear) statement \"which can be entangled\".\n- Is $X_\\text{train}$ the (ordered) sequence of inputs or is it a set that contains all the input data? (same for $Y_\\text{train}$ and test)\n- \"The values for each factor $i$ are included in the training output\" is also unclear\n- \"A model $f$ maps input $X$ to the prediction of output $f (X)$\": so is $X$ the set of all inputs or a single input?\n\nExamples of unclear parts in Section 2.2:\n- \"deep learning more or equally prefers $f$ over $g$\" -> \"deep learning prefers $f$ over $g$ more or equally\" (a bit more clear in my opinion)\n\nExamples of unclear parts in Section 3.1:\n- \"$Y_1$ is chosen from all possible labels\": what does this mean? That $Y_1$ is the set of all labels?\n- The two datasets share the same input space $X$?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "OPHp_0G0jGx", "reviewer": "Reviewer_w8Ad", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper hypothesise that function sharing is one of the reasons why deep learning models can’t perform systematic generalization. The paper demonstrate that as the degree of parameter sharing increases, the systematic generalization drops. From the practical stand point, the papers argues for sparsity in models that somewhat learn symbolic functions (in term of disentangling feature attributes). Although it’s not touched upon but I believe the paper can be seen from modularity perspective where each module encampasses a particular underlying function, describing certain factor of the input. ", "summary_of_the_review": "This work need major re-writing as most of the concepts were not framed correctly. Moreover, they are some shortcomings in the evaluation section as I explained in the weaknesses section. I believe there is some value in this work, but it needs written with clarity.", "strength_and_weaknesses": "### Strengths\n- The paper provides empirical evidence of why parameter sharing (function sharing) leads to performance drop in systematic generalization for deep learning models.\n- I like the problem space and believe that the authors are on to something tangible here but the lack of rigour in analysis didn't convince me.\n\n### Weaknesses\n- I have issues with problem formulation and writing. Some details of the work are not framed correctly and it’s hard to understand it since no context from previous literature is provided while introducing and explaining new concepts.\n- Eg. what is implied as functions in deep neural networks? Is it an individual parameter or a set of parameters?\n- What does the three equations at the end of sec 3.1 refer to?\n- There is no clear description of the dataset. I can see that it contains factor but what are those factors? How are those factors combined?\n- On the same note, could you please provide description of the datasets and models used, separately?\n- The results section just provide information on the models tried. No information on training, and test, train splits of dataset provided.\n- The author try to ablate different model architectures, however they use different datasets across those architectures thus it’s hard judge if the results are consistent across those architectures or the differences arise from the difference in datasets.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject