Controlling Neural Network Smoothness for Neural Algorithmic Reasoning
Authors: David A. Klindt
Venue: Accepted by TMLR ()

Abstract: The modelling framework of neural algorithmic reasoning (Veličković & Blundell, 2021) postulates that a continuous neural network may learn to emulate the discrete reasoning steps of a symbolic algorithm. We investigate the underlying hypothesis in the most simple conceivable scenario – the addition of real numbers. Our results show that two layer neural networks fail to learn the structure of the task, despite containing multiple solutions of the true function within their hypothesis class. Growing the network’s width leads to highly complex error regions in the input space. Moreover, we find that the network fails to generalise with increasing severity i) in the training domain, ii) outside of the training domain but within its convex hull, and iii) outside the training domain’s convex hull. This behaviour can be emulated with Gaussian process regressors that use radial basis function kernels of decreasing length scale. Classical results establish an equivalence between Gaussian processes and infinitely wide neural networks. We demonstrate a tight linkage between the scaling of a network weights’ standard deviation and its effective length scale on a sinusoidal regression problem, suggesting simple modifications to control the length scale of the function learned by a neural network and, thus, its smoothness. This has important applications for the different generalisation scenarios suggested above, but it also suggests a partial remedy to the brittleness of neural network predictions as exposed by adversarial examples. We demonstrate the gains in adversarial robustness that our modification achieves on a standard classification problem of handwritten digit recognition. In conclusion, this work shows inherent problems of neural networks even for the simplest algorithmic tasks which, however, may be partially remedied through links to Gaussian processes.

Published in Transactions on Machine Learning Research (02/2023)
Controlling Neural Network Smoothness for Neural Algorithmic Reasoning
David A. Klindt.
klindt. david com
Reviewed on OpenReview: /ptp::///peeeeveeee net/ forum? Jdsyggutt
Abstract
The modelling framework of neural algorithmic reasoning (Veličković Blundell 2021 pos- p tulates that a continuous neural network may learn to emulate the discrete reasoning steps of a symbolic algorithm. We investigate the underlying hypothesis in the most simple con- ceivable scenario the addition of real numbers. Our results show that two layer neural networks fail to learn the structure of the task, despite containing multiple solutions of the true function within their hypothesis class. Growing the network's width leads to highly complex error regions in the imput space, Moreover, we find that the network fails to gener- alise with increasing severity i) in the training domain, ii) outside of the training domain but within its convex hull, and iii) outside the training domain's 's convex hull. This behaviour can be emulated with Gaussian process regressors that use radial basis function kernels of de- crcasing length scale. Classical results establish equivalence between Gaussian processes and infinitely wide neural networks. We demonstrate tight linkage between the scaling of a network weights' standard deviation and its effective length scale on sinusoidal re- gression problem, suggesting simple modifications to control the length scale of the function learned by a neural network and, thus, its smoothness. This has important applications for the different generalisation scenarios suggested above, but it also suggests a partial remedy to the brittleness of neural network predictions as exposed by adversarial examples. We demonstrate the gains in adversarial robustness that our modification achieves on simple image classification problems. In conclusion, this work shows inherent problems of neural networks even for the simplest algorithmic tasks which, however, may be partially remedied through links to Gaussian processes.
1 Introduction
The two most prominent paradigms in artificial intelligence research are discrete, symbolic algorithms on the one side, and continuous, neural information processing systems on the other (Fodor & Pylyshyn 1988 Natarajan 1989 Marcus 2003). While systems of the latter kind have caused revolutionary transformation of the field, they are often plagued by hard challenges, such as robustness to changes in the input distributions, for which algorithmic approaches can provide worst-case performance guarantees. Crucially, we know that both approaches are deployed by humans, akin to Kahneman's 1 and 2 reasoning systems (Kahneman, 2011 and that, therefore, algorithms must be implemented in biological neural networks the human brain Zador cet al., 2022). For instance, observing a scene the world, we know that it is represented and processed the distributed representation of neural activity in visual cortex. However, the same scene is also represented when we describe it with the use of symbols and the syntax of our language (Yildirim et al. 2020). Thus, one of the most mysterious questions in neuroscience as well as in artificial intelligence rescarch is: where and how do these two representation systems interact?
The concept of neural algorithmic reasoning (Veličković & Blundell 2021 a recent proposal for a modeling framework at the intersoction between symbol processing algorithms and continuous distributed information processing systems (see also Smoolnskyy] 1990 Bear et al., 2020 Sabour al., (0017). As an illustrative example of this hybrid approach, can think of robot equipped with neural network that processes

---

Published in Transactions on Machine Learning Research (02/2023)
input images to extract, c.g., its position in space (visual encoder); this is coupled with planning algorithm (e.g-: Dijkstma that computes the shortest path from the current location to the target; that output is then fed into another neural network (motor controller) which translates the symbolic action plan into continuous motor control outputs. The obvious question is how such a hybrid architecture may be trained, since we usually require differentiability of the whole system for end-to-end training. To solve this, /Vlićković & Blundell (2021) propose training a neural network to approximate the output of the algorithm in the middle of the model. This differentiable approximation would then allowing gradient flow from motor controller to visual encoder for effective end-to-end training.
The purpose of our work is to investigate the fcasibility of neural algorithmic reasoning in one of the most simple conceivable settings: the addition of real numbers. Integer calculus and foating-poing arithmetic in binary (symbolic) representations have previously received more attention (Nogueira et al.
2021 Talmor
et al. 2020: Jiang et al.] 2019) (Thawani et al.) [021 Zhou et al., 2022 Hendrycks et al. 2021 Bansal et al.
2022 By contrast, we want to know if a simple multilayer perceptron (MLP, (Rosenblatt 1958 ), i.c. the
basic building block of most neural networks, can learn to add real-valued numbers on compact domain such the unit disc. Specifically, we look at two layer MLPs which are particularly interesting because: i) they contain paramcter subspace that perfectly solves the task (sce below), ii) they can provide an approximation of unlimited precision in the infinite width limit (Hornik et al. 1989. iii) they reveal an interesting failure case that prompts further study of neural network functions.
In summary, this paper we find that artificial neural networks are unable to learn the simple function of adding real numbers, even if abundant training data is available, leaving nonlinear, uneven regions of error within the training domain and struggling to extrapolate beyond. A comparison to Gaussian processes suggests simple partial remedy, exploiting classic results (Neal, 1996 about the equivalence between these two model classes, based on correct adjustment of the smoothness of the learned function. We show that these modifications also translate to increased adversarial robustness on handwritten character image recognition [Goodfellow et al. 2014]. This has important implications for real world applications, such as self-driving car vision controllers that need be robust to shilting input statistics, where smooth, generalisable model functions are required. It also aligns with the motivation behind neural algorithmic reasoning to build diffcrentiable models that generalise as broadly analytical performance guarantees. Finally, are interested in this minimal setting where seeing how neural network fails to learn the correct algorithm reveals an interesting phenomenon that ad advances our basic understanding of the functions learned by neural networks and how to control their smoothness.
classical algorithms with worst case
2
Background
Simple mathematical reasoning (Saxton et al. 2019 Charton 2021 and, specifically, the addition of real mumbers a particularly interesting setting because it is so simple, while still elucidating important functional complexity of neural networks (Hendrycks & Dietterich 2019) (Fig. D and clearly exposing the difficult inductive inference from an infinite look-up table to the proper representation of an algorithm (Henderson) 2022). Moreover, we know that the hypothesis class of neural networks with rectified linear unit (ReLU) activation functions trivially contains a parameter subspace with the correct solution, iee. given inputs E IR? the following set of functions
Q 0
a
3
8-1
f(x) = W2 max Wix + b1) b2,
W ==
W2 =
(1)
7
7
0 8
8-1
with Y E Ry0, 8.8 E Reo and b,, by := 0 are a subspace of the network parameters that correspond to perfect representation of the desired output y = x2 + 42-
Given input and output training pairs, a neural network can learn to approximate this function. However, it is an open question how accurate the approximation will be within the compact domain of the training data; possibly in the limit of infinite data and a model that is a universal function approximator (Hornik et al. 1989 Secondly, it is unclear how the model will generalise outside the domain of the training data.
2

---

Published in Transactions on Machine Learning Research (02/2023)
Ideally, the concept of addition should only require limited amount of training examples to understand the underlying algorithm for producing the correct answer on any pair of inputs.
A crucial difference between algorithms and neural network solutions is the way they generalise to different inputs Marcus 2003; Veličković & Blundell 2021; Zador et al.) 2022). Addition is defined on all numbers in R. but the approximation learned by a NN can only observe subset of those inputs its training data (ie., i..dd independent identically distributed). Recent discussions have investigated this from the point of view of interpolation versus extrapolation (Nakkiran al.) 2021; Schott et al., 2021 Overparameterised NN exhibit a double descent phenomenon, which is thought to improve their generalisation performance by interpolating between training points (Chatterji et al.) 2021 although see (Balestricro et al. 2021) Going beyond the convex hull of the training data would, by contrast, require the ability to extrapolate to a new domain (o.o.d. - out of distribution). This is one of the key motivations behind neural algorithmic
reasoning and has recently been explored with transformers (Nogucira et al., 2021 Kim et al. 2021 Anil
ct al. 2022; Zhou et al. 2022 Charton
2021 Zhang et al. (021) and recurrent neural networks Bansal ct al.
(2022) ; Linsley et. al. (2018);
; Schwarzschild al. (2021). Apart from extensions of the compact training data
domain, also study robustness to specific distribution shifts such as added Gaussian noise (Hendrycks
& Dietterich 2019 Rusak et al.] 2020). We also study adversarial robustness (Goodfellow et al. 2014)
which can be thought of as worst-case distribution shifts, this has recently been studied in two layer MLPs
Dohmmatbb & Bietti 2022) and is specifically interesting as it relates to Lipschitz constants (Virmaux &
Scaman 2018) and a neural networks smoothness (see section 3.5).
An important caveat is that the point of this study is not to find a tailored solution to the problem of adding real numbers. There many handcrafted fixes the specific experimental setup in this paper, which i) reduce the size of the network down to the exact required dimensions (i.e., 4, see equation D ii) encourage sparsity to switch off all superfluous units, or iii) use more involved transformer-base
models Vaswani et al. 2017 Zhou et al.) 2022) to solve the task. However, none of these settings are of interest for the present rescarch question: i) and ii) provide solutions only to this specific problem without any transferable insights into neural network functions as we oblain in this paper (section 3.3); and iii) obscures the simple failure cases of MLPs, which are the minimal building blocks to study we want to understand neural networks (Wang et al. 2020
Dohmatob & Bietti 2022
3
Results
3.1
Neural Networks and Gaussian Processes Learning Addition
We first investigate learning addition real numbers in two dimensions. For this, we uniformly draw D = 128 points on the unit disc (see also Fig. 6
D: (x x E RR, |||||2 < 1).
(2)
We train a two layer neural network with ReLU nonlinearities after the first layer to solve the addition task using a simple squared loss function (for additional experimental details see Appendix section A.1
CMSE(f = (f(x) (a) + x2))²²
Across the top row of Fig. 1 we change the number of hidden units N (i.e., the width) of the network and observe the effect on the learned solution. With few units (N 16), the model exhibits the recently proposed polytope structure of neural network approximated functions (Balestriero et al., 2018). With more units entering the interpolation regime (i.e., zero training error), where recent work on double descent might suggest better performance (Nakkiran et al. 2021). While the performance docs increase (Appendix A.2.1 we can see that the model uses the additional capacity lo cut up the input space into increasingly complex, uneven regions. Interestingly, the network learns intricate, nonlinear ridges of good performance on which the training data lies. These ridges appear to be connected on continuous paths an intriguing observation for future NN rescarch. Importantly, these patterns suggest that larger ReLU networks learn sieve-like solutions of increasing complexity (see also Fig. 12 but they do not enter a qualitatively different
3

---

Published in Transactions on Machine Learning Research (02/2023)
ReLU (Nm16)
RelU
ReLU
ReLU (N-1024)
(A 0.2)
50)
Figure 1: Unit Disc Loss Surfaces. These plots show the network error in input space (remember, we are trying to model f R f(z) 21 2 x D C R²), where brighter regions indicate lower error, i.c., colorscale shows the log of the loss function equation (red dots training data points). Top row shows the loss surface for ReLU neural network with increasing number of hidden units (N, left to right). Bottom row shows the loss surface for GP with RBF kernel of increasing length scale (A).
regime that would resemble the simple algorithmic solution to the task. We provide initial exploration for why NNs fail to reach the optimal solution (equation D during optimisation in Appendix A.2.2
We can produce a similar sequence of model behaviours with Gaussian process (GP) regressors with varying radial basis function (RBF) kernel scales. We use the standard GP implementation in sklearn (Pedrogosa et al. 2011). Usually, this would include maximum likelihood length scale selection (optimum near À 150). However, setting this by hand lets us visualise the different solutions for suboptimal length scales. Specifically, that setting the length scale too low (Fig. [ bottom left.) forces the model to learn ridges of good solutions through the training data similar to the NN model
Note that the diagonal line passing through the origin indicates the mull space of the target function. Precisely, adding any vector from the subspace A = ((1-1)) = {a (1,-1) | a E R} to an input x, does not change the output of the algorithm y(z) = I1 + I2, i.c.
v()) 2 + F2 (x) + a) + (x2 a) y( Vn a (1,-1)
(4)
Thus, the anti-diagonal passing through the origin, the correct output y(w) 0 for all possible inputs (0, 0) +. We see that the chosen GP with outside of the training data. For detailed average performance levels see Appendix A.2.1 and for density dependent solutions see Appendix A.2.6
mean function, naturally, converges to this solution
3.2 Out of Domain Generalisation
Building models that generalise outside of the training data is key motivation behind neural algorithmic reasoning (Veličković & Blundell 2021 Zhou et al. 2022). We adjust our setting slightly to study all aspects
The geometry of these ridges studied analytically for GPs. We recognise that this might give meaningful insights into the appearance similar ridges for NNs. However, we leave this open for future work.
4

---

Published in Transactions on Machine Learning Research (02/2023)
ReLU (N 10, DO0; D oo)
Generalization Performance
Color Scheme (ang
20
35
18
16
Accuracy
15
0.0
0.5
1.0
1.5
2.0
Generalization Performance
1e-5
30-
2.00
Acouracy [nig (os0)
Unceraintty (..d)
1.75
1.50
50
26
1.25
40
(E train
0.75
30
22
ood [E chull
kde (a 10"!
0.25
20
Input Norm
Accuracy (neg. log loss]
Figure 2: Annulus Loss Surfaces and Generalisation Performance. The first column shows the loss surfaces for NN (top) with N hidden units, and nearly unlimited amount of training data; and (bottom) the loss surfaces for GP with optimal length scale (A) and D 256 training data points, shown red dots. The second column shows the negative log loss function of eccentricity (dots coloured by angle, black dotted lines radially averaged kernel density estimate 'kde') in the three regimes of o.o.d. (within convex hull), i.i.d. and o.o.d. (outside convex hull) generalisation for the NN (lop) and GP (bottom). The third column (top) shows the colour scheme of the second column; and (bottom) the GP's uncertainty estimate.
of generalisation. Specifically, the training data is now randomly sampled from the annulus
A = fo RR? 0.5 s | 1..
(5)
This allows us to test how well the model generalises:
Within the training domain: A (Fig. middle column, green).
Outside of the training domain but within its convex hull: {or e R || 0.5) (Fig. 2 yellow).
Outside of the training domain and outside its convex hull: (* E R2 |
> 1} (Fig. red).
Moreover, to ensure that the findings from Fig. 1 do not depend on limited training data or finite network size, we set the number of hidden units to N 10, 000 and generate a new random batch for every gradient step (totalling D 256 50, 000 12, 800, 000 training examples). As before, for full experimental details, please refer to Appendix section A.1
The learned solution by the NN is shown in Fig. 2 top left. Again, emerging within the training data domain (i.id..) with ridges of good performance but large valleys of bad predictions. We also observe that the model predictions deteriorate both inside the convex hull of the training data as well as outside. This is quantified in Fig. top middle as a function of the input norm (we can also see the different ridges outside the training domain distinguished by their angles). For the GP, even with limited training data (D 256), we see higher performance levels (Fig. bottom middle) both on training data (green), in
intricate, uneven pattern
its convex hull (yellow) well outside (red). Moreover, for o.o.d. regions outside the

---

Published in Transactions on Machine Learning Research (02/2023)
GP - 0.5)
GP fA 1151
GP (.5)
2.0-
10.
1.3
1.5
1.-
Morral
-1.
-10
-1.0
-2.
-2.8
-2.
ANo 0:31
Ni 1
3.30
2.0
1.3
1.5
1.0-
0.
0.8
0.0
-1.0
-1.-
-1.0
-13-
-1.
Figure 3: Sinusoidal Regression. Top row shows GP models with different length scales (A) fitted to a sinusoidal regression problem with: true function (yellow i..d.,, purple o.o.d.), training points (red), model prediction (light blue), uncertainty (shaded, two standard deviations). The top right plot shows the different losses (mean squared error MSE, and log likelihood) a function of length scale (dotted lines indicate length scales in plots to the left). The bottom row gives the same regression plots for the NN with TanH activation and varying length scales (o- and (right) the training and test error as a function thereof.
training data's convex hull, where the performance starts dropping, the GP at least produces meaningful uncertainty estimates (Fig. bottom right).
3.3 Setting the Length Scale of Neural Networks
The similarity between the solution patterns for neural networks with many hidden units (Fig. [ top right) and GPs with short length scales (Fig. 1 bottom left) is not surprising given classical results Neal 1996) Briefly, Neal established that for NNs with a hyperbolic tangent (TanH) nonlinearity
sinh (x)
2x -1
tanh(x) == cosh(x))
(6)
2 +1
and an infinite number of hidden units (with appropriate scaling of their initialisation variances) the distri- bution over learned functions (ab inilio) becomes equivalent that of a GP. This opens an interesting path forward in understanding and improving the neural network solution.
An important step in fitting GP with a RBF kernel to data is finding the best length scale for the kernel. Effectively, this controls the smoothness [Lederer et al. 2019) of the resulting non-parametric model and, thus, the complexity of the hypothesis class [Vapnik] 1999). ). Translating this into the space of neural network functions is nontrivial in the general setting with ongoing work into the Lipschitz constants of trained models (Virmaux & Scaman
2018 Fazlyab et al., 2019). In this section, thanks to the equivalence in the considered settings, we are able to investigate, parallel, the length scale selection process both in GPs and NNs.
For easier visualisation of the learned pnput-output mapping we reduce complexity even further: The dataset is now simply a noisy sine function on E [-2m, 2m] (for o.o.d., extrapolate to [-37, Precisely, we are trying to model the function y sin(x) e with <~ (0, 0.25). In Fig. 3 we can see that there exists a sweet spot for the kernel's length scale (top middle), and that the model overfits (top left) for smaller length scales and underfits (top right) for larger length scales. This is confirmed quantitatively (top right) by looking at the likelihood (blue) as well as the training and test error as a function of the length scale.
To get a NN behave like a GP, we closely follow the construction in Neal 1996) making the following changes to NN model: i) We use a TanH nonlinearity; ii) We restrict the standard deviation (sld)

---

observe Published in Transactions on Machine Learning Research (02/2023)
g- 0.3
-- 0.9
0-1=2.0
0 33
Figure 4: Controlling the Smoothness of Neural Network Functions. The four columns show learned TanH two layer NN loss surfaces for the two datascts: top, disc (soction 3.1 N 10, 000, D 128); bottom, annulus (section 3.2 N 10, 000, oc) for increasing length scales (left to right.) producing
increasingly smooth mapping (for comparison, see also Neal (9966 Fig. 2.3).
of the weights Lo N where N is the number of inputs to layer and scaling factor. Importantly in contrast to Neal (996)., we enforce this standard deviation (let u(w) Σx wy denote the mean of the weights w) throughout training by using the scaled weights
1 W
w ΣN (w: - p(w))²
(7)
VNst((w)
Intuitively, -1 behaves like the length scale in GPs (Fig. 3 bottom), i.e., smaller o means larger weights and higher curvature/lower smoothness (potentially overfitting to the training data) whereas too large o-1 mcans very small weights and lower curvature/ /higher smoothness (potentially underfitting the training data) (see also Neal (1996) Fig. 2.3). Again, we establish the existence of an optimum length scale a- (Fig. 3 bottom right) that produces the smallest test error. In summary, these results suggest a simple remedy to improve NN approximations to simple algorithmic functions by making them more similar GPs (with TanH nonlinearity) and correctly adjusting their length scale and, thus, their functional smoothness.
3.4 Controlling Neural Network Smoothness: Addition
Returning the initial two examples of adding real numbers from disc or annulus, the effect of varying the length scale (a-1) of two layer NN with large number of hidden units (N = 10, 000, approaching the GP regime) and TanH activation functions (see Appendix A.1 for additional details).
On both datasets (Fig, 4 it apparent that increasing the length scale makes the learned output function more smooth. Thus, the NN becomes qualitatively more similar to GP with a well adjusted learning scale for its RBF kernel. We show in Appendix A.2.8 that the weight normalisation (equation 7 is crucial for these results, both in the case of addition and, for further verification and extension, multiplication.

---

Published in Transactions on Machine Learning Research (02/2023)
3.5 Controlling Neural Network Smoothness: Classification
As further evidence in support of the claim that this approach effectively controls the smoothness of the Iearned NN input-output mapping, we turn towards the more complicated problem of performing image recognition on MNIST {LeCun et al. 1989) and CIFARIO (Krizhevsky al. 2009) under different distri- bution shifts. Briefly, using the same two layer TanH model as above, with the minimal modification that the second layer output dimension is now 10 - i.c. the logits. All other details remain the same (see Appendix A.1], except for the objective function. Over training, we are minimising the standard cross-entropy loss between the output logits and the target label CoE (f, Zz, y:) = (Z(()
have a dataset of flattened images I; E 12 and labels yi {I,..., 10). We are
Since the imput domain is now much more complex and high dimensional than in the previous settings, we have to adjust the different generalisation settings. Firstly, we look at additive white (Gaussian) noise of varying standard deviation (Fig, 5 right) i.e. exploring random L2 epsilon balls around the training data
manifold (Hendrycks & Dietterich 2019). Secondly, we look at worst case (adversarial) distribution shifts
(Fig. 5 left) where we search for minimal perturbations that maximally change the output, which is a proxy for the model's smoothness (Goodfellow et al.) 2014 Dohmatob & Bictti 2022). More specifically, for a fixed maximum L2 perturbation norm c (horizontal axis Fig. we look for
+ max L)) s.t. l| S.
(8)
This second setting is particularly interesting: a) in high dimensions and complex input manifolds (here digit images) that defy simple separation of scttings (Fig. 2 [Balestriero et al., 2021 and b) because those
MNIST Adversarial Robustness
MNIST White Noise Robustness
1.0
1.0
-.00
0-38
0.9
-
0.8
Claziiiiation Accuracy
0'1-2.71
0.7
0.7
0.6
0.5
=0.20
Clessifiation
0.5
0.4
-4.34
0.4
021000
0.3
0.2
0.2
0.1
0.0
0.00
0.15
0.30 Porturbation strangth: g
White noise standard deviation
CIFAR10 Adversarial Rabustness
CIFAR10 White Noise Robustness
0.45
0.45
+ . 1 1.00
0.40
0.40
0.35
0.95
0'1
Clasification Accuracy
0.35
0.30
0379
0.30
Ciassitcttion
0.25
0.177.37
o51=522
0.25
0.20
o-1-10.20
0.20
04.34
0.15
0 --000
0.15
0.10
0.10
0.05
0.00
0.00
0.05
0.10
0.15
0.25 0.30 Perturbation strength: c
0.20
2
10
White noise standard deviation
Figure 5: Neural Network Classification Robustness. Top, MNIST: Left) Accuracy as function of (FGSM) perturbation strength for TanH NN models with different length scales (a-1); Right) Accuracy as a function of white noise standard deviation for the same models. Bottom, same for CIFAR10 dataset.
8

---

Published in Transactions on Machine Learning Research (02/2023)
are the edge cases analysed when assessing the theoretical performance guarantees for classical algorithms (Veličković & Blundell 2021).
Fig. 5 (left) shows that controlling the length scale of a neural network on handwritten character recognition does indeed increase the robustness to Lg adversarial perturbations. Fig. 3 (right) shows that the robustness to white noise perturbations can be controlled in a similar way. In both cases, we can see a trade-off ween the clean and the robust accuracy with large length scales increasing the robustness while decreasing the clean accuracy (Rusak al.) 2020). In control experiment, test whether the same robustness gains may be achieved with a simple control model using ReLU activations and standard weight decay (Appendix A.2.5 We find that this also increases adversarial robustness, however, to smaller extent. To reduce computational costs, we have used the fast gradient sign (FGSM) attack (Goodfellow et al. 2014]. However, in Appendix [.22. we show that we obtain the same qualitative results with the much stronger adaptive attacks implemented in AutoAttack (Croce & Hein 2020
4 Limitations
The proposed modifications are only applicable to simple two layer MLPs i.c., the minimal example for which the complex input space behaviour (Fig. 1 arises (Wang et al. 2020). It is insight to observe how the smoothness of this simple model class can be controlled in the GP regime (Lederer et al., 2019p. However, this does not generalise to more layers (confirmed by preliminary experiments) and it is open for future work to investigate how the composite function of more than two layers contributes to global smoothness of NN. potentially building on recent work extending the NN-GP equivalence to multilayer networks (Lee et al. 2017). To extend the current results we would need a closed form, or at least simple enough, expression for the length scale of a NN with more than two layers that allows us to control this parameter. Preliminary experiments on deeper models (Appendix (A.2.4) reveal that gains in robustness do not transfer to multi-layer (convolutional) models in a straightforward way.
Even with the modifications, the network only becomes more similar to tuned GP. However, there is still gap between this smoother function approximation and the qualitative inference step of actually understanding the underlying arithmetic and learning the right solution for all possible inputs Henderson (2022). Put differently, the non-zero error rates, even for the GP, indicate that even smooth NNs are only taking us closer to the bold goal of learning how to infer and emulate simple algorithms.
While GPs show more desired behaviour on this task, again we want to highlight the fact that they do not offer a general solution the problem that we are interested in, because they: i) fail to generalise properly to o.o.d. settings (see Figs. I 2); ii) scale poorly to problems with more data and larger input dimensions; and iii) cannot be integrated into differentiable model pipeline, which is the original motivation for neural algorithmic reasoning. Moreover, using GPs instead of NNs (rather than as an inspiring analogy) would not allow us to extract the insights into neural network functions which we obtain in this paper (section 3.3 Therefore, we did not devote more time into studying different GP kernel and mean functions.
5 Discussion
The promise of neural algorithmic reasoning to combine distributed and discrete reasoning systems via dif- ferentiable NN approximations is intriguing (Veličković & Blundell 2021 Here, we show that the functions learned by neural networks, even in one of the most simple conceivable examples of symbolic manipulation (real addition), is prone to learning a highly complex and uneven output mapping that falls short of learn- ing the proper algorithmic target. There are many different perspectives on this problem, with previous approaches mostly focusing on transformers for manipulating arithmetic expressions (Thawani et al. 2021 Zhou et al.] 2022 Anil et al. 2022, Nogucira et al. 2021 Jiang et al.) 2019 Talmor et al. 2020). However, looking at the loss surface of the NN solution for a simple two layer MLP, reveals an intriguing structure of the loss surface and analogy to GPs that we decided to pursue in this work and that may also provide a useful pointer for future directions in basic NN theory. Specifically, as challenges prior results stating that MLPs are able to perfectly learn linear functions (Xu et al., (2020) (under stricter theoretical assumptions) and that they are biased towards low frequency (i.e., smooth) solutions (Rahaman al. 2019
9

---

Published in Transactions on Machine Learning Research (02/2023)
We have demonstrated how the NN mapping can be made more similar to that of well-calibrated GP, specifically, by making it more smooth which also improved adversarial robustness. The smoothness is closely related to a neural networks' Lipschitz constant (Virrmax & Scaman, 2018). Whereas prior work derived GP Lipschitz constants depending on the specific kernel (Lederer et al. 2019). note that designing Lipschitz neural networks with (provably) bounded smoothness is an open research area (aazlya et al. 2019; Jordan et al. 2019). Also, the performance levels in Fig- 5 are far from state of the art robust models on this task, however, they eschew the need for (expensive) adversarial training (Kurakin al. 2016] WWong et al,, 2020 or generative inference (Schott et al.) 2018). Thus, this is
that the smoothness of NN function can indeed be controlled with the construction proposed in this paper.
A different inductive bias (other than smoothness), is symmelry (Vapnik, 1999 For instance, the real numbers with the addition operation form an Abelian group and, thus, addition is commutative. Finding the right symmetries, such as permutation invariance of the inputs in the case of commutative algorithm, is crucial in controlling the complexity of learning problem (Bronstein al.) 2021). ). Intuitively, symmetries reduce the size of the search space by grouping parameter combinations that lead to the same function (ef., Entezari et al. 2021). We see in preliminary experiments that adding this additional constraint, does force the model to become permutation invariant (commutative) in its inputs (Appendix A.2.7 but it does not take us closer to properly learning the correct algorithm.
Here, we have closely followed the construction in Neal 1996 by normalising the network weights; future research might investigate the effect of normalising the network activations on the phenomena studied in this work. Prior findings suggests that batch normalisation (BN) (loffe & Szegedy 2015), , one of the most commonly used activation normalisation schemes (Ren et al. 2016), helps network training and generalisation by making the loss landscape smoother and reducing the norm of the weight gradient during optimisation (Santurkar et al. 2018). However, despite this increased smoothness of the loss landscape, BN does actually seem to hurt adversarial robustness (Benz et al. 2021 Singla et al., 2021) and make adversarial training more difficult (Wang et al. 2022 Walter ct al.] 2022j. By contrast, test-time BN can help models deal with dist distribution ion shifts (Schneider et al. 2020), and normalisation in the form of feature competition (e.g., sparse coding, divisive normalisation) has been shown to increase adversarial robustness (Paiton et al. 2020
Finally, while this study focuses on the smoothness of NN approximations to simple algorithms, the larger question remains still open how NNs can make the inferential step from any finile amount of data to an infinite look-up table (see MLST episode 061} this amounts to Hume's problem of induction (Hume 1896 Henderson 2022) which may not be solvable with connectionist architectures (Fodor & Pylyshyn
1988). p. Indeed, there remains a gap between the correct algorithm and its neural approximation which future research may seek to close, for instance, by taking closer look at the biological solution (Zador et al. 2022). Surely, humans are a proof-of-coneept that noisy and distributed processing systems (i.e., brains) can implement discrete symbolic algorithms (Fias et al. 2021). We hope that future research in this area will benefit more from interdisciplinary approaches that take inspiration across fields.
Broader Impact Statement
As neural network applications become more and more ubiquitous in transforming data, for instance, in self-driving cars or autonomous flying drones, the need for reliable system grows proportionally. By studying the smoothness of neural networks, we hope to contribute to these efforts. Specifically, algorithms in high stake scenarios (such as traffic control) undergo thorough theoretical evaluation to ensure that they reach minimal performance targets even for worst case inputs. Analogously, controlling the smoothness of neural networks to equip them with this crucial aspect (in the spirit of neural algorithmic reasoning), takes small steps towards the same desideratum.
Acknowledgements
The present paper has been shaped by insightful foodback from many people, including: Lukas Schott; Jing Yang Zhou, Eero Simoncelli and members of his group; Anthony Zador, Tatiana Engel, Alexei Koulakov and the Ae community at Cold Spring Harbor Laboratory; well as the anonymous reviewers who
10

---

Published in Transactions on Machine Learning Research (02/2023)
have suggested number of additional experiments and valuable clarifications. We would like to thank all of these people for their time, thought and consideration which have helped to shape this paper into its final form, and which have, hopefully, provided them with some inspiration for their future scientific journeys.
References
Samuel к Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models modulo permutation symmetries. ar preprint arXiw:2209.0400 04836, 2022.
Cem Anil, Yuhuai Wu, Anders Andreassen, Aitor Lewkowyez, Vedant Misra, Vinay Ramasesh, Ambrose Slone, Guy Gur-Ari, Ethan Dyer, and Behnam Neyshabur. Exploring length generalization in large language models. arXiv preprint arXiv:2207. 04901,, 2022.
Randall Balestriero, Jerome Pesenti, and Yann LeCun. Learning in high dimension always amounts to extrapolation. arXiv preprint orXiv:21 10. 09.499855 2021.
Randall Balestriero et al. spline theory of deep learning. In International Conference on Machine Learning, pp. 374-383. PMLR, 2018.
Arpit Bansal, Avi Schwarzschild, Eitan Borgnia, Zeyad Emam, Furong Huang, Micah Goldblum, and Tom Goldstein. End-to-end algorithm synthesis with recurrent networks: Logical extrapolation without over- thinking. arXiv preprinl ar Xix: 05826, 2022.
Daniel Bear, Chaofei Fan, Damian Mrowea, Yunzhu Li, Seth Alter, Aran Nayebi, Jeremy Schwartz, Li F Fei-Fei, Jiajun Wu, Josh Tenenbaum, et al. Learning physical graph representations from visual scenes. Advances in Newral Information Processing Systems, 33:6027 6039, 2020.
Philipp Benz, Chaoning Zhang, and In So Kweon. Batch normalization increases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective. In Proceedings of the IEEE/CVP Internaiional Conference on Compuler Vision, pp. 7818 7827, 2021.
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprinl arXin:2104.13/ 134778 2021.
François Charton. Linear algebra with transformers. arXiv preprint arXiv:2112.01890 2021.
Niladri Chatterji, Philip M Long, and Peter L Bartlett. When does gradient descent with logistic loss find interpolating two-layer networks? J. Mach. Learn. Res., 22:159-1 2021.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fi freee attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020.
Elvis Dohmatob and Alberto Bietti. On the (non-) robustness of two-layer neural networks in different learning regimes. Xi preprint wrXii:2203.118h0 2022.
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. The role of permutation invariance linear mode connectivity of neural networks, arXiv preprint arXiv:2110.000 06296, 2021.
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George Pappas. Efficient and accurate estimation of lipschitz constants for deep neural networks. Advances Neural Information Processing Systems, 32, 2019.
Wim Fias, Muhammet Ikbal Sahan, Daniel Ansari, and lan M Lyons. From counting to retrieving: Neural networks underlying alphabet arithmetic learning. Journal of Cognilive Neuroscience, 34(1):16 33, 2021.
Jerry A Fodor and Zenon W Pylyshyn. Connectionism and cognitive architecture: A critical analysis. Cognilion, 28(1-2):3-77 71, 1988.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint ar Xio: 14l.:1412.6500 6572, 2014.
11

---

Published in Transactions on Machine Learning Research (02/2023)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern vecognilion, pp. 770-778, 2016.
Leah Henderson. The Problem of Induction. In Edward N. Zalta and Uri Nodelman (eds.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter 2022 edition, 2022.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint orXiv:1933.12 12261, 2019.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. ar preprinl arXiv:2103.038 2021.
Kurt. Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359 366, 1989.
David Hume. A treatise of human nature. Clarendon Press, 1896.
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. International conference on machine learning, pp. 448 456. PMLR, 2015.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
Chengyue Jiang, Zhonglin Nian, Kaíhao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, and Kewei Tu. Tearning numeral embeddings. arXie preprinl arXiv:2001.00003, 2019.
Keller Jordan, Hanie Sedghi, Olga Saukh, Rahim Entezari, and Behnam Neyshabur. Repair: Renormalizing permuted activations for interpolation repair. arXiv preprint arXim:2211.08/ 1.08403, 2022.
Matt Jordan, Justin Lewis, and Alexandros G Dimakis. Provable certificates for adversarial examples: Fitting a ball in the union of polytopes. Advances in neural information processing systems, 32, 2019.
Danicl Kahneman. Thinking, fast and slow. Macmillan, 2011.
Jeonghwan Kim, Giwon Hong, Kyung-min Kim, Junmo Kang, and Sung-Hyon Myaeng. Have you seen that number? investigating extrapolation in question answering models. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7031-7037, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1412.690 2014.
arXio preprint
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, lan Goodfellow, and Samy Bengio. Adversarial machine learning at scale, arXiv preprint wrXiu:161..01290 2016.
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1 (4):541-55) 1989.
Armin Lederer, Jonas Umlauft, and Sandra Hirche. Uniform error bounds for gaussian process regression with application safe control. Advances in Neural Information Processing Systems, 32, 2019.
Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl- Dickstein. Deep neural networks as gaussian processes. arXiv preprint 1:Xiv:1711.000 2017.
Drew Linsley, Junkyung Kim, Vijay Vecrabadran, Charles Windolf, and Thomas Serre Learning long-range spatial dependencies with horizontal gated recurrent units. Advances in neural information processing systems, 31, 2018.
12

---

Published in Transactions on Machine Learning Research (02/2023)
James Lucas, Juhan Bae, Michael R Zhang, Stanislav Fort, Richard Zemel, and Roger Grosse. Analyzing monotonic linear interpolation in neural network loss landscapes. arviv preprinl or arXi::2144.1 11044, 2021.
Gary F Marcus. The algebraic mind: Inlegraling connectionism and cognilive science. MIT press, 2003.
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and llya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021
2021.
Balas K Natarajan. On learning sets and functions. Machine Learning, 4(1):67 97, 1989.
Radford M Neal. Priors for infinite networks. In Bayesion Learning for Neural Networks, pp. 29-53. Springer, 1996.
Rodrigo Nogueira, Zhiying Jiang, and Jimmy Lin. Investigating the limitations of transformers with simple arithmetic tasks. arviv preprint or Xix:21102. 13019, 2021.
Dylan M Paiton, Charles G Frye, Sheng Y Lundquist, Joel D Bowen, Ryan Zarcone, and Bruno A Olshausen. Selectivity and robustness of sparse coding networks. Journal of vision, 20(12):10-144 10, 2020.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825 -2300, 2011.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International Conference on Machine Learning, pp. 5301-5310. PMLR, 2019.
Comparing and extending network normalization schemes. arXiv preprint Xr XXi::1611.0450) 1.44500, 2016. Mengye Ren, Renjie Liao, Raquel Urtasun, Fabian H Sinz, and Richard S Zemel. Normalizing the normalizers:
Frank Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6):386, 1958.
Evgenia Rusak, Lukas Schott, Roland Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. A simple way to make neural networks robust against diverse image corruptions. In European Conference on Computer Vision, pp. 53 69. Springer, 2020.
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. Advances in neural information processing systems, 30, 2017.
Shibani Santurkar, Dimitris Tsipras, Andrew Tlyas, and Aleksander Madry. How does batch normalization help optimization? Advances in neural information processing systems, 31, 2018.
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical reasoning abilities of neural models. arXie preprint 19Xiv:1904.01300 2019.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, 33:11530-11551 2020.
Lukas Scholt, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model on mnist. arXie preprint arXix:1808 hat:1800009900 09190, 2018.
Lukas Schott, Julius Von Kügelgen, Frederik Träuble, Peter Gehler, Chris Russell, Matthias Bethge, Bern- hard Schölkopf, Francesco Locatello, and Wieland Brendel. Visual representation learning does not gen- cralize strongly within the same domain. arXio preprint arXtv:2107.0822. 2021.
Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein. Can you learn an algorithm? generalizing from easy hard problems with recurrent networks. Advances in Neural Information Processing Systems, 34:6695 (6665,67766660 2021.
13

---

Published in Transactions on Machine Learning Research (02/2023)
Vasu Singla, Songwei Ge, Basri Ronen, and David Jacobs. Shift invariance can reduce adversarial robustness. Aduances in Newral Information. Processing Systems, 34:1858-1877 2021.
Paul Smolensky. Tensor product variable binding and the representation of symbolic structures in connec- tionist systems. Artificial intelligence, 46(1-2):159- 216, 1990.
Alon Talmor, Yanai Flazar, Yoav Goldberg, and Jonathan Berant. olmpics-on what language model pre- training captures. Transactions of the Association for Computational Linguistics, 8:743 758, 2020.
Avijit Thawani, Jay Pujara, Pedro A Szekely, and Filip Tlievski. Representing numbers in nlp: a survey and a vision. Xi preprint arXiv:2103.13136, 2021.
Vladimir Vapnik. An overview of statistical learning theory. IEEE transactions on neural networks, 10 (5):988-999, 999, 1999.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Atlention all you need. Advances in neural informalion processing systems, 30, 2017.
Petar Veličković and Charles Blundell. Neural algorithmic reasoning. Potterns, 2(7):100273, 2021.
Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Newral Information Processing Systems, 31, 2018.
Tiffany J Vlaar and Jonathan Frankle. What can linear interpolation of neural network loss landscapes tell us? In International Conference on Machine Learning, pp. 22325 22341. PMLR, 2022.
Nils Philipp Walter, David Stutz, and Bernt Schiele. On fragile features and batch normalization in adver- sarial training. arXiv preprint arxiv:2044. 12399, 2022.
Haotao Wang, Aston Zhang, Shuai Zheng, Xingjian Shi, Mu Li, and Zhangyang Wang. Removing batch normalization boosts adversarial training. In International Conference on Machine Learning, pp. 23433- 23445. PMLR, 2022.
Zhunxuan Wang, Linyun He, Chunchuan Lyu, and Shay B Cohen. Nonparametric learning of two-layer relu residual units. arXiv preprint 0rXtc:2008.076// 2020.
Eric Wong, Leslie Rice, and Zico Kolter. Fast better than free: Revisiting adversarial training. arXie preprint ar Xxxx:2001.03900 2020.
Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S Du, Ken-ichi Kawarabayashi, and Stefanie Jegelka. How neu- ral networks extrapolatc: From foodforward to graph neural networks. arXiv preprint arXiv:2009.1184 2020.
l1ker Yildirim, Mario Belledonne, Winrich Freiwald, and Josh Tenenbaum. Efficient inverse graphics in biological face processing. Science advances, 6(10)ceaax5979, 2020.
Anthony Zador, Blake Richards, Bence Ölveczky, Sean Escola, Yoshua Bengio, Kwabena Boahen, Matthew Botvinick, Dmitri Chklovskii, Anne Churchland, Claudia Clopath, et al. Toward next-generation artificial intelligence: Catalyzing the neuroai revolution. ar preprint wrXiv:22100083/p 2022.
Chiyuan Zhang, Maithra Raghu, Jon Kleinberg, and Samy Bengio. Pointer value retrieval: A new benchmark for understanding the limits of neural network generalization. arXie preprint brXiw:2107.12558: 2021.
Hattie Zhou, Azade Nova, Hugo Larochelle, Aaron Courville, Behnam Neyshabur, and Hanie Sedghi. Teach- ing algorithmic reasoning via in-context learning. arXiv preprint arXiw:2211.09066, 2022.
14

---

Published in Transactions on Machine Learning Research (02/2023)
A Appendix
A.1
Experimental Details
Training
Testing Data
(visualization) Data
100
15
10
10
0.50
050
05
0.00
0.00
00
00
-05
05
-0.50
0.50
-10
-10
-0.75
-1.00
-100
0.0
Figure 6: Visualisation of Addition on Disc Datasets.
To sample uniformly from the disc : {x x E R², ||a||2 S 1}, we first sample a random angle U(, 2x) and then a random length r' -((0,1) which we take the square root r vr to get uniform sampling on the disc. These polar coordinates are then transform into standard euclidean coordinates æ1 r cos 0 and 22 sin f (sce Fig. 6). For the Annulus == {x R2 |0.5 < | 1}, from r' (0.255,11 1).
we simply sample the length
We train for 50, 000 steps with the Adam optimizer Kingma & Ba 2014), an initial learning rate of 0.001 and a learning rate decay of 0.9. We verified in initial experiments that these settings led to best held- out performance and convergence gradient descent for the lested models. For the adversarial robustness experiments in section 3.5 we use the fast gradient sign attack (Goodfellow et al., 2014).
A.2
Additional Experiments
A.2.1
Performance on Disc Addition
(Model)
MSE
MSEtest
MSE....d.
ReLU (N=16)
2.93e-11
(2.39e-14
8.46e-06
(2.89e-06 )
1.99e-02
(4.49e-03
ReLU (N=64)
8.05e-08
(1.20e-08
6.22o-06
(1.10e-06
6.94e-03
(1.91c-03
ReLU (N=256)
1.46e-09
(3.20e-10 10
6.10e-06
(5.94e-07
2.46e-03
(4.31e-04
ReLU (N=1024)
1.95e-10 10 (6.30e-11
5.54e-06
(6. (6.10
1.48e-03
(1.51e-04
GP (X=0.2)
2.09e-14 14 (1.79e-14)
1.58c-03
(5.11e-04
1.55e-00
(1.12c-02)
GP (X=1)
3.67e-12
(1.09e-13 13
1.12e-10
(3.41e-11 )
9.29e-04 (8. (8.08e-05)
GP (A=5)
1.18c-12
(4.50e114)
2.12c-12
(2.71c-13 )
4.24c-08 (3. 73.779
GP (A=50)
2.43e-1 (2.69e-15 3.04e-14 (4.34e-15) )
6.97e-12 (9.52e-13
Table 1: Performance on Disc Addition. For ReLU networks with different numbers of hidden units
(N) and GPs with different length scales (X) corresponding to the models in Fig. [ Reported is the mean squared error (MSE) on the training and test sets, well as out of distribution X := [-1.5, 1.5] D.
15

---

Published in Transactions on Machine Learning Research (02/2023)
A.2.2
Optimal Solution and Interpolation Experiments
Interpolation solutions
Interpolation solutions (modulo OLS)
Interpolation between solutions
10
10
10a1
10
10
10
10
10
10
Test
10-10
10-10
10-10
10-11
9g to Byy
10-11
Bg to Bag (OL5)
10-11
Paz to Past
Oo to Oope
0o to Ospe (OLS)
0gr to Best (OLS)
10-1
0.6 Interpolation confficient: a [valus]
0.4
0.6
Interpolation confficiant: [valus]
Interpolation confficient: a [valus]
10
10
10-7
10-7
10-
Test
10-19
10-10
10-10
10
9o to Bey
10
8g to Bazz (QLS)
10
Par to Bot
B, to Riee
8, to Rir (OLS)
Bay to Bur (OLSS
10-16
10-14
10-16
80
100
20
40
80
100
interpoletion coefficient: a [index]
Interpoletion coefficient: [index]
Interpoletion coefficient a [index]
Figure 7: Interpolation Experiments. Test loss as a function of the interpolation coefficient (a) between models (see text) at initialisation (0o), after training (0sga) and with the optimal solution (Oapt; equation D. Top (bottom) row shows the value (index) of the horizontal axis for clearer visualisation. OLS (ordinary least squares) denotes models where the second layer weights (W2) were set to the least squares solution on the training data set. Light colours indicate 10 random seeds and darker colours averages.
In this section, we provide a preliminary exploration for why NNs fail to find the optimal solution (equation ! We primarily rely on one dimensional cross sections of the loss surface. Specifically, we look at interpolations from initialisation trained models (Lucas ct al.) 2021 Vlaar & Frankle 2022) as well as between trained models (Entezari al., 2021: Ainsworth al. 2022 Jordan al.) 2019). We choose the same settings as in Fig. D (top row, third column), i..,, D = 128 data points from the disc D, N 256 hidden units, standard two layer MLP with ReLU, As before, we train the network with the same settings (see section A.] and denote the learned weights Ssgd. Further, we denote the weights at initialisation Gu and the optimal solution Oopt (equation D. We repeat. this experiment for 10 different random seeds.
First, we investigate whether interpolating from the network initialisation (0o) to a solution produces a monotonically decreasing test loss (Lucas al. 2021). In Fig. (left column), we see that this the case both when interpolating to the learned Osgd well as the optimal Oupt solution. Note that the loss curve of the former (0, to @gga) is lower for most of the interpolation distance, which might explain why (greedy) gradient descent algorithm would prefer that direction. However, the interpolating to the optinal solution (0o to oppt) ultimately achieves a much lower test error (up to machine precision), as expected.
Sccond, an alternative explanation for the failure to find the optimal solution is that as the number of units grows (N + xo), the (first layer) weights are virtually fixed and the model forced to utilise the feature space provided initialisation (cf. neural tangent kernel aJacot al. 2018)). Consequently, during gradient descent, the model cannot reach the optimal solution (cquation 1). To investigate this, compute the ordinary least squares (OLS) solution for the second layer weights (W2). Precisely, given the hidden
activations H E RND as H; = max(0, X, + b1) for the training set (X E Dx2 Y E RRD) we compute
β H
where (HTH) is the (Moore-Penroso, pseudo-inverse matrix of H H. We then use this as second layer weights W2 3 (with bias set to ba 0) and test the model performance on the test set.
16

---

Published in Transactions on Machine Learning Research (02/2023)
In Fig. 7 (middle column) we see that interpolating from fu to Oagd but computing the OLS solution for the second layer for each interpolation value (a), we obtain nearly constant loss curve. This suggests, that over training the quality of the first layer feature space (or equivalently, the quality of the model modulo optimal/OLS second layer weights) does not change and that most increase in performance during training comes from solving the (convex) optimisation of the second layer. By contrast, interpolating (modulo OLS) to the optimal solution (fopt) stays approximately constant until, at the very end of the interpolation path, there is a sharp decrease in test loss (i.e., increase in the quality of the first layer feature space). This experiment suggests that: i) most training progress can be recovered by optimising the second layer during learning, which alone cannot achieve the optimal solution; and that ii) there is a sharp transition in model quality close to the optimal solution, until then the loss (modulo OLS) is almost constant.
Third, we may also ask: how does the loss surface behave for an interpolation between solutions, i..., from egd to opt (Entezari et al.; 2021: Ainsworth et al., 2022 Jordan et al. 2019)? We see Fig. 7 (right column, blue) that there is loss barrier (Vlaar Frankle 2022), i..., the loss increases from 8 to 0opt. Recent work (Entezari network units are taken into account. In simple model can easily test this, again, by computing the OLS solution for the second layer which will account for any permutation P E SN. since this can casily be absorbed into the OLS P3 = 3 estimate. We see in Fig. 7 (right column, green) that the loss is now approximately constant along the interpolation path again almost up to Oopt 2
al. 2021 suggested that this barrier may be removed if permulations bet
In summary, these interpolation experiments suggest that: i) Interpolating to the trained solution leads to faster (but ultimately smaller) decrease in loss (Fig. 7 left); ii) Most training progress can be recovered by simply optimising the second layer weights (Fig. middle); iii) The quality of the first layer features (i.e., the loss modulo OLS) is nearly constant on the triangle (0o, Osgd, Oopt), but with sharp transition close to opt (Fig. 7 middle, right). Thus, it seems that the model does not find the optimal solution during optimisation because it stays in a regime where the quality of the first layer feature space is nearly constant and where all training progress can be reduced to solving the (convex) optimisation of the second layer weights. There is only a very sharp and narrow peak where the first layer feature quality rapidly increases around the optimal solution. Reaching this point in a vast surrounding space of constant first layer quality may be rather difficult with gradient descent optimisation.
Thhis supports the idca of lincar mode connectivity modulo permutations (Ainsworth 2022 Later work Jordan el 2022) showed that. rescaling individual units maintain variance can also be important linear mode connectivity; again; this linear transformation can be subsumed into the OL.S B weights.
17

---

Published in Transactions on Machine Learning Research (02/2023)
A.2.3 Stronger Adversarial Attacks
MNIST Adversarial Robustness
1.0
o- = 1.00
0.9
a 1.39
0.8
a=1 = 1.95
Classification Accuracy
o 1 =271
0.7
a² =379
0.6
a = =528
R =7.7
0.5
a = 10.28
0.4
14.34
a 20.00
0.3
0.2
0.1
0.0
0
1
3
4
Perturbation norm: ||e||2
Figure 8: Neural Network Classification Robustness. Accuracy on MNIST as a function of (L2) adversarial perturbation size ([le]12) for TanH NN models with different levels of weight decay. Attacks computed with the stronger adaptive attacks implemented AutoAttack (Croce Hein 2020).
A.2.4
Multilayer Model for CIFAR10 Classification
CIFAR10 Adversarial Robustness
CIFAR10 White Noise Robustness
a-1 1.00
0.8
=154
0.7
Classification Accuracy
8ー1 3.68
Classification Accuracy
0.6
R² 5.69
0.6
0.5
o 8.79
13.57
0.5
0.4
0 20.96
32.37
0.4
0.3
55.000
0.3
0.2
0.1
0.00
0.05
0.10
0.15
0.25 0.30
10
Perturbation strength:
White noise standard deviation
Figure CIFAR10 Classification with ResNet Models. Accuracy on CIFAR10 as a function of (FGSM) perturbation strength for models with different length scales (a- 1); Right) Accuracy function of white noise standard deviation for the same models.
We using standard ResNet9 He et al. 2016 implementation based on this public repository (kaggle) Importantly, we exchange every fully connected layer with the construction proposed in this paper (section 3.3 i.e., normalising the weights and using TanH nonlinearities. Secondly, we apply the same normalisation to every input channel of convolutional layers and also change their activation functions to TanH nonlin- carities. We are using the same o for all layers. Notice, that these are specific design choices and that we leave it open future research explore different normalisation strategies or layer-dopendent smoothness parameters that could, potentially, improve the present results deeper (vision) models.
18

---

Published in Transactions on Machine Learning Research (02/2023)
A.2.5
Weight Decay Robustness Comparison
Adversarial Robustness
White Noise Robustness
1.0
1.0
Classifcation Accuracy
Classification Accuracy
weight decay
peegg decci
0.5
0.4
0.4
0.3
0.3
0.2
0.2
Perturbation strength:
White noise standard deviation
Adversarial Robustness
White Noise Robustness
1.0
Baseline TanH (a 1.001
Baseline TanH (c 1 1.00)
Best TanH
TanH
Baseline ReLU
0.8
Baseline 0.0)
Class ficatton Accuracy
0.7
Best RelU
flcatiniction Accurapy
0.7
Best ReLU
0.6
0.5
0.4
0.4
0.3
0.3
0.1
0.0
0.0
Perturbation strength:
White noise standard deviation
0.25
0.4
RelU
ReLU
0.20
Accuracy Gain
Accuracy Gain
0.2
0.10
0.05
Perturbation strength:
White noise standard deviation
Figure 10: Neural Network Classification Robustness. Top left.) Accuracy MNIST as function of (FGSM) perturbation strength for ReLU NN models with different levels of weight decay (wd). Top right) Accuracy on MNIST as a function of white noise standard deviation for the same models. Middle) Accuracy for baseline TanH (a 1.00) and ReLU (wd = 0.0) models for adversarial (left) and noise (right) perturbations, and best models, i.c., maximum performance over a (wnd) for every perturbation size. Bottom) Accuracy Gain, i.e, difference between baseline and best (middle) for both TanH and ReLU models.
19

---

Published in Transactions on Machine Learning Research (02/2023)
A.2.6
Density Dependent Fitting
Figure 11: TanH Neural Network Learning Addition on Gaussian Data. Data drawn from standard normal distribution (00, 1). Colorbar indicates the log of the loss function (equation
A.2.7
Inductive Bias Commutative Symmetry
Figure 12: Exploring Commutative Symmetry as Inductive Bias. TanH model (a-1 1.0) trained on disc addition task (Fig. D. however, forcing commutativity in the outputs by changing the network function f to faym (x) (2) 1/22 Colorbar indicates the log of the loss function (equation
20

---

Published in Transactions on Machine Learning Research (02/2023)
A.2.8 Results Addition and Multiplication
additio
multipiiation
09
49
47
-10
Log CORK
Log L03
00
02
02
-30
-35
13
111
addition
multplication
12-
37-
08-
Lgg L200
64
0.4
88
0.-
30
20
08
addition
maltipiiatiin
3.6
-35
2.4
2.4
12-
-10
Log Lo0o
24
-20
24
36
36-
-30
4.8
43
24
42
24
36
4.0
-35
244
422
3.
Figure 13: Addition and Multiplication for ReLU Network. Results of ReLU network trained on both
addition (Fig. and multiplication. Colorbar indicates the log of the loss function (equation 3
21

---

Published in Transactions on Machine Learning Research (02/2023)
mulbplication
0.4
44
02
-0
4.2
-1
04
-00
07
07
3.1
0.
47
24
02
00
42
04
07
09
111
49
47
24
42
0.0
04
07
69
multiplication
2.0
2.0
05
0.4
04
(og Lane
Log
-20
12
12
1.6
3.2
4.8
0.
00
24
08
122
16
4.5
3.2
4.5
44
04
08
122
multipiicaton
6.0
6.0
48
12
Log Loca
60
0.
36
40
43
60
2.4
1.2
2.
36
24
12
24
36
Figure 14: Addition and Multiplication for TanH Network. Results of TanH network, but without weight normalisation (equation 7 trained on both addition (Fig. 1 and multiplication. Colorbar indicates the log of the loss function (equation
22


Reviews:
[{"review_id": "2b5GzIXZXJ", "reviewer": "Reviewer_ksSX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- This submission motivates the need to investigate addition through the lens of neural algorithmic reasoning. The motivation for investigation on a simple problem is clear.\n- They illustrate that a simple two layer MLP is not capable of learning the addition task.\n- The authors connect their work to Gaussian processes and motivate a length scale for neural networks, which show benefits to smoothing the learned function and in adversarial robustness. \n\n## Weaknesses\n- An important scientific question that feels unsatisfactorily addressed after reading the paper is an explanation of why existing neural networks fail to generalize to solve addition, given that many such solutions exist in a subspace. It seems important to diagnose which part of the training algorithm fails (e.g. is it due to network initialization, learning rate, how data is generated, how the data is batched, or whether we use SGD or Adam?). A clear diagnosis for why simple MLPs fail could also be useful if it illustrated a more general failure cause of neural networks.\n- It's not clear how to set the variance hyperparameter, beyond retraining the network multiple times. Moreover, it seems important to justify the choice of scaling weights throughout training by comparing to something like weight decay.\n- Some experimental design choices need to be better motivated. The initial motivation studies the ReLU activation, but the authors later switch to the tanh activation. It's not clear from the existing experiments how much the choice of number of units matters e.g. what happens when N > 10000?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission seeks to understand the properties of multi layer perceptrons when solving the task of adding real numbers. They investigate generalization and extrapolation properties, contrasting them with those of gaussian processes. Motivated by GPs, they propose to regularize the variance of the weights at initialization and throughout training, which helps improve the functional smoothness.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would recommend acceptance if the authors conduct a more thorough investigation of why neural networks are not able to solve the addition task. I am not sure how exactly this could be done, but listed some potential ideas above in \"Weaknesses.\" Perhaps it is useful to understand how the behavior in function space evolves throughout training.\n\nIt would also be nice to have experiments with convolutional neural networks or attention-based networks (which have been shown to perform addition by Zhou et al.), which would also potentially help answer the questions of what ails simple MLPs, since we know more complicated architectures can solve the task. This would also show whether benefits to regularizing the variance scale to tasks and architectures more relevant to practitioners.", "broader_impact_concerns": "No broader impact concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OE0VkxHDPd", "reviewer": "Reviewer_oixe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. **Relevant research question**: Understanding the inductive bias of neural networks when solving algorithmic tasks is a topic of great interest for the ML/DL research community.\n2. **Educative illustration of error surface of NNs and GPs**: Personally, I find the collection of figures illustrating the error surface of the different NNs and GPs quite interesting and pedagogical. They clearly show what is the role of smoothness of the GP prior, how the representation of the NNs and GPs adapts to the training support, and how both hypotheses spaces favor solutions with connected paths of low error in the input space.\n3. **Clear writing**: Overall, this manuscript is very well-written and it is very easy to read.\n\n## Weaknesses\n\n1. **Anecdotal empirical evidence on extremely toy settings**:  In their introduction, the authors claim that the main weakness of the prior work on this topic was that \"...it provided solutions to a specific problem without any transferable insights into neural network functions as we obtain in this paper\". However, this work only provides a few empirical results on an extremely toy setting (e.g., two-layer neural networks on a single two-dimensional linear problem) that vastly differs from real practice. In fact, in Section 4, the authors directly claim as their main limitation that \"the proposed modifications are only applicale to simple two layer MLPs\".\n2. **Shallow observations with little added value over deeper prior work**: The previous weakness would not be major if the performed analysis provided a deep understanding of new phenomena. However, the observations described in this paper are only superficial and provide little additional insights over prior work:\n\t- The connections between GPs and NNs have been widely studied in previous literature (far beyond Neal (1996)).\n\t- There exist deep empirical and theoretical studies analyzing the extrapolation abilities of wide MLPs with far stronger insights than the ones described in this work, e.g., (Xu et al. 2021).\n\t- The toy sinusoidal regression studied in Section 3.3. has been a deep object of study of many works on the spectral bias of NNs, e.g., (Rahaman et al. 2019).\n3. **Lack of rigor in robustness study**: The connections between function smoothness and robustness are widely known in the robustness community and have been studied to a much greater extent than what is presented in this study. The minimal improvements in robustnesss presented in this work for MNIST on a trained MLP and evaluated using FGSM are not relevant for a community which in 2022, after having shown several times that weak robustness evaluations in toy settings did not provide meaningful research signals, has developed strong standardized benchmarks and codes of good practices to avoid pitfalls in robustness evaluations due to lack of rigor in research (Croce et al. 2021).\n4. **Inconsistent experimental settings**: I find intriguing that despite the simplicity of the problem under study, depending on the section, some experiments are arbitrarily performed using different sets of parameters. For example, why are the experiments on the ring performed with $N\\to\\infty$ , but the experiments on the disk for $N=256$? Similarly, some figures have colormaps and some others do not, and while the first half of the paper deals with ReLU networks, the second half uses $\\tanh$ and weight normalization.\n\n- Xu et al. \"How neural networks extrapolate:  from feedforward to graph neural networks\". ICLR 2021\n- Rahaman et al. \"On the spectral bias of neural networks\". ICML 2019.\n- Croce et all. \"RobustBench: a standardized adversarial robustness benchmark\". NeurIPS 2021\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work presents an empirical study on how a few design choices affect the performance of a two-layer neural network (NN) when trained to approximate the addition of two real numbers, i.e., ($f(x_1, x_2)=x_1+x_2$). The key result is a visualization of how the error surface of the NN changes depending on the level of overparameterization (i.e., width). The authors visually compare these plots with the error surfaces obtained by training Gaussian proceesses (GPs) with varying degrees of smoothness on the same task. They highlight the qualitative similarity between the complex error surfaces of the GPs with low smoothness and the heavily overparameterized NNS. As higher degrees of smoothness for GPs lead to better performance on the addition task, the authors borrow inspiration from the equivalence results between GPs and NNs in certain settings (e.g., infinfinite-width, certain initializations, and activation functions) and propose a simple wight normalization strategy which they claim can control the smoothness of a certain small class of NNs. The authors support this claim showing better qualitative results and a better visual correspondence between GPs and the NN in a few settings. Finally, the paper presents preliminary results on MNIST in which the proposed weight normalization strategy applied to a two-layer NN slightly improves the robustness of the NN to additive Gaussian perturbations and FGSM adversarial attacks.\n\nOverall the main contributions of this work are:\n1. A visual comparison between the error surfaces of two-layer NNs and GPs trained to approximate $f(x_1, x_2)=x_1, x_2$ varying the network width and the GP smoothness.\n2. A quantitative analysis of the generalization error in and out of the support of the input distribution when this support is a narrow ring. GPs generalize better on this ring, but also inside. Neither the NNs nor the GPs extrapolate outside the ring.\n3. A few experimental results showing that a GP-inspired weight normalization strategy can control the smoothness of a two-layer NN in a one-dimensional sinusoidal regression task, and the previously described two-dimensional addition task.\n4. Preliminary results on MNIST showing a slight increase in robustness to additive noise and weak adversarial attacks using this weight normalization strategy.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "Unfortunately, I believe a lot of work would be required for this manuscript to meet the bar of acceptance to TMLR. In particular, I think it is critical that the presented results go beyond the shallow anecdotal evidence they currently provide, and give new insights which add value on top of the prior literature. This could be done by thoroughly studying theoretically the simple setting described in this work (note that there exist many mathematical tools today to study two-layer MLPs on simple tasks, e.g., NTK limit, mean-field, etc), or scaling up the experiments to more interesting settings from a relevant practice, isolating new phenomena that could inform future practice or theory on deep learning. I personally believe such studies would be beyond the scope of what is possible in a major revision of this work.", "broader_impact_concerns": "I see no clear broad impact concerns stemming from this theoretical work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FJd9dPElh7", "reviewer": "Reviewer_yr94", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- Overall, the idea of studying simple problems where it is well known what is contained within the networks training set seems like a very promising way to measure OOD performance. Another example of this is [1].\n- The analysis leads the authors to propose and test a modification to neural networks such that the functions which they learn are more smooth. They do so by explicitly controlling the standard deviation of the weights. Experiments show that this helps learn smoother network functions, and improves robustness to adversarial attacks and noise.\n\nWeaknesses:\n- Experiments are on MNIST difficulty problems.\n- No discussion on normalization layers (e.g., [2]) which are present in neural networks today and how the proposed modification relates to those. While those apply to activations instead of weights, they presumably have similar effects.\n- No discussion on prior work relating smoothness and robustness (e.g., [3]).\n- No discussion on that modern neural networks can do addition (e.g., [4]).\n\nMinor:\n- Eq 6 contains a typo - should be $\\frac{sinh(x)}{cosh(x)}$.\n\n[1] https://arxiv.org/abs/2107.12580\n[2] https://arxiv.org/abs/1607.06450\n[3] https://proceedings.neurips.cc/paper/2019/file/ae3f4c649fb55c2ee3ef4d1abdb79ce5-Paper.pdf\n[4] https://arxiv.org/abs/2211.09066", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the problem of learning to add real numbers with MLPs. They find that neural networks are not able to do so in a way that generalizes, and find very rough decision boundaries as networks grow wider. They partially remedy this by making sure the functions learned by neural networks are smooth by controlling the standard deviation of the weights.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It would be great to see:\n- experiments on more difficult tasks\n- discussion on normalization layers in neural networks\n- discussion on how LLMs can do addition\n- discussion on prior work relating smoothness and robustness", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The claims of the paper could have been more significant if authors would try a larger range of architectures and normalization techniques. Authors could have also covered a few other algorithmic/arithmetic tasks. Nevertheless, I think the current limited setting is still interesting. I think perhaps understanding the relationship between this suggestion and known normalization techniques is the most important part that could be added to the paper.
Decision: Accept (poster)