SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Other Architectures. While we consider fully connected two-layer networks as the most important case, we are also interested in  $\exists \mathbb{R}$ -hardness results for other network architectures. Specifically, fully connected three-layer neural networks and convolutional neural networks are interesting. While it is hard to imagine that more complicated architectures are easier to train, a formal proof of this intuition would strengthen our result and show that  $\exists \mathbb{R}$ -completeness is a robust phenomenon, in other words independent of a choice of a specific network type.
5 Further Related Work
Complexity of Training Neural Networks. It is well-known that minimizing the training error of a neural network is a computationally hard problem for a large variety of activation functions and architectures [80]. For ReLU networks, NP-hardness, parameterized hardness and inapproximability
results have been established even for the simplest possible architecture consisting of only a single ReLU neuron [14, 27, 35, 40]. While all these results require non-constant input-dimension, Froese and Hertrich [34] showed that it is also NP-hard to train a two-layer ReLU network with input dimension two and output dimension one. On the positive side, the seminal algorithm by Arora, Basu, Miany and Mukherjee [6] solves empirical risk minimization for two-layer ReLU networks and one-dimensional output to global optimality, placing the problem in NP. It was later extended to a more general class of loss functions by Froese, Hertrich and Niedermeier [35]. The running time is exponential in the number of neurons in the hidden layer and in the input dimension, but polynomial in the number of data points if the former two parameters are considered to be a constant. This NP-containment of TRAIN-F2NN with one-dimensional output is in sharp contrast to our  $\exists \mathbb{R}$ -completeness result for two-dimensional outputs.
While minimizing training and generalization errors are different problems, the hardness of the former also imposes challenges on the latter. Strategies to circumvent hardness from the perspective of learning theory include allowing improper learners, restricting the type of weights allowed in a neural network, or imposing assumptions on the underlying distribution. For example, Chen, Klivans and Meka [20] show fixed-parameter tractability of learning a ReLU network under some assumptions including Gaussian data and Lipschitz continuity of the network. We refer to [7, 19, 28, 38, 41, 39] as a non-exhaustive list of other results about (non-)learnability of ReLU networks in different settings.
Expressivity of ReLU Networks. It is essential for our reduction to understand the classes of functions representable by certain ReLU network architectures. So-called universal approximation theorems state that a single hidden layer (with arbitrary width) is already sufficient to approximate every continuous function on a bounded domain with arbitrary precision [22, 52]. However, deeper networks require much fewer neurons to reach the same expressive power, yielding a potential theoretical explanation of the dominance of deep networks in practice [6, 31, 45, 47, 58, 67, 69, 72, 85, 86, 89]. Other related work includes counting and bounding the number of linear regions [46, 64, 65, 68, 69, 79], classifying the set of functions exactly representable by different architectures [6, 26, 44, 49, 50, 51, 66, 92], or analyzing the memorization capacity of ReLU networks [88, 90, 91]. Huchette, Muñoz, Serra, and Tsay [53] provide a survey on the interactions of neural networks and polyhedral geometry, including implications on training, verification, and expressivity.
Existential Theory of the Reals. The complexity class  $\exists \mathbb{R}$  gains its importance by numerous important algorithmic problems that have been shown to be complete for this class in recent years. The name  $\exists \mathbb{R}$  was introduced by Schaefer in [73] who also pointed out that several NP-hardness reductions from the literature actually implied  $\exists \mathbb{R}$ -hardness. For this reason, several important  $\exists \mathbb{R}$ -completeness results were obtained before the need for a dedicated complexity class became apparent.
Common features of  $\exists \mathbb{R}$ -complete problems are their continuous solution space and the nonlinear relations between their variables. Important  $\exists \mathbb{R}$ -completeness results include the realizability of abstract order types [63, 83] and geometric linkages [74], as well as the recognition of geometric segment [57, 60], unit-disk [55, 61], and ray intersection graphs [18]. More results appeared in the graph drawing community [30, 32, 59, 75], regarding polytopes [29, 71], the study of Nash-equilibria [8, 10, 11, 36, 76], matrix factorization [21, 78, 81, 82], or continuous constraint satisfaction problems [62]. In computational geometry, we would like to mention the art gallery problem [2, 84] and covering polygons with convex polygons [1].
Recently, the community started to pay more attention to higher levels of the real polynomial hierarchy, which also capture several interesting algorithmic problems [12, 16, 23, 30, 54, 77].
6 Proof Ideas
To show  $\exists \mathbb{R}$ -hardness, we reduce from a classical version of ETR called ETR-INV. An instance of ETR-INV consists of real variables and a conjunction of constraints in these variables, where each constraint is either and addition constraint  $X + Y = Z$  or an inversion constraint  $X \cdot Y = 1$ . Given such an instance, we construct a TRAIN-F2NN instance that models the variables and whose training corresponds to satisfying all addition and inversion constraints.
Variables. A natural candidate for encoding variables are the weights and biases of the neural network. However, those did not prove to be suitable for our purposes. The main problem with using the parameters of the neural network as variables is that the same function can be computed by many neural networks with different combinations of these parameters. We are not aware of an easy way to normalize the parameters.
To circumvent this issue, we work with the functions representable by fully connected two-layer neural networks directly. We frequently make use of the geometry of these functions. For now, it is only important to understand that each hidden ReLU neuron encodes a continuous piecewise linear function with exactly two pieces (both of constant gradient). These two pieces are separated by a so-called breakline. Now, if we have  $m$  middle neurons, their individually encoded functions add up such that the function computed by the whole neural network is a continuous piecewise linear function with at most  $m$  breaklines. Between these breaklines all pieces of the function have constant gradient.
To keep things simple for now, let us first consider a neural network with only one input and one output neuron. We place a series of data points  $(x_{i},y_{i})\in \mathbb{R}^{2}$  as seen in Figure 2. All continuous piecewise linear functions  $f(\cdot ,\Theta)$  computed by a neural network with only four hidden neurons (i.e., only four breaklines and therefore at most five pieces) that fit these data points exactly must be very similar. In fact, they can only differ in one degree of freedom, namely the slope of the piece going through point  $p$ . In our construction, this slope represents the value of a variable. The whole set of data points enforcing this configuration is called a variable gadget.
Figure 2: The value of  $f(\cdot, \Theta)$  is fixed (black part), except for the segment through data point  $p$ . The red, orange and blue segments are just three out of uncountably many possibilities. Its slope can be used to encode a real-valued variable.
Linear Dependencies. The key insight for encoding constraints between variables is that we can relate the values of several variable gadgets by a data point: By placing a data point  $p$  at a location where several variable gadgets overlap, each of the variable gadgets contributes its part towards fitting  $p$ . The exact contribution of each variable gadget depends on its slope. Consequently, if one variable gadget contributes more, the others have to proportionally contribute less. This enforces linear dependencies between different variable gadgets and can be used to design addition and copy gadgets.
To be able to intersect multiple variable gadgets, we need a second input dimension. Each variable gadget as seen in Figure 2 is now extended into a stripe in  $\mathbb{R}^2$ , with Figure 2 showing only an orthogonal cross section of this stripe. See Figure 3 for two intersecting variable gadgets. Much of the technical difficulties lie in the subtleties to enforce the presence of multiple (possibly intersecting) gadgets using a finite number of data points.
To construct an addition gadget we need to be able to encode linear relations between three variables. In  $\mathbb{R}^2$ , three lines (or stripes) usually do not intersect in a single point. Thus, for addition we have to carefully place them to guarantee such a single intersection point. This might involve creating copies of a variable gadget, which is a linear relation between just two variable gadgets.
Inversion Constraint. Within only a single output dimension, we are not able to encode nonlinear constraints [6]. We therefore add a second output dimension, which implies that the neural network
Figure 3: Two variable gadgets (slopes of the blue and red regions encode the values). Point  $p$  lies in the intersection of both and can encode a linear relationship between them.
represents two functions  $f^1(\cdot, \Theta)$  and  $f^2(\cdot, \Theta)$ . Consequently, we are allowed to use data points with two different output labels, one for each output dimension.
One important observation is that the locations of breaklines of  $f = f(\cdot, \Theta) = (f_1(\cdot, \Theta), f_2(\cdot, \Theta))$  only depend on the weights and biases in the first layer of the neural network. Thus, every breakline is present in both functions  $f^1$  and  $f^2$ , except if the weight to the corresponding output neuron is zero. This connects the two output dimensions in a way that we can encode nonlinear constraints.
We define an inversion gadget (realizing the constraint  $X \cdot Y = 1$ ), which also corresponds to a stripe in  $\mathbb{R}^2$ . For simplicity, we only show a cross section here, see Figure 4. In each output dimension individually, the inversion gadget looks exactly like a variable gadget. The inversion gadget can therefore be understood as a variable gadget that carries two values.
We prove that by allowing only five breaklines in total, a function  $f$  can only fit all data points exactly if  $f^1$  and  $f^2$  share three of their four breaklines (while both having one "exclusive" breakline each). This enforces a nonlinear dependency between the slopes of  $f^1$  and  $f^2$ . By choosing the right parameters, this nonlinear relation models exactly an inversion constraint.
Figure 4: Data points  $p$  and  $q$  have different labels in the two output dimensions, enforcing that the slopes of the red and the blue pieces are related via a nonlinear dependency.
Reduction. Let us illustrate the reduction by giving a simple example. Note that this is not yet the "complete picture", which follows in the supplementary material. We start with the ETR-INV instance that is to decide whether the following sentence
$$
\exists X _ {1}, X _ {2}, X _ {3}, X _ {4} \in \mathbb {R}: \left(X _ {1} + X _ {2} = X _ {3}\right) \wedge \left(X _ {1} + X _ {3} = X _ {4}\right) \wedge \left(X _ {1} \cdot X _ {4} = 1\right) \wedge \left(X _ {4} \cdot X _ {3} = 1\right)
$$
is true. This instance has four variables  $X_{1}, X_{2}, X_{3}, X_{4}$  and four constraints: two additions and two inversions. Recall that every gadget corresponds to a stripe in the input space  $\mathbb{R}^2$ . See Figure 5 for the following construction (for better readability the stripes are drawn as lines).
- For each variable we add a variable gadget. We place all of these such that their corresponding stripes are parallel and do not overlap, see the horizontal lines in Figure 5.
- For each addition constraint we introduce three new variable gadgets, one per involved variable. We place these such that they have a common intersection point while also intersecting their corresponding variable gadget. In Figure 5, see the two bundles to the left. A data point at the triple intersection enforces the addition constraint, while data points labelled  $\bullet_{=}$  encode that the values of the two intersecting variable gadgets are equal.
- Lastly, we add an inversion gadget for each inversion constraint and place it such that it intersects the variable gadgets of the two involved variables. See the two dashed lines in
Figure 5. Data points labelled  $\bullet^{=1}$  ( $\bullet^{=2}$ ) enforce that the values stored in the first (second) output dimension are equal.
Figure 5: Overview of the global arrangement of the gadgets.
To see that above reduction is correct, assume that the sentence in the ETR-INV instance is indeed true. Then there exists real values for the four variables and these values can be used as the slopes of their corresponding variable gadgets. By the correctness of the individual gadgets (which is to be proven in the supplementary material) it follows that each data point is fit exactly.
Conversely, if all data points are fit exactly, then the correctness of the gadgets implies that the slopes of the variable gadgets give a solution to the ETR-INV instance. Intuitively, this holds because the addition and inversion constraints are encoded exactly by the gadgets of our construction.
Outline of the Supplementary Material. We sketch  $\exists \mathbb{R}$ -membership of TRAIN-F2NN in Appendix A before turning to  $\exists \mathbb{R}$ -hardness in Appendix B. As Appendix B is by far the longest part of the paper, it is further broken down into several parts: After a short reduction overview, we describe suitable abstractions to make defining and analyzing the gadgets easier. Then we introduce the gadgets formally one by one and in isolation before finally putting everything together. We finish the section by proving correctness of our reduction. Finally, we prove algebraic universality in Appendix C.
Christoph Hertrich is supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreements ScaleOpt-757481 and ForEFront-615640). Tillmann Miltzow is generously supported by the Netherlands Organisation for Scientific Research (NWO) under project numbers 016.Veni.192.250 and VI.Vidi.213.150. Simon Weber is supported by the Swiss National Science Foundation under project no. 204320.
[1] Mikkel Abrahamsen. Covering Polygons is Even Harder. In Nisheeth K. Vishnoi, editor, 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pages 375-386, 2022. doi:10.1109/FOCS52979.2021.00045.
[2] Mikkel Abrahamsen, Anna Adamaszek, and Tillmann Miltzow. The Art Gallery Problem is  $\exists \mathbb{R}$ -complete. Journal of the ACM, 69(1):1-70, 2022. doi:10.1145/3486220.
[3] Mikkel Abrahamsen, Linda Kleist, and Tillmann Miltzow. Training Neural Networks is ER-complete. In Marc A. Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, and Jennifer W Vaughan, editors, Advances in Neural Information Processing Systems (NeurIPS 2021), volume 34, 2021.
[4] Mikkel Abrahamsen and Tillmann Miltzow. Dynamic Toolbox for ETRINV. arXiv preprint, 2019. arXiv:1912.08674.
[5] Reyan Ahmed, Felice De Luca, Sabin Devkota, Stephen Kobourov, and Mingwei Li. Graph Drawing via Gradient Descent,  $(GD)^{2}$ . In David Auber and Pavel Valtr, editors, Graph Drawing
[90] Chulhee Yun, Suvrit Sra, and Ali Jadbabaie. Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity. In Hanna Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché Buc, Emily Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems (NeurIPS 2019), volume 32, 2019.
[91] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding Deep Learning (Still) Requires Rethinking Generalization. Communications of the ACM, 64(3):107-115, 2021. doi:10.1145/3446776.
[92] Liwen Zhang, Gregory Naitzat, and Lek-Heng Lim. Tropical Geometry of Deep Neural Networks. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning (ICML 2018), volume 80 of Proceedings of Machine Learning Research, pages 5824-5832, 2018.