SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[26] Haggai Maron et al. "Provably powerful graph networks". In: Advances in neural information processing systems 32 (2019).
[27] Chris Miller. "Tameness in expansions of the real field". In: Logic Colloquium. Vol. 1. 2005, pp. 281-316.
[28] Boris Samuilovich Mityagin. "The zero set of a real analytic function". In: Mathematical Notes 107.3-4 (2020), pp. 529-530.
[29] Guido F Montufar et al. "On the number of linear regions of deep neural networks". In: Advances in neural information processing systems 27 (2014).
[30] Christopher Morris et al. “TUDataset: A collection of benchmark datasets for learning with graphs”. In: ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+2020). 2020. arXiv: 2007.08663. URL: www.graphlearning.io.
[31] Christopher Morris et al. "Weisfeiler and leman go neural: Higher-order graph neural networks". In: Proceedings of the AAAI conference on artificial intelligence. Vol. 33. 01. 2019, pp. 4602-4609.
[32] Allan Pinkus. "Approximation theory of the MLP model in neural networks". In: Acta numerica 8 (1999), pp. 143-195.
[33] Sergey N Pozdnyakov and Michele Ceriotti. "Incompleteness of graph neural networks for points clouds in three dimensions". In: Machine Learning: Science and Technology 3.4 (2022), p. 045020.
[34] Charles R Qi et al. "Pointnet: Deep learning on point sets for 3d classification and segmentation". In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, pp. 652-660.
[35] Maithra Raghu et al. "On the Expressive Power of Deep Neural Networks". In: arxiv preprint arXiv:1606.05336 (2017).
[36] Walter Rudin. Real and Complex Analysis. McGraw-Hill Science/Engineering/Math, 1986.
[37] Nimrod Segol and Yaron Lipman. "On Universal Equivariant Set Networks". In: International Conference on Learning Representations. 2019.
[38] Guillaume Valette. On subanalytic geometry. June 2023.
[39] Lou Van den Dries and Chris Miller. "Geometric categories and o-minimal structures". In: (1996).
[40] Cédric Villani et al. Optimal transport: old and new. Vol. 338. Springer, 2009.
[41] Edward Wagstaff et al. "On the limitations of representing functions on sets". In: International Conference on Machine Learning. PMLR. 2019, pp. 6487-6494.
[42] Edward Wagstaff et al. "Universal approximation of functions on sets". In: Journal of Machine Learning Research 23.151 (2022), pp. 1-56.
[43] Peihao Wang et al. "Polynomial Width is Sufficient for Set Representation with High-dimensional Features". In: arXiv preprint arXiv:2307.04001 (2023).
[44] Keyulu Xu et al. "How Powerful are Graph Neural Networks?" In: International Conference on Learning Representations. 2018.
[45] Manzil Zaheer et al. "Deep sets". In: Advances in neural information processing systems 30 (2017).
[46] Yan Zhang, Jonathon Hare, and Adam Prügel-Bennett. "FSPool: Learning Set Representations with Featurewise Sort Pooling". In: International Conference on Learning Representations.
A Finite witness theorem
In this section, we state and prove the full version of the finite witness theorem (Theorem A.2), which is more general than Theorem 3.4 stated in the main text. Before laying out the formal definitions and proofs, we begin by describing the context of these results. While this section makes use of notions from algebraic geometry and real analytical functions, it is self-contained and most of it only requires knowledge of elementary calculus and some topology.
The finite witness theorem is essentially a tool for reducing an infinite, continuously parameterized family of constraints  $p(z; \theta) = 0 \forall \theta$  to a finite subset of constraints  $p(z; \theta_i) = 0, i = 1, \dots, m$ , defined by random parameters  $\theta_1, \ldots, \theta_m$ . This general approach seems to have originated from the famous proof of uniqueness for phase-retrieval measurements in [2]. In that work, functions  $p(z, \theta): \mathbb{C}^n \times \mathbb{C}^n \to \mathbb{R}$  of the form  $p(z; \theta) = |\langle z, \theta \rangle|$  were considered, and it was proved that a finite number of  $\sim 4n$  random measurements  $\theta_i$  are sufficient to uniquely determine a signal, up to unavoidable global phase ambiguity. The proof in [2] achieves this result by showing that the values  $p(z; \theta)$  for all possible  $\theta$ s are sufficient to determine the signal uniquely, and then uses a real algebraic-geometric and dimension-counting argument to show that this continuous family of measurements can be replaced by a finite subset, defined by  $m \sim 4n$  random vectors (witnesses)  $\{\theta_i\}_{i=1}^m$ , without losing information.
In [9], the authors provide a generalization of the results in [2], by defining conditions under which a continuously parameterized family of functions  $p(z; \theta)$  that fully determines  $z$  up to equivalence, can be replaced by a finite subset  $p(z; \theta_i)$ ,  $i = 1, \dots, m$  determined by random witness-vectors  $\theta_i$ . This theorem is based on similar arguments as those used in [2], and on similar assumptions required for machinery from real algebraic geometry. Specifically, sets in [9] are assumed to be semialgebraic, which means that they are finite unions of subsets of  $\mathbb{R}^D$  that can be defined by polynomial equalities and inequalities. This class of sets includes, for example, finite unions of spheres, balls, and convex polyhedra. The functions in the theorem are assumed to be semialgebraic as well, which means that their graphs are semialgebraic sets. This class of functions includes polynomials, as well as rational functions and piecewise-linear functions.
The main theorem in [9] can be essentially<sup>3</sup> formulated as
Theorem A.1. Let  $\mathbb{M}$  be a semialgebraic set of dimension  $D$ , and let  $F: \mathbb{M} \times \mathbb{R}^{D_{\theta}} \to \mathbb{R}$  be a semialgebraic function. Define the set
$$
\mathcal {N} = \left\{\boldsymbol {z} \in \mathbb {M} \mid F (\boldsymbol {z}; \boldsymbol {\theta}) = 0, \forall \boldsymbol {\theta} \in \mathbb {R} ^ {D _ {\boldsymbol {\theta}}} \right\}
$$
and assume that for all  $z\in \mathbb{M}\setminus \mathcal{N}$  , we have that
$$
\dim \left\{\boldsymbol {\theta} \in \mathbb {R} ^ {D _ {\boldsymbol {\theta}}} \mid F (\boldsymbol {z}; \boldsymbol {\theta}) = 0 \right\} \leq D _ {\boldsymbol {\theta}} - 1, \tag {12}
$$
then for Lebesgue almost every  $\pmb{\theta}^{(1)},\dots,\pmb{\theta}^{(D + 1)}$
$$
\mathcal {N} = \left\{\boldsymbol {z} \in \mathbb {M} | F (\boldsymbol {z}; \boldsymbol {\theta} ^ {(i)}) = 0, \forall i = 1, \dots D + 1 \right\}.
$$
The notion of dimension used in condition (12) and throughout this section is the Hausdorff dimension, explained in Appendix A.4 below.
Theorem A.1 is similar to the simple version of the finite witness theorem (Theorem 3.4), with four notable differences: (1) The domain  $\mathbb{M}$  in Theorem 3.4 is a countable union of affine sets; this is not, in general, a semialgebraic set, and thus does not qualify for the conditions of Theorem A.1. (2) The function  $F$  in Theorem 3.4 is analytic. Analytic functions are not necessarily semialgebraic. (3) Theorem A.1 requires the extra condition (12), which does not appear in Theorem 3.4. This condition is not required in Theorem 3.4 because with analytic functions, it is always satisfied; however, it will be required in our full version of the theorem, since it admits a more general class of functions. (4) Theorem A.1 deals with sets of the form  $\{z|F(z,\pmb {\theta}) = 0,\forall \pmb {\theta}\}$  while Theorem 3.4 deals with sets of the form  $\{(x,y)|F(x,\pmb {\theta}) = F(\pmb {y},\pmb {\theta}),\forall \pmb {\theta}\}$ . This difference is not essential and can be handled by the change of variables  $z = (x,y)$  and  $\tilde{F} (\pmb {x},\pmb {y},\pmb {\theta}) = F(\pmb {x},\pmb {\theta}) - F(\pmb {y},\pmb {\theta})$ .
3To obtain this theorem from the formulation in [9], use the change of variables  $z = (x,y)$  and  $F(z;\theta) = p(x;\theta) - p(y;\theta)$ . The formulation in [9] makes some additional requirements on  $F$  which are relevant to the specific applications considered there, but going through the details of the proof shows that it is sufficient for proving Theorem A.1 as stated here.
To address the first two differences, we shall generalize Theorem A.1 to support a large class of domains  $\mathbb{M}$  and functions  $F$ . The admissible domains shall include a vast class of sets, among which are countable unions of semialgebraic sets, as well as all open sets, and sets defined by analytic, rather than polynomial, equations. The admissible functions  $F$  shall include semialgebraic functions, analytic functions, and many other types of functions. To achieve this goal, we use results from the study of o-minimal structures (see below), which aim at finding families of sets that have the same tame properties as semialgebraic sets.
A good starting point to achieve this generalization is to consider the family of globally subanalytic sets (formally defined below). This family contains all semialgebraic sets and is an o-minimal system; consequently, it is possible to generalize Theorem A.1 so that the domain  $\mathbb{M}$  can include all globally subanalytic sets, and the function  $F$  could be any globally subanalytic function (meaning that the graph of  $F$  is a globally subanalytic set). However, this still will not allow for a general analytic  $F$ , nor for  $\mathbb{M}$  to include countable unions of affine sets.
To address this, we will show that an analogue to Theorem A.1 holds even when considering countable unions of globally subanalytic sets. We call such sets  $\sigma$ -subanalytic sets. To the best of our knowledge, such sets have not been studied to date. The family of  $\sigma$ -subanalytic sets is not an o-minimal structure, since it not closed under taking complements. However, it is closed under linear projections, countable unions, finite intersections and Cartesian products. Moreover,  $\sigma$ -subanalytic sets are countable unions of  $C^\infty$  manifolds. We find that these properties are sufficient to generalize Theorem A.1 and obtain a finite witness theorem for the  $\sigma$ -subanalytic category.
Clearly, the class of  $\sigma$ -subanalytic sets is larger than the class of globally subanalytic sets. In particular, it includes countable unions of semialgebraic sets. This enables the domain  $\mathbb{M}$  in the theorem statement to be a countable union of affine spaces, as in Theorem 3.4 from the main text. This also implies that any open set is  $\sigma$ -subanalytic, since it is a countable union of open balls — which are semialgebraic sets.
As for the function  $F$ , our theorem admits all  $\sigma$ -subanalytic functions: functions whose graph is  $\sigma$ -subanalytic. This class includes all analytic functions, as well as all semialgebraic functions. Moreover, we show below that it is closed under composition and other elementary operations.
We now state the full version of the finite witness theorem.
Theorem A.2 (Finite Witness Theorem, full version). Let  $\mathbb{M} \subseteq \mathbb{R}^p$ ,  $\mathbb{W} \subseteq \mathbb{R}^q$  be  $\sigma$ -subanalytic sets of dimension  $D$  and  $D_{\theta}$  respectively. Let  $F: \mathbb{M} \times \mathbb{W} \to \mathbb{R}$  be a  $\sigma$ -subanalytic function. Define the set
$$
\mathcal {N} = \{\boldsymbol {z} \in \mathbb {M} \mid F (\boldsymbol {z}; \boldsymbol {\theta}) = 0, \forall \boldsymbol {\theta} \in \mathbb {W} \}.
$$
Suppose that for all  $z\in \mathbb{M}\setminus \mathcal{N}$
$$
\dim \left\{\boldsymbol {\theta} \in \mathbb {W} \mid F (\boldsymbol {z}; \boldsymbol {\theta}) = 0 \right\} \leq D _ {\boldsymbol {\theta}} - 1. \tag {13}
$$
Then for generic  $\left(\pmb{\theta}^{(1)},\dots ,\pmb{\theta}^{(D + 1)}\right)\in \mathbb{W}^{D + 1}$
$$
\mathcal {N} = \left\{\boldsymbol {z} \in \mathbb {M} \mid F (\boldsymbol {z}; \boldsymbol {\theta} ^ {(i)}) = 0, \forall i = 1, \dots D + 1 \right\}. \tag {14}
$$
Moreover, if  $\mathbb{W}$  is an open and connected subset of  $\mathbb{R}^q$ , and  $F(z; \theta)$  is analytic as a function of  $\theta$  for all fixed  $z \in \mathbb{M}$ , then condition (13) is not required, as it is automatically satisfied.
The notion of dimension in (13) is the Hausdorff dimension (discussed below), and the term generic means that the set of  $\left(\pmb{\theta}^{(1)},\dots,\pmb{\theta}^{(D + 1)}\right)$  for which (14) fails is a subset of  $\mathbb{W}^{D + 1}$  whose Hausdorff dimension is strictly lower than  $\dim \left(\mathbb{W}^{D + 1}\right)$ . In particular, in the common case where  $\mathbb{W} = \mathbb{R}^q$ , or is an open subset of  $\mathbb{R}^q$ , we have that the theorem holds for almost any  $\left(\pmb{\theta}^{(1)},\dots,\pmb{\theta}^{(D + 1)}\right)$ , with respect to the Lebesgue measure on  $\mathbb{R}^{q(D + 1)}$ .
We now begin to rigorously define the notions discussed in this section and then prove Theorem A.2. In Appendices A.1 to A.4, we construct the theoretical framework step by step. Then, in Appendix A.5 we prove the theorem, and in Appendix A.6 we show that the simple version of the theorem (Theorem 3.4) follows from the full version. Finally, we present in Appendix A.6 several corollaries to the theorem.
A.1 Definitions and Background
A.1.1 Globally subanalytic sets: an o-minimal structure
We begin by defining analytic functions, subanalytic sets and functions, and various other related concepts that will be required for our discussion. These definitions are taken from [19] and [38].
Definition A.3 (Analytic function [19]). Let  $U \subseteq \mathbb{R}^D$  be an open set. We say that  $f: U \to \mathbb{R}$  is analytic if for all  $z \in U$ , there exists an open ball  $V \subseteq U$  centered at  $z$ , and  $(a_{\alpha})_{\alpha \in \mathbb{N}^D}$  such that
$$
f (\boldsymbol {y}) = \sum_ {\alpha \in \mathbb {N} ^ {D}} a _ {\alpha} (\boldsymbol {y} - \boldsymbol {z}) ^ {\alpha} \quad \forall \boldsymbol {y} \in V, \tag {15}
$$
and the power series in (15) converges absolutely.
Our next goal is to define the family of globally subanalytic sets, which contains all semialgebraic sets; moreover, it is an  $o$ -minimal structure, meaning that it shares the essential tame properties of semialgebraic sets. This requires some preliminary definitions:
Definition A.4 (Semianalytic sets [38]). A subset  $E \subseteq \mathbb{R}^n$  is called semianalytic if it is locally defined by finitely many real analytic equalities and inequalities. Namely, for each  $a \in \mathbb{R}^n$ , there is a neighborhood  $U$  of  $a$ , and real analytic functions  $f_{ij}, g_{ij}$  on  $U$ , where  $i = 1, \ldots, r$  and  $j = 1 \ldots s_i$ , such that
$$
E \cap U = \bigcup_ {i = 1} ^ {r} \bigcap_ {j = 1} ^ {s _ {i}} \left\{\boldsymbol {z} \in U \mid g _ {i j} (\boldsymbol {z}) > 0 \text {a n d} f _ {i j} (\boldsymbol {z}) = 0 \right\}. \tag {16}
$$
Example A.5. As shown in Example 1.1.2 in [38], the graph of the analytic function  $f:(0,1)\to \mathbb{R}$  defined by  $f(x) = \sin (1 / x)$  is not a semianalytic set. This is because there is no neighborhood of  $a = (0,0)$  for which (16) holds. On the other hand, if  $B\subseteq \mathbb{R}^n$  is a closed ball and  $f:B\to \mathbb{R}$  can be extended to an analytic function in an open set containing  $B$ , then the graph of  $f$  (as a function defined on  $B$ ) will be semianalytic.
The example above suggests that the class of semianalytic sets may be too restrictive for our purposes. More importantly, linear projections of semianalytic sets may not be semianalytic [38], so the family of semianalytic sets does not form an o-minimal system. This can be remedied by defining globally semianalytic sets (which don't form an o-minimal system) and then using these sets to define globally subanalytic sets (which do form an o-minimal system):
Definition A.6 (Globally semianalytic sets [38]). A subset  $Z \subseteq \mathbb{R}^n$  is globally semianalytic if  $V_{n}(Z)$  is a semianalytic subset of  $\mathbb{R}^n$ , where  $V_{n}: \mathbb{R}^{n} \to (-1,1)^{n}$  is the homeomorphism defined by
$$
V _ {n} \left(\boldsymbol {z} = \left(z _ {1}, \dots , z _ {n}\right)\right) = \left(\frac {z _ {1}}{\sqrt {1 + | \boldsymbol {z} | ^ {2}}}, \dots , \frac {z _ {n}}{\sqrt {1 + | \boldsymbol {z} | ^ {2}}}\right).
$$
Globally semianalytic sets can be thought of as semianalytic sets that remain semianalytic when 'compactified' by  $V_{n}$ . This is useful to rule out bad behaviour as  $z$  goes out to infinity. Important examples of globally semianalytic sets include all bounded semianalytic sets and all semialgebraic sets [38].
Globally semianalytic sets are still not closed under linear projection. Their projections are called globally subanalytic sets:
Definition A.7 (Globally subanalytic sets [38]). A subset  $E \subseteq \mathbb{R}^n$  is globally subanalytic if it can be presented as a linear projection of a globally semianalytic set; more precisely, if there exists a globally semianalytic set  $Z \subseteq \mathbb{R}^{n+p}$  such that  $E = \pi(Z)$ , with  $\pi: \mathbb{R}^{n+p} \to \mathbb{R}^n$  being the projection operator that omits the last  $p$  coordinates while leaving the remaining coordinates unchanged.
Globally semianalytic sets are by definition globally subanalytic. Globally subanalytic sets do form an  $o$ -minimal structure. In particular, they have the following properties:
Proposition A.8 (Properties of globally subanalytic sets [38]). Let  $A, B \subseteq \mathbb{R}^D$  and  $C \subseteq \mathbb{R}^M$  be globally subanalytic sets. Then:
1.  $\mathbb{R}^D\setminus A$  is globally subanalytic.
2.  $A\cup B$  is globally subanalytic.
3.  $A\cap B$  is globally subanalytic.
4.  $A\times C$  is globally subanalytic.
5. If  $\pi : \mathbb{R}^D \to \mathbb{R}^L$  is a linear projection, then  $\pi(A)$  is globally subanalytic.
6.  $A$  is a finite union of  $C^\infty$  manifolds.
The first property in Proposition A.8 is proven in Gabrielov's Complement Theorem; see Theorem 1.8.8 in [38]. The last property is a weaker version of Theorem 1.2.3 in [38]. The remaining properties are proved<sup>4</sup> in [38, Basic properties 1.1.8].
A.2  $\sigma$ -subanalytic sets
As mentioned above, the fact that globally subanalytic sets form an o-minimal structure is already sufficient to prove a finite witness theorem for this class. However, this class is still too restrictive for our purposes, as it does not allow us to support arbitrary analytic functions  $F$ , as the following example shows:
Example A.9. Analogously to semialgebraic functions, we say that a function is globally subanalytic if its graph is globally subanalytic. Consider the function  $\sin (x)$  defined on the real line. Then by Example 1.1.7 in [38], this function is not globally subanalytic.
Fortunately, we find that a finite witness theorem can be proven for a larger class of sets: countable unions of globally subanalytic sets. We name such sets  $\sigma$ -subanalytic sets. To the best of our knowledge, this class of sets has not been studied to date<sup>5</sup>. The class of  $\sigma$ -subanalytic sets is rather large: we will show below that it contains all open sets, while the classes discussed previously do not (see Example A.12 below). This is illustrated in the Venn diagram on the left-hand side of Figure 3. Correspondingly, the class of  $\sigma$ -subanalytic functions is larger than the classes of functions considered previously, and in particular it contains any analytic function  $F$  defined on any open set. This is illustrated in the Venn diagram on the right-hand side of Figure 3, and will be discussed rigorously in Appendix A.3 below. Thus, moving to  $\sigma$ -subanalytic sets and functions is the crucial step that enables us to achieve our goal of proving a finite witness theorem for analytic functions (though the theorem covers a much larger class).
We now define  $\sigma$ -subanalytic sets and study their properties.
Definition A.10 ( $\sigma$ -subanalytic sets). We say that a subset  $A \subseteq \mathbb{R}^D$  is  $\sigma$ -subanalytic if it is a countable union of globally subanalytic subsets of  $\mathbb{R}^D$ .
The class of  $\sigma$ -subanalytic sets is rather large. For example, any open set in  $\mathbb{R}^D$  can be written as a countable union of open balls. Since open balls are semialgebraic sets, it follows that all open sets are  $\sigma$ -subanalytic. Also, any semianalytic set is  $\sigma$ -subanalytic, since semianalytic sets are a countable union of bounded semianalytic sets, and these are globally subanalytic.
Properties of  $\sigma$ -subanalytic sets
As the following proposition shows,  $\sigma$ -subanalytic sets inherit most of the properties enjoyed by globally subanalytic sets, described in Proposition A.8.
Proposition A.11 (Properties of  $\sigma$ -subanalytic sets). Assume that  $A, B \subseteq \mathbb{R}^D$  and  $C \subseteq \mathbb{R}^M$  are  $\sigma$ -subanalytic sets, then
1.  $A \cup B$  is  $\sigma$ -subanalytic. More generally, any countable union of  $\sigma$ -subanalytic sets is  $\sigma$ -subanalytic.
<sup>4</sup>Regarding the fifth property in Proposition A.8, note that it does not follow directly from property 1 in [38] which only discusses a special class of projections, but rather from property 4 in [38] which states that the image of a globally subanalytic set under a globally subanalytic mapping is globally subanalytic. Any linear projection is a semialgebraic mapping, and hence a globally subanalytic mapping.
Model theorists and analytic geometers have researched other structures that are larger than o-minimal structures and enjoy some of their tame properties; see, e.g., [39, 27].
Figure 3: Left: The class of semialgebraic sets is contained in the class of globally subanalytic sets, which in turn is contained in the class of  $\sigma$ -subanalytic sets — on which we focus in this paper. One of the advantages of this larger class is that it contains all open sets, whereas the two smaller classes do not. Right: The classes of semialgebraic, globally subanalytic, and  $\sigma$ -subanalytic functions are related in the same way. The class of  $\sigma$ -subanalytic functions is the only one of the three that contains all analytic functions.
2.  $A\cap B$  is  $\sigma$  -subanalytic.
3.  $A\times C$  is  $\sigma$  -subanalytic.
4. If  $\pi : \mathbb{R}^D \to \mathbb{R}^L$  is a linear projection, then  $\pi(A)$  is a  $\sigma$ -subanalytic set.
5.  $A$  is a countable union of  $C^\infty$  manifolds.
Proof. Let  $A, B \subseteq \mathbb{R}^D$  be  $\sigma$ -subanalytic sets. Then  $A = \cup_{n \in \mathbb{N}} A_n$  and  $B = \cup_{m \in \mathbb{N}} B_m$ , where each  $A_n$  and  $B_m$  is globally subanalytic. We then have
1. A countable union of  $\sigma$ -analytic sets, each of which being itself a countable union of globally subanalytic set, is clearly a countable union of globally subanalytic sets, and therefore is  $\sigma$ -subanalytic by definition.
2. We have that
$$
A \cap B = \left(\cup_ {n \in \mathbb {N}} A _ {n}\right) \cap \left(\cup_ {m \in \mathbb {N}} B _ {m}\right) = \cup_ {(n, m) \in \mathbb {N} ^ {2}} \left(A _ {n} \cap B _ {m}\right)
$$
and  $A_{n}\cap B_{m}$  is globally subanalytic as the intersection of globally subanalytic sets.
3. The set  $A\times \mathbb{R}^{M}$  can be presented as
$$
A \times \mathbb {R} ^ {M} = \left(\cup_ {n \in \mathbb {N}} A _ {n}\right) \times \mathbb {R} ^ {M} = \cup_ {n \in \mathbb {N}} \left(A _ {n} \times \mathbb {R} ^ {M}\right),
$$
with  $A_{m}$  being globally subanalytic. Since globally subanalytic sets are closed to Cartesian products, each  $A_{n} \times \mathbb{R}^{M}$  is globally subanalytic, and their countable union is  $\sigma$ -subanalytic. Finally,
$$
A \times C = (A \times \mathbb {R} ^ {M}) \cap (\mathbb {R} ^ {D} \times C),
$$
which is  $\sigma$ -subanalytic by part 2.
4. Follows from globally subanalytic sets being closed to projections, since
$$
\pi \left(\cup_ {n \in \mathbb {N}} A _ {n}\right) = \cup_ {n \in \mathbb {N}} \left(\pi (A _ {n})\right).
$$
5. A  $\sigma$ -subanalytic set is a countable union of globally subanalytic sets, each of which is a finite union of  $C^\infty$  manifolds. Therefore, it is a countable union of  $C^\infty$  manifolds.