SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In comparison to Proposition A.8, we see that we have lost two properties by moving from globally subanalytic sets to countable unions thereof: Firstly, a  $\sigma$ -subanalytic set is a countable union of manifolds rather than a finite one. Secondly, the complement of a  $\sigma$ -subanalytic set may not be  $\sigma$ -subanalytic, as our next example shows. As it turns out, the properties noted in Proposition A.11 are sufficient for our purposes.
Example A.12. The Cantor set is not  $\sigma$ -subanalytic, since it has fractal Hausdorff dimension and thus is not a countable union of manifolds. However, the complement of the Cantor set, which we denote by  $U$ , is an open set and hence is  $\sigma$ -subanalytic. This shows that the class of  $\sigma$ -subanalytic sets is not closed to taking complements, as well as countable intersections: this is since the Cantor set is a countable intersection of finite unions of intervals, which are  $\sigma$ -subanalytic. The set  $U$  is also an example of an open set that is not semialgebraic or globally subanalytic, since these classes of sets are closed to complements, and do not contain the Cantor set.
A.3  $\sigma$ -subanalytic functions
We now define the class of functions our theorem can admit as the function  $F$ :  $\sigma$ -subanalytic functions.
Definition A.13 ( $\sigma$ -subanalytic function). Let  $\mathbb{M} \subseteq \mathbb{R}^D$  be a  $\sigma$ -subanalytic set. We say that  $f: \mathbb{M} \to \mathbb{R}^L$  is a  $\sigma$ -subanalytic function if its graph is a  $\sigma$ -subanalytic subset of  $\mathbb{R}^{D + L}$ .
If  $\mathbb{M} \subseteq \mathbb{R}^D$  is a semialgebraic set and  $f: \mathbb{M} \to \mathbb{R}^L$  is a semialgebraic function, then the graph of  $f$  is semialgebraic, and thus is  $\sigma$ -subanalytic. Therefore, a semialgebraic function  $f$  defined on a semialgebraic domain  $\mathbb{M}$  is a  $\sigma$ -subanalytic function. A similar result holds also for analytic functions defined on open sets, as shown in the following lemma:
Lemma A.14. If  $U \subseteq \mathbb{R}^D$  is open and  $f: U \to \mathbb{R}^L$  is analytic, then  $f$  is  $\sigma$ -subanalytic.
Proof. First recall that since  $U$  is open, it is a  $\sigma$ -subanalytic set. Next, we can write  $U$  as a countable union of closed balls  $B_{i}$ , and then the graph of  $f$  can be written as a countable union of sets of the form
$$
\left\{\left(\boldsymbol {x}, \boldsymbol {y}\right) \in B _ {i} \times \mathbb {R} ^ {L} \mid \boldsymbol {y} = f (\boldsymbol {x}) \right\}.
$$
These subsets of  $U\times \mathbb{R}^L$  are compact and semianalytic, and hence they are globally subanalytic.
Properties of  $\sigma$ -subanalytic functions
The following proposition shows that the class of  $\sigma$ -subanalytic functions is closed under composition and elementary arithmetic operations. In particular, this allows for functions that combine analytic and semialgebraic functions via compositions and elementary arithmetic operations.
Proposition A.15. Let  $A \subseteq \mathbb{R}^a$  and  $B \subseteq \mathbb{R}^b$  be  $\sigma$ -subanalytic sets.
1. (Composition) If  $f: A \to B$  and  $g: B \to \mathbb{R}^c$  are  $\sigma$ -subanalytic functions, then  $g \circ f$  is a  $\sigma$ -subanalytic function.
2. (Concatenation) If  $f: A \to \mathbb{R}^m$  and  $g: A \to \mathbb{R}^n$  are  $\sigma$ -subanalytic functions, then  $h: A \to \mathbb{R}^{m + n}$  given by  $h(x) = (f(x), g(x))$  is a  $\sigma$ -subanalytic function.
3. (Addition and multiplication) If  $f, g: A \to \mathbb{R}$  are  $\sigma$ -subanalytic functions, then  $f + g$  and  $f \cdot g$  are  $\sigma$ -subanalytic functions.
Proof. 1. To prove that  $g \circ f$  is  $\sigma$ -subanalytic, we need to prove that its graph is a  $\sigma$ -subanalytic set. Indeed, this graph is of the form
$$
\left\{\left(\boldsymbol {x}, \boldsymbol {z}\right) \in A \times \mathbb {R} ^ {c} \mid g (f (\boldsymbol {x})) = \boldsymbol {z} \right\},
$$
which is the projection of the intersection
$$
\{(x, y, z) \in A \times B \times \mathbb {R} ^ {c} \mid f (x) = y \} \cap \{(x, y, z) \in A \times B \times \mathbb {R} ^ {c} \mid g (y) = z \}
$$
onto the  $(\pmb{x},\pmb{z})$  coordinates. The two sets in the intersection above are Cartesian products of the graph of  $f$  (respectively  $g$ ) with a  $\sigma$ -subanalytic set, and hence are  $\sigma$ -subanalytic.
2. The graph of  $h$  is the intersection of the sets  $\{(x, y, z) | f(x) = y\}$  and  $\{(x, y, z) | g(x) = z\}$ . Each of these two sets is a Cartesian product of a Euclidean space with the graph of a  $\sigma$ -subanalytic function.
3. The functions  $f + g$  and  $f \cdot g$  can be presented as the composition of the function  $\mathbb{R}^a \ni \pmb{x} \mapsto (f(\pmb{x}), g(\pmb{x}))$ , which is  $\sigma$ -subanalytic, with the addition or multiplications functions. Thus, the claim follows.
A.4 Dimension
The proof of Theorem A.2 is based on a dimension-counting argument. Accordingly, we will need an appropriate definition of dimension for  $\sigma$ -subanalytic sets. It is convenient to work with the Hausdorff dimension, which is defined for every subset of a Euclidean space  $\mathbb{R}^D$ , and coincides with the standard notions of dimension for vector spaces and manifolds. The definition of Hausdorff dimension can be found, e.g., in [14]. For our purposes, we will only need to use some of its properties, stated below.
Proposition A.16 (Taken from [14]). The Hausdorff dimension has the following properties:
1. The Hausdorff dimension of a  $k$ -dimensional  $C^1$  submanifold of  $\mathbb{R}^D$  is  $k$ .
2. If  $B_{1}, B_{2}, \ldots$  are subsets of  $\mathbb{R}^{D}$  then
$$
\dim \left(\cup_ {n \in \mathbb {N}} B _ {n}\right) = \max  _ {n \in \mathbb {N}} \dim \left(B _ {n}\right).
$$
3. For any  $B \subseteq \mathbb{R}^D$  and Lipschitz function  $f: B \to \mathbb{R}^C$ ,  $\dim(f(B)) \leq \dim(B)$ .
Recall that  $\sigma$ -subanalytic sets are countable unions  $A = \cup_{n \in \mathbb{N}} A_n$  of  $C^\infty$  manifolds. As we saw, their Hausdorff dimension is just the maximal dimension of all manifolds  $A_n$ . We shall now see that for such sets, the Hausdorff dimension has two nice properties that do not hold for general sets. These properties will be used in the proof of the finite witness theorem.
The first property is the dimension of Cartesian products. For two general sets  $A, B$ , it is not always true that  $\dim(A \times B) = \dim(A) + \dim(B)$  (see [14]). However, this does hold if  $A$  and  $B$  are countable unions of manifolds.
Lemma A.17. If  $A, B$  are countable unions of  $C^1$  sub-manifolds of  $\mathbb{R}^a$  and  $\mathbb{R}^b$ , then
$$
\dim (A \times B) = \dim (A) + \dim (B).
$$
Proof. By assumption  $A = \cup_{n\in \mathbb{N}}A_n$  and  $B = \cup_{m\in \mathbb{N}}B_m$ , where  $A_{n},B_{m}$  are  $C^1$  manifolds. We have that
$$
A \times B = \cup_ {(n, m) \in \mathbb {N} ^ {2}} \left(A _ {n} \times B _ {m}\right)
$$
is again a countable union of the product manifolds  $A_{n} \times B_{m}$ , which are known to be of dimension  $\dim(A_{n}) + \dim(B_{m})$  (see e.g. [22]). It follows that the dimension of  $A \times B$  is equal to
$$
\max  _ {(n, m) \in \mathbb {N} ^ {2}} (\dim (A _ {n}) + \dim (B _ {m})) = \max  _ {n \in \mathbb {N}} \dim (A _ {n}) + \max  _ {m \in \mathbb {N}} \dim (B _ {m}) = \dim (A) + \dim (B).
$$
The second nice property of the Hausdorff dimension for countable unions of manifolds, which does not always hold for general sets, is dimension conservation, in the following sense.
Lemma A.18 (Dimension Conservation). Let  $S \subseteq \mathbb{R}^{D_1}$  be a countable union of  $C^\infty$  manifolds and  $f: \mathbb{R}^{D_1} \to \mathbb{R}^{D_2}$  a  $C^\infty$  function. Then
$$
\dim (S) \leq \dim (f (S)) + \max  _ {t \in f (S)} \dim (f ^ {- 1} (t) \cap S). \tag {17}
$$
For failure of dimension conservation for general sets see, e.g., the discussion in [12].
A good intuition for the concept of dimension conservation comes from the linear case, in which  $f: \mathbb{R}^{D_1} \to \mathbb{R}^{D_2}$  is a linear transformation and  $S \subseteq \mathbb{R}^{D_1}$  is a linear subspace. In this case, the Rank-Nullity Theorem from elementary linear algebra states that
$$
\begin{array}{l} \dim (S) = \dim (f (S)) + \dim (\operatorname {K e r n e l} (f) \cap S) \tag {18} \\ = \dim (f (S)) + \dim (f ^ {- 1} (0) \cap S). \\ \end{array}
$$
The same statement also holds if we replace  $f^{-1}(0)$  by  $f^{-1}(t)$ , with  $t$  being any point in the image of  $f$ . Thus, for linear transformations, any loss of dimension when moving from a set  $S$  to its image, is accounted for by the dimension of the fibers above points in the image. The image conservation in Lemma A.18 is a weaker since it is only an inequality, but it applies to the more general class of  $\sigma$ -semianalytic functions.
Proof of Lemma A.18. We know that  $S$  is a countable union of manifolds, and at least one of these, which we denote by  $S_{1}$ , has the maximal dimension  $\dim(S_{1}) = \dim(S)$ . This equality shall enable us to argue about  $\dim(S)$  by arguing only about  $\dim(S_{1})$ .
Let  $r$  be the maximal rank of the differential of  $f_{|S_1}$ . Fix some  $s_1 \in S_1$  so that the differential of  $f_{|S_1}$  at  $s_1$  has rank  $r$ . The set of  $s \in S_1$  whose differential has rank  $r$  is open, and so there is a neighborhood  $V$  of  $s_1$ , such that  $V \cap S_1$  is a manifold of the same dimension as  $S_1$ , and the restriction of  $f$  to  $V \cap S_1$  has constant rank  $r$ . Consider the restriction of  $f$  to  $V \cap S_1$ , denoted by  $f_{|V \cap S_1}$ . By the constant rank theorem [21],  $f_{|V \cap S_1}$  is a projection, up to a diffeomorphic change of coordinates. Since diffeomorphisms of manifolds preserve dimensions, and projections (like all linear transformations) satisfy dimension conservation as in (18), this implies that locally we have dimension conservation: for all  $t \in f(V \cap S_1)$ ,
$$
\dim (V \cap S _ {1}) = \dim \left(f (V \cap S _ {1})\right) + \dim \left(f ^ {- 1} (t) \cap V \cap S _ {1}\right).
$$
$$
\begin{array}{l} \dim (S) = \dim (S _ {1}) = \dim (V \cap S _ {1}) \\ = \dim (f (V \cap S _ {1})) + \dim (f ^ {- 1} (t) \cap V \cap S _ {1}), \quad \forall t \in f (V \cap S _ {1}) \\ \leq \dim \left(f (S)\right) + \max  _ {t \in f (V \cap S _ {1})} \dim \left(f ^ {- 1} (t) \cap S\right) \\ \leq \dim (f (S)) + \max  _ {t \in f (S)} \dim \left(f ^ {- 1} (t) \cap S\right). \\ \end{array}
$$
We conclude our discussion of the Hausdorff dimension with a natural corollary which will be useful later on for our proof of Proposition 3.6.
Corollary A.19. Let  $\mathbb{M} \subseteq \mathbb{R}^D$  be a  $\sigma$ -subanalytic set, and let  $F: \mathbb{M} \to \mathbb{R}^L$  a  $\sigma$ -subanalytic function. Then the graph and image of  $F$  are  $\sigma$ -subanalytic sets, and
$$
\dim \left(\operatorname {g r a p h} (F)\right) = \dim (\mathbb {M}), \qquad \dim (F (\mathbb {M})) \leq \dim (\mathbb {M}).
$$
Proof. Note that by the definition of  $\sigma$ -subanalytic functions, the graph of  $f$  is a  $\sigma$ -subanalytic set. We begin by showing that it has the same dimension as  $\mathbb{M}$ . Denote by  $\pi$  the projection from
$$
\operatorname {g r a p h} (F) = \left\{\left(\boldsymbol {x}, F (\boldsymbol {x})\right) \mid \boldsymbol {x} \in \mathbb {M} \right\}
$$
onto the first coordinate. Firstly, since  $\pi$  is a Lipschitz function, it cannot increase the Hausdorff dimension, so
$$
\dim (\operatorname {g r a p h} (F)) \geq \dim (\pi (\operatorname {g r a p h} (F))) = \dim (\mathbb {M}).
$$
In the other direction, note that for every  $(\pmb{x}, F(\pmb{x})) \in \mathrm{graph}(F)$  we have that
$$
\pi^ {- 1} (\pi (\boldsymbol {x}, F (\boldsymbol {x}))) = \pi^ {- 1} (\boldsymbol {x}) = \left\{(\boldsymbol {x}, F (\boldsymbol {x})) \right\}
$$
is a set containing a single point, and thus has dimension zero. Therefore, using Lemma A.18 we have that
$$
\dim (\operatorname {g r a p h} (F)) \leq \dim (\pi (\operatorname {g r a p h} (F))) + 0 = \dim (\mathbb {M}),
$$
and so we have shown that  $\mathbb{M}$  and the graph of  $F$  have the same dimension. Finally,  $F(\mathbb{M})$  is the projection of the graph of  $F$  onto the second coordinate and so it is a  $\sigma$ -subanalytic set. Moreover, as projections cannot increase the Hausdorff dimension, we obtain that
$$
\dim (F (\mathbb {M})) \leq \dim (\operatorname {g r a p h} (F)) = \dim (\mathbb {M}),
$$
which concludes the proof of the corollary.
A.5 Proof of the finite witness theorem
We are finally ready to prove the full version of the finite witness theorem.
Proof of Theorem A.2. Let  $m = D + 1$ . Let  $F_{m}:\mathbb{M}\times \mathbb{W}^{m + 1}\to \mathbb{R}^{m + 1}$  be given by
$$
F _ {m} \left(\boldsymbol {z}, \boldsymbol {\theta} _ {0}, \boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m}\right) = \left(F \left(\boldsymbol {z}; \boldsymbol {\theta} _ {0}\right), \dots , F \left(\boldsymbol {z}; \boldsymbol {\theta} _ {m}\right)\right).
$$
Since  $F_{m}$  is a concatenation of  $\sigma$ -subanalytic functions (each such function is the composition of  $F$  with a different linear projection), by Proposition A.15 it is also  $\sigma$ -subanalytic. Therefore, the graph of  $F_{m}$ , given by  $\mathcal{A}_{m}$  below, is  $\sigma$ -subanalytic:
$$
\mathcal {A} _ {m} = \left\{\left(\boldsymbol {z}, \boldsymbol {\theta} _ {0}, \boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m}, s _ {0}, s _ {1}, \dots , s _ {m}\right) \in \mathbb {M} \times \mathbb {W} ^ {m + 1} \times \mathbb {R} ^ {m + 1} \mid F (\boldsymbol {z}; \boldsymbol {\theta} _ {i}) = s _ {i}, \forall i = 0, \dots , m \right\}.
$$
Next, we can intersect  $\mathcal{A}_m$  with the semialgebraic set defined by the equations  $s_0 \neq 0, s_1 = s_2 = \ldots = s_m = 0$  to obtain the  $\sigma$ -subanalytic set
$$
\begin{array}{l} \tilde {\mathcal {A}} _ {m} = \left\{\left(\boldsymbol {z}, \boldsymbol {\theta} _ {0}, \boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m}, F (\boldsymbol {z}; \boldsymbol {\theta} _ {0}), 0, \dots , 0\right) \in \mathbb {M} \times \mathbb {W} ^ {m + 1} \times \mathbb {R} ^ {m + 1} \mid \right. \\ F (\boldsymbol {z}; \boldsymbol {\theta} _ {0}) \neq 0 \text {a n d} F (\boldsymbol {z}; \boldsymbol {\theta} _ {i}) = 0, \forall i = 1, \dots , m \}. \\ \end{array}
$$
We can then remove by projection the last  $m + 1$  coordinates and the  $\theta_0$  coordinate, to obtain the set
$$
\mathcal {B} _ {m} = \left\{\left(\boldsymbol {z}, \boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m}\right) \in \mathbb {M} \times \mathbb {W} ^ {m} \mid \boldsymbol {z} \in \mathbb {M} \backslash \mathcal {N} \text {a n d} F (\boldsymbol {z}; \boldsymbol {\theta} _ {i}) = 0, \forall i = 1, \dots , m \right\},
$$
which is  $\sigma$ -subanalytic as well. Let  $\pi$  and  $\pi_{\theta}$  denote the projections
$$
\pi (\boldsymbol {z}, \boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m}) = \boldsymbol {z}, \quad \pi_ {\boldsymbol {\theta}} (\boldsymbol {z}, \boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m}) = (\boldsymbol {\theta} _ {1}, \dots , \boldsymbol {\theta} _ {m}).
$$
The set  $\pi_{\pmb{\theta}}(\mathcal{B}_m) \subseteq \mathbb{W}^m$  is exactly the set of  $m$ -tuples  $(\pmb{\theta}_1, \dots, \pmb{\theta}_m)$  that are not sufficient to determine  $\mathcal{N}$ . To prove the theorem, we thus need to show that the dimension of  $\pi_{\pmb{\theta}}(\mathcal{B}_m)$  is lower than  $\dim(\mathbb{W}^m) = mD_{\pmb{\theta}}$ . We do so by bounding the dimension of  $\mathcal{B}_m$ .
Let  $\mathbf{z}_0 \in \pi(\mathcal{B}_m)$ . The set  $\pi^{-1}(\mathbf{z}_0) \cap \mathcal{B}_m$  can be presented as the Cartesian product
$$
\pi^ {- 1} \left(\boldsymbol {z} _ {0}\right) \cap \mathcal {B} _ {m} = \left\{\boldsymbol {z} _ {0} \right\} \times \underbrace {\Theta_ {\boldsymbol {z} _ {0}} \times \Theta_ {\boldsymbol {z} _ {0}} \times \dots \times \Theta_ {\boldsymbol {z} _ {0}}} _ {m \text {t i m e s}}, \tag {19}
$$
$$
\Theta_ {\boldsymbol {z} _ {0}} = \{\boldsymbol {\theta} \in \mathbb {W} \mid F (\boldsymbol {z} _ {0}; \boldsymbol {\theta}) = 0 \}.
$$
The set  $\Theta_{z_0}$  is also  $\sigma$ -subanalytic: the graph of  $F$
$$
\{(z, \boldsymbol {\theta}, s) \in \mathbb {M} \times \mathbb {W} \times \mathbb {R} \mid s = F (z; \theta) \}
$$
is  $\sigma$ -subanalytic by assumption; intersecting it with the space  $z = z_0$ ,  $s = 0$ , and projecting to the  $\theta$  coordinate, yields the set  $\Theta_{z_0}$ . Since  $\Theta_{z_0}$  is  $\sigma$ -subanalytic, it is a countable union of manifolds. By the theorem assumption (13), the Hausdorff dimension of  $\Theta_{z_0}$  satisfies
$$
\dim (\Theta_ {z _ {0}}) \leq D _ {\theta} - 1.
$$
Therefore by Lemma A.17 and Equation (19),
$$
\dim \left(\pi^ {- 1} \left(\boldsymbol {z} _ {0}\right) \cap \mathcal {B} _ {m}\right) = m \dim \left(\Theta_ {\boldsymbol {z} _ {0}}\right) \leq m D _ {\boldsymbol {\theta}} - m. \tag {20}
$$
So far,  $z_0$  is only assumed to be an arbitrary point in  $\pi(\mathcal{B}_m)$ . Now, fix  $z_0 \in \pi(\mathcal{B}_m)$  to be a maximizer of the left-hand side of Equation (20). By the dimension conservation Lemma A.18,
$$
\begin{array}{l} \dim (\mathcal {B} _ {m}) \leq \dim (\pi (\mathcal {B} _ {m})) + \max  _ {\boldsymbol {z} \in \pi (\mathcal {B} _ {m})} \dim \left(\pi^ {- 1} (\boldsymbol {z}) \cap \mathcal {B} _ {m}\right) \\ \stackrel {(a)} {=} \dim (\pi (\mathcal {B} _ {m})) + \dim \left(\pi^ {- 1} (\boldsymbol {z} _ {0}) \cap \mathcal {B} _ {m}\right) \\ \stackrel {(b)} {\leq} \dim (\pi (\mathcal {B} _ {m})) + m D _ {\boldsymbol {\theta}} - m \tag {21} \\ \stackrel {(c)} {\leq} D + m D _ {\boldsymbol {\theta}} - m = m D _ {\boldsymbol {\theta}} - 1, \\ \end{array}
$$
where (a) holds by the choice of  $z_0$ , (b) is by Equation (20), and (c) holds since  $\pi(\mathcal{B}_m) \subseteq \mathbb{M}$  and  $\dim(\mathbb{M}) = D$ . Since  $\pi_\theta$  is Lipschitz, Equation (21) implies that
$$
\dim (\pi_ {\boldsymbol {\theta}} (\mathcal {B} _ {m})) \leq \dim (\mathcal {B} _ {m}) \leq m D _ {\boldsymbol {\theta}} - 1
$$
Now, suppose that  $\mathbb{W}$  is an open and connected subset of  $\mathbb{R}^q$ . We need to show that (13) holds for all  $z \in \mathbb{M} \setminus \mathcal{N}$ . Fix  $z \in \mathbb{M} \setminus \mathcal{N}$  and let  $f: \mathbb{W} \to \mathbb{R}$  be given by  $f(\theta) = F(z; \theta)$ . Since  $z \notin \mathcal{N}$ , there exists some  $\theta$  for which  $F(z; \theta) \neq 0$ . Hence,  $f$  is an analytic function defined on an open connected domain, and is not identically zero. Therefore, its zero-set must be dimension-deficient (see [28, Proposition 3]) and so we obtain (13).
This concludes the proof of the theorem.
A.6 Corollaries
In this subsection, we present several corollaries to the finite witness theorem, and show that the simple version Theorem 3.4 follows from it as a special case.
A useful application of the finite witness theorem, which often arises in invariant learning, is when one wishes to assert that two points  $\pmb{x}, \pmb{y} \in \mathbb{M}$  are indistinguishable by a parametric family of functions  $F(\cdot; \pmb{\theta})$ ; namely, that  $F(\pmb{x}; \pmb{\theta}) = F(\pmb{y}; \pmb{\theta})$  for all  $\pmb{\theta} \in \mathbb{W}$ . As the following corollary shows, this can be asserted almost surely by testing whether  $F(\pmb{x}; \pmb{\theta}) = F(\pmb{y}; \pmb{\theta})$  on  $2\dim(\mathbb{M}) + 1$  random  $\pmb{\theta}$ s.
Corollary A.20 (Separation by Finite Witnesses). Let  $\mathbb{M} \subseteq \mathbb{R}^p$ ,  $\mathbb{W} \subseteq \mathbb{R}^q$  be  $\sigma$ -subanalytic sets of dimension  $D$  and  $D_{\theta}$  respectively. Let  $F: \mathbb{M} \times \mathbb{W} \to \mathbb{R}$  be a  $\sigma$ -subanalytic function. Define the set
$$
\mathcal {N} = \left\{\left(\boldsymbol {x}, \boldsymbol {y}\right) \in \mathbb {M} \times \mathbb {M} \mid F (\boldsymbol {x}; \boldsymbol {\theta}) = F (\boldsymbol {y}; \boldsymbol {\theta}), \forall \boldsymbol {\theta} \in \mathbb {W} \right\}.
$$
Suppose that for all  $(\pmb{x},\pmb{y})\in \mathbb{M}\times \mathbb{M}\setminus \mathcal{N}$
$$
\dim \left\{\boldsymbol {\theta} \in \mathbb {W} \mid F (\boldsymbol {x}; \boldsymbol {\theta}) = F (\boldsymbol {y}; \boldsymbol {\theta}) \right\} \leq D _ {\boldsymbol {\theta}} - 1. \tag {22}
$$
Then for generic  $\left(\pmb{\theta}^{(1)},\dots ,\pmb{\theta}^{(2D + 1)}\right)\in \mathbb{W}^{D + 1}$
$$
\mathcal {N} = \left\{\left(\boldsymbol {x}, \boldsymbol {y}\right) \in \mathbb {M} \times \mathbb {M} \mid F \left(\boldsymbol {x}; \boldsymbol {\theta} ^ {(i)}\right) = F \left(\boldsymbol {y}; \boldsymbol {\theta} ^ {(i)}\right), \forall i = 1, \dots 2 D + 1 \right\}. \tag {23}
$$
Moreover, if  $\mathbb{W}$  is an open and connected subset of  $\mathbb{R}^q$ , and  $F(\pmb{x};\pmb{\theta})$  is analytic as a function of  $\pmb{\theta}$  for all fixed  $\pmb{x} \in \mathbb{M}$ , then condition (22) is not required, as it is automatically satisfied.
Proof. Set  $\tilde{\mathbb{M}} = \mathbb{M}\times \mathbb{M}$ , and define  $G:\tilde{\mathbb{M}}\times \mathbb{W}\to \mathbb{R}$  by
$$
G \left(\left(\boldsymbol {x}, \boldsymbol {y}\right); \boldsymbol {\theta}\right) = F (\boldsymbol {x}; \boldsymbol {\theta}) - F (\boldsymbol {y}; \boldsymbol {\theta}).
$$
Then  $\tilde{\mathbb{M}}$  is a  $\sigma$ -subanalytic set of dimension  $\dim(\tilde{\mathbb{M}}) = 2D$ , and  $G$  is a  $\sigma$ -subanalytic function. Moreover, if  $F(\boldsymbol{x}; \boldsymbol{\theta})$  is analytic as a function of  $\boldsymbol{\theta}$  for any fixed  $\boldsymbol{x} \in \mathbb{M}$ , then the function  $\boldsymbol{\theta} \mapsto G((\boldsymbol{x}, \boldsymbol{y}); \boldsymbol{\theta})$  is analytic for any fixed  $(\boldsymbol{x}, \boldsymbol{y}) \in \tilde{\mathbb{M}}$ . The result follows from applying Theorem A.2 to  $G$ .
Using Corollary A.20, we can now easily prove Theorem 3.4 and Proposition 3.6 from the main text, which we restate here for convenience:
Theorem 3.4. (Finite Witness Theorem, simple version) Let  $\mathbb{M} \subseteq \mathbb{R}^L$  be a countable union of affine sets, each of which is of dimension  $\leq D$ . Let  $\mathbb{W} \subseteq \mathbb{R}^{D_\theta}$  be open and connected. Let  $F(x; \pmb{\theta}): \mathbb{M} \times \mathbb{W} \to \mathbb{R}$  be an analytic function. Then for almost any  $(\pmb{\theta}^{(1)}, \dots, \pmb{\theta}^{(2D + 1)}) \in \mathbb{W}^{2D + 1}$ , the following set equality holds:
$$
\begin{array}{l} \{(x, y) \in \mathbb {M} \times \mathbb {M} \mid F (x; \boldsymbol {\theta}) = F (y; \boldsymbol {\theta}), \forall \boldsymbol {\theta} \in \mathbb {W} \} = \\ \left\{\left(\boldsymbol {x}, \boldsymbol {y}\right) \in \mathbb {M} \times \mathbb {M} \mid F \left(\boldsymbol {x}; \boldsymbol {\theta} ^ {(i)}\right) = F \left(\boldsymbol {y}; \boldsymbol {\theta} ^ {(i)}\right), \forall i = 1, \dots 2 D + 1 \right\}. \\ \end{array}
$$
Proof of Theorem 3.4. Since affine sets are semialgebraic, countable unions of affine sets are  $\sigma$ -subanalytic. Therefore, Theorem 3.4 follows from Corollary A.20. Note that by Corollary A.20, the claim holds for all  $(\theta^{(1)},\ldots ,\theta^{(2D + 1)})$  except for possibly a dimension-deficient subset of  $\mathbb{W}^{D + 1}$ , which is slightly stronger than the 'for almost any' statement in the theorem.
We now prove Proposition 3.6 from the main text.
Proof. Let  $\mathcal{V} = f\left(\mathbb{R}^d\right) \subseteq \mathbb{R}^L$  be the image of  $f$ . Since  $f$  is injective, it naturally induces an injective pushforward map of measures  $f_*: \mathcal{M}_{\leq n}(\mathbb{R}^d) \to \mathcal{M}_{\leq n}(\mathcal{V})$ , defined by
$$
\mathcal {M} _ {\leq n} (\mathbb {R} ^ {d}) \ni \sum_ {i = 1} ^ {n} w _ {i} \delta_ {x _ {i}} \mapsto \sum_ {i = 1} ^ {n} w _ {i} \delta_ {f (x _ {i})} \in \mathcal {M} _ {\leq n} (\mathcal {Y}).
$$