SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\det  \left(\mathbf {J} _ {h} (\mathbf {z})\right) = \sum_ {\sigma \in \mathcal {P} _ {d _ {z}}} \left(\operatorname {s i g n} (\sigma) \prod_ {j = 1} ^ {d _ {z}} \mathbf {J} _ {g} (\mathbf {z}) _ {\sigma (j), j}\right) \neq 0, \tag {12}
$$
where  $\mathcal{P}_{d_z}$  is the set of all  $d_{z}$ -permutations.
Equation 12 indicates that there exists  $\sigma \in \mathcal{P}_{d_z}$ , such that  $\prod_{j=1}^{d_z} \mathbf{J}_g(\mathbf{z})_{\sigma(j),j} \neq 0$ . Equivalently, we have
$$
\forall j \in [ d _ {z} ], (\sigma (j), j) \in \mathcal {T}. \tag {13}
$$
Therefore, for a specific  $j_{\hat{s}} \in \{d_c + 1, \dots, d_z\}$ , it follows that  $(\sigma(j_{\hat{s}}), j_{\hat{s}}) \in \mathcal{T}$ . Further, Equation 11 shows that for any  $i_x \in [d_x]$ , s.t.,  $(i_x, \sigma(j_{\hat{s}})) \in \mathcal{G}$ , we have  $\{i_x\} \times \mathcal{T}_{\sigma(j_{\hat{s}}),:} \subseteq \hat{\mathcal{G}}$ . Together, it follows that
$$
\left(i _ {x}, \sigma \left(j _ {\hat {s}}\right)\right) \in \mathcal {G} \Longrightarrow \left(i _ {x}, j _ {\hat {s}}\right) \in \hat {\mathcal {G}}. \tag {14}
$$
Equation 14 suggests that the column  $\sigma(j_{\hat{s}})$  of the true generating function support  $\mathcal{G}$  is included in the column  $j_{\hat{s}}$  of the estimated generating function support  $\hat{\mathcal{G}}$ . Together with Assumption 1-iii., it follows that
$$
\sum_ {j _ {\hat {s}} \in \left\{d _ {c} + 1, \dots , d _ {z} \right\}} \left\| \hat {\mathcal {G}} _ {., j _ {\hat {s}}} \right\| _ {0} \geq \sum_ {j _ {s} \in \left\{d _ {c} + 1, \dots , d _ {z} \right\}} \| \mathcal {G} _ {., j _ {s}} \| _ {0}, \tag {15}
$$
where the permutation  $\sigma(\cdot)$  connects the indices of  $\mathbf{s}$  and those of  $\hat{\mathbf{s}}$ . We note that Equation 15 is a lower-bound of the objective Equation 3, which can be attained by a minimizer  $\hat{g} = g$ .
In the following, we show by contradiction that the support of  $\mathbf{J}_h^{-1}(\mathbf{z})$  does not contain  $(j_c,j_{\hat{s}})$ , for any  $j_{c}\in [d_{c}]$  and any  $j_{\hat{s}}\in \{d_c + 1,\dots ,d_c\}$ , i.e.,  $(j_{c},j_{\hat{s}})\notin \mathcal{T}$ .
We suppose that a specific  $(j_c', j_{\hat{s}}') \in \mathcal{T}$ , where  $j_c' \in [d_c]$  and any  $j_{\hat{s}}' \in \{d_c + 1, \dots, d_c\}$ . We note that the argument for Equation 14 also applies to  $(j_1, j_2) \in \mathcal{T}$  for any  $j_1, j_2 \in [d_z]$ . Thus, we would have
$$
\left(j _ {c} ^ {\prime}, j _ {\tilde {s}} ^ {\prime}\right) \in \mathcal {T} \Longrightarrow \left(i _ {x}, j _ {\tilde {s}} ^ {\prime}\right) \in \hat {\mathcal {G}}, \quad \forall i _ {x} \in \{i \in [ d _ {x} ]: \left(i _ {x}, j _ {c} ^ {\prime}\right) \in \mathcal {G} \}. \tag {16}
$$
It would follow that
$$
\begin{array}{l} \sum_ {j _ {\hat {s}} \in \{d _ {c} + 1, \dots , d _ {z} \} \backslash \{j _ {\hat {s}} ^ {\prime} \}} \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {s}}} \right\| _ {0} + \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {s}} ^ {\prime}} \right\| _ {0} \geq \sum_ {j _ {\hat {s}} \in \{d _ {c} + 1, \dots , d _ {z} \} \backslash \{j _ {\hat {s}} ^ {\prime} \}} \left\| \mathcal {G} _ {:, \sigma (j _ {\hat {s}})} \right\| _ {0} + \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {s}} ^ {\prime}} \right\| _ {0} \\ \geq \sum_ {j _ {\hat {s}} \in \{d _ {c} + 1, \dots , d _ {z} \} \backslash \{j _ {\hat {s} ^ {\prime}} ^ {\prime} \}} \left\| \mathcal {G} _ {:, \sigma (j _ {\hat {s}})} \right\| _ {0} + \left\| \mathcal {G} _ {:, \sigma (j _ {\hat {s}} ^ {\prime})} \cup \mathcal {G} _ {:, j _ {c} ^ {\prime}} \right\| _ {0} \\ \geq \sum_ {j _ {\hat {s}} \in \{d _ {c} + 1, \dots , d _ {z} \} \backslash \{j _ {\hat {s}} ^ {\prime} \}} \left\| \mathcal {G} _ {:, \sigma (j _ {\hat {s}})} \right\| _ {0} + \left\| \mathcal {G} _ {:, j _ {c} ^ {\prime}} \right\| _ {0} \\ \underbrace {>} _ {(1)} \sum_ {j _ {s} \in \left\{d _ {c} + 1, \dots , d _ {z} \right\}} \left\| \mathcal {G} _ {:, j _ {s}} \right\| _ {0}, \tag {17} \\ \end{array}
$$
where the inequality (1) is due to Assumption 1-iii. that the influence of  $\mathbf{c}$  on  $\mathbf{x}$  is denser than that of  $\mathbf{s}$ .
However, as discussed above, there exists an optimizer that attains the lower-bound Equation 15. Equation 17 contradicts the minimization objective Equation 3. Therefore,  $(j_c', j_{\hat{s}}') \notin \mathcal{T}$ , for any  $j_c' \in [d_c]$  and any  $j_{\hat{s}}' \in \{d_c + 1, \dots, d_c\}$ .
As discussed above, this implies that  $\mathbf{c}$  is not influenced by  $\hat{\mathbf{s}}$ . Further, it follows from the invertibility of  $h(\cdot)$  that  $[\mathbf{J}_h(\mathbf{z})]_{j_{\hat{c}},j_s} = 0$ , for any  $j_{\hat{c}} \in \{1,\dots,d_c\}$  and  $j_s \in \{d_c + 1,\dots,d_c + d_s\}$ , which implies that  $\hat{\mathbf{c}}$  is not influenced by  $\mathbf{s}$ . These two conditions and the invertibility of  $h(\cdot)$  imply that  $\hat{\mathbf{c}}$  and  $\mathbf{c}$  form a one-to-one mapping.
A.6 Proof for Theorem 2
The original Assumption iii. and Theorem 2 are copied below for reference.
Assumption 2 (Partially intersecting influence supports). For every pair  $(c_{j_c}, s_{j_s})$ , the supports of their influences on  $\mathbf{x}$  do not fully intersect, i.e.,  $\| \mathcal{G}_{:, j_c} \cap \mathcal{G}_{:, j_s} \|_0 < \min \{\| \mathcal{G}_{:, j_c} \|_0, \| \mathcal{G}_{:, j_s} \|_0\}$ .
$$
\min  \sum_ {\left(j _ {\hat {c}}, j _ {\hat {s}}\right) \in \{1, \dots , d _ {c} \} \times \left\{d _ {c} + 1, \dots , d _ {z} \right\}} \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {c}}} \cap \hat {\mathcal {G}} _ {:, j _ {\hat {s}}} \right\| _ {0}. \tag {4}
$$
Proof. As shown in Section A.5, there exists a  $d_z$ -permutation  $\sigma(\cdot)$  such that  $\forall j \in [d_z]$ ,  $(\sigma(j), j) \in \mathcal{T}$ . Also, we have shown in Theorem 1 that  $(j_c, j_{\hat{s}}) \notin \mathcal{T}$  for  $j_c \in [d_c]$  and  $j_{\hat{s}} \in \{d_c + 1, \dots, d_z\}$ , which implies that  $\sigma(j_{\hat{s}}) \in \{d_c + 1, \dots, d_z\}$ . Thus, it follows that for any  $j_{\hat{c}} \in [d_c]$ ,  $\sigma(j_{\hat{c}}) \in [d_c]$ .
In the following, we show by contradiction that  $(j_s,j_{\hat{c}})\notin \mathcal{T}$  for any  $j_{s}\in \{d_{c} + 1,\dots ,d_{z}\}$  and  $j_{\hat{c}}\in [d_c]$ . We suppose that  $(j_s',j_{\hat{c}}')\in \mathcal{T}$ . Analogous to Equation 16, we would have that
$$
\left(j _ {s} ^ {\prime}, j _ {\hat {c}} ^ {\prime}\right) \in \mathcal {T} \Longrightarrow \left(i _ {x}, j _ {\hat {c}} ^ {\prime}\right) \in \hat {\mathcal {G}}, \quad \forall i _ {x} \in \{i \in [ d _ {x} ]: \left(i _ {x}, j _ {s} ^ {\prime}\right) \in \mathcal {G} \}. \tag {18}
$$
It would follow that  $\hat{\mathcal{G}}_{:,j_{\hat{s}}^{\prime}}\supseteq \mathcal{G}_{:, \sigma(j_{\hat{s}}^{\prime})}\cup \mathcal{G}_{:,j_{s}^{\prime}}$ . Also, to attain the objective Equation 3 in Theorem 1, we have  $j_{\hat{s}}^{\prime}:= \sigma^{-1}(j_{s}^{\prime})\in \{d_{c} + 1,\dots ,d_{z}\}$ , s.t.,  $\hat{\mathcal{G}}_{:,j_{\hat{s}}^{\prime}} = \mathcal{G}_{:,j_{s}^{\prime}}$ . It would follow that  $\hat{\mathcal{G}}_{:,j_{\hat{s}}^{\prime}}\supseteq \hat{\mathcal{G}}_{:,j_{\hat{s}}^{\prime}}$ .
Further, we would have
$$
\left\| \hat {\mathcal {G}} _ {:, j _ {\hat {c}} ^ {\prime}} \cap \hat {\mathcal {G}} _ {:, j _ {\hat {s}} ^ {\prime}} \right\| _ {0} = \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {s}} ^ {\prime}} \right\| _ {0} = \left\| \mathcal {G} _ {:, j _ {\hat {s}} ^ {\prime}} \right\| _ {0} \underbrace {>} _ {(2)} \left\| \mathcal {G} _ {:, \sigma \left(j _ {\hat {c}} ^ {\prime}\right)} \cap \mathcal {G} _ {:, \sigma \left(j _ {\hat {s}} ^ {\prime}\right)} \right\| _ {0}, \tag {19}
$$
where (2) is due to Assumption 2.
We note that the lower-bound for Equation 4 is
$$
\begin{array}{l} \sum_ {\left(j _ {\hat {c}}, j _ {\hat {s}}\right) \in \{1, \dots , d _ {c} \} \times \left\{d _ {c} + 1, \dots , d _ {z} \right\}} \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {c}}} \cap \hat {\mathcal {G}} _ {:, j _ {\hat {s}}} \right\| _ {0} \geq \sum_ {\left(j _ {\hat {c}}, j _ {\hat {s}}\right) \in \{1, \dots , d _ {c} \} \times \left\{d _ {c} + 1, \dots , d _ {z} \right\}} \left\| \mathcal {G} _ {:, \sigma \left(j _ {\hat {c}}\right)} \cap \mathcal {G} _ {:, \sigma \left(j _ {\hat {s}}\right)} \right\| _ {0} (20) \\ = \sum_ {\left(j _ {c}, j _ {s}\right) \in \{1, \dots , d _ {c} \} \times \left\{d _ {c} + 1, \dots , d _ {z} \right\}} \| \mathcal {G} _ {:, j _ {c}} \cap \mathcal {G} _ {:, j _ {s}} \| _ {0}, (21) \\ \end{array}
$$
which can be achieved by  $\mathcal{G} = \hat{\mathcal{G}}$ . Note that the lower-bounds for both Equation 3 and Equation 4 can be attained simultaneously by  $\mathcal{G} = \hat{\mathcal{G}}$ . Hence, optimizing the sum of the two objectives does not alter the optimal value of either.
Applying a similar argument as that in Equation 15, we would have that
$$
\begin{array}{l} \sum_ {\left(j _ {\hat {c}}, j _ {\hat {s}}\right) \in \{1, \dots , d _ {c} \} \times \{d _ {c} + 1, \dots , d _ {z} \}} \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {c}}} \cap \hat {\mathcal {G}} _ {:, j _ {\hat {s}}} \right\| _ {0} \\ = \sum_ {\left(j _ {\hat {c}}, j _ {\hat {s}}\right) \in \{1, \dots , d _ {c} \} \times \{d _ {c} + 1, \dots , d _ {z} \} \backslash \left\{\left(j _ {\hat {c}} ^ {\prime}, j _ {\hat {s}} ^ {\prime}\right) \right\}} \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {c}}} \cap \hat {\mathcal {G}} _ {:, j _ {\hat {s}}} \right\| _ {0} + \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {c}} ^ {\prime}} ^ {\prime} \cap \hat {\mathcal {G}} _ {:, j _ {\hat {s}} ^ {\prime}} \right\| _ {0} \\ \geq \sum_ {\left(j _ {\hat {c}}, j _ {\hat {s}}\right) \in \{1, \dots , d _ {c} \} \times \left\{d _ {c} + 1, \dots , d _ {z} \right\} \backslash \left\{\left(j _ {\hat {c}} ^ {\prime}, j _ {\hat {s}} ^ {\prime}\right) \right\}} \left\| \mathcal {G} _ {:, \sigma \left(j _ {\hat {c}}\right)} \cap \mathcal {G} _ {:, \sigma \left(j _ {\hat {s}}\right)} \right\| _ {0} + \left\| \hat {\mathcal {G}} _ {:, j _ {\hat {c}} ^ {\prime}} \cap \hat {\mathcal {G}} _ {:, j _ {\hat {s}} ^ {\prime}} \right\| _ {0} \tag {22} \\ \end{array}
$$
$$
\begin{array}{l} \underbrace{\geq}_{(3)}\sum_{(j_{\hat{c}},j_{\hat{s}})\in \{1,\dots,d_{c}\} \times \{d_{c} + 1,\dots,d_{z}\} \setminus \{(j_{\hat{c}}^{\prime},j_{\hat{s}}^{\prime})\}}\left\| \mathcal{G}_{:, \sigma (j_{\hat{c}})}\cap \mathcal{G}_{:, \sigma (j_{\hat{s}})}\right\|_{0} + \left\| \mathcal{G}_{:, \sigma (j_{\hat{c}}^{\prime})}\cap \mathcal{G}_{:, \sigma (j_{\hat{s}}^{\prime})}\right\|_{0} \\ = \sum_ {(j _ {c}, j _ {s}) \in \{1, \dots , d _ {c} \} \times \{d _ {c} + 1, \dots , d _ {z} \}} \| \mathcal {G} _ {:, j _ {c}} \cap \mathcal {G} _ {:, j _ {s}} \| _ {0}, \\ \end{array}
$$
where (3) is due to Equation 19. Hence, this was not the minimizer of Equation 4. By contradiction, we have that  $(j_{s}, j_{\hat{c}}) \notin \mathcal{T}$  for any  $j_{s} \in \{d_{c} + 1, \ldots, d_{z}\}$  and  $j_{\hat{c}} \in [d_{c}]$ . This implies that  $\mathbf{s}$  is not influenced by  $\hat{\mathbf{c}}$ . Further, it follows from the invertibility of  $h(\cdot)$  that  $[\mathbf{J}_{h}(\mathbf{z})]_{j_{\hat{s}}, j_{c}} = 0$ , for any  $j_{\hat{s}} \in \{d_{c} + 1, \ldots, d_{z}\}$  and  $j_{c} \in [d_{c}]$ , which implies that  $\hat{\mathbf{s}}$  is not influenced by  $\mathbf{c}$ . These two conditions and the invertibility of  $h(\cdot)$  imply that  $\hat{\mathbf{s}}$  and  $\mathbf{s}$  form a one-to-one mapping.