SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left. \frac {\partial \operatorname {o f f} (\boldsymbol {\Sigma})}{\partial \operatorname {o f f} (\mathbf {A})} \right| _ {\mathbf {A} = \mathbf {I} _ {n}} = \left[ \begin{array}{c} \mathbf {I} _ {\parallel \mathbf {A} \parallel_ {0} - d} \\ \mathbf {I} _ {\parallel \mathbf {A} \parallel_ {0} - d} \\ \mathbf {0} \end{array} \right],
$$
$$
\operatorname {r a n k} \left(\left. \frac {\partial \text {o f f} (\boldsymbol {\Sigma})}{\partial \text {o f f} (\mathbf {A})} \right| _ {\mathbf {A} = \mathbf {I} _ {n}}\right) = \| \mathbf {A} \| _ {0} - d. \tag {14}
$$
Substituting Eq. (13) into Eq. (12), we have
$$
\left. \frac {\partial \boldsymbol {\Sigma}}{\partial \mathbf {A}} \right| _ {\mathbf {A} = \mathbf {I} _ {n}} = \left[ \begin{array}{c c} \left. \frac {\partial \operatorname {o f f} (\boldsymbol {\Sigma})}{\partial \operatorname {o f f} (\mathbf {A})} \right| _ {\mathbf {A} = \mathbf {I} _ {n}} & \mathbf {0} \\ \mathbf {0} & 2 \mathbf {I} _ {d} \end{array} \right],
$$
which, with Eq. (14), implies
$$
\begin{array}{l} \operatorname {r a n k} \left(\frac {\partial \boldsymbol {\Sigma}}{\partial \mathbf {A}} \Big | _ {\mathbf {A} = \mathbf {I} _ {n}}\right) = \operatorname {r a n k} \left(\frac {\partial \operatorname {o f f} (\boldsymbol {\Sigma})}{\partial \operatorname {o f f} (\mathbf {A})} \Big | _ {\mathbf {A} = \mathbf {I} _ {n}}\right) + \operatorname {r a n k} (2 \mathbf {I} _ {d}) \\ = \| \mathbf {A} \| _ {0} - d + d \\ = \| \mathbf {A} \| _ {0}. \\ \end{array}
$$
Proposition 2 (Dimension of Covariance Set). Let  $\xi$  be a support matrix that satisfies Assumption 2. Then, its covariance set has a dimension of  $\| \pmb {\xi}\| _0$ , i.e.,  $\dim (\Sigma (\pmb {\xi})) = \| \pmb {\xi}\| _0$ .
Proof. Since  $\xi$  satisfies Assumption 2, it can be permuted by column and row permutations to be lower triangular; in this case, the resulting covariance matrix and the original covariance matrix differ in equal row and column permutations. Note that the dimension of the covariance set remains the same after simultaneous equal row and column permutations of the covariance matrices. Therefore, it suffices to consider the case where  $\xi$  is lower triangular, and show that it has a dimension of  $\| \pmb{\xi} \|_0$ .
As indicated by Geiger et al. [19, Theorem 10], the dimension of  $\dim(\Sigma(\xi))$  equals the maximum rank of the corresponding Jacobian matrix. In this case, it suffices to consider the columns of the Jacobian matrix that correspond to the nonzero entries of support matrix  $\xi$ , i.e., the free parameters of matrix  $\mathbf{A}$ , which we denote by  $\frac{\partial \Sigma}{\partial \mathbf{A}}$ . By Lemma 9, when  $\mathbf{A} = \mathbf{I}_n$ , the Jacobian matrix  $\frac{\partial \Sigma}{\partial \mathbf{A}}$  has full column rank that is equal to  $\|\xi\|_0$ . Therefore, the dimension of the covariance set is  $\|\xi\|_0$ .
C.2 Covariance Equivalence and Column Permutation of Support
We now state a result that is adapted from Ghassami et al. [20, Proposition 5] to the context of ICA. In our proof of Theorem 1, only the "only if part" of the following result is used.
Proposition 6 (Ghassami et al. [20, Proposition 5]). Consider two support matrices  $\xi_{1}$  and  $\xi_{2}$ . If every pair of columns of  $\xi_{1}$  differ in more than one entry, then  $\xi_{1}$  and  $\xi_{2}$  are covariance equivalent if and only if the columns of  $\xi_{2}$  are a permutation of columns of  $\xi_{1}$ .
C.3 Equality Constraints and Covariance Equivalence
In this section, we show how Assumption 2 allows one to go from two support matrices  $\pmb{\xi}_1$  and  $\pmb{\xi}_2$  having the same equality constraints to covariance equivalence.
Following Ghassami et al. [20], we first denote the covariance set of a directed graph  $\mathcal{G}$  by
$$
\boldsymbol {\Theta} (\mathcal {G}) := \left\{\left(\mathbf {I} - \mathbf {B}\right) \boldsymbol {\Omega} ^ {- 1} \left(\mathbf {I} - \mathbf {B}\right) ^ {\top}: \mathbf {B} \in \mathbb {R} _ {\text {o f f}} ^ {n \times n}, \boldsymbol {\Omega} \in \operatorname {d i a g} \left(\mathbb {R} _ {> 0} ^ {n}\right), \operatorname {s u p p} (\mathbf {B}) \subseteq \operatorname {s u p p} \left(\mathbf {B} _ {\mathcal {G}}\right) \right\},
$$
where  $\mathbf{B}_{\mathcal{G}}$  is the adjacency matrix of  $\mathcal{G}$ . With a slight abuse of notation, we denote by  $H(\mathcal{G})$  the set of the equality constraints imposed by  $\mathcal{G}$  on the resulting matrix  $(\mathbf{I} - \mathbf{B})\boldsymbol{\Omega}^{-1}(\mathbf{I} - \mathbf{B})^{\top}$ .
Lemma 10. Let  $\pmb{\xi}$  be a non-singular support matrix that satisfies Assumption 2. Then, there exists a DAG  $\mathcal{G}$  such that  $\pmb{\Sigma}(\pmb{\xi}) = \pmb{\Theta}(\mathcal{G})$ .
Proof. Since  $\pmb{\xi}$  is non-singular, by Lemma 2, it can be mapped via column permutations to another support matrix  $\pmb{\xi}'$  with diagonal entries being nonzero. Since  $\pmb{\xi}$  satisfies Assumption 2, Proposition 10 implies that  $\pmb{\xi}'$  represents a DAG, say  $\mathcal{G}$ , where the support of its adjacency matrix is  $\mathrm{supp}(\mathbf{B}_{\mathcal{G}}) = \mathrm{supp}(\mathrm{off}(\pmb{\xi}'))$ . Since column permutations of support matrices do not affect the resulting covariance matrices, we have  $\pmb{\Sigma}(\pmb{\xi}) = \pmb{\Sigma}(\pmb{\xi}')$ . Therefore, it suffices to prove  $\pmb{\Sigma}(\pmb{\xi}') = \pmb{\Theta}(\mathcal{G})$ .
Since  $\pmb{\xi}'$  is a support matrix with diagonal entries being nonzero, we have
$$
\operatorname {s u p p} \left(\boldsymbol {\xi} ^ {\prime}\right) = \operatorname {s u p p} (\mathbf {I}) \cup \operatorname {s u p p} \left(\operatorname {o f f} \left(\boldsymbol {\xi} ^ {\prime}\right)\right). \tag {15}
$$
We consider both parts of the statements.
Proof of  $\Theta (\mathcal{G})\subseteq \Sigma^{+}(\pmb{\xi}^{\prime})$
Suppose  $\mathbf{M} \in \Theta(\mathcal{G})$ . By definition of  $\Theta(\mathcal{G})$ , there exist  $\mathbf{B} \in \mathbb{R}_{\mathrm{off}}^{n \times n}$  and  $\Omega \in \mathrm{diag}(\mathbb{R}_{>0}^n)$  such that
$$
\operatorname {s u p p} (\mathbf {B}) \subseteq \operatorname {s u p p} \left(\operatorname {o f f} \left(\boldsymbol {\xi} ^ {\prime}\right)\right) \quad \text {a n d} \quad \mathbf {M} = (\mathbf {I} - \mathbf {B}) \boldsymbol {\Omega} ^ {- 1} (\mathbf {I} - \mathbf {B}) ^ {\top}. \tag {16}
$$
Let  $\mathbf{A} \coloneqq (\mathbf{I} - \mathbf{B})\Omega^{-\frac{1}{2}}$ , which, with Eq. (16), implies  $\mathbf{A}\mathbf{A}^{\top} = \mathbf{M}$ . Recall that  $\mathcal{G}$  is a DAG, which indicates  $\mathbf{B}$  also represents a DAG. Therefore,  $\mathbf{I} - \mathbf{B}$ , and thus  $\mathbf{A}$ , are non-singular. Since right multiplication of  $\Omega^{-\frac{1}{2}}$  does not affect the support of  $\mathbf{I} - \mathbf{B}$ , by Eqs. (15) and (16), we have
$$
\operatorname {s u p p} (\mathbf {A}) = \operatorname {s u p p} (\mathbf {I} - \mathbf {B}) = \operatorname {s u p p} (\mathbf {I}) \cup \operatorname {s u p p} (\mathbf {B}) \subseteq \operatorname {s u p p} (\mathbf {I}) \cup \operatorname {s u p p} (\text {o f f} (\boldsymbol {\xi} ^ {\prime})) = \operatorname {s u p p} (\boldsymbol {\xi} ^ {\prime}).
$$
Therefore, we have  $\mathrm{supp}(\mathbf{A})\subseteq \mathrm{supp}(\boldsymbol{\xi}^{\prime})$ , which, with the non-singularity of matrix  $\mathbf{A}$ , implies  $\mathbf{M} = \mathbf{A}\mathbf{A}^{\top}\in \boldsymbol {\Sigma}(\boldsymbol{\xi}^{\prime})$
2We say that a support matrix  $\pmb{\xi}$  is non-singular if there exists non-singular matrix  $\mathbf{A}$  such that  $\mathrm{supp}(\mathbf{A})\subseteq$ $\operatorname {supp}(\pmb {\xi})$
Proof of  $\Sigma (\pmb{\xi}^{\prime})\subseteq \Theta (\mathcal{G})$  ..
Suppose  $\mathbf{M} \in \Sigma(\pmb{\xi}')$ . By definition of  $\Sigma(\pmb{\xi}')$ , there exists non-singular matrix  $\mathbf{A} \in \mathbb{R}^{n \times n}$  such that
$$
\operatorname {s u p p} (\mathbf {A}) \subseteq \operatorname {s u p p} \left(\boldsymbol {\xi} ^ {\prime}\right) \quad \text {a n d} \quad \mathbf {M} = \mathbf {A} \mathbf {A} ^ {\top}. \tag {17}
$$
Since matrix  $\mathbf{A}$  is non-singular, all of its diagonal entries must be nonzero, because otherwise the corresponding determinant will be zero, which contradicts its non-singularity. By Lemma 4, there exists matrix  $\mathbf{B} \in \mathbb{R}_{\mathrm{off}}^{n \times n}$  and  $\Omega \in \mathrm{diag}(\mathbb{R}_{>0}^n)$  such that  $\mathbf{A} = (\mathbf{I} - \mathbf{B})\Omega^{-\frac{1}{2}}\mathbf{D}$ , where  $\mathbf{D}$  is a diagonal matrix with diagonal entries being  $\pm 1$ . By Eq. (17), we have  $(\mathbf{I} - \mathbf{B})\Omega^{-1}(\mathbf{I} - \mathbf{B})^\top = \mathbf{M}$ . Since right multiplication of  $\Omega^{-\frac{1}{2}}\mathbf{D}$  does not affect the support of  $\mathbf{I} - \mathbf{B}$ , by Eqs. (15) and (17), we have
$$
\operatorname {s u p p} (\mathbf {I}) \cup \operatorname {s u p p} (\mathbf {B}) = \operatorname {s u p p} (\mathbf {I} - \mathbf {B}) = \operatorname {s u p p} (\mathbf {A}) \subseteq \operatorname {s u p p} (\boldsymbol {\xi}) = \operatorname {s u p p} (\mathbf {I}) \cup \operatorname {s u p p} (\text {o f f} (\boldsymbol {\xi} ^ {\prime})).
$$
Note that  $\operatorname{supp}(\mathbf{I})$  and  $\operatorname{supp}(\mathbf{B})$  are disjoint. Furthermore,  $\operatorname{supp}(\mathbf{I})$  and  $\operatorname{supp}(\mathrm{off}(\boldsymbol{\xi}'))$  are disjoint. Therefore, we have  $\operatorname{supp}(\mathbf{B}) \subseteq \operatorname{supp}(\mathrm{off}(\boldsymbol{\xi}'))$  and thus  $\mathbf{M} = (\mathbf{I} - \mathbf{B})\boldsymbol{\Omega}^{-1}(\mathbf{I} - \mathbf{B})^\top \in \boldsymbol{\Theta}(\mathcal{G})$ .
Proposition 7. Let  $\xi_{1}$  and  $\xi_{2}$  be non-singular support matrices that satisfy Assumption 2. If they have the same set of equality constraints, i.e.,  $H(\pmb{\xi}_2) = H(\pmb{\xi}_2)$ , then they are covariance equivalent.
Proof. Since matrices  $\xi_{1}$  and  $\xi_{2}$  are non-singular and satisfy Assumption 2, by Lemma 10, there exist DAGs  $\mathcal{G}_1$  and  $\mathcal{G}_2$  such that
$$
\boldsymbol {\Sigma} \left(\boldsymbol {\xi} _ {1}\right) = \boldsymbol {\Theta} \left(\mathcal {G} _ {1}\right) \quad \text {a n d} \quad \boldsymbol {\Sigma} \left(\boldsymbol {\xi} _ {2}\right) = \boldsymbol {\Theta} \left(\mathcal {G} _ {2}\right), \tag {18}
$$
which imply
$$
H \left(\boldsymbol {\xi} _ {1}\right) = H \left(\mathcal {G} _ {1}\right) \quad \text {a n d} \quad H \left(\boldsymbol {\xi} _ {2}\right) = H \left(\mathcal {G} _ {2}\right). \tag {19}
$$
We now provide a proof by contrapositive. Suppose that  $\xi_{1}$  and  $\xi_{2}$  are not covariance equivalent, i.e.,  $\Sigma (\xi_1)\neq \Sigma (\xi_2)$ . By Eq. (18), we have  $\Theta (\mathcal{G}_1)\neq \Theta (\mathcal{G}_2)$ , i.e., DAGs  $\mathcal{G}_1$  and  $\mathcal{G}_2$  are not covariance equivalent. By Ghassami et al. [20, Proposition 1], DAGs  $\mathcal{G}_1$  and  $\mathcal{G}_2$  are not Markov equivalent, which indicates that they do not have the same skeleton and v-structures [43]. This implies that they lead to different sets of conditional independence constraints, which in this case correspond to different sets of polynomial equality constraints [39], i.e.,  $H(\mathcal{G}_1)\neq H(\mathcal{G}_2)$ . By Eq. (19), we have  $H(\pmb {\xi}_1)\neq H(\pmb {\xi}_2)$ .
C.4 Identifiability of Parameters from Support
In the following, we provide a result that establish the identifiability of the parameters in mixing matrix from its support.
Proposition 8. Consider two non-singular matrices  $\mathbf{A}_1$  and  $\mathbf{A}_2$  that satisfy Assumption 2 and that entail the same covariance matrix, i.e.,  $\pmb{\Sigma} = \mathbf{A}_1\mathbf{A}_1^\top = \mathbf{A}_2\mathbf{A}_2^\top$ . If matrices  $\mathbf{A}_1$  and  $\mathbf{A}_2$  have the same support, then they differ only in sign changes of columns.
Proof. Since matrix  $\mathbf{A}_1$  satisfies Assumption 2, there exist permutation matrices  $\mathbf{P}_1$  and  $\mathbf{P}_2$  such that  $\mathbf{P}_1^\top \mathbf{A}_1\mathbf{P}_2$  is lower triangular. Clearly,  $\mathbf{P}_1^\top \mathbf{A}_2\mathbf{P}_2$  is also lower triangular, because matrices  $\mathbf{A}_1$  and  $\mathbf{A}_2$  have the same support. Since matrices  $\mathbf{P}_1^\top \mathbf{A}_1\mathbf{P}_2$  and  $\mathbf{P}_1^\top \mathbf{A}_2\mathbf{P}_2$  are non-singular, all diagonal entries of these two matrices must be nonzero, because otherwise the corresponding determinant will be zero, which contradict their non-singularity. Let  $\mathbf{D}_1$  and  $\mathbf{D}_2$  be diagonal matrices with diagonal entries being  $\pm 1$  such that the diagonal entries of  $\mathbf{P}_1^\top \mathbf{A}_1\mathbf{D}_1\mathbf{P}_2$  and  $\mathbf{P}_1^\top \mathbf{A}_2\mathbf{D}_2\mathbf{P}_2$  are positive. (The procedure for constructing such diagonal matrices  $\mathbf{D}_1$  and  $\mathbf{D}_2$  is straightforward and omitted here.)
$$
\left(\mathbf {P} _ {1} ^ {\top} \mathbf {A} _ {1} \mathbf {D} _ {1} \mathbf {P} _ {2}\right) \left(\mathbf {P} _ {1} ^ {\top} \mathbf {A} _ {1} \mathbf {D} _ {1} \mathbf {P} _ {2}\right) ^ {\top} = \left(\mathbf {P} _ {1} ^ {\top} \mathbf {A} _ {2} \mathbf {D} _ {2} \mathbf {P} _ {2}\right) \left(\mathbf {P} _ {1} ^ {\top} \mathbf {A} _ {2} \mathbf {D} _ {2} \mathbf {P} _ {2}\right) ^ {\top} = \mathbf {P} _ {1} ^ {\top} \boldsymbol {\Sigma} \mathbf {P} _ {1}. \tag {20}
$$
Since matrix  $\mathbf{A}_1$  is non-singular, matrices  $\boldsymbol{\Sigma}$ , and thus  $\mathbf{P}_1^\top \boldsymbol{\Sigma} \mathbf{P}_1$ , are symmetric positive definite. Here, Eq. (20) can be viewed as the Cholesky decomposition of  $\mathbf{P}_1^\top \boldsymbol{\Sigma} \mathbf{P}_1$ , where  $\mathbf{P}_1^\top \mathbf{A}_1 \mathbf{D}_1 \mathbf{P}_2$  and  $\mathbf{P}_1^\top \mathbf{A}_2 \mathbf{D}_2 \mathbf{P}_2$  are the Cholesky factors. Recall that they are lower triangular matrices with all diagonal entries being positive; in this case, it is known that such Cholesky factor is unique [42]. Therefore, we have
$$
\mathbf {P} _ {1} ^ {\top} \mathbf {A} _ {1} \mathbf {D} _ {1} \mathbf {P} _ {2} = \mathbf {P} _ {1} ^ {\top} \mathbf {A} _ {2} \mathbf {D} _ {2} \mathbf {P} _ {2},
$$
$$
\mathbf {A} _ {1} = \mathbf {A} _ {2} \mathbf {D} _ {2} \mathbf {D} _ {1} ^ {- 1} = \mathbf {A} _ {2} \mathbf {D} _ {2} \mathbf {D} _ {1}.
$$
Since  $\mathbf{D}_2\mathbf{D}_1$  is a diagonal matrix with diagonal entries being  $\pm 1$ , we conclude that matrices  $\mathbf{A}_1$  and  $\mathbf{A}_2$  differ only in sign changes of columns.
Corollary 1 (Identifiability of Parameters from Support). Consider two non-singular matrices  $\mathbf{A}_1$  and  $\mathbf{A}_2$  that satisfy Assumption 2 and that entail the same covariance matrix, i.e.,  $\boldsymbol{\Sigma} = \mathbf{A}_1\mathbf{A}_1^\top = \mathbf{A}_2\mathbf{A}_2^\top$ . If the columns of  $\xi_{\mathbf{A}_1}$  are a permutation of those of  $\xi_{\mathbf{A}_2}$ , then we have  $\mathbf{A}_1 \sim \mathbf{A}_2$ .
Proof. Suppose by contradiction that  $\mathbf{A}_1 \not\sim \mathbf{A}_2$ . This implies that, for every permutation matrix  $\mathbf{P}$  such that  $\xi_{\mathbf{A}_1} = \xi_{\mathbf{A}_2\mathbf{P}}$ , matrices  $\mathbf{A}_1$  and  $\mathbf{A}_2\mathbf{P}$  differ in more than sign changes of columns. By Proposition 8, this cannot happen.
D Proofs of Other Results
D.1 Proof of Example 1
$$
\bigl (\Sigma_ {1, 1} \Sigma_ {2, 2} \Sigma_ {3, 3} + \Sigma_ {1, 1} \Sigma_ {2, 3} ^ {2} - \Sigma_ {2, 2} \Sigma_ {1, 3} ^ {2} - \Sigma_ {3, 3} \Sigma_ {1, 2} ^ {2} \bigr) ^ {2} - 4 \bigl (\Sigma_ {1, 1} \Sigma_ {2, 2} - \Sigma_ {1, 2} ^ {2} \bigr) \bigl (\Sigma_ {1, 1} \Sigma_ {3, 3} \Sigma_ {2, 3} ^ {2} - \Sigma_ {1, 3} ^ {2} \Sigma_ {2, 3} ^ {2} \bigr) \geq 0.
$$
Proof. We first consider matrix  $\mathbf{A}_1$  with the support  $\xi_1$ . That is, matrix  $\mathbf{A}_1$  is of the form
$$
\mathbf {A} _ {1} = \left[ \begin{array}{c c c} a _ {1, 1} & 0 & 0 \\ a _ {2, 1} & a _ {2, 2} & 0 \\ a _ {3, 1} & 0 & a _ {3, 3} \end{array} \right].
$$
The entries of the resulting covariance matrix  $\pmb{\Sigma} = \mathbf{A}_1\mathbf{A}_1^\top$  are then given by
$$
\Sigma_ {1, 1} = a _ {1, 1} ^ {2}, \qquad \Sigma_ {1, 2} = a _ {1, 1} a _ {2, 1},
$$
$$
\Sigma_ {2, 2} = a _ {2, 1} ^ {2} + a _ {2, 2} ^ {2}, \quad \Sigma_ {1, 3} = a _ {1, 1} a _ {3, 1},
$$
$$
\Sigma_ {3, 3} = a _ {3, 1} ^ {2} + a _ {3, 3} ^ {2}, \quad \Sigma_ {2, 3} = a _ {2, 1} a _ {3, 1},
$$
$$
\Sigma_ {1, 1} \Sigma_ {2, 3} - \Sigma_ {1, 2} \Sigma_ {1, 3} = 0.
$$
We now consider matrix  $\mathbf{A}_2$  with the support  $\xi_2$ . That is, matrix  $\mathbf{A}_2$  is of the form
$$
\mathbf {A} _ {2} = \left[ \begin{array}{c c c} a _ {1, 1} & 0 & a _ {1, 3} \\ a _ {2, 1} & a _ {2, 2} & 0 \\ 0 & a _ {3, 2} & a _ {3, 3} \end{array} \right].
$$
The entries of the resulting covariance matrix  $\pmb{\Sigma} = \mathbf{A}_2\mathbf{A}_2^\top$  are then given by
$$
\Sigma_ {1, 1} = a _ {1, 1} ^ {2} + a _ {1, 3} ^ {2}, \quad \Sigma_ {1, 2} = a _ {1, 1} a _ {2, 1},
$$
$$
\Sigma_ {2, 2} = a _ {2, 1} ^ {2} + a _ {2, 2} ^ {2}, \quad \Sigma_ {1, 3} = a _ {1, 3} a _ {3, 3},
$$
$$
\Sigma_ {3, 3} = a _ {3, 2} ^ {2} + a _ {3, 3} ^ {2}, \quad \Sigma_ {2, 3} = a _ {2, 2} a _ {3, 2}.
$$
Suppose we fix the value of  $a_{3,2}$ . This leads to
$$
a _ {3, 2} ^ {2} = \Sigma_ {3, 3} - \frac {\Sigma_ {1 , 3} ^ {2}}{\Sigma_ {1 , 1} - \frac {\Sigma_ {1 , 2} ^ {2}}{\Sigma_ {2 , 2} - \frac {\Sigma_ {2 , 3} ^ {2}}{a _ {3 , 2} ^ {2}}}},
$$
$$
\bigl (\Sigma_ {1, 1} \Sigma_ {2, 2} - \Sigma_ {1, 2} ^ {2} \bigr) a _ {3, 2} ^ {4} + \bigl (- \Sigma_ {1, 1} \Sigma_ {2, 2} \Sigma_ {3, 3} - \Sigma_ {1, 1} \Sigma_ {2, 3} ^ {2} + \Sigma_ {2, 2} \Sigma_ {1, 3} ^ {2} + \Sigma_ {3, 3} \Sigma_ {1, 2} ^ {2} \bigr) a _ {3, 2} ^ {2} + \bigl (\Sigma_ {1, 1} \Sigma_ {3, 3} \Sigma_ {2, 3} ^ {2} - \Sigma_ {1, 3} ^ {2} \Sigma_ {2, 3} ^ {2} \bigr) = 0.
$$
Since the value of  $a_{3,2}$  is a real number, we have
$$
\begin{array}{c} \big (\Sigma_ {1, 1} \Sigma_ {2, 2} \Sigma_ {3, 3} + \Sigma_ {1, 1} \Sigma_ {2 3} ^ {2} - \Sigma_ {2, 2} \Sigma_ {1, 3} ^ {2} - \Sigma_ {3, 3} \Sigma_ {1, 2} ^ {2} \big) ^ {2} - 4 \big (\Sigma_ {1, 1} \Sigma_ {2, 2} - \Sigma_ {1, 2} ^ {2} \big) \big (\Sigma_ {1, 1} \Sigma_ {3, 3} \Sigma_ {2, 3} ^ {2} - \Sigma_ {1, 3} ^ {2} \Sigma_ {2, 3} ^ {2} \big) \geq 0. \\ \square \end{array}
$$
D.2 Proof of Proposition 1
Proof. Since the true mixing matrix  $\tilde{\mathbf{A}}$  does not satisfy Assumption 1, there exist  $i,j\in [n],i\neq j$  such that
$$
| \operatorname {s u p p} (\tilde {\mathbf {a}} _ {i}) \cup \operatorname {s u p p} (\tilde {\mathbf {a}} _ {j}) | - | \operatorname {s u p p} (\tilde {\mathbf {a}} _ {i}) \cap \operatorname {s u p p} (\tilde {\mathbf {a}} _ {j}) | \leq 1.
$$
This leads to the following two cases:
- Case 1:  $|\operatorname{supp}(\tilde{\mathbf{a}}_i) \cup \operatorname{supp}(\tilde{\mathbf{a}}_j)| - |\operatorname{supp}(\tilde{\mathbf{a}}_i) \cap \operatorname{supp}(\tilde{\mathbf{a}}_j)| = 1$ . In this case, since the mixing matrix is of full column rank, there must exist a  $k \in [n]$  such that  $(\xi_{\tilde{\mathbf{A}}})_{k,i} = (\xi_{\tilde{\mathbf{A}}})_{k,j} = \times$ . Thus, we can always apply a reversible acute rotation (see Remark 3) to the  $i$ -th and  $j$ -th columns of matrix  $\tilde{\mathbf{A}}$ . This operation leads to another matrix  $\tilde{\mathbf{A}}$  with  $\| \tilde{\mathbf{A}} \|_0 = \| \tilde{\mathbf{A}} \|_0$  and  $\tilde{\mathbf{A}} \tilde{\mathbf{A}}^\top = \tilde{\mathbf{A}} \tilde{\mathbf{A}}^\top$ . In the reversible acute rotation, we can set either  $(\xi_{\tilde{\mathbf{A}}})_{k,i}$  or  $(\xi_{\tilde{\mathbf{A}}})_{k,j}$  to 0. This implies  $\| \tilde{\mathbf{a}}_{k,:} \|_0 < \| \tilde{\mathbf{a}}_{k,:} \|_0$ , and therefore  $\tilde{\mathbf{A}} \not\sim \tilde{\mathbf{A}}$ . Now, suppose that the true mixing matrix  $\tilde{\mathbf{A}}$  is not a solution to Problem (1). Clearly, there must exist a solution  $\hat{\mathbf{A}}$  to Problem (1) such that  $\| \hat{\mathbf{A}} \|_0 < \| \tilde{\mathbf{A}} \|_0$ , which indicates  $\hat{\mathbf{A}} \not\sim \tilde{\mathbf{A}}$ . It remains to consider the case where matrix  $\tilde{\mathbf{A}}$  is a solution to Problem (1). In this case, since  $\| \tilde{\mathbf{A}} \|_0 = \| \tilde{\mathbf{A}} \|_0$ , matrix  $\tilde{\mathbf{A}}$  is also a solution to Problem (1), and we have shown that  $\tilde{\mathbf{A}} \not\sim \tilde{\mathbf{A}}$ .
- Case 2:  $|\operatorname{supp}(\tilde{\mathbf{a}}_i) \cup \operatorname{supp}(\tilde{\mathbf{a}}_j)| - |\operatorname{supp}(\tilde{\mathbf{a}}_i) \cap \operatorname{supp}(\tilde{\mathbf{a}}_j)| = 0$ . In this case, we can always apply a reduction (see Remark 3) to the  $i$ -th and  $j$ -th columns of matrix  $\tilde{\mathbf{A}}$ . This operation leads to another matrix  $\tilde{\mathbf{A}}$  with  $\| \tilde{\mathbf{A}} \|_0 < \| \tilde{\mathbf{A}} \|_0$  and  $\tilde{\mathbf{A}} \tilde{\mathbf{A}}^\top = \tilde{\mathbf{A}} \tilde{\mathbf{A}}^\top$ . Therefore, there must exist a solution  $\hat{\mathbf{A}}$  to Problem (1) such that  $\| \hat{\mathbf{A}} \|_0 \leq \| \tilde{\mathbf{A}} \|_0 < \| \tilde{\mathbf{A}} \|_0$ , which indicates  $\hat{\mathbf{A}} \not\sim \tilde{\mathbf{A}}$ .
In either case, there exists a solution to Problem (1) whose columns are not signed permutations of the columns of  $\tilde{\mathbf{A}}$ .
D.3 Proof of Example 2
Proof. Suppose that the matrix  $\mathbf{A}$  does not satisfy Assumption 2. This means that there does not exist permutation matrices  $\mathbf{P}_1$  and  $\mathbf{P}_2$  such that  $\mathbf{P}_1^\top \mathbf{A}\mathbf{P}_2$  is lower triangular, which implies that, in the connective structure  $\mathcal{G}_{\mathbf{A}}$ , there exists a path that alternates between source and observed variable nodes, i.e., a sequence of nodes  $\{s_{i_1}, x_{j_1}, s_{i_2}, x_{j_2}, \ldots, s_{i_k}, x_{j_k}, s_{i_1}\}$  where each pair  $(s_{i_t}, x_{j_t})$  corresponds to a nonzero entry  $a_{j_t, i_t}$  in  $\mathbf{A}$ , for  $t = 1, \ldots, k$ . Thus, by replacing the directed edges on this path with undirected edges, we obtain a cycle. Therefore,  $\mathcal{G}_{\mathbf{A}}$  cannot be a polytree.
D.4 Proof of Proposition 3
The proof of the following proposition is adapted from that of Ghassami et al. [20, Proposition 8].
Proof. Let  $\phi$  be the set of possible equality constraints of any covariance matrix with the same size as  $\pmb{\Sigma}$ , which is a finite set because the number of variables is finite. Consider a matrix  $\hat{\mathbf{A}}$  with the same support as  $\mathbf{A}$  (i.e.,  $\xi_{\hat{\mathbf{A}}} = \xi_{\mathbf{A}}$ ) that violates Assumption 3, where the corresponding covariance matrix is  $\hat{\Sigma}$ . To violate Assumption 3,  $\hat{\Sigma}$  has to satisfy an equality constraint  $\kappa \in \phi \setminus H(\xi_{\mathbf{A}})$ . Thus, the set of possible matrices with the same support as  $\mathbf{A}$  that violate Assumption 3 is a subset of
$$
\bigcup_ {\kappa \in \phi \backslash H (\boldsymbol {\xi} _ {\mathbf {A}})} \left\{\hat {\mathbf {A}}: \boldsymbol {\xi} _ {\hat {\mathbf {A}}} = \boldsymbol {\xi} _ {\mathbf {A}} \text {a n d} \hat {\boldsymbol {\Sigma}} \text {s a t i s f i e s e q u a l i t y c o n s t r a i n t} \kappa \right\}.
$$
By the definition of equality constraint, each set in the union above has zero Lebesgue measure, and thus the finite union above also has zero Lebesgue measure. This implies that the set of possible
matrices with the same support as  $\mathbf{A}$  that violate Assumption 3 has zero Lebesgue measure. Therefore, Assumption 3 is satisfied with probability one.
D.5 Proof of Theorem 2
We first prove the following proposition that will be used to prove both Theorems 2 and 3.
Proposition 9. If mixing matrix  $\mathbf{A}$  satisfies Assumption 6, then it satisfies Assumption 1.
Proof. We provide a proof by contrapositive. Suppose  $\mathbf{A}$  does not satisfy Assumption 1. This means that there exist some  $i, j \in [n]$  with  $i \neq j$  such that
$$
| \operatorname {s u p p} (\mathbf {a} _ {i}) \cup \operatorname {s u p p} (\mathbf {a} _ {j}) | - | \operatorname {s u p p} (\mathbf {a} _ {i}) \cap \operatorname {s u p p} (\mathbf {a} _ {j}) | \leq 1.
$$
The difference can be either 0 or 1.