SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where Eq.  $(\star)$  is directly from the result that all entries in  $\mathbf{C}$ , i.e., those in  $\mathbf{J}_{\mathbf{h}^{-1}}(\hat{s})_{n_I + 1:::n_I}$ , are zero. Moreover, in the proof of Thm. 4.1, we have also shown that all entries in the submatrix  $\mathbf{B}$  are zero. Let  $\mathbf{D}(\mathbf{s})$  represent a diagonal matrix and  $\mathbf{P}$  represent a permutation matrix. Thus, our goal is equivalent to show that  $\mathbf{J}_{\mathbf{h}}(\mathbf{s}):_{n_I,:n_I} = \mathbf{P}_I\mathbf{D}_I(\mathbf{s})$  or  $\mathbf{J}_{\mathbf{h}^{-1}}(\hat{\mathbf{s}}):_{n_I,:n_I} = \mathbf{J}_{\mathbf{h}^{-1}}(\hat{\mathbf{s}}):_{n_I,:n_I} = \mathbf{J}_{\mathbf{h}}(\mathbf{s})_{:n_I,:n_I}^{-1} = \mathbf{D}_I(\mathbf{s})^{-1}\mathbf{P}_I^{-1}$ . Then we need to prove that
$$
\mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}) _ {:,: n _ {I}} = \mathbf {J} _ {\mathbf {f}} (\mathbf {s}) _ {:,: n _ {I}} \mathbf {D} _ {I} (\mathbf {s}) ^ {- 1} \mathbf {P} _ {I} ^ {- 1}. \tag {42}
$$
$$
\mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}) _ {:,: n _ {I}} = \mathbf {J} _ {\mathbf {f}} (\mathbf {s}) _ {:,: n _ {I}} \mathbf {T} (\mathbf {s}), \tag {43}
$$
where  $\mathbf{T}(\mathbf{s})\in \mathbb{R}^{n_I\times n_I}$  is a square matrix. Note that we have denoted  $\mathcal{F}$  as the support of  $\mathbf{J}_{\mathbf{f}}(\mathbf{s})$ ,  $\hat{\mathcal{F}}$  as the support of  $\mathbf{J}_{\hat{\mathbf{f}}}(\hat{\mathbf{s}})$  and  $\mathcal{T}$  as the support of  $\mathbf{T}(\mathbf{s})$ . Besides, we have also denoted  $\mathrm{T}$  as a matrix with the same support of  $\mathcal{T}$ . According to Assumption i, we have
$$
\operatorname {s p a n} \left\{\mathbf {J} _ {\mathbf {f}} \left(\mathbf {s} ^ {(\ell)}\right) _ {i,: n _ {I}} \right\} _ {\ell = 1} ^ {\left| \mathcal {F} _ {i,: n _ {I}} \right|} = \mathbb {R} _ {\mathcal {F} _ {i,: n _ {I}}} ^ {n}. \tag {44}
$$
Since  $\{\mathbf{J}_{\mathbf{f}}(\mathbf{s}^{(\ell)})_{i:n_I}\}_{\ell = 1}^{\lvert\mathcal{F}_{i:n_I}\rvert}$  forms a basis of  $\mathbb{R}_{\mathcal{F}_{i:n_I}}^{n_I}$ , for any  $j_0\in \mathcal{F}_{i:n_I}$ , we are able to rewrite the one-hot vector  $e_{j_0}\in \mathbb{R}_{\mathcal{F}_{i:n_I}}^{n_I}$  as
$$
e _ {j _ {0}} = \sum_ {\ell \in \mathcal {F} _ {i,: n _ {I}}} \alpha_ {\ell} \mathbf {J} _ {\mathbf {f}} \left(\mathbf {s} ^ {(\ell)}\right) _ {i,: n _ {I}}, \tag {45}
$$
where  $\alpha_{\ell}$  is the corresponding coefficient. Then
$$
\mathrm {T} _ {j _ {0,:}} = e _ {j _ {0}} \mathrm {T} = \sum_ {\ell \in \mathcal {F} _ {i,: n _ {I}}} \alpha_ {\ell} \mathbf {J} _ {\mathbf {f}} \left(\mathbf {s} ^ {(\ell)}\right) _ {i,: n _ {I}} \mathrm {T}, \tag {46}
$$
According to Assumption i, each term in the above summation belongs to the space  $\mathbb{R}_{\hat{\mathcal{F}}_{i,:n_I}}^{n_I}$ . Therefore,  $\mathrm{T}_{j_0,:}$  itself resides in  $\mathbb{R}_{\hat{\mathcal{F}}_{i,:n_I}}^{n_I}$ , i.e.,  $\mathrm{T}_{j_0,:} \in \mathbb{R}_{\hat{\mathcal{F}}_{i,:n_I}}^{n_I}$ . Thus
$$
\forall j \in \mathcal {F} _ {i,: n _ {I}}, \mathrm {T} _ {j,:} \in \mathbb {R} _ {\hat {\mathcal {F}} _ {i,: n _ {I}}} ^ {n _ {I}}. \tag {47}
$$
$$
\forall (i, j) \in \mathcal {F} _ {:; n _ {I}}, \{i \} \times \mathcal {T} _ {j,:} \subset \hat {\mathcal {F}} _ {:; n _ {I}}. \tag {48}
$$
Since  $\mathbf{J}_{\mathbf{f}}(\mathbf{s}^{(\ell)})_{:,n_I}$  and  $\mathbf{J}_{\hat{\mathbf{f}}}(\hat{\mathbf{s}}^{(\ell)})_{:,n_I}$  have full column rank  $n_I$ ,  $\mathbf{T}(\mathbf{s}^{(\ell)})$  must have a non-zero determinant. Otherwise, it would follow that the rank of  $\mathbf{T}(\mathbf{s}^{(\ell)})$  is less than  $n_I$ , which would imply a contradiction that  $\mathbf{J}_{\hat{\mathbf{f}}}(\hat{\mathbf{s}}^{(\ell)})_{:,n_I} = \mathbf{J}_{\mathbf{f}}(\mathbf{s}^{(\ell)})_{:,n_I}\mathbf{T}(\mathbf{s}^{(\ell)})$  has a column rank less than  $n_I$ .
The determinant of the matrix  $\mathbf{T}(\mathbf{s}^{(\ell)})$  can be represented as its Leibniz formula as
$$
\det  (\mathbf {T} (\mathbf {s} ^ {(\ell)})) = \sum_ {\sigma \in \mathcal {S} _ {n _ {I}}} \left(\operatorname {s g n} (\sigma) \prod_ {i = 1} ^ {n _ {I}} \mathbf {T} (\mathbf {s} ^ {(\ell)}) _ {i, \sigma (i)}\right) \neq 0, \tag {49}
$$
where  $S_{n_I}$  is the set of  $n_I$ -permutations. Therefore, there is at least one term in the sum that is non-zero, i.e.,
$$
\exists \sigma \in \mathcal {S} _ {n _ {I}}, \forall i \in \{1, \dots , n _ {I} \}, \operatorname {s g n} (\sigma) \prod_ {i = 1} ^ {n _ {I}} \mathbf {T} \left(\mathbf {s} ^ {(\ell)}\right) _ {i, \sigma (i)} \neq 0, \tag {50}
$$
$$
\exists \sigma \in \mathcal {S} _ {n _ {I}}, \forall i \in \{1, \dots , n _ {I} \}, \mathbf {T} (\mathbf {s} ^ {(\ell)}) _ {i, \sigma (i)} \neq 0. \tag {51}
$$
Then we can conclude that this  $\sigma$  must present in the support of  $\mathbf{T}(\mathbf{s})$  since  $\mathbf{s}^{(\ell)}\in \mathbf{s}$ . Therefore, it follows that
$$
\forall j \in \{1, \dots , n _ {I} \}, \sigma (j) \in \mathcal {T} _ {j,:}. \tag {52}
$$
Together with Eq. (48), we have
$$
\forall (i, j) \in \mathcal {F} _ {:; n _ {I}}, (i, \sigma (j)) \in \{i \} \times \mathcal {T} _ {j,:} \subset \hat {\mathcal {F}} _ {:; n _ {I}}. \tag {53}
$$
$$
\sigma \left(\mathcal {F} _ {:,: n _ {I}}\right) = \{(i, \sigma (j)) \mid (i, j) \in \mathcal {F} _ {:,: n _ {I}} \}. \tag {54}
$$
$$
\sigma \left(\mathcal {F} _ {:; n _ {I}}\right) \subset \hat {\mathcal {F}} _ {:; n _ {I}}. \tag {55}
$$
$$
\left| \hat {\mathcal {F}} _ {:,: n _ {I}} \right| \leq \left| \mathcal {F} _ {:,: n _ {I}} \right| = \left| \sigma \left(\mathcal {F} _ {:,: n _ {I}}\right) \right|. \tag {56}
$$
Combined with Eq. (55), we have
$$
\sigma \left(\mathcal {F} _ {:,: n _ {I}}\right) = \hat {\mathcal {F}} _ {:,: n _ {I}}. \tag {57}
$$
Suppose  $\mathbf{T}(\mathbf{s})\neq \mathbf{D}_I(\mathbf{s})^{-1}\mathbf{P}_I^{-1}$ , then
$$
\exists j _ {1} \neq j _ {2}, \mathcal {T} _ {j _ {1},:} \cap \mathcal {T} _ {j _ {2},:} \neq \emptyset . \tag {58}
$$
Additionally, consider  $j_3 \in \{1, \dots, n_I\}$  for which
$$
\sigma \left(j _ {3}\right) \in \mathcal {T} _ {j _ {1},:} \cap \mathcal {T} _ {j _ {2},:}. \tag {59}
$$
Since  $j_1 \neq j_2$ , we can assume  $j_3 \neq j_1$  without loss of generality. Based on Assumption ii, there exists  $\mathcal{C}_{j_1} \ni j_1$  such that  $\bigcap_{i \in \mathcal{C}_{j_1}} \mathcal{F}_{i,:n_I} = \{j_1\}$ . Because
$$
j _ {3} \notin \{j _ {1} \} = \bigcap_ {i \in \mathcal {C} _ {j _ {1}}} \mathcal {F} _ {i,: n _ {I}}, \tag {60}
$$
$$
j _ {3} \notin \mathcal {F} _ {i _ {3},: n _ {I}}. \tag {61}
$$
Since  $j_{1}\in \mathcal{F}_{i_{3},:n_{I}}$ , it follows that  $(i_3,j_1)\in \mathcal{F}_{\therefore ,n_I}$ . Therefore, according to Eq. (48), we have
$$
\left\{i _ {3} \right\} \times \mathcal {T} _ {j _ {1},:} \subset \hat {\mathcal {F}} _ {:,: n _ {I}}. \tag {62}
$$
$$
\left(i _ {3}, \sigma \left(j _ {3}\right)\right) \in \left\{i _ {3} \right\} \times \mathcal {T} _ {j _ {1},..} \tag {63}
$$
Then by Eqs. (62) and (63), we have
$$
\left(i _ {3}, \sigma \left(j _ {3}\right)\right) \in \hat {\mathcal {F}} _ {:,: n _ {I}}. \tag {64}
$$
This further implies  $(i_3,j_3)\in \mathcal{F}_{:,n_I}$  by Eqs. (54) and (57), which contradicts Eq. (61). Thus, we have proven by contradiction that  $\mathbf{T}(\mathbf{s}) = \mathbf{D}_I(\mathbf{s})^{-1}\mathbf{P}_I^{-1}$ . By replacing  $\mathbf{T}(\mathbf{s})$  with  $\mathbf{D}_I(\mathbf{s})^{-1}\mathbf{P}_I^{-1}$  in Eq. (43), we obtain Eq. (42), which is the goal.
i. For each  $i \in \{1, \dots, n_I\}$ , there exist  $\{\mathbf{s}^{(\ell)}\}_{\ell=1}^{\left|\mathcal{F}_{i:n_I}\right|}$  and a matrix  $T \in \mathcal{T}$  s.t.  $\operatorname{span}\{\mathbf{J}_{\mathbf{f}}(\mathbf{s}^{(\ell)})_{i:n_I}\}_{\ell=1}^{\left|\mathcal{F}_{i:n_I}\right|} = \mathbb{R}_{\mathcal{F}_{i:n_I}}^{n_I}$  and  $\left[\mathbf{J}_{\mathbf{f}}(\mathbf{s}^{(\ell)})\mathbf{T}\right]_{i:n_I} \in \mathbb{R}_{\hat{\mathcal{F}}_{i:n_I}}^{n_I}$ .
ii. There exist  $2n_{D} + 1$  values of  $\mathbf{u}$ , i.e.,  $\mathbf{u}_i$  with  $i \in \{0,1,\dots,2n_{D}\}$ , s.t. the  $2n_{D}$  vectors  $\mathbf{w}(\mathbf{s}_{D},\mathbf{u}_{i}) - \mathbf{w}(\mathbf{s}_{D},\mathbf{u}_{0})$  with  $i \in \{1,\dots,2n_{D}\}$  are linearly independent, where vector  $\mathbf{w}(\mathbf{s}_{D},\mathbf{u}_{i})$  is defined as follows:
$$
\begin{array}{l} \mathbf {J} _ {\mathbf {f}} (\mathbf {s}) = \mathbf {J} _ {\hat {\mathbf {f}} \circ \mathbf {h}} (\mathbf {s}) \tag {65} \\ = \mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}) \mathbf {J} _ {\mathbf {h}} (\mathbf {s}). \\ \end{array}
$$
Because  $\mathbf{J}_{\hat{\mathbf{f}}}(\hat{\mathbf{s}})$  and  $\mathbf{J}_{\mathbf{f}}(\mathbf{s})$  have full column rank,  $\mathbf{J}_{\mathbf{h}}(\mathbf{s})$  must be invertible and have a non-zero determinant. Otherwise, one of them would not be of full column rank, which leads to a contradiction.
$$
p _ {\mathbf {s} | \mathbf {u}} (\mathbf {s} | \mathbf {u}) \left| \det  \left(\mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}})\right) \right| = p _ {\mathbf {s}} (\hat {\mathbf {s}} | \mathbf {u}). \tag {66}
$$
$$
\log p _ {\mathbf {s} | \mathbf {u}} (\mathbf {s} | \mathbf {u}) + \log | \det  (\mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}})) | = \log p _ {\hat {\mathbf {s}} | \mathbf {u}} (\hat {\mathbf {s}} | \mathbf {u}). \tag {67}
$$
$$
p _ {\mathbf {s} | \mathbf {u}} (\mathbf {s} | \mathbf {u}) = \prod_ {i = 1} ^ {n _ {I}} p _ {s _ {i}} \left(s _ {i}\right) \prod_ {j = c _ {1}} ^ {c _ {d}} p _ {\mathbf {s} _ {j} | \mathbf {u}} \left(\mathbf {s} _ {j} | \mathbf {u}\right), \tag {68}
$$
$$
p _ {\hat {\mathbf {s}} | \mathbf {u}} (\hat {\mathbf {s}} | \mathbf {u}) = \prod_ {i = 1} ^ {n _ {I}} p _ {\hat {s} _ {i}} (\hat {s} _ {i}) \prod_ {j = c _ {1}} ^ {c _ {d}} p _ {\hat {\mathbf {s}} _ {j} | \mathbf {u}} (\hat {\mathbf {s}} _ {j} | \mathbf {u}).
$$
Together with Eq. (67), we have
$$
\begin{array}{l} \sum_ { \begin{array}{c} i \\ n _ {1} \end{array} } ^ {n _ {I}} \log p _ {s _ {i}} (s _ {i}) + \sum_ { \begin{array}{c} j = c _ {1} \\ c _ {1} \end{array} } ^ {c _ {d}} \log p _ {\mathbf {s} _ {j} | \mathbf {u}} (\mathbf {s} _ {j} | \mathbf {u}) + \log | \det  (\mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}})) | \tag {69} \\ = \sum_ {i} ^ {n _ {I}} \log p _ {\hat {s} _ {i}} (\hat {s} _ {i}) + \sum_ {j = c _ {1}} ^ {c _ {d}} \log p _ {\hat {\mathbf {s}} _ {j} | \mathbf {u}} (\hat {\mathbf {s}} _ {j} | \mathbf {u}). \\ \end{array}
$$
Therefore, for  $\mathbf{u} = \mathbf{u}_0,\dots ,\mathbf{u}_{2n_D}$ , we have  $2n_{D} + 1$  such equations. Subtracting each equation corresponding to  $\mathbf{u}_1,\ldots ,\mathbf{u}_{2n_D}$  with the equation corresponding to  $\mathbf{u}_0$  results in  $2n_{D}$  equations:
$$
\begin{array}{l} \sum_ {i = c _ {1}} ^ {c _ {d}} \left(\log p _ {\mathbf {s} _ {i} | \mathbf {u} _ {j}} (\mathbf {s} _ {i} | \mathbf {u} _ {j}) - \log p _ {\mathbf {s} _ {i} | \mathbf {u} _ {0}} (\mathbf {s} _ {i} | \mathbf {u} _ {0})\right) \\ = \sum_ {i = c _ {1}} ^ {c _ {d}} \left(\log p _ {\hat {\mathbf {s}} _ {i} | \mathbf {u} _ {j}} (\hat {\mathbf {s}} _ {i} | \mathbf {u} _ {j}) - \log p _ {\hat {\mathbf {s}} _ {i} | \mathbf {u} _ {0}} (\hat {\mathbf {s}} _ {i} | \mathbf {u} _ {0})\right). \\ \end{array}
$$
Then we take the derivatives of both sides of Eq. (70) w.r.t.  $\hat{\mathbf{s}}_k$  and  $\hat{\mathbf{s}}_v$  where  $k, v \in \{1, \dots, n\}$  and  $k \neq v$ . Besides, if both  $k > n_I$  and  $v > n_I$ , then they are not indices of the same subspace. It is clear that the RHS of Eq. (70) equals to zero. For the  $i$ -th term of the summation on the LHS, we have the following equation after taking the derivatives:
$$
\begin{array}{l} \sum_ {l = i ^ {(l)}} ^ {i ^ {(h)}} \left(\left(\frac {\partial^ {2} \log p _ {\mathbf {s} _ {i} | \mathbf {u} _ {j}} (\mathbf {s} _ {i} | \mathbf {u} _ {j})}{(\partial s _ {l}) ^ {2}} - \frac {\partial^ {2} \log p _ {\mathbf {s} _ {i} | \mathbf {u} _ {0}} (\mathbf {s} _ {i} | \mathbf {u} _ {0})}{(\partial s _ {l}) ^ {2}}\right) \cdot \frac {\partial s _ {l}}{\partial \hat {s} _ {k}} \frac {\partial s _ {l}}{\partial \hat {s} _ {v}} \right. \tag {71} \\ + \left(\frac {\partial \log p _ {\mathbf {s} _ {i} | \mathbf {u} _ {j}} (\mathbf {s} _ {i} | \mathbf {u} _ {j})}{\partial s _ {l}} - \frac {\partial \log p _ {\mathbf {s} _ {i} | \mathbf {u} _ {0}} (\mathbf {s} _ {i} | \mathbf {u} _ {0})}{\partial s _ {l}}\right) \cdot \left. \frac {\partial^ {2} s _ {l}}{\partial \hat {s} _ {k} \partial \hat {s} _ {v}}\right) = 0, \\ \end{array}
$$
where  $i_{l}$  and  $i_{h}$  corresponds to the minimum and maximum indices of elements in  $\mathbf{s}_i = (s_{i_l},\dots ,s_{i_h})$ . By iterating  $i$  from  $c_{1}$  to  $c_{d}$ , we are also iterating  $l$  from  $n_I + 1$  to  $n$ . Thus by considering all those equations as well as iterating  $j$  in  $\mathbf{u}_j$  from 0 to  $2n_{D}$ , we have a linear system with a  $2n_{D}\times 2n_{D}$  coefficient matrix.
Then, according to Assumption ii, the coefficient matrix of the linear system has full rank. Thus, the only solution of Eq. (71) is  $\frac{\partial s_l}{\partial\hat{s}_k}\frac{\partial s_l}{\partial\hat{s}_v} = 0$  and  $\frac{\partial^2s_l}{\partial\hat{s}_k\partial\hat{s}_v} = 0$
Note that  $k \neq v$  and, if both  $k > n_I$  and  $v > n_I$ , they are not indices of sources in the same subspace. Besides,  $\frac{\partial s_l}{\partial \hat{s}_k} \frac{\partial s_l}{\partial \hat{s}_v} = 0$  indicates that it is impossible for both  $\frac{\partial s_l}{\partial \hat{s}_k}$  and  $\frac{\partial s_l}{\partial \hat{s}_v}$  to be non-zero. Furthermore, because  $h$  is invertible, they cannot be both zero, indicating that it is either  $\frac{\partial s_l}{\partial \hat{s}_k} = 0$  or  $\frac{\partial s_l}{\partial \hat{s}_v} = 0$ . Then, because  $\mathbf{s}_I$  has invariant distribution w.r.t.  $\mathbf{u}$ , if  $\frac{\partial s_l}{\partial \hat{s}_k} \neq 0$  (i.e.,  $\frac{\partial s_l}{\partial \hat{s}_v} = 0$ ), then  $\hat{k} \notin \{1, \dots, n_I\}$ . Otherwise,  $s_l$  will also be invariant, which is a contradiction. Therefore,  $\hat{k}$  can only be the index of an estimated source from one independent subspace, which, together with the invertibility, leads to the conclusion that  $\mathbf{s}_D$  is a composition of an invertible subspace-wise transformation and a subspace-wise permutation of  $\mathbf{s}_D'$ . So it is the mapping from  $\hat{\mathbf{s}}_D$  to  $\mathbf{s}_D$  since the subspace-wise transformation is invertible and the inverse of a block-wise permutation matrix is still a block-wise invertible matrix.
Now we have shown the identifiability result for  $\mathbf{s}_D = (s_{n_I + 1},\dots ,s_n)$ , then we need to show that for the remaining sources  $\mathbf{s}_I = (s_1,\dots ,s_{n_I})$
By using chain rule repeatedly, we have
$$
\begin{array}{l} \mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}) = \mathbf {J} _ {\mathbf {f} \circ \mathbf {h} ^ {- 1}} (\hat {\mathbf {s}}) \\ = \mathbf {J} _ {\mathbf {f}} \left(\mathbf {h} ^ {- 1} (\hat {\mathbf {s}})\right) \mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}}) \tag {72} \\ = \mathbf {J} _ {\mathbf {f}} (\mathbf {s}) \mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}}). \\ \end{array}
$$
Since we have shown that, for every  $l \in \{n_I + 1, \dots, n\}$  and  $k \in \{1, \dots, n_I\}$ ,  $\frac{\partial s_l}{\partial \hat{s}_k} = 0$ , all entries of  $\mathbf{J}_{\mathbf{h}^{-1}}(\hat{\mathbf{s}})_{n_I + 1:n,:n_I}$  must be zero.
$$
\begin{array}{l} \mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}) _ {:,: n _ {I}} = \mathbf {J} _ {\mathbf {f}} (\mathbf {s}) \mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}}) _ {:,: n _ {I}} \\ \stackrel {(\star)} {=} \mathbf {J} _ {\mathbf {f}} (\mathbf {s}): _ {:: n _ {I}} \mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}}): _ {n _ {I},: n _ {I}}, \tag {73} \\ \end{array}
$$
where Eq.  $(\star)$  is directly from the result that all entries in  $\mathbf{J}_{\mathbf{h}^{-1}}(\mathbf{s})_{n_I + 1::: n_I}$  are zero.
Based on the proof of Theorem 4.2 in Kong et al. (2022) and Assumption iii, particularly, steps 1, 2, and 3, one can show that  $\hat{\mathbf{s}}_I$  does not depend on  $\mathbf{s}_D$ . Let  $\mathbf{D}(\mathbf{s})$  represents a diagonal matrix and  $\mathbf{P}$
represent a permutation matrix. Thus, our goal is equivalent to show that  $\mathbf{J}_{\mathbf{h}}(\mathbf{s}):_{n_I,:n_I} = \mathbf{P}_I\mathbf{D}_I(\mathbf{s})$  or  $\mathbf{J}_{\mathbf{h}^{-1}}(\hat{\mathbf{s}}):_{n_I,:n_I} = \mathbf{J}_{\mathbf{h}^{-1}}(\hat{\mathbf{s}}):_{n_I,:n_I} = \mathbf{J}_{\mathbf{h}}(\mathbf{s})^{-1}:_{n_I,:n_I} = \mathbf{D}_I(\mathbf{s})^{-1}\mathbf{P}_I^{-1}$ . Then we need to prove that
$$
\mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}) _ {:,: n _ {I}} = \mathbf {J} _ {\mathbf {f}} (\mathbf {s}) _ {:,: n _ {I}} \mathbf {D} _ {I} (\mathbf {s}) ^ {- 1} \mathbf {P} _ {I} ^ {- 1}. \tag {74}
$$
$$
\mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}):;: n _ {I} = \mathbf {J} _ {\mathbf {f}} (\mathbf {s}):;: n _ {I} \mathbf {T} (\mathbf {s}), \tag {75}
$$
$$
\operatorname {s p a n} \left\{\mathbf {J} _ {\mathbf {f}} \left(\mathbf {s} ^ {(\ell)}\right) _ {i,: n _ {I}} \right\} _ {\ell = 1} ^ {\left| \mathcal {F} _ {i,: n _ {I}} \right|} = \mathbb {R} _ {\mathcal {F} _ {i,: n _ {I}}} ^ {n}. \tag {76}
$$
$$
e _ {j _ {0}} = \sum_ {\ell \in \mathcal {F} _ {i,: n _ {I}}} \alpha_ {\ell} \mathbf {J} _ {\mathbf {f}} \left(\mathbf {s} ^ {(\ell)}\right) _ {i,: n _ {I}}, \tag {77}
$$
$$
\mathrm {T} _ {j _ {0,:}} = e _ {j _ {0}} \mathrm {T} = \sum_ {\ell \in \mathcal {F} _ {i,: n _ {I}}} \alpha_ {\ell} \mathbf {J} _ {\mathbf {f}} \left(\mathbf {s} ^ {(\ell)}\right) _ {i,: n _ {I}} \mathrm {T} \in \mathbb {R} _ {\hat {\mathcal {F}} _ {i,: n _ {I}}} ^ {n _ {I}}, \tag {78}
$$
where the final “ $\in$ ” follows from Assumption i that each element in the summation belongs to  $\mathbb{R}_{\hat{\mathcal{F}}_{i:n_I}}^{n_I}$ . Thus
$$
\forall j \in \mathcal {F} _ {i,: n _ {I}}, \mathrm {T} _ {j,:} \in \mathbb {R} _ {\hat {\mathcal {F}} _ {i,: n _ {I}}} ^ {n _ {I}}. \tag {79}
$$
$$
\forall (i, j) \in \mathcal {F} _ {:,: n _ {I}}, \{i \} \times \mathcal {T} _ {j,:} \subset \hat {\mathcal {F}} _ {:,: n _ {I}}. \tag {80}
$$
$$
\det  (\mathbf {T} (\mathbf {s} ^ {(\ell)})) = \sum_ {\sigma \in \mathcal {S} _ {n _ {I}}} \left(\operatorname {s g n} (\sigma) \prod_ {i = 1} ^ {n _ {I}} \mathbf {T} (\mathbf {s} ^ {(\ell)}) _ {i, \sigma (i)}\right) \neq 0, \tag {81}
$$
where  $S_{n_I}$  is the set of  $n_I$ -permutations. Thus, there is at least one non-zero term in the sum, i.e.,
$$
\exists \sigma \in \mathcal {S} _ {n _ {I}}, \forall i \in \{1, \dots , n _ {I} \}, \operatorname {s g n} (\sigma) \prod_ {i = 1} ^ {n _ {I}} \mathbf {T} \left(\mathbf {s} ^ {(\ell)}\right) _ {i, \sigma (i)} \neq 0, \tag {82}
$$
$$
\exists \sigma \in \mathcal {S} _ {n _ {I}}, \forall i \in \{1, \dots , n _ {I} \}, \mathbf {T} \left(\mathbf {s} ^ {(\ell)}\right) _ {i, \sigma (i)} \neq 0. \tag {83}
$$
Then we can see that this  $\sigma$  must present in the support of  $\mathbf{T}(\mathbf{s})$  since  $\mathbf{s}^{(\ell)}\in \mathbf{s}$ . It follows that
$$
\forall j \in \{1, \dots , n _ {I} \}, \sigma (j) \in \mathcal {T} _ {j,:}. \tag {84}
$$
Together with Eq. (80), we have
$$
\forall (i, j) \in \mathcal {F} _ {:; n _ {I}}, (i, \sigma (j)) \in \{i \} \times \mathcal {T} _ {j,:} \subset \hat {\mathcal {F}} _ {:; n _ {I}}. \tag {85}
$$
$$
\sigma \left(\mathcal {F} _ {:; n _ {I}}\right) = \{(i, \sigma (j)) \mid (i, j) \in \mathcal {F} _ {:; n _ {I}} \}. \tag {86}
$$
$$
\sigma \left(\mathcal {F} _ {:,: n _ {I}}\right) \subset \hat {\mathcal {F}} _ {:,: n _ {I}}. \tag {87}
$$
$$
| \hat {\mathcal {F}} _ {:,: n _ {I}} | \leq | \mathcal {F} _ {:,: n _ {I}} | = | \sigma (\mathcal {F} _ {:,: n _ {I}}) |. \tag {88}
$$
Together with Eq. (87), we have
$$
\sigma \left(\mathcal {F} _ {:; n _ {I}}\right) = \hat {\mathcal {F}} _ {:; n _ {I}}. \tag {89}
$$
$$
\exists j _ {1} \neq j _ {2}, \mathcal {T} _ {j _ {1},:} \cap \mathcal {T} _ {j _ {2},:} \neq \emptyset . \tag {90}
$$
$$
\sigma \left(j _ {3}\right) \in \mathcal {T} _ {j _ {1},:} \cap \mathcal {T} _ {j _ {2},:}. \tag {91}
$$
Since  $j_1 \neq j_2$ , we can assume  $j_3 \neq j_1$  without loss of generality. Based on Assumption iv, there exists  $\mathcal{C}_{j_1} \ni j_1$  such that  $\bigcap_{i \in \mathcal{C}_{j_1}} \mathcal{F}_{i:n_I} = \{j_1\}$ . Because
$$
j _ {3} \notin \{j _ {1} \} = \bigcap_ {i \in \mathcal {C} _ {j _ {1}}} \mathcal {F} _ {i,: n _ {I}}, \tag {92}
$$
$$
j _ {3} \notin \mathcal {F} _ {i _ {3},: n _ {I}}. \tag {93}
$$
Since  $j_{1}\in \mathcal{F}_{i_{3},:n_{I}}$ , we have  $(i_3,j_1)\in \mathcal{F}_{:,n_I}$ . Therefore, according to Eq. (80), we have
$$
\left\{i _ {3} \right\} \times \mathcal {T} _ {j _ {1,:}} \subset \hat {\mathcal {F}} _ {:,: n _ {I}}. \tag {94}
$$
$$
\left(i _ {3}, \sigma \left(j _ {3}\right)\right) \in \left\{i _ {3} \right\} \times \mathcal {T} _ {j _ {1},..} \tag {95}
$$
Then by Eqs. (94) and (95), we have
$$
\left(i _ {3}, \sigma \left(j _ {3}\right)\right) \in \hat {\mathcal {F}} _ {:,: n _ {I}}, \tag {96}
$$
which implies  $(i_3,j_3)\in \mathcal{F}_{\therefore n_I}$  by Eqs. (86) and (89), therefore contradicting Eq. (93). Thus, we have proven by contradiction that  $\mathbf{T}(\hat{\mathbf{s}}) = \mathbf{D}_I(\mathbf{s})^{-1}\mathbf{P}_I^{-1}$ . By replacing  $\mathbf{T}(\mathbf{s})$  with  $\mathbf{D}_I(\mathbf{s})^{-1}\mathbf{P}_I^{-1}$  in Eq. (75), we obtain Eq. (74), which is the goal.
Therefore,  $\mathbf{s}_I$  is identifiable up to a composition of a component-wise invertible transformation and a permutation, and  $\mathbf{s}_D$  is identifiable up to a composition of a subspace-wise invertible transformation and a subspace-wise permutation.
i. For each  $i \in \{1, \dots, n_I\}$ , there exist  $\{\mathbf{s}^{(\ell)}\}_{\ell=1}^{|F_{i:n_I}|}$  and a matrix  $T \in \mathcal{T}$  s.t.  $\operatorname{span}\{\mathbf{J}_{\mathbf{f}}(\mathbf{s}^{(\ell)})_{i:n_I}\}_{\ell=1}^{|F_{i:n_I}|} = \mathbb{R}_{\mathcal{F}_{i:n_I}}^{n_I}$  and  $[\mathbf{J}_{\mathbf{f}}(\mathbf{s}^{(\ell)})\mathrm{T}]_{i:n_I} \in \mathbb{R}_{\hat{\mathcal{F}}_{i:n_I}}^{n_I}$ .
ii. There exist  $2n_{D} + 1$  values of  $\mathbf{u}$ , i.e.,  $\mathbf{u}_i$  with  $i \in \{0,1,\dots,2n_{D}\}$ , s.t. the  $2n_{D}$  vectors  $\mathbf{w}(\mathbf{s}_{D},\mathbf{u}_{i}) - \mathbf{w}(\mathbf{s}_{D},\mathbf{u}_{0})$  with  $i \in \{1,\dots,2n_{D}\}$  are linearly independent, where vector  $\mathbf{w}(\mathbf{s}_D,\mathbf{u})$  is defined as follows:
$$
\mathbf {v} ^ {\prime} (\mathbf {s} _ {D}, \mathbf {u} _ {i}) = \Big (\frac {\partial^ {2} \log p (s _ {n _ {I} + 1} | \mathbf {u} _ {i})}{(\partial s _ {n _ {I} + 1}) ^ {2}}, \dots , \frac {\partial^ {2} \log p (s _ {n} | \mathbf {u} _ {i})}{(\partial s _ {n}) ^ {2}} \Big).
$$
$$
\begin{array}{l} \mathbf {J} _ {\mathbf {f}} (\mathbf {s}) = \mathbf {J} _ {\hat {\mathbf {f}} \circ \mathbf {h}} (\mathbf {s}) \tag {97} \\ = \mathbf {J} _ {\hat {\mathbf {f}}} (\hat {\mathbf {s}}) \mathbf {J} _ {\mathbf {h}} (\mathbf {s}). \\ \end{array}
$$
$$
p _ {\mathbf {s} | \mathbf {u}} (\mathbf {s} | \mathbf {u}) | \det  (\mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}})) | = p _ {\tilde {\mathbf {s}}} (\hat {\mathbf {s}} | \mathbf {u}). \tag {98}
$$
$$
\log p _ {\mathbf {s} \mid \mathbf {u}} (\mathbf {s} \mid \mathbf {u}) + \log | \det  (\mathbf {J} _ {\mathbf {h} ^ {- 1}} (\hat {\mathbf {s}})) | = \log p _ {\tilde {\mathbf {s}} \mid \mathbf {u}} (\hat {\mathbf {s}} \mid \mathbf {u}). \tag {99}
$$
$$
\begin{array}{l} p _ {\mathbf {s} | \mathbf {u}} (\mathbf {s} | \mathbf {u}) = \prod_ {i = 1} ^ {n _ {I}} p _ {s _ {i}} \left(s _ {i}\right) \prod_ {j = n _ {I} + 1} ^ {n} p _ {s _ {j} | \mathbf {u}} \left(s _ {j} | \mathbf {u}\right) = \prod_ {i = 1} ^ {n} p _ {s _ {i} | \mathbf {u}} \left(\hat {s} _ {i} | \mathbf {u}\right), \tag {100} \\ p _ {\hat {\mathbf {s}} | \mathbf {u}} (\hat {\mathbf {s}} | \mathbf {u}) = \prod_ {i = 1} ^ {n _ {I}} p _ {\hat {s} _ {i}} (\hat {s} _ {i}) \prod_ {j = n _ {I} + 1} ^ {n} p _ {\hat {s} _ {j} | \mathbf {u}} (\hat {s} _ {j} | \mathbf {u}) = \prod_ {i = 1} ^ {n} p _ {\hat {s} _ {i} | \mathbf {u}} (\hat {s} _ {i} | \mathbf {u}). \\ \end{array}
$$
Together with Eq. (99), we have