SlowGuess/NeurIPS_2023_OCR huggingface dataset:
If  $\widehat{\mathbf{C}}_{:,i} \neq \mathbf{0}$  then  $i \in \operatorname{supp}\left(\widehat{\mathbf{C}}\right)$ , and the expression above constitutes a linear combination of the support columns with not all coefficients non-zero. This contradicts Lemma C.9. Thus  $\mathbf{C}_{:,i} = \mathbf{0}$  and  $\overline{\hat{a}}_{ji} = 0 \quad \forall j \in \operatorname{supp}\left(\widehat{\mathbf{C}}\right)$ .
We are now ready to prove Prop. C.16.
Proposition C.16. If the data  $\mathbf{X}$  are indexed such that  $\mathbf{A}$  is (strictly) upper triangular, then so is  $\hat{\mathbf{A}}$ .
Remark C.17. Based on lemma C.14, in our proof of Prop. C.16 next we may only care about the transitive weights, namely we will only need to show that  $\bar{a}_{ji} = 0$  for all  $i < j$ .
Proof. We first choose a  $k$ -pair  $\mathbf{C}$ ,  $\widehat{\mathbf{C}}$  such that the support  $S$  of  $\mathbf{C}$  is concentrated to the last  $k$  columns, namely  $\mathbf{C}_{\cdot,i} = 0$  for  $i = 1,\dots,d - k$ . Then the corresponding data submatrix  $\mathbf{X}$  will necessarily have the same sparsity pattern since the values are computed according to predecessors, which in our case lie in smaller indices. Thus  $\mathbf{X}_{\cdot,i} = \mathbf{0}\quad \forall i = 1,\dots,d - k$  and necessarily  $\widehat{\mathbf{C}}$  will follow the same sparsity pattern. Moreover, according to Lemma C.15 we get that
$$
\bar {\hat {a}} _ {j i} = 0, \text {f o r a l l} 1 \leq i \leq d - k \text {a n d} d - k + 1 \leq j \leq d. \tag {34}
$$
We notice that the desired condition is fulfilled for  $i = d - k$ , i.e. it has zero influence from a node with larger index. We now prove the same sequentially for  $i = d - k - 1, d - k - 2, \ldots, 1$  by moving each time (one step) left the leftmost index of the support  $S$ . We prove the following statement by induction:
$$
P (l): \bar {\hat {a}} _ {j i} = 0 \text {f o r} l <   j, i <   j, i \leq d - k \tag {35}
$$
We know that  $P(d - k)$  is true and  $P(1)$  gives the required relation for all indices  $i = 1, \dots, d - k$ . Now we assume that  $P(l)$  holds. If we pick  $k$ -pair  $\mathbf{C}$ ,  $\widehat{\mathbf{C}}$  such that  $\mathbf{C}$  has support  $S = \{l, d - k + 2, d - k + 3, \dots, d\}$ . Then  $\mathbf{X}_{:,i} = \mathbf{0}$  for all  $i < l$  which with Lemma C.15 gives  $\widehat{\mathbf{C}}_{:,i} = \mathbf{0}$  for  $i < l$  which means  $\text{supp}(\widehat{\mathbf{C}}) \subset \{l, l + 1, \dots, d\}$  and  $\overline{\hat{a}}_{ji} = 0$  for  $j \in \text{supp}(\widehat{\mathbf{C}})$ . Note, that by the induction hypothesis we have that  $\overline{\hat{a}}_{jl} = 0$  for all  $l < j$ . However it is true that  $\mathbf{X}_{:,l} = \mathbf{C}_{:,l}$  and also
$$
\mathbf {X} _ {:, l} = \sum_ {j \in \operatorname {s u p p} (\widehat {\mathbf {C}})} ^ {d} \bar {\widehat {a}} _ {j l} \widehat {\mathbf {C}} _ {:, j} + \widehat {\mathbf {C}} _ {:, l} = \widehat {\mathbf {C}} _ {:, l} \tag {36}
$$
Therefore  $l \in \operatorname{supp}\left(\widehat{\mathbf{C}}\right)$  and thus  $\overline{\hat{a}}_{li} = 0$  for all  $i < l$  which combined with  $P(l)$  gives
$$
\bar {\hat {a}} _ {j i} = 0 \text {f o r} l - 1 <   j, i <   j, i \leq d - k \tag {37}
$$
which is exactly  $P(l - 1)$  and the induction is complete.
Now it remains to show that  $\bar{\hat{a}}_{ji} = 0$  for  $d - k + 1\leq i\leq d$  and  $i < j$ . We will again proceed constructively using induction. This time we will sequentially choose support  $S$  that is concentrated on the last  $k + 1$  columns and at each step we will move the zero column one index to the right. For  $l = d - k + 1,\dots,d$  let's define:
$$
Q (l): \bar {\bar {a}} _ {j l} = 0 \text {f o r} l <   j \text {a n d} \bar {\bar {a}} _ {j l} = \bar {a} _ {j l} \text {f o r} d - k \leq j <   l \tag {38}
$$
First we show the base case  $Q(d - k + 1)$ . For this we choose a  $k$ -pair  $\mathbf{C}, \widehat{\mathbf{C}}$  such that  $\mathbf{C}$  has support  $S = \{d - k, d - k + 2, d - k + 3, \dots, d\}$ . It is true that  $\mathbf{X}_{:i} = \mathbf{0}$  for  $i = 1, \dots, d - k - 1$  hence  $\widehat{\mathbf{C}}_{:,i} = \mathbf{0}$  for  $i \leq d - k - 1$  and therefore the node  $d - k$  doesn't have any non-zero parents, since also  $\widehat{\bar{a}}_{j(d - k)} = 0$  for  $d - k < j$  from previous claim. Therefore  $\widehat{\mathbf{C}}_{:,d - k} = \mathbf{X}_{:d - k} = \mathbf{C}_{:d - k}$ . Also, for  $l = d - k + 1$
$$
\mathbf {X} _ {:, l} = \bar {a} _ {d - k, l} \mathbf {C} _ {:, d - k} = \bar {a} _ {d - k, l} \widehat {\mathbf {C}} _ {:, d - k} \tag {39}
$$
The equation from  $\widehat{\mathbf{A}}$  gives:
$$
\mathbf {X} _ {:, l} = \sum_ {j = d - k} ^ {d} \bar {\hat {a}} _ {j l} \widehat {\mathbf {C}} _ {:, j} + \widehat {\mathbf {C}} _ {:, l} \Rightarrow (\bar {a} _ {d - k, l} - \bar {\hat {a}} _ {d - k, l}) \widehat {\mathbf {C}} _ {:, d - k} + \sum_ {j = d - k + 1} ^ {d} \bar {\hat {a}} _ {j l} \widehat {\mathbf {C}} _ {:, j} + \widehat {\mathbf {C}} _ {:, l} = \mathbf {0} \tag {40}
$$
From the linear Independence Lemma C.9 of the support of  $\widehat{\mathbf{C}}$  we necessarily need to have  $\widehat{\mathbf{C}}_{:,l} = \mathbf{0}$ ,  $\overline{a}_{d - k,l} = \overline{\hat{a}}_{d - k,l}$  and  $\overline{\hat{a}}_{jl} = 0$  for  $l < j$  which gives the base case.
For the rest of the induction we proceed in a similar manner. We assume with strong induction that all  $Q(d - k + 1), \ldots, Q(l)$  are true and proceed to prove  $Q(l + 1)$ . Given these assumptions we have that
$$
\bar {\bar {a}} _ {j i} = \bar {a} _ {j i} \text {f o r} d - k \leq j <   i, d - k \leq i \leq l \text {a n d} \bar {\bar {a}} _ {j i} = 0 \text {f o r} i <   j, d - k \leq i \leq l \tag {41}
$$
Consider root causes support  $S = \{d - k, d - k + 1, \dots, l, l + 2, \dots, d\}$  (the  $(l + 1)$ -th column is 0) for the  $k$ -pair  $\mathbf{C}, \hat{\mathbf{C}}$ . Then we have the equations:
$$
\mathbf {X} _ {:, d - k} = \mathbf {C} _ {:, d - k} = \widehat {\mathbf {C}} _ {:, d - k}
$$
$$
\mathbf {X} _ {:, d - k + 1} = \bar {a} _ {d - k, d - k + 1} \mathbf {C} _ {:, d - k} + \mathbf {C} _ {:, d - k + 1} = \bar {\hat {a}} _ {d - k, d - k + 1} \widehat {\mathbf {C}} _ {:, d - k} + \widehat {\mathbf {C}} _ {:, d - k + 1} \Rightarrow \widehat {\mathbf {C}} _ {:, d - k + 1} = \mathbf {C} _ {:, d - k + 1}
$$
$$
\mathbf {X} _ {:, l} = \sum_ {j = d - k} ^ {l - 1} \bar {a} _ {j l} \mathbf {C} _ {:, j} + \mathbf {C} _ {:, l} = \sum_ {j = d - k} ^ {l - 1} \hat {\bar {a}} _ {j l} \widehat {\mathbf {C}} _ {:, j} + \widehat {\mathbf {C}} _ {:, l} \Rightarrow \widehat {\mathbf {C}} _ {:, l} = \mathbf {C} _ {:, l}
$$
Where we used the linear independence lemma and sequentially proved that the root causes columns up to  $l$  are equal. The equation for the  $(l + 1)$ -th column now becomes:
$$
\mathbf {X} _ {:, l + 1} = \sum_ {j = d - k} ^ {l} \bar {a} _ {j, l + 1} \mathbf {C} _ {:, j} + \mathbf {C} _ {:, l} = \sum_ {j = d - k} ^ {d} \bar {\hat {a}} _ {j, l + 1} \widehat {\mathbf {C}} _ {:, j} + \widehat {\mathbf {C}} _ {:, l + 1} \tag {42}
$$
$$
\Leftrightarrow \sum_ {j = d - k} ^ {d} \left(\bar {\hat {a}} _ {j, l + 1} - \bar {a} _ {j, l + 1}\right) \widehat {\mathbf {C}} _ {:, j} + \widehat {\mathbf {C}} _ {:, l + 1} = \mathbf {0} \Rightarrow \left\{ \begin{array}{l} \bar {\hat {a}} _ {j, l + 1} = 0 \text {f o r} l + 1 <   j \\ \bar {\hat {a}} _ {j, l + 1} = \bar {a} _ {j, l + 1} \text {f o r} j <   l + 1 \\ \widehat {\mathbf {C}} _ {:, l + 1} = \mathbf {0} \end{array} \right. \tag {43}
$$
where the last set of equalities follows from linear independence. This concludes the induction and the proof.
To complete the proof of Theorem 3.2 it remains to show the following proposition.
Proposition C.18. If both  $\mathbf{A}$ ,  $\widehat{\mathbf{A}}$  are upper triangular then  $\widehat{\mathbf{A}} = \mathbf{A}$ .
For our proof we use the following definition.
Definition C.19. We denote by  $\mathcal{P}_k$  the set of all  $k$ -pairs  $\mathbf{C}_R$ ,  $\widehat{\mathbf{C}}_R$  for all possible support patterns.
Now we proceed to the proof.
Proof. We will show equivalently that  $\widehat{\overline{\mathbf{A}}} = \overline{\mathbf{A}}$  using two inductions. First we show for  $l = 1, \ldots, k$  the following statement.
$P(l): \text{For all } k-\text{pairs } \mathbf{C}, \widehat{\mathbf{C}} \text{ in } \mathcal{P}_k \text{ the first } l \text{ non-zero columns } \mathbf{C}_{:,i_1}, \mathbf{C}_{:,i_2}, \dots, \mathbf{C}_{:,i_l} \text{ and } \widehat{\mathbf{C}}_{:,i_1}, \widehat{\mathbf{C}}_{:,i_2}, \dots, \widehat{\mathbf{C}}_{:,i_l}$  are in the same positions, i.e.  $i_j = \hat{i}_j$  and
- either they are respectively equal  $\mathbf{C}_{:,i_j} = \widehat{\mathbf{C}}_{:,i_j}$
- or  $\mathbf{C}_{:,i_l}$  is in the last possible index, namely  $i_{l} = d - (l - 1)$
For the base case  $P(1)$ , consider a  $k$ -pair  $\mathbf{C}$ ,  $\widehat{\mathbf{C}}$  and let  $i_1$  be the position of the first non-zero root causes column of  $\mathbf{C}$ . Then  $\mathbf{X}_{:,i} = \mathbf{0}$  for  $i < i_1$  and therefore from Lemma C.15  $\widehat{\mathbf{C}}_{:,i} = \mathbf{0}$  for  $i < i_1$ . Hence
$$
\mathbf {X} _ {:, i _ {1}} = \sum_ {j <   i _ {1}} \bar {a} _ {j i _ {1}} \widehat {\mathbf {C}} _ {:, j} + \widehat {\mathbf {C}} _ {:, i _ {1}} = \widehat {\mathbf {C}} _ {:, i _ {1}} \tag {44}
$$
Therefore  $\widehat{\mathbf{C}}_{:,i_1} = \mathbf{C}_{:,i_1}$  and we proved  $P(1)$ , by satisfying both the positioning and the first requirement.
Assuming now that  $P(l)$  holds, we will show  $P(l + 1)$ . Take any  $k$ -pair of  $\mathcal{P}_k$  which we denote by  $\mathbf{C}$ ,  $\widehat{\mathbf{C}}$ . Then, if  $\mathbf{C}_{:,i_l}$  is in the last possible position, then necessarily  $\mathbf{C}_{:,i_{l + 1}}$  is in the last possible position. Moreover, from the induction hypothesis the first  $l$  root causes columns are in the same positions. Therefore in the same manner,  $\widehat{\mathbf{C}}_{:,i_l}$  is in the last position and  $\widehat{\mathbf{C}}_{:,i_{l + 1}}$ , too. This case fulfills the desired statement.
If  $\mathbf{C}_{:,i_l}$  is not in the last position, then from induction hypothesis, the first  $l$  root causes columns are equal. If  $\mathbf{C}_{:,i_{l + 1}}$  is in the last position and the same holds  $\widehat{\mathbf{C}}_{:,i_{l + 1}}$ , the requirement is satisfied. Otherwise  $\hat{i}_{l + 1} < i_{l + 1}$  and the equation for  $\hat{i}_{l + 1}$  gives:
$$
\mathbf {X} _ {:, \hat {i} _ {l + 1}} = \sum_ {j = 1} ^ {l} \overline {{\hat {a}}} _ {i _ {j}, \hat {i} _ {l + 1}} \widehat {\mathbf {C}} _ {:, i _ {j}} + \widehat {\mathbf {C}} _ {:, \hat {i} _ {l + 1}} = \sum_ {j = 1} ^ {l} \bar {a} _ {i _ {j}, \hat {i} _ {l + 1}} \mathbf {C} _ {:, i _ {j}} + \mathbf {0} \Leftrightarrow \sum_ {j = 1} ^ {l} \left(\overline {{\hat {a}}} _ {i _ {j}, \hat {i} _ {l + 1}} - \bar {a} _ {i _ {j}, \hat {i} _ {l + 1}}\right) \widehat {\mathbf {C}} _ {:, i _ {j}} + \widehat {\mathbf {C}} _ {:, \hat {i} _ {l + 1}} = \mathbf {0}
$$
According to linear independence Lemma C.9 we necessarily derive  $\widehat{\mathbf{C}}_{:,i_{l + 1}} = \mathbf{0}$ , absurd. Thus  $\hat{i}_{l + 1} = i_{l + 1}$  and the induction statement is fulfilled in that case.
It remains to consider the case where  $\mathbf{C}_{:,i_{l + 1}}$  is not in the last position. Since,  $i_{l + 1}$  is not the last position there exists a  $k$ -pair  $\mathbf{C}'$ ,  $\widehat{\mathbf{C}}'$  such that the column  $i_{l + 1}$  is zero and the  $(l + 1)$ -root causes column of  $\mathbf{C}'$  lies at  $i_{l + 1}' > i_{l + 1}$ . The equation at  $i_{l + 1}$  for  $\mathbf{X}'$  gives:
$$
\mathbf {X} _ {:, i _ {l + 1}} ^ {\prime} = \sum_ {j = 1} ^ {l} \bar {a} _ {i _ {j}, i _ {l + 1}} \mathbf {C} _ {:, i _ {j}} ^ {\prime} = \sum_ {j = 1} ^ {l} \hat {\bar {a}} _ {i _ {j}, \hat {i} _ {l + 1}} \hat {\mathbf {C}} _ {:, i _ {j}} ^ {\prime} + \hat {\mathbf {C}} _ {:, \hat {i} _ {l + 1}} ^ {\prime} \tag {45}
$$
For the pair  $\mathbf{C}'$ ,  $\widehat{\mathbf{C}}'$  the induction hypothesis holds, thus we derive:
$$
\sum_ {j = 1} ^ {l} \left(\bar {\hat {a}} _ {i _ {j}, \hat {i} _ {l + 1}} - \bar {a} _ {i _ {j}, i _ {l + 1}}\right) \widehat {\mathbf {C}} _ {:, i _ {j}} ^ {\prime} + \widehat {\mathbf {C}} _ {:, \hat {i} _ {l + 1}} ^ {\prime} = \mathbf {0} \tag {46}
$$
Therefore, Lemma C.9 gives  $\overline{a}_{i_j,i_{l + 1}} = \overline{a}_{i_j,i_{l + 1}}$  for all  $j = 1,\dots,l$  and returning back to the equation for  $\mathbf{X}_{\cdot ,i_{l + 1}}$  we derive:
$$
\begin{array}{l} \mathbf {X} _ {:, i _ {l + 1}} = \sum_ {j = 1} ^ {l} \bar {a} _ {i _ {j}, i _ {l + 1}} \mathbf {C} _ {:, i _ {j}} + \mathbf {C} _ {:, i _ {l + 1}} = \sum_ {j = 1} ^ {l} \bar {\hat {a}} _ {i _ {j}, \hat {i} _ {l + 1}} \widehat {\mathbf {C}} _ {:, i _ {j}} + \widehat {\mathbf {C}} _ {:, \hat {i} _ {l + 1}} (47) \\ \stackrel {P (l)} {=} \sum_ {j = 1} ^ {l} \bar {a} _ {i _ {j}, \hat {i} _ {l + 1}} \mathbf {C} _ {:, i _ {j}} + \widehat {\mathbf {C}} _ {:, \hat {i} _ {l + 1}} (48) \\ \Rightarrow \widehat {\mathbf {C}} _ {:, \widehat {i} _ {l + 1}} = \mathbf {C} _ {:, \widehat {i} _ {l + 1}} (49) \\ \end{array}
$$
which completes the induction step. Notice that  $P(k)$  gives that for all  $k$ -pairs, the  $k$  root causes columns are in the same position. Given this fact we will now show that  $\widehat{\mathbf{A}} = \mathbf{A}$ . We will prove by induction that for  $l = k - 1, \dots, 1, 0$
$$
Q (l): \bar {a} _ {i j} = \bar {a} _ {i j} \text {f o r} 1 \leq i <   j <   d - l \tag {50}
$$
To prove  $Q(k - 1)$  we choose all the  $k$ -pairs  $\mathbf{C}$ ,  $\widehat{\mathbf{C}}$ , such that  $\mathbf{C}_{:,i_2}$  is in the last possible position,  $i_2 = d - k + 2$ . Then for  $i_1 \leq d - k$  the columns  $\mathbf{C}_{:,i_1}$ ,  $\widehat{\mathbf{C}}_{:,i_1}$  lie at the same position and are equal. Choosing  $i_1 = i$  and computing the equation for  $\mathbf{X}_{:,j}$  where  $i < j \leq d - k + 1$  gives:
$$
\mathbf {X} _ {:, j} = \bar {a} _ {i j} \mathbf {C} _ {:, i} = \bar {\hat {a}} _ {i j} \hat {\mathbf {C}} _ {:, i} = \bar {\hat {a}} _ {i j} \mathbf {C} _ {:, i} \tag {51}
$$
Therefore  $\overline{a}_{ij} = \overline{a}_{ij}$  for all  $1\leq i < j\leq d - (k - 1)$  and  $Q(k - 1)$  is satisfied. Next, assume that  $Q(k - l)$  is true. We want to show  $Q(k - l - 1)$ . Similarly to the base case we consider all  $k$ -pairs such that  $\mathbf{C}_{:,l + 2}$  lies in its last possible position  $i_{l + 2} = d - k + l + 2$ , and  $i_{l + 1}\leq d - k + l$ . Since the  $(l + 1)$ -th column is not in the last position, from the previous induction we have that:
$$
\left\{ \begin{array}{l} \widehat {\mathbf {C}} _ {:, i _ {1}} = \mathbf {C} _ {:, i _ {1}} \\ \widehat {\mathbf {C}} _ {:, i _ {2}} = \mathbf {C} _ {:, i _ {2}} \\ \vdots \\ \widehat {\mathbf {C}} _ {:, i _ {l + 1}} = \mathbf {C} _ {:, i _ {l + 1}} \end{array} \right. \tag {52}
$$
The equation for  $d - k + l + 1$  gives:
$$
\mathbf {X} _ {:, d - k + l + 1} = \sum_ {j = 1} ^ {l + 1} \bar {a} _ {i _ {j}, d - k + l + 1} \mathbf {C} _ {:, i _ {j}} \tag {53}
$$
$$
= \sum_ {j = 1} ^ {l + 1} \bar {a} _ {i _ {j}, d - k + l + 1} \widehat {\mathbf {C}} _ {:, i _ {j}} \tag {54}
$$
$$
= \sum_ {j = 1} ^ {l + 1} \hat {a} _ {i _ {j}, d - k + l + 1} \mathbf {C} _ {:, i _ {j}} \tag {55}
$$
$$
\xlongequal {\text {L e m m a C . 9}} \bar {\hat {a}} _ {i _ {j}, d - k + l + 1} = \bar {a} _ {i _ {j}, d - k + l + 1} \tag {56}
$$
By choosing all such possible  $k$ -pairs the indices  $i_j$  span all possibilities  $1 \leq i < d - k + l + 1$ . Combining this with  $Q(k - l)$  we get that  $Q(k - l - 1)$  is true and the desired result follows.