SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} q _ {(j _ {1}, j _ {0})} \left(\omega_ {N} ^ {i _ {1} \sqrt {N} + i _ {0}}\right) = \ell_ {j _ {0}} \left(\omega_ {N} ^ {i _ {1} \sqrt {N} + i _ {0}}\right) \cdot \tilde {r} _ {j _ {0}, j _ {1}} \left(\omega_ {N} ^ {\left(i _ {1} \sqrt {N} + i _ {0}\right) \sqrt {N}}\right) \\ = \ell_ {j _ {0}} \left(\omega_ {N} ^ {i _ {1} \sqrt {N} + i _ {0}}\right) \cdot \tilde {r} _ {j _ {0}, j _ {1}} \left(\omega_ {\sqrt {N}} ^ {i _ {0}}\right). \tag {45} \\ \end{array}
$$
By definition we have,
$$
\mathbf {M} ^ {\prime} \left[ \left(i _ {1}, i _ {0}\right), \left(j _ {1}, j _ {0}\right) \right] = \overline {{\mathbf {L}}} _ {i _ {1}, j _ {1}} \left[ i _ {0}, i _ {0} \right] \cdot \mathbf {R} _ {j _ {1}, j _ {1}} \left[ i _ {0}, j _ {0} \right].
$$
By (39) we have
$$
\mathbf {R} _ {j _ {1}, j _ {1}} \left[ i _ {0}, j _ {0} \right] = \tilde {r} _ {j _ {1}, j _ {0}} \left(\omega_ {\sqrt {N}} ^ {i _ {0}}\right)
$$
and by (41) and the fact that
$$
\overline {{\mathbf {L}}} = \mathbf {P L P},
$$
$$
\overline {{\mathbf {L}}} _ {i _ {1}, j _ {1}} \left[ i _ {0}, i _ {0} \right] = \ell_ {j _ {1}} \left(\omega_ {N} ^ {i _ {1} \sqrt {N} + i _ {0}}\right).
$$
$$
\mathbf {M} ^ {\prime} \left[ \left(i _ {1}, i _ {0}\right), \left(j _ {1}, j _ {0}\right) \right] = \ell_ {j _ {1}} \left(\omega_ {N} ^ {i _ {1} \sqrt {N} + i _ {0}}\right) \cdot \tilde {r} _ {j _ {1}, j _ {0}} \left(\omega_ {\sqrt {N}} ^ {i _ {0}}\right).
$$
$$
\mathbf {M} \cdot \mathbf {P} = \mathbf {M} ^ {\prime},
$$
$$
\begin{array}{l} \mathbf {M} \left[ \left(i _ {1}, i _ {0}\right), \left(j _ {1}, j _ {0}\right) \right] = \mathbf {M} ^ {\prime} \left[ \left(i _ {1}, i _ {0}\right), \left(j _ {0}, j _ {1}\right) \right] \\ = \ell_ {j _ {0}} \left(\omega_ {N} ^ {i _ {1} \sqrt {N} + i _ {0}}\right) \cdot r _ {j _ {0}, j _ {1}} \left(\omega_ {\sqrt {N}} ^ {i _ {0}}\right) = q _ {(j _ {1}, j _ {0})} \left(\omega_ {N} ^ {i _ {1} \sqrt {N} + i _ {0}}\right), \\ \end{array}
$$
where the last equality follows from (45).
Corollary 6. The DFT is  $\mathbf{M}$  as in Theorem 10 when all blocks of  $\widetilde{\mathbf{R}}$  and the top block of  $\widetilde{\mathbf{L}}$  are the identity matrix of size  $\sqrt{N} \times \sqrt{N}$  (the rest of  $\widetilde{\mathbf{L}}$  is all 0's).
Proof. Since only the diagonal of the top block of  $\widetilde{\mathbf{L}}$  will contain any non-zero values we only index the first  $0, \ldots, \sqrt{N} - 1$  rows. And since all blocks of  $\widetilde{\mathbf{R}}$  are the identity matrix we get
$$
\widetilde {\mathbf {L}} [ j _ {1}, j _ {1} ] = \tilde {\ell} _ {j _ {1}} [ j _ {1} ] = 1
$$
$$
\widetilde {\mathbf {R}} _ {j _ {1}} [ j _ {0}, j _ {0} ] = \tilde {r} _ {j _ {1}, j _ {0}} [ j _ {0} ] = 1.
$$
All other entries of  $\widetilde{\mathbf{L}}$  and  $\widetilde{\mathbf{R}}$  are 0. Thus, we have
$$
\ell_ {j _ {1}} (Z) = Z ^ {j _ {1}}
$$
$$
\tilde {r} _ {j _ {1}, j _ {0}} (Z) = Z ^ {j _ {0}}.
$$
As per Theorem 10,
$$
\begin{array}{l} q _ {(j _ {1}, j _ {0})} (Z) = \ell_ {j _ {0}} (Z) \cdot \tilde {r} _ {j _ {0}, j _ {1}} \left(Z ^ {\sqrt {N}}\right) \\ = Z ^ {j _ {0}} \cdot Z ^ {j _ {1} \sqrt {N}} = Z ^ {j _ {0} + j _ {1} \sqrt {N}} = Z ^ {j}. \\ \end{array}
$$
Then by Theorem 10 note
$$
q _ {\left(j _ {1}, j _ {0}\right)} \left(\omega_ {N} ^ {i}\right) = \omega_ {N} ^ {i j} = \mathbf {M} \left[ \left(i _ {1}, i _ {0}\right), \left(j _ {1}, j _ {0}\right) \right],
$$
which implies  $\mathbf{M}$  is  $\mathbf{F}_N$  as desired.
Algorithm 3 is pseudo code for the Monarch convolution algorithm. It maps an input space of  $\widetilde{\mathbf{L}}$ ,  $\widetilde{\mathbf{R}}$  matrices, a kernel vector  $\mathbf{k}$ , and input vector  $\mathbf{u}$  to a vector  $\mathbf{f}$ .
Algorithm 3 BLOCKMONARCHCONV( $\widetilde{\mathbf{L}}, \widetilde{\mathbf{R}}, \mathbf{k}, \mathbf{u}$ )
Input:  $\widetilde{\mathbf{L}},\widetilde{\mathbf{R}}\in \mathbb{R}^{N\times \sqrt{N}}$  , k, u  $\in \mathbb{R}^N$    
Output:  $\mathbf{f}\in \mathbb{R}^N$    
1: L, R  $\leftarrow$  BLOCKY MONARCH(L, R)   
2: M  $\leftarrow$  PLPRP  $>$  Get M' from BLOCKY MONARCH   
3:  $\mathbf{k}_f\gets \mathbf{M}\cdot \mathbf{k}$    
4:  $\mathbf{u}_f\gets \mathbf{M}\cdot \mathbf{u}$    
5: f  $\leftarrow$  M-1. (kf 0 u) Compute f   
6: return f
We next outline how to make the map in Algorithm 3 causal. Towards that end, we observe:
Lemma 6. For any fixed  $n \geq 1$ , let  $N = \left\lceil \sqrt{2n} \right\rceil^2$ . Define  $\widetilde{\mathbf{L}}, \widetilde{\mathbf{R}} \in \mathbb{R}^{N \times \sqrt{N}}$  such that for every  $0 \leq j_1 < \sqrt{N}$ ,
$$
\widetilde {\mathbf {L}} = \left[ \begin{array}{c} \widetilde {\mathbf {L}} _ {0} \\ \vdots \\ \widetilde {\mathbf {L}} _ {\sqrt {N} - 1} \end{array} \right] a n d \widetilde {\mathbf {R}} = \left[ \begin{array}{c} \widetilde {\mathbf {R}} _ {0} \\ \vdots \\ \widetilde {\mathbf {R}} _ {\sqrt {N} - 1} \end{array} \right].
$$
Define  $\widetilde{\mathbf{L}}', \widetilde{\mathbf{R}}' \in \mathbb{R}^{N \times \sqrt{N}}$  with corresponding coefficient blocks  $\left(\widetilde{\mathbf{L}}_k'\right)_{0 \leq k < \sqrt{N}}$ ,  $\left(\widetilde{\mathbf{R}}_k'\right)_{0 \leq k < \sqrt{N}}$  as
$$
\widetilde {\mathbf {L}} ^ {\prime} = \left[ \begin{array}{c} \widetilde {\mathbf {L}} _ {0} ^ {\prime} \\ \mathbf {0} _ {\sqrt {N} \times \sqrt {N}} \\ \vdots \\ \mathbf {0} _ {\sqrt {N} \times \sqrt {N}} \end{array} \right] a n d \widetilde {\mathbf {R}} ^ {\prime} = \left[ \begin{array}{c} \widetilde {\mathbf {R}} _ {0} ^ {\prime} \\ \widetilde {\mathbf {R}} _ {1} ^ {\prime} \\ \vdots \\ \widetilde {\mathbf {R}} _ {\sqrt {N} - 1} ^ {\prime} \end{array} \right]
$$
$$
\widetilde {\mathbf {L}} _ {0} ^ {\prime} \left[ \left(i _ {1}, i _ {0}\right), j _ {1} \right] = \left\{ \begin{array}{l} \widetilde {\mathbf {L}} _ {0} \left[ i _ {0}, j _ {1} \right] \text {i f} i _ {1} = 0 \text {a n d} i _ {0} \geq j _ {1} \\ 0 \text {o t h e r w i s e} \end{array} \right., \quad \widetilde {\mathbf {L}} _ {k} ^ {\prime} = \mathbf {0} _ {\sqrt {N} \times \sqrt {N}} \text {f o r} k = 1, \dots , \sqrt {N} - 1
$$
$$
\widetilde {\mathbf {R}} _ {j _ {1}} ^ {\prime} [ i _ {0}, j _ {0} ] = \left\{ \begin{array}{l l} 0 & \text {i f} i _ {0} <   j _ {0} \text {o r} \big ((i _ {0} \geq \left\lfloor \frac {\sqrt {N}}{2} \right\rfloor) \text {a n d} (j _ {0} <   \left\lfloor \frac {\sqrt {N}}{2} \right\rfloor)) \\ \widetilde {\mathbf {R}} _ {j _ {1}} [ i _ {0}, j _ {0} ] & \text {o t h e r w i s e} \end{array} \right. \quad \text {f o r} j _ {1} = 0, \ldots , \sqrt {N} - 1.
$$
Further, we require that  $\widetilde{\mathbf{R}}_{j_1}'[j_0,j_0]$ ,  $\widetilde{\mathbf{L}}_0'[j_1,j_1]$  are all non-zero entries for all  $0\leq j_0,j_1 < \sqrt{N}$ . Then for  $j_0\sqrt{N} +j_1 < \sqrt{N}\left\lfloor \frac{\sqrt{N}}{2}\right\rfloor$ , the basis polynomial  $q_{(j_1,j_0)}'(Z) = \ell_{j_1}'(Z)r_{j_1,j_0}'(Z^{\sqrt{N}})$  of  $\mathbf{M}' = \mathbf{PLPR}$  has minimum degree  $j_0\sqrt{N} +j_1$  and maximum degree  $\leq \frac{N}{2} -1$ , and for  $\sqrt{N}\left\lfloor \frac{\sqrt{N}}{2}\right\rfloor \leq j_0\sqrt{N} +j_1 < N$ , the basis polynomial  $q_{(j_1,j_0)}'(Z)$  has minimum degree  $j_0\sqrt{N} +j_1$  and maximum degree  $\leq N - 1$ .
Note that in  $\mathbf{M} = \mathbf{M}'\mathbf{P}$  the  $(j_1,j_0)$  basis polynomial  $q_{j_1,j_0}(Z) = q_{j_0,j_1}'(Z)$  has the required degree bound.
Proof. Let  $\mathbf{L}'$ ,  $\mathbf{R}' \gets \text{BLOCKY}$ $\text{MONARCH}(\widetilde{\mathbf{L}}', \widetilde{\mathbf{R}}')$ , and denote their corresponding polynomials as  $\ell_{j_1}'(Z), r_{j_1,j_0}'(Z^{\sqrt{N}})$ , respectively for every  $0 \leq j_1, j_0 < \sqrt{N}$ . By our definition, for all  $0 \leq j_0, j_1 < \sqrt{N}$ ,  $r_{j_1,j_0}'(Z^{\sqrt{N}})$  has minimum degree  $j_0\sqrt{N}$  and  $\ell_{j_1}'(Z)$  has minimum degree  $j_1$ . Then it follows that the basis polynomial  $q_{(j_1,j_0)}'(Z) = \ell_{j_1}'(Z)r_{j_1,j_0}'(Z^{\sqrt{N}})$  has minimum degree  $j_1 + j_0\sqrt{N}$ .
We now look at the degrees of each basis polynomial. From our definition of  $\mathbf{R}_{j_1}'$ , all entries  $\mathbf{R}_{j_1}'[i_0,j_0] = 0$  for  $j_0 < \left\lfloor \frac{\sqrt{N}}{2} \right\rfloor$  and  $i_0 \geq \left\lfloor \frac{\sqrt{N}}{2} \right\rfloor$ . This implies that for  $0 \leq j_0 < \left\lfloor \frac{\sqrt{N}}{2} \right\rfloor$ , and  $0 \leq j_1 < \sqrt{N}$ , we have  $\deg(r_{j_1,j_0}') < \left\lfloor \frac{\sqrt{N}}{2} \right\rfloor$ . Since we are only looking at  $\mathbf{L}_0'(Z)$ , note that degree
$\deg (\ell_{j_1}^{\prime}) \leq \sqrt{N} - 1$ . Then it follows that
$$
\begin{array}{l} \deg \left(q _ {(j _ {1}, j _ {0})} ^ {\prime}\right) \leq \sqrt {N} - 1 + \left(\left\lfloor \frac {\sqrt {N}}{2} \right\rfloor - 1\right) \sqrt {N} \\ = \sqrt {N} - 1 + \left(\sqrt {N} \left\lfloor \frac {\sqrt {N}}{2} \right\rfloor - \sqrt {N}\right) \\ \leq \sqrt {N} \frac {\sqrt {N}}{2} - 1 \\ = \frac {N}{2} - 1. \\ \end{array}
$$
(Note that in the above  $j_0\sqrt{N} + j_1 \leq \sqrt{N}\left\lfloor \frac{\sqrt{N}}{2}\right\rfloor - 1$  by same calculations above as needed.)
For  $\left\lfloor \frac{\sqrt{N}}{2} \right\rfloor \leq j_0 < \sqrt{N}$  and  $0 \leq j_1 < \sqrt{N}$ ,  $\deg(r_{j_1,j_0}') = \sqrt{N} - 1$ , and  $\ell_{j_1}'(Z)$  degree  $\sqrt{N} - 1$ . Then it follows that
$$
\begin{array}{l} \deg \left(q _ {(j _ {1}, j _ {0})} ^ {\prime}\right) \leq \sqrt {N} - 1 + \left(\sqrt {N} - 1\right) \sqrt {N} \\ = N - 1, \\ \end{array}
$$
as desired. (Note that  $j_0\sqrt{N} + j_1 \geq \left\lfloor \frac{\sqrt{N}}{2} \right\rfloor \sqrt{N}$  as needed.)
Finally, we use Lemma 6 and Theorem 8 to conclude the following:
Theorem 11. For any fixed  $n \geq 1$ , let  $N = \lceil \sqrt{2n} \rceil^2$ . Let  $\widetilde{\mathbf{L}}', \widetilde{\mathbf{R}}' \in \mathbb{R}^{N \times N}$  with corresponding coefficient blocks  $\left(\widetilde{\mathbf{L}}_k'\right)_{0 \leq k < \sqrt{N}}, \left(\widetilde{\mathbf{R}}_k'\right)_{0 \leq k < \sqrt{N}}$  be defined as in Lemma 6. Then for any  $\mathbf{k}, \mathbf{u} \in \mathbb{R}^n$ , BLOCKMONARCHCONV  $\left(\widetilde{\mathbf{L}}', \widetilde{\mathbf{R}}', \mathbf{k}', \mathbf{u}'\right) [0:n-1]$ , where  $\mathbf{k}' = (\mathbf{k}, \mathbf{0}_{N-n})$ ,  $\mathbf{u}' = (\mathbf{u}, \mathbf{0}_{N-n})$ , is causal in  $\mathbf{u}$ .
Proof. Note that this is the same setting as Theorem 8. If  $N = \left\lceil \sqrt{2n} \right\rceil^2$ , then  $\left\lfloor \frac{N}{2} \right\rfloor \geq n$ . Then by Lemma 6, the basis polynomials  $q_{j_1,j_0}(Z)$  of  $\mathbf{M} = \mathbf{PLPR}$  have  $\deg(q_{(j_1,j_0)}) < \frac{N}{2}$  for  $0 \leq j_1\sqrt{N} + j_0 < \sqrt{N} \left\lfloor \frac{\sqrt{N}}{2} \right\rfloor \leq \left\lfloor \frac{N}{2} \right\rfloor$ .
Then (34) computes BLOCKMONARCHCONV  $\left(\widetilde{\mathbf{L}}',\widetilde{\mathbf{R}}',\mathbf{k}',\mathbf{u}'\right)$ . Since Algorithm 3 performs the same operation as (34), the result follows from Lemma 6 and Theorem 8.
D.5 Bivariate Polynomials with Kronecker Substitution Over Reals
Our earlier results pertain to complex evaluation points. In this section we define causal convolution over real evaluation points. To do this, we redefine our basis polynomials in terms of the Chebyshev polynomials of the first kind.
In Appendix D.5.1 we recover Theorem 8 for univariate polynomials defined over real evaluation points. This identifies a class of matrices that we use to define to define a structured subclass in Appendix D.5.2. We show that these matrices can be used to perform (31). Finally, in Appendix D.5.3 we give the analog of Theorem 11 over real evaluation points. However, the resulting matrices are not Monarch matrices but they are close. In Appendix D.5.4 and Appendix D.5.5 we discuss how we can exploit this closeness, and compute (31) more efficiently.
<sup>6</sup>Recall that Lemma 6 is stated for basis polynomials  $q_{j_1,j_0}'(Z)$  for  $\mathbf{M}' = \mathbf{PLPR}$  where  $q_{j_1,j_0}(Z) = q_{j_0,j_1}'(Z)$ .
D.5.1 Univariate Evaluation over Real Numbers
For any integer  $a \geq 0$ , the Chebyshev polynomial of the first kind with degree  $a$  is denoted as  $T_{a}(Z)$  and is defined as
$$
T _ {a} (\cos \theta) \stackrel {\text {d e f}} {=} \cos (a \theta), \tag {46}
$$
and has the property
$$
T _ {a} (- \cos \theta) = (- 1) ^ {a} \cos (a \theta). \tag {47}
$$
To parallel Appendix D.3.2, we consider the class of basis polynomials with the form
$$
\bar {q} _ {j} ^ {N} (Z) = \sum_ {a = 0} ^ {N - j - 1} \bar {q} _ {j} [ a ] T _ {a} (Z), \tag {48}
$$
evaluated over  $(\omega_{N,i})_{0\leq i < N}$  where
$$
\omega_ {N, i} \stackrel {\text {d e f}} {=} \cos \left(\frac {\pi \left(i + \frac {1}{2}\right)}{N}\right). \tag {49}
$$
Let us consider the class of matrices defined over (48) and (49).
Lemma 7. For sequence of points  $A = \{\omega_{N,0},\dots ,\omega_{N,N - 1}\}$ , let  $\mathbf{M}$  be defined as
$$
\mathbf {M} [ i, j ] = \bar {q} _ {j} ^ {N} \left(\omega_ {N, i}\right) \tag {50}
$$
where  $\bar{q}_j^N (Z)$  and  $\omega_{N,i}$  are defined as in (48) and (49), respectively. Then for any vector  $\mathbf{u}$ ,  $\mathbf{M} \cdot \mathbf{u}$  is equivalent to evaluating the polynomial
$$
u (Z) = \sum_ {j = 0} ^ {N - 1} u _ {j} \cdot \bar {q} _ {j} ^ {N} (Z) \tag {51}
$$
at each point in  $A$
Proof. By our definition of  $\mathbf{M}$ , the column  $\mathbf{M}[:, j]$  is exactly the evaluation of the polynomial  $\bar{q}_j^N(Z)$  at each point in  $A$ . The claimed result comes from the definition of matrix vector multiplication and (51).
This leads to the following analog of Theorem 7.
Theorem 12. For matrices  $\mathbf{M}_0, \mathbf{M}_1, \mathbf{M}_2$ , each of form as in Lemma 7, the operation
$$
\mathbf {f} = \mathbf {M} _ {0} ^ {- 1} \cdot \left(\left(\mathbf {M} _ {1} \cdot \mathbf {k}\right) \odot \left(\mathbf {M} _ {2} \cdot \mathbf {u}\right)\right). \tag {52}
$$
$$
f (Z) = k (Z) \cdot u (Z) \mod T _ {N} (Z)
$$
$$
\hat {q} _ {j} ^ {N} (Z) f o r j = 0, \dots , N - 1
$$
where  $k(Z), u(Z)$  are defined in terms of the respective basis polynomials corresponding to  $\mathbf{M}_1$  and  $\mathbf{M}_2$  as in Lemma 7, and  $\left(\hat{q}_j^N (Z)\right)_{0\leq j < N}$  corresponds to  $\mathbf{M}_0$ .
Proof. For  $A = \{\omega_{N,i}\}_{0\leq i < N}$ , define
$$
q _ {A} (Z) = \prod_ {\alpha \in A} (Z - \alpha). \tag {53}
$$
Then (52) follows since for
$$
f (Z) = k (Z) \cdot u (Z) \mod q _ {A} (Z),
$$
we have the following for any  $\alpha \in A$ :
$$
f (\alpha) = k (\alpha) \cdot u (\alpha).
$$
The claim follows from Lemma 7, (52), the invertibility of  $\mathbf{M}_0$ , and the known fact that  $T_N(Z) = q_A(Z)$ .
We also utilize the following result:
Lemma 8. Let  $\bar{q}_j^{\left\lfloor \frac{N}{2}\right\rfloor}(Z)$  be defined as in (48). Then for any  $0 \leq j, j' < N$
$$
\bar {q} _ {j} ^ {\lfloor \frac {N}{2} \rfloor} (Z) \cdot \bar {q} _ {j ^ {\prime}} ^ {\lfloor \frac {N}{2} \rfloor} (Z) = \sum_ {i = j + j ^ {\prime}} ^ {N - 1} \alpha_ {j + j ^ {\prime}, i} \cdot \bar {q} _ {i} ^ {N} (Z). \tag {54}
$$
for some set of coefficients  $\alpha_{j + j',i}$ .
Proof. From (48), we have
$$
\bar {q} _ {j} ^ {\lfloor \frac {N}{2} \rfloor} (Z) \cdot \bar {q} _ {j ^ {\prime}} ^ {\lfloor \frac {N}{2} \rfloor} (Z) = \sum_ {a = 0} ^ {\lfloor \frac {N}{2} \rfloor - j - 1} q _ {j} [ a ] T _ {a} (Z) \cdot \sum_ {a ^ {\prime} = 0} ^ {\lfloor \frac {N}{2} \rfloor - j ^ {\prime} - 1} q _ {j ^ {\prime}} ^ {\prime} [ a ^ {\prime} ] T _ {a ^ {\prime}} (Z).
$$
Recall that within a  $m$  dimensional vector space of polynomials, we can define a basis by choosing any set of polynomials with degrees  $0,\ldots ,m - 1$ . Because  $\deg \left(\bar{q}_j^{\lfloor \frac{N}{2}\rfloor}\cdot \bar{q}_{j'}^{\lfloor \frac{N}{2}\rfloor}\right)\leq 2\lfloor \frac{N}{2}\rfloor -(j + j') - 2,$  it can be written as a linear combination of any set of  $2\lfloor \frac{N}{2}\rfloor -(j + j') - 1$  polynomials where for  $0\leq a\leq 2\lfloor \frac{N}{2}\rfloor -(j + j') - 2$ , the  $a$ -th polynomial has degree  $a$ . In other words we can choose the  $a$ -th polynomial as  $q_{N - a - 1}^{N}(Z)$ . Thus we have
$$
\bar {q} _ {j} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z) \cdot \bar {q} _ {j ^ {\prime}} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z) = \sum_ {a = 0} ^ {2 \left\lfloor \frac {N}{2} \right\rfloor - (j + j ^ {\prime}) - 2} \bar {\alpha} _ {j + j ^ {\prime}, a} \cdot \bar {q} _ {N - a - 1} ^ {N} (Z)
$$
for some set of coefficients  $\alpha_{j + j',a}$ . Then after reindexing  $i\gets N - a - 1$  we have
$$
\bar {q} _ {j} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z) \cdot \bar {q} _ {j ^ {\prime}} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z) = \sum_ {i = \left(N - 2 \left\lfloor \frac {N}{2} \right\rfloor\right) + j + j ^ {\prime} + 1} ^ {N - 1} \bar {\alpha} _ {j + j ^ {\prime}, N - i - 1} \cdot \bar {q} _ {i} ^ {N} (Z).
$$
The claim follows by setting  $\alpha_{j + j',j + j' + 1} = 0$ , and if  $N$  is odd,  $\alpha_{j + j' + 1,j + j' + 2} = 0$ , and  $\alpha_{j + j',i} = \bar{\alpha}_{j + j',N - i - 1}$  for other  $i$ .
This allows us to prove the following causality result for convolutions over real evaluation points.
Theorem 13. Fix a family of basis polynomials  $\bar{q}_0^N, \bar{q}_1^N, \dots, \bar{q}_{N-1}^N$  as defined in (48). Let  $N \geq 1$  be a perfect square,  $n \leq \left\lfloor \frac{N}{2} \right\rfloor$ ,  $\mathbf{k}, \mathbf{u} \in \mathbb{R}^n$  and  $\mathbf{M}_N$  defined by basis  $\left(\bar{q}_j^N(Z)\right)_{0 \leq j < N}$ . Let  $\mathbf{k}'' = \left(\mathbf{k}, \mathbf{0}_{\left\lfloor \frac{N}{2} \right\rfloor - n}\right)$  and  $\mathbf{u}'' = \left(\mathbf{u}, \mathbf{0}_{\left\lfloor \frac{N}{2} \right\rfloor - n}\right)$ . Then the operation
$$
\mathbf {u} \mapsto \left(\mathbf {M} _ {N} ^ {- 1} \left(\mathbf {M} _ {N} \cdot \left(\mathbf {0} _ {\lceil \frac {N}{2} \rceil}, \mathbf {k} ^ {\prime \prime}\right) \circ \mathbf {M} _ {N} \cdot \left(\mathbf {0} _ {\lceil \frac {N}{2} \rceil}, \mathbf {u} ^ {\prime \prime}\right)\right)\right) [ 0: n - 1 ] \tag {55}
$$
defines a causal map in  $\mathbf{u}$ .
$$
\mathbf {f} = \left(\mathbf {M} _ {N} ^ {- 1} \left(\mathbf {M} _ {N} \cdot \left(\mathbf {0} _ {\lceil \frac {N}{2} \rceil}, \mathbf {k} ^ {\prime \prime}\right) \circ \mathbf {M} _ {N} \cdot \left(\mathbf {0} _ {\lceil \frac {N}{2} \rceil}, \mathbf {u} ^ {\prime \prime}\right)\right)\right) [ 0: n - 1 ]. \tag {56}
$$
In order to prove that (55) is actually causal in the input  $\mathbf{u} \in \mathbb{R}^n$ , we must show that for all  $0 \leq i < N$ ,  $\mathbf{f}[i]$  is dependent only on  $\mathbf{u}[0], \mathbf{u}[1], \ldots, \mathbf{u}[i]$ . Let  $\mathbf{k}' = \left(\mathbf{0}_{\lceil \frac{N}{2} \rceil}, \mathbf{k}''\right)$  and  $\mathbf{u}' = \left(\mathbf{0}_{\lceil \frac{N}{2} \rceil}, \mathbf{u}''\right)$ . By Lemma 7,  $\mathbf{M}_N \cdot \mathbf{k}'$  and  $\mathbf{M}_N \cdot \mathbf{u}'$  correspond to the evaluations of the polynomials
$$
k ^ {\prime} (Z) = \sum_ {j = 0} ^ {N - 1} k _ {j} ^ {\prime} \cdot \bar {q} _ {j} ^ {N} (Z), \quad \text {a n d} \quad u ^ {\prime} (Z) = \sum_ {j ^ {\prime} = 0} ^ {N - 1} u _ {j ^ {\prime}} ^ {\prime} \cdot \bar {q} _ {j ^ {\prime}} ^ {N} (Z). \tag {57}
$$
$$
k (Z) = \sum_ {j = 0} ^ {n - 1} k _ {j} \cdot \bar {q} _ {j} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z), \quad \text {a n d} \quad u (Z) = \sum_ {j ^ {\prime} = 0} ^ {n - 1} u _ {j} \cdot \bar {q} _ {j ^ {\prime}} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z). \tag {58}
$$
Note that for  $0 \leq j < \left\lceil \frac{N}{2} \right\rceil$ , the coefficients  $k_{j}' = u_{j}' = 0$ . Then (57) becomes
$$
k ^ {\prime} (Z) = \sum_ {j = \left\lceil \frac {N}{2} \right\rceil} ^ {N - 1} k _ {j} ^ {\prime} \cdot \bar {q} _ {j} ^ {N} (Z), \quad \text {a n d} \quad u ^ {\prime} (Z) = \sum_ {j ^ {\prime} = \left\lceil \frac {N}{2} \right\rceil} ^ {N - 1} u _ {j} ^ {\prime} \cdot \bar {q} _ {j ^ {\prime}} ^ {N} (Z),
$$
$$
k ^ {\prime} (Z) = \sum_ {j = 0} ^ {\left\lfloor \frac {N}{2} \right\rfloor - 1} k _ {j + \left\lceil \frac {N}{2} \right\rceil} ^ {\prime} \cdot \bar {q} _ {j + \left\lceil \frac {N}{2} \right\rceil} ^ {N} (Z), \quad \text {a n d} \quad u ^ {\prime} (Z) = \sum_ {j ^ {\prime} = 0} ^ {\left\lfloor \frac {N}{2} \right\rfloor - 1} u _ {j ^ {\prime} + \left\lceil \frac {N}{2} \right\rceil} ^ {\prime} \cdot \bar {q} _ {j ^ {\prime} + \left\lceil \frac {N}{2} \right\rceil} ^ {N} (Z).
$$
For  $0 \leq j < \left\lfloor \frac{N}{2} \right\rfloor$ ,  $\deg \left(\bar{q}_j^{\left\lfloor \frac{N}{2} \right\rfloor}\right) = \left\lfloor \frac{N}{2} \right\rfloor - j - 1$ , and  $\deg \left(\bar{q}_{j + \left\lceil \frac{N}{2} \right\rceil}\right) = N - \left\lceil \frac{N}{2} \right\rceil - j - 1 = \left\lfloor \frac{N}{2} \right\rfloor - j - 1$ . This implies that  $\bar{q}_j^{\left\lfloor \frac{N}{2} \right\rfloor}(Z)$  and  $\bar{q}_{j + \left\lceil \frac{N}{2} \right\rceil}^{N}(Z)$  are both linear combinations of  $\left\lfloor \frac{N}{2} \right\rfloor - j - 1$  Chebychev polynomials. Then we can set  $\bar{q}_j^{\left\lfloor \frac{N}{2} \right\rfloor}(Z) = \bar{q}_{j + \left\lceil \frac{N}{2} \right\rceil}^{N}(Z)$ . Similarly, note that for  $0 \leq j < n$ ,  $k_{j + \left\lceil \frac{N}{2} \right\rceil}^{\prime} = k_{j}$ . Then it follows that  $k(Z) = k^{\prime}(Z)$ , and by a similar argument,  $u(Z) = u^{\prime}(Z)$ . Then by Theorem 12 we have
$$
\begin{array}{l} f (Z) = k (Z) \cdot u (Z) \mod T _ {N} (Z) \\ = k (Z) \cdot u (Z) \tag {59} \\ \end{array}
$$
where the last statement follows since  $\deg (k(Z)),\deg (u(Z))\leq n - 1 < \left\lfloor \frac{N}{2}\right\rfloor$  implying that their product has  $\deg (\bar{k} (Z)\cdot \bar{u} (Z)) <   \deg (T_N(Z)) = N$  . We want to write  $f(Z)$  in the form
$$
f (Z) = \sum_ {i = 0} ^ {N - 1} f _ {i} \cdot \bar {q} _ {i} ^ {N} (Z)
$$
for some set of coefficients set of coefficients  $f_{i}$ . From (58), (59) becomes
$$
f (Z) = \sum_ {j = 0} ^ {n - 1} \sum_ {j ^ {\prime} = 0} ^ {n - 1} k _ {j ^ {\prime}} \cdot u _ {j} \cdot \bar {q} _ {j ^ {\prime}} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z) \cdot \bar {q} _ {j} ^ {\left\lfloor \frac {N}{2} \right\rfloor} (Z).
$$
Then by Lemma 8 we have
$$
\sum_ {i = 0} ^ {N - 1} f _ {i} \cdot \bar {q} _ {i} ^ {N} (Z) = \sum_ {j, j ^ {\prime} = 0} ^ {n - 1} k _ {j ^ {\prime}} \cdot u _ {j} \cdot \sum_ {i = j + j ^ {\prime}} ^ {N - 1} \alpha_ {j + j ^ {\prime}, i} \bar {q} _ {i} ^ {N} (Z).
$$
We show that for all  $0 \leq i < N$ ,  $f_{i}$  is a function of  $(u_{i'})_{0 \leq i' \leq i}$ . Then note from the above that each  $k_{j}$  and  $u_{j'}$  appears in terms of  $\bar{q}_{i}^{N}$  where  $i \geq j + j'$ . Then we have
$$
f_{i} = \sum_{\substack{j,j = 0\\ j + j^{\prime}\leq i}}^{N - 1}\alpha_{j + j^{\prime},i}\cdot k_{j^{\prime}}\cdot u_{j},
$$
D.5.2 Structured Causal Matrices
In this section we narrow our scope from the general class of matrices defined in Appendix D.5.1 to a particular structured subclass. Now let us define the class of structured causal matrices over the real numbers.
Definition 6. Define
$$
\ell_ {j _ {1}} (Z) \stackrel {\text {d e f}} {=} \sum_ {a = 0} ^ {j _ {1}} \ell_ {j _ {1}} [ a ] T _ {a} (Z), \quad \tilde {r} _ {j _ {1}, j _ {0}} (Z) \stackrel {\text {d e f}} {=} \sum_ {a = 0} ^ {j _ {0}} \tilde {r} _ {j _ {1}, j _ {0}} [ a ] T _ {a} (Z) \tag {60}
$$
$$
\tilde {r} _ {j _ {1}, j _ {0}} [ a ] = 0 \text {i f} (j _ {0} - a) \text {i s o d d}, \tag {61}
$$
We define structured causal (SC) matrix polynomials as
$$
q _ {\left(j _ {1}, j _ {0}\right) \sqrt {N}} ^ {N} (Z) = \ell_ {\sqrt {N} - j _ {1} - 1} (Z) \cdot \tilde {r} _ {\sqrt {N} - j _ {1} - 1, \sqrt {N} - j _ {0} - 1} \left(T _ {\sqrt {N}} (Z)\right). \tag {62}
$$
A  $N\times N$  SC matrix is defined over the set of real evaluation points as
$$
\mathbf {M} ^ {\prime} [ i, \left(j _ {0}, j _ {1}\right) ] = q _ {\left(j _ {1}, j _ {0}\right) \sqrt {N}} ^ {N} \left(\omega_ {N, i}\right) = q _ {\left(j _ {0}, j _ {1}\right) \sqrt {N}} ^ {N} \left(\omega_ {N, i}\right). \tag {63}
$$
We note that in (63) we deviate from the usual ordering of indices  $(j_{1}, j_{0})$  to  $(j_{0}, j_{1})$ . We show that the SC matrix falls under the category of matrices defined by (50).