SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\ell_ {\mathbf {j}} ^ {(a)} \left(\omega_ {\bar {v} \sqrt {N}, i _ {0}}, \dots , \omega_ {\bar {v} \sqrt {N}, i _ {a}}\right) = \ell_ {\mathbf {j}, \mathbf {i}} ^ {(a)} \left(\omega_ {\bar {v} \sqrt {N}, i _ {a}}\right). \tag {76}
$$
${}^{8}b = 2$  gives the well known bit reversal permutation.
We are now ready to define our basis  $p$ -variate polynomials. For any index  $\mathbf{j} \in [0, \sqrt[p]{N})^p$ , define
$$
q _ {\mathbf {j}} ^ {N} \left(X _ {0}, \dots , X _ {p - 1}\right) = \prod_ {a = 0} ^ {p - 1} \ell_ {\mathbf {j}} ^ {(a)} \left(X _ {0}, \dots , X _ {a}\right). \tag {77}
$$
Then (75) can be re-written as
$$
\mathbf {M} [ \mathbf {i}, \mathbf {j} ] = q _ {\mathbf {j}} ^ {N} \left(\omega_ {\bar {\nu} \sqrt {N}, i _ {0}}, \dots , \omega_ {\bar {\nu} \sqrt {N}, i _ {p - 1}}\right). \tag {78}
$$
The above leads to the following result, which generalizes Theorem 1 to general  $p$ :
Theorem 15. Let  $\mathbf{M}'$ ,  $A$  and  $q_{\mathbf{j}}^{N}(X_{0},\ldots ,X_{p - 1})$  be as defined in (75), (74) and (77). Then for any vector  $\mathbf{u}$ ,  $\mathbf{M} \cdot \mathbf{u}$  is equivalent to evaluating the polynomial
$$
u \left(X _ {0}, \dots , X _ {p - 1}\right) = \sum_ {\mathbf {j}} u _ {\mathbf {j}} \cdot q _ {\mathbf {j}} ^ {N} \left(X _ {0}, \dots , X _ {p - 1}\right) \tag {79}
$$
at each point in  $A^p$
Proof. By (78), the column  $\mathbf{M}'[:,j]$  is exactly the evaluation of the polynomial (77) at each point in  $A^p$ . Then the claim follows from the definition of matrix vector multiplication and (79).
This then leads to the following result, (which generalizes Theorem 2):
Theorem 16. For matrices  $\mathbf{M}_0, \mathbf{M}_1, \mathbf{M}_2$ , each of form as in Theorem 15, the operation
$$
\mathbf {f} = \mathbf {M} _ {0} ^ {- 1} \cdot \left(\left(\mathbf {M} _ {1} \cdot \mathbf {k}\right) \odot \left(\mathbf {M} _ {2} \cdot \mathbf {u}\right)\right). \tag {80}
$$
$$
f (\mathbf {X}) = k (\mathbf {X}) \cdot u (\mathbf {X}) \mod \left(T _ {\mathbb {V} / N} \left(X _ {0}\right), \dots , T _ {\mathbb {V} / N} \left(X _ {p - 1}\right)\right)
$$
$$
\hat {q} _ {\mathbf {j}} ^ {N} (\mathbf {X})
$$
where  $k(\mathbf{X}), u(\mathbf{X})$  are defined in terms of the respective basis polynomials corresponding to  $\mathbf{M}_1$  and  $\mathbf{M}_2$  as in Theorem 15, and  $\hat{q}_{\mathbf{j}}^N(\mathbf{X})s$  corresponds to  $\mathbf{M}_0$ .
$$
q _ {A} (Z) = \prod_ {\alpha \in A} (Z - \alpha). \tag {81}
$$
Then (80) follows since for
$$
f (\mathbf {X}) = k (\mathbf {X}) \cdot u (\mathbf {X}) \mod (q _ {A} (X _ {0}), \dots q _ {A} (X _ {p - 1}),
$$
we have the following for any  $\mathbf{a} \in A^p$  we have:
$$
f (\mathbf {a}) = k (\mathbf {a}) \cdot u (\mathbf {a}).
$$
Using the known fact that  $T_{\overline{\mathcal{V}}\overline{N}}(Z) = \prod_{c=0}^{\overline{\mathcal{V}}\overline{N} - 1}\left(Z - \omega_{\overline{\mathcal{V}}\overline{N},c}\right) = q_A(Z)$ , the claim follows from Theorem 15, (80), and the invertibility of  $\mathbf{M}_0$ .
D.6.3 Generalizing Theorem 3 for  $p \geq 2$
To convert Theorem 16 into a causal map we basically have to blow up  $n \to 2^p \cdot n$ . Further, paralleling (48) we need to change the definition in (73) to have degree  $\sqrt[p]{N} - j_a - 1$  instead of the earlier  $\sqrt[p]{N} - 1$ :
$$
\tilde {\ell} _ {\mathbf {j}, \mathbf {i}} ^ {(a)} (X _ {a}) = \sum_ {m = 0} ^ {\sqrt [ p ]{N} - j _ {a} - 1} \tilde {\ell} _ {(j _ {a})} ^ {(a)} [ m ] \cdot T _ {m} (X _ {a}). \tag {82}
$$
Note that now the RHS only depends on  $a$  and  $j_{a}$  (let us call the RHS  $\widetilde{\ell}_{j_a}^{(a,\sqrt[p]{N})}(X_a)$ ), so the next definition becomes easier:
$$
\widetilde {\ell} _ {\mathbf {j}} ^ {(a)} \left(X _ {0}, \dots , X _ {a}\right) = \widetilde {\ell} _ {j _ {a}} ^ {\left(a, \sqrt [ 4 ]{N}\right)} \left(X _ {a}\right).
$$
We are now ready to define our causal basis polynomials. For any index  $\mathbf{j}$ , define
$$
\widetilde {q} _ {\mathbf {j}} ^ {N} \left(X _ {0}, \dots , X _ {p - 1}\right) = \prod_ {a = 0} ^ {p - 1} \widetilde {\ell} _ {\mathbf {j}} ^ {(a)} \left(X _ {0}, \dots , X _ {a}\right). \tag {83}
$$
These polynomials form a structured subclass of the polynomials defined in (77).
Lemma 17. The class of polynomials  $\widetilde{q}_{\mathbf{j}}^{N}\left(X_{0},\ldots ,X_{p - 1}\right)$  defined in (83) are a special case of (77).
Proof. This follows from the fact that (82) is a special case of (73) for every  $\mathbf{i},\mathbf{j},0\leq a < p$
We show that the product of two  $\widetilde{q}_{\mathbf{j}}^{\left\lfloor \frac{pN}{2}\right\rfloor^{p}}(X_{0},\ldots ,X_{p - 1})$  type polynomials can be written has a linear combination of  $\widetilde{q}_{\mathbf{m}}^{N}(X_{0},\ldots ,X_{p - 1})$  with the indices of  $\mathbf{m}$  being lexicographically larger than the original indices.
Lemma 18. Let  $\widetilde{q}_{\mathbf{j}}^{\left\lfloor \frac{p\sqrt{N}}{2}\right\rfloor^{p}}(X_{0},\ldots ,X_{p - 1})$  be defined as in (83). Then for any  $\mathbf{j},\mathbf{j}^{\prime}\in \left[0,\frac{p\sqrt{N}}{2}\right)^{p}$
$$
\tilde {q} _ {\mathbf {j}} ^ {\left\lfloor \frac {\sqrt {N}}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right) \cdot \tilde {q} _ {\mathbf {j} ^ {\prime}} ^ {\left\lfloor \frac {\sqrt {N}}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right) = \sum_ {\mathbf {j} + \mathbf {j} ^ {\prime} \preceq \mathbf {m} \in [ 0, \sqrt [ p ]{N}) ^ {p}} \alpha_ {\mathbf {j} + \mathbf {j} ^ {\prime}, \mathbf {m}} \tilde {q} _ {\mathbf {m}} ^ {N} \left(X _ {0}, \dots , X _ {p - 1}\right) \tag {84}
$$
for some set of coefficients  $\alpha_{\mathbf{j} + \mathbf{j}^{\prime},\mathbf{m}}$
Proof. From (83) we have,
$$
\tilde {q} _ {\mathbf {j}} ^ {\left\lfloor \frac {\nu_ {N}}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right) \cdot \tilde {q} _ {\mathbf {j} ^ {\prime}} ^ {\left\lfloor \frac {\nu_ {N}}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right) = \prod_ {a = 0} ^ {p - 1} \tilde {\ell} _ {j _ {a}} ^ {\left(a, \left\lfloor \frac {\nu_ {N}}{2} \right\rfloor\right)} \left(X _ {a}\right) \cdot \tilde {\ell} _ {j _ {a} ^ {\prime}} ^ {\left(a, \left\lfloor \frac {\nu_ {N}}{2} \right\rfloor\right)} \left(X _ {a}\right) \tag {85}
$$
Let us fix  $0 \leq a < p$ .
Because (82) is of the same form as in (48), we can apply Lemma 8 to each product  $\widetilde{\ell}_{j_a}^{(a,\left\lfloor \frac{p_N}{2}\right\rfloor)}(X_a)$ .  $\widetilde{\ell}_{j_a'}^{(a,\left\lfloor \frac{p_N}{2}\right\rfloor)}(X_a)$ , which gives us
$$
\widetilde {\ell} _ {j _ {a}} ^ {\left(a, \left\lfloor \frac {\mathcal {P} \sqrt {N}}{2} \right\rfloor\right)} \left(X _ {a}\right) \cdot \widetilde {\ell} _ {j _ {a} ^ {\prime}} ^ {\left(a, \left\lfloor \frac {\mathcal {P} \sqrt {N}}{2} \right\rfloor\right)} \left(X _ {a}\right) = \sum_ {m _ {a} = j _ {a} + j _ {a} ^ {\prime}} ^ {\mathcal {P} \sqrt {N} - 1} \alpha_ {j _ {a} + j _ {a} ^ {\prime}, m _ {a}} ^ {(a)} \cdot \widetilde {\ell} _ {m _ {a}} ^ {\left(a, \mathcal {P} \sqrt {N}\right)} \left(X _ {a}\right).
$$
Going back to (85), we get
$$
\widetilde {q} _ {\mathbf {j}} ^ {\left\lfloor \frac {\nu \sqrt {N}}{2} \right\rfloor^ {p}} (X _ {0}, \ldots , X _ {p - 1}) \cdot \widetilde {q} _ {\mathbf {j} ^ {\prime}} ^ {\left\lfloor \frac {\nu \sqrt {N}}{2} \right\rfloor^ {p}} (X _ {0}, \ldots , X _ {p - 1}) = \prod_ {a = 0} ^ {p - 1} \sum_ {m _ {a} = j _ {a} + j _ {a} ^ {\prime}} ^ {N - 1} \alpha_ {j _ {a} + j _ {a} ^ {\prime}, m _ {a}} ^ {(a)} \cdot \widetilde {\ell} _ {m _ {a}} ^ {\left(a, \frac {\nu \sqrt {N}}{2}\right)} (X _ {a}).
$$
Let  $\alpha_{\mathbf{j} + \mathbf{j}',\mathbf{m}} = \prod_{a = 0}^{p - 1}\alpha_{j_a + j'_a,m_a}^{(a)}$ . Then we get
$$
\widetilde {q} _ {\mathbf {j}} ^ {\left\lfloor \frac {p \sqrt {N}}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right) \cdot \widetilde {q} _ {\mathbf {j} ^ {\prime}} ^ {\left\lfloor \frac {p \sqrt {N}}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right) = \sum_ {\mathbf {j} + \mathbf {j} ^ {\prime} \preceq \mathbf {m} \in [ 0, \sqrt [ p ]{N}) ^ {p}} \alpha_ {\mathbf {j} + \mathbf {j} ^ {\prime}, \mathbf {m}} \widetilde {q} _ {\mathbf {m}} ^ {N} \left(X _ {0}, \dots , X _ {p - 1}\right),
$$
Algorithm 5  $\mathrm{PAD}(\mathbf{k})$
Input:  $\mathbf{k} \in \left(\mathbb{R}^{\left\lfloor \frac{p\sqrt{N}}{2}\right\rfloor}\right)^{p}$ , indexed as  $\mathbf{k_j}$  for  $\mathbf{j} \in \left[0, \frac{p\sqrt{N}}{2}\right)^p$
Output:  $\mathbf{k}' \in \left(\mathbb{R}^{\sqrt[p]{N}}\right)^p$
1: for  $\mathbf{j} \in \left[0, \frac{pN}{2}\right)^p$  do
2: if  $j_{a} \geq \left\lfloor \frac{\sqrt[2]{N}}{2} \right\rfloor$  for  $0 \leq a < p$  then
3:  $\mathbf{k}_{(j_0,\dots ,j_{p - 1})}'\gets \mathbf{k}_{\left(j_0 + \left\lceil \frac{\mathcal{V} / N}{2}\right\rceil ,\dots ,j_{p - 1} + \left\lceil \frac{\mathcal{V} / N}{2}\right\rceil}\right)$
4: else
5:  $\mathbf{k}_{\mathbf{j}}^{\prime} = 0$
6: return  $\mathbf{k}'$
The above basis and padding scheme allows us to extend Theorem 3 to general  $p$ .
Theorem 17. Fix a family of basis polynomials  $\widetilde{q}_{\mathbf{j}}^{N}(\mathbf{X})$  as defined in (83). Let  $N \geq 1$  be a perfect power of  $p$ ,  $n \leq \left\lfloor \frac{\sqrt[p]{N}}{2} \right\rfloor^p$ ,  $\mathbf{k}, \mathbf{u} \in \mathbb{R}^n$  and  $\mathbf{M}_N'$  defined by basis  $\widetilde{q}_{\mathbf{j}}^{N}(\mathbf{X})$ . Then the operation
$$
\mathbf {u} \mapsto \left(\mathbf {M} _ {N} ^ {\prime - 1} \left(\mathbf {M} _ {N} ^ {\prime} \cdot \operatorname {P A D} (\mathbf {k}) \circ \mathbf {M} _ {N} ^ {\prime} \cdot \operatorname {P A D} (\mathbf {u})\right)\right) [ 0: n - 1 ] \tag {86}
$$
Proof. Let  $\mathbf{k}' = \mathrm{PAD}(\mathbf{k})$ ,  $\mathbf{u}' = \mathrm{PAD}(\mathbf{u})$ , and
$$
\mathbf {f} = \left(\mathbf {M} _ {N} ^ {\prime - 1} \left(\mathbf {M} _ {N} ^ {\prime} \cdot \mathbf {k} ^ {\prime} \circ \mathbf {M} _ {N} ^ {\prime} \cdot \mathbf {u} ^ {\prime}\right)\right) [ 0: n - 1 ]. \tag {87}
$$
In order to prove that (86) is causal in the input  $\mathbf{u} \in \mathbb{R}^n$ , we must show that for all  $\mathbf{i} \in \left[0, \sqrt[p]{N}\right)^p$ ,  $\mathbf{f}_{\mathbf{i}}$  is dependent only on  $\mathbf{u}_{\mathbf{i}'}$  for  $\mathbf{i}' \preceq \mathbf{i}$ .
By Theorem 15,  $\mathbf{M}_N^{\prime} \cdot \mathbf{k}^{\prime}$  and  $\mathbf{M}_N^{\prime} \cdot \mathbf{u}^{\prime}$  correspond to the evaluations of the polynomials
$$
k ^ {\prime} \left(X _ {0}, \dots , X _ {p - 1}\right) = \sum_ {\mathbf {j} ^ {\prime} \in \left[ 0, \sqrt [ p ]{N}\right) ^ {p}} k _ {\mathbf {j} ^ {\prime}} ^ {\prime} \cdot \widetilde {q} _ {\mathbf {j} ^ {\prime}} ^ {N} \left(X _ {0}, \dots , X _ {p - 1}\right), \quad \text {a n d}
$$
$$
u ^ {\prime} \left(X _ {0}, \dots , X _ {p - 1}\right) = \sum_ {\mathbf {j} ^ {\prime} \in \left[ 0, \sqrt [ n ]{N}\right) ^ {p}} u _ {\mathbf {j} ^ {\prime}} ^ {\prime} \cdot \widetilde {q} _ {\mathbf {j} ^ {\prime}} ^ {N} \left(X _ {0}, \dots , X _ {p - 1}\right), \tag {88}
$$
respectively. Let us define
$$
k \left(X _ {0}, \dots , X _ {p - 1}\right) = \sum_ {\mathbf {j} \in \left[ 0, \left\lfloor \frac {R / N}{2} \right\rfloor\right) ^ {p}} k _ {\mathbf {j}} \cdot \widetilde {q} _ {\mathbf {j}} ^ {\left\lfloor \frac {R / N}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right), \quad \text {a n d}
$$
$$
u \left(X _ {0}, \dots , X _ {p - 1}\right) = \sum_ {\mathbf {j} \in \left[ 0, \left\lfloor \frac {\mathcal {R} / N}{2} \right\rfloor\right) ^ {p}} u _ {\mathbf {j}} \cdot \widetilde {q} _ {\mathbf {j}} ^ {\left\lfloor \frac {\mathcal {R} / N}{2} \right\rfloor^ {p}} \left(X _ {0}, \dots , X _ {p - 1}\right). \tag {89}
$$
Let us define  $\deg \left(\widetilde{q}_{\mathbf{j}}^{N}\right) = \left(\sqrt[p]{N} - j_{a} - 1\right)_{a=0}^{p-1}$  as the vector of length  $p$  consisting of the degrees of the component univariate polynomials  $\widetilde{\ell}_{\mathbf{j},\mathbf{i}}^{(a)}(X_{a})$  defined as in (82). Then we have  $\deg \left(\widetilde{q}_{\mathbf{j}}^{\left\lfloor \frac{p\sqrt{N}}{2}\right\rfloor^{p}}\right) =$
We now define the following padding scheme,  $\mathrm{PAD}(\mathbf{k})$ .
$\left(\left\lfloor \frac{\sqrt[p]{N}}{2}\right\rfloor -j_0 - 1,\dots ,\left\lfloor \frac{\sqrt[p]{N}}{2}\right\rfloor -j_{p - 1} - 1\right)$  for  $\mathbf{j} = (j_0,\dots ,j_{p - 1})$  such that  $0\leq j_{a} <   \left\lceil \frac{\sqrt[p]{N}}{2}\right\rceil$  for  $0\le a <   p$  . Further, for  $\mathbf{j}' = \left(j_0 + \left\lceil \frac{\sqrt[p]{N}}{2}\right\rceil ,\dots ,j_{p - 1} + \left\lceil \frac{\sqrt[p]{N}}{2}\right\rceil\right)$  we have
$$
\begin{array}{l} \deg \left(\widetilde {q} _ {\mathbf {j} ^ {\prime}} ^ {N}\right) = \left(\sqrt [ p ]{N} - j _ {0} ^ {\prime} - 1, \dots , \sqrt [ p ]{N} - j _ {p - 1} ^ {\prime} - 1\right) \\ = \left(\sqrt [ p ]{N} - \left\lceil \frac {\sqrt [ p ]{N}}{2} \right\rceil - j _ {0} - 1, \dots , \sqrt [ p ]{N} - \left\lceil \frac {\sqrt [ p ]{N}}{2} \right\rceil - j _ {p - 1} - 1\right) \\ = \left(\left\lfloor \frac {\sqrt [ p ]{N}}{2} \right\rfloor^ {p} - j _ {0} - 1, \ldots , \left\lfloor \frac {\sqrt [ p ]{N}}{2} \right\rfloor^ {p} - j _ {p - 1} - 1\right). \\ \end{array}
$$
Since  $\deg \left(\widetilde{q}_{\mathbf{j}}^{\left\lfloor \frac{R / N}{2}\right\rfloor^{p}}\right) = \deg \left(\widetilde{q}_{\mathbf{j}^{\prime}}^{N}\right)$ , we can set  $\widetilde{\ell}_{\mathbf{j},\mathbf{i}}^{(a)}(X_a) = \widetilde{\ell}_{\mathbf{j}^{\prime},\mathbf{i}}^{(a)}(X_a)$ . Similarly, note that for  $\mathbf{j},\mathbf{j}^{\prime}$  as above,  $k_{\mathbf{j}^{\prime}}^{\prime} = k_{\mathbf{j}}$  and  $u_{\mathbf{j}^{\prime}}^{\prime} = u_{\mathbf{j}}$ . Then it follows that  $k(X_0,\ldots ,X_{p - 1}) = k'(X_0,\ldots ,X_{p - 1})$ , and by a similar argument,  $u(X_0,\ldots ,X_{p - 1}) = u'(X_0,\ldots ,X_{p - 1})$ . Then by Theorem 16 we have
$$
\begin{array}{l} f \left(X _ {0}, \ldots , X _ {p - 1}\right) = k \left(X _ {0}, \ldots , X _ {p - 1}\right) \cdot u \left(X _ {0}, \ldots , X _ {p - 1}\right) \mod \left(T _ {\mathbb {P} \sqrt {N}} \left(X _ {0}\right), \ldots , T _ {\mathbb {P} \sqrt {N}} \left(X _ {p - 1}\right)\right) \\ = k \left(X _ {0}, \dots , X _ {p - 1}\right) \cdot u \left(X _ {0}, \dots , X _ {p - 1}\right) \tag {90} \\ \end{array}
$$
where the second line follows by observing that each  $0 \leq a < p$ , we have  $\deg_{X_a}(k(\mathbf{X}) \cdot u(\mathbf{X})) < 2\left\lfloor \frac{\sqrt[p]{N}}{2} \right\rfloor$  and observing that  $2\left\lfloor \frac{\sqrt[p]{N}}{2} \right\rfloor \leq \sqrt[p]{N}$ . We want to write  $f(X_0, \ldots, X_{p-1})$  in the form
$$
f \left(X _ {0}, \ldots , X _ {p - 1}\right) = \sum_ {\mathbf {m} \in \left[ 0, \sqrt [ p ]{N}\right) ^ {p}} f _ {\mathbf {m}} \cdot \widetilde {q} _ {\mathbf {m}} ^ {N} \left(X _ {0}, \ldots , X _ {p - 1}\right),
$$
for a set of coefficients  $f_{\mathbf{m}}$ . From (89) and (90) we get
$$
f\left(X_{0},\ldots ,X_{p - 1}\right) = \sum_{\substack{\mathbf{j},\mathbf{j}^{\prime}\in \left[0,\left\lfloor \frac{V / N}{2}\right\rfloor\right)^{p}}}k_{\mathbf{j}^{\prime}}u_{\mathbf{j}}\cdot \widetilde{q}_{\mathbf{j}^{\prime}}^{\left\lfloor \frac{V / N}{2}\right\rfloor^{p}}\left(X_{0},\ldots ,X_{p - 1}\right)\cdot \widetilde{q}_{\mathbf{j}}^{\left\lfloor \frac{V / N}{2}\right\rfloor^{p}}\left(X_{0},\ldots ,X_{p - 1}\right).
$$
Then by Lemma 18 we have
$$
f \left(X _ {0}, \ldots , X _ {p - 1}\right) = \sum_ {\mathbf {j}, \mathbf {j} ^ {\prime} \in \left[ 0, \left\lfloor \frac {\mathcal {P} N}{2} \right\rfloor\right) ^ {p}} k _ {\mathbf {j} ^ {\prime}} u _ {\mathbf {j}} \cdot \sum_ {\mathbf {j} + \mathbf {j} ^ {\prime} \preceq \mathbf {m} \in [ 0, \mathcal {P} N) ^ {p}} \alpha_ {\mathbf {j} + \mathbf {j} ^ {\prime}, \mathbf {m}} \widetilde {q} _ {\mathbf {m}} ^ {N} \left(X _ {0}, \ldots , X _ {p - 1}\right).
$$
Thus, for any  $\mathbf{m} \in \left[0, \sqrt[p]{N}\right)^p$ , we have
$$
f _ {\mathbf {m}} = \sum_ {\mathbf {j} + \mathbf {j} ^ {\prime} \preceq \mathbf {m}} \alpha_ {\mathbf {j} + \mathbf {j} ^ {\prime}, \mathbf {m}} \cdot k _ {\mathbf {j} ^ {\prime}} u _ {\mathbf {j}}
$$
implying that  $f_{\mathbf{m}}$  depends only on  $\mathbf{u_j}$  for  $\mathbf{j}\preceq \mathbf{m}$ , as desired.
D.6.4  $p$ -variate Monarch
Recall that we have fixed  $b = \sqrt[p]{N}$  (and hence  $N = b^p$ ).
Define the  $p$ -variate Monarch matrix as follows:
$$
\mathbf {M} ^ {\prime} = \mathbf {P} _ {b, N} ^ {R} \left(\prod_ {a = 0} ^ {p - 2} \mathbf {B} _ {p - 1 - a} \cdot \left(\mathbf {P} _ {b ^ {a + 1}, b ^ {a + 2}, N}\right) ^ {\top}\right) \mathbf {B} _ {0}, \tag {91}
$$
Where each  $\mathbf{B}_a$  is block diagonal with  $b \times b$  blocks for every  $0 \leq a < p$ . Recall that Equation (1) has a permutation  $\mathbf{P}_0$  at the end while the above definition does not have any permutation at the end. One trivial way to show the equivalence of above to Equation (1) is to define  $\mathbf{P}_0 = \mathbf{I}$ . In Appendix D.6.5,
we show that exists other non-trivial choices for  $\mathbf{P}_0$ . Further for  $1 \leq i \leq p$ , the  $\mathbf{P}_i$  in Equation (1) connect to the above definition as follows:
$$
\mathbf {P} _ {i} = \left\{ \begin{array}{l l} \left(\mathbf {P} _ {b ^ {p - i}, b ^ {p - i + 1}, N}\right) ^ {\top} & \text {f o r} 1 \leq i \leq p - 1 \\ \mathbf {P} _ {b, N} ^ {R} & \text {f o r} i = p \end{array} \right..
$$
Finally we connect the above definition of  $p$ -variate Monarch to the earlier definition based on polynomial evaluation:
Lemma 19. Equation (75) can be written as Equation (91).
Proof. In (91),  $\mathbf{B}_a$  would correspond to the evaluations  $\ell_{\mathbf{j},\mathbf{i}}^{(a)}\left(\omega_{\sqrt[N],i_a}\right)$  from earlier. Specifically the following holds for any  $0 \leq a < p$ . We index the  $q^{p-1}$  blocks of  $\mathbf{B}_a$  by  $(i_0, \ldots, i_{a-1})$ ,  $(j_{p-1}, \ldots, j_{a+1})$  and the row and column 'offsets' within each such block are indexed by  $i_a$  and  $j_a$  respectively. Connecting back to  $\ell_{\mathbf{j},\mathbf{i}}^{(a)}(\cdot)$  we set
$$
\mathbf {B} _ {a} ^ {\left(\left(i _ {0}, \dots , i _ {a - 1}\right), \left(j _ {p - 1}, \dots , j _ {a + 1}\right)\right)} \left[ i _ {a}, j _ {a} \right] = \ell_ {\mathbf {j}, \mathbf {i}} ^ {(a)} \left(\omega_ {\sqrt [ N ]{N}, i _ {a}}\right). \tag {92}
$$
We will prove the claim as follows. Let  $\mathbf{e_j} \in \mathbb{R}^N$  have a 1 in the  $\mathbf{j}^{\mathrm{th}}$  location and 0's elsewhere. Our goal is to multiply this vector on the right of Equation (91) and show that we get the  $\mathbf{j}^{\mathrm{th}}$  column of  $\mathbf{M}'$  as defined in Equation (75). We will do so by induction.
Define  $\mathbf{y}_0 = \mathbf{e_j}$  and  $\mathbf{y}_1 = \mathbf{B}_0\cdot \mathbf{y}_0$  . Then for every  $1\leq a <   p$  , define
$$
\mathbf {y} _ {a + 1} = \mathbf {B} _ {a} \left(\mathbf {P} _ {b ^ {p - a}, b ^ {p - a + 1}, N}\right) ^ {\top} \mathbf {y} _ {a}.
$$
$$
\mathbf {M} ^ {\prime} \cdot \mathbf {e} _ {\mathbf {j}} = \mathbf {P} _ {b, N} ^ {R} \cdot \mathbf {y} _ {p}. \tag {93}
$$
Next, we claim that for every  $1 \leq a \leq p$ , we have for every  $(i_0, \ldots, i_{a-1}) \in [0, \sqrt[p]{N})^{a-1}$
$$
\mathbf {y} _ {a} \left[ \left(\left(i _ {0}, \dots , i _ {a - 2}\right), \left(j _ {p - 1}, \dots , j _ {a}\right), i _ {a - 1}\right) \right] = \prod_ {b = 0} ^ {a - 1} \ell_ {\mathbf {j}, \mathbf {i}} ^ {(b)} \left(\omega_ {\sqrt [ n ]{N}, b}\right), \tag {94}
$$
where for  $a = p$ , we think of  $((i_0, \ldots, i_{a-2}), (j_{p-1}, \ldots, j_a), i_{a-1}) = (i_0, \ldots, i_{p-1})$ .
We first note that Equation (94) is enough to prove the claim. Indeed we have that
$$
\mathbf {y} _ {p} \left[ \left(\left(i _ {0}, \dots , i _ {p - 1}\right)\right) \right] = \prod_ {b = 0} ^ {p - 1} \ell_ {\mathbf {j}, \mathbf {i}} ^ {(b)} \left(\omega_ {\sqrt [ n ]{N}, b}\right),
$$
where the RHS in the above is the same as RHS in Equation (75). To see the claim note that by definition of  $\mathbf{P}_{b,N}^{R}$ , we have  $\left(\mathbf{P}_{b,N}^{R}\cdot \mathbf{y}_{p}\right)\left[(i_{p - 1},\ldots ,i_{0})\right] = \mathbf{y}_{p}\left[(i_{0},\ldots ,i_{p - 1})\right]$ . Equation (93) then established the claim.
To complete the proof we prove Equation (94) by induction on  $a$ .
We next consider the base case of  $a = 1$ . Note that  $\mathbf{y}_1$  is just the jth column of  $\mathbf{B}_0$  (as  $\mathbf{y}_0 = \mathbf{e_j}$ ) and in that case Equation (94) follows from Equation (92).
For the inductive hypothesis, assume that Equation (94) is true for  $a$  for some  $a \geq 1$ . We now want to argue Equation (94) for  $\mathbf{y}_{a + 1}$ . Towards that end define
$$
\mathbf {z} _ {a} = \left(\mathbf {P} _ {b ^ {p - a}, b ^ {p - a + 1}, N}\right) ^ {\top} \mathbf {y} _ {a} = \mathbf {P} _ {b, b ^ {p - a + 1}, N} \mathbf {y} _ {a}.
$$
$$
\mathbf {y} _ {a + 1} = \mathbf {B} _ {a} \cdot \mathbf {z} _ {a}. \tag {95}
$$
Now by definition of  $\mathbf{P}_{b,b^{p - a + 1},N}$ , we have
$$
\mathbf {z} _ {a} \left[ \left(\left(i _ {0}, \dots , i _ {a - 1}\right), \left(j _ {p - 1}, \dots , j _ {a}\right)\right) \right] = \mathbf {y} _ {a} \left[ \left(\left(i _ {0}, \dots , i _ {a - 2}\right), \left(j _ {p - 1}, \dots , j _ {a}\right), i _ {a - 1}\right) \right].
$$
We claim that Equation (94) is true for  $a + 1$  from the above along with Equation (95) and Equation (92). Indeed, fix any  $(i_0, \ldots, i_{a-1})$ . Then in Equation (95) the entry  $\mathbf{z}_a[(i_0, \ldots, i_{a-1}), (j_{p-1}, \ldots, j_a)]$  gets multiplied by the entries  $\mathbf{B}_a^{((i_0, \ldots, i_{a-1}), (j_{p-1}, \ldots, j_{a+1}))} [i_a, j_a]$  for all values of  $i_a \in [0, \sqrt[p]{N}]$ . The inductive hypothesis and Equation (92) then proves the inductive step, as desired.
Finally, we note that we can generalize Algorithm 4 for constructing  $\mathbf{B}_a$  for  $0 \leq a < p$  (since this is the multivariate case some of the steps are a bit simplified):
Algorithm 6 BLOCKY MULTIVAR MONARCH( $\widetilde{\mathbf{B}}_{(0)}, \ldots, \widetilde{\mathbf{B}}_{(p-1)}, N, p$ )
Input:  $\widetilde{\mathbf{B}}_{(0)},\ldots ,\widetilde{\mathbf{B}}_{(p - 1)}\in \mathbb{R}^{N\times b}$  where  $b = \sqrt[p]{N}$    
Output: Block diagonal matrices  $\mathbf{B}_0,\dots ,\mathbf{B}_{p - 1}\in \mathbb{R}^{N\times N}$ $\triangleright$  Compose each output matrix from corresponding input matrix
1: for  $y \gets 0$  to  $p - 1$  do
2: for  $a \gets 0$  to  $\frac{N}{b} - 1$  do
3:  $\mathbf{B}_y^{(a)}\gets \mathbf{C}_b\cdot \widetilde{\mathbf{B}}_{(y)}^{(a)}[ab:ab + b - 1,:]$
4:  $\mathbf{B}_y\gets \mathrm{diag}(\mathbf{B}_y^{(0)},\ldots \mathbf{B}_y^{(\frac{N}{b} -1)})$
5: return  $\mathbf{B}_0, \ldots, \mathbf{B}_{p-1}$
D.6.5 Extensions and Open Questions
In this sub-section we outline certain (fairly) straightforward extension of our theoretical results and conclude with some open questions.
Comparing Equation (91) to Equation (1) We note that we can post multiply  $\mathbf{M}$  in Equation (91) with a large class of permutations for which Theorem 17 still holds. We outline the technical reason why this is true. At the heart of the argument for why Equation (86) gives a causal map is Lemma 18. Specifically note that the sum in RHS in Equation (84), is over all  $\mathbf{j} + \mathbf{j}' \preceq \mathbf{m}$ . The main observation is that this partial order still holds if we permute the  $b$ -variate representation of  $\mathbf{j}, \mathbf{j}'$  and  $\mathbf{m}$  in the same way. In other words, for any permutation  $\sigma : [0, p) \to [0, p)$  if we define  $\sigma(\mathbf{j}) = (j_{\sigma(0)}, \ldots, j_{\sigma(p-1)})$  and similarly  $\sigma(\mathbf{j}')$ ,  $\sigma(\mathbf{m})$ . Then we still have  $\sigma(\mathbf{j}) + \sigma(\mathbf{j}') \preceq \sigma(\mathbf{m})$ . This in turn implies the following. Let  $\mathbf{P}_{\sigma}$  be a permutation that maps  $\mathbf{j} \in [0, \sqrt[p]{N})^p$  to  $\sigma(\mathbf{j}) \in [0, \sqrt[p]{N})^p$ . Then Theorem 17 holds if we replace  $\mathbf{M}'$  by  $\mathbf{M} \cdot \mathbf{P}_{\sigma}$  with  $\mathbf{M}$  as in Equation (91).