SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yuka Hashimoto
NTT Network Service Systems Laboratories /
RIKEN AIP,
yuka hashimoto@ntt.com
Masahiro Ikeda
RIKEN AIP / Keio University,
masahiro.ikeda@riken.jp
Hachem Kadri
Aix-Marseille University, CNRS, LIS,
hachem.kadri@lis-lab.fr
Reproducing kernel Hilbert  $C^*$ -module (RKHM) is a generalization of reproducing kernel Hilbert space (RKHS) by means of  $C^*$ -algebra, and the Perron-Frobenius operator is a linear operator related to the composition of functions. Combining these two concepts, we present deep RKHM, a deep learning framework for kernel methods. We derive a new Rademacher generalization bound in this setting and provide a theoretical interpretation of benign overfitting by means of Perron-Frobenius operators. By virtue of  $C^*$ -algebra, the dependency of the bound on output dimension is milder than existing bounds. We show that  $C^*$ -algebra is a suitable tool for deep learning with kernels, enabling us to take advantage of the product structure of operators and to provide a clear connection with convolutional neural networks. Our theoretical analysis provides a new lens through which one can design and analyze deep kernel methods.
Kernel methods and deep neural networks are two major topics in machine learning. Originally, they had been investigated independently. However, their interactions have been researched recently. One important perspective is deep kernel learning [1, 2, 3]. In this framework, we construct a function with the composition of functions in RKHSs, which is learned by given training data. Representer theorems were shown for deep kernel methods, which guarantee the representation of solutions of minimization problems only with given training data [4, 5]. We can combine the flexibility of deep neural networks with the representation power and solid theoretical understanding of kernel methods. Other important perspectives are neural tangent kernel [6, 7] and convolutional kernel [8], which enable us to understand neural networks using the theory of kernel methods. In addition, Bietti et al. [9] proposed a regularization of deep neural network through a kernel perspective.
The generalization properties of kernel methods and deep neural networks have been investigated. One typical technique for bounding generalization errors is to use the Rademacher complexity [10, 11]. For kernel methods, generalization bounds based on the Rademacher complexity can be derived by the reproducing property. Bounds for deep kernel methods and vector-valued RKHSs (vvRKHSs) were also derived [5, 12, 13]. Table 1 shows existing Rademacher generalization bounds for kernel methods. Generalization bounds for deep neural networks have also been actively studied [14, 15, 16, 17, 18, 19]. Recently, analyzing the generalization property using the concept of benign overfitting
Reproducing space	Output dimension	Shallow	Deep
RKHS	1	O(√1/n) [10]	O(AL√1/n)
vvRKHS	d	O(√d/n) [12, 13]	O(AL√d/n) [5]
RKHM (existing)	d	O(√d/n) [27]	-
RKHM (ours)	d	O(d1/4/√n)	O(BLD1/4√n)
Table 1: Existing generalization bounds for kernel methods based on the Rademacher complexity and our bound (  $n$  : sample size,  $A$  : Lipschitz constant regarding the positive definite kernel,  $B$  : The norm of the Perron-Frobenius operator)
has emerged [20, 21]. Unlike the classical interpretation of overfitting, which is called catastrophic overfitting, it explains the phenomenon that the model fits both training and test data. For kernel regression, it has been shown that the type of overfitting can be described by an integral operator associated with the kernel [22].
In this paper, we propose deep RKHM to make the deep kernel methods more powerful. RKHM is a generalization of RKHS by means of  $C^*$ -algebra [23, 24, 25], where  $C^*$ -algebra is a generalization of the space of complex numbers, regarded as space of operators. We focus on the  $C^*$ -algebra of matrices in this paper. Applications of RKHMs to kernel methods have been investigated recently [26, 27]. We generalize the concept of deep kernel learning to RKHM, which constructs a map from an operator to an operator as the composition of functions in RKHMs. The product structure of operators induces interactions among elements of matrices, which enables us to capture relations between data components. Then, we derive a generalization bound of the proposed deep RKHM. By virtue of  $C^*$ -algebras, we can use the operator norm, which alleviates the dependency of the generalization bound on the output dimension.
We also use Perron-Frobenius operators, which are linear operators describing the composition of functions and have been applied to analyzing dynamical systems [28, 29, 30, 31], to derive the bound. The compositions in the deep RKHM are effectively analyzed by the Perron-Frobenius operators.
$C^*$ -algebra and Perron-Frobenius operator are powerful tools that provide connections of the proposed deep RKHM with existing studies. Since the norm of the Perron-Frobenius operator is described by the Gram matrix associated with the kernel, our bound shows a connection of the deep RKHM with benign overfitting. In addition, the product structure of a  $C^*$ -algebra enables us to provide a connection between the deep RKHMs and convolutional neural networks (CNNs).
Our main contributions are as follows.
- We propose deep RKHM, which is a generalization of deep kernel method by means of  $C^*$ -algebra. We can make use of the products in  $C^*$ -algebra to induce interactions among data components. We also show a representative theorem to guarantee the representation of solutions only with given data.
- We derive a generalization bound for deep RKHM. The dependency of the bound on the output dimension is milder than existing bounds by virtue of  $C^*$ -algebras. In addition, the Perron-Frobenius operators provide a connection of our bound with benign overfitting.
- We show connections of our study with existing studies such as CNNs and neural tangent kernel. We emphasize that our theoretical analysis using  $C^*$ -algebra and Perron-Frobenius operators gives a new and powerful lens through which one can design and analyze kernel methods.
2.1  $C^*$ -algebra and reproducing kernel  $C^*$ -module
$C^*$ -algebra, which is denoted by  $\mathcal{A}$  in the following, is a Banach space equipped with a product and an involution satisfying the  $C^*$  identity (condition 3 below).
Definition 2.1 ( $C^*$ -algebra) A set  $\mathcal{A}$  is called a  $C^*$ -algebra if it satisfies the following conditions:
1.  $\mathcal{A}$  is an algebra over  $\mathbb{C}$  and equipped with a bijection  $(\cdot)^{*}:\mathcal{A}\to \mathcal{A}$  that satisfies the following conditions for  $\alpha ,\beta \in \mathbb{C}$  and  $a,b\in \mathcal{A}$ :
$$
\bullet (\alpha a + \beta b) ^ {*} = \bar {\alpha} a ^ {*} + \bar {\beta} b ^ {*}, \quad \bullet (a b) ^ {*} = b ^ {*} a ^ {*}, \quad \bullet (a ^ {*}) ^ {*} = a.
$$
2.  $\mathcal{A}$  is a normed space endowed with  $\| \cdot \|_{\mathcal{A}}$ , and for  $a, b \in \mathcal{A}$ ,  $\| ab \|_{\mathcal{A}} \leq \| a \|_{\mathcal{A}} \| b \|_{\mathcal{A}}$  holds. In addition,  $\mathcal{A}$  is complete with respect to  $\| \cdot \|_{\mathcal{A}}$ .
3. For  $a \in \mathcal{A}$ , the  $C^*$  identity  $\| a^* a \|_{\mathcal{A}} = \| a \|_{\mathcal{A}}^2$  holds.
Example 2.2 A typical example of  $C^*$ -algebras is the  $C^*$ -algebra of  $d$  by  $d$  circulant matrices. Another example is the  $C^*$ -algebra of  $d$  by  $d$  block matrices with  $M$  blocks and their block sizes are  $\mathbf{m} = (m_1, \ldots, m_M)$ . We denote them by  $\operatorname{Circ}(d)$  and  $\operatorname{Block}(\mathbf{m}, d)$ , respectively. See [27, 32] for more details about these examples.
We now define RKHM. Let  $\mathcal{X}$  be a non-empty set for data.
Definition 2.3 ( $\mathcal{A}$ -valued positive definite kernel) An  $\mathcal{A}$ -valued map  $k: \mathcal{X} \times \mathcal{X} \to \mathcal{A}$  is called a positive definite kernel if it satisfies the following conditions:
-  $k(x, y) = k(y, x)^*$  for  $x, y \in \mathcal{X}$ ,
-  $\sum_{i,j=1}^{n} c_{i}^{*} k(x_{i}, x_{j}) c_{j}$  is positive semi-definite for  $n \in \mathbb{N}$ ,  $c_{i} \in \mathcal{A}$ ,  $x_{i} \in \mathcal{X}$ .
Let  $\phi : \mathcal{X} \to \mathcal{A}^{\mathcal{X}}$  be the feature map associated with  $k$ , defined as  $\phi(x) = k(\cdot, x)$  for  $x \in \mathcal{X}$  and let  $\mathcal{M}_{k,0} = \{\sum_{i=1}^{n} \phi(x_i)c_i | n \in \mathbb{N}, c_i \in \mathcal{A}, x_i \in \mathcal{X} (i = 1, \dots, n)\}$ . We can define an  $\mathcal{A}$ -valued map  $\langle \cdot, \cdot \rangle_{\mathcal{M}_k}: \mathcal{M}_{k,0} \times \mathcal{M}_{k,0} \to \mathcal{A}$  as
$$
\left\langle \sum_ {i = 1} ^ {n} \phi \left(x _ {i}\right) c _ {i}, \sum_ {j = 1} ^ {l} \phi \left(y _ {j}\right) b _ {j} \right\rangle_ {\mathcal {M} _ {k}} := \sum_ {i = 1} ^ {n} \sum_ {j = 1} ^ {l} c _ {i} ^ {*} k \left(x _ {i}, y _ {j}\right) b _ {j},
$$
which enjoys the reproducing property  $\langle \phi (x),f\rangle_{\mathcal{M}_k} = f(x)$  for  $f\in \mathcal{M}_{k,0}$  and  $x\in \mathcal{X}$ . The reproducing kernel Hilbert  $\mathcal{A}$ -module (RKHM)  $\mathcal{M}_k$  associated with  $k$  is defined as the completion of  $\mathcal{M}_{k,0}$ . See, for example, the references [33, 34, 26] for more details about  $C^*$ -algebra and RKHM.
2.2 Perron-Frobenius operator on RKHM
We introduce Perron-Frobenius operator on RKHM [35]. Let  $\mathcal{X}_1$  and  $\mathcal{X}_2$  be nonempty sets and let  $k_{1}$  and  $k_{2}$  be  $\mathcal{A}$ -valued positive definite kernels. Let  $\mathcal{M}_1$  and  $\mathcal{M}_2$  be RKHMs associated with  $k_{1}$  and  $k_{2}$ , respectively. Let  $\phi_1$  and  $\phi_2$  be the feature maps of  $\mathcal{M}_1$  and  $\mathcal{M}_2$ , respectively. We begin with the standard notion of linearity in RKHMs.
Definition 2.4 ( $\mathcal{A}$ -linear) A linear map  $A: \mathcal{M}_1 \to \mathcal{M}_2$  is called  $\mathcal{A}$ -linear if for any  $a \in \mathcal{A}$  and  $w \in \mathcal{M}_1$ , we have  $A(wa) = (Aw)a$ .
Definition 2.5 (Perron-Frobenius operator) Let  $f: \mathcal{X}_1 \to \mathcal{X}_2$  be a map. The Perron-Frobenius operator with respect to  $f$  is an  $\mathcal{A}$ -linear map  $P_f: \mathcal{M}_1 \to \mathcal{M}_2$  satisfying
$$
P _ {f} \phi_ {1} (x) = \phi_ {2} (f (x)).
$$
Note that a Perron-Frobenius operator is not always well-defined since  $ac = bc$  for  $a, b \in \mathcal{A}$  and nonzero  $c \in \mathcal{A}$  does not always mean  $a = b$ .
Definition 2.6 ( $\mathcal{A}$ -linearly independent) The set  $\{\phi_1(x) \mid x \in \mathcal{X}\}$  is called  $\mathcal{A}$ -linearly independent if it satisfies the following condition: For any  $n \in \mathbb{N}$ ,  $x_1, \ldots, x_n \in \mathcal{X}$ , and  $c_1, \ldots, c_n \in \mathcal{A}$ , “ $\sum_{i=1}^{n} \phi_1(x_i)c_i = 0$ ” is equivalent to “ $c_i = 0$  for all  $i = 1, \ldots, n$ ”.
Lemma 2.7 If  $\{\phi_1(x) \mid x \in \mathcal{X}\}$  is  $\mathcal{A}$ -linearly independent, then  $P_f$  is well-defined.
The following lemma gives a sufficient condition for  $\mathcal{A}$ -linearly independence.
Lemma 2.8 Let  $k_{1} = \tilde{k}a$ , i.e.,  $k$  is separable, for an invertible operator  $a$  and a complex-valued kernel  $\tilde{k}$ . Assume  $\{\tilde{\phi}(x) \mid x \in \mathcal{X}\}$  is linearly independent (e.g.  $\tilde{k}$  is Gaussian or Laplacian), where  $\tilde{\phi}$  is the feature map associated with  $\tilde{k}$ . Then,  $\{\phi_1(x) \mid x \in \mathcal{X}\}$  is  $\mathcal{A}$ -linearly independent.
Note that separable kernels are widely used in existing literature of vvRKHS (see e.g. [36]). Lemma 2.8 guarantees the validity of the analysis with Perron-Frobenius operators, at least in the separable case. This provides a condition for "good kernels" by means of the well-definedness of the Perron-Frobenius operator.
Figure 1: Overview of the proposed deep RKHM. The small blue squares represent matrix elements. In the case of the autoencoder (see Example 3.1),  $f_{1} \circ f_{2}$  is the encoder, and  $f_{3} \circ f_{4}$  is the decoder.
Notation We denote the Euclidean inner product and norm by  $\langle \cdot, \cdot \rangle$  and  $\|\cdot\|$  without the subscript. For  $a \in \mathcal{A}$ , let  $|a|_{\mathcal{A}}$  be the unique element in  $\mathcal{A}$  satisfying  $|a|_{\mathcal{A}}^2 = a^* a$ . If  $a$  is a matrix,  $\|a\|_{\mathrm{HS}}$  is the Hilbert-Schmidt norm of  $a$ . The operator norm of a linear operator  $A$  on an RKHM is denoted by  $\|A\|_{\mathrm{op}}$ . All the technical proofs are documented in the supplementary.
3 Deep RKHM
We now construct an  $L$ -layer deep RKHM. Let  $\mathcal{A} = \mathbb{C}^{d\times d}$  be the  $C^*$ -algebra of  $d$  by  $d$  matrices. Let  $\mathcal{A}_0,\ldots ,\mathcal{A}_L$  be  $C^*$ -subalgebras of  $\mathcal{A}$  and for  $j = 1,\dots ,L$  and  $k_{j}:\mathcal{A}_{j - 1}\times \mathcal{A}_{j - 1}\to \mathcal{A}_{j}$  be an  $\mathcal{A}_j$ -valued positive definite kernel for the  $j$ th layer. For  $j = 1,\dots ,L$ , we denote by  $\mathcal{M}_j$  the RKHM over  $\mathcal{A}_j$  associated with  $k_{j}$ . In addition, we denote by  $\tilde{\mathcal{M}}_j$  the RKHM over  $\mathcal{A}$  associated with  $k_{j}$ . Note that  $\mathcal{M}_j$  is a subspace of  $\tilde{\mathcal{M}}_j$  and for  $u,v\in \mathcal{M}_j$ , we have  $\langle u,v\rangle_{\tilde{\mathcal{M}}_j} = \langle u,v\rangle_{\mathcal{M}_j}$ . We set the function space corresponding to each layer as  $\mathcal{F}_L = \{f\in \mathcal{M}_L\mid f(x)\in \mathbb{R}^{d\times d}$  for any  $x\in \mathcal{A}_{L - 1}, \| f\|_{\mathcal{M}_L}\leq B_L\}$  and  $\mathcal{F}_j = \{f\in \mathcal{M}_j\mid f(x)\in \mathbb{R}^{d\times d}$  for any  $x\in \mathcal{A}_{j - 1}, \| P_f\|_{\mathrm{op}}\leq B_j\}$  for  $j = 1,\ldots ,L - 1$ . Here, for  $f\in \mathcal{M}_j$  with  $j = 1,\ldots ,L - 1$ ,  $P_{f}$  is the Perron-Frobenius operator with respect to  $f$  from  $\tilde{\mathcal{M}}_j$  to  $\tilde{\mathcal{M}}_{j + 1}$ . We assume the well-definedness of these operators. Then, we set the class of deep RKHM as
$$
\mathcal {F} _ {L} ^ {\text {d e e p}} = \left\{f _ {L} \circ \dots \circ f _ {1} \mid f _ {j} \in \mathcal {F} _ {j} (j = 1, \dots , L) \right\}.
$$
Figure 1 schematically shows the structure of the deep RKHM.
Example 3.1 We can set  $\mathcal{A}_j = \text{Block}((m_{1,j},\ldots,m_{M_j,j}),d)$ , the  $C^*$ -algebra of block diagonal matrices. By setting  $M_1 \leq \dots \leq M_l$  and  $M_l \geq \dots \geq M_L$  for  $l < L$ , the number of nonzero elements in  $\mathcal{A}_j$  decreases during  $1 \leq j \leq l$  and increases during  $l \leq j \leq L$ . This construction is regarded as an autoencoder, where the  $1 \sim l$ th layer corresponds to the encoder and the  $l + 1 \sim L$ th layer corresponds to the decoder.
Advantage over existing deep kernel methods Note that deep kernel methods with RKHSs and vvRKHSs have been proposed [1, 4, 2]. Autoencoders using RKHSs and vvRKHSs were also proposed [3, 5]. We have at least three advantages of deep RKHM over deep vvRKHSs or RKHSs: 1) useful structures of matrices stated in Remark 5.2, 2) availability of the operator norm in the generalization bound stated in Section 4, 3) connection with CNNs stated in Subsection 6.1.
4 Generalization Bound
We derive a generalization error for deep RKHM. To derive the bound, we bound the Rademacher complexity, which is one of the typical methods on deriving generalization bounds [11]. Let  $\Omega$  be a probability space equipped with a probability measure  $P$ . We denote the integral  $\int_{\Omega} s(\omega) \mathrm{d}P(\omega)$  of a measurable function  $s$  on  $\Omega$  by  $\operatorname{E}[s]$ . Let  $x_1, \ldots, x_n$  and  $y_1, \ldots, y_n$  be input and output samples from the distributions of  $\mathcal{A}_0$  and  $\mathcal{A}_L$ -valued random variables  $x$  and  $y$ , respectively. Let  $\sigma_{i,j}$  ( $i = 1, \ldots, d$ ,  $j = 1, \ldots, n$ ) be i.i.d. Rademacher variables. Let  $\sigma_j = (\sigma_{1,j}, \dots, \sigma_{d,j})$ . For an  $\mathbb{R}^d$ -valued function class  $\mathcal{F}$  and  $\mathbf{x} = (x_1, \dots, x_n)$ , the empirical Rademacher complexity  $\hat{R}_n(\mathbf{x}, \mathcal{F})$  is defined by  $\hat{R}_n(\mathbf{x}, \mathcal{F}) := \operatorname{E}[\sup_{f \in \mathcal{F}} \sum_{i=1}^{n} \langle \sigma_i, f(x_i) \rangle] / n$ .
4.1 Bound for shallow RKHMs
We use the operator norm to derive a bound, whose dependency on output dimension is milder than existing bounds. Availability of the operator norm is one of the advantages of considering
$C^*$ -algebras (RKHMs) instead of vectors (vvRKHSs). Note that although we can also use the Hilbert-Schmidt norm for matrices, it tends to be large as the dimension  $d$  becomes large. On the other hand, the operator norm is defined independently of the dimension  $d$ . Indeed, the Hilbert-Schmidt norm of a matrix  $a \in \mathbb{C}^{d \times d}$  is calculated as  $\left(\sum_{i=1}^{d} s_i^2\right)^{1/2}$ , where  $s_i$  is the  $i$ th singular value of  $a$ . The operator norm of  $a$  is the largest singular value of  $a$ .
To see the derivation of the bound, we first focus on the case of  $L = 1$ , i.e., the network is shallow. Let  $E > 0$ . For a space  $\mathcal{F}$  of  $\mathcal{A}$ -valued functions on  $\mathcal{A}_0$ , let  $\mathcal{G}(\mathcal{F}) = \{(x,y) \mapsto f(x) - y \mid f \in \mathcal{F}, \|y\|_{\mathcal{A}} \leq E\}$ . The following theorem shows a bound for RKHMs with the operator norm.
Theorem 4.1 Assume there exists  $D > 0$  such that  $\| k_1(x,x)\|_{\mathcal{A}}\leq D$  for any  $x\in \mathcal{A}_0$ . Let  $\tilde{K} = 4\sqrt{2} (\sqrt{DB_1} +E)B_1$  and  $\tilde{M} = 6(\sqrt{DB_1} +E)^2$ . Let  $\delta \in (0,1)$ . Then, for any  $g\in \mathcal{G}(\mathcal{F}_1)$  where  $\mathcal{F}_1$  is defined in Section 3, with probability at least  $1 - \delta$ , we have
$$
\| \mathrm {E} [ | g (x, y) | _ {\mathcal {A}} ^ {2} ] \| _ {\mathcal {A}} \leq \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} | g (x _ {i}, y _ {i}) | _ {\mathcal {A}} ^ {2} \right\| _ {\mathcal {A}} + \frac {\tilde {K}}{n} \left(\sum_ {i = 1} ^ {n} \mathrm {t r} k _ {1} (x _ {i}, x _ {i})\right) ^ {1 / 2} + \tilde {M} \sqrt {\frac {\log (2 / \delta)}{2 n}}.
$$
Theorem 4.1 is derived by the following lemmas. We first fix a vector  $p \in \mathbb{R}^d$  and consider the operator-valued loss function acting on  $p$ . We first show a relation between the generalization error and the Rademacher complexity of vector-valued functions. Then, we bound the Rademacher complexity. Since the bound does not depend on  $p$ , we can finally remove the dependency on  $p$ .
Lemma 4.2 Let  $\mathcal{F}$  be a function class of  $\mathbb{R}^{d\times d}$ -valued functions on  $\mathcal{A}_0$  bounded by  $C$  (i.e.,  $\| f(x)\|_{\mathcal{A}}\leq C$  for any  $x\in \mathcal{A}_0$ ). Let  $\tilde{\mathcal{G}} (\mathcal{F},p) = \{(x,y)\mapsto \| (f(x) - y)p\| ^2\mid f\in \mathcal{F},\| y\|_{\mathcal{A}}\leq E\}$  and  $M = 2(C + E)^2$ . Let  $p\in \mathbb{R}^d$  satisfy  $\| p\| = 1$  and let  $\delta \in (0,1)$ . Then, for any  $g\in \tilde{\mathcal{G}} (\mathcal{F},p)$  with probability at least  $1 - \delta$ , we have
$$
\| \mathrm {E} [ | g (x, y) | _ {\mathcal {A}} ^ {2} ] ^ {1 / 2} p \| ^ {2} \leq \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} | g (x _ {i}, y _ {i}) | _ {\mathcal {A}} ^ {2} \right\| _ {\mathcal {A}} + 2 \hat {R} _ {n} (\mathbf {x}, \tilde {\mathcal {G}} (\mathcal {F}, p)) + 3 M \sqrt {\frac {\log (2 / \delta)}{2 n}}.
$$
Lemma 4.3 With the same notations in Lemma 4.2, let  $K = 2\sqrt{2} (C + E)$ . Then, we have  $\hat{R}_n(\mathbf{x},\mathcal{G}(\mathcal{F},p)) \leq K\hat{R}_n(\mathbf{x},\mathcal{F}p)$ , where  $\mathcal{F}p = \{x \mapsto f(x)p \mid f \in \mathcal{F}\}$ .
Lemma 4.4 Let  $p \in \mathbb{R}^d$  satisfy  $\| p \| = 1$ . For  $\mathcal{F}_1$  defined in Section 3, we have
$$
\hat {R} _ {n} (\mathbf {x}, \mathcal {F} _ {1} p) \leq \frac {B _ {1}}{n} \left(\sum_ {i = 1} ^ {n} \operatorname {t r} \left(k \left(x _ {i}, x _ {i}\right)\right)\right) ^ {1 / 2}.
$$
4.2 Bound for deep RKHMs
We now generalize Theorem 4.1 to the deep setting  $(L\geq 2)$  using the Perron-Frobenius operators.
Theorem 4.5 Assume there exists  $D > 0$  such that  $\| k_L(x,x)\|_{\mathcal{A}} \leq D$  for any  $x \in \mathcal{A}_0$ . Let  $\tilde{K} = 4\sqrt{2} (\sqrt{D} B_L + E)B_1 \cdots B_L$  and  $\tilde{M} = 6(\sqrt{D} B_L + E)^2$ . Let  $\delta \in (0,1)$ . Then, for any  $g \in \mathcal{G}(\mathcal{F}_L^{\mathrm{deep}})$ , with probability at least  $1 - \delta$ , we have
$$
\| \mathrm {E} [ | g (x, y) | _ {\mathcal {A}} ^ {2} ] \| _ {\mathcal {A}} \leq \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} | g (x _ {i}, y _ {i}) | _ {\mathcal {A}} ^ {2} \right\| _ {\mathcal {A}} + \frac {\tilde {K}}{n} \Big (\sum_ {i = 1} ^ {n} \mathrm {t r} k _ {1} (x _ {i}, x _ {i}) \Big) ^ {1 / 2} + \tilde {M} \sqrt {\frac {\log (2 / \delta)}{2 n}}.
$$
We use the following proposition and Lemmas 4.2 and 4.3 to show Theorem 4.5. The key idea of the proof is that by the reproducing property and the definition of the Perron-Frobenius operator, we get  $f_{L}\circ \dots \circ f_{1}(x) = \langle \phi_{L}(f_{L - 1}\circ \dots \circ f_{1}(x)),f_{L}\rangle_{\tilde{\mathcal{M}}_{L}} = \bigl {\langle}P_{f_{L - 1}}\dots P_{f_{1}}\phi (x),f_{L}\bigr{\rangle}_{\tilde{\mathcal{M}}_{L}}$
Proposition 4.6 Let  $p \in \mathbb{R}^d$  satisfy  $\| p \| = 1$ . Then, we have
$$
\hat {R} _ {n} (\mathbf {x}, \mathcal {F} _ {L} ^ {\mathrm {d e e p}} p) \leq \frac {1}{n} \sup  _ {(f _ {j} \in \mathcal {F} _ {j}) _ {j}} \| P _ {f _ {L - 1}} \dots P _ {f _ {1}} | _ {\tilde {\mathcal {V}} (\mathbf {x})} \| _ {\mathrm {o p}} \| f _ {L} \| _ {\mathcal {M} _ {L}} \left(\sum_ {i = 1} ^ {n} \operatorname {t r} (k _ {1} (x _ {i}, x _ {i}))\right) ^ {1 / 2}.
$$
Here,  $\tilde{\mathcal{V}} (\mathbf{x})$  is the submodule of  $\tilde{\mathcal{M}}_1$  generated by  $\phi_1(x_1),\ldots \phi_1(x_n)$
Corollary 4.7 Let  $p \in \mathbb{R}^d$  satisfy  $\| p \| = 1$ . Then, we have
$$
\hat {R} _ {n} (\mathbf {x}, \mathcal {F} _ {L} ^ {\mathrm {d e e p}} p) \leq \frac {1}{n} B _ {1} \dots B _ {L} \left(\sum_ {i = 1} ^ {n} \operatorname {t r} \left(k _ {1} \left(x _ {i}, x _ {i}\right)\right)\right) ^ {1 / 2}.
$$
Comparison to deep vvRKHS We can also regard  $\mathbb{C}^{d\times d}$  as the Hilbert space equipped with the Hilbert-Schmidt inner product, i.e., we can flatten matrices and get  $d^2$ -dimensional Hilbert space. In this case, the corresponding operator-valued kernel is the multiplication operator of  $k(x,y)$ , which we denote by  $M_{k(x,y)}$ . Then, we can apply existing results for vvRKHSs [5, 12], which involve the term  $(\sum_{i=1}^{n}\mathrm{tr}(M_{k(x_i,x_i)}))^{1/2}$ . It is calculated as
$$
\sum_ {i = 1} ^ {n} \operatorname {t r} \left(M _ {k \left(x _ {i}, x _ {i}\right)}\right) = \sum_ {i = 1} ^ {n} \sum_ {j, l = 1} ^ {d} \left\langle e _ {j l}, k \left(x _ {i}, x _ {i}\right) e _ {j l} \right\rangle_ {\mathrm {H S}} = \sum_ {i = 1} ^ {n} \sum_ {j, l = 1} ^ {d} k \left(x _ {i}, x _ {i}\right) _ {l, l} = d \sum_ {i = 1} ^ {n} \operatorname {t r} k \left(x _ {i}, x _ {i}\right).
$$
Thus, using the existing approaches, we have the factor  $(d\sum_{i=1}^{n}\operatorname{tr}k(x_i,x_i))^{1/2}$ . On the other hand, we have the smaller factor  $\left(\sum_{i=1}^{n}\operatorname{tr}k(x_i,x_i)\right)^{1/2}$  in Theorems 4.1 and 4.5. Using the operator norm, we can reduce the dependency on the dimension  $d$ .
5 Learning Deep RKHMs
We focus on the practical learning problem. To learn deep RKHMs, we consider the following minimization problem based on the generalization bound derived in Section 4:
$$
\min  _ {\left(f _ {j} \in \mathcal {M} _ {j}\right) _ {j}} \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} \left| f _ {L} \circ \dots \circ f _ {1} \left(x _ {i}\right) - y _ {i} \right| _ {\mathcal {A}} ^ {2} \right\| _ {\mathcal {A}} + \lambda_ {1} \| P _ {f _ {L - 1}} \dots P _ {f _ {1}} | _ {\tilde {\mathcal {V}} (\mathbf {x})} \| _ {\mathrm {o p}} + \lambda_ {2} \| f _ {L} \| _ {\mathcal {M} _ {L}}. \tag {1}
$$