SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Chen Zhang $^{1}$  Xiaofeng Cao $^{1,\boxtimes}$  Weiyang Liu $^{2,3}$  Ivor W. Tsang $^{4}$  James T. Kwok $^{5}$
$^{1}$ School of Artificial Intelligence, Jilin University, China
$^{2}$ Max Planck Institute for Intelligent Systems, Germany,  $^{3}$ University of Cambridge, UK
$^{4}$ CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore
$^{5}$ Hong Kong University of Science and Technology, Hong Kong, China
u3567831@connect.hku.hk, xiaofengcao@jlu.edu.cn, wl396@cam.ac.uk
ivorTsang@cfar.a-star.edu.sg, jamesk@cse.ust.hk
We study the problem of teaching multiple learners simultaneously in the nonparametric iterative teaching setting, where the teacher iteratively provides examples to the learner for accelerating the acquisition of a target concept. This problem is motivated by the gap between current single-learner teaching setting and the real-world scenario of human instruction where a teacher typically imparts knowledge to multiple students. Under the new problem formulation, we introduce a novel framework - Multi-learner Nonparametric Teaching (MINT). In MINT, the teacher aims to instruct multiple learners, with each learner focusing on learning a scalar-valued target model. To achieve this, we frame the problem as teaching a vector-valued target model and extend the target model space from a scalar-valued reproducing kernel Hilbert space used in single-learner scenarios to a vector-valued space. Furthermore, we demonstrate that MINT offers significant teaching speed-up over repeated single-learner teaching, particularly when the multiple learners can communicate with each other. Lastly, we conduct extensive experiments to validate the practicality and efficiency of MINT.
Machine teaching [81, 83] considers the problem of how to design the most effective teaching set, typically with the smallest amount of (teaching) examples possible, to facilitate rapid learning of the target models by learners based on these examples. It can be thought of as an inverse problem of machine learning, in the sense that the student aims to learn a target model on a given dataset, while the teacher constructs such a (minimal) dataset. Machine teaching has many applications in computer vision [67, 68], crowd sourcing [59, 60, 78, 79] and cyber security [2, 3, 39, 53].
Roughly speaking, machine teaching can be carried out in a batch [80, 81, 33] or iterative [36, 37, 38, 52] fashion, depending on how teachers and learners interact with each other. Batch teaching focuses on single-round interaction, that is, the most representative and effective teaching dataset are designed to be fed to the learner in one shot. After that, the leaner solely and assiduously learns a target model from this dataset without further interaction. With practical considerations, iterative teaching extends such a single-round mode to a multi-round one. It studies the case where the teacher feeds examples based on learners' status (current learnt models) round by round, such that the learner can converge to a target model within fewer rounds. The minimal count of such rounds (or iterations) is referred to as iterative teaching dimension [36, 37].
Considering that previous works on iterative machine teaching usually limit target models in a parameterized family, that is, assuming the target model can be represented by some parameters, nonparametric iterative machine teaching [73] extends such single family to a general nonparametric one. This allows multiple possibility of the target model family. Specifically, by formulating nonparametric teaching in a reproducing kernel Hilbert space (RKHS), [73] introduce various families of target models associated with kernels, e.g., Gaussian and Laplacian kernels in RKHS.
However, existing nonparametric teaching only focuses on the single-learner setting (i.e., teaching a scalar-valued target model or function to a single learner), and it is computationally inefficient to carry out this same single-learner teaching repeatedly for the multi-learner scenario, where the teacher needs to teach numerous scalar-valued target functions to multiple learners and a single learner can only learn one. For example, when taking a colored picture with three (RGB) channels as a multi-learner target function (e.g., [62, 20]), the iteration number of repeatedly performing single-learner teaching for each channel is intuitively triple that of carrying out them simultaneously. Another example is the scenario where the single-learner target model having a large input space. Such a model can be divided into multiple smaller ones with an input space of appropriate size, and these smaller models can be formulated together into a multi-learner target model
[7, 64, 66, 61, 74]. Concretely, one can divide a single high-revolution picture into multiple subregions, and the original single-learner target function will become a multi-learner one. These examples motivate us to study a generalized framework, called multi-learner nonparametric teaching (MINT), where a vector-valued target model (instead of a scalar-valued model) is being taught. A comparison between single-learner teaching and MINT is illustrated in Figure 1.
It is therefore of great significance to generalize the recent single-learner nonparametric teaching [73] to MINT [11, 14, 77]. MINT is guided by the insight that repeatedly (or sequentially) teaching multiple scalar-valued target functions can be viewed as teaching a vector-valued target function. The theoretical motivation comes from the well-developed results of kernels for vector-valued functions [56, 44, 21], an important approach to deal with multiple data sources. This inspires us to formulate MINT as a teaching problem of a vector-valued target function, where sequentially teaching multiple scalar-valued functions (for single-learner teaching) becomes a special case of teaching a vector-valued function [43, 21, 14]. We emphasize that, compared to the case where a single learner is learning a vector-valued target function, the multi-learner setting offers a general framework that can be generalized to more complicated scenarios, e.g., learners operate within different feature spaces, and learners are able to communicate with each other. We summarize our major contributions below:
- By analyzing general vector-valued RKHS [10, 45, 4], we study the multi-learner nonparametric teaching (MINT), where the teacher selects examples based on a vector-valued target function (each component of the vector-valued function is a scalar-valued function for a single learner)<sup>2</sup>, such that multiple learners can learn their own target models simultaneously.
- By enabling the communication among multiple learners, learners can update themselves with a linear combination of current learnt functions of all learners [23, 12]. We study a communicated MINT where the teacher not only selects examples but also injects the guidance of communication.
- Under mild assumptions, we characterize the efficiency of our multi-learner generalization of nonparametric teaching. More importantly, we also empirically demonstrate its efficiency.
Each channel in a colored picture can be viewed as a single-learner (scalar-valued) target function whose inputs and outputs are the pixel location and values, respectively [73].
2When components of it are highly correlated with each other, the teacher in each iteration can select a teaching set based on one component, and use it to teach all learners simultaneously.
Machine teaching. Recently, there has been a surge of interest in the field of machine teaching, see [81, 83, 36, 37, 67] and references therein. Batch machine teaching has examined the behaviors of various types of learners, including linear learners [33], forgetful learners [27, 37], version space learners [13, 65], hypothesis learner [40] and reinforcement learners [29, 76]. Further, by extending the single-round teaching mode to a multi-round one, iterative teaching has attracted growing attention in recent studies [36, 37, 40, 50, 32, 38, 69, 52, 73]. Specifically, [38] focuses on label synthesis teaching, while [52] proposes generative teaching. Additionally, [73] relaxes the parametric assumption on target models and generalizes the previous iterative teaching to a nonparametric iterative one [30, 51]. In contrast to previous works that mainly concentrate on the single-learner teaching, this work aims to address a more practical task - teaching multi-learner (vector-valued) target models. In this regard, the realistic practical scenario, classroom teaching [82, 71], is highly relevant, where it examines multiple learners by partitioning them into groups in batch and iterative setting, respectively. However, their works are also limited to the parametric setting, and their methods therefore are not immediately generalizable to nonparametric situations. In contrast, our work investigates multi-learner teaching in the nonparametric setting.
Multi-task functional optimization. Functional optimization [58, 6, 58, 22, 84, 63, 75] is a fundamental and significant task in various fields, such as variational inference [35, 34], barycenter problem [57, 70], and Residual Networks [48, 49, 26]. It involves mapping from input to output without having pre-defined parameters, optimized over a more general function space such as the reproducing kernel Hilbert space (RKHS), Sobolev space [1, 46], and Fréchet space [47]. Notably, the functional gradient descent algorithm has been studied extensively for functional optimization in RKHS due to its regular properties [41, 42, 16, 35, 46, 5, 57]. Meanwhile, modeling in RKHS of vector-valued functions [56, 21, 44, 10, 9, 14, 19, 45] is an important approach to handle multi-task problem. Specifically, [21, 9, 4] focus on the analysis of the kernel and [28, 12] study multi-task versions of online mirror descent, which displays the similarity to multi-learner teaching in the sense of simultaneous execution. These theoretical and empirical works motivate us to extend single-learner teaching to a multi-learner one by analyzing functional gradient in vector-valued RKHS.
Notation. Let  $\mathcal{X} \subseteq \mathbb{R}^n$  be a  $n$  dimensional input (i.e., feature) space and  $\mathcal{Y} \subseteq \mathbb{R}$  be a output (i.e., label) space. By  $\mathcal{X}^d = \mathcal{X}_1 \times \dots \times \mathcal{X}_d \subseteq \mathbb{R}^{n \times d}$  we denote a  $d$ -learner input space, and let  $\mathcal{Y}^d = \mathcal{Y}_1 \times \dots \times \mathcal{Y}_d \subseteq \mathbb{R}^d$  be a  $d$ -learner output space. Let a  $d$ -dimensional column vector with  $a_i$ , entries indexed by  $i \in \mathbb{N}_d$  ( $\mathbb{N}_k := \{1, \dots, k\}$ ), be  $[a_i]^d = (a_1, \dots, a_d)^T$  (we may denote it by  $a$  for simplicity), and a 1-vector of size  $d$  be  $[1]^d = (1, \dots, 1)^T \in \mathbb{R}^d$ . By  $M_{(i,\cdot)}$  we denote  $i$ -th row vector of a matrix  $M$ , and let  $M_{(\cdot,i)}$  be its  $i$ -th column vector. A  $d$ -learner teaching sequence in size  $d \times k$  is a collection of examples, notated as  $\mathcal{D} = \{(x_{i,j}, y_{i,j}) \in \mathcal{X} \times \mathcal{Y}\}^3$  with the learner index  $i \in \mathbb{N}_d$  and the example index  $j \in \mathbb{N}_k$ . We note the collection of such teaching sequence candidates by  $\mathbb{D}^d \ni \mathcal{D}$ , which is referred to as the knowledge domain of the teacher [36].
Let  $K(x,x') : \mathcal{X} \times \mathcal{X} \mapsto \mathbb{R}$  be a scalar-valued positive definite kernel function, which can be equivalently notated by  $K(x,x') = K_x(x') = K_{x'}(x)$ , and one can abbreviate  $K_x(\cdot)$  by  $K_x$ . The scalar-valued reproducing kernel Hilbert space (RKHS)  $\mathcal{H}$  defined by  $K(x,x')$  is the closure of linear span  $\{f : f(\cdot) = \sum_{i=1}^{r} \alpha_i K(x_i, \cdot), \alpha_i \in \mathbb{R}, r \in \mathbb{N}, x_i \in \mathcal{X}\}$  equipped with inner product  $\langle f, g \rangle_{\mathcal{H}} = \sum_{ij} \alpha_i \beta_j K(x_i, x_j)$  when  $g = \sum_j \beta_j K_{x_j}$ . We assume that given the scalar-valued target model  $f^* \in \mathcal{H}$ , one can uniquely identify a teaching example by its  $x^\dagger$  for brevity,  $(x^\dagger, y^\dagger) = (x^\dagger, f^*(x^\dagger))$ . Let  $\mathcal{H}^d = \mathcal{H}_i \times \dots \times \mathcal{H}_d$  be a RKHS of vector-valued functions  $\pmb{f} = [f_i]^d$  with  $f_i \in \mathcal{H}^4$ , equipped with inner product  $\langle f, g \rangle_{\mathcal{H}^d} = \sum_{i=1}^d \langle f_i, g_i \rangle_{\mathcal{H}}$ . For simplicity, we use the vector-input  $K(\pmb{x}, \pmb{x}') = [K(x_i, x_i')]^d$  to denote kernels in vector-valued RKHS  $\mathcal{H}^d$ . For a functional  $F : \mathcal{H}^d \mapsto \mathbb{R}$ , its Fréchet derivative [16, 34, 57] is defined as following:
3To avoid clutter in the notation, we assume that all learners share same input and output spaces, i.e.,  $\mathcal{X}_i = \mathcal{X}$  and  $\mathcal{Y}_i = \mathcal{Y}$ . The results for different input and output spaces can be derived by plugging into specific  $\mathcal{X}_i$  and  $\mathcal{Y}_i$  directly.
4To simplify the notation, we assume that the RKHS of target models are the same for all learners, i.e.,  $\mathcal{H}_i = \mathcal{H}$
Definition 1. (Fréchet derivative in vector-valued RKHS) For a vector-valued functional  $F: \mathcal{H}^d \mapsto \mathbb{R}$ , its Fréchet derivative  $\nabla_{\pmb{f}} F[\pmb{f}]$  at  $\pmb{f} \in \mathcal{H}^d$  is defined implicitly as  $F[\pmb{f} + \epsilon \pmb{g}] = F[\pmb{f}] + \epsilon \langle \nabla_{\pmb{f}} F[\pmb{f}], \pmb{g} \rangle_{\mathcal{H}^d} + \mathcal{O}(\epsilon^2)$  for any  $\pmb{g} \in \mathcal{H}^d$  and  $\epsilon \in \mathbb{R}$ , which is a function in  $\mathcal{H}^d$ .
Using the Riesz-Fréchet representation theorem [31, 55], the evaluation functional of vector-valued functions is defined in the following:
Definition 2. For a vector-valued reproducing kernel Hilbert space  $\mathcal{H}^d$  with a positive definite kernel  $K_{\pmb{x}}\in \mathcal{H}^{d}$ , where  $\pmb {x} = [x_{i,j_i}]^d\in \mathcal{X}^d$  and the example index  $j_{i}\in \mathbb{N}_{k}$ , we define evaluation functional  $E_{\pmb{x}}[\cdot ]:\mathcal{H}^{d}\mapsto \mathbb{R}$  as
$$
E _ {\boldsymbol {x}} [ \boldsymbol {f} ] = \langle \boldsymbol {f}, K _ {\boldsymbol {x}} (\cdot) \rangle_ {\mathcal {H} ^ {d}} = \sum_ {i = 1} ^ {d} \left\langle f _ {i}, K _ {x _ {i, j _ {i}}} \right\rangle_ {\mathcal {H}} = \sum_ {i = 1} ^ {d} f _ {i} \left(x _ {i, j _ {i}}\right), \boldsymbol {f} = \left(f _ {1}, \dots , f _ {d}\right) ^ {T} \in \mathcal {H} ^ {d}. \tag {1}
$$
Single-learner nonparametric teaching. [73] formulates the single-learner nonparametric teaching as a functional minimization over single-learner  $\mathbb{D}$  in scalar-valued RKHS:
$$
\mathcal {D} ^ {*} = \underset {\mathcal {D} \in \mathbb {D}} {\arg \min } \quad \mathcal {M} (\hat {f} ^ {*}, f ^ {*}) + \lambda \cdot \operatorname {l e n} (\mathcal {D}) \quad \text {s . t .} \quad \hat {f} ^ {*} = \mathcal {A} (\mathcal {D}), \tag {2}
$$
where  $\mathcal{M}$  is a disagreement between  $\hat{f}^*$  and  $f^*$  (e.g.,  $L_{2}$  distance defined in RKHS  $\mathcal{M}(\hat{f}^*, f^*) = \|\hat{f}^* - f^*\|_{\mathcal{H}}$ ),  $\mathrm{len}(\cdot)$  is the length of the teaching sequence  $\mathcal{D}$  (i.e., the ITD defined in [36]) controlled by a regularized constant  $\lambda$ , and  $\mathcal{A}$  denotes the learning algorithm of learners. Usually,  $\mathcal{A}(\mathcal{D})$  is taken as empirical risk minimization:
$$
\hat {f} ^ {*} = \underset {f \in \mathcal {H}} {\arg \min } \mathbb {E} _ {(x, y) \sim \mathbb {Q} (x, y)} [ \mathcal {L} (f (x), y) ], \tag {3}
$$
with single-learner convex loss function  $\mathcal{L}$ . As introduced in Section 1, iterative teaching [36, 37] focuses on some specific optimization algorithm that the learner adopts [38]. In the nonparametric setting, we consider the functional gradient descent:
$$
f ^ {t + 1} \leftarrow f ^ {t} - \eta^ {t} \mathcal {G} (\mathcal {L}; f ^ {t}; \mathcal {D} ^ {t}), \tag {4}
$$
where  $t = 0,1,\dots ,T$  serves as an iteration index,  $\eta^t >0$  (i.e., a small constant) denotes the learning rate for the  $t$ -th iteration, and  $\mathcal{G}$  represents the gradient functional evaluated at  $\mathcal{D}^t$ .
Specifically, [73] investigates the teaching algorithms within a practical teaching protocol and gray-box setting. This involves a teacher that has no knowledge about the learner, including the learning rate and specific loss function, but still is able to keep track of the learnt model during each iteration. Two functional teaching algorithms are proposed: Random Functional Teaching (RFT) and Greedy FT (GFT). The former essentially adopts random sampling, and it serves as a simple baseline, which can also be viewed as a functional analogue of stochastic gradient descent [54, 25]. In contrast, GFT picks examples by maximizing the corresponding disagreement between the target and current models [5, 18], and has been shown to be more effective than RFT both theoretically and experimentally.
4 MINT: Multi-learner nonparametric teaching
In this section, we begin by defining multi-learner nonparametric teaching as a functional minimization in a vector-valued RKHS. Next, we analyze a vanilla MINT where multiple learners independently and simultaneously learn corresponding components of a vector-valued target function. Lastly, we investigate a communicated MINT where the teacher does not only provide examples but also guide multiple learners in the process of linearly combining present learnt functions.
4.1 Teaching settings
To define MINT, we expand scalar-valued target models in single-learner teaching to vector-valued ones and modify other notations to suit the multi-learner setting. More specifically, we redefine functional minimization of Eq. 2 as follows:
$$
\boldsymbol {\mathcal {D}} ^ {*} = \underset {\boldsymbol {\mathcal {D}} \in \mathbb {D} ^ {d}} {\arg \min } \quad \mathcal {M} (\hat {\boldsymbol {f}} ^ {*}, \boldsymbol {f} ^ {*}) + \lambda \cdot \operatorname {l e n} (\boldsymbol {\mathcal {D}}) \quad \text {s . t .} \quad \hat {\boldsymbol {f}} ^ {*} = \mathcal {A} (\boldsymbol {\mathcal {D}}), \tag {5}
$$
where  $\pmb{f}^{*} \in \mathcal{H}^{d}$  refers to a vector-valued target model, and other notations bear the same meaning as in Eq. 2. The learning algorithm  $\mathcal{A}$  arrives at the following solution:
$$
\hat {\boldsymbol {f}} ^ {*} = \underset {\boldsymbol {f} \in \mathcal {H} ^ {d}} {\arg \min } \mathbb {E} _ {(\boldsymbol {x}, \boldsymbol {y})} [ \mathcal {L} (\boldsymbol {f} (\boldsymbol {x}), \boldsymbol {y}) ], \tag {6}
$$
where  $(\pmb{x},\pmb{y})\in \mathcal{X}^d\times \mathcal{Y}^d$  and  $(\pmb {x},\pmb {y})\sim [\mathbb{Q}_i(x_i,y_i)]^d$  . Evaluated at an example vector  $(\pmb {x},\pmb {y}) = [(x_{i,j_i},y_{i,j_i})]^d$  with the example index  $j_{i}\in \mathbb{N}_{k}$  , the multi-learner convex loss  $\mathcal{L}$  therein is
$$
\mathcal {L} (\boldsymbol {f} (\boldsymbol {x}), \boldsymbol {y}) = \sum_ {i = 1} ^ {d} \mathcal {L} _ {i} \left(f _ {i} \left(x _ {i, j _ {i}}\right), y _ {i, j _ {i}}\right) = E _ {\boldsymbol {x}} \left[ \left[ \mathcal {L} _ {i} \left(f _ {i}, y _ {i, j _ {i}}\right) \right] ^ {d} \right], \tag {7}
$$
where  $\mathcal{L}_i$  is the convex loss for  $i$ -th learner. We can also express it as  $\langle [1]^d, [\mathcal{L}_i(f_i(x_{i,j_i}), y_{i,j_i})]^d \rangle_{\mathcal{H}^d}$ , where the vector  $[1]^d$  can be replaced by a weight vector  $[w_i]^d \in \mathbb{R}^d$  to adjust the significance of each learner relative to others. Under iterative setting [36, 37] which explores teaching algorithms from the viewpoint of optimization and approximation, we present vector-valued functional gradient descent:
$$
\boldsymbol {f} _ {A ^ {t}} ^ {t + 1} \leftarrow A ^ {t} \cdot \boldsymbol {f} ^ {t} - \boldsymbol {\eta} ^ {t} \odot \mathcal {G} (\mathscr {L}; A ^ {t} \cdot \boldsymbol {f} ^ {t}; \mathscr {D} ^ {t}), \tag {8}
$$
where  $\odot$  denotes the element-wise multiplication,  $\pmb{\eta}^t = [\eta_i]^d = (\eta_1^t,\dots ,\eta_d^t)^T$  is a vector of learning rates that corresponds to  $d$  learners and the communication matrix  $A^{t} = \arg \min_{A\in \mathbb{R}^{d\times d}}\| A\pmb{f}^{t} - \pmb{f}^{*}\|_{\mathcal{H}^{d}}$  signifies a matrix with row sums that are equal to one in order to maintain the output's scale. Equivalently, by denoting that  $A_{(i,\cdot)}^{t} = \arg \min_{M_{(i,\cdot)}^{t}\in \mathbb{R}^{1\times d}}\| M_{(i,\cdot)}^{t}\cdot \pmb{f}^{t} - f_{i}^{*}\|_{\mathcal{H}}$ , it can also be expressed in a learner-specific (i.e., component-wise) fashion as  $f_{i}^{t + 1}\gets A_{(i,\cdot)}^{t}\cdot f_{i}^{t} - \eta_{i}^{t}\mathcal{G}_{i}(\mathcal{L};A_{(i,\cdot)}^{t}\cdot \pmb{f}^{t};\pmb{\mathcal{D}}^{t})$ , where  $i\in \mathbb{N}_d$  is the learner index.
We investigate MINT in the gray-box setting, which is equivalent to the one considered in [73]. To facilitate the theoretical analysis, we adopt some moderate assumptions regarding  $\mathcal{L}_i$  and kernels, which align with those made in [73].
Assumption 3. Each loss  $\mathcal{L}_i(f_i), i \in \mathbb{N}_d$  is  $L_{\mathcal{L}_i}$ -Lipschitz smooth, i.e.,  $\forall f_i, f_i' \in \mathcal{H}$ ,  $x_i \in \mathcal{X}$  and  $i \in \mathbb{N}_d$
$$
\left| E _ {x _ {i}} \left[ \nabla_ {f} \mathcal {L} _ {i} \left(f _ {i}\right) \right] - E _ {x _ {i}} \left[ \nabla_ {f} \mathcal {L} _ {i} \left(f _ {i} ^ {\prime}\right) \right] \right| \leq L _ {\mathcal {L} _ {i}} \left| E _ {x _ {i}} \left[ f _ {i} \right] - E _ {x _ {i}} \left[ f _ {i} ^ {\prime} \right] \right|,
$$
where  $L_{\mathcal{L}_i} \geq 0$  is a constant. To simplify the notation, we assume that  $L_{\mathcal{L}_i} = L_{\mathcal{L}}$  for all  $i \in \mathbb{N}_d$ .
Assumption 4. Each kernel  $K(x,x^{\prime})\in \mathcal{H}$  is bounded, i.e.,  $\forall x,x^{\prime}\in \mathcal{X}$ ,  $K(x,x^{\prime})\leq M_{K}$ , where  $M_K\geq 0$  is a constant.
With regards to diverse knowledge domains, we narrow the scope of investigation in this study to the synthesis-based teacher setting [36]. Furthermore, it's worth noting that by limiting the knowledge domain to a specific pool, it can result in multiple learners converging to a suboptimal  $f^{*\prime}$ , such findings of pool-based teachers are comparable and can be deduced accordingly, as discussed in Remark 7 of [73].
4.2 Vanilla multi-learner teaching
In tackling MINT, we begin by examining a basic scenario in which multiple learners concurrently learn corresponding components of a vector-valued target function without communication between them [28, 12], that is,  $A^t$  in Eq. 8 is assigned the identity matrix  $I_d$ . This simplifies Eq. 8 to
$$
\boldsymbol {f} ^ {t + 1} \leftarrow \boldsymbol {f} ^ {t} - \boldsymbol {\eta} ^ {t} \odot \mathcal {G} (\mathcal {L}; \boldsymbol {f} ^ {t}; \boldsymbol {\mathcal {D}} ^ {t}). \tag {9}
$$
In this vanilla setting, multiple learners do not linearly combine learned functions of all learners; rather, it updates its functions by Eq. 9 alone.
In light of the definition of Fréchet derivative in vector-valued RKHS (as presented in Defi.1), we present Chain Rule for vector-valued functional gradients [24] as a Lemma.
Lemma 5. (Chain rule for vector-valued functional gradients) For differentiable functions  $G: \mathbb{R} \mapsto \mathbb{R}$  that are functions of functionals  $F$ ,  $G(F[f])$ , the expression
$$
\nabla_ {f} G (F [ \boldsymbol {f} ]) = \frac {\partial G (F [ \boldsymbol {f} ])}{\partial F [ \boldsymbol {f} ]} \cdot \nabla_ {\boldsymbol {f}} F [ \boldsymbol {f} ] \tag {10}
$$
is usually referred to as the chain rule.
To obtain the derivative of the evaluation functional [16], we introduce Lemma 6, with the proof of this lemma deferred to Appendix B.
Lemma 6. For an evaluation functional in vector-valued RKHS  $E_{\mathbf{x}}[\pmb{f}] = \sum_{i=1}^{d} f_i(x_{i,j_i}) : \mathcal{H}^d \mapsto \mathbb{R}$  where  $\pmb{x} = [x_{i,j_i}]^d \in \mathcal{X}^d$ , its gradient is a  $d$ -dimensional vector  $\nabla_{\pmb{f}} E_{\mathbf{x}}[\pmb{f}] = K_{\mathbf{x}} = [K_{x_{i,j_i}}]^d \in \mathcal{H}^d$ .