Online model selection by learning how compositional kernels evolve
Authors: Eura Shin
Predrag Klasnja
Susan Murphy
Finale Doshi-Velez
Venue: Accepted by TMLR ()

Abstract: Motivated by the need for efficient, personalized learning in health, we investigate the problem of online compositional kernel selection for multi-task Gaussian Process regression. Existing composition selection methods do not satisfy our strict criteria in health; selection must occur quickly, and the selected kernels must maintain the appropriate level of complexity, sparsity, and stability as data arrives online. We introduce the Kernel Evolution Model (KEM), a generative process on how to evolve kernel compositions in a way that manages the bias--variance trade-off as we observe more data about a user. Using pilot data, we learn a set of kernel evolutions that can be used to quickly select kernels for new test users. KEM reliably selects high-performing kernels for a range of synthetic and real data sets, including two health data sets.

Published in Transactions on Machine Learning Research (10/2023)
Online model selection by learning how compositional ker- nels evolve
Eura Shin
surashin@g.haroord edu
Department of Computer Science Harvard University
Predrag Klasnja
Hasnjo@uuiich. edu
School Information
University of Michigan
Susan A. Murphy
samarphy@@g.hrraad edu
Department of Computer Science Harvard University
Finale Doshi-
finale edu
Department of Computer Science Harvard Universily
Reviewed on OpenReview: https: // openrevie.. net/ forum? id= 23WZFQBUNS
Abstract
Motivated by the need for efficient, personalized learning in mobile health, we investigate the problem of online compositional kernel selection for multi-task Gaussian Process regres- sion. Existing composition selection methods do not satisfy our striet criteria in health; selection must occur quickly, and the selected kernels must maintain the appropriate level of complexity, sparsity, and stability data arrives online. We introduce the Kernel Evo- lution Model (KEM), generative process how to evolve kernel compositions in way that manages the bias-variance trade-off data, we learn set of kernel evolutions that can be used to quickly select kernels for new lest users. KEM reliably selects high-performing kernels for a range of synthetic and real data sets, including two health data sets.
observe more data about user. Using pilot
1 Introduction
Online, multi-task learning is common in many machine learning applications, ranging from recommender systems (Luo et al. 2020 to online education (Sayed et al.) 2020). In these settings, pools of available data grow for each user as they continue to interact with the application platform; a single task consists of maintaining the best model for an individual at any given time. We are specifically motivated to model online, multi-lask problems mobile health (mHeallth. For example, mobile fitness, we want model a user's daily activity levels given other mobility data to plan the delivery personalized messages. Each user's data is a time series in which model outputs are needed, online, in order personalize health interventions.
Mobile health applications have number of domain-specific assumptions and constraints. For applica- tions, we can assume the availability of pilot training data (on the order of tens to hundreds of users Lec et al. (2016); Calazzo el al. (2012); ; Choi el. al.] (2016)) that may be used to prepare a model for deployment in a high-stakes clinical trial. We can assume a generous pre-deployment period to work with the pilot data (on the order of months to years) and commit to a set of modcling docisions [Trella et al.) (022). To maintain the integrity of the study during deployment, our models are held to strict criteria: we must be able to update the model for each user as data arrives online, and the updated model must be appropriately

---

Published in Transactions on Machine Learning Research (10/2023)
Selected LINO-PERO Test likelihood: -3.36
Selected kernel: LINO-PERO
Selected kemel: LIND
Selected kemel: LINO-PERO
Test likelihood: -0.15
Test likelihood: .1.05
Test likelihood: -0.15
7.5
15
5.0
50
255
25
5
10
15
20
10 15
10
20
(a) Baseline model, N 5 (b) Baseline model, N 30
(c) Our model, N = 5 (d) Our model, N = 30
Our selection model (in red, fig. 1c and fig. 1d) infers the kernel composition for a new user and adjusts its complexity given more data, N. We demonstrate the quality of the kernels selected by our method (rod) versus the bascline method (bluc) by plotting the GP predictions given varying numbers of observations, shown in red "X"s. We want the selection method to choose kernels that result in good predictions when we have observed little data (fig, [a] fig. [c] and sufficient data (fig, [b fig, [d] The GP mcan predictions are dotted lines and the uncertainties are shaded; a good prediction as close as possible to the true function (the solid, black line) and has high test likelihood.
Figure 1:
complez and interpretable. By appropriate complexity, we mean that the model should not overfit when there is initially small amount of data per user, underfit as the available data grows; in mHealth, a poorly fit model risks sending inappropriate interventions that lead to user dropout. By interpretable, we mean that the model must predict uncertainties, include only a sparse number of relevant features, identify the structural relationship of the feature to the predictions, and mainlain slabilily in the features and rela- tionships between updates. Such characteristics allow for verification of the model and scientific discovery of meaningful relationships from the data.
Since Gaussian Processes naturally handle scarce, noisy data with the capacity to model complex trends and their uncertainties, they are good candidate to meet such interpretability standards in health (Tomkins
2020 Cheng et al. 2020 Ghassemi et al. 2015). As result, we focus on GP regression for modeling online health data. However, poor choice of kernel is detrimental to GP performance (Oyetunde & Liem, 2022 Stephenson et al. 2021 and current. kernel selection approaches do not meet the strict needs- adaptive complexity and interpretability of our multi-task, online health settings.
et al.
For single-task learning, compositional kernel search (CKS) methods use arithmetic combinations of simple kernels to encode structures in data (Duvenaud et al. 2013). Structural kernels selected by CKS are sparse and encode relationships that are intuitive to understand (Lloyd et al., 2014 Schulz et al. 2017). However, restarling the selection procedure from scratch risks losing the stability of the structure: the kernel included in one time step may differ in the next. Furthermore, these methods do not scale to the update cadence that is required for multi-task settings we need to repeat this expensive search every time a new user joins the app or the model for an existing user is updated. With multiple users (tasks), we can share information about the best, kernel across users. CKS can be made scalable through multi-task learning, in which a small set of representative kernels is selected on pilot users and then used for modeling new users in general deployment. However, existing multi-task approaches consider the final data sets of pilot users when identifying representative kernels (Titsias & Lázaro-Gredila, 2011). For online selection, directly transferring a complex kernel composition, learned on a pilot user with many data points, to a new user with few points leads to overfitting (see fig. la].
Our method for choosing representative kernels explicitly addresses the challenges of online, multi-task model selection in mHealth. We increase the complexity of the kernel only when demanded by data. To do so, we learn the best sequences of kernels that will balance the bias-variance trade-off for users with growing amounts of data (see fig. Ic and fig. [dp. These sequences, which we call kernel evolutions, encode the relationship between former and current kernels, which can be transferred from pilot to test users. By design, our kernel evolutions encode stability and sparsity, since they demonstrate how user compositions change from one time step to the next.
We illustrate the efficacy of our approach on synthetic examples, UCI datasets, and real health data selected to match the characteristics of data we would find in mHealth. When data arrive online, our method selects kernels that are (1) sparse, including only features that are relevant to the prediction target; (2)
2

---

Published in Transactions on Machine Learning Research (10/2023)
stable, includes the same kernel components (i.e. features) consistently across time steps; (3) of adaptive complexity, selecting kernels that manage the bias-variance trade-off.
2
Related Work
Batch Compositional Kernel Selection. Our compositional kernel selection strategy is specifically designed for online settings. That said, one way to use batch selection methods for online data is to re- run the selection algorithm on the user's cumulative data set each time new data arrives. The Automatic Statistician literature describes batch methods that search over grammar of compositions (Steinruccken) et al. 2019 Duvenaud et. al. (2013); and [loyd al. (2014) introduced Compositional Kernel Search (CKS), a greedy search over the sums and products of simpler base kernels. Recent work has made CKS more scalable by approximating the model evidence (Kim & Teh 2018) or representing the search neural network (Sun et al.) 2018). The greedy method of Duvenaud et al. (013 is most similar to our own, in that it prioritizes building the strongest kernels from the data into the composition. However, the space of compositions these batch methods must search over is already extremely large (Gardner et al. 2017 and infeasible to repeat from scratch each time new data arrives.
Online Composition Selection. Probabilistic composition selection methods (Malkomes et al.) 2016; Gardner et al. 2017 Titsias & Lázaro-Gredilla 2011 Tong et al.) 2021 Zhang al. 2019) are better suited for online data since one can use the previous posterior over kernels as the updated prior for training on new data. For example, Tong et al. (0021) uses shrinkage priors to select. a sparse subset of kernel components.
A recent subset of CKS methods attempt kernel selection specifically for non-stationary functions online data whose generating kernel changes over time. Adaptations of Duvenaud et al. (2013) use a "sliding window" on the data and a change-point detection algorithm for initializing CKS (Hüwel et al. 2021 2022 These methods are more scalable in time but not in users, as they still require many iterations of re-sampling or re-approximating the updated posterior per user. contrast, our method scales to new users by directly transferring the compositions learned from pilot users.
Finally, there are arcas of online (multi) kernel selection that are distinct from online compositional kernel selection. Some ouline moth are designed specifically for Support Vector Machines (Zhang & Liao 2018 Yang et al. 2012: Orabona et al.) 2010 Hoi et al. 2013), which lack predictive uncertainties and bypass dealing with the cubic time complexity of calculating a GP marginal likelihood. Lu et al. (2020) use random features to approximate an ensemble of stationary GPs but does not describe how to combine different base kernels (e.g. linear and periodic), sparsely select over components, or search over hyperparameters. Similarly, Zhang et al. (2019) learn mixture of (infinite) GPs that evolves over time, but the GPs that they consider have the same kernel composition and differ by the hyperparameters: they do not describe how to jointly search over compositions and their hyperparameterrs. Finally, Levi & Ullman (2099) poses model-agnostic algorithm that incrementally increases the model's capacity but does not address how to search over a large composition space. In summary, these works are missing at least one key element of compositional kernels for GPs, which are desirable for their interpretability in our setting.
Multi-task Sclection. One way to make CKS amenable to multi-task scttings by leveraging kernel similarities across tasks. Methods from Tong Choil (2019) and Titsias & Lázaro-Gredille (2011) learn set of kernel compositions multiple users that can be transferred to new users. However, these approaches risk overfitting, as they do not restrict the complexity of the composition that is being transferred.
KEM's approach of transferring kernel evolutions from a set of pilot users to a set of test users is related to the meta-learning paradigm. Classical MAML (Finn et al.) 2017) and its variants (Yoon et al. 2018 Finn 2018) require differentiable loss functions- relaxing our discrete search over compositions to a soft
ct al.
search would reduce sparsity. Similarly, con-compositional approaches from Garnelo et al. (2018): Rootfuss
et al. 2021a b) sacrifice the sparsity and stability that our additive compositions provide.
3

---

Published in Transactions on Machine Learning Research (10/2023)
3 Background
Compositional kernels. We assume that the reader is familiar with GP regression (Rasmussen & Williams, 2006p. Compositional kernels express complex functions through sums and products of simpler kernels Duvenaud et al. (2013). The sum of two functions with independent priors, J1 ~ GP(O, k1) and ʃ2 the same operations on the kernels: fi + f2 GP(O, k1 + k2). Furthermore, the product of two kernels that are defined on different dimensions of the data allows us to account for interactions between the dimensions. Compositional kernel selection uses these rules to form an expressive grammar over functions. For example, functions that can be recovered with a compositional kernel include generalized additive models (2 SEx) and automatic relevance determination (Tla SEa), where SEa is a squared exponential kernel on the d-th dimension of the data.
GP(O, k2), corresponds
Dirichlet process mixture models. A Dirichlet Process (DP) mixture model generalizes the Dirichlet distribution from being a conjugate prior over a fized number clusters to an infinite mumber of clusters et al. 2019). The generative process for data under the DP is: H DP(a, G); 0, ~ H, X,10, ~V p(X,; On), where X, the observation, On are the parameters of the distribution that generated X,, and H is distribution over the parameters (G the base distribution). The Dirichlet Process assumes I is discrete. Consequently, all X, with the same 8; f can be thought of as belonging to the same cluster. Let (0e) represent the C clusters that exist among the observations and Zn E (I,...,C) be the assignment of an observation to one of these clusters.
One can track cluster assignments under this DP using a Chinese Restaurant Process (CRP). Given the current cluster assignments of all other observations, the n-th observation is assigned cluster e with the following probability:
ifc=C+1
p(Z, cl 1n-
N Nta-I'
(1)
otherwise
where No is the number of observations currently assigned to the c-th cluster of N total observations. Under this model, the probability of starting a new cluster is determined by a. The probability of assignment to an existing cluster is proportional to the number observations already in that cluster. For more information on DPs, see Tch (2010).
4 Problem Setting
During deployment, user data arrives online- in batches over multiple time steps and want to efficiently select the "best" kernel for every user at every time step. Below, we define our kernel notation and what it moans to select the "best" kernel for a user at given point in time.
Kernel notation. We denote kernel compositions as K, kernel hyperparameters (e.g., lengthscale, period) as 0, and kernels (combination of a composition and hyperparameters) as Kg. Bold notation represents a set; for example, K set of kernel compositions. We attribute any of these entitics to a user u and/or time | through subscripts. For example, Ke,., is kernel and K,,t is set of kernel compositions for the u-th user at time t.
Kernel compositions. As in Tong et al. 2021 we define a kernel composition as weighted sum of "candidate kernels":
K(æ, Σ w,Ki(z,''),
(2)
where k; is one of 7 candidate kernels and w; > 0 is the weight.
Formalizing online model selection. We begin by defining what it means for the user's data to arrive online. Let Du, = (X..,, represent the cumulative data set of all observations for user u up to time t. We assume that the user's data is generated by a fixed, latent function f.(x) RD - R for D dimensional inputs x. We assume fa was sampled from a GP with an unknown kernel. The observed targets are corrupted by user-specific level of noise, such that y fx(x) + en for en ())
4

---

Published in Transactions on Machine Learning Research (10/2023)
Now we are prepared to define what it means to select the best kernel for user's online data. At time t, we must select a kernel composition Ka and hyperparameters ou, so that fu, R Su where fu, ~ GP(, K) Using our observations from the user up to this point Pu,,, the kernel Ku, can be selected by maximizing the marginal likelihood, or, as we will later do, by minimizing the BIC for a restricted set of compositions.
Stationarity assumptions. We make two stationarity assumptions on (1) the distributions between pilot and deployment users; and (2) the data distribution within a single user. First, we assume there is no distribution shift between pilot users and deployment users. In health, the pilot trial is designed to target the same population as in deployment. For example, an earlier and follow-up version of the HeartSteps trial targeted adults newly diagnosed with stage hypertension in the Seattle area (Ghosh et al. 2023). Second, we assume that the data-generating function for a single user, fu, does not change over time. The evolution of the "best" composition between time steps does not indicate non-stationarity, but rather, our adjustments to the bias-nariance trade-off selecting the best-fitting model for the data we currently have about a user. In health scttings, stationarity is often assumed because the noisiness and sparsity of the data make it difficult to directly model non-stationary effects online (Trella et al. 2022
5 Kernel Evolution Model (KEM)
Problem statement. During deployment, we must select kernel compositions for new users in a manner that is of appropriate complexity to the currently available data, and interpretable- sparse and stable- in the components that are included.
Our solution. We propose the Kernel Evolution Model (KEM), Bayesian non-parametric model that describes how a user's kernel evolves as a function of observing additional data. Our solution happens in two stages, pilot training and deployment testing, which we describe in section 5.1 We use KEM to find a set of kernel evolutions offline (pilot training, section 5.3), which enables us to efficiently select kernel compositions online (deployment testing, section 5.4 in a sparse and stable manner.
5.1 Pilot training vs. deployment testing.
Example timeline. KEM learns how to select kernels during a long pilot training phase in order to efficiently select kernels for new users during deployment. In pilot training, we have the time and resources to perform any necessary search operations over compositions. For example, there was over one year between the pilot trial of HeartSteps V1, a micro-randomizze trial for mobile health activity notifications (Klasnja et al. 2019)). and HcartSteps V2 (Liao et al.) 2020). In contrast, during deployment testing, new usors and their data arrive ouline, and we must elficientl select kernels for these users with initially limited data. For example, the kernel selection must occur daily (Liao et al. (2220) or weekly (Trella et al., (022), in a stable manner, for all users. We do not re-learn the set of kernel evolutions between the arrival of test users, so that kernel selection remains stable during deployment.
Pilot training. Before the main study, we use data from a pilot study to identify sequences of kernel compositions which we call kernel evolutions that manage the bias variance trade-off as the amount of data increases, per user, for different users. To do so, we process the training users' data incrementally, as though it were arriving "online." " For example, for some users, a linear kernel might best model the initial data, but at later time, a lincar + periodic may be best; for others, a squared-cxponential kernel may always be preferred. In section 5.2 and section 5.3 we detail the pilot training process that identifies a set of kernel evolutions- with emphasis on sparsity and stability- among the training users.
Deployment testing. During deployment, we use the previously Icarned evolutions to omit compositions that are unlikely to fit,, too complicated, not sparse, and not stable. In the above example, if a lest user's data is currently modeled by a linear kernel, given new data we might consider either a linear or a -perooo kernel, but not try a squarod-cxponential. (Note: while the selection model helps us efficiently select a kernel composition for a new user, the hyperporameters of the composition must still be optimized to that test user). In section 5.4 we detail how evolutions are used to select kernels for a new test user.
5

---

Published in Transactions on Machine Learning Research (10/2023)
White
LIND
LINO
Figure 2: Example visualization of sclection model learned by KEM synthetic data. This directed graph is one way to represent the Kparent + Kehila relationships that are learned in pilot training, Fach node in the graph is DP. The arrows leaving from each node represent the "clusters" found at this DP. It is possible for two separate DP to identify clusters with the same composition; this is why SEO and LINO both have arrows leading to LINO + PERO.
5.2 KEM: A Generative Process for Online Data
Modeling evolutions of kernel compositions with Dirichlet Process. For u, model the evolution of single composition from Ku,t-1 to Ku,t with a DP mixture model. Generically, refer to an evolution transition of the form K parrent + Kehiaa- Since the distribution over Kehia is only conditioned on its immediate predecessor K Kparent, our process is Markovian- not hierarchical- in how it represents evolutions over time.
Why a micture model? We will learn the set of evolutions Kparent - Kehild by observing the evolutions that occurred among all our pilot users. It possible that different pilot users, who started with the same K parent: roquired different Kehild composition. Each cluster in mixture model corrosponds to unique composition for Kehild, and each cluster is populated with instances of the given evolution occurring among the pilot users. Why DP? Using DP allows us to avoid having to pre-specify the number of clusters- the number of unique compositions (Kehild) per Kparent that will exist.
Concretely, for every kernel composition Kparent, there is a DP over the composition of Kehit:
KK (Kehita ~ DP(a, Gparent K parent))
(3)
where Hygaren defines a distribution over the next composition lcaving from Kparent- Gparen is the base distril distribution (whose support is over kernel compositions) and the concentration parameter of the DP.
We will now discuss how our choice of Gparent encourages sparsity and stability in the evolution from K parent to Kehild Recall from eq. (2) that kernel composition is weighted over candidate kernels, where is the weight on the i-th candidate kernel. Gparent uses spike-and-slab distribution for the weights: a33;, ~ Bernoull(z;)), af pp(a)). The "spike" is represented by Bernoulli(T;), and determines whether or not the candidate kernel is included in the composition. The "slab" is represented by p(a²) and is a prior on the weights themselves (note that this is equivalent to a prior on the kernel's amplitude hyperparameter). Our experiment settings for the DP parameters are given in appendix 10.2.5
For sparsily, the spike-and-slab distribution encourages w; to be zero, which induces sparsity in the total number of candidate kernels included. Smaller values of n, encourage greater sparsity (since w, = 0f 0). For stability, when Gparent is conditioned on K pareett: it places a higher probability of including the same candidate kernels as K Kpareut- Specifically, if candidate kernel j was in the previous composition, then we set its probability ny close to 1. Since the same candidate kernels are more likely to be included from one time step to the next, the evolutions will be more stable.
Modeling online data generated by kernel evolutions. Now that we have defined the evolution of composition between time steps, are prepared to define how Drt t is generated a result of such

---

Published in Transactions on Machine Learning Research (10/2023)
evolutions:
Ku,t ~ HK
(4)
ou. plo ku,-1)
(5)
0utt ~ p(0u.1 Ku,))
(6)
fu, GP(O, Kow,)
Yu, ((fa,e.0,.11
Eq. 4 The kernel composition at time step t is sampled from the distribution sitions defined by the preceding composition at t - The DP from eq. (3) allows us to transfer information about the likeliest next composition (evolution) between users with the same parent composi- tion. As discussed, the base distribution of the DP also encourages stability; a user has a "linear" kernel one time step, they are likely to build from the "linear" kernel at the next time step.
compo-
Eq. and The prior over byperparameters depends on the composition. If a composition increases in complexity from one time step to the next, then the corresponding observation noise should decrease, since the data-generating function is stationary. To encode this behavior, prior on the noise of is decrcasing function of the number of components. Our prior over the remaining kernel hyperparameters helps separale the model classes. For example, a squared-exponential kernel with a large lengthscale maps to functions that behave linearly; a prior on the lengthscale would minimize overlap between the Linear and SE kernels. See appendix 10.2.5 for the priors used in experiments.
Eq. 7 and eq. (8): Each user's function is sampled from Gaussian Process. The GP is defined by the kernel and the final observations overlap in data with Yu,t-1, we use different generative processes for each by noting that the bias variance trade-off, and therefore the best composition, may differ between the two overlapping data sets.
corrupted by Gaussian noise. Though in our model Y., will
5.3 Pilot Training: Offline Inference for KEM
We now prepared to infer the set of kernel evolutions that generated the pilot user's "online" data. We represent each DP Chinese Restaurant Process (CRP) to track the assignments of pilot data to clusters. We will describe inference for the parameters of a single CRP (conditioned on K parent composition). This inference jointly occurs for each unique Kparent composition.
We must infer the set of Kenna clusters and the assigmments of pilot datasets (datasets for each user, at each time step) to these clusters. The unknown parameters are = Zz K, 0, a²). Assuming there are N datasets and C clusters in the -pa CRP, the cluster assignment of the n-th dataset is given by Zn E {l...., C). Each cluster represents a kernel. The c-th kernel is defined composition Ke, hyperparameters oz, and observation noise o²? Inference of the unknown parameters involves sampling from the joint posterior, p(Z, K, 0, (D)), where D = (Da,e} is the set of all training user's data (separated by time step so that it is processed as though it is "online"). Each posterior sample results in a selection model like the one shown in fig. 2 Following the marginal Gibbs sampler from Gelman et al. (2013), we alternate between the following two steps (with further detail in appendix 10.3):
1. Assigning datasets to clusters. The goal is sample from the posterior p( 1Z| 02, single dataset Du,t, sample from multinomial posterior defined by the likelihood that the kernel within the cluster generated Dr,t and the prior probability the cluster assignment in eq. 1
For a
2. Assigning kernels to clusters. The goal is to sample from the posterior p(K,0, D). Each dataset in the cluster is treated as a sample from a GP with the same kernel, and we use this data to select kernel to represent the cluster. is possible for two different clusters to have the same composition but different hyperparameters. We obtain samples from the posterior distribution of the kernel via the Metropolis- Hastiing algorithm (appendix 10.3.2
7

---

Published in Transactions on Machine Learning Research (10/2023)
5.4 Deployment testing: Using KEM for New Users Online
Up to this point, we have described how to infer set of kernel evolutions defined by the generalive model section 5.2 by applying the inference procedure in section 5.3 on pilot data. In algorithm [ we define a
sclection model that leverages these learned cvolutions to select a kernel for a new test user u at time t.
Algorithm Selection method for KEM
Input Test user's current data Dur,t ( preceding composition K Output kernel with composition K and hyperparameters 0x-
Define set of potential compositions K {K K (-1) (- for K K do
Initialize hyperparameters 0g to pilot user's hyperparameters Optimize 0K = max log p(yua, Xx 0x) Pk
end
Select K = min BIC(K;0x) KEK*
Return K", K',0k-
Our method minimizes the Bayesian Information Criterion (BIC): BIC(K)) = [0(ln(n) - 2ln(Lg), where Lg is the log marginal likelihood of the data, for GP with composition K and MLE hyperparameters The BIC is common model selection metric that penalizes hyperparameter complexity (Duvenaud et al. 2013 Kim & Teh 2018]; though it can be replaced with any model selection metric, KEM is still needed in addition, to restrict the set of compositions to those that are stable and sparse for online selection. KEM restricts K", the set of compositions considered, to those that were identified for pilot users with the same preceding composition. We also allow no evolution to occur, by including the previous composition K in the set. When l 1, K* initialized to those that were found at / = 1 for pilot users. Conceptually, our approach corresponds to "traversing" the graph in fig. 2 to select kernel; the test user is currently at node K 1-1 and K contains the children of the current node.
6 Experimental Setup
We evaluate a method's ability to select compositions that are sparse (includes only relevant features), stable (the same fcatures are consistently included across time steps), and of adaptive complerity (the kernels perform well across different data sizes). Furthermore, any method for online deployment must be sufficiently scalable to multiple users and time steps. All empirical experiments over 10 independent trials.
6.1 Baselines
KEM has three essential aspects, and each of our baselines omits least one of these aspects: KEM learns a model selection strategy from pilot users (pilot training), learns the kernel compositions at different time points (adaptive complexity), and models composition evolutions between time points (stable). The Memoryless method (as in [Tong et al. 2021), but with no variational GP approximations) does not have a pilot training phase and always requires a new round of selection in deployment. The ARD method represents using a GP with a fixed kernel composition- the ARD kernel- throughout the deployment, with kernel re-selection. The ARD kernel is composition of SE kernels applied to each dimension d of
D dimensional data set (Rasmussen & Williams, 2006 II SE.. Note that the lengthscales of ARD must
still be optimized to new data. The Final [Titsias & Lázaro-Gredill 2011 method transfers knowledge obtained on the final data set of pilot users, ignoring the potential to overfit when there are fewer data points for the Lest user. Finally, Stratified is an ablation of our approach that only transfers kernels from pilot to test users if they were found at the same time step. It still has our innovation suggesting kernels based on how much data has been collected but does not enforce stability, since it treats kernels subsoquent time steps independently. Overall, the methods with pilot training (Final, Stratified, KEM) transfer the kernel compositions, but the kernel huperparrmeterss are re-optimized to the new user's data during selection.
8

---

Published in Transactions on Machine Learning Research (10/2023)
6.2 Candidate Kernels
The set of candidate kernels contains "atomic kernels," " which are combinations of three commonly-used kernel functions (Tong et al.) 2021; Duvenaud et al. 2013) linear (LIN), periodic (PER), and squared- exponential (SE) applied to single dimension of the data. For example, LINO denotes a linear kernel function that operates on the 0-th dimension of the data. WHITE means no kernel is selected. The candidate kernel set also includes first-order interactions between these "atomic kernels". For example LIN1 LIN1 or LINI x SE1. Finally, the ARD kernel is included, so that the selection methods are comparable to the SE-ARD method.
6.3
Overview of Datasets
Data set
# instances
# features
# users
# instances user
# time sleps
Synthetic
1800
60 (10 train/50 550 test)
30
6
UCI: Energy
768
8
15 (7 train/8 test)
50
10
UCI: Housing
506
13
10 (5 Irain/5 test)
50
10
UCI: Fires
244
15
(4 train/4 test)
30
10
UCI: Concrete
1030
8
20 (10 train/10 /10 test)
50
10
MIMIC-III
975
14
16 train/8 test)
varies
varies
HeartSteps 1535
23
37 (18 train/19 test)
varies
varies
Table 1: Descriptions of datasots. For MIMIC and HeartSteps, different users had different available data sizes, due to missingness in the data different lengths of stay in the ICU.
The datasets used in our experiments, reflected in table 1 have different properties. By design, the users in the UCI data sets are homogeneous, because they were artificially constructed by randomly splitting the data. The MIMIC, HeartSteps, and Synthelic data are expected to have more heterogeneity- "clusters" of users for whom different compositions are more appropriate. In terms of noise, the Syuthetic (by design) and UCL: Fires (a simpler prediction task) data are relatively low noise compared to the remaining data sets. We are particularly interested in the performance of the kernel selection methods on the high noise, high heterogeneity data- HeartSteps and MIMIC-III- since these characteristics reflect our health setting.
7 Results
In section 7.1 we begin with a pedagogical comparison of the methods on synthotic data and then demon- strate that the same takeaways hold on real data sets in section 7.2
7.1 Demonstrative Results on Synthetic Data
We will begin by examining what the selection models learned during pilot training and link them to per- formance on selecting kernels for new users in testing. We find that KEM's ability to identify simpler, intermediary compositions during pilot training is crucial to performance in low-data regimes.
Synthetic data. We constructed a 1-D data set that contains two "types" of users. For half of the users, we sampled cach function from a GP prior with LINO + PERO kernel. The other half's functions were sampled from SEO kernel. Each user's data arrives in batches of 5 data points over 6 total time steps (for a total of 30 data points in the end). For test user u", the train set is the cumulative data Du observed the user up to time t. The test set is composed of 200 uniformly spaced points along the x-axis from [0, 20).
Sparsity and stability: KEM tells a cohesive story about how kernels evolve with more data. In fig. 3 we show an example of the kernel compositions learned by the Final, Stratified, and KEM methods during pilot training; the ARD and Memoryless methods do not have pilot phase. As expected, Stratified and KEM learn simpler, intermediary kernels account for varying time steps, while Final does not. However, Stratified chooses kernels that lack congruence over time, since it does not share information about
9

---

Published in Transactions on Machine Learning Research (10/2023)
(PERO PERO LIND
1-2
LINO
143
SEP
LINO PERO
LINO PERD
SDD
LINO
1-4
SF3
LINE + PERO
(a) Final
t-5
SED
LINO + PERO
LINO - PERO
SBO
1
SE0 x PERO
LINO SDD
(c) KEM (ours)
(b) Stratified (our ab.)
Figure 3: During pilot training, KEM learns evolutions in which simple kernels consistently precede complex kernels, whereas baselines do not. Final identifies two clusters, the two final compositions (as expected). Stratified randomly adds and drops components, such as when goes from a PERO x PERO LINO composition to LINO PERO composition from t to t = 2.
the composition across time steps. On the other hand, KEM tells a clear story about how the data is initially best fit by a LINO kernel and subsequently evolves into either a LINO+PERO or SEO kernel (the true kernels). These relationships were enforcod by our modcl's definition of an "evolution."
Test likeli. for Synthetic
Selection runtimes
Test Log Likeli.
Log runtime
Memoryless
-50
10'
ARD
Strat. (our ab.)
Final
5
15
20
30
5
15
30
Num. data points
Num. data points
KEM (ours)
(a) Test likelihoods
(b) Runtimes
Figure 4: KEM quickly selects high-performing kernels (log-likelihod)) for new users across varying amounts of synthetic data, by successfully managing the bias-variance trade-off. Left: log-likelihood. Right: runtime in log scale. Error bars: two standard deviations. Memoryless is included, but overlapping, with KEM.
Feasibility: The pre-trained selection methods select. kernel compositions at a rate that is feasible for online learning. In fig. 4b the methods with pilot training phase are orders of magnitude faster at selecting kernel than Memoryless, which must re-perform kernel selection for each new user and time step. ARD is quickest because it performs no selection. Though one might update the kernel on weekly or monthly time scale in a health setting, the runtime required for Memoryless would grow rapidly with the number of users. Alternatively, the computation cost for the pre-trained methods is primarily spent optimizing the hyperparameters for a small set of kernels (on the order of or 3 comparisons for new data).
Adaptive complexity: KEM manages the bias-variance trade-off by regularizing the kernel composition in low data regimes. We expect method that manages the bias-variance trade-off to have a high test likelihood when there is little data and after more data has been observed. In fig, 4 we see evidence of KEM managing the bias-variande trade-off by maintaining the best test performance across time steps. The mechanism through which KEM manages this trade-off is by regularizing the kernel composition when there is little data. In fig. 5 KEM initially selects simpler compositions LINO and White- for new users. KEM eventually selects more complicated compositions- LINO + PERO, LINO + SEO, and SE0- as necessitated by the test user's data. KEM's use of simpler compositions when data size low reduces the chances of overfitting the hyperparameters. Memoryless requires more data from the test user identify the true composition and underfits the data.
10

---

Published in Transactions on Machine Learning Research (10/2023)
Test compositions selected w/5 data points
Test compositions selected w/30 data points
68
LINO+PERO
LINO
LINO-SEO
SEO
White
LINO-PERO
LINO
LINO-SEO
SEO
White
KEM (ours)
Strat. (our ab.)
Final
Memoryless
ARD
Figure 5: KEM chooses simpler kernels where there are data points and more complex kernels (including ground truth) when there are 30 data points. We display distributions over 5 most common compositions selected across all 10 trials when the ground truth is LINO PERO. ARD method is SEO kernel by default.
On the other hand, we see evidence that the more complicated compositions initially selected by the baseline
methods (e.g. a LINO + PERO kernel) lead overfitting hyperparameters (fig. 9 Fig. that overfitting in low data regimes results in poor test likelihoods.
demonstrates
7.2 Generalization to Real Data
In this section, verify that the behaviors of the selection methods from section 7.1 regarding sparsity, stability, and adaptive complexity hold on more complex data. We omitted further experiments with Mem- oryless due to impracticality with respect to computation time in online settings. For test user u", the train set is the cumulative data Du-, observed on the user up to time I. The lest set is the incoming data Duur,e. This procedure mimics the way that models are used practice; a kernel selected with the user's cumulative data and then used make predictions on incoming data, until the next time step, when the kernel must be selected again.
from the next time step: Da ,11 ,tt,
UCI data. We include four UCI regression tasks: Energy (Taaas & Xifara 2012). Concrete Yeh 1998
Boston Housing (Harrison Jr & Rubinfeld 1978), and Fires [Abid & jen 2019). We randomly split these batch, Sngle-task: datasets into "users" " and further into "batches" of points each to reffect our setting.
MIMIC-III: Predicting plateau pressure. We use the Medical Information Mart for Intensive Care (MIMIC-III) data set Johnson et al. (0166), which contains patient's physiological readings from the ICU, The regression goal is to predict plateau pressure the amount of pressure applied to the airways during mechanical ventilation- from the patient's other vitals (e.g. heart rate). We grouped patients into 16 tasks via their diagnosis upon admission, such as sepsis, and assumed data arrive batches of 10 points.
mHealth: Imputing missing wearable data. HcartSteps V1 (Klasnja et al. 2019 was an mHealth study that helped sedentary individuals increase physical activity (ie number of active minutes) through contextually-tailore interventions. In mHealth, a major source of missing data is when users forget to wear their tracking devices (Secwald et al., 2019). Our goal is to actively impute each participant's missing values of "daily active minutes" (square-root transformed). from other data sources (e.g. location data). We assume that the data from 37 users arrive in batches of 5 points.
Adaptive complexity. We expect KEM to have high test-likelihood for varying levels of data. In fig, patterns from the synthetic experiments hold; KEM maintains high likelihood, while Final consistently overfits when there are small amounts of data. We believe ARD performs worse on real data because it has more opportunity overfit the lengthscales the multi-dimensional datasets. All methods perform similarly Fires, simpler regression task with low noise.
Stability. On synthetic data, we noticed Stratified chose kernels that are inconsistent across time steps, and we demonstrate that this behavior is exacerbated on real data in table 2
Sparsity. In fig. 7 we provide a basic interpretation of the evolutions that KEM learned on the health data. This is not replace domain expert but demonstrate that KEM selects sparse compositions that are easy to examine. In contrast, the composition implied by ARD not sparse and is difficult to use in relating the features to the predictions.
11

---

Published in Transactions on Machine Learning Research (10/2023)
+
ARD
Strat. (our ab.)
Final
KEM (ours)
Test likeli. for UCI: Energy
Test likeli. for UCI: Housing
Test likeli. for UCE Fires
Test Log Likeli.
Log Likeli.
Test Log Likeli.
0.0
-2.5
20
10
30
50
10
Num. data points
Num. data points
Num. data points
Test likeli. for UCI: Concrete
Test likeli. for MIMIC
Test likeli. for Heartsteps
Test Log Likeli.
Test Log Likeli.
Test Test Log Likeli.
-2
-5
10 5
15
20
25
30
35
40
45
50
20
40
60
5
20
25
35
40
45
55
Num. data points
Num. data points
Num. data points
Figure 6: KEM is the only kernel selection method to consistently perform well for all levels datasets (including real health settings). All methods perform similarly on the simplest of the data (fires). Error bars are 95% confidence intervals.
data
Datasel
UCI: Energy
UCI: Housing
UCI: Fires
UCI:Concrete
MIMIC
Heart.Seps
KEM (ours)
0.08 + 0.05
0.07 + 0.04
0.29 + 0.17
0.06 + 0.04
0.07 + 0.1
0.22 + 0.05
Strat. (our ab.) 1.09 + 0.14
0.73 = 0.3
0.56 + 0.2
0.46 0.14
0.22 0.12
0.62 = 0.15
Table 2: KEM consistently includes the same features across time steps. Table shows avg. # of features included in kernel at one time step but dropped the next for each test user (lower is better). We report 95% confidence intervals.
For the MTMIC application of predicting plateau pressure, KEM prioritizes the "peak inspiratory pressure" feature earlier in the evolution. Plateau pressure is directly related to the peak pressure through the resistance of airflow the lungs. Features farther down the evolution, such as inspired oxygen and respiratory rate, also relate to passive and mechanical ventilation (Hagberg Fasa 2022). For imputing missing wearable values HcartSteps, we expected "GoogleFit (daily) steps" to be the most predictive feature. Interestingly, some evolutions lead to a linear kernel the "day in study," " which we hypothesize because some decrease activity throughout the course of study; this trend may only be apparent enough days (and data points) are observed for user.
SEjsys, blod pressure] SEjurine
WHTTE
WHITE
x SE/menn blood pressurel
SElplateau pressure]
SElpeal pressure)
SE SElgoogle fit steps) LINigoogle fit steps]
...
x SEffrac. insp oxygen] ICU]
ENNrrsps ratej
PER|frac. insp. oxygun] SElpeak insp- pressure)
LINistudy dayl
SP[google fir steps)
(a) ARD on MIMIC
(c) KEM on Heartsteps
(b) KEM on MIMIC
Figure 7: KEM selects sparse kernel compositions that are easily examined. Examples of the selection model learned by KEM on MIMIC and HeartSteps shown. ARD "selects" a composition formed by multiplying an SE kernel for each feature; remaining 8 14 features omitted due to space.
12

---

Published in Transactions on Machine Learning Research (10/2023)
Runtime per itcration (15 base kemels))
Runtime per iteration
Runtime per itcration (60 base kernels)
10 (30 base kernels)
Hours
2
Hours
2
Um6
Hours
U-6
U-12
U-12
U-12
U-30
U-30
U U030
0
20
40
20
40
20
40
T
T
T
Runtime per iteration D=5 (15 base kemelss
Runtime per iteration
Runtime per iteration = (60 basc kernels)
= 10 (30 base kemcls)
T-20
T-5
T-20
T-20
2
2
Hours
10
T-50
Hours
T-10
T:50
Hours
10
T T-50
0
0
0
10
20
30
10
20
30
10
20
30
U
Figure 8: KEM train times are most affected by the number of time steps. In top row, KEM scales exponentially for increasing T. In bottom row, KEM scales linearly for increasing M. The runtime per iteration does not scale with number of dimensions (each column).
7.3 Scalability of KEM in pilot training
Our experiments have demonstrated the benefits of using KEM during online deployment to select sparse and adaptive kernel compositions. Though by design, KEM achieves these benefits online by frontloading the computational cost of kernel selection to a long pilot training phase, for practical consideration, we explore how pilot training scales with the size of the pilot data.
Experimental outline. Recall that our pilot training for KEM involves Gibbs sampling. The dependent variable is the runtime per sample drawn from the posterior. The independent variables relate to the size of the pilot data and include the number of users U, the number of time steps T, and the number of input dimensions D. We constructed the pilot data as D-dimensional data set that contains three "types" of users LINO + PERO composition, +INO+ PER1 composition, and a SE1 composition and there are the same number of users per type. Each user's data arrives in batches of 5 data points over T total time steps. We train on each pilot dataset for 100 posterior samples, though in practice the total number of samples required for convergence will vary. The reported runtimes are on a 4 core Intel "Cascade Lake" CPU.
KEM training scales better to more users (linearly) than to more time steps (exponentially). Figure X indicates that in general, KEM is most suitable for applications where the pilot data has a large number of users but a small number of time steps. In our experiments on performance during deployment, we demonstrated that KEM is useful for managing the bias-variance trade-off in earlier time steps. Therefore, even when trained pilot data with small T, KEM can achieve its purpose of appropriately adapting the complexity of the kernel early on. Note that it is also possible to form the time steps in the pilot data with irregular batch sizes (that is, smaller batch sizes initially, and then larger batch sizes later on).
The runtime per posterior sample is not affected by the dimensionality of the training data. This is due to the fact that our sampling scheme searches over kernel compositions by randomly sampling subsets of the base kernels. These operations do not scale with the size of the basc kernel sct; however, larger number of base keruels will make this sampling-based search more difficult, and therefore, require a larger number of samples overall.
8 Discussion
Limitations and Future Work
13

---

Published in Transactions on Machine Learning Research (10/2023)
Scalability of KEM in pilot training. KEM scales poorly with more pilot users and time steps, since require repeated GP marginal likelihood per user and time step. However, in our setting, this is manageable one- time cost incurred during the long pilot period. Our only goal for pilot training is that it enables us to perform online kernel selection for a large number of users in the real deployment (e.g. in fig. 4b only 10 seconds per time step, on average, is required). In future work, we are excited to incorporate advancements in scalable GPs, such as (Kim & Teh 2018), to roduce the cost of the marginal likelihood calculations.
Limited interaction terms. The number of interaction terms considered in this approach is restricted to those in the candidate kernel set. This is acceptable in health, where we do not expect to find higher-order effects due to noise and sparse data (Trella et al., 2022). In future work, we may consider actively adapting the candidate set to include more interactions.
Rigidity of the kernel evolutions from pilot to test. Though KEM's strategy, which limits the space of kernels considered for test users, is exactly what allows kernel selection to occur scalably in our sctting, it also means that lest. users are limited to the compositions found on training users. An interesting direction for future work is to use the set of kernel evolutions from pilot training a prior for new users, rather than hard constraint. The task of adapting the kernel evolution set online remains challenge.
Conclusion In this work, we presented a method for compositional kernel selection in the online, multitask setting. We illustrated that learning "evolutions" of kernel compositions is most beneficial in the low- volume and/or heterogeneous user regime, when overfitting poses a significant concern. We demonstrated across variety of data sets, including two health applications, that our approach quickly selects kernels that outperform baselines in predictive performance, sparsity, and stability all crucial considerations for real-world deployment.
9 Acknowledgements
This material is based upon work supported by the National Science Foundation under Grant No. IIS- 1750358 and the Graduate Research Fellowship Program under Grant No. DGE 2140743. Any opinions, findings, and conchusions rocommendations exprossed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under award nunber OD P41EB028242. ES's work on the project was supported by gift fund from Benshi.ai.
References
Faroudja Abid and Nouma Izeboudjen. Predicting forest fire in algeria using data mining techniques: Case study of the decision tree algorithm. In International Conference on Advonced Intelligent Systems for Suslainable Development, pp. 363 370. Springer, 2019.
Joseph A Calazzo, Mark Casselman, Nathaniel Hamming, Debra K Katzman, and Mark R Palmert. Design of an mhealth app for the self- management of adolescent type diabetes: pilot study. Journal of medical Internet research, 14(3),22058 2012.
Li-Fang Cheng, Bianca Dumitrascu, Gregory Darnell, Corey Chivers, Michael Draugelis, Kai Li, and Bar- bara E Engelhardt. Sparse multi-output gaussian processes for online medical time series prediction. BMC medical informatics and decision making, 1011):1-23, 2020.
JiWon Choi, Ji Hyeon Lee, Eric Vittinghoff, and Yoshimi Fukuoka. mhealth physical activity intervention: a randomized pilot study in physically inactive pregnant women. Maternal and child health journal, 20: 1091-1101, 2016.
David Duvenaud, James Lloyd, Roger Grosse, Joshua Tenenbaum, and Ghahramani Zoubin. Structure discovery in nonparametric regression through compositional kernel search. In International Conference on Machine Learning, pp. 1166-1174. PMLR, 2013.
14

---

Published in Transactions on Machine Learning Research (10/2023)
Chelsea Finn, Picter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pp- 1136-1135. PMLR, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. arXiv preprint w:Xi:1886.028111 2018.
Jacob Gardner, Chuan Guo, Kilian Weinberger, Roman Garnelt, and Roger Grosse. Discovering and exploit- ing additive structure for bayesian optimization. In Artificial Intelligence and Statistics, pp. 1311-1319. PMLR, 2017.
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint wrXiv:1807.01622 2018.
Andrew Gelman, John B Carlin, Hal S Stern, David B Dunson, Aki Vchtari, and Donald B Rubin. Bayesian dala analysis. CRC press, 2013.
Marzyeh Ghassemi, Marco Pimentel, Tristan Naumann, Thomas Brennan, David Clifton, Peter Szolovits, and Mengling Feng. A multivariate timeseries modeling approach to severity of illness assessment and forecasting in icu with sparse, heterogeneous clinical data. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015.
Susobhan Ghosh, Raphael Kim, Prasidh Chhabria, Raaz Dwivedi, Predrag Klasjna, Peng Liao, Kelly Zhang, and Susan Murphy. Did we personalize? assessing personalization by an online reinforcement learning algorithm using resampling. arxie preprint arXiv:2304.054 003665 2023.
Carin A Hagberg and MD Fasa. Benumof and Hagberg's 's airway manogement. Elsevier Health Sciences, 2022.
David Harrison Jr and Daniel L Rubinfeld. Hedonic housing prices and the demand for clean air. Journal of environmental economics and management, 5(1):81-102, 1978.
Steven CH Hoi, Rong Jin, Peilin Zhao, and Tianbao Yang. Online multiple kernel classification. Machine learning, 90(2))280-314 316, 2013.
Jan David Hüwel, Fabian Berns, and Christian Beecks. Automated kernel search for gaussian processes on data streams. In 2021 IEEE International Conference on Big Data (Big Data), pp. 3584-3588. IEEE, 2021.
Jan David Hüwel, Florian Haselbeck, Dominik G Grimm, and Christian Beecks. Dynamically sclf-adjusting gaussian processes for data stream modelling. In German Conference on Arlificial Intelligence (Künslliche Intelligenz), pp. 114. Springer, 2022.
Alistair EW Johnson, Tom J Pollard, Ta Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, freely accessible critical care database. Scientific data, 3(1):1 9, 2016.
Hyunjik Kim and Yee Whye Teh. Scaling up the automatic statistician: Scalable structure discovery using gaussian processes. In International Conference on Artificial Intelligence and Statistics, pp. 575-584 PMLR, 2018
Predrag Klasnja, Shawna Smith, Nicholas J Seewald, Andy Lee, Kelly Hall, Brook Luers, Eric B Hekler, and Susan A Murphy. Efficacy of contextually tailored suggestions for physical activity: ranro-zrdd optimization trial of heartsteps. Annals of Behavioral Medicine, 53(6):573-582, 2019.
Hee Yun Lee, Joseph S Koopmeiners, Jennifer McHugh, Victoria H Raveis, and Jasjit Ahluwalia. mhealth pilot study: text messaging intervention to promote hpv vaccination. American journal of health behavior, 40(1):67-70) 76, 2016.
Dan Levi and Shimon Ullman. Learning model complexity in an online environment. In 2009 Canadian Conference on Computer and Robot Vision, pp. 260-267. TEEE, 2009.
15

---

Published in Transactions on Machine Learning Research (10/2023)
Yuelin Li, Elizabeth Schofield, and Mithat Gönen. A tutorial on dirichlet process mixture modeling. Journal of mathematical psychology, 91:128-148 144, 2019.
Peng Liao, Kristjan Greenewald, Predrag Klasnja, and Susan Murphy. Personalized heartsteps: A reinforce- ment learning algorithm for optimizing physical activity. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(1):1 22, 2020.
James Lloyd, David Duvenaud, Roger Grosse, Joshua Tenenbaum, and Zoubin Ghahramani. Automatic construction and natural-language description of nomparameuric regression models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
Qin Tu, Georgios Karanikolas, Yanning Shen, and Georgios B Giannakis. Ensemble gaussian processes with spectral features for online interactive learning with scalability, In International Conference on Artificial Intelligence and Statistics, pp. 1910-1920. PMLR, 2020.
Mi Luo, Fei Chen, Pengxiang Cheng, Zhenhua Dong, Xiugiang He, Jiashi Feng, and Zhenguo Li. Metase- lector: Meta-ngarning for recommendation with user-level adaptive model selection. In Proceedings of The Web Conference 2020, pp. 2507 2513, 2020.
Gustavo Malkomes, Chip Schaff, and Roman Garnett. Bayesian optimization for automated model selection. In Workshop on Aulomatic Machine Learning, pp. 41 47. PMLR, 2016.
Francesco Orabona, Luo Jie, and Barbara Caputo. Online-batch strongly convex multi kernel learning. In 2010 IEFE compuler sociely conference on computer vision and pallern recognilion, pp. 787 794. IEEE, 2010.
Kehinde Sikirulai Oyetunde and Rhea P Liem. Navigating kernel selections in kernel-based methods: The issues and possible solutions. In ALAA SCITECH 2022 Forum, pp. 0507, 2022.
Carl Edward Rasmussen and Christopher K. I. Williams. Goussian processes for machine learning. Adaptive computation and machine learning. MIT Press, 2006. ISBN 026218253X.
Jonas Rothfuss, Vincent Fortuin, Martin Josifoski, and Andreas Krause. Pacoh: Bayes-optimal mota-lcarning with pac-guarantees. In International Conference on Machine Learning, pp. 9116 9126. PMLR, 2021a.
Jonas Rothfuss, Dominique Heyn, Andreas Krause, et al. Meta-learning reliable priors in the function space. Advances in Newral Information Processing Systems, 34:280-2000 293, 2021b.
Wafaa Sayed, Mostafa Gamal, Moemen Abdelrazek, and Samah Tl-Tantawy. Towards a learning style and knowledge level-basod adaptive personalized platform for an effective and advanced lcarning for school students. In Recend Advances in Engineering Malhemalics and Physics, pp. 261 273. Springer, 2020.
Eric Schulz, Joshua B Tenenbaum, David Duvenaud, Maarten Speekenbrink, and Samuel J Gershman. Compositional inductive biases in function learning. Cognilive psychology, 99:44 79, 2017.
Nicholas Seewald, Shawna Smith, Andy Jinseok Lee, Predrag Klasnja, and Susan A Murphy, Practical considerations for data collection and management in mobile health micro-randomized trials. Statistics in biosciences, 11(2):355 370, 2019.
Christian Steinruccken, Emma Smith, David Janz, James Lloyd, and Zoubin Ghahramani. The automatic statistician. In Aulomated Machine Learning, pp.
Springer, Cham, 2019.
William T Stephenson, Soumya Ghosh, Tin D Nguyen, Mikhail Yurochkin, Sameer K Deshpande, and Tamara Broderick. Measuring the sensitivity of gaussian processes to kernel choice. arXiv preprint arXiv:2106.06311 2021.
Shengyang Sun, Guodong Zhang, Chaoqi Wang, Wenyuan Zeng, Jiaman Li, and Roger Grosse. Differentiable compositional kernel learning for gaussian processes. In International Conference on Machine Learning. pp. 4828 4837. PMIRR 2018.
16

---

Published in Transactions on Machine Learning Research (10/2023)
Yee Whye Teh. Dirichlet process., 2010.
Michalis Titsias and Miguel Lázaro-Gredilla, Spike and slab variational inference for multi-task and multiple kernel learning. Advances in neural information processing systems, 24:2339 14:2339-2347, 2011.
Sabina Tomkins, Peng Liao, Predrag Klasnja, Serena Yeung, and Susan Murphy. Rapidly personalizing mobile health treatment policies with limited data. ar preprint orXi::2002.0999 2020.
Anh Tong and Jaesik Choi. Discovering latent covariance structures for multiple time series. In International Conference on Machine Learning, pp. 6285-6294. PMT.R, 2019.
Anh Tong, Toan Tran, Hung Bui, and Jaesik Choi. Learning compositional sparse gaussian processes with a shrinkage prior. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9906 9914, 2021.
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shami Vivek Shetty, Finale Doshi-Velez, and Susan A. Mur- phy. Designing reinforcement learning algorith for digital interventions: Pre-implerrentatiio guidelines. Algorithms, 15(8):255, Jul 2022. ISSN 1999-4893.
Athanasios Tsanas and Angeliki Xifara. Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy and buildings, 49:560-561 2012.
Tianbao Yang, Mehrdad Mahdavi, Rong Jin, Jinfeng Yi, and Steven Hoi. Online kernel selection: Algorithms and evaluations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, pp. 1197- 1203, 2012.
I-C Yeh. Modeling of strength of high- high-performance concrete using artificial neural networks. Cement and Concrete research, (8(1)):1797-1800 1998.
Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian model-agnostic meta-learning. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp.
2018
Michael Minyi Zhang, Bianca Dumitrascu, Sinead A Williamson, and Barbara Engelhardt. Sequential gaussian processes for online learning of nonstationary functions. arxiv preprint Xii:1995510000 2019.
Xiao Zhang and Shizhong Liao. Online kernel selection via incremental sketched kernel alignment. In IJCAI, pp. 3118-3124 2018.
17

---

Published in Transactions on Machine Learning Research (10/2023)
10
Appendix
10.1
Additional Results
Noise dist. KEM (ours)
Noise dist.
Noise dist. for Stral. (our ab.)
Noise for ARD
Noise for Memoryless
Count
Count
200
Count
200
300
log
log
log
log o²
log
Figure 9: Bascline methods frequently underestimate the observation noise hyperparameter, a sign of overfitting. We plot the log-distribution of o² of test users with 5 training points. True o² = 0.5 (dotted black line). We constrain o² 2 0.01 during optimization for numerical stability (hence the spikes around -5).
10.2
Experimental Details
10.2.1 Datasets
Generation of synthetic data. Half of the user's functions were generated from LINO + PERO kernel composition. The other half's functions were generated from SEO kernel. To generate the synthetic data for each user, we first sample the hyperparameters of the ground-truth kernel. The users with LINO+PERO compostion had hyperparameters that were sampled from as follows:
Osngghscalee
Log-No (0g-NNrrallO,,, 0.1)
period
Log-Normal(1132) 0.1)
aaplitude Nog--oorma(1.60090000 0.1)
Sshitt Normal(5,
The users with SEO kernel had hyperparameters that were sampled as follows:
lengthscale
Log-NNrmaa(O. 0.1)
amplitude LlogNorrmal(23002 0.1)
The observation-noiie for the synthetic experiments was fixed for all users Onoise 0.5. After defining the user's kernel- by assigning the composition and hyperparameters- we sample the user's function from GP prior with the kernel.
Pre-pree processeesngg of real data. For the real data experiments, features are scaled to the range of [o, 1] using min-max normalization. The prediction targets are standardized (y/ g-mean(y) ). The prediction targets std(y)
for each of these data sets follows:
UCI Energy: heating load
UCI Housing: housing price
UCI Fires: Build up Index (BUI) for Bejaia data, Initial Spread Index (ISI) for Sidi data.
UCI Concrete: concrete compressive strength
MIMIC-III: plateau pressure
HeartSteps: square-root. transformed daily values of "number of active minutes."
18

---

Published in Transactions on Machine Learning Research (10/2023)
MIMIC-III Pre-processing The pre-processing steps for MIMIC given below:
Original number of data points: 331532 points
Select data points where user is on ventilation: 39125 points
Remove data points outside valid range (as defined by medical expert): 22046 points
Remove data points missing the prediction target (plateau pressure): 3321 points
Group data points by diagnosis upon admission. Remove diagnoses with fewer than 30 points per diagnosis: 975 points
HeartSteps Pre-processing The preeeeeeessnng steps for HeartSteps are as follows:
Original number of data points, aggregated across all users: 1658 points
Take square-root transformation of the target value (daily active minutes): 1658 points
Drop points where daily active minutes are zero: 1535 points
10.2.2 Train/Test Splits
In the synthetic experiments, we have access to the ground-truth kernel which can be used to evaluate the quality of the kernels selected for test users. This is not the case for real data, As result, the training and testing data splits differed slightly between the experiments involving real and synthetic data.
Synthetic Experiments
Training Users (10 training users)
Testing Users (50 test users)
Training data is used to select the kernel. The training data the cumulative data set at each time step.
Testing data is used to evaluate the kernel. Since we have access to the ground-truth kernel, the test data is the function evaluated at 200 uniformly spaced points along the X-axis with noisc.
Real Data Experiments For the real data experiments, users were randomly assigned to either the training or testing set in a 50 50 split.
Training Users (50% of total users randomly selected)
Testing Users (remaining 50% of users)
Training data is used to select the kernel. The training data is the cumulative data set at cach time step.
Testing data used to evaluate the kernel. Since we do not have the ground truth kernel, the test data refers the future (unobserved) data that will arrive the next time step.
The train/test split for the test users mirrors the realistic process of selecting a model for a user based on his cumulative data and then using the selected model to make predictions the data observed until the next update point.
19

---

Published in Transactions on Machine Learning Research (10/2023)
10.2.3
Candidate Kernel Pool
Kernel functions Consistent with the previous kernel selection literature, we consider the linear, periodic, and squaredl-xponeettal kernel functions.
ksE(s, a² exp
(-
(-
(x 110--'0 x')2
(9)
282
( a² exp (- 2sin
( x² 1/1)
(10)
82
(1, = d² -c) cc)
(11)
kWhite(Z,x') a a' Sūs if =zj
(12)
The squared-exponential kernel (eq. 99 has two hyperparameters, the lengthscale l and amplitude a². The periodic kernel (eq. 10 has three hyperparameters, the lengthscale l, amplitude a², and period p. The lincar kernel (eq. 111)) has two hyperparameters, the amplitude a² and offsct c. Finally the white kernel (eq.
12 has one hyperparameter, the amplitude noise parameter, a².
Data specific Kernel pools The basic candidate kernel pool is formed by pairing each kernel function with each dimension of the data. The synthetic experiments include finst-order interaction terms between these candidate kernels. The real data experiments include interaction termss on features that are identified as important to the prediction target in the literature (for example, whether or not is a weekend is an important feature in determining how active a user will be for that day). Finally, the ARD kernel (the product of an SE kernel on each dimension of the data) is included in the candidate pool well, so that our selection method can be directly compared the SE-ARD method.
Synthetic experiments: basic kernel functions on the first (and only) dimension of the data, along with interaction kernels up to the second degree: (LINO, PERO, SEO, LINO x PERO, LINO x SE0, LINO LINO, PERO SEO, PERO x SFO}
Real data experiments:
UCI Energy: 16 total candidate kernels.
UCI Housing:
38 total candidate kernels.
UCI Fires:
Interaction term on windspeed. 35 total candidate kernels.
UCT Concrete:
Interaction term cement and age. 39 total candidate kernels. MIMIC-III: Interaction term on whether patient on vasos (binary). 50 total candidate kernels.
HcartSteps: Interaction term on whether user is "traveling" (binary) and whether or not it is "weekend" (binary). 95 total candidate kernels.
10.2.4 KEM Inference Parameters
Gibbs sampling during pilot training:
until convergence, up to 200 iterations
10 iterations of global updates (assigning datasets to clusters)
100 samples from MH sampling algorithm during local updates (assigning kernels to clusters)
MH sampling algorithm:
Proposal distribution for kernel hyperparameters: No(0, onyer)) where Oyper 0.1 for synthetic experiments and hyper 0.05 for real data experiments (because features normalized)
20

---

Published in Transactions on Machine Learning Research (10/2023)
10.2.5
KEM Priors
Dirichlet Process
1
For the base distribution Gaareut [ parent::
For Bernoulli(Tr)
" = 0.1 if K2 is an atomic candidate kernel and K2 not K Kpavent
* π. 0.02 if K2 is an interaction candidate kernel and K₃ not in Kparent
Ty = 0.9 for Ký in Karrent
For ot ~ p(uf): Because of determines how the candidate kernel scaled, it also refers to the common "amplitude" hyperparameter included in most kernel definitions (sce section 10.2.3) therefore, p(a²) is prior over the amplitude hyperparameter defined in the next paragraph.
Kernel Hyperparameter Priors The prior distribution on the kernel hyperparameters are as follows:
Lengthscale: log p(0 ((0Jengthseale) = (0, for synthetic data, log p p(Orengthecale) = -V(022, 0.5) for real data
Period: log perriod) NV5, 0.25) for synthetic data, log p(0 p(Operiod) (..22,00.25 for real data
Amplitude: log p(famplitude) = 2)
Observation noise: log p( p(Qnoise) = (0,2)
Shift: p(fstun)) 0.0,0.11
The lengthscale and period priors differ between the synthetic and real experiments because the features are normalized between [0, 1] for the real experiments. For the synthelic experiments, the choice of = 5 for the prior on peerood reflects our general estimate that the period may be around 1 4 the domain the input
space.
Prior on the observation noise for the KEM. The KEM (ours) model enforces a prior on the obser- vation noise which depends on the parent composition. Lct Npareut denote the number of kernel components in the parent composition:
N(2,0.5),
for N Nparent
log p(0 noinoi parent)
N(1, 1), ((0,2),
for N Nparent 1
for N pparent >
10.3
Inference Details for Kernel Evolution Model
10.3.1
Details of Gibbs sampler steps
Assigning datasets to clusters. The seating assignment for the mtt customer is sampled from multi- nomial posterior with the probability:
p(z, =
K. 0, c
plyn X,, K. 0_ oz K. Zn = c)p(Zn = c|
(13)
where MT 0 o² and Z are the remaining kernel compositions, hyperparameters, noise levels, and seating assignments occupied tables after unscating customer m. Here p(y,,X,,, K 0- a²-) is the model likelihood. The table asssignment probability p(Z,, clZ as determined by a Chinese Restaurant Process.
21

---

Published in Transactions on Machine Learning Research (10/2023)
Assigning kernels to cluster. We assign kernel to cluster by sampling from the posterior distribution over kernels defined by the data sets in the cluster. We obtain these samples via the Beetrpolis-Hastingg algorithm described in section 10.3.2
Oc o. p(e, o² K.)G parent (Ko) x
II
p(yn Xn. Ke, 0c, 02
(14)
cZ =c
kernel
Likelihond of datasets
Since we are assigning a kernel at each cluster, it is possible for two different clusters to have the same composition but different hyperparameters.
10.3.2
Metropolis- Sampler for Kernel Selection
In the following, we detail the proposal distribution over kernel compositions used by the Metropolis-Hasse Hastings sampling algorithm in the paper.
10.3.3 Kernel Composition Proposal
NR
Let K by the current kernel composition, which is a sum of kernel components: K Cn, where Cn E K.
==0
is a kernel component and part of the total candidate kernel pool K. Tet N be the total number of candidate kernels in the kernel pool. We propose new compositions by randomly adding component, removing a component, doing nothing to the current composition. Doing nothing is an option because may alter the kernel by sampling the hyperparameeers, even though the composition remains the same. The Padd: Premoves and Puothing are parameters for the proposal distribution and must sum to 1.
P(add|K)
P(remove|K)
P(nothing/A)
1 NK < N
0.2
0.4
0.4
NK <1
0.5
0.5
NK =
0
0.5
0.5
If the action is to add remove component, need to choose which kernel component, C", to add remove to the composition. If the action is to add, C' is uniformly sampled from the kernel pool excluding the components that are already in K (there are N-N options). If the action is to remove, C' is uniformly sampled from the set of kernel components that currently compose K (there are NR options).
if action add if action remove
N-Nx 1
p(C'laction, K)
(15)
Nx* 1,
otherwise
The probability of proposing kernel K' from K is then p(C', action| K) p(C'laction, K))((ctiio) K). Note that this proposal distribution is not symmetric, and the corresponding acceptance ratio must incorporate p(K(K)) and K) appropriately.
10.3.4 Kernel Hyperparameter Proposal
The proposal distribution for the kernel hyperparamcters random normal distribution centered at the current bog-transformed hyperparameter values. We apply the log transformation because the lengthscale, period, amplitude, and observation noise parameters must be positive.
22

---

Published in Transactions on Machine Learning Research (10/2023)
The kernel composition and hyperparameters closely related; Linear kernel will not hyperparme- ter on the period, just as Periodic kernel will not use "shift" hyperparameter. In order to track the current hyperparaaeeeers under changing kernel composition, the proposal distrbution samples the two independently: p(K, 0) = p(K) pp()). The current hyperparameters are represented by H E R( where D the dimensionality of the data and each column corresponds to a type of hyperparameter a lengthscale, period, amplitude, shift, and observation noise parameter respectively. The last row in the matrix is always reserved for the observation noise, which applies to the entire composition (not specific to a dimension). The benefit of this representation is that hyperparameters can be sampled independently of the composition: when the composition is sampled, it imposes "mask" over the hyperparameter matrix. For example, for a D = 3 dimensional data set, a SEO + PERI + LIN2 kernel composition would impose the following mask:
0
1
1
M
0
0
1
0 0.
0
0
0
0
To reemphasize, each column corresponds to a lengthscale, period, amplitude, shift, and observation noise parameter, respectively. Each row represents a different base kernel (kernel function type and the dimension that it applies to). The final kernel hyperparameters would be represented by applying mask M hyper- parameter matrix H. In this example, where have an SE kernel on the dimension of the data, the mask would ensure that the columns containing the relevant hyperparameters, which is the lengthscale (first column) and amplitude (second column), used with the composition. We recognize that limitation of this representation is that composition that requires multiple copies of the same hyperparameter on the same dimension, such as a PERO x PERO kernel, could only be represented with one value for the period. However, this is a compact representation which avoids the need to maintain a hyperparameter matrix for each potential kernel composition in the kernel space. To be clear, our proposal method does not imply that different types of kernels would use the same lengthscale, since each row is different base kernel (type of kernel and dimension).
23


Reviews:
[{"review_id": "2GIkOcUMJl", "reviewer": "Reviewer_U5o6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors address an interesting problem in ML, where the complexity of the kernel can evolve as more users are added. Overall, the English level is satisfactory.\nWhile the empirical results demonstrate the merits of the approach, I have some concerns about the paper and think it should be revised significantly prior to publication.\nSpecifically, the presentation should be dramatically improved. The empirical evaluation is only focused on relatively small datasets and only one extremely small synthetic evaluation. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "\nThe authors address the task of multi-task Gaussian Process regression and present a method that selects compositional kernels in an online setting. The idea is to enable the evolution of the kernel as more users are observed; this allows the method to handle the bias-variance trade-off better. The new method is termed Kernel Evolution Model (KEM) and is motivated by online multitask applications in healthcare. The authors evaluate KEM on synthetic and real data, demonstrating its flexibility and stability.  \n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think the paper could be organized better, specifically moving the related work forward and briefly introducing the problem in the introduction to help the reader connect to the problem. \nThe example in Figure 1 is unclear; it is hard to identify differences between the blue and red results. Can you quantify this numerically? Also, the explanation of this example could be improved; for example, each element should be explained in the caption. \nCommas are missing after some of the equations (eq. 1…)\nThe authors discuss the bias-variance tradeoff several times; intuitively, I understand why it provides benefits here, but is there a way to demonstrate this?\nThe method is designed for sparse selection; why not demonstrate the feature selection capabilities of the method as in Figure 8 but with synthetic data where the informative features are known?\nFigure 6e is introduced before figures 4 and 5; is e a typo?\nARD performs very similarly to KEM but much faster in the synthetic example. Are there examples in which the merits of the approach are more significant? \nARD uses a constant kernel, so why present it in Figure 5?\nWhat are the properties of the real datasets?\nWhat is the model's accuracy (and baselines ) for the real data?\n", "broader_impact_concerns": "No ethical concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8OXZY9KwP0", "reviewer": "Reviewer_t2mc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+Given the strict setting discussed by the authors, the method seems reasonable and shows good experimental results.\n\n-The limitations of the proposed method should be discussed in more detail, see next section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In a multi-task setting, authors propose a kernel method to learn an appropriate kernel evolution that would select more complex kernels when more training data is available for a task.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "-the number of observations seen for users in the Piolot training phase should be similar to the ones expected in the online phase, otherwise the kernels may result too simple. This is another strong assumption that may be discussed in more detail.\n\n-related works: When you discuss e.g. Zhang et al., 2019 you mention that those methods are suited for non-stationary functions. Assuming a function as stationary in online learning is a strong assumption. Can you motivate more in-depth this choice? Also, is it possible to apply the method proposed by Zhang et al., 2019 also in a stationary setting? If so, an experimental comparison would be nice.\n\n-The proposed method does not perform well on the UCI: Fires dataset. Can authors analyse this dataset more in-depth and try to understand why this happens? e.g. which are the dataset characteristics that make the proposed method not perform as well as the baselines?\n\n-Comparison with other models: neural networks are commonly adopted for sequential data, e.g. LSTMs. With NNs it is easy to perform multi-task, transfer or meta learning (MAML as mentoned by authors) and, with the proper hardware, they tend to be quite fast. Can you include this comparison or motivate why it is not reasonable to include it? Authors explain it with a lack of sparsity, that in my understanding would mean a lack of explainability, which brings me to the last point in the next comment. However, it would be nice to see how these methods perform, also to understand the difference in predictive performances we are paying for a sparse solution.\n\n-you mention sparsity as a requirement for high-stakes applications. I guess this is directly related to the interpretability of the resulting models. Can you discuss more in depth this characteristic of the proposed method? Can it identify the features that would be relevant for experts, so they can trust the system?", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KaOmZNJ5CO", "reviewer": "Reviewer_jRnG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The explored setup of online multi-task learning (with new tasks appearing) is common in many modern applications and thus is highly relevant. The paper explains clearly the proposed approach, striking a good balance between the necessity to review basic concepts and the expected prior knowledge on the reader's side, making it fairly easy to follow. The experiments as well as the selected tables and graphics clarify and document the performance well.\n\nI am concerned about the scalability - in terms of input features, time points and number of tasks. You mention the challenge of number of tasks in pilot and interaction terms briefly in the the Discussion section, but I believe the scalability considerations deserve more attention. The search for kernel evolutions becomes more complex with higher number of features (and even more with higher number of considered interactions) - how much more complex (some quantification to help user to estimate the usability of the algo)? This also increases the complexity of the search during deployment (BIC over all possible evolutions) - how much?\n\nAll experiments are based on a rather limited number of examples. Do you have any recommendation for longer streams? Shall the evolutions be continued or frozen at some point (e.g. when $\\mathcal{D_{u,t}}$ is large enough based on some learning theory principals)?\n\nIn my understanding, the underlying assumption is some level of distributional stability between the tasks in the pilot and the deployment online setup. I didn't find this in the text. (Or have I missed it? I only found the stability within the stream of a single task.)\n\nThere are pieces of information missing (some I found in the appendix) which shall in my view be covered in the main text. See next section for details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses a particular online learning setup, where new models need to be developed quickly for newly appearing tasks (e.g. users). It proposes a kernel learning method (learning kernel compositions) for GPs, which shall be sparse (selects limited number of input features), stable (the features selected do not change much when additional online data arrive), adaptive in complexity (limited complexity at the start when only few data available and gradually increasing), and scalable (not getting unwieldy with increasing time). The method is based on learning a set of ``kernel evolutions'' over readily available pilot data that can then be used in the actual online regime. These evolutions are learned as a Markovian process across the time-steps (online batches of data) with Chinese Restaurant Process across the tasks in the pilot for each evolution step. The properties of the method are clarified on a toy synthetic dataset, and documented on 4 UCI and 2 real medical datasets. The proposed methods performs well in terms of test log-likelihoods, tends to avoid overfitting in small data regimes in the beginning of the data stream, prefers sparse solutions (BIC criteria), and is more stable in variable selection than the presented baselines.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I recommend addressing the points raised under `Strength and Weakness`.\n\nOther minor comments, that shall, however be addressed before publication:\n1. In equation (2) do the weights need to be non-negative?\n2. Please clarify briefly the difference between the ARD and SE kernels.\n3. Figure 4 - caption mentions lef and middle diagram. There is no middle diagram.\n4. You use ARD as a kernel as well as a method. I find this very confusing. Please make it clear in the text. (Also the difference between SE and ARD kernels.)\n5. Please Clarify the split to pilot and deployment - nums of tasks in each - in your experiments.\n6. Please clarify the total number of time steps in your experiments (length of kernel evolutions)\n7. In your online setup there is no separate test set, correct? The only test is the actual online deployment data on which you fit the model parameters and directly evaluate the likelihood (e.g. Figure 7 ``Test'' log likelihood). Please make this clearer to the reader.\n8. Page 9 Feasibility paragraph - you refer to figure 6e, I think this is not correct. Please check/correct.\n9. Table 1, UCI:Concrete - formatting of +-\n10. General formatting of latex quotes - somehow the end quote is difficult to read. Possible to ``correct''?", "broader_impact_concerns": "I do not see any ethical concerns requiring Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Dear authors,

All three reviewers have praised the interest and soundness of your contributions. I am happy to recommend acceptance provided that you can satisfactorily expand some of the comments on the limitations (see paragraph below), and include the new experiments that you suggested to better study how scalable your method is.

One of the reviewer wrote to me (not verbatim, these are my words):
An argument for a minor revision is issues related to scalability (number of features, expected interactions, users, time-steps). The paper describes the setting for when the method works well (the specific mHealth setup with large pilot small deployment, etc.) but does not provide enough guidance to be able to judge when it no longer should be used. The authors suggested an experiment to better explore the scalability of their method and this should included and commented upon before publication.

Congratulations on a fine piece of work!

Benjamin
Decision: Accept (poster)