SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The target task is to predict the lattice thermal conductivity (LTC) of inorganic crystalline materials, where the LTC is the amount of vibrational energy propagated by phonons in a crystal. In general, LTC can be calculated ab initio by performing many-body electronic structure calculations based on quantum mechanics. However, it is quite time-consuming to perform the first-principles calculations for thousands of crystals, which will be used as a training sample set to create a surrogate statistical model. Therefore, we perform TL with the source task of predicting an alternative, computationally tractable physical property called scattering phase space (SPS), which is known to be physically related to LTC.
We used the dataset from [8] that records SPS and LTC for 320 and 45 inorganic compounds, respectively. The input compounds were translated to 290-dimensional compositional descriptors using XenonPy [6, 33, 34, 35] $^1$ .
B.2.2 Model Definition and Hyperparameter Search
Fully connected neural networks were used for both the source and target models, with a LeakyReLU activation function with  $\alpha = 0.01$ . The model training was conducted using the Adam optimizer [36]. Hyperparameters such as the width of the hidden layer, learning rate, number of epochs, and regularization parameters were adjusted with 5-fold cross-validation. For more details on the experimental conditions and procedure, refer to the provided Python code.
Source Model For the preliminary step, neural networks with three hidden layers that predict SPS were trained using  $80\%$  of the 320 samples. 100 models with different numbers of neurons were randomly generated and the top 10 source models that showed the highest generalization performance in the source domain were selected. The hidden layer width  $L$  was randomly chosen from the range [50, 100], and we trained a neural network with a structure of (input)- $L-L-L-1$ . Each of the three hidden layers of the source model was used as an input to the transfer models, and we examined the difference in prediction performance for the three layers.
Target Model In the target task, an intermediate layer of a source model was used as the feature extractor. A model was trained using 40 randomly chosen samples of LTC, and its performance was evaluated with the remaining 5 samples. For each of the 10 source models, we performed the
1https://github.com/yoshida-lab/XenonPy
Figure S.3: Change of RMSE values between the affine transfer model and the ordinary feature extractor when using different levels of intermediate layers as the source features. The line plot shows the mean and  $95\%$  confidence interval. As a baseline, RMSE values for direct learning without transfer and fine-tuned neural networks are shown as dotted and dashed lines, respectively.
training and testing 10 times with different sample partitions and compared the mean values of RMSE among four different methods: (i) the affine model transfer using neural networks to model the three functions  $g_{1}, g_{2}$  and  $g_{3}$ , (ii) a neural network using the XenonPy compositional descriptors as input without transfer, (iii) a neural network using the source features as input, and (iv) fine-tuning of the pre-trained neural networks. The width of the layers of each neural network, the number of training epochs, and the dropout rate were optimized during 5-fold cross-validation looped within each training set. For the affine model transfer, the functions  $g_{1}, g_{2}$ , and  $g_{3}$  were modeled by neural networks. We used neural networks with one hidden layer for  $g_{1}, g_{2}$  and  $g_{3}$ .
B.2.3 Results
Figure S.3 shows the change in prediction performance of TL models using source features obtained from different intermediate layers from the first to the third layers. The affine transfer model and the ordinary feature extractor showed opposite patterns. The performance of the feature extractor improved when the first intermediate layer closest to the input layer was used as the source features and gradually degraded when layers closer to the output were used. When the third intermediate layer was used, a negative transfer occurred in the feature extractor as its performance became worse than that of the direct learning. In contrast, the affine transfer model performs better as the second and third intermediate layers closer to the output were used. The affine transfer model using the third intermediate layer reached a level of accuracy slightly better than fine-tuning, which intuitively uses more information to transfer than the extracted features.
In general, the features encoded in an intermediate layer of a neural network are more task-independent as the layer is closer to the input, and the features are more task-specific as the layer is closer to the output [9]. Because the first layer does not differ much from the original input, using both  $x$  and  $f_{s}$  in the affine model transfer does not contribute much to performance improvement. However, when using the second and third layers as the feature extractors, the use of both  $x$  and  $f_{s}$  contributes to improving the expressive power of the model, because the feature extractors have acquired different representational capabilities from the original input. In contrast, a model based only on  $f_{s}$  from a source task-specific feature extractor could not account for data in the target domain, so its performance would become worse than direct learning without transfer, i.e., a negative transfer would occur.
B.3 Heat Capacity of Organic Polymers
We highlight the benefits of separately modeling and estimating domain-specific factors through a case study in polymer chemistry. The objective is to predict the specific heat capacity at constant pressure  $C_{\mathrm{P}}$  of any given organic polymer with its chemical structure in the polymer's repeating unit. Specifically, we conduct TL to bridge the gap between experimental values and physical properties calculated from molecular dynamics (MD) simulations.
Figure S.4: MD-calculated (vertical axis) and experimental values (horizontal axis) of the specific heat capacity at constant pressure for various amorphous polymers.
Table S.1: Force field parameters that form the General AMBER force field [37] version 2 (GAFF2), and their detailed descriptions.
Parameter	Description
mass	Atomic mass
σ	Equilibrium radius of van der Waals (vdW) interactions
ε	Depth of the potential well of vdW interactions
charge	Atomic charge of Gasteiger model
r0	Equilibrium length of chemical bonds
Kbond	Force constant of bond stretching
polar	Bond polarization defined by the absolute value of charge difference between atoms in a bond
θ0	Equilibrium angle of bond angles
Kangle	Force constant of bond bending
Kdih	Rotation barrier height of dihedral angles
As shown in Figure S.4, there was a large systematic bias between experimental and calculated values; the MD-calculated properties  $C_{\mathrm{P}}^{\mathrm{MD}}$  exhibited an evident overestimation with respect to their experimental values. As discussed in [38], this bias is inevitable because classical MD calculations do not reflect the presence of quantum effects in the real system. According to Einstein's theory for the specific heat in physical chemistry, the logarithmic ratio between  $C_{\mathrm{P}}^{\mathrm{exp}}$  and  $C_{\mathrm{P}}^{\mathrm{MD}}$  can be calibrated by the following equation:
$$
\log C _ {\mathrm {P}} ^ {\exp} = \log C _ {\mathrm {P}} ^ {\mathrm {M D}} + 2 \log \left(\frac {\hbar \omega}{k _ {B} T}\right) + \log \frac {\exp \left(\frac {\hbar \omega}{k _ {B} T}\right)}{\left[ \exp \left(\frac {\hbar \omega}{k _ {B} T}\right) - 1 \right] ^ {2}}, \tag {S.3}
$$
where  $k_{B}$  is the Boltzmann constant,  $\hbar$  is the Planck constant,  $\omega$  is the frequency of molecular vibrations, and  $T$  is the temperature. The bias is a monotonically decreasing function of frequency  $\omega$ , which is described as a black-box function of polymers with their molecular features. Hereafter, we consider the calibration of this systematic bias using the affine transfer model.
Experimental values of the specific heat capacity of the 70 polymers were collected from PoLyInfo [39]. The MD simulation was also applied to calculate their heat capacities. For models to predict the log-transformed heat capacity, a given polymer with its chemical structure was translated into the 190-dimensional force field descriptors, using RadonPy  $[38]^2$ .
The force field descriptor represents the distribution of the ten different force field parameters ( $t \in \mathcal{T} = \{\mathrm{mass}, \sigma, \epsilon, \mathrm{charge}, r_0, K_{\mathrm{bond}}, \mathrm{polar}, \theta_0, K_{\mathrm{angle}}, K_{\mathrm{dih}}\}$  that make up the empirical potential (i.e., the General AMBER force field [37] version 2 (GAFF2)) of the classical MD simulation. The
$^{2}$ https://github.com/RadonPy/RadonPy
Algorithm S.1 Block relaxation algorithm for the model in Eq. (S.4).
repeat
Initialize:  $\alpha_0\gets \hat{\alpha}_{0,\mathrm{olr}}$ $\alpha_{1}\leftarrow \hat{\alpha}_{1,\mathrm{olr}},\beta \leftarrow 0,\gamma \leftarrow \hat{\gamma}_{\mathrm{diff}}$
$\pmb{\alpha} \gets \arg \min_{\pmb{\alpha}} F_{\alpha, \beta, \gamma}$
$\beta \gets \arg \min_{\beta}F_{\alpha ,\beta ,\gamma}$
until convergence
$\gamma \gets \arg \min_{\gamma}F_{\alpha ,\beta ,\gamma}$
detailed descriptions for each parameter are listed in Table S.1. For each  $t$ , pre-defined values are assigned to their constituent elements in a polymer, such as individual atoms (mass, charge,  $\sigma$ , and  $\epsilon$ ), bonds ( $r_0$ ,  $K_{\mathrm{bond}}$ , and polar), angles ( $\theta_0$  and  $K_{\mathrm{angle}}$ ), or dihedral angles ( $K_{\mathrm{dih}}$ ), respectively. The probability density function of the assigned values of  $t$  is then estimated and discretized into 10 points corresponding to 10 different element species such as hydrogen and carbon for mass, and 20 equally spaced grid points for the other parameters. The details of the descriptor calculations are described in [40].
The source feature  $f_{s}$  was given as the log-transformed value of  $C_{\mathrm{P}}^{\mathrm{MD}}$ . Therefore,  $f_{s}$  is no longer a function of  $x$ ; this modeling was intended for calibrating the MD-calculated properties.
We randomly sampled 60 training polymers and tested the prediction performance of a trained model on the remaining 10 polymers 20 times. The PoLyInfo sample identifiers for the selected polymers are listed in the code.
B.3.2 Model Definition and Hyperparameter Search
As described above, the 190-dimensional force field descriptor consists of ten blocks corresponding to different types of features. The  $J_{t}$  features that make up block  $t$  represent discretized values of the density function of the force field parameters assigned to the atoms, bonds, or dihedral angles that constitute the given polymer. Therefore, the regression coefficients of the features within a block should be estimated smoothly. To this end, we imposed fused regularization on the parameters as
$$
\lambda_ {1} \| \boldsymbol {\gamma} \| _ {2} ^ {2} + \lambda_ {2} \sum_ {t \in \mathcal {T}} \sum_ {j = 2} ^ {J _ {t}} \left(\gamma_ {t, j} - \gamma_ {t, j - 1}\right) ^ {2},
$$
where  $\mathcal{T} = \{\mathrm{mass},\mathrm{charge},\epsilon ,\sigma ,K_{\mathrm{bond}},r_0,K_{\mathrm{angle}},\theta ,K_{\mathrm{dih}}\}$ , and  $J_{t} = 10$  for  $t = \mathrm{mass}$  and  $J_{t} = 20$  otherwise. The regression coefficient  $\gamma_{t,j}$  corresponds to the  $j$ -th feature of block  $t$ .
Ordinary Linear Regression The experimental heat capacity  $y = \log C_{\mathrm{P}}^{\mathrm{exp}}$  was regressed on the MD-calculated property, without regularization, as  $\hat{y} = \alpha_0 + \alpha_1 f_s$  where  $\hat{y}$  denotes the conditional expectation and  $f_s = \log C_{\mathrm{P}}^{\mathrm{MD}}$ .
Learning the Log-Difference We calculated the log-difference  $\log C_{\mathrm{P}}^{\mathrm{exp}} - \log C_{\mathrm{P}}^{\mathrm{MD}}$  and trained the linear model with the ridge penalty. The hyperparameters  $\lambda_{1}$  and  $\lambda_{2}$  for the scale- and smoothness-regularizers were determined based on 5-fold cross validation across 25 equally space grids in the interval  $[10^{-2}, 10^{2}]$  for  $\lambda_{1}$  and across the set  $\{50, 100, 150\}$  for  $\lambda_{2}$ .
Affine Transfer The log-transformed value of  $C_{\mathrm{P}}^{\mathrm{exp}}$  is modeled as
$$
y := \log C _ {\mathrm {P}} ^ {\exp} = \underbrace {\alpha_ {0} + \alpha_ {1} f _ {s}} _ {g _ {1}} - \underbrace {(\beta f _ {s} + 1)} _ {g _ {2}} \cdot \underbrace {\boldsymbol {x} ^ {\top} \boldsymbol {\gamma}} _ {g _ {3}} + \epsilon_ {\sigma}, \tag {S.4}
$$
where  $\epsilon_{\sigma}$  represents observation noise, and  $\alpha_0,\alpha_1,\beta$  and  $\gamma$  are unknown parameters to be estimated. When  $\alpha_{1} = 1$  and  $\beta = 0$ , Eq. (S.4) is consistent with the theoretical equation in Eq. (S.3) in which the quantum effect is linearly modeled as  $\alpha_0 + \pmb{x}^\top \pmb{\gamma}$ .
Table S.2: Mean and standard deviation of RMSE of three prediction models.
Model	RMSE (log J/kg·K)
y = α0 + α1fs + εσ	0.1403 ± 0.0461
y = fs + xTγ + εσ	0.1368 ± 0.04265
y = α0 + α1fs - (βfs + 1)xTγ + εσ	0.1357 ± 0.04173
In the model training, the objective function was given as follows:
$$
\begin{array}{l} F _ {\boldsymbol {\alpha}, \beta , \boldsymbol {\gamma}} = \frac {1}{n} \sum_ {i = 1} ^ {n} \left\{y _ {i} - \left(\alpha_ {0} + \alpha_ {1} f _ {s, i} - (\beta f _ {s, i} + 1) \boldsymbol {x} ^ {\top} \boldsymbol {\gamma}\right) \right\} ^ {2} \\ + \lambda_ {\beta} \beta^ {2} + \lambda_ {\gamma , 1} \| \boldsymbol {\gamma} \| _ {2} ^ {2} + \lambda_ {\gamma , 2} \sum_ {t \in T} \sum_ {j = 2} ^ {J _ {t}} \left(\gamma_ {t, j} - \gamma_ {t, j - 1}\right) ^ {2}, \\ \end{array}
$$
where  $\alpha = [\alpha_0\alpha_1]^\top$ . With a fixed  $\lambda_{\beta} = 1$ , the remaining hyperparameters  $\lambda_{\gamma,1}$  and  $\lambda_{\gamma,2}$  were optimized through 5-fold cross validation over 25 equally space grids in the interval  $[10^{-2},10^{2}]$  for  $\lambda_{\gamma,1}$  and across the set  $\{50,100,150\}$  for  $\lambda_{\gamma,2}$ .
The algorithm to estimate the parameters  $\alpha, \beta$  and  $\gamma$  is described in Algorithm S.1, where  $\alpha_{0,\mathrm{olr}}$  and  $\alpha_{1,\mathrm{olr}}$  are the estimated parameters of the ordinary linear regression model, and  $\hat{\gamma}_{\mathrm{diff}}$  is the estimated parameter of the log-difference model. For each step, the full conditional minimization of  $F_{\alpha, \beta, \gamma}$  with respect to each parameter can be made analytically as
$$
\arg \min  _ {\boldsymbol {\alpha}} F _ {\boldsymbol {\alpha}, \beta , \gamma} = \left(F _ {s} ^ {\top} F _ {s}\right) ^ {- 1} F _ {s} ^ {\top} (\boldsymbol {y} + \left(\beta \boldsymbol {f} _ {\boldsymbol {s}, 1: n} + 1\right) \circ (X \boldsymbol {\gamma})),
$$
$$
\arg \min  _ {\beta} F _ {\boldsymbol {\alpha}, \beta , \gamma} = - \left(\boldsymbol {f} _ {\boldsymbol {s}, 1: n} ^ {\top} \operatorname {d i a g} (X \boldsymbol {\gamma}) ^ {2} \boldsymbol {f} _ {\boldsymbol {s}, 1: n} + n \lambda_ {2}\right) ^ {- 1} \boldsymbol {f} _ {\boldsymbol {s}, 1: n} ^ {\top} \operatorname {d i a g} (X \boldsymbol {\gamma}) (\boldsymbol {y} - F _ {s} \boldsymbol {\alpha} + X \boldsymbol {\gamma}),
$$
$$
\arg \min _ {\boldsymbol {\gamma}} F _ {\boldsymbol {\alpha}, \beta , \boldsymbol {\gamma}} = - (X ^ {\top} \operatorname {d i a g} (\beta \boldsymbol {f} _ {\boldsymbol {s}, 1: n} + 1) ^ {2} X + \Lambda) ^ {- 1} X ^ {\top} \operatorname {d i a g} (\beta \boldsymbol {f} _ {\boldsymbol {s}, 1: n} + 1) (\boldsymbol {y} - F _ {s} \boldsymbol {\alpha}),
$$
where  $X$  denote the matrix in which the  $i$ -th row is  $x_{i}$ ,  $\pmb{y} = [y_{1}\dots y_{n}]^{\top}$ ,  $\pmb{f}_{s,1:n} = [f_{s,1}\dots f_{s,n}]^{\top}$ ,  $F_{s} = [\pmb{f}_{s,1:n} \mathbf{1}]$ , and  $d = 190$ .  $\Lambda$  is a matrix including the two regularization parameters  $\lambda_{\gamma,1}$  and  $\lambda_{\gamma,2}$  as
$$
\Lambda = D ^ {\top} D, \text {w h e r e} D = \left[ \begin{array}{l} \lambda_ {\gamma , 1} I _ {d} \\ \lambda_ {\gamma , 2} M \end{array} \right], M = \left[ \begin{array}{c c c c c c} - 1 & 1 & 0 & \dots & 0 & 0 \\ 0 & - 1 & 1 & \dots & 0 & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & 0 & \dots & 0 & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & 0 & \dots & - 1 & 1 \end{array} \right] \leftarrow m \text {- t h r o w s},
$$
where  $m \in \{10, 30, 50, 70, 90, 110, 130, 150, 170\}$ . Note that the matrix  $M$  is the same as the matrix  $\left[\mathbf{0} I_{189}\right] - \left[I_{189} \mathbf{0}\right]$  except that the  $m$ -th row is all zeros. Note also that  $M \in \mathbb{R}^{189 \times 190}$ , and therefore  $D \in \mathbb{R}^{279 \times 190}$  and  $\Lambda \in \mathbb{R}^{190 \times 190}$ .
The stopping criterion of the algorithm was set as
$$
\max  _ {\theta \in \{\boldsymbol {\alpha}, \beta , \boldsymbol {\gamma} \}} \frac {\max  _ {i} \left| \theta_ {i} ^ {(\text {n e w})} - \theta_ {i} ^ {(\text {o l d})} \right|}{\max  _ {i} \left| \theta_ {i} ^ {(\text {o l d})} \right|} <   1 0 ^ {- 4}, \tag {S.5}
$$
where  $\theta_{i}$  denotes the  $i$ -th element of the parameter  $\theta$ . This convergence criterion is employed in several existing machine learning libraries, e.g., scikit-learn<sup>3</sup>.
B.3.3 Results
Table S.2 summarizes the prediction performance (RMSE) of the three models. The ordinary linear model  $y = \alpha_{0} + \alpha_{1}f_{s} + \epsilon_{\sigma}$ , which ignores the force field descriptors, exhibited the lowest prediction
<sup>3</sup>https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html
Figure S.5: Bar plot of regression coefficients  $\gamma$  of linear calibrator filling the discrepancy between experimental and MD-calculated specific heat capacity of amorphous polymers.
performance. The other two calibration models  $y = f_{s} + \boldsymbol{x}^{\top}\boldsymbol{\gamma} + \epsilon_{\sigma}$  and the full model in Eq. (S.4) reached almost the same accuracy, but the latter had achieved slightly better prediction accuracy. The estimated parameters of the full model were  $\alpha_{1} \approx 0.889$  and  $\beta \approx -0.004$ . The model form is highly consistent with the theoretical equation in Eq. (S.3) as well as the restricted model  $(\alpha_{1} = 1, \beta = 0)$ . This supports the validity of the theoretical model in [38].
It is expected that physicochemical insights can be obtained by examining the estimated coefficient  $\gamma$ , which would capture the contribution of the force field parameters to the quantum effects. The magnitude of the quantum effect is a monotonically increasing function of the frequency  $\omega$ , and is known to be highly related to the descriptors  $\epsilon$ ,  $K_{\mathrm{dih}}$ ,  $K_{\mathrm{bond}}$ ,  $K_{\mathrm{angle}}$  and mass. According to physicochemical intuition, it is considered that as  $\epsilon$ ,  $K_{\mathrm{bond}}$ ,  $K_{\mathrm{angle}}$ , and  $K_{\mathrm{dih}}$  decrease, their potential energy surface becomes shallow, which leads to the decrease of  $\omega$ , and in turn the decrease of quantum effects. Furthermore, because the molecular vibration of light-weight atoms is faster than that of heavy atoms,  $\omega$  and quantum effects should theoretically increase with decreasing mass.
Figure S.5 shows the mean values of the estimated parameter  $\gamma$  for the full calibration model. The physical relationships described above can be captured consistently with the estimated coefficients. The coefficients in lower regions of  $\epsilon$ ,  $K_{\mathrm{bond}}$ ,  $K_{\mathrm{angle}}$  and  $K_{\mathrm{dih}}$  showed large negative values, indicating that polymers containing more atoms, bonds, angles, and dihedral angles with lower values will have smaller quantum effects. Conversely, the coefficients in lower regions of mass showed positive large values, meaning that polymers containing more atoms with smaller masses will have larger quantum effects. As illustrated in this example, separate inclusion of the domain-common and domain-specific factors in the affine transfer model enables us to infer the features relevant to the cross-domain differences.
Instructions for obtaining the datasets used in the experiments are described in the code.
C.1 Kinematics of the Robot Arm
C.1.1 Data
We used the SARCOS dataset in [23]. The task is to predict the feed-forward torque required to follow the desired trajectory in the seven joints of the SARCOS anthropomorphic robot arm. The twenty one features representing the joints' position, velocity, and acceleration were used as  $\pmb{x}$ . The observed values of six torques other than the torque at the joint in the target domain were given to the source features  $\pmb{f}_s$ . The dataset includes 44,484 training samples and 4,449 test samples. We selected  $\{5,10,15,20,30,40,50\}$  samples randomly from the training set. The prediction performances of the trained models were evaluated using the 4,449 test samples. Repeated experiments were conducted 20 times with different independently sampled datasets.
C.1.2 Model Definition and Hyperparameter Search
Source model For each target task, a multi-task neural network was trained to predict the torque values of the remaining six source tasks. The source model shares four layers (256-128-64-32) up to the final layer, and only the output layer is task-specific. We used all training data and Adagrad [41] with learning rate of 0.01.
Algorithm S.2 Block relaxation algorithm for AffineTL-full.
Initialize:  $a_0\gets (K_1 + \lambda_1I_n)^{-1}\pmb {y},\pmb {b}_0\sim \mathcal{N}(\pmb {0},I_n),\pmb {c}_0\sim \mathcal{N}(\pmb {0},I_n),d_0\gets 0.5$  repeat  $\begin{array}{rl} & {\pmb {a}\leftarrow (K_{1} + \lambda_{1}I_{n})^{-1}(\pmb {y} - (K_{2}\pmb {b} + \pmb {1})\circ (K_{3}\pmb {c}) - d\pmb {1})}\\ & {\pmb {b}\leftarrow (\mathrm{Diag}(K_3\pmb {c})^2 K_2 + \lambda_2I_n)^{-1}((K_3\pmb {c})\circ (\pmb {y} - K_1\pmb {a} - K_3\pmb {c} - d\pmb {1}))}\\ & {\pmb {c}\leftarrow (\mathrm{Diag}(K_2\pmb {b} + \pmb {1})^2 K_3 + \lambda_3I_n)^{-1}((K_2\pmb {b} + \pmb {1})\circ (\pmb {y} - K_1\pmb {a} - d\pmb {1}))\\ & {d}\leftarrow \langle \pmb {y} - K_1\pmb {a} - (K_2\pmb {b} + \pmb {1})\circ (K_3\pmb {c}),\pmb {1}\rangle /n} \end{array}$  until convergence
Direct, Only source, Augmented, HTL-offset, HTL-scale For each procedure, we used kernel ridge regression with the RBF kernel  $k(\pmb{x}, \pmb{x}^{\prime}) = \exp\left(-\frac{1}{2\ell^{2}}\|\pmb{x} - \pmb{x}^{\prime}\|_{2}^{2}\right)$ . The scale parameter  $\ell$  was set to the square root of the input dimension as  $\ell = \sqrt{21}$  for Direct, HTL-offset and HTL-scale,  $\ell = \sqrt{6}$  for Only source and  $\ell = \sqrt{27}$  for Augmented. The regularization parameter  $\lambda$  was selected in 5-fold cross-validation in which the grid search was performed over 50 grid points in the interval [10-4, 10²].
AffineTL-full, AffineTL-const We considered the following kernels:
$$
\begin{array}{l} k _ {1} \left(\boldsymbol {f} _ {\boldsymbol {s}} (\boldsymbol {x}), \boldsymbol {f} _ {\boldsymbol {s}} \left(\boldsymbol {x} ^ {\prime}\right)\right) = \exp \left(- \frac {1}{2 \ell^ {2}} \| \boldsymbol {f} _ {\boldsymbol {s}} (\boldsymbol {x}) - \boldsymbol {f} _ {\boldsymbol {s}} \left(\boldsymbol {x} ^ {\prime}\right) \| _ {2} ^ {2}\right) (\ell = \sqrt {6}), \\ k _ {2} (\boldsymbol {f} _ {\boldsymbol {s}} (\boldsymbol {x}), \boldsymbol {f} _ {\boldsymbol {s}} (\boldsymbol {x} ^ {\prime})) = \exp \left(- \frac {1}{2 \ell^ {2}} \| \boldsymbol {f} _ {\boldsymbol {s}} (\boldsymbol {x}) - \boldsymbol {f} _ {\boldsymbol {s}} (\boldsymbol {x} ^ {\prime}) \| _ {2} ^ {2}\right) (\ell = \sqrt {6}), \\ k _ {3} (\boldsymbol {x}, \boldsymbol {x} ^ {\prime}) = \exp \left(- \frac {1}{2 \ell^ {2}} \| \boldsymbol {x} - \boldsymbol {x} ^ {\prime} \| _ {2} ^ {2}\right) (\ell = \sqrt {2 7}), \\ \end{array}
$$
for  $g_{1}, g_{2}$  and  $g_{3}$  in the affine transfer model, respectively.