SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\frac {y (\mathbf {x} , t _ {T}) - y (\mathbf {x} , t _ {0})}{t _ {T} - t _ {0}} = \frac {\alpha}{t _ {T} - t _ {0}} y (\mathbf {x}, t _ {0}) + \frac {1}{t _ {T} - t _ {0}} v (\mathbf {x}, t _ {0}). \tag {A.8}
$$
Since  $\alpha$  and  $v(\mathbf{x},t_0)$  are values and function to be estimated, we can absorb the constant  $t_T - t_0$  into  $\alpha$  and  $v(\mathbf{x},t_0)$  and write
$$
\frac {y (\mathbf {x} , t _ {T}) - y (\mathbf {x} , t _ {0})}{t _ {T} - t _ {0}} = \beta_ {0} y (\mathbf {x}, t _ {0}) + u (\mathbf {x}, t _ {0}). \tag {A.9}
$$
where  $\beta_0\equiv \alpha /t_T - t_0$  and  $u(\mathbf{x},t_0)\equiv v(\mathbf{x},t_0) / t_T - t_0$ . We recognize that is a explicit solution of a different equation. If we take the limit of  $(t_{T} - t_{0})\to 0$ , we have
$$
\frac {\mathrm {d} y (\mathbf {x} , t)}{\mathrm {d} t} = \beta (t) y (\mathbf {x}, t) + u (\mathbf {x}, t), \tag {A.10}
$$
which is in Proposition II. Notice that, in this equation, we turn the constant  $\beta$  into a function  $\beta(t)$ , which allows us to control differential level of information transfer depending on its fidelity level as a general formulation.
Taking a forward Euler solution to solve Eq. (A10), we have
$$
\frac {y (\mathbf {x} , t + \Delta t) - y (\mathbf {x} , t)}{\Delta t} + \beta (t) y (\mathbf {x}, t) = u (\mathbf {x}, t) \tag {A.11}
$$
$$
y (\mathbf {x}, t + \Delta t) = (1 + \Delta t \beta (t)) y (\mathbf {x}, t) + \Delta t u (\mathbf {x}, t).
$$
We recognize that this formulation is exactly the same as the classic AR with  $\rho = 1 + \Delta t\beta (t)$  with a residual  $\mathrm{GP}\Delta tu(\mathbf{x},t)$ .
C A General Solution to LiFiDE
We derive the general solution to the derived linear fidelity differential equation (LiFiDE)
$$
\frac {\mathrm {d} y (\mathbf {x} , t)}{\mathrm {d} t} + \beta (t) y (\mathbf {x}, t) = u (\mathbf {x}, t), \tag {A.12}
$$
which is a standard non-homogeneous first order differential equation. We know that for the homogeneous equation, i.e.,  $u(\mathbf{x},t) = 0$ , the general solution is
$$
y (\mathbf {x}, t) = C (\mathbf {x}) e ^ {- B (t)}, \tag {A.13}
$$
where  $C(\mathbf{x})$  is a functional of  $\mathbf{x}$ , and  $B(t) = \int^{t} \beta(\tau) \, \mathrm{d}\tau$  is the antiderivative of  $\beta(t)$ . Thus, we assume the general solution of the non-homogeneous equation is of the form  $y(\mathbf{x}, t) = v(\mathbf{x}, t)e^{-B(t)}$ , where  $v(\mathbf{x}, t)$  is a functional of  $\mathbf{x}$  and  $t$ . Substituting this into Eq. (A12), we have
$$
\begin{array}{l} \frac {\mathrm {d} v (\mathbf {x} , t)}{\mathrm {d} t} e ^ {B (t)} - v (\mathbf {x}, t) e ^ {- B (t)} \beta (t) + \beta (t) v (\mathbf {x}, t) e ^ {- B (t)} = u (\mathbf {x}, t), \tag {A.14} \\ \frac {\mathrm {d} v (\mathbf {x} , t)}{\mathrm {d} t} = u (\mathbf {x}, t) e ^ {- B (t)}. \\ \end{array}
$$
Integrating both sides, we have
$$
v (\mathbf {x}, t) = \int^ {t} u (\mathbf {x}, \tau) e ^ {- B (\tau)} \mathrm {d} \tau + C (\mathbf {x}). \tag {A.15}
$$
Putting this back into the assumed solution, we have
$$
y (\mathbf {x}, t) = e ^ {- B (t)} \int^ {t} u (\mathbf {x}, \tau) e ^ {- B (\tau)} \mathrm {d} \tau + C (\mathbf {x}) e ^ {- B (t)}. \tag {A.16}
$$
If we assume a constant for  $\beta (t)$  , i.e.,  $\beta (t) = \beta$  , then we have
$$
\begin{array}{l} y (\mathbf {x}, t) = e ^ {- \beta t} \int_ {c t} ^ {t} u (\mathbf {x}, \tau) e ^ {\beta \tau} \mathrm {d} \tau + C (\mathbf {x}) e ^ {- \beta t} \tag {A.17} \\ = C (\mathbf {x}) e ^ {- \beta t} + \int^ {t} u (\mathbf {x}, \tau) e ^ {- \beta (t - \tau)} d \tau . \\ \end{array}
$$
For a practical from where the integral starts from  $t_0$ , the lowest-fidelity index, we have
$$
\begin{array}{l} y (\mathbf {x}, t _ {0}) = C (\mathbf {x}) e ^ {- \beta t _ {0}} + \int_ {0} ^ {t _ {0}} u (\mathbf {x}, \tau) e ^ {- \beta (t - \tau)} d \tau \tag {A.18} \\ = C (\mathbf {x}) e ^ {- \beta t _ {0}}, \\ \end{array}
$$
where we have assumed  $u(\mathbf{x}, t) = 0$  for  $t < t_0$  because we are not interested in  $t < t_0$ . Substituting this into Eq. (A17), we have
$$
\begin{array}{l} y (\mathbf {x}, t) = C (\mathbf {x}) e ^ {- \beta t} + \int_ {0} ^ {t} u (\mathbf {x}, \tau) e ^ {- \beta (t - \tau)} d \tau \\ = y (\mathbf {x}, t _ {0}) e ^ {\beta t _ {0}} e ^ {- \beta t} + \int_ {0} ^ {t _ {0}} u (\mathbf {x}, \tau) e ^ {- \beta (t - \tau)} d \tau + \int_ {t _ {0}} ^ {t} u (\mathbf {x}, \tau) e ^ {- \beta (t - \tau)} d \tau \tag {A.19} \\ = y (\mathbf {x}, t _ {0}) e ^ {- \beta (t - t _ {0})} + \int_ {t _ {0}} ^ {t} u (\mathbf {x}, \tau) e ^ {- \beta (t - \tau)} d \tau . \\ \end{array}
$$
This formulation is equivalent to Eq. (A17). However, it allows for more flexibility in practice as we can give start the model with the lowest-fidelity  $t_0$  and then use the model to predict the higher-fidelity  $t > t_0$ .
D General Solutions to LiFiDEs
Consider LiFiDEs taking this general form
$$
\frac {d \mathbf {y} (\mathbf {x} , t)}{d t} + \mathbf {B} (t) \mathbf {y} (\mathbf {x}, t) = \mathbf {S u} (\mathbf {x}, t), \tag {A.20}
$$
where  $\mathbf{B}(t)$  and  $\mathbf{S}$  are matrices,  $\mathbf{y}(\mathbf{x}, t)$  is a vector of unknown functions, and  $\mathbf{u}(\mathbf{x}, t)$  is a non-zero vector.
The general solution is the sum of the homogeneous solution  $\mathbf{y}_h(\mathbf{x},t)$  and a particular solution  $\mathbf{y}_p(\mathbf{x},t)$ .
The homogeneous solution comes from the homogeneous equation:
$$
\frac {d \mathbf {y} _ {h} (\mathbf {x} , t)}{d t} + \mathbf {B} (t) \mathbf {y} _ {h} (\mathbf {x}, t) = 0. \tag {A.21}
$$
This has the solution:
$$
\mathbf {y} _ {h} (\mathbf {x}, t) = e ^ {- \int \mathbf {B} (t) d t} \mathbf {c}, \tag {A.22}
$$
where  $\mathbf{c}$  is a constant vector. The particular solution comes from the non-homogeneous equation and can be obtained using the variation of parameters. The method involves finding a function  $\mathbf{v}(\mathbf{x},t)$  such that:
$$
\mathbf {y} _ {p} (\mathbf {x}, t) = e ^ {- \int \mathbf {B} (t) d t} \mathbf {v} (\mathbf {x}, t) \tag {A.23}
$$
is a solution to the non-homogeneous system. Substituting  $\mathbf{y}_p$  into the non-homogeneous system gives:
$$
\frac {d \mathbf {v} (\mathbf {x} , t)}{d t} = e ^ {\int \mathbf {B} (t) d t} \mathbf {S u} (\mathbf {x}, t), \tag {A.24}
$$
which can be integrated to find  $\mathbf{v}(\mathbf{x},t)$ . Thus, the particular solution is:
$$
\mathbf {y} _ {p} (\mathbf {x}, t) = e ^ {- \int \mathbf {B} (t) d t} \int \left[ e ^ {\int \mathbf {B} (t) d t} \mathbf {S u} (\mathbf {x}, t) \right] d t. \tag {A.25}
$$
The general solution to the non-homogeneous system is the sum of the homogeneous and particular solutions:
$$
\begin{array}{l} \mathbf {y} (\mathbf {x}, t) = \mathbf {y} _ {h} (\mathbf {x}, t) + \mathbf {y} _ {p} (\mathbf {x}, t) \\ = e ^ {- \int \mathbf {B} (t) d t} \mathbf {c} + e ^ {- \int \mathbf {B} (t) d t} \int \left[ e ^ {\int \mathbf {B} (t) d t} \mathbf {S u} (\mathbf {x}, t) \right] d t, \tag {A.26} \\ \end{array}
$$
which is similar to the solution to LiFiDE in Eq. (A17). Note that the matrix exponential computation is done by
$$
e ^ {B (t)} = I + B (t) + \frac {(B (t)) ^ {2}}{2 !} + \frac {(B (t)) ^ {3}}{3 !} + \frac {(B (t)) ^ {4}}{4 !} + \dots , \tag {A.27}
$$
which is not easy to compute. However, if the matrix  $\mathbf{B}(t)$  is a constant matrix  $\beta \mathbf{I}$ , then the matrix exponential can be computed by
$$
e ^ {B (t)} = e ^ {\beta I} = I e ^ {\beta} = \left[ \begin{array}{c c c c} e ^ {\beta} & 0 & \dots & 0 \\ 0 & e ^ {\beta} & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & e ^ {\beta} \end{array} \right], \tag {A.28}
$$
which gives us a efficient way to derive a solution to LiFiDEs,
$$
\mathbf {y} (\mathbf {x}, t) = e ^ {- \beta t} \mathbf {C} + e ^ {- \beta t} \int e ^ {\beta t} \mathbf {S u} (\mathbf {x}, t) d t. \tag {A.29}
$$
E Proof of Lemma 1
Proof. The autokrigeability in ContinuAR is easy to derive once we derive the predictive posterior for the subset case in Appendix G and the non-subset case in Appendix H. We can see the matrix  $\mathbf{H}$  and  $\mathbf{S}\mathbf{S}^{\top}$  are canceled out in the predictive mean.
F Joint Likelihood of FiDEs-1
We derive the joint likelihood of FiDEs for observations  $\mathbf{Y} = [\vec{\mathbf{y}}^{(0)};\vec{\mathbf{y}}^{(1)}]$  as an illustrating example. For clarity, we slightly abuse the notations by replacing the previous spatial correlation  $H$  in the main paper with  $\mathbf{S}^{(0)}$  and  $\mathbf{SS}^{\top}$  with  $\mathbf{S}^{(1)}$ . The joint probability for  $\mathbf{Y}$  is
$$
\left( \begin{array}{c} \vec {\mathbf {y}} ^ {(0)} \\ \vec {\mathbf {y}} ^ {(1)} \end{array} \right) \sim \mathcal {N} \left(\mathbf {0}, \begin{array}{c c} k ^ {0} (\mathbf {X} ^ {(0)} \mathbf {X} ^ {(0)}) \otimes \mathbf {S} ^ {(0)} & e ^ {- \beta (t _ {1} - t _ {0})} k ^ {0} (\mathbf {X} ^ {(0)}, \mathbf {X} ^ {(0)}) \otimes \mathbf {S} ^ {(0)} \\ e ^ {- \beta (t _ {1} - t _ {0})} k ^ {0} (\mathbf {X} ^ {(0)}, \mathbf {X} ^ {(1)}) \otimes \mathbf {S} ^ {(0)} & e ^ {- 2 \beta (t _ {1} - t _ {0})} k ^ {0} (\mathbf {X} ^ {(1)}, \mathbf {X} ^ {(1)}) \otimes \mathbf {S} ^ {(0)} + k ^ {u _ {1}} (\mathbf {X} ^ {(1)}, \mathbf {X} ^ {(1)}) \otimes \mathbf {S} ^ {(1)} \end{array} \right),
$$
where  $k^{u_1}(\mathbf{X}^{(1)},\mathbf{X}^{(1)}) = \int_{t_0}^{t_1}e^{-B(t - \tau)}\int_{t_0}^{t_1}e^{-B(t' - \tau ')}k^u (\mathbf{X}^{(1)},\tau ,\mathbf{X}^{(1)},\tau ') \mathrm{d}\tau '\mathrm{d}\tau$
Eq. (A30) reveals that the for a two fidelity problem, ContinuAR is equivalent to AR. Thus, the likelihood decomposition for subset structure data [LO] also holds for ContinuAR, which lays out the foundation for our fast inference algorithm for non-subset problems.
G Predictive Posterior With Subset Structure
In order to conduct the proof of decomposition for the likelihood function with non-subset data, we need to derive the form of predictive posterior distribution with multi-fidelity with subset structure i.e.,  $\mathbf{X}^{(T)}\subset ,\dots ,\subset$ $\mathbf{X}^{(0)}$  , first. Here, we take the simplest situation, i.e.,  $T = 1$  as an example, and the assumption  $\mathbf{X}^{(1)}\subset \mathbf{X}^{(0)}$  holds. To stay consistent with AR, we denote  $\rho = e^{-\beta (t_1 - t_0)}$  for clarity. Following the format of the covariance
matrix in Eq. (A30) and Eq. (A6), the mean function and covariance matrix have the following expression,
$$
\begin{array}{l} \bar {\mathbf {y}} (x _ {*}, 1) \\ = \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}, \rho^ {2} \mathbf {k} _ {*} ^ {(0)} (\mathbf {x} ^ {(1)}) \otimes \mathbf {S} ^ {(0)} + \mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) \boldsymbol {\Sigma} ^ {- 1} \left( \begin{array}{c} \vec {\mathbf {y}} ^ {(0)} \\ \vec {\mathbf {y}} ^ {(1)} \end{array} \right) \\ = \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\mathbf {K} ^ {(0 0)} \otimes \mathbf {S} ^ {(0)}\right) ^ {- 1} \vec {\mathbf {y}} ^ {(0)} + \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\mathbf {E} ^ {(1)} \left(\rho^ {2} \mathbf {K} ^ {(0 0)}\right) ^ {- 1} \left(\mathbf {E} ^ {(1)}\right) ^ {\top} \otimes \left(\mathbf {S} ^ {(1)}\right) ^ {- 1}\right) \vec {\mathbf {y}} ^ {(0)} \\ - \left(\rho^ {2} \mathbf {k} _ {*} ^ {(0)} (\mathbf {x} ^ {(1)}) \otimes \mathbf {S} ^ {(0)}\right) \left(\rho \mathbf {K} _ {a} ^ {(1 1)} (\mathbf {E} ^ {(1)}) ^ {\top} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \vec {\mathbf {y}} ^ {(0)} - \left(\mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\rho \mathbf {K} _ {a} ^ {(1 1)} (\mathbf {E} ^ {(1)}) ^ {\top} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \vec {\mathbf {y}} ^ {(0)} \\ - \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\mathbf {E} ^ {(1)} \left(\rho \mathbf {K} _ {a} ^ {(1 1)}\right) ^ {- 1} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \mathbf {y} _ {a} ^ {(1)} \\ + \left(\rho^ {2} \mathbf {k} _ {*} ^ {(0)} (\mathbf {x} ^ {(1)}) \otimes \mathbf {S}\right) \left(\mathbf {K} _ {a} ^ {(1 1)} \otimes \mathbf {S}\right) ^ {- 1} \mathbf {y} _ {a} ^ {(1)} + \left(\mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(0)}\right) \left(\mathbf {K} _ {a} ^ {(1 1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \mathbf {y} _ {a} ^ {(1)} \\ = \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\mathbf {K} ^ {(0 0)} \otimes \mathbf {S} ^ {(0)}\right) ^ {- 1} \vec {\mathbf {y}} ^ {(0)} - \left(\mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\rho \mathbf {K} _ {a} ^ {(1 1)} (\mathbf {E} ^ {(1)}) ^ {\top} \otimes (\mathbf {S} ^ {(1)}) ^ {- 1}\right) \vec {\mathbf {y}} ^ {(0)} \\ + \left(\mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\rho \mathbf {K} _ {a} ^ {(1 1)} (\mathbf {E} ^ {(1)}) ^ {\top} \otimes (\mathbf {S} ^ {(0)}) ^ {- 1}\right) \vec {\mathbf {y}} ^ {(0)} + \left(\mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) \left(\mathbf {K} _ {a} ^ {(1 1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \mathbf {y} _ {a} ^ {(1)} \\ = \left(\rho \mathbf {k} _ {*} ^ {(0)} \left(\mathbf {K} ^ {(0 0)}\right) ^ {- 1} \otimes \mathbf {I}\right) \vec {\mathbf {y}} ^ {(0)} + \left(\mathbf {k} _ {a *} ^ {(1)} \left(\mathbf {K} _ {a} ^ {(1 1)}\right) ^ {- 1} \otimes \mathbf {I}\right) \mathbf {y} _ {a} ^ {(1)}, \tag {A.31} \\ \end{array}
$$
$$
\begin{array}{l} \boldsymbol {\Sigma} _ {*} ^ {(1)} \\ = \left(\rho^ {2} \mathbf {k} _ {*} ^ {(0 0)} \otimes \mathbf {S} ^ {(0)} + \mathbf {k} _ {a *} ^ {(1 1)} \otimes \mathbf {S} ^ {(1)}\right) \\ - \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}, \rho^ {2} \mathbf {k} _ {*} ^ {(0)} (\mathbf {x} ^ {(1)}) \otimes \mathbf {S} ^ {(0)} + \mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) \boldsymbol {\Sigma} ^ {- 1} \left( \begin{array}{c} \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) ^ {\top} \\ \rho^ {2} \left(\mathbf {k} _ {*} ^ {(0)} (\mathbf {x} ^ {(1)}) \otimes \mathbf {S} ^ {(0)}\right) ^ {\top} + \left(\mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {\top} \end{array} \right) \\ = \left(\rho^ {2} \mathbf {k} _ {*} ^ {(0 0)} \otimes \mathbf {S} ^ {(0)} + \mathbf {k} _ {a *} ^ {(1 1)} \otimes \mathbf {S} ^ {(1)}\right) - \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) \left(\mathbf {K} ^ {(0 0)} \otimes \mathbf {S} ^ {(0)}\right) ^ {- 1} \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) ^ {\top} \\ + \left(\mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) \left(\rho \mathbf {K} _ {a} ^ {(1 1)} \otimes \mathbf {S} ^ {(1)}\right) \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right) ^ {\top} - \left(\mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) \left(\rho \mathbf {K} _ {a} ^ {(1 1)} \otimes \mathbf {S} ^ {(1)}\right) \left(\rho \mathbf {k} _ {*} ^ {(0)} \otimes \mathbf {S} ^ {(0)}\right). \\ - \left(\mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) \left(\mathbf {K} _ {a} ^ {(1 1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \left(\mathbf {k} _ {a *} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {\top} \\ = \rho^ {2} \left(\mathbf {k} _ {*} ^ {(0 0)} - \left(\mathbf {k} _ {*} ^ {(0)}\right) ^ {\top} \left(\mathbf {K} ^ {(0 0)}\right) ^ {- 1} \mathbf {k} _ {*} ^ {(0)}\right) \otimes \mathbf {S} ^ {(0)} + \left(\mathbf {k} _ {a *} ^ {(1 1)} - \left(\mathbf {k} _ {a *} ^ {(1)}\right) ^ {\top} \left(\mathbf {K} _ {a} ^ {(1 1)}\right) ^ {- 1} \mathbf {k} _ {a *} ^ {(1)}\right) \otimes \mathbf {S} ^ {(1)}, \tag {A.32} \\ \end{array}
$$
where  $\mathbf{k}_{*}^{(0)} = k^{0}(\mathbf{X}_{*},\mathbf{X}^{(0)})$ $\mathbf{k}_{*}^{(00)} = k^{0}(\mathbf{X}_{*},\mathbf{X}_{*})$ $\mathbf{K}^{(00)} = k^0 (\mathbf{X}^{(0)},\mathbf{X}^{(0)})$  denotes the covariance in the lowest fidelity, and  $\mathbf{k}_{a*}^{(1)} = k^{u_1}(\mathbf{X}_*,\mathbf{X}^{(1)})$ $\mathbf{k}_{*}^{(11)} = k^{u_{1}}(\mathbf{X}_{*},\mathbf{X}_{*})$  and  $\mathbf{K}_a^{(11)} = k^{u_1}(\mathbf{X}^{(1)},\mathbf{X}^{(1)})$
Notice that Eq. (A32) decomposes the predictive posterior into two parts one related to the low-fidelity data and the other to the high-fidelity data. This lays the foundation for our later derivation for the non-subset structure. In the following section, we will derive the predictive posterior for non-subset structure, and show that the autokrigeability also holds.
H Predictive Posterior With Non-Subset Structure
In this section, we derive the mean function and covariance matrix in the predictive posterior for non-subset structure following the previous two-fidelity setup.
$$
\begin{array}{l} p (\bar {\mathbf {Y}} (\mathbf {x} _ {*}, 2) | \mathbf {Y} ^ {(2)}, \mathbf {Y} ^ {(1)}) = \int p (\bar {\mathbf {Y}} (\mathbf {x} _ {*}, 2), \hat {\mathbf {Y}} ^ {(1)} | \mathbf {Y} ^ {(2)}, \mathbf {Y} ^ {(1)}) d \hat {\mathbf {Y}} ^ {(1)} \\ = \int \overbrace {p \left(\bar {\mathbf {Y}} \left(\mathbf {x} _ {*}, 2\right) \mid \mathbf {Y} ^ {(2)} , \mathbf {Y} ^ {(1)} , \hat {\mathbf {Y}} ^ {(1)}\right)} ^ {\text {f i d e l i t y - 2 p o s t e r i o r}} \overbrace {p \left(\hat {\mathbf {Y}} ^ {(1)}\right)} ^ {\text {f i d e l i t y - 1 p o s t e r i o r}} d \hat {\mathbf {Y}} ^ {(1)}. \tag {A.33} \\ \end{array}
$$
Figure 7: Illustration for the selection matrix and multi-fidelity data structure. Let  $\mathbf{X}^{(t)}$  be the available system inputs for  $t$  fidelity. The inputs for  $t$  fidelity will be two parts: the subset part  $\check{\mathbf{X}}^{(t)}$  contained in  $\mathbf{X}^{(t-1)}$ , and the part that is not contained in  $\mathbf{X}^{(t-1)}$  is denoted as  $\hat{\mathbf{X}}^{(t-1)}$  (where the hat and superscript indicate that it is a complement set for the  $t-1$  fidelity). To extract these two parts, we define  $\tilde{\mathbf{X}}^{(t)} = \mathbf{E}^{(t)}\mathbf{X}^{(t-1)}$  and  $\hat{\mathbf{X}}^{(t-1)} = \hat{\mathbf{E}}^{(t)}\mathbf{X}^{(t)}$ .
As we know, once the  $\hat{\mathbf{Y}}^{(1)}$  is decided, the high-fidelity posterior part can be written as the subset posterior distribution, in the following way,
$$
p \left(\bar {\mathbf {Y}} \left(\mathbf {x} _ {*}, 2\right) \mid \mathbf {Y} ^ {(2)}, \mathbf {Y} ^ {(1)}, \hat {\mathbf {Y}} ^ {(1)}\right) = 2 \pi^ {- \frac {N _ {p} D}{2}} \times \left| \boldsymbol {\Sigma} _ {*} ^ {(2)} \right| ^ {- \frac {1}{2}} \times \exp - \frac {1}{2} \left[ \left(\mathbf {y} _ {*} ^ {(2)} - \bar {\mathbf {y}} ^ {(2)}\right) ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(2)}\right) ^ {- 1} \left(\mathbf {y} _ {*} ^ {(2)} - \bar {\mathbf {y}} ^ {(2)}\right) \right]. \tag {A.34}
$$
$$
\boldsymbol {\Sigma} _ {*} ^ {(2)} = \boldsymbol {\Sigma} (\mathbf {x} _ {*}, 1) + \tilde {\boldsymbol {\Sigma}} ^ {(2)},
$$
$$
\bar {\mathbf {y}} ^ {(2)} = \left[ \left(\mathbf {k} _ {*} ^ {(1 1)}\right) \left(\mathbf {K} ^ {(1 1)}\right) ^ {- 1} \otimes \mathbf {I} \right] \left( \begin{array}{l} \mathbf {Y} ^ {(1)} \\ \hat {\mathbf {y}} ^ {(1)} \end{array} \right) + \left[ \left(\mathbf {k} _ {a *} ^ {(2 2)}\right) \left(\mathbf {K} _ {a} ^ {(2 2)}\right) ^ {- 1} \otimes \mathbf {I} \right] \mathbf {y} ^ {(2)}. \tag {A.35}
$$
This is depends on the conclusion in Eq. (A31).  $\pmb{\Sigma}_{*}^{(2)}$  and  $\bar{\mathbf{y}}^{(2)}$  are posterior covariance matrix and mean function.
At the same time, we know that  $\hat{\mathbf{Y}}^{(1)}$  are samples from fidelity 1 model, therefore the probability of getting  $\hat{\mathbf{Y}}^{(1)}$  can be written as:
$$
p \left(\hat {\mathbf {Y}} ^ {(1)}\right) = 2 \pi^ {- \frac {N _ {n} ^ {(2)} D}{2}} \times \left| \boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)} \right| ^ {- \frac {1}{2}} \times \exp \left[ - \frac {1}{2} \left(\hat {\mathbf {y}} ^ {(1)} - \bar {\mathbf {y}} ^ {(1)}\right) ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \left(\hat {\mathbf {y}} ^ {(1)} - \bar {\mathbf {y}} ^ {(1)}\right) \right], \tag {A.36}
$$
where the  $\pmb{\Sigma}_{*}^{(1)}\otimes \mathbf{S}$  means the posterior covariance matrix of sampling  $\hat{\mathbf{Y}}^{(1)}$  from lower fidelity model and  $\bar{\mathbf{y}}^{(1)}$  are predicted mean for the non-subset data. Therefore, the posterior distribution of non-subset data structure is
where  $\tilde{\mathbf{y}}$  and  $\Gamma$  is defined by the following equations,
$$
\tilde {\mathbf {y}} = \mathbf {y} _ {*} ^ {(2)} - \rho \left[ \left(\mathbf {k} _ {a *} ^ {(1 1)}\right) \left(\mathbf {K} _ {a} ^ {(1 1)}\right) ^ {- 1} \otimes \mathbf {I} \right] \left( \begin{array}{c} \vec {\mathbf {y}} ^ {(1)} \\ \mathbf {0} \end{array} \right) - \left[ \left(\mathbf {k} _ {a *} ^ {(2 2)}\right) \left(\mathbf {K} _ {a} ^ {(2 2)}\right) ^ {- 1} \otimes \mathbf {I} \right] \left[ \mathbf {y} ^ {(2)} - \left( \begin{array}{c} \breve {\mathbf {y}} ^ {(1)} \\ \mathbf {0} \end{array} \right) \right],
$$
$$
\boldsymbol {\Gamma} = \left[ \mathbf {k} _ {a *} ^ {(2, 2)} \left(\mathbf {K} _ {a} ^ {(2, 2)}\right) ^ {- 1} \left(\mathbf {E} _ {n} ^ {(2)}\right) ^ {\top} \tilde {\mathbf {k}} _ {a *} ^ {(1, 1)} \left(\tilde {\mathbf {K}} _ {a} ^ {(1, 1)}\right) ^ {- 1} \right] \left(\mathbf {E} _ {m} ^ {(2)}\right). \tag {A.38}
$$
where  $\mathbf{E}_m^{(2)}$  denotes the selection matrix which selects the non-subset parts between two fidelities.  $\hat{\mathbf{X}}^{(1)} = \hat{\mathbf{X}}^{(1)} = \left(\mathbf{E}_m^{(2)}\right)^\top \left[\mathbf{X}^{(1)},\hat{\mathbf{X}}^{(1)}\right]$  and  $\mathbf{E}_n^{(2)}$  also denotes the selection matrix but which selects all inputs for the second fidelity,  $\mathbf{X}^{(2)} = \left(\mathbf{E}_n^{(2)}\right)^\top \left[\mathbf{X}^{(1)},\hat{\mathbf{X}}^{(1)}\right]$ . Please see Fig. 7 for the illustration of the selection matrix for the multi-fidelity data structure.
After that, we can simplify the determinant and the exponential parts by decomposing them into different parts and using the Sherman-Morrison formula to obtain conclusions.
For the determinant part, we can derive
$$
\begin{array}{l} \left| \boldsymbol {\Sigma} _ {*} ^ {(2)} \right| ^ {- \frac {1}{2}} \times \left| \boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)} \right| ^ {- \frac {1}{2}} \times \left| \boldsymbol {\Gamma} ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(2)}\right) ^ {- 1} \boldsymbol {\Gamma} + \boldsymbol {\Sigma} _ {*} ^ {(1)} \right| ^ {- \frac {1}{2}} \\ = \left| \boldsymbol {\Sigma} _ {*} ^ {(2)} \right| ^ {- \frac {1}{2}} \times \left| \boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)} \right| ^ {- \frac {1}{2}} \times \left| \left(\boldsymbol {\Sigma} _ {*} ^ {(2)}\right) ^ {- 1} \right| ^ {- \frac {1}{2}} \times \left| \left(\boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \right| ^ {- \frac {1}{2}} \times \left| \boldsymbol {\Sigma} _ {*} ^ {(2)} + \boldsymbol {\Gamma} ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \boldsymbol {\Gamma} \right| ^ {- \frac {1}{2}} \\ = \left| \boldsymbol {\Sigma} _ {*} ^ {(2)} + \boldsymbol {\Gamma} ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \boldsymbol {\Gamma} \right| ^ {- \frac {1}{2}}. \tag {A.39} \\ \end{array}
$$
For the exponential parts in Eq. (A37),
$$
\left(\boldsymbol {\Sigma} _ {*} ^ {(2)}\right) ^ {- 1} - \left(\boldsymbol {\Sigma} _ {*} ^ {(2)}\right) ^ {- 1} \boldsymbol {\Gamma} \left(\boldsymbol {\Gamma} ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(2)}\right) ^ {- 1} \boldsymbol {\Gamma} + \boldsymbol {\Sigma} _ {*} ^ {(1)}\right) ^ {- 1} \boldsymbol {\Gamma} ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(2)}\right) ^ {- 1} = \left(\boldsymbol {\Sigma} _ {*} ^ {(2)} + \boldsymbol {\Gamma} ^ {\top} \left(\boldsymbol {\Sigma} _ {*} ^ {(1)} \otimes \mathbf {S} ^ {(1)}\right) ^ {- 1} \boldsymbol {\Gamma}\right) ^ {- 1}
$$