Adaptive Transfer Kernel Learning for Transfer Gaussian Process Regression

Transfer regression is a practical and challenging problem with important applications in various domains, such as engineering design and localization. Capturing the relatedness of different domains is the key of adaptive knowledge transfer. In this paper, we investigate an effective way of explicitly modelling domain relatedness through transfer kernel, a transfer-specified kernel that considers domain information in the covariance calculation. Specifically, we first give the formal definition of transfer kernel, and introduce three basic general forms that well cover existing related works. To cope with the limitations of the basic forms in handling complex real-world data, we further propose two advanced forms. Corresponding instantiations of the two forms are developed, namely <inline-formula><tex-math notation="LaTeX">${Trk}_{\alpha \beta }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow><mml:mi>T</mml:mi><mml:mi>r</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>α</mml:mi><mml:mi>β</mml:mi></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq1-3219121.gif"/></alternatives></inline-formula> and <inline-formula><tex-math notation="LaTeX">${Trk}_{\omega }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow><mml:mi>T</mml:mi><mml:mi>r</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mi>ω</mml:mi></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq2-3219121.gif"/></alternatives></inline-formula> based on multiple kernel learning and neural networks, respectively. For each instantiation, we present a condition with which the positive semi-definiteness is guaranteed and a semantic meaning is interpreted to the learned domain relatedness. Moreover, the condition can be easily used in the learning of <italic>TrGP</italic><inline-formula><tex-math notation="LaTeX">$_{\alpha \beta }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow/><mml:mrow><mml:mi>α</mml:mi><mml:mi>β</mml:mi></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq3-3219121.gif"/></alternatives></inline-formula> and <italic>TrGP</italic><inline-formula><tex-math notation="LaTeX">$_{\omega }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow/><mml:mi>ω</mml:mi></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq4-3219121.gif"/></alternatives></inline-formula> that are the Gaussian process models with the transfer kernels <inline-formula><tex-math notation="LaTeX">${Trk}_{\alpha \beta }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow><mml:mi>T</mml:mi><mml:mi>r</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>α</mml:mi><mml:mi>β</mml:mi></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq5-3219121.gif"/></alternatives></inline-formula> and <inline-formula><tex-math notation="LaTeX">${Trk}_{\omega }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow><mml:mi>T</mml:mi><mml:mi>r</mml:mi><mml:mi>k</mml:mi></mml:mrow><mml:mi>ω</mml:mi></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq6-3219121.gif"/></alternatives></inline-formula> respectively. Extensive empirical studies show the effectiveness of <italic>TrGP</italic><inline-formula><tex-math notation="LaTeX">$_{\alpha \beta }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow/><mml:mrow><mml:mi>α</mml:mi><mml:mi>β</mml:mi></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq7-3219121.gif"/></alternatives></inline-formula> and <italic>TrGP</italic><inline-formula><tex-math notation="LaTeX">$_{\omega }$</tex-math><alternatives><mml:math><mml:msub><mml:mrow/><mml:mi>ω</mml:mi></mml:msub></mml:math><inline-graphic xlink:href="wei-ieq8-3219121.gif"/></alternatives></inline-formula> on domain relatedness modelling and transfer adaptiveness.