Extrapolation and Latent-Variable Identification
Given the labeled source distribution p(\mathbf{x}_{\mathrm{src}},\mathbf{y}_{\mathrm{src}}) , our goal is to make predictions on a target sample \mathbf{x}_{\mathrm{tgt}} outside the source support ( \mathbf{x}_{\mathrm{tgt}}\not\in\mathcal{X}_{\text{src}} ). While more target samples would provide better information about the distribution shift, in practice, we often have only a handful of samples to work with. Therefore, we focus on the challenging scenario where only one target sample \mathbf{x}_{\mathrm{tgt}} is available.

Making reliable predictions on out-of-support samples \mathbf{x}_{\mathrm{tgt}} is infeasible without additional structure.
Real-world problems where humans successfully extrapolate often follow a minimal change principle: they involve sparse, non-semantic intrinsic shifts despite complex raw data changes.
For example, a person who has only seen a cow on a pasture can recognize the same cow on a beach, even if the background pixels change significantly.
Here, the cow corresponds to the part of the latent variable that remains within the support of the source data, which we call the invariant variable \mathbf{c} ( \mathbf{c}_{\mathrm{tgt}}\in\mathcal{C}_{\text{src}} ), while the background change corresponds to the complement that drifts off the source support, which we call the changing variable \mathbf{s} ( \mathbf{s}_{\mathrm{tgt}}\notin\mathcal{S}_{\text{src}} ).
Clearly, extrapolation is impossible if the intrinsic shift is dense (i.e., all dimensions change, \mathbf{z}=\mathbf{s} ) or semantic (i.e., \mathbf{y} is a function of \mathbf{s} ). For instance, if the variable \mathbf{s} also alters the cow’s appearance drastically, making it unrecognizable, extrapolation fails. We define the data-generating process to encapsulate this minimal change principle, as follows:

\mathbf{c}\sim p(\mathbf{c}),\;\mathbf{s}\sim p(\mathbf{s}|\mathbf{c}); (1) \mathbf{x}=g(\mathbf{z}),\;\mathbf{y}=g_{\mathbf{y}}(\mathbf{c}).

In this process, the latent space \mathbf{z}\in\mathcal{Z}\subset\mathbb{R}^{d_{\mathbf{z}}} comprises two subspaces: the invariant variable \mathbf{c}\in\mathcal{C}\subset\mathbb{R}^{d_{\mathbf{c}}} and the changing variable \mathbf{s}\in\mathcal{S}\subset\mathbb{R}^{d_{\mathbf{s}}} .
We define \mathcal{Z}:=\mathcal{Z}_{\mathrm{src}}\cup\{\mathbf{z}_{\mathrm{tgt}}\} as the source support augmented with the target sample \mathbf{z}_{\mathrm{tgt}} and similarly \mathcal{X} and \mathcal{S} .
The invariant variable \mathbf{c} encodes shared information between the source distribution p(\mathbf{x}_{\mathrm{src}}) and the out-of-support target sample \mathbf{x}_{\mathrm{tgt}} , while the changing variable \mathbf{s} describes the shift from the source support \mathcal{X}_{\mathrm{src}} . Hence, \mathbf{c}_{\mathrm{tgt}}\in\mathcal{C}_{\text{src}} and \mathbf{s}_{\mathrm{tgt}}\notin\mathcal{S}_{\text{src}} . The variables \mathbf{z}:=[\mathbf{c},\mathbf{s}] jointly generate the observed variable \mathbf{x}\in\mathcal{X}\subset\mathbb{R}^{d_{\mathbf{x}}} through an invertible generating function g:\mathbb{R}^{d_{\mathbf{z}}}\to\mathbb{R}^{d_{\mathbf{x}}} .
Furthermore, we assume that the label \mathbf{y} originates from the invariant variable \mathbf{c} .
This assumption reflects the reality that factors such as camera angles and lighting do not affect the object’s class in an image.

Our latent-variable model adheres to the minimal change principle in two key ways: (1) the target sample’s out-of-support nature arises from only a subset of latent variables \mathbf{s} , and (2) these changing variables \mathbf{s} are non-semantic, thus not influencing the label \mathbf{y} .