Fréchet Regression

As data types are becoming more complex, attention has turned to regression in more abstract settings. A setting that is increasingly encountered is that of a response variable taking values in a metric space. Examples of such data include Covariance matrix, Graph (discrete mathematics) and Probability distribution function. The presence of a metric provides a natural connection to the work of Fréchet, where the Fréchet mean is defined for Random element of a metric space as a direct generalization of the standard mean. As Regression analysis can be viewed as the modeling of Conditional expectation, a key feature of Fréchet regression is the generalization of the classical Fréchet mean to the conditional Fréchet mean. Recently, regression is not technical at all due to its versatility and importance. Fréchet regression is just the extension of the classical regression to the complex data.
== Definition ==
Let (\Omega,d) be a metric space. Denote a stochastic process (X,Y)\sim F, where X and Y take values in \mathcal{R}^{p} and \Omega, respectively, and F is the joint distribution of (X,Y) on \mathcal{R}^{p}\times \Omega. Let F_{X} and F_{Y} be the marginal distribution of X and Y. It is assumed that \mu=\mathrm{E}(X), \Sigma=\text{Var}(X), Conditional probability distribution F_{X|Y} and F_{Y|X} exist, with \Sigma Definite matrix.
=== Fréchet mean, Fréchet variance ===
The notions of mean and variance were generalized to random objects of a metric space by Fréchet.
The Fréchet mean is defined as w_{\oplus} = \text{argmin}_{w\in\Omega}E(d^{2}(Y,w)), and the Fréchet variance is V_{\oplus} = E(d^{2}(Y,w_{\oplus})).
=== Global Fréchet regression ===
The global Fréchet regression m_{\oplus}(x) on an arbitrary metric space (\Omega,d) is defined as
m_{\oplus}(x) := \text{argmin}_{w \in \Omega}M(w,x), M(\cdot,x) = E[s(X,x)d^{2}(Y,\cdot)], (1) 
where the weight function s is 
 s(z,x) = 1+(z-\mu)^{T}\Sigma^{-1}(x-\mu). (2)
=== Local Fréchet regression ===
Consider the case of a scalar predictor X \in \mathcal{R}^{p}, where p=1 for ease of representation. This method can be generalized for any p with p=1. Denote a Kernel (linear algebra) function K and Bandwidth (computing) as h with the scaled kernel K_{h}(\cdot)=h^{-1}K(\cdot/h). Let \mu_{j} = E[K_{h}(X-x)(X-x)^{j}], r_{j} = E[K_{h}(X-x)(X-x)^{j}Y]and \sigma_{0}^{2}=\mu_{0}\mu_{2}-\mu_{1}^{2}. 
The local Fréchet regression l_{\oplus}(x) is defined as
l_{\oplus}(x) := \text{argmin}_{w \in \Omega}L_{n}(w,x), L_{n}(w) = E[s(X,x,h)d^{2}(Y,w)], (3) 
where the weight function s is
 s(z,x,h) = \sigma_{0}^{-2}\{K_{h}(z-x)[\mu_{2}-\mu_{1}(z-x)]\}. (4)
== Examples ==
=== Regression for probability distributions with the Wasserstein metric ===
Let the space \Omega be the set of distribution functions equipped with the Wasserstein metric. Consider a sample (X_{i},Y_{i}),i=1,\ldots,n on \Omega. Set \hat{g}_{x}=n^{-1}\Sigma_{i=1}^{n}s_{in}(x)Q(Y_{i}), where the weights s_{in}(x) in (5) and Q(w) to be the quantile function corresponding to w, for any w \in \Omega. Since \hat{g}_{x} \in L^{2}[0,1], the global Fréchet regression estimator is
 \hat{m}_{\oplus}(x)=\text{argmin}_{w\in\Omega}d^{2}_{L^{2}}(\hat{g}_{x},Q(w))=Q^{-1}(\text{argmin}_{q\in Q(\Omega)}d^{2}_{L^{2}}(\hat{g}_{x},q)),
where d_{L^{2}} is the standard Lp space| L^{2} metric.
=== Regression for correlation matrices with the Frobenius metric ===
Consider a space of random objects \Omega which consists of Correlation matrices with Frobenius inner product d_{F}. From a sample (X_{i},Y_{i}),i=1,\ldots,n the minimization in (6) is reformulated by setting \hat{B}(x)=n^{-1}\Sigma_{i=1}^{n}s_{in}(x)Y_{i}, where the weights s_{in}(x) in (5), and computing
 \hat{m}_{\oplus}(x)=\text{argmin}_{w\in\Omega}d_{F}(\hat{B}(x),w). 
Therefore, this problem is reduced to finding the correlation matrix which is nearest to the matrix \hat{B}(x), which has been well studied.
Category:Regression analysis
Category:Regression models