Title: Wasserstein Transfer Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: data aggregation

multitask learning

non-Euclidean output

optimal transport

Wasserstein metric
Abstract: Transfer learning is a powerful paradigm for leveraging knowledge from source domains to enhance learning in a target domain. However, traditional transfer learning approaches often focus on scalar or multivariate data within Euclidean spaces, limiting their applicability to complex data structures such as probability distributions. To address this, we introduce a novel framework for transfer learning in regression models, where outputs are probability distributions residing in the Wasserstein space. When the informative subset of transferable source domains is known, we propose an estimator with provable asymptotic convergence rates, quantifying the impact of domain similarity on transfer efficiency. For cases where the informative subset is unknown, we develop a data-driven transfer learning procedure designed to mitigate negative transfer. The proposed methods are supported by rigorous theoretical analysis and are validated through extensive simulations and real-world applications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an approach to transfer learning called Wasserstein Transfer Learning, designed for regression models where the outputs are probability distributions. \nThis paper introduces two algorithms: one for scenarios where the informative subset of source domains (the available source domains that are similar enough to the target one to yield relevant information for the learning process) is known, and another adaptive algorithm for cases where this subset is unknown which includes a way of identifying this subset. \nA rigorous theoretical analysis is given, establishing asymptotic convergence rates of the estimators. \nThe methodology is validated through simulations and real-world applications. \nThe results show that WaTL outperforms methods using only target or only source data.\nOverall, the paper presents a new approach on transfer learning by addressing the challenges of regression models with distributional outputs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1- The paper tackles an interesting issue: transfer learning for probability distributions output. \n\n2-The theoretical contributions are thorough : convergence rates for the learnt estimators and guidelines for the choice of the regularization parameter are given in section 4.\n\n3- In section 3.3 and Algorithm 2 an adaptative selection procedure to identify informative subset of the source data is given and its ability to find those subset is illustrated in the numerical experiments of section 5.\n\nWeaknesses:\n\n1- The paper deserves more pedagogical clarification. Due to the focus on mathematical content some part of the methodology lack motivation/explanation. For example in section 3 in the algorithms the step that connects the source and target data via regularization (step 2 of Algorithm 1, step 3 of Algorithm 3) are only explained in two sentences while that are at the core of the transfer learning objective.\n\n2- While the problem tackled is explained, it is not clear how this paper fits into the existing literature on Wasserstein (and optimal transport) methods in transfer learning (and domain adaptation).\n\n3- Some key aspects are not defined early for example the term : \"informative subset of transferable source domains\" is used multiple times before being explained.\n\n4- The assumption \"all source data are informative enough.\" is not enough detailed.\nEven if I agree that this kind of assumption is common un transfer learning/domain adaptation, but it can be formalized (and sometimes estimated from samples) by explicitly defined a measure of distance between the distributions (see for example, On the analysis of adaptability in multi-source domain adaptation, Redko et et. 2019) \n\n5- In the empirical study there is no comparison with a baseline that would be computed on both the source and target data at the same time. It would be useful to compare it with the learnt estimator since the transfer learning estimator uses labeled target datas anyway.\n\nQUESTIONS:\nQuestions:\n\na. Is the transfer learning learnt estimator what would happen if an estimator was learnt using directly every available data, source and target at the same time? How worse would it be compared to the transfer learning one ?\n\nb. In the experiments what is the reason not to use the second algorithm with informative subset selection was used ? Which groups would be selected/rejected and why?\n\nc. Could the authors give more details on how their work connects with existing literature on transfer learning and the use of Wasserstein?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose a framework, called Wasserstein transfer learning (WaTL), for transfer learning in regression models, with complex data structures. Specifically, WaTL can well hand the cases where the informative subset of transferable source domains is known or unknown.  Experiments on numerical simulations and real-world applications are done to verify the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1.The paper introduces a Wasserstein transfer learning for transfer regression setting. \n\n2.WaTL algorithms for cases where the informative subset of transferable source domains is known, or unknown are introduced.\n\n3.Asymptotic convergence rates for the WaTL algorithm in the above two cases are presented.\n\n4.Empirical studies are done, and well support the effectiveness of the proposed method.\n\nWeakness:\n\n1.The paper is not well written and hard to follow. There are quite a lot of equations written in lines, which easily confused the reader. Moreover, a clear notation table is needed for better understanding of the algorithms.\n\n2.The motivation is not clear. See detailed questions below.\n\n3.The related works are insufficient. Wasserstein distance is a widely used technique in Transfer learning, but related works are not included. Moreover, transfer regression is a well-define transfer setting, and there are many methods, e.g., transfer Gaussian process, proposed for this setting, but they are not well discussed in this paper.\n\n4.The comparison baselines can be further improved by including more transfer regression methods, e.g. transfer Gaussian process.\n\nQUESTIONS:\n1.The authors motivate the method by stating the growing interest in extending transfer learning to more complex data structure but fail to give concrete examples. What are the examples of complex data structure, how the data structure complexity presents new challenges to existing transfer learning methods? Moreover, in second paragraph, the authors specifically use samples of univariate probability distributions, which, to this reviewer, is not considered as “more complex data structure” than multimedia data, e.g., images or videos. \n\n2.The line 28-29, it is not clear how this gap affects the effectiveness of the corresponding transfer learning methods. Moreover, it seems that it is restricted to methods using Wasserstein space, while the comparison with other transfer regression methods, e.g., transfer gaussian process, is not well discussed.\n\n3.Due to negative transfer, AwaTL is a more practical algorithm. The current way of measuring ‘relatedness’ or ‘discrepancy’ through AwaTl is a deterministic way of calculating $||f_0(x) -f_k(x)||$, the reviewer wonders if it can be done by using some learnable parameters? \n\n4.How the number of target labelled instances affect the performance of AwaTL?\n\n5.Time series extrapolation is a typical transfer regression setting. It is encouraged to include empirical results on such tasks.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

comments: SUMMARY:
The paper studies transfer learning in regression tasks where the response is a distribution in Wasserstein space. The goal is to utilize relevant source samples to achieve a small Wasserstein error. The paper proposes two algorithms:

- WaTL (Wasserstein Transfer Learning), which assumes that all source domains are informative.

- AWaTL (Adaptive WaTL), which selects a subset of informative source domains in a data-driven manner.

The authors provide theoretical guarantees for the rate of convergence, corroborated by some experimental results.

STRENGTHS AND WEAKNESSES:
Strengths:

 - The paper studies a new setting where the responses are probability distributions, a setting not covered by the existing transfer learning literature.

- It provides finite-sample error bounds that strictly improve upon target-only Fréchet regression when the sources are sufficiently close.

Weaknesses and limitations:

- The setting assumes that each response distribution is known in its entirety, whereas in practice we usually observe only finite samples. Although this issue is discussed in the appendix, there is no formal result for the case where the distributions are not observed directly, but only through samples.

- The rates are compared only to vanilla Fréchet regression, and no minimax lower bound is provided.

- The results are limited to univariate distributions in Wasserstein space.

QUESTIONS:
What is the main obstacle to extending the current results to the multivariate case? Would the rates change in this extended setting?

LIMITATIONS:
The paper does not have any potential negative societal impact.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes WaTL, a novel transfer learning framework designed for regression models where the outputs are probability distributions residing in the Wasserstein space. The core of the method builds on Fréchet regression: it first constructs a weighted estimator using both source and target domain data, then corrects for distribution shift bias using the target domain data, and finally projects the estimator back into the Wasserstein space. To mitigate negative transfer, the authors also introduce an extended variant of WaTL that performs a pre-selection of source domain datasets based on a discrepancy score, selecting only those most similar to the target domain. The paper provides a theoretical convergence rate analysis of WaTL and validates its performance on both synthetic and real-world datasets, demonstrating promising empirical results.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths\n1. The paper tackles an important and underexplored area in transfer learning, i.e., regression models where the outputs are probability distributions in Wasserstein space. This non-Euclidean setting naturally arises in many modern applications. The motivation for addressing distribution shift in such geometric spaces is well-justified, and the paper fills a noticeable gap in the literature.\n2. The paper is well-written, with a clear structure and precise technical presentation. Notation is generally well-defined, and the flow of ideas is easy to follow. The balance between theoretical depth and practical applicability is well-maintained, making the paper accessible to both theoretical and applied audiences.\n\n- Weaknesses\n1. WaTL requires training a separate estimator for each source domain. When multiple source domains are needed, this could become computationally intensive, especially if training each estimator is expensive.\n2. The method assumes access to data that sufficiently captures full probability distributions, which may not always be feasible in real-world applications.\n\nQUESTIONS:\n1. Instead of training a new estimator for each source domain from scratch, could WaTL be adapted to use existing pre-trained models to reduce computational overhead?\n2. Have the authors considered using RKHS-based metrics such as MMD in place of the Wasserstein distance? It would be helpful to discuss the trade-offs and motivations for choosing Wasserstein geometry over kernel-based alternatives.\n3. The experimental comparison is limited to models trained only on the source or only on the target domain. A more informative baseline might include a model trained on combined (source + target) data, to better investigate WaTL’s performance improvements.\n4. The projection back into the Wasserstein space is an essential step in WaTL. Could the authors elaborate on whether this step is easy to compute?\n5. A brief clarification or context would be helpful for the notation E in line 80 and later.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}