Paper:
Extracting Relationships by Multi-Domain Matching
In many biological and medical contexts, we construct a large labeled corpus by aggregating many sources to use in target prediction tasks. Unfortunately, many of the sources may be irrelevant to our target task, so ignoring the structure of the dataset is detrimental. This work proposes a novel approach, the Multiple Domain Matching Network (MDMN), to exploit this structure. MDMN embeds all data into a shared feature space while learning which domains share strong statistical relationships. These relationships are often insightful in their own right, and they allow domains to share strength without interference from irrelevant data. This methodology builds on existing distribution-matching approaches by assuming that source domains are varied and outcomes multi-factorial. Therefore, each domain should only match a relevant subset. Theoretical analysis shows that the proposed approach can have a tighter generalization bound than existing multiple-domain adaptation approaches. Empirically, we show that the proposed methodology handles higher numbers of source domains (up to 21 empirically), and provides state-of-the-art performance on image, text, and multi-channel time series classification, including clinical outcome data in an open label trial evaluating a novel treatment for Autism Spectrum Disorder.

Review:
Title: Extracting Relationships by Multi-Domain Matching  Summary  Assuming that a corpus is compiled from many sources belonging to different to domains, of which only a strict subset of domains is suitable to learn how to do prediction in a target domain, this paper proposes a novel approach (called Multiple Domain Matching Network (MDMN)) that aims at learning which domains share strong statistical relationships, and which source domains are best at supporting to learn the target domain prediction tasks.  While many approaches to multiple-domain adaptation aim to match the feature-space distribution of *every* source domain to that of the target space, this paper suggests to not only map the distribution between sources and target, but also *within* source domains. The latter allows for identifying subsets of source domains that share a strong statistical relationship.  Strengths  Paper provides a theoretical analysis that yields a tighter bound on the weighted multi-source discrepancy.  Approach yields state-of-the-art performance on image, text and multi-channel time series classification tasks.  Weaknesses  Tighter bound on multi-source discrepancy depends on the assumption that source domains that are less relevant for the target domain have lower weights. While intuitively, this may seem obvious, there is no guarantee that in practice, the irrelevant source domains can reliably be identified.  No commitment that source code may get released.  Questions  L 96: Is it intended that the sum runs over all domains but s, including the target domain S? L120: Why is the Wasserstein distance not affected by a varying feature scale in practice?  There is a shift in notation in Section 3 where the target domain is not denoted by T while the source domains are denoted by s=1...S. In Section 2, the source domains were defined as s=1,...,S-1 while the single target domain was defined as S.  Theorem 3.3 shows that weighting yields a tighter bound given that irrelevant domains are assigned small weights. However, what happens if the algorithm fails to assign small weights to irrelevant domains or, in the most adverse case, if the least relevant domains get assigned the highest weights? More general: for which weight distributions does Theorem 3.3 provide tighter bounds?  To what number of source domains does the provided method scale? A total of 21 domains may still be a small number if this method were to be applied to other tasks. What potential does MDMN have to be extended or applied to fuzzy domains, i.e., where the source data set does not induce a canonical partitioning into domains?  Comments  Editorial Notes L 36 is be -> is L 219 develop -> development L 287 An -> A L 320 [add reference] L 321 domains is large L 342 state-of-the0art -> state-of-the-art  ---------------  Thanks to the authors for their detailed responses and addressing my questions. I am looking forward to the release of the code.