Transfer learning has yielded state-of-the-art  results in many supervised natural language processing tasks. However, annotated data for every target task in every target language is rare, especially for low-resource languages. In this work, we propose MultiMix, a novel data augmentation method for semi-supervised learning in zero-shot transfer learning scenarios. In particular, MultiMix targets to solve cross-lingual adaptation problems from a source (language ) distribution to an unknown target (language ) distributionassuming it has no training labels in the target language task. In its heart , MultiMix performs simultaneous self-training with data augmentation and unsupervised sample selection. To show its effectiveness, we have performed extensive experiments on zero-shot transfers for cross-lingual named entity recognition (XNER) and natural language inference (XNLI). Our experiments show sizeable improvements in both tasks  outperforming the baselines by a good margin  .
Transfer learning has yielded state-of-the-art (SoTA) results in many supervised natural language processing tasks. However, annotated data for every target task in every target language is rare, especially for low-resource languages. We propose MultiMix, a novel data augmentation framework for self-supervised learning in zero-resource transfer learning scenarios. In particular, MultiMix targets to solve cross-lingual adaptation problems from a source language distribution to an unknown target language distribution, assuming no training labels are available for the target language task. At its core , MultiMix performs simultaneous self-training with data augmentation and unsupervised sample selection. To show its effectiveness, we conduct extensive experiments on zero-resource cross-lingual transfer tasks for Named Entity Recognition and Natural Language Inference. MultiMix achieves SoTA results in both tasks , outperforming the baselines by a good margin . With an in-depth model dissection, we demonstrate the cumulative contributions of different components to MultiMix's success .
A
(SoTA)
R
In this work, we
We
R
method for semi-supervised learning in zero-shot
framework for self-supervised learning in zero-resource
R
(language )
language
R
(language ) distributionassuming it has
language distribution, assuming
R
in
are available for
R
In its heart
At its core
R
have performed
conduct
R
zero-shot transfers for
zero-resource
R
named entity recognition (XNER) and natural language inference (XNLI). Our experiments show sizeable improvements
transfer tasks for Named Entity Recognition and Natural Language Inference. MultiMix achieves SoTA results
A
,
A
. With an in-depth model dissection, we demonstrate the cumulative contributions of different components to MultiMix's success