We propose an ensemble model for predictingthe lexical complexity of words and multiwordexpressions (MWEs). The model receives asinput a sentence with a target word or MWEand outputs its complexity score. Given thata key challenge with this task is the limitedsize of annotated data, our model relies onpretrained contextual representations from dif-ferent state-of-the-art transformer-based lan-guage models (i.e., BERT and RoBERTa), andon a variety of training methods for further en-hancing model generalization and robustness:multi-step fine-tuning and multi-task learning, and adversarial training. Additionally, we pro-pose to enrich contextual representations byadding hand-crafted features during training. Our model achieved competitive results andranked among the top-10 systems in both sub-tasks.
We propose an ensemble model for predicting the lexical complexity of words and multiword expressions (MWEs). The model receives as input a sentence with a target word or MWEand outputs its complexity score. Given that a key challenge with this task is the limited size of annotated data, our model relies on pretrained contextual representations from different state-of-the-art transformer-based language models (i.e., BERT and RoBERTa), and on a variety of training methods for further enhancing model generalization and robustness:multi-step fine-tuning and multi-task learning, and adversarial training. Additionally, we propose to enrich contextual representations by adding hand-crafted features during training. Our model achieved competitive results and ranked among the top-10 systems in both sub-tasks.
R
predictingthe
predicting the
R
multiwordexpressions
multiword expressions
R
asinput
as input
R
thata
that a
R
limitedsize
limited size
R
onpretrained
on pretrained
R
dif-ferent
different
R
lan-guage
language
R
andon
and on
R
en-hancing
enhancing
R
pro-pose
propose
R
byadding
by adding
R
andranked
and ranked