  Many downstream NLP tasks have shown significant improvement through
continual pre-training, transfer learning and multi-task learning.
State-of-the-art approaches in Word Sense Disambiguation today benefit from
some of these approaches in conjunction with information sources such as
semantic relationships and gloss definitions contained within WordNet. Our work
builds upon these systems and uses data augmentation along with extensive
pre-training on various different NLP tasks and datasets. Our transfer learning
and augmentation pipeline achieves state-of-the-art single model performance in
WSD and is at par with the best ensemble results.
