We present WiC-TSV, a new multi-domain evaluation benchmark for  Target Sense Verification of Words in Context . The benchmark is a new framework for evaluating Word Sense Disambiguation and Entity Linking, which is different from conventional benchmarks for it covering various domains, while being independent of external sense inventories  . This makes the dataset highly flexible for the evaluation of a diverse set of models and systems in and across domains. WiC-TSV provides three different evaluation settings, depending on the input signals provided to the model. We set baseline performance on the dataset using state-of-the-art language models. Experimental results show that even though these models can perform decently on the task, there remains a gap between machine and human performance, especially in out-of-domain settings. WiC-TSV data is available at URL/competitions/ 23683.
We present WiC-TSV, a new multi-domain evaluation benchmark for Word Sense Disambiguation. More specifically, we introduce a framework for Target Sense Verification of Words in Context which grounds its uniqueness in the formulation as a binary classification task thus being independent of external sense inventories , and the coverage of various domains . This makes the dataset highly flexible for the evaluation of a diverse set of models and systems in and across domains. WiC-TSV provides three different evaluation settings, depending on the input signals provided to the model. We set baseline performance on the dataset using state-of-the-art language models. Experimental results show that even though these models can perform decently on the task, there remains a gap between machine and human performance, especially in out-of-domain settings. WiC-TSV data is available at URL/competitions/ 23683
A
Word Sense Disambiguation. More specifically, we introduce a framework for
R
. The benchmark is a new framework for evaluating Word Sense Disambiguation and Entity Linking, which is different from conventional benchmarks for it covering various domains, while
which grounds its uniqueness in the formulation as a binary classification task thus
A
, and the coverage of various domains
R
23683.
23683