The goal of this task is to allow for comparison across sense-induction and discrimination systems, and also to compare these systems to other supervised and knowledgebased systems. In total there were 6 participating systems. We reused the SemEval-2007 English lexical sample subtask of task 17, and set up both clustering-style unsupervised evaluation (using OntoNotes senses as gold-standard) and a supervised evaluation (using the part of the dataset for mapping). We provide a comparison to the results of the systems participating in the lexical sample subtask of task 17.
The evaluation of the collocational-graph method in the SemEval-2007 sense induction task REF showed promising results.
SemEval-2007 Task 02: Evaluating Word Sense Induction and Discrimination Systems
Fourth International Workshop on Semantic Evaluations (SemEval-2007)
Computer Science