The NLP community has shown a renewed interest in deeper semantic analyses, among them automatic recognition of relations between pairs of words in a text. We present an evaluation task designed to provide a framework for comparing different approaches to classifying semantic relations between nominals in a sentence. This is part of SemEval, the 4 th edition of the semantic evaluation event previously known as SensEval. We define the task, describe the training/test data and their creation, list the participating systems and discuss their results. There were 14 teams who submitted 15 systems. The theme of Task 4 is the classification of semantic relations between simple nominals (nouns or base noun phrases) other than named entities -honey bee, for example, shows an instance of the ProductProducer relation. The classification occurs in the context of a sentence in a written English text. Algorithms for classifying semantic relations can be applied in information retrieval, information extraction, text summarization, question answering and so on. The recognition of textual entailment is an example of successful use of this type of deeper analysis in high-end NLP applications. The literature shows a wide variety of methods of nominal relation classification. They depend as much on the training data as on the domain of application and the available resources. Rosario and Hearst (2001) classify noun compounds from the domain of medicine, using 13 classes that describe the semantic relation between the head noun and the modifier in a given noun compound. Rosario et al. (2002) There is little consensus on the relation sets and algorithms for analyzing semantic relations, and it seems unlikely that any single scheme could work for all applications. For example, the gene-gene relation scheme of Stephens et al. (2001) , with relations like X phosphorylates Y, is unlikely to be transferred easily to general text. We have created a benchmark data set to allow the evaluation of different semantic relation classification algorithms. We do not presume to propose a single classification scheme, however alluring it would 13
SemEval-2007 Task 4 "Classification of Semantic Relations between Nominals" REF gives access to a corpus containing nominal causal relations among others, as causality is one of the considered semantic relations in the task.
SemEval-2007 Task 04: Classification of Semantic Relations between Nominals
Fourth International Workshop on Semantic Evaluations (SemEval-2007)
Computer Science