The ISA -17 Quantification Challenge: Background and introduction (2021)

Abstract: This paper, intended for the ISA-17 Quantification Annotation track, provides background information for the shared quantification annotation task at the ISA-17 workshop, a.k.a. the Quantification Challenge. In particular, the role of the abstract and concrete syntax of the QuantML markup language are explained, and the semantic interpretation of QuantML annotations in relation to the ISO principles of semantic annotation. Additionally, the choice is motivated of the test suite of the Quantification Challenge, along with the suggested markables for the sentences of the suite.

## introduction the isa-17 quantification challenge was motivated by the decision of the international organisation for standardisation iso to develop an international standard for the annotation of quantification in natural language, extending the series of standards for semantic annotation called the iso semantic annotation framework (semaf, iso 24617)). other parts of this series include standards for the annotation of (1) time and events (iso 24617-1, 'iso-timeml'); also belonging to this series is the meta-standard iso 24617-6, 'principles of semantic annotation', which defines a common methodological framework for developing other parts of semaf. as the first steps in the development of an annotation standard for quantification, preliminary studies have been conducted and reported at lrec 2018 @xcite , at iwcs 2019 @xcite , and in a technical report of tilburg university @xcite in which the markup language quantml is defined. on the basis of these studies, the document iso wd 24617-12 was drafted. the isa-17 quantification challenge is intended to identify the strengths, limitations, and deficiencies of the quantml proposal by inviting experts in quantification and/or in semantic annotation to explore the application of quantml in the annotation of a range of test sentences that display some of the phenomena that the future standard would hope to cover. this paper is organised as follows. section 2 briefly summarizes the iso principles of semantic annotation, especially where it concerns the architecture of a semantic annotation scheme, including the design of an abstract syntax and the specification of a concrete syntax plus the significance of a compositional semantics of the (abstract syntactic structures of the) annotations, and applies this to the quantml annotation scheme. section 3 introduces and motivates the choices in the test suite used in the quantification challenge.