Semantic tagging of a corpus using the Topic Navigation Map standard

Our work carried out as part of the Scriptorium project has confronted us with a variety of problems which shed light on important issues related to corpus architectural design, such as the definition of fine-grained textual units, extraction of relevant subsections of the corpus, and in particular linking techniques enabling , text annotation with arbitrary and at times conflicting meta-data. The need for greater flexibility and expressive power for our annotation schemes has led us to apply linking techniques as defined in HyTime (ISO 10744:1992) and Topic Maps (ISO13250). We have used these techniques in particular to construct a semantic map of the corpus which enables hypertext navigation in terms of the topics inductively acquired through text-mining software. Navigation is aided by a 3D geometric representation of the semantic space of the corpus. 1. What is Scriptorium ? Scriptorium (Lahlou et al., 1995a), is a project developed in the Research & Development Division of EDF (Electricité de France) in collaboration with ENS (Ecole Normale Supérieure). The aim of this project is to identify prominent and emerging topics from the automatic analysis of the discourse of the company's (EDF) different social players (managers, trade-unions, employees, etc. by way of textual data analysis methods. The corpus under study in this project has 8 million words and is very heterogeneous (it contains book extracts, corporate press, union press, summaries of corporate meetings, transcriptions of taped trade union messages, etc. All documents are SGML tagged following the TEI (Dunlop, 1995) recommendations. Each document is provided with a descriptive header and is segmented into minimal textual units or chunks (which correspond to paragraphs). Scriptorium is a modular architecture which provides an open framework where different text-mining software can be plugged in. At present the following software has been used : ALCESTE (Reinert, 1987) and ZELLIG (Habert et al. 1999). The results of these text mining tools are integrated into the architecture as structured annotation layers pointing at the relevant locations in the corpus. 2. Current approaches in semantic annotation Increasing availability of textual data in electronic form has made semantic access to document collections a crucial issue. Semantic annotation of corpora is mainly performed today on the basis of pre-existing categories which are defined in external resources and then projected onto the corpus. These resources include lexical resources such as dictionaries, thesauri or domain-specific terminologies and conceptual resources such as generalpurpose or application-specific ontologies. Work in the field of Word Sense Disambiguation (WSD) for instance, exploits lexical resources with the aim of selecting the relevant meaning of a word in a given context from a pre-established list of all the word's potential meanings. Reducing ambiguity by tagging each word of a corpus with its contextual meaning is then exploited in Information Retrieval (IR) to increase the precision of the results generated in response to a query. Semantic ontologies and networks are also exploited in view of enhancing navigation or IR performance. To date, the most widespread lexical semantic network is WordNet (Miller et al., 1990), developed at Princeton University since 1985. The nodes of this network are lists of synonyms or synsets. Each synset represents a meaning which can be lexicalized through a set of synonymous words. A meaning is defined differentially in terms of the relations a given synset has with other synsets of the network. The network is partitioned into different areas which correspond to parts of speech (POS) categories (nouns, adjectives, verbs ). Different types of relations structure the network which depend on the POS category. For instance, hyponomy relations are defined for nouns, antonomy for adjectives, implication for verbs, etc. The network is essentially conceptual as the relations are established between meanings and not between lexical items. Corpus annotation with WordNet categories has been used to optimize navigation and IR, in particular to expand queries by generating synonyms that can be inferred through the relations of the network. This is aimed at improving recall in response to a query. Semantic annotation can also be performed on the basis of an application-specific conceptual model. Given the growing merging between XML and database technologies and the increasing use of XML as a standard format for exchanging data among heterogeneous computer systems, document markup is used for annotating metadata whose structure is described in an external model comparable to a database schema. This model specifies the entities and relations of a given application domain. In this approach too, semantic annotation is based on a pre-defined categorization scheme, which is in this case similar to the fields and records of a database table. In contrast to the approaches described above, the focus is different within the context of document-based systems aimed at technology watch (or social watch as is the case for Scriptorium), given that the aim is not to access the corpus by means of a pre-defined list of topic categories, but to identify emerging topics in the corpus. Furthermore, the nature of these topics is also different, as they are defined inductively by text mining tools based on data-driven methods. They are in consequence contextual and endogenous to a given corpus. It is essential for text mining software to run on homogeneous corpora in order to yield relevant results. As we have pointed out above, our document collection is extremely heterogeneous. The concepts underlying the documents cannot be represented in one single semantic space. Therefore we do not attempt to build a semantic representation of the entire corpus through one only analysis. Our approach therefore consists in building subcorpora of exploitable size which are homogeneous with respect to a given parameter. To this aim, we use an extractor developed using the XML Python libraries which dynamically assembles subsets of text chunks in response to a query. This extractor runs queries concerning the descriptive parameters stored in each document's header as well as fulltext searching constraints. These sub-corpora are then analyzed by the text mining tools, which at present include ALCESTE and ZELLIG. In the following two sections we show how we construct semantic classes with ALCESTE and exploit them to construct a navigable topic map overlaid on the corpus. 3. Generating semantic classes with ALCESTE ALCESTE is a textual data analysis software developed by M. Reinert at CNRS. This program performs a classification of the text chunks of a given corpus in order to produce classes of statistically related chunks. The notion of a chunk corresponds to that of an elementary textual sequence of variable size. In our case we have chosen the paragraph, as that is our basic segmentation unit. Similarity of text chunks is based on a distributional criteria. It depends on the number of word stems that chunks have in common, taking into account the frequency of occurrence of the words in the corpus. Classification is based on a hierarchical descending clustering algorithm in which successive dichotomies are carried out using the first axis of a factor analysis. Details on the algorithm can be found in (Reinert, 1985). The important point is that the method is contrastive, in other words a class of chunks is formed in opposition to the rest, not in terms of an absolute criteria. After these classes have been determined, the program seeks the list of words most characteristic or specific to each class according to a chi2 metric. This also produces a geometric representation of classes and words in the space, based on the axis yielded by the factor analysis. We use the coordinates of the classes and words on the first three axis to obtain a 3D representation of the semantic space of a given ALCESTE analysis. The resulting ALCESTE classes reveal underlying representations or concepts which are lexicalized through a set of related chunks and a characteristic vocabulary. These classes cannot be interpreted as absolute semantic categories but rather as points of view or semantic contexts relative to the sub-corpus under study. More on the interpretation of ALCESTE classes in (Lahlou, 1995b). 4. The Topic Navigation Map Standard Topic Navigation Maps (TNM) is an international standard (ISO/IEC 13250) providing a language expressed in SGML and HyTime to construct a layer of topics and relations aimed at classifying and semantically annotating a collection of documents. A topic map is implemented as a set of independent links. A topic map can be compared to a semantic network overlaid on a pool of textual data. The important point is that there is a separation between the textual data and the topic map structure. As an independent structure, a topic map can be associated to different textual resources; likewise, and more relevant to our aims, different topic maps can be layered over the same document collection providing thus multiple views on the same corpus. A topic map structure is optimized for navigation, but in contrast to HTML provides the mechanisms for explicitly defining a semantic model for this structure, aimed at for instance, describing the relation expressed by a link. The main element of a Topic Map is a topic which is an independent multi-headed link pointing to a set of occurrences. A topic link groups all textual occurrences (of any granularity) which are related to the topic. A topic has a topic type (or multiple topic types), which correspond to link types in HyTime. A topic occurrence is any addressable textual unit in the document collection. Topic occurrences can be described by the role they play. This corresponds to the facility provided in HyTime for describing the role