The I celandic W ord W eb: A language technology-focused redesign of a lexicosemantic database (2021)

Abstract: The new Icelandic Word Web (IW) is a language technology focused redesign of a lexicosemantic database of semantically related entries. The IW’s entities, relations, metadata and categorization scheme have all been implemented from scratch in two systems, OntoLex and SKOS. After certain adjustments were made to OntoLex and SKOS interoperability, it was also possible to implement specific IW features that, while potentially nonstandard, form an integral part of the Word Web’s lexicosemantic functionality. Also new in this implementation are access to a larger amount of linguistic data, a greater variety of search options, the possibility of automated processing, and the ability to conduct research through SPARQL without possessing a mastery of Icelandic.

## introduction we introduce the new icelandic word web (iw; icel. íslenskt orðanet), a language technology focused overhaul and redesign of a lexicosemantic database of semantically related icelandic words and phrases @xcite . this modernization improves access to the iw's intricate systems, makes its data more malleable, enables the use of a greater variety of metadata, and allows for a new, open-ended approach to conducting research on its various elements. the iw is the only database of its kind for the icelandic language. although there does exist a number of other semantic databases, e.g. arabic wordnet @xcite , balkanet @xcite , eurowordnet @xcite , indo-wordnet @xcite , and the multi-wordnet project @xcite , there is a strong tendency for these to be modeled on the princeton wordnet (princeton university, 2010), arguably one of the best known databases of semantic word relations. while comparisons might be made between the iw and the princeton word-net, the iw diverges considerably in its overall structure and approach to semantic relations; its structure is more fluid and its focus more on the relations between core entries rather than the complex hierarchy around them @xcite . the implementation of the new iw itself represents a novel application of the two models with which the iw is encoded, and may prove useful in other projects involving the encoding of lexical databases with nonstandard structures and elements. we begin by describing the core structure of the original iw, focusing on the aspects that remained unaltered. we then move on to the details of the overhaul. we discuss the choice of implementation models, how we applied them to the iw and what benefits we derived, and how we adapted them to certain aspects of the iw that were vital to its design but could not be represented by standard model features. we subsequently describe how the redesign has increased search scope, both in terms of the amount of accessible data and of the ways in which that data may now be searched for and inspected. lastly, we touch on the potential future development and use of the iw, now that it is in this new form. ## core structure of the icelandic word web the iw is effectively composed of two separate but interconnected systems: entries and categories. the former, entries, contains the words themselves and their semantic relations, and forms the bulk of the iw. entries come in many varieties: monolexical and polylexical, unordered and ordered (including phrasemes), sourced both from reference works and primary sources, and accompanied by varying degrees of explanatory and morphosyntactic metadata @xcite . as is common with these types of collections, the entries do not have definitions except in cases where glosses are necessary to differentiate word forms; rather, their meanings are considered to be implicit in the relations they have to other entries or to their respective categories. the semantic relations themselves are similarly sourced both from primary sources and older reference works, with the majority being derived from the former. the latter system, categories, contains a semantic classification scheme, and effectively functions as an ontology for the iw's entries. unlike the entries and their relations, which are primarily derived from source material, the categories have been created and implemented over the years by the iw's past administrators. the scheme is descriptive rather than prescriptive, and is not intended to be all-encompassing; each entry may thus belong to one, none, or multiple categories. all categories have equal priority, there are no category hierarchies, and from a semantic perspective their subjects may overlap. although categories do exist as separate entities in the iw, there are no direct category-to-category relations. they are connected only through the relations of the entities that belong to them. the iw's primary type of semantic relation is a specific kind of parallel construction, which we will call pairings for short. this relation indicates that two given entries, x and y, have at some point appeared in a source text with the conjunction og (eng. and) between them. pairings are unordered by design, with a sourced x og y being considered the equivalent of y og x. most of the other relation types in the original iw build in some way on pairings, aside from a relatively small set of synonyms and antonyms whose handcrafted relations are drawn from preexisting entries in the iw's database. pairings combine semantics and syntax, albeit with an emphasis on the former; and this amalgamated nature, coupled with their status as a cornerstone of the iw's full span of relation types, was a major design factor in the development of the new iw. the creation of the original iw involved the work of several people, over a period of decades rather than years, collating relational information that initially described syntax and morphology but later shifted in focus to involve semantics as well, all of which culminated in a deep and complex collection of data. while the original iw is presented through a web interface foot_0 , there is no single, fully standardized type of entry in its underlying database. some entries may be written or encoded differently from others, some have more metadata, and in certain cases the metadata itself may also be encoded differently between entries. in short, a direct conversion to an established format was not an option. the only way of bringing the iw to a language technology friendly format while simultaneously maintaining its breadth of data and functionality was to design and implement it in the new format almost from scratch -a process that not only allowed for greater standardization, but also for greater inclusivity of information that up until now had either been difficult to use or entirely inaccessible. ## general implementation given that the iw's original structure is divided into two separate systems whose functionality is not the same, we approached the reimplementation from the very beginning with the idea that it would not necessarily contain only one system. we therefore expected -and later found -one of the major points of complexity not to lie in how to fit the entire structure into one paradigm, but rather how well two new schemes could interact. we modeled the new iw using two separate systems: ontolex @xcite and skos @xcite . not only was each of these well suited to represent its respective part of the iw, but their point of intersection also turned out to be both fully workable -ontolex's functionality has been designed with skos's interoperability in mind, so the two models mesh well when applied to the iw -and useful to model certain important and nonstandard aspects of the iw. moreover, the use of these systems opened up the possibility of a range of new queries into the data that had not been possible until now, both through the new systemized encoding of its metadata, which made it available to users for the first time, and through the option of user-created sparql queries rather than fixed web site user patterns. ontolex was used to encode the iw's basic entities: lemmas, their semantic relations, and per-tinent morphosyntactic information. it supports complex linguistic modeling, and was the model of choice for structuring an rdf version of the princeton wordnet. it has already been used to recreate dictionaries in such a way that they are easily integrable with certain outside resources, an important point for the iw's two systems. moreover, it is the only data model of its kind that can reasonably be applied to a morphologically rich language such as icelandic @xcite . ontolex allowed us to recreate with relative ease a myriad of the original iw's features, and, moreover, it enabled us to codify data that had been present in the original iw but had not been directly available to the user. as an example, most of the new iw's monolexical entries are now accompanied by data drawn (when available and applicable) from the database of icelandic morphology (dim) @xcite . the data cover not only each entry's lemmatized form but also its various inflectional forms and associated morphosyntactic features as well. this enables users to conduct both context-based searches for inflectional forms, and searches based on the morphosyntactic features themselves. these include gender, case, number, voice, mood, tense, person, and definiteness. additionally, certain entries (both mono-and polylexical) are labeled as exclamations, conjunctions, prepositions, numerals, set phrases and proper nouns, to the degree that the iw's original data allows. the encoded data for polylexical entries is somewhat more sparse than for monolexical ones, partly to save space and avoid reduplicating data. however, a valuable new feature of the iw is interlinking: wherever possible we have added to each single word of a polylexical entry a link to that word's corresponding monolexical entry. this new feature grants access by proxy to the word's morphosyntactic data, removing the need to reduplicate all that information in the polylexical entry, and overall greatly increases both the accessibility and interrelatedness of the iw's data. skos, a popular rdf-based ontology model, was used to encode the iw's categories. while ontolex has a great variety of various encoding options but a stringently ordered design dictating their use, the base version of skos has a comparatively smaller range of options but is far more malleable, a fact that makes it well-suited for the iw. instead of being a standalone entity with a com-plete and systematic internal structure, the iw's category system draws heavily on the content of its other system of entities -categories are only created and put into use if existing lemmas support them -and there is a great deal of commingling and cross-referencing between the categories and lemmas, which means that the category system needs to be represented by a model that does not require all its entities to be discrete. using skos, modeling the categories was a straightforward process. where skos really comes into play is at the point where the iw's two systems -and hence these two models -intersect. ## model convergence while ontolex is an ideal option for representing the iw's complex grammar, it is unable to encapsulate certain other aspects of the iw's design. most notably, ontolex cannot comfortably represent semantic relationships such as the pairings noted in the previous section, nor can it encode entities at all if, as is sometimes the case in the iw, they do not have an ontological connection to at least one category. skos, meanwhile, may be used to implement both these features but cannot store the entities themselves, which must be kept in ontolex if we are to hold on to that linguistic modeling mentioned earlier. our solution was to develop a separate conceptual layer that hovers between these two models and serves as an intermediary. the change can be seen in figures 1 and 2 below. figure 1 shows what the iw would be like if implemented directly in ontolex and skos, without taking into account the aforementioned issues. the iw's entities would be encoded in ontolex as lexical entries, while its categories would be encoded in skos as concepts. the two would then be connected by an ontolex relation called lexical sense. entities that did not belong to an iw category would not have this kind of relation, which would render them invalid in onto-lex. moreover, the iw's pairings relation would need to be directly between lexical senses, and although ontolex does support a number of senseto-sense relations, none of them are a suitable fit for our purpose. figure 2 , on the other hand, shows our final implementation of the iw in ontolex and skos, where these issues are taken into account. here, we have added the separate conceptual layer. note the replication of entities: after we have encoded every one of them in ontolex as lexical entries, we mirror them in skos as concepts. we call these mirrored entities lemma-as-concept, or lac for short (lemma being a more direct translation of the iw's icelandic term for entities, fletta) @xcite . from the viewpoint of ontolex, lacs serve as connectors to an ontology. from a skos viewpoint, lacs are effectively a new category layer where each respective unit represents exactly one entity: the lemma in question. in those cases where there does exist an actual category, the lac merely functions as its subset. not only does this ensure that we always have the lexical entry/concept connection mandatory for sustaining each lexical sense, but it also allows us to encode lemma pairings by using the skos senseof keyword to relate lacs as appropriate. in implementing this separate layer and its functionality, we have thus avoided creating nonstandard keywords that might have otherwise complicated the iw's use, and have confined any somewhat atypical use of the models to a clearly delineated section of our system, while simultaneously maintaining vital core functionality of the original iw. if new types of semantic word rela-tions were to be added to the iw, they could comfortably be fitted into this layer. ## data accessibility and augmentation in terms of existing entity relations, the new iw has greatly increased their scope. the original iw, which is accessible through a bespoke web site, contains one fundamental semantic relation -pairings -and three ancillary ones (synonyms, near-synonyms and antonyms), plus a half-dozen derived relations that build on these. while these relations could often be highly informative, both in content and presentation, the precise nature of each relation was fixed and could not be altered by the user. search functionality, likewise, was simple and clean but unmalleable, with a focus on textual searches for entities. parameters both for the search and the relations themselves could not be altered, and the results could not be exported for further examination. the original data was stored in multiple tables in a database backend behind the web site, and was either not accessible except through the web site's search options or, in the case of morphosyntactic metadata, not accessible to regular users in any way. the new iw, by contrast, effectively offers a limitless variety of potential searches. it is stored in a single rdf file accessible directly through clarin foot_1 under a cc by 4.0 license. we have encoded only the pairings and ancillary relations into the system, and the derived relations are not formally encoded in the system. instead, everything may now be produced through queries written in sparql, and the raw data itself may be viewed at will as needed. the sample query in figure 3 shows a search for all words and phrases whose written form, irrespective of grammatical categorization, has more than one "gloss", or explicitly mentioned definition. the sparql code is on the left, and the results on the right, with the results' left-hand column listing word web entries and the right-hand one listing their corresponding glosses. these queries extend to practically every attribute encoded into the iw, including all those listed in the chapter on general implementation. so long as the user can formulate their intent into a valid sparql query, it may be applied to the iw's data. (this includes lacs, which may be trivially folded into sparql queries.) ## conclusion and future developments by its encoding in a publicly accessible form, the new iw reduces the barriers to entry for anyone wishing to make use of its stores of information. it also encodes that information in such a way that far more of it is accessible and usable for research, while maintaining, wherever possible, an adherence to official standards that ensure the iw's functionality is well-documented and comparable to that of any other models encoded using those same standards. on those occasions where that adherence is not possible or practical, we have tried to ensure that non-standard use is properly documented, kept to a minimum, and contained within a specific, clearly-defined part of the iw. the iw's data storage is kept current, with new information added on a regular basis. as noted earlier, updates of existing data will grant the iw even greater usability. in addition, the models in which the iw is encoded support a range of potential information such as bilingualism and phonetics that, although not currently a part of the iw, may now be added to a system designed to store and handle this kind of data. overall, the iw is a deep and extensive system that models varying degrees of semantic relations between single-and multiword lemmas, drawing its information not from third party schemas and design, but rather directly from first-party sources. there is ample reason to think that the iw will be relevant to any number of research projects in the future, particularly now that it has been redesigned and reimplemented with depth of reach and ease of access in mind.