Multilingual Automatic Extraction of Linguistic Data from Grammars (2023)

Abstract: One of the goals of field linguistics is compilation of descriptive grammars for relatively little-studied languages. Until recently, extracting linguistic characteristics from grammatical descriptions and creating databases based on them was done manually. The aim of this paper is to apply methods of multilingual automatic information extraction to grammatical descriptions written in different languages of the world: we present a search engine for grammars, which would accelerate the tedious and time-consuming process of searching for information about linguistic features and facilitate research in the field of linguistic typology.

## introduction this work is dedicated to methods of information extraction, one of the subtasks of natural language processing. methods of information extraction are widely used to create search engines. in addition to web services designed to search for internet websites that are relevant to the user's request, there is a need for highly specialized search engines for scientific publications, including linguistic ones. one of the publication types in field linguistics is a descriptive grammar, which is a description of phonetic, morphological, syntactic, semantic and other characteristics of a particular language. until recently, extracting language characteristics from descriptive grammars and creating databases based on them was done manually. for instance, the world atlas of language structures 1 , originally published as a book @xcite , contains information on 144 characteristics for over 2600 languages. searching for information about a multitude of features is a long and labor-intensive process, albeit a portion of grammars is available not only in paper form, but also in digitized form: grammars from different time periods (from missionary 1 https://wals.info/ grammars to modern papers) created by researchers from different countries do not have a single structure. furthermore, a simple search for a word in a document can return dozens of occurrences, and not all of them will be relevant to the query. the purpose of this work is to create a search engine for grammars, which would facilitate and speed up the process of finding information about language characteristics. the paper considers two methods of information extraction (bm25 and a reranking model based on bert). the materials for the demonstration of the search engine include grammars presented on google drive foot_0 . section 2 will analyze the already existing works pertaining to the task of automatic extraction of data from grammars; in section 3, the methods used for data preprocessing will be described. section 4 will discuss the two methods used for information extraction. in section 5, we will compare the results obtained using the two methods and demonstrate the features of the search engine web application. ## review of existing approaches at the moment, the subject of automatic information extraction of data from grammars is relatively little-studied. several scientific papers regarding the methodology for extracting information from grammars using frame semantic parsers have been published by members of språkbanken, a research and development unit at the university of @xcite @xcite . the methodology proposed by språkbanken is illustrated in @xcite using the following hypothetical sentence from a grammar as an example: the adjectives follow the noun they qualify. an answer to the following question: "what is the order of adjectives and nouns in the language?" is to be chosen from the values "noun-adjective", "adjective-noun", and "both". based on the labels assigned to the predicate "follow", the subject "adjectives", and the object "noun" by the semantic parser, the option "noun-adjective" is selected as an answer to the question and entered into the database. semantic parsers based on tagged text corpora are usually not sufficient to describe semantic frames found in grammars. @xcite describes the functionality of a highly specialized semantic parser for linguistic publications, created on the basis of lingfn. lingfn is a corpus of grammars in english with annotated semantic frames, described in in @xcite . extracting information from grammars written in languages other than english would require creating highly specialized parsers for each of the languages. since a single specialized parser is not a multilingual solution, further this paper will discuss methods that are not based on frame semantics. a simpler method is used in @xcite : to find out if a certain phenomenon is present in the language, the frequency of the corresponding term in the text of the grammar is counted. occurrences of a term in the context of negative polarity items ("in language x [there is no phenomenon y] | [missing category y] | [category y not found]") are excluded. based on the distribution of occurrences of each term in grammars, a frequency threshold is calculated. only terms with a frequency above the threshold are categories potentially present in the language. this method does not require significant time spent on annotating corpora and is universal for grammars written in any language, which greatly facilitates automatic creation of databases of linguistic characteristics. however, the methods described in @xcite @xcite @xcite are effective for building language databases in the form of tables, where at the intersection of a row with the name of the language and the column with the name of the category is an answer to a question (for example, "noun-adjective") or a truth value indicating presence or absence of a particular category in the language. the table format does not fully meet the goals of our work, since it is not enough for a search engine to extract a single truth value; it is more crucial to extract a paragraph which describes the specific features of the desired language characteristic, together with the glosses and examples. therefore, it has been decided to use methods that rank documents (paragraphs) according to their relevance to the search query entered by the user in order to return the original paragraph from the grammar in response to the query. the chosen approach does not perform any final feature extraction, but leaves the ultimate decision to the linguist. ## data the grammars from which the search engine application extracts information are presented on google drive in the grammars folder. the source code of the application is available on foot_1 . each grammar is presented in a .pdf file. the table grammars_database.xslx (stored in the source code repository) contains meta-information for each grammar: the full path to the file, availability of an ocr layer ("searchable"/"not searchable"), the language described in the grammar, and the language the grammar is in. initially, some files did not have an ocr layer. such files were processed using the ocrmypdf foot_2 library. for the subsequent information extraction, the contents of each file were preprocessed. the grammars were parsed using the pdftotext foot_3 library and divided into paragraphs. a combination of two spaces was taken as a separator. after separation, extra spaces were removed from the beginning and end of each paragraph. since there are frequent cases of a paragraph being split between two pages, after separation, each pair of adjacent paragraphs was checked: if the second paragraph does not start with a capital letter and/or the first one does not end with a dot, ellipsis, question mark or exclamation mark, then they were connected again into a single paragraph. further, each paragraph was divided into tokens using the spacy 6 library. spacy was chosen because it currently implements text preprocessing methods for 22 languages. the paragraphs underwent tokenization; punctuation marks, numbers, and stop words were removed. the lists of tokens and their corresponding page numbers were saved in .json files in the grammars_page_numbers folder in the repository in order that the search algorithm would work with preprocessed data and not with the original .pdf file. after the tokenization, the paragraphs were lemmatized, and the lists of lemmas were saved as .json files in the grammars_lemmas folder. ## methods for ranking paragraphs by relevance to the query after the data has been divided into paragraphs and preprocessed, the search engine itself was implemented. it accepts a query from the user, determines which of the paragraphs are relevant to the query, and returns them. to calculate relevance, this paper uses the bm25 algorithm and a combination of bm25 with bert embeddings. ## a solution to the problem of multilinguality since the goal of this paper is to create a search engine that is not exclusive to grammars written in english, it is necessary to implement an algorithm for automatically translating the user's query from english into other languages. google translate and libraries based on it are not suitable for this task: results for translating linguistic terms into other languages are in most cases incorrect. for instance, the term "reduplication" is translated from english into german as "verdoppelung" ("doubling"), not "reduplikation". consequently, it was decided to use another method of translating linguistic terms into different languages: using wikipedia. the html code of the wikipedia page called "reduplication" in english contains links to pages about the same term in other languages. the method for extracting page titles in the desired language was implemented using the beautiful-soup4 9 library. in addition to titles of articles, it was decided to extract their summaries using the wikipedia 10 library. example: summary for the term "ergative case" 11 in english (accessed 23 feb. 2023): in grammar, the ergative case (abbreviated erg) is the grammatical case that identifies a nominal phrase as the agent of a transitive verb in ergative-absolutive languages. using a summary as a query increases the likelihood of extracting a relevant paragraph from the grammar, as it may contain words, linguistic terms, and abbreviations that are often found in the context of the term requested by the user: for instance, the summary for the wikipedia article "ergative case" contains the abbreviation "erg" and related terms "agent", "transitive verb", and "absolutive". each summary is extracted from wikipedia, tokenized, and lemmatized only once. the summaries themselves and the lists of tokens corresponding to them are saved in .json files in the grammars_summaries folder in the repository. ## conclusion this paper presents a search engine web application that allows automatic extraction of information from grammars written in different languages of the world. two information extraction methods (classical bm25 and the combined method based on bm25 + reranking with bert) have been compared to each other regarding the task of extracting linguistic information relevant to the user's query. the search algorithm has been integrated with wikipedia. the implemented system makes it possible to get an impression of the total complexity of the task of automatic information extraction from scientific publications and opens up the possibility for massive automated research in the field of linguistic typology, facilitating the routine task of extracting information from grammatical descriptions and allowing researchers to direct the time to solving problems that require advanced expertise.