Introducing the Digital Language Equality Metric: Contextual Factors (2022)

Abstract: In our digital age, digital language equality is an important goal to enable participation in society for all citizens, independent of the language they speak. To assess the current state of play with regard to Europe’s languages, we developed, in the project European Language Equality, a metric for digital language equality that consists of two parts, technological and contextual (i.e., non-technological) factors. We present a metric for calculating the contextual factors for over 80 European languages. For each language, a score is calculated that reflects the broader context or socio-economic ecosystem of a language, which has, for a given language, a direct impact for technology and resource development; it is important to note, though, that Language Technologies and Resources related aspects are reflected by the technological factors. To reduce the vast number of potential contextual factors to an adequate number, five different configurations were calculated and evaluated with a panel of experts. The best results were achieved by a configuration in which 12 manually curated factors were included. In the factor selection process, attention was paid to data quality, automatic updatability, inclusion of data from different domains, and a balance between different data types. The evaluation shows that this specific configuration is stable for the official EU languages; while for regional and minority languages, as well as national non-official EU languages, there is room for improvement.

## introduction the rising influence of the internet on the daily life impacts the relevance of the automated understanding and production of language in the digital age since natural language is an important part of human-computerinteraction (hci). from a technological perspective, language technologies (lt) can add the "ability to analyze, understand and generate information expressed in natural language" @xcite to digital systems. especially many languages with smaller numbers of speakers are typically under-served in terms of resources and technologies, because of factors as missing economic interest, etc.. to analyse the situation of a language community in the digital sphere, it is necessary to develop a metric which is able to assess the current state of technological support, but that is also able to position the results in the broader socio-economic context of a language and its community. hence, our suggested digital language equality (dle) metric consists of two broader groups of factors, technological and contextual factors. especially in multilingual societies, the importance and relevance of dle is growing every day. in europe, we are still far away from the ideal situation of dle which would be "the state of affairs in which all languages have the technological support and situational context necessary for them to continue to exist and to prosper as living languages in the digital age." @xcite . back in 2012, the meta-net white paper series @xcite demonstrated a strong imbalance in terms of technology support for 31 european languages, even though at least the 24 official eu member state languages have the same status and rights. additionally, more than 60 regional and minority languages (rml) are protected via the european charter for regional or minority languages and the charter of fundamental rights of the eu (article 21), which declare the prohibition of discrimination grounded on language (european union, 1992; european union, 2010). the eu-funded project european language equality (ele) addresses the challenge how to solve this existing imbalances. its main goal is the preparation of a strategic research, innovation and implementation agenda and roadmap that specifies the necessary steps and instruments to achieve dle in europe by 2030 @xcite . the project covers a total of 89 languages: all 24 official eu languages, 11 official national languages without an official status in the eu and 54 regional and minority languages. 1 for the preparation of the strategic agenda, the current state of each language needs to be determined as the starting point. in all previous attempts, such as the meta-net white paper series, the role of a language's context on the development (or lack thereof) of technologies for that language has been neglected. accordingly, in this article we focus upon the contextual factors (cf). 2 we prepare different configurations of the metric based on simple classifications of the cfs to assess which factors can and should be included. section 2 provides the theoretical background about dle in europe and the measurement of the context of languages in the digital world. section 3 describes the data collection, preparation and calculation of the metric. section 4 presents the results and evaluation. section 5 discusses the work and its limitations. ## discussion the dle metric for the contextual factors has some limitations (see section 3). first, expanding the data set to include regional or national sources would result in (i) a higher number of factors, (ii) improved data quality, as gaps in individual indicators may be filled, (iii) quantification of more factors with more than one indicator, to reflect different perspectives and (vi) a more complex mapping to language communities based on regional data resulting in a significant impact on rml. moreover, including the factors suggested by the experts, such as membership in the eea or language vitality status, could help improve the results. second, the data cleaning procedure can be improved. for the calculation of the innovation scoreboard (bielinska-dusza and hamerska, 2021), outliers with values outside twice the standard deviation were replaced by the respective maxima or minima of the data series. data gaps could be filled using data from previous years and skewed data could be corrected using a square root transformation. these steps would most likely affect the results of configurations 1 and 2, since they use the data with poorer quality. the mapping of country-specific data to languagespecific data can be improved, e. g., @xcite show how a possible regional mapping of data using the world geodatasets foot_8 could be realised. for large countries with bigger regional or urban-rural divides, a regional mapping would represent reality more accurately. in particular, the missing mapping of proportional data, scores and total numbers per capita has a major impact on the results. here, regional data could help to calculate the average deviation of individual regions or language communities from other proportional data and to transfer this deviation to proportional data suitable too high too low contrary opinion english irish norwegian french dutch italian spanish german danish swedish portuguese saami, northern polish hungarian czech latvian greek croatian romanian finnish maltese bulgarian estonian faroese icelandic slovene scottish gaelic emilian slovak cornish sicilian lithuanian manx serbian saami, southern basque saami, pite catalan saami, lule galician saami, skolt asturian saami, inari aragonese sardinian welsh romagnol griko lombard ligurian venetian southern italian friulian piemontese ladin 25 17 9 4 the lowest and highest value of the range for the calculation represent the lowest or the highest value from the last years, which reduces fluctuations. another approach would be to measure the prediction accuracy of the cfs with regard to the tfs after some time. in this way, each single factor could be evalu-ated and unrecognized distortion in the results could be examined and ruled out in the future. the results show a need for an improvement regarding the context for lt development for all languages except english, french and german. despite the lack of data about non-european countries with english as the official national language, english achieves the best results in every configuration. thus, the dominance of english in business and science is reflected in the data. the good results for french and german are grounded in the size of the countries and their economies. spanish reaches only half the score, even though it has many more speakers. some experts criticise this result since the context of spanish for lt development should score higher. as shown by the meta-net white papers @xcite , lt support for spanish is similar to german and french. since the cfs are supposed to show the achievability of dle and thus give a 'prediction' for lt development, the results do not fit. in the meta-net white paper comparison of the technical support of europe's languages, the languages that were assessed as having a better technical support in 2012 also perform better in the calculation of the cfs. always reaching the highest contextual scores, english, dutch, french, german, spanish and italian achieved "moderate support" in at least three of the four lt areas @xcite . the next set of languages according to the results of the cfs, i. e., polish, czech, swedish, hungarian and finnish, also achieved "moderate support" in at least one area in 2012. the fact that these languages achieved better results in 2012 indicates that their context has probably been better ten years ago than for the remaining languages. greek, croatian and danish stand out because these three languages did not reach the "moderate support" level in any of the four groups in 2012. however, since the score for croatian is considered too high by the experts for all configurations, it can be assumed that the score is distorted by the data. the context for greek and danish seems to have improved. @xcite and @xcite highlight the marginal representation in research of languages with a small language community and a low economic weight. the results based on an academic context are not deviating from results based on the entire context as presented in the present paper @xcite . additionally, @xcite point out the more complex the technical task, the worse the technical support languages with a small number of speakers have, i. e., the size of the language community seems to have an influence on the technical support. @xcite predict the correlation between data sets and the country of origin with three factors: gdp, size of the language community and geographic proximity. most of the data sets came from economically prosperous countries, thus the best predictive value was the gdp. additionally, @xcite show that the gdp has a better predictive power regarding the publication of papers than the number of speakers of a language. according to these results, the gdp has a stronger influence in academia than the size of the language community. however, if language communities have both, a low gdp and few speakers, special effort and support are needed to ensure technical support. according to the northwest to southeast divide identified, it is the context of language communities in the east and south of europe that needs to be strengthened to achieve dle. in the meta-net white paper series, only three languages spoken in eastern europe achieved "moderate support" once in the four areas. in comparison, the technical support of nine languages spoken in the west was rated as "moderate" at least once. since no other related studies exist, these results can only be discussed in a broader context. for example, @xcite identified an east-west difference in europe using data on economic and social development patterns. although fewer factors were examined, the same pattern emerges. the difference between northwest and southeast needs to be reduced to enable all language communities to participate in the digital society. the results are particularly poor for small language communities. in order for the eu to be a truly equal association of countries and language communities, the differences must be evened out. otherwise, the impact of language barriers (section 2.1) will remain and even reinforce inequalities. the results of the cfs along with the technologi-cal scores form the digital language equality metric. both scores will be presented in an interactive, webbased dashboard and will provide information about the current state of lt support based on the tfs and about the situation of the language communities regarding the further development. together, tf and cf scores/results can be used as the basis for strategic recommendations regarding the future development of languages in the digital world. a language that is poorly supported technologically and has a bad contextual score is unlikely to exhibit significant improvement regarding lt support without changing its context. a language lacking lt support but with a better situational context could indeed take the next steps towards dle in the coming years. currently well-supported languages will continue to do well if their good situational context stays intact, while languages with a good technological score and a rather low context, are likely to stagnate technologically. ## conclusion we present a first approach for the calculation of a score, which is meant to reflect the context of a language with respect to the development of lts. the dle metric consists of technological factors representing the current state of technical support and contextual factors describing the situation for lt development and achievability of dle, especially with regard to the languages covered by ele. the scores can also be used to create initial predictions about the further lt development if the context does not change. our initial methodological approach exhibits room for improvement. this applies in particular to the data collection and preparation. the mapping of data from the country to the language level can be improved, reducing inherent inaccuracies affecting data from language communities with few speakers. another approach could be the calculation of predictive values for individual cfs based on tf scores. this would allow each individual factor to be tested for its predictive power regarding lt development. the results of the five tested configurations show a clear pattern once they are reduced by the factors that distort the results due poor data quality. there exists a greater difference between the scores of the official eu languages and rmls, as well as a gradient from northwest to southeast within the groups. the heuristic expert evaluation has shown that the results of the fifth configuration correspond most closely to reality. the scores of some languages, especially those in a more complicated political environment, do not yet adequately represent their language community's context. these results can be improved using the suggestions presented. the result of this initial approach provides a first starting point from which further development regarding aspects as clarity and reproducibility can be pursued.