Advancing Digital Language Equality in E urope: A Market Study and Open-Source Solutions for Multilingual Websites (2024)

Abstract: The paper presents findings from a comprehensive market study commissioned by the European Commission, aimed at analysing multilinguality of European websites and automated website translation services across various sectors. The findings show that the majority of websites offer content in one or two languages, while only less than 25% of European websites provide content in 3 or more languages. Additionally, we introduce Web-T, a collection of open-source solutions facilitating automated website translation with a help of free MT service eTranslation provided by the European Commission and possibility to integrate other MT providers. Web-T solutions include local plug-ins for Content Management Systems, universal plug-ins, and an MT API Integrator, thus contributing to the broader goal of digital language equality in Europe.

## introduction within the european union, a diverse linguistic landscape is comprised of 24 official languages and more than 60 regional and minority languages. several research studies @xcite @xcite and official resolutions @xcite have underscored a stark discrepancy in the -----------------------© 2024 the authors. this article is licensed under a creative commons 4.0 licence, no derivative works, attribution, cc-by-nd. the challenge of limited multilingual support on websites extends beyond europe and has been highlighted by research in other parts of the world @xcite @xcite @xcite . supporting a website in multiple languages (translating both ui and the content) can be a time-consuming and expensive process. automated translations have revolutionised website localization, making it more accessible to businesses, including smaller enterprises and individuals. despite limitations of automated translations that may not always accurately convey the intended message or account for cultural differences, businesses can benefit from cost savings, speed, and scalability, which allow them to expand their global presence. in many cases, a precise translation is needed (government, legislation, healthcare, industry specifications, brand identity, etc.), and to reach that high quality of translation, so far, the automated translation must be followed by post-editing by humans. however, automated translation is enough in cases when the users need to get a general understanding of the content of a webpage and non-perfect translation won't have critical consequences. the automated translation accelerates the translation process and as far as the language models can be trained on specific topics, languages, jargons, and dialects, the quality of the translations can be improved to a level that requires minimum or no human intervention, making the process even more productive @xcite . although there are numerous automated translation solutions provided by market players, their use is not dominant on the european web space. to assess and improve the situation, the european commission (ec) has commissioned an extensive market study on multilingualism of websites in europe and the development of solutions to support the use of automated translations on websites. the project is implemented in the scope of the digital europe programme's strategic objective 5, "accelerating best use of technologies," and aims to enhance language technologies' capacity within the european public sector and their broader deployment across public and private sectors, ngos, and academia (european commission, 2021). in this paper we present the project findings in analysing the language diversity on the european web space, the use of solutions ensuring automated website translation, and the machine translation services underpinning these solutions. we also introduce a collection of open-source solutions developed under the project, collectively known as web-t. these solutions offer free-ofcharge automated website translations utilizing the european commission's etranslation machine translation (mt) service 1 and are adaptable for integration with other mt providers. ## assessing multilingualism of european websites according to a recent study by idc, there are slightly more than 1 million websites managed by public sector enterprises in europe (eu 27 plus albania, north macedonia, switzerland, serbia, montenegro, and bosnia and herzegovina) and about 8.4 million websites managed by private sector entities in europe. to assess the multilinguality of european websites, two randomised sample lists of european ----------------------- 1 https://commission.europa.eu/resources-partners/etranslation_en websites were compiled. the lists of websites for analysis were compiled by combining a list of websites per country available on builtwith.com, lists of small and medium enterprises (smes) provided by national registries, trade organisations, and lists of government institutions, universities, schools, and healthcare institutions on a national level and a regional/city level for each country. a random subset of the lists was used for analysis. one list contains websites sampled from domains of the largest economies: germany, france, italy, austria, and the netherlands. the list is balanced to include about 20% public sector and about 55% smes, with the rest being large or medium companies. the second list contains links to websites of enterprises in eu 27 member states and is balanced between big companies, smes, and the public sector. the multilingualism scoring tool @xcite was used to measure the multilingualism of a website. it analyses the textual content of the website and identifies the number of languages used, the distribution of content in various languages, and the presence of multilingual features. multilingual features are website features that point to this webpage being available in other languages and offering user access to this content, such as language switcher tool/button/link, machine-readable links to translated content, or blocks of text in various languages available for display using javascript. from the first list of largest economies, 426 websites were crawled with a depth of 2 links. most websites contain at least one page in two languages (42%), 30% of crawled websites are monolingual when crawled to a depth of two links, while the rest (28%) have at least one page of content in 3 or more languages. from the second list of eu 27, 401 websites were crawled and analysed (figure 1 ). in this case, when crawled 2 links deep, one-third of websites have content in two languages, 24% of websites are monolingual and the rest (~43%) are multilingual, i.e., having at least one page of content in 3 or more languages. as usually links to valuable content in other languages are provided on the landing page of the website, we compared these results with those from crawling with the depth of one link. in this case, less than 20% of websites from the countries with the largest economies and less than 25% of eu 27 websites yielded content in 3 or more languages. this shows that for many websites multilingual content is available only for some of the content on deeper levels. a detailed analysis was performed on 698 public sector websites across 31 european countries (eu 27 countries plus albania, serbia, north macedonia, and switzerland) for the number of languages in which the websites are published (figure 2 ). we found that government websites are significantly more multilingual than websites of education (nace (the statistical classification of economic activities in the european community) code 85) and healthcare (nace codes 86, 87, 88) institutions. ## solutions for website multilinguality website multilingualism is enabled by an ecosystem of solutions that includes website builders, content management systems (cms), machine translation services, systems to manage translation workflows, and plug-ins. tools such as website builders like elementor and wpbakery, along with ecommerce platforms like wix and shopify, provide a range of built-in translation tools to enable website owners to create multilingual websites without any coding experience. cms such as wordpress, drupal, and joomla integrate plug-ins and extensions that allow website owners to easily translate their websites. in the backend, automated translation services such as google translate, microsoft translator and deepl provide machine translation to automatically translate website content into multiple languages. the translation management systems (tms) of language technology vendors like phrase, unbabel, tilde, and many others also enable website translations. ## web-t solutions to address the need for website translation, we have developed a web-t website translation solution. in accordance with ec requirements, the website translation solutions that are developed are free of charge, easy to use, secure, implementable on various platforms, flexible, adaptable to different cmss, integrate free mt service etranslation provided by the european commission, as well as open for other machine translation providers. the key findings from the user survey and existing website translation plugin review were included in the requirements when designing the solutions. ## conclusion our study underscores the limited diversity of languages in the european web space and the pivotal role of automated translation tools in streamlining website localization. it highlights the need for user-friendly, accurate, and costeffective solutions. the analysis of user requirements, the web-t architecture, and opensource solutions offer practical guidance for extending the availability and use of automated website translation solutions. this contributes to the goal of achieving true multilinguality of european web space and advancing digital language equality in europe.