CEF Data Marketplace: Powering a Long-term Supply of Language Data (2020)

Abstract: We describe the CEF Data Marketplace project, which focuses on the development of a trading platform of translation data for language professionals: translators, machine translation (MT) developers, language service providers (LSPs), translation buyers and government bodies. The CEF Data Marketplace platform will be designed and built to manage and trade data for all languages and domains. This project will open a continuous and longterm supply of language data for MT and other machine learning applications.

## introduction the cef data marketplace project is an initiative co-funded by the european union under the connecting europe facility programme, under grant agreement inea/cef/ict/a2018/1816453. the project has a duration of 24 months and started in november 2019. with over 350 1 million new internet users in 2019 and the annual digital growth of 9%, there is insufficient content available in the local languages. the automated translation platforms support merely about a hundred of the 4,000 languages with an established writing system. the cef data marketplace will be the first platform that facilitates the buying and selling of language data to help businesses and communities reach c 2020 the authors. this article is licensed under a creative commons 3.0 licence, no derivative works, attribution, cc-by-nd. 1 https://wearesocial.com/blog/2019/ 01/digital-2019-global-internet-useaccelerates scale with their language technologies while offering a way for the language data creators to monetize their work. ## platform description the platform focuses on the integration and maintenance of the already available technologies for managing and trading translation data. specifically, the following features will be added to an existing underlying translation data repository: ## state-of-the-art processing tools advanced data processing services will be integrated to enable and facilitate data exchange through the marketplace and to encourage data sellers and buyers to join the platform. these services consist of software for cleaning, anonymizing and clustering the data to ensure that the data-sets available in the marketplace are of high quality. these services will be provided through apis and will be available free of charge to data providers or against a fee for users not publishing their data through marketplace. the software will also be released open source for the industrial and research communities. ## data acquisition strategy to acquire as much data as possible with added value for the cef automated translation (cef-at) core platform our strategy is to create a vibrant, broader market serving the needs of translation providers and translation buyers across various desired language combinations and domains. the legal framework of taus data is updated to build trust foot_0 of the data owners to participate in the marketplace. clear guidelines are provided about data ownership to safeguard the copyrights and to support the royalty-based model. ## acknowledgement the sole responsibility of this publication lies with the author. the european union is not responsible for any use that may be made of the information contained therein.