plain X - AI Supported Multilingual Video Workflow Platform (2022)

Abstract: The plain X platform is a toolbox for multilingual adaptation, for video, audio, and text content. The software is a 4-in-1 tool, combining several steps in the adaptation process, i.e., transcription, translation, subtitling, and voice-over, all automatically generated, but with a high level of editorial control. Users can choose which translation engine is used (e.g., MS Azure, Google, DeepL) depending on best performance. As a result, plain X enables a smooth semi-automated production of subtitles or voice-over, much faster than with older, manual workflows. The software was developed out of EU research projects and has recently been rolled out for professional use. It brings Artificial Intelligence (AI) into the multilingual media production process, while keeping the human in the loop.

## introduction the platform simplifies the multilingual adaptation process to a large degree, enabling easy subtitling in source and any target language requirement. after a full year of preparation, we are currently rolling out the platform for daily use in deutsche welle. some other organizations are trialing the tool. in the future the software will be available to others, based on a software-as-aservice subscription model. ## challenges the concept for plain x originated from the need to produce more with less, i.e., to use automation in the production process, so media producers can increase the volume of certain target languages, distribute content in more languages, or use synthetic voice, allowing to reach more people in their own spoken tongue, including in specific african or asian regions. as dw produces content in so many languages, it is essential to cover as many languages as possible, in the best possible quality, through a combination of engines from carefully selected providers, for instance for transcription or translation. in plain x users can freely switch between different translation engines. the software allows for the inclusion of additional engines in the future. as the tool wasand isco-developed by user partner deutsche welle, direct access to user requirements and feedback is ensured. this revealed that integration with internal systems and customization is a must to reach the highest level of user acceptance. plain x initially came out of the summa multilingual media platform, funded by the european commission's h-2020 project as a basic prototype for controlled transcription and translation. this prototype was then further developed and funded through the google digital news initiative projects speech.media and news.bridge. finally, deutsche welle, world broadcaster in need of such platform, and priberam, a natural language processing developer, decided to turn the prototype into a scalable, fully operational multilingual platform for wider use, supporting the needs of broadcasters and other multilingual content producers. that was the birth of plain x, a platform which turns content from virtually any language into almost any target language. ## workflow the task-based workflow is easy to use, but very powerful, offering editorial users the comfort of their familiar workflow, yet encompassing advanced automated technologies to support them in the creative process. the first step is ingestion of content, be it video, audio, or text, with many input formats. the next step for audiovisual content is transcription, through speech-to-text in the source language. that could be an end-goal, for instance for interviews. this also allows for a primary output of automatically generated source-language subtitles, which can be used as open or closed captions. the next step is automated translation to a selected target language, which can be post-edited to any level. again, the translation can be an endgoal on its own, and used as input text for respeaking, for example. one file can be translated to multiple languages. however, it can also generate automated subtitling in the same target language. as a final step, the translation can be used for voice-over, by converting text to speech in the target language after selecting a synthetic voice. post-editing and review by colleagues can be added in every step, as required. subsequently, other target languages can be added and produce equivalent content. ## integration it was vital to integrate this tool into the existing workflow infrastructure at deutsche welle and to allow for customization. this meant connecting it to input platforms for a smooth ingestion, as well as output tools for an efficient postproduction and publication in the company style and branding. subtitle templates help to prepare the output in a particular house format. other customizations include library management and access, setting subtitling rules, assigning roles to users, keeping track of usage and billing. it is possible to create fully automated processes for subtitling. working directly in a user environment from the start, with user input and feedback at every stage, allowed us to build a user-oriented platform to support editors in their adaptation process with the help of ai, while minimizing the feeling of insecurity and threat coming from automated processing. more enhancements are planned to cater for different use cases, improve the quality of the output and strengthen post-editing options.