TICO -19: the Translation Initiative for CO vid-19 (2020)

Abstract: The COVID-19 pandemic is the worst pandemic to strike the world in over a century. Crucial to stemming the tide of the SARS-CoV-2 virus is communicating to vulnerable populations the means by which they can protect themselves. To this end, the collaborators forming the Translation Initiative for COvid-19 (TICO-19) have made test and development data available to AI and MT researchers in 35 different languages in order to foster the development of tools and resources for improving access to information about COVID-19 in these languages. In addition to 9 high-resourced, ”pivot” languages, the team is targeting 26 lesser resourced languages, in particular languages of Africa, South Asia and South-East Asia, whose populations may be the most vulnerable to the spread of the virus. The same data is translated into all of the languages represented, meaning that testing or development can be done for any pairing of languages in the set. Further, the team is converting the test and development data into translation memories (TMXs) that can be used by localizers from and to any of the languages.

## introduction the covid-19 pandemic marks the worst pandemic to strike the world since 1918. at the time of this writing, 3 the sars-cov-2 coronavirus responsible for covid-19 has infected over ten million people worldwide, with over a half a million deaths. while these numbers are likely under-reported, they are growing at an alarming rate, and many millions of people could become infected or perish without proper prevention measures. effective communication from health authorities is essential to protect at-risk populations, slow down the spread of the disease, and decrease its morbidity and mortality (unocha, 2020). yet, preventive measures such as stay-at-home orders, social distancing, and requirements to wear personal protective equipment (e.g. masks, gloves, etc.) have proven difficult to relay. that's not accounting for the difficulty to disseminate correct technical information about the disease, such as symptoms (e.g., fever, chills, etc.), specifics about testing (e.g., viral vs. antibody testing), and treatments (e.g., intubation, plasma transfusion). while official communications from the world's health organization (who) are constantly published and revised, they are mostly limited to major languages. this has resulted in a vacuum in many languages that has been filled by an infodemic of misinformation, as described by the who. nongovernmental organizations (ngos) such as translators without borders (twb) play an important role in delivering multilingual communication in emergencies, such as the covid-19 pandemic, but their reach and capacity has been outsized by the needs presented by the pandemic. to date, twb has translated over 3.5 million words with over 80 non-profit organizations for more than 100 language pairs as part of their covid-19 response. translation technologies such as automatic machine translation (mt) and computer assisted translation (cat) present unique opportunities to scale the throughput of human translators. however, given the sensitivity of the content, it is critical that the translations produced automatically are of the highest possible quality. the translation initiative for covid-19 (tico-19) effort marks a unique collaboration between public and private entities that came together shortly after the beginning of the pandemic. @xcite the focus of tico-19 is to enable the translation of content related to covid-19 into a wide range of languages. first, we make available a collection of translation memories and technical glossaries so that language service providers (lsps), translators and volunteers can make use of them to expedite their work and ensure consistency and accuracy. second, we provide an open-source, multi-lingual benchmark set (which includes data for very-lowresource languages) specialized in the medical domain, which is intended to track the quality of current machine translation systems, thus enabling future research in the area. lastly, we provide monolingual and bi-lingual resources for mt practitioners to use in order to advance the state-of-the-art in medical and humanitarian machine translation, as well as other natural language processing (nlp) applications. our hope is that our work will in the short-term enable the translation of important communications into multiple languages, and that in the long-term, it will serve to foster the research on mt for specialized content into low-resource languages. through these resources we hope that our society is better prepared to quickly respond to the needs of translation in the midst of crises (e.g., for future crises, a la lewis et al. (2011)). ## the value of translation technologies in crisis scenarios during a crisis, whether it is local to one region or is a worldwide pandemic, communicating effectively in the languages and formats people understand is central to effective programs on the ground. for example, as part of the effort to control the spread of covid-19, the global humanitarian response plan recognizes community engagement in relevant languages as a key strategy (unocha, 2020). @xcite in some countries, this will be all the more @xcite the world health organization (who) declared covid-19 a pandemic on march 11th, 2020. the tico-19 collaborators came together in the days following and first met as a group (over zoom) on march 20th. it cannot be understated the rapidity with which this collaboration came together and how seamlessly the participants, many erstwhile competitors, have worked in harmony and without animosity. it is truly a testament to the needs of the greater good outweighing personal differences or potentially conflicting objectives. @xcite the plan does not call out machine translation per se, but does call out the need for content to be produced and disseminated in "accessible languages", and the need for communication in "local languages". historically, mt, nlp and translation technologies have played a crucial role in crisis scenarios. the response to the haitian earthquake in 2010 was notable for the broad use of technology in the humanitarian response, relying more on crowdsourced translations and geolocation, but notably, translation technology was also used. in the days following the earthquake, haitian citizens were encouraged to text messages requesting assistance to "4636", and as many as 5,000 messages were texted to this number per hour. unfortunately for the aid agencies, whose dominant languages were english and french (aid agencies included the us navy, the red cross, and doctors without borders), most of the sms messages were in haitian kreyòl. quickly, the haitian kreyòl speaking diaspora around the world were activated by the mission 4636 consortium to translate the sms messages and geolocate (munro, 2010), and the translated messages were handed off to aid agencies for triage and action. the mission 4636 infrastructure included a high-precision rule-based mt (lewis et al., 2011), and within days to weeks after the earthquake, statistical mt engines were brought online by microsoft (lewis, 2010) and google. @xcite translation technology continues to be used in a variety of crisis and relief scenarios. notable among these is translators without borders (twb) use of translation memories for translating to a number of under-resourced languages in relief settings. likewise, the standby task force, 7 who are activated in a variety of relief settings, note the use of mt in various deployments around the world, e.g., for urdu in the pakistan earthquake of 2011 and for spanish in the ecuador earthquake in 2016. the eu funded international network on crisis translation (interact) 8 project, started a couple of years before the covid-19 pandemic, focused on crisis translation, specifically in health crises such as pandemics, with a focus on improving resilience in times of crises through communication, ultimately with the goal of reducing loss of life. @xcite likewise, during the current pandemic, several community-driven efforts have sprung up to fulfil the need for information communication. the endangered languages project foot_4 , for example, has collected community-produced translations of public health information in hundreds of languages in various formats. what is not tracked is the degree to which publicly available mt tools and resources are used in crisis and relief settings, e.g., translation apps and tools from amazon, google and microsoft, or the translation feature built into facebook (e.g., automatically translating posts). the authors suspect use may be broad, but there are no published accounts documenting just how broadly and how much these tools are used in crises. tantalizing evidence of the use of publicly available tools was noted by lewis et al. (2011) who documented traffic in the microsoft translator apps in the weeks following the haitian earthquake: they noted that at least 5 percent of the haitian kreyòl traffic was relief-related. it is likely that google's and microsoft's apps are used even when cell phone infrastructure is unavailable or destroyed, since the tools permit users to download models to their devices so they can perform offline translations. @xcite in crises, it is clear that organizations need the capacity to communicate critical information and key messages into the languages people understand, at speed and at scale. crisis affected communities could access content in local languages through various channels such as sms, online chatbots, or more traditional printed materials. their questions and feedback can be used to refine content to better meet their needs. likewise, relief agencies need access to sms and other communiques in local languages in order to more effectively and equitably distribute aid. mt can help the various actors to translate and disseminate essential communications in a timely manner without the need to wait on human transla-tors. this is particularly important in low resourced languages where professional translators are not readily available. domain-specific mt can also assist translators with the right terminology to convey the correct response and standardize concepts. furthermore, people who are unable to understand major languages could get access to vital information (such as news sources, websites, etc) first hand via a mt-driven tool set. we emphatically note that communication is not just a translation problem. especially in the case of under-represented indigenous communities, messaging needs to also remain respectful of cultural norms (e.g. communicate through appropriate channels, without undermining cultural authorities and practices) and to not minimize the agency of such communities through "deficit framing" (wanambi et al., 2020). nevertheless, translation is a crucial component of the information flow. however, to be useful for translating specialized content such as medical texts, we require that automatic translations be of the highest possible accuracy. to advance the research in machine translation, we require both high-quality benchmark sets and access to basic training resources, both monolingual and parallel. likewise, translation memories in a broader set of languages can help localizers around the world translate into these languages. in the remainder of this paper we describe the resources created by the tico-19 initiative, and some evaluations against them. ## the tico-19 translation benchmark we created the tico-19 benchmark with three criteria in mind: diversity, relevance and quality. first, we sampled from a variety of public sources containing covid-19 related content, representing different domains. second, to make our content relevant for relief organizations, we chose the languages to translate into based on the requests from relief organizations on-the-ground. third, we established a stringent quality assurance process, to ensure that the content is translated according to the highest industry standard. ## translator resources translation memories because of the breadth of languages covered by tico-19, and the fact that so many are under-resourced, the translations themselves can be of significant value to localizers. as part of the effort, the tico-19 collaborators have converted 20 the translated data to translation memories, cast as tmx files, for all @xcite conversion was carried out with the tikal command included in the okapi framework: https://okapiframework.org/ . english-x pairings, as well as some other pairings of languages focusing on potential local needs (e.g. french-congolese swahili, farsi-dari, and kurdish kurmanji-sorani). these tmx files, in addition to the test and development data, have been made available to the public through the project's website. terminologies two sets of translation terminologies were provided by facebook and google (the complete set of the english source terms and of the translated languages are listed in appendix e): additional translations translators without borders (twb) worked with its network of translators to provide translations in hard-to-source languages (e.g. congolese swahili and kanuri). it also provided covid-19 specific sources from its diverse humanitarian partners to augment the dataset. this augmented dataset will be available under license on twb's gamayun portal. foot_13 ## mt developer resources as part of our project the cmu team also collected some covid-19-related monolingual data in multiple languages. they are available online, @xcite but we note that some of these data might not be available under the same license as our datasets (and hence might not be appropriate for commercial system development). these are detailed in the next sections. ## conclusion enabling efficient and accurate communication through translations still has a ways to go for the majority of the world's languages and particularly the most vulnerable ones. with this effort we only address a fraction of the needs for a fraction of the world's languages. nevertheless, we hope that the mt resources that we release will have an immediate impact for the languages we cover. more importantly, the benchmark we release will allow the mt research community, both academic and industrial, to be more prepared for the next crisis where translation technologies will be needed. 8 : qa score across languages. we report the initial qa score and whether re-work was required on the produced translations (beyond the qa sample, due to serious translation errors). in cases where the initial qa yielded very poor results, the translations were corrected in their entirety and a new qa process was performed, and we report both the initial and the final qa results. note: a "n/a" final qa score in the score indicates that the final score is temporarily not available and we will report it in an updated version of the paper.