Design of an Open-Source Architecture for Neural Machine Translation (2023)

Abstract: adaptNMT is an open-source application that offers a streamlined approach to the development and deployment of Recurrent Neural Networks and Transformer models. This application is built upon the widely-adopted OpenNMT ecosystem, and is particularly useful for new entrants to the field, as it simplifies the setup of the development environment and creation of train, validation, and test splits. The application offers a graphing feature that illustrates the progress of model training, and employs SentencePiece for creating subword segmentation models. Furthermore, the application provides an intuitive user interface that facilitates hyperparameter customization. Notably, a single-click model development approach has been implemented, and models developed by adaptNMT can be evaluated using a range of metrics. To encourage eco-friendly research, adaptNMT incorporates a green report that flags the power consumption and kgCO2 emissions generated during model development. The application is freely available.

## introduction explainable artificial intelligence (xai) @xcite aims to ensure that the outcomes of ai solutions are easily comprehensible to humans. in light of this goal, adaptnmt has been developed to provide users with a form of explainable neural machine translation (xnmt). the typical nmt process comprises several independent stages, including setting up the environment, preparing the dataset, training subword models, parameterizing and training the main models, evaluating and deploying them. by adopting a modular approach, this framework has established an effective nmt model development process that caters to both technical and non-technical practitioners in the field. to address the environmental impact of building and running large ai models @xcite , we have also produced a "green report" that calculates carbon emissions. while primarily intended as an information aid, this report will hopefully encourage the development of reusable and sustainable models. this research endeavors to create models and applications that address the challenges of language technology, which will be particularly beneficial for those new to the field of machine translation (mt) and those seeking to learn more about nmt. the application is built on opennmt foot_0 @xcite and thus inherits all of its features. unlike many nmt toolkits, a command line interface (cli) is not used, and the interface is designed and fully implemented in google colab. 3 for both educational and research purposes, a cloud-hosted solution like colab is often more user-friendly. additionally, the training of models can be monitored and controlled via a google colab mobile app, which is useful for long-run builds. the adaptnmt framework also includes gui controls that allow for the customization of all crucial parameters needed for nmt model training. the application can be run in local mode to utilize existing infrastructure or hosted mode for rapid infrastructure scaling. a deploy function is also included to allow for the immediate deployment of trained models. this paper begins by presenting background information on nmt and nmt tools in section 3, followed by a detailed description of the adapt-nmt architecture and its key features in section 4. the system is discussed in section 5 before concluding with a discussion of future work in section 6. a more in-depth system description, coupled with an empirical evaluation of models developed using the application, is outlined in a separate paper @xcite . ## architecture of adaptnmt after providing a general overview of nmt and nmt development systems, we introduce the adaptnmt tool, which enables users to configure the components of the nmt development process. the platform's system architecture is depicted in figure 1 . the tool is built as an ipython notebook and leverages the pytorch implementation of opennmt for training models. additionally, sen-tencepiece is used to train subword models. using a jupyter notebook facilitates sharing the application with other members of the mt community, and the application's setup is simplified since all necessary packages are downloaded dynamically as the application runs. the system has two deployment options: running it locally or as a colab instance via google cloud. in order to build translation models, the system requires parallel text corpora for both the source and target languages. a tensorboard visualization allows for real-time monitoring of the model training process. at runtime, users can select to use the system for either model building or translation services, or both. additionally, as depicted in figure 1 , the system enables the generation of an ensemble output during translation. finally, trained models can be easily deployed to a pre-configured location. ## discussion numerous tools have been developed to assess the carbon footprint of nlp @xcite . the notion of sustainable nlp has also gained momentum as an independent research track, with high-profile conferences such as the eacl 2021 green and sustainable nlp track dedicating resources to this area. 9 given these developments, we have incorporated a "green report" into adaptnmt that logs the kgco 2 generated during model development. this aligns with the industry's increasing focus on quantifying the environmental impact of nlp. in fact, it has been demonstrated that highperforming mt systems can be developed with much lower carbon footprints, leading to significant energy cost savings for a real translation company @xcite . the risks associated with relying on large language models (llms) have been welldocumented in the literature. the discussion surrounding these models emphasizes not only their environmental impact but also the inherent biases and dangers they pose for low-resource languages @xcite . it is important to note that smaller, in-domain datasets can yield high-performing nmt models, and the adaptnmt framework makes this approach easily accessible and understandable. ## conclusion and future work we have introduced adaptnmt, an application that manages the entire nmt model development, evaluation, and deployment workflow. as for future work, our development efforts will be directed towards incorporating new transfer learning methods and improving our ability to track environmental costs. we will integrate modern zero-shot and few-shot approaches, as seen in the gpt3 @xcite and facebook laser @xcite frameworks. while the existing adaptnmt application is focused on customizing nmt models, we will also develop a separate application, adaptllm @xcite , for finetuning llms. in particular, adaptllm will cater for low-resource language pairs such as nllb @xcite . the green report integrated into the application represents our first implementation of a sustainable nlp feature within adaptnmt. we plan to enhance this feature by improving the user interface and providing recommendations on how to develop greener models. as an open-source project, we invite the community to contribute new ideas and improvements to the development of this feature.