Usability Features	In addition to the main algorithms to train and decode with an NMT model, Nematus includes features aimed towards facilitating experimentation with the models, and their visualisation. Various model parameters are configurable via a command-line interface, and we provide extensive documentation of options, and sample set-ups for training systems. Nematus provides support for applying single models, as well as using multiple models in an ensemble – the latter is possible even if the model architectures differ, as long as the output vocabulary is the same. At each time step, the probability distribution of the ensemble is the geometric average of the individual models' probability distributions. The toolkit includes scripts for beam search decoding, parallel corpus scoring and n-best-list rescoring. Nematus includes utilities to visualise the attention weights for a given sentence pair, and to visualise the beam search graph. An example of the latter is shown in Figure FIGREF16 . Our demonstration will cover how to train a model using the command-line interface, and showing various functionalities of Nematus, including decoding and visualisation, with pre-trained models.

Neural Network Architecture	Nematus implements an attentional encoder–decoder architecture similar to the one described by DBLP:journals/corr/BahdanauCB14, but with several implementation differences. The main differences are as follows: We will here describe some differences in more detail: Given a source sequence INLINEFORM0 of length INLINEFORM1 and a target sequence INLINEFORM2 of length INLINEFORM3 , let INLINEFORM4 be the annotation of the source symbol at position INLINEFORM5 , obtained by concatenating the forward and backward encoder RNN hidden states, INLINEFORM6 , and INLINEFORM7 be the decoder hidden state at position INLINEFORM8 .

Introduction	Neural Machine Translation (NMT)  ,  has recently established itself as a new state-of-the art in machine translation. We present Nematus, a new toolkit for Neural Machine Translation. Nematus has its roots in the dl4mt-tutorial. We found the codebase of the tutorial to be compact, simple and easy to extend, while also producing high translation quality. These characteristics make it a good starting point for research in NMT. Nematus has been extended to include new functionality based on recent research, and has been used to build top-performing systems to last year's shared translation tasks at WMT  and IWSLT  . Nematus is implemented in Python, and based on the Theano framework  . It implements an attentional encoder–decoder architecture similar to DBLP:journals/corr/BahdanauCB14. Our neural network architecture differs in some aspect from theirs, and we will discuss differences in more detail. We will also describe additional functionality, aimed to enhance usability and performance, which has been implemented in Nematus.

Conclusion	We have presented Nematus, a toolkit for Neural Machine Translation. We have described implementation differences to the architecture by DBLP:journals/corr/BahdanauCB14; due to the empirically strong performance of Nematus, we consider these to be of wider interest. We hope that researchers will find Nematus an accessible and well documented toolkit to support their research. The toolkit is by no means limited to research, and has been used to train MT systems that are currently in production  . Nematus is available under a permissive BSD license.

Acknowledgments	This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreements 645452 (QT21), 644333 (TraMOOC), 644402 (HimL) and 688139 (SUMMA).

Training Algorithms	By default, the training objective in Nematus is cross-entropy minimization on a parallel training corpus. Training is performed via stochastic gradient descent, or one of its variants with adaptive learning rate (Adadelta  , RmsProp  , Adam  ). Additionally, Nematus supports minimum risk training (MRT)  to optimize towards an arbitrary, sentence-level loss function. Various MT metrics are supported as loss function, including smoothed sentence-level Bleu  , METEOR  , BEER  , and any interpolation of implemented metrics. To stabilize training, Nematus supports early stopping based on cross entropy, or an arbitrary loss function defined by the user.

Nematus: a Toolkit for Neural Machine Translation	We present Nematus, a toolkit for Neural Machine Translation. The toolkit prioritizes high translation accuracy, usability, and extensibility. Nematus has been used to build top-performing submissions to shared translation tasks at WMT and IWSLT, and has been used to train systems for production environments.

Question: What is the architecture of the model?
Answer: attentional encoder–decoder