Self-Modifying State Modeling for Simultaneous Machine Translation (2024)

Abstract: Simultaneous Machine Translation (SiMT) generates target outputs while receiving stream source inputs and requires a read/write policy to decide whether to wait for the next source token or generate a new target token, whose decisions form a decision path. Existing SiMT methods, which learn the policy by exploring various decision paths in training, face inherent limitations. These methods not only fail to precisely optimize the policy due to the inability to accurately assess the individual impact of each decision on SiMT performance, but also cannot sufficiently explore all potential paths because of their vast number. Besides, building decision paths requires unidirectional encoders to simulate streaming source inputs, which impairs the translation quality of SiMT models. To solve these issues, we propose Self-Modifying State Modeling (SM ^2 ), a novel training paradigm for SiMT task. Without building decision paths, SM ^2 individually optimizes decisions at each state during training. To precisely optimize the policy, SM ^2 introduces Self-Modifying process to independently assess and adjust decisions at each state. For sufficient exploration, SM ^2 proposes Prefix Sampling to efficiently traverse all potential states. Moreover, SM ^2 ensures compatibility with bidirectional encoders, thus achieving higher translation quality. Experiments show that SM ^2 outperforms strong baselines. Furthermore, SM ^2 allows offline machine translation models to acquire SiMT ability with fine-tuning.

## introduction simultaneous machine translation (simt) @xcite @xcite outputs translation while receiving the streaming source sentence. different from normal offline machine translation (omt) @xcite , simt needs a suitable read/write policy to decide whether to wait for the coming source inputs (read) or generate target tokens (write). as shown in figure 1 (a), to learn a suitable policy, existing simt methods usually require building a decision path (i.e., a series of read and write decisions made by the policy) to simulate the complete simt process during training @xcite . methods of fixed policies @xcite build the decision path based on pre-defined rules, and only optimize translation quality along the path. methods of adaptive policies @xcite @xcite dynamically build the decision path and optimize the policy based on the simt performance along this path. however, the current training paradigm based on decision paths faces inherent limitations. first, it can lead to imprecise optimization of the pol-icy during training. for fixed policies, pre-defined rules cannot ensure optimal decisions at each state. for adaptive policies, there exists a credit assignment problem @xcite , which means it is difficult to identify the impact of each individual decision on the global simt performance along a path, thus hindering the precise optimization of each decision. second, due to numerous potential decision paths, existing methods @xcite @xcite often prohibit the exploration of some paths during training, but this insufficient exploration cannot ensure the optimal policy. third, for building decision paths in training, existing methods require unidirectional encoders to simulate streaming source inputs and avoid the leakage of source future information @xcite , which impairs simt models' translation quality @xcite . to address these issues, we propose self-modifying state modeling (sm 2 ), a novel training paradigm for simt task. as shown in figure 1 (b), instead of constructing complete decision paths, sm 2 individually optimizes decisions at all potential states during training. this paradigm necessitates addressing two critical issues: firstly, how to independently optimize each decision based on its own contribution to simt performance; and secondly, how to sufficiently explore all potential states during training. to realize the independent optimization, sm 2 assesses each decision by estimating confidence values which measure the translation credibility. high confidence means the simt model can predict a credible target token at current state and write is beneficial for simt performance; otherwise, read is preferred. since golden confidence values are unavailable, sm 2 introduces self-modifying process to learn accurate confidence estimation @xcite . specifically, during training, the simt model is allowed to modify its prediction based on the received source prefix with the prediction based on the complete source sentence, and the confidence is estimated to determine whether the modification is necessary to ensure a credible prediction at current state. to sufficiently explore all potential states, sm 2 conducts prefix sampling to divide all states into groups according to the number of their received source prefix tokens, and sample one group for optimization in each iteration. compared to the training paradigm based on decision paths, sm 2 presents significant advantages. first, the self-modifying process can assess each decision independently, which realizes the precise optimization of policy without the credit assignment problem. second, prefix sampling ensures sufficient exploration of all potential states, promoting the discovery of the optimal policy. these benefits enable sm 2 to learn a more effective policy. furthermore, without building decision paths in training, sm 2 ensures compatibility with bidirectional encoders, thereby improving translation quality. this compatibility also allows omt models to acquire the simt capability via fine-tuning. our contributions are outlined in the following: ## background simultaneous machine translation for simt task, we respectively denote the source sentence as @xmath0, ..., x m ) and the corresponding target sentence as @xmath1, .., y n ). since the source inputs are streaming, we denote the number of source tokens available when generating y i as g i , and hence the prediction probability of y i is p(y i | @xmath2, @xmath3. thus, the decoding probability of y is given by: decision state and decision path we define the state s ij as the condition in which the source prefix @xmath4. at s ij , a decision d @xmath5,read} can be made based on the context (@xmath6, @xmath7. specifically, if @xmath8, d ij should be write; otherwise, d ij should be read. as shown in figure of decisions [d 00 , ..., d n m ] are made in the simt process, which forms a decision path from s 00 to s n m . along the decision path, the simt model can finish reading the whole x and outputting the complete y @xcite . since these concepts are usually used in simt methods based on reinforcement learning (rl) @xcite , we compare our method with rl-based methods in appendix a for clarity. ## the proposed method we propose self-modifying state modeling (sm 2 ), which individually optimizes decisions at all states. the overview of sm 2 is shown in figure 2 . to independently optimize each decision, sm 2 learns confidence estimation to assess decisions at each state by modeling the self-modifying process (sec.3.1). to ensure sufficient exploration during training, sm 2 conducts prefix sampling to traverse all potential states (sec. 3.2). then, based on estimated confidence at each state, sm 2 can determine whether the received source tokens are sufficient to generate a credible token and make suitable decisions during inference (sec.3.3). ## conclusion in this paper, we propose self-modifying state modeling (sm 2 ), a novel training paradigm for simt. sm 2 eschews the construction of complete decision paths during training, opting to explore all potential states individually instead. by introducing the self-modifying process, sm 2 independently assesses each state to precisely optimize the read/write policy without the credit assignment problem. through prefix sampling, sm 2 ensures sufficient exploration of all potential states. experimental results across three language pairs validate the superior performance of sm 2 , and our analyses further confirm that sm 2 can learn a more effective read/write policy. more promisingly, sm 2 demonstrates the potential to endow omt models with simt capability through fine-tuning.