# Neural Machine Translation of Electrical Engineering Based on Vector Fusion
## 3 Neural Machine Translation Models for Vector Fusion
### Multi-Attention Mechanism Translation Model Based on Vector Fusion  
The attention mechanism in the Transformer model is mainly divided into two types: self-attention and context-attention. As shown in Figure 3, the internal structure and calculation method of these two attention mechanisms are the same. The difference between the two is the query vector Q (query). In self-attention, the query vectors Q (query), K (key) and V (value) are exactly the same. However, the context-attention is used to pass the encoder information to the decoder so that in the decoding process, the source language representation vector obtained by the encoder can be better assigned weights according to the target information. Therefore, the query vector Q (query) comes from the decoder unit, while the key-value pairs K (key) and V (value) still come from the encoder's output vector and have the same value.  
Its calculation formula is:  
\[Attention(Q,K,V)=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V \tag{8}\]  
where \(d_{k}=d_{model}/h\), \(d_{model}\) is the vector dimension within the translation model, and \(h\) is the number of heads of multi-head attention mechanisms.  
The decoder unit of the Transformer model consists of a multi-head self-attention mechanism, a multi-head context-attention mechanism, a fully connected layer, residual connections, and layer normalization structures. The underlying self-attention extracts internal information from the input of the target side. The intermediate context attention layer dynamically assigns weights to the incoming encoder-side source language feature vectors based on the target language feature vectors extracted by the decoder for subsequent decoding.  
The hidden layer vector obtained after weight allocation integrates the source language information into the decoder side but also reduces the target language information contained in it. Therefore, in order to further improve the ability of the decoder to extract features, the decoder unit has been improved in this paper, and an attention mechanism is superimposed after the context-attention layer. Further, on the basis of vector fusion, a multi-attention mechanism translation model based on vector fusion is proposed, as shown in Figure 4.  
Figure 4: Multi-Attention Mechanism Translation Model Based on Vector Fusion.  
Figure 3: Attention mechanism.  
Inside each decoder unit, when the fusion vector output by the encoder is input to the decoder as a key-value pair K (key) and V (value), it first enters the context-attention layer in the model and transmits the fused source language information to the decoder, and the output of the self-attention layer is used as the query vector Q (query) to calculate the attention mechanism. The target language information is used as guidance to extract the source language information contained in the fusion vector at this layer. Then, the output vector obtained from this layer is used as the key-value pair K (key) and V (value) to input into the additional attention mechanism layer. The target language features output from the bottom self-attention layer are still used as the query vector Q (query) to calculate the multi-attention mechanism. The calculation method and parameter setting of the attention layer are consistent with the multi-head attention mechanism in the original model, which is the same as Formula 8. The improved decoder structure can further extract the internal features of the fusion vector and better realize the transformation process from the source language features to the target language features.