# Text Detection and Recognition for X-ray Weld Seam Images
## 2 X-ray Weld Seam Image Text Recognition Model Framework
### A Text Recognition Model Based on MATRN  
We employ a text recognition model based on MATRN to recognize characters within the extracted text boxes containing various weld seam information. In order to enhance the accuracy of the text line recognition, we have introduced the CA, CBAM, and HFA modules to improve the network. In the MATRN model, the detected text boxes undergo initial processing through the Feature Extraction Backbone, composed of ResNet and Transformer units [22], to generate visual features. Subsequently, the position attention module is employed to obtain the preliminary text recognition results. Following this, a language model is utilized for further optimization and correction of the obtained text recognition results, resulting in semantic features. To facilitate effective fusion of the visual and semantic features, a Multi-Modal Feature Enhancement module is employed to enhance both types of features. Finally, through the Output Fusion module, the enhanced visual features and semantic features are integrated to produce the ultimate recognition result. The formula is as follows:  
\[V=F^{V.T}\Big{(}F^{V.R}(X)+P^{V}\Big{)} \tag{3}\]  
where \(X\) represents the input image; \(F^{V.R}\) corresponds to a ResNet unit; \(P^{V}\) denotes spatial position encoding, which is added to the input features through an addition operation; \(F^{V.T}\) is a transformer unit; and \(V\) represents the visual features extracted through the backbone network. \[A^{V-S}=softmax\Bigg{(}\frac{P^{S}\mathcal{G}(V)^{T}}{\sqrt{D}}\Bigg{)} \tag{4}\]where \(P^{S}\) serves as the positional encoding for the character order, acting as the query vector. \(T\) represents the length of the character sequence, \(\mathcal{G}(\cdot)\) is a small U-Net unit, \(\mathcal{G}(V)\) functions as the key vector, \(D\) is the feature dimension, and \(A^{V-S}\) is the attention map calculated from the query vector and key vector. \[Y_{(0)}=softmax\bigg{(}A^{V-S}\overset{\sim}{V}W\bigg{)} \tag{5}\]  
where \(\overset{\sim}{V}\) represents the flattened visual features, and through the attention map \(A^{V-S}\), the visual features are abstracted into sequence features, denoted as \(E^{V}=A^{V-S}\overset{\sim}{V}\). \(W\) is a linear transformation matrix, and \(Y_{(0)}\) is the character sequence feature obtained by applying a linear layer and softmax function. \[S=F^{LM}\bigg{(}Y_{(0)}\bigg{)} \tag{6}\]  
where \(F^{LM}\) represents the language model, comprising four transformer decoder units. \(S\) corresponds to the semantic features obtained by processing \(Y_{(0)}\) through the language model. \[P^{Align}=A^{V-S}\overset{\sim}{P} \tag{7}\]  
\[S^{Align}=S+P^{Align} \tag{8}\]  
As \(A^{V-S}\) effectively provides visual features for the character estimation at each position, spatial position encoding can be introduced through flattened spatial position encoding \(\overset{\sim}{P}^{V}\) to determine the spatial positions of the semantic features, denoted as \(P^{Align}\). Subsequently, the spatial position information is combined with the semantic features to obtain spatially aligned semantic features, \(S^{Align}\). \[G=\sigma\bigg{(}\big{[}E^{VM};S^{M}\big{]}W^{sated}\bigg{)} \tag{9}\]  
\[F=G\bigodot E^{VM}+(1-G)\bigodot S^{M} \tag{10}\]  
where \(W^{sated}\) is a weight matrix, \([:]\) denotes concatenation, and \(\bigodot\) represents element-wise multiplication. \(S^{M}\) and \(V^{M}\), respectively, represent the multimodal semantic features and multimodal visual features obtained by enhancing the aligned semantic feature \(S^{Align}\) and visual feature \(V\) using a multimodal transformer unit [23]. \(E^{VM}\) is the serialized representation of \(V^{M}\). Subsequently, a gating mechanism is employed to combine the two sequence features, resulting in the feature \(F\). \[P=softmax(linear(F)) \tag{11}\]  
where \(F\) is passed through a linear layer and softmax function to obtain the final predicted character sequence. We introduce the CA module and integrate it into the backbone.