# CommuSpotter: Scene Text Spotting with Multi-Task Communication
## 3 Text Spotter with Communication
### Text Detection Communication  
Generally, the modality interaction [34] is either conducted by concatenating features and interacting together or by directly conducting interactions among features. Different tasks generate information from different perspectives for the final text spotting in text spotters. Existing text detection and recognition modules share the same backbone for different tasks. Considering the modeling efficiency, we keep the same backbone but introduce Extensive Representation (ER) on pyramid features for different tasks, as shown in Figure 3a. Prior to applying RoIAlign, as detailed in [32], which builds local coherence for feature alignments, we embed high-level pyramid features on a large receptive field for text detection. Specifically, the shared pyramid features \(P_{2}\) to \(P_{5}\) are updated as follows:  
\[P_{i-1}^{*}=Gcm(Ur(P_{i})\cdot P_{i-1}),i=3,4,5, \tag{1}\]  
where \(P_{2}\) is the shape of \(C\times H/4\times W/4\), \(P_{3}\) is the shape of \(C\times H/8\times W/8\), \(P_{4}\) is the shape of \(C\times H/16\times W/16\), and \(P_{5}\) is the shape of \(C\times H/32\times W/32\). \(C\) represents the dimension of channels, set at 256, while \(H\) and \(W\) denote the input resolutions in terms of height and width, respectively. The operation \(Ur(\cdot)\) is used for up-sampling. \(Gcm(\cdot)\) denotes a sequence of convolution layers, consisting of \(1\times 3\) and \(3\times 1\) convolutions. They are applied to reduce dimensionality gradually. Finally, the results are combined as \(P^{*}\), consisting of \(P_{2}^{*}\) to \(P_{5}^{*}\).  
Compared with the general object detection of integrated targets, split characters inside texts always cause false positive detection results due to background interruptions and character-like non-texts. To achieve concurrent semantic guidance, we developed a Conversation Mechanism (CM) through a series of straightforward operations. This mechanism incorporates the expertise of a text recognition module, comprising (1) text semantic segmentation expertise \(S\) (as detailed in Section 3.3) to identify character objects (other than non-texts) and (2) text instance segmentation expertise \(F^{\prime}\) to construct exact locations. Specifically, the refined features, \(P^{*}\), and proposals, \(R\), are fed into the RoIAlign with text semantic segmentation expertise \(S\) to generate the aligned RoI feature \(F_{RoI}\) of the shape \(N\times C\times H_{RoI}\times W_{RoI}\), where \(N\) is the number of mapped proposals, and \(H_{RoI}\) and \(W_{RoI}\) are feature map resolutions set to 7. To obtain text instance segmentation expertise \(F^{\prime}\), we  
Figure 2: Architecture of the proposed CommuSpotter. From the backbone, the pyramid features, \(P\), and the proposals, \(R\), are processed by text detection and text recognition modules. “ER” denotes Extensive Representation, and Conversation Mechanisms are designed for different communications. From the backbone, text semantic segmentation expertise is generated for the text recognition module. Multiple forms of expertise interact with each other concurrently. The black arrows indicate the forward flow of information, while the colorful arrows represent the backward interaction of multi-task expertise.  
conducted a sequence of four convolutional layers \(Conv\) and one transposed convolutional layer \(TrConv\) on the RoI features \(F_{RoI}\). Then, the expertise was combined.  
\[F_{RoI}=RoIAlign(P^{*},R)+S, \tag{2}\] \[F^{\prime}=TrConv(Conv(F_{RoI})),\] (3) \[F^{*}_{RoI}=F^{\prime}+F_{RoI}. \tag{4}\]