# CommuSpotter: Scene Text Spotting with Multi-Task Communication
## 1 Introduction  
Scene text spotting aims to detect and recognize various texts in complex scenes simultaneously. Derived from general object detection, scene text detection faces many challenges, such as scattered and changing characters in different word detection, text style and font variations, background interruptions, etc. On the other hand, scene text recognition also incorporates language processing techniques to decode sequence features for text prediction. The end-to-end training and application of scene text detection and recognition have garnered increased research interest in recent years. It not only integrates the two main tasks of text detection and recognition, but it also bridges the training schemas of different tasks for optimization. It has wide applications in artificial intelligence and computer vision, such as criminal investigation [1], video information retrieval [2], robotic assistance [3], and autonomous driving [4]. Modern scene text spotting comes up with challenges, such as artificially embellished texts, various text perspectives, and complex text shapes. Existing two-stage paradigm text spotters conduct text detection tasks for text localization and text recognition tasks for words, while one-stage paradigm text spotters remove text detection but introduce customized modules for connecting the backbone with text recognition modules. The configuration examples of typical text spotters are collected in Table 1. Transformer-based backbones and text detectors greatly simplify proposalgeneration and boost performance at the expense of the computation cost. The customized modules are specifically designed to shrink the text instances or refine the features. Although current end-to-end scene text spotters [5; 6; 7; 8; 9; 10; 11; 12] have achieved substantial progress, there are three limitations. Firstly, the information or expertise from various multi-tasks compensates for each other in the final text spotting, but some expertise is lost in the current independent multi-task methods. For example, many studies improve information usage by sharing the network backbone [9; 12; 13], integrating the recognition loss [14; 15], or bridging customized modules [5; 6; 7; 8], as shown in Figure 1a. However, the interaction between tasks is not adequately identified. As indicated by the yellow arrow, there is only compensation from the text detection task (or customized modules) to the text recognition task, but not inversely, where the expertise from customized modules or text recognition is scarcely utilized to interact with early text detection tasks or proposal generations. There is no communication or usage of each other's information for multi-tasks in the text-spotting process. Recently, some text spotters [14; 15] have adopted a Transformer to understand the relationship between text instances but not between tasks. As a result, independent multi-task modulation leads to the loss of concurrent compensatory expertise from multiple tasks in text spotting. Secondly, apart from task-specific information loss, pipeline errors accumulate in current unidirectional text spotters. Errors from early tasks are not identified and, thus, accumulate in later tasks, leading to back-and-forth training. During inference, the methods become unreliable from the start when there is no ground truth in each step. Thirdly, the sophisticated customized modules increase model complexity and require expensive extra annotations. For example, some customized modules involve designs and labels for text lines [16], text strokes [17], text center points [18], and so on. In this work, we propose CommuSpotter, which explicitly and concurrently communicates multi-task expertise with others. Multi-task communication involves information