## 3 Method of Recognizing Line Objects
### Method of Recognizing Line Objects  
Figure 2b shows the result of recognized continuous lines from the image format P&ID (Figure 2a) when pixel processing-based method adopted in [35; 38] are applied. As shown in Figure 2b, the objects that only correspond to continuous lines are correctly recognized. However, in other types of objects, occurred problems are shown as error case 1, 2, and 3 of Figure 2. Error case 1 in Figure 2 shows a situation in which a line incorporated with a symbol or text is recognized as a continuous line. Error case 2 in Figure 2 illustrates a problem when the line type was not recognized to fit the line sign embedded in the continuous line. The error case 3 in Figure 2 shows a problem that the information of the flow arrow detection is not merged with the line object. A way to solve these problems is to develop a method for robustly recognizing continuous lines and determining the line type after detecting line signals and flow arrow, or to develop a new method for directly recognizing different types of lines. Image processing techniques including pixel processing show high performance in recognizing continuous lines. However, it is not efficient for detecting line signs and flow arrows. On the other hand, deep artificial neural networks [21; 22; 23; 24] show high performance in detecting objects of a set of anchor sizes and ratios. Therefore, it is suitable for detecting line signs and flow arrows. However, it is difficult to detect a continuous line in which a line sign is embedded and a change in length is very large. Considering this, it is feasible and effective to apply the first method. Figure 1: Definition of line objects to be recognized in P&IDs. Following the first approach to the problems, we propose a method for the recognition of multiple types of lines included in P&IDs by utilizing object detection techniques based on deep neural networks and image processing, which is a methodology clearly differentiated from previous related works. The following techniques are applied for each error case. In error case 1 of Figure 2, after detecting the continuous lines first, the lines overlapping with the symbol and text objects are removed using the symbol and text recognition data. In error case 2 in Figure 2, the line signs and flow arrows are detected using a deep neural network that shows a high level of performance in object detection from image. Then, the continuous lines overlapping with the detected line signs are identified, and the line types are changed according to the detected line signs. In error case 3 in Figure 2, after the recognition of line objects, the detected flow arrows are merged with the overlapping line objects. Figure 3 shows the procedure of line object recognition of P&ID in consideration of the methods described above. The recognition process consists of a preprocessing step (Figure 3a,b), a line detection step (Figure 3c-e), and a post-processing step (Figure 3f-i). When the image-format P&ID is input, it is first converted into a binary image (Figure 3a). Then, by identifying the outer border and title box of P&ID and removing them, a diagram composed of high-level objects is extracted (Figure 3b). The line signs and flow arrows embedded in the line object are detected using a deep neural network (Figure 3c). At the same time, continuous lines are detected by applying image processing techniques, including line thinning [27], pixel processing, Hough transform, and bitwise AND(&) operation (Figure 3d). At this time, among the recognized continuous lines, only the pure continuous lines are left by removing the continuous lines that overlap with the symbols and texts (Figure 3e). By checking whether the pure continuous lines and the recognized line signs overlap each other (Figure 3f), the type of the overlapping lines is changed to match the line signs (Figure 3g). After checking the line type changes for all continuous lines, a list of corrected lines is generated (Figure 3h).