# Multi-Constrained Optimization Method of Line Segment Extraction Based on Multi-Scale Image Space
## 1 Introduction  
The line feature is an important part of an image's geometric information and plays a crucial role in photogrammetry and remote sensing [1], three-dimensional (3D) urban modeling [2; 3], computer vision and robot navigation positioning [4; 5], and so on. Simultaneously, the importance of line elements is emphasized in cartography and the even more user-oriented discipline, spatial cognition [6; 7]. The line feature has the following advantages as a more advanced feature than the point feature: (1) It has rich structural information for expressing edge information of 3D objects, such as structuredbuildings; (2) the extraction of the line feature is less affected by image noise, occlusion, geometric distortion, and grayscale distortion; (3) it has higher matching accuracy; (4) it is not necessary to completely determine the position of its two endpoints; and (5) in case of missing texture or uniformity, the line feature is easier to extract than the point feature. Particularly, a line feature is the kind of rich structural information which can intuitively express edge contours of the structured scene. This has achieved good practical application results in scientific and engineering decision-making applications. [PERSON] [8] and [PERSON] et al. [9] used line features to realize map creation and navigation positioning. [PERSON] et al. [10] utilized lines to express environmental features, enabling simultaneous localization and mapping (SLAM) of mobile robots for sensing unknown environments. [PERSON] et al. [11] extracted line segments from the surface of structured buildings to achieve a shape analysis of the target. [PERSON] et al. [12] detected the location of cracks in pipes by judging whether the line feature was continuous. [PERSON] et al. [13] proposed a bundle-adjustment method combined with line segments to achieve relative orientation of stereo-images and the estimated position and pose information of the camera. [PERSON] [14] and [PERSON] [15] used line segments as the constraint information for a camera's relative orientation. [PERSON] et al. [16] combined the line feature extraction algorithm to implement unmanned aerial vehicle (UAV) image-based power line inspections. [PERSON] [17] extracted straight lines from a building feature and used them for the regularized processing of buildings. [PERSON] et al. [18] used reconstructed 3D line segments as auxiliary information to improve the quality of the real orthophoto that reduced the sawtooth pull problem at the edge of a building. Therefore, it is valuable to extract good line features in many fields. Line segment extraction algorithms can be classified into three categories: (1) Algorithms that transform domain-based methods [19]; (2) algorithms based on gradient phase feature [20]; and (3) segmentation edge approaches [21]. The first type of algorithm is less influenced by image noise, but the calculation cost is higher, with lower extraction accuracy. The second type of algorithm has a low memory-occupancy rate with better extraction efficiency in complex scenes, but it is sensitive to grayscale distortion. When the grouping error is large, line segments are prone to the phenomenon of obvious segmentation fracture. The third type of method can detect local features of the image and it is easy to obtain the geometric attribute information of the line segment. Its extraction speed is fast, yet these algorithms depend on the quality of built-in edge tracking algorithms. The theoretical basis of line segment extraction is relatively mature. The Hough transform technique [22] extracts line segments accurately in a single texture background scene, but it is prone to large error-extracted line segments in complex texture scenes. [PERSON] et al. [23] proposed a multi-scale Hough transform with a pre-stored weight matrix to detect straight lines with higher precision and less constraints by studying the limitation of accuracy, efficiency, and image size for detecting line segments of Hough transform.