AsymcNet：a document images-relevant asymmetric geometry correction network

目的 文档图形的几何校正是指通过图像处理的方法对图像采集过程中存在的扭曲、畸变和歪斜等几何干扰进行处理，以提升原始图像的视觉效果与光学字符识别（optical character recognition，OCR）精度。在深度学习普及以前，传统的图像处理方法需要使用激光扫描仪等辅助硬件或在多视角下对文档进行拍摄，且算法的鲁棒性欠佳。深度学习方法构建模型能规避传统算法的不足，但在现阶段这些模型还存在一定的局限性。针对现有算法的缺陷，提出了一种集成文档区域定位与校正的轻量化几何校正网络（asymmetric geometry correction network，AsymcNet），端到端地实现文档图像的几何校正。方法 AsymcNet由用于文档区域定位的分割网络和用于校正网格回归的回归网络构成，两个子网络以级联的形式搭设。由于分割网络的存在，AsymcNet对于各种视野下的文档图像均能取得良好的校正效果。在回归网络部分，通过减小输出回归网格的分辨率来降低AsymcNet在训练及推理时的显存耗用和时长。结果 在自制的测试数据集中与业内最新的4种方法进行了比较，使用AsymcNet可以将原始图像的多尺度结构相似度（multi-scale structural similarity，MS-SSIM）从0.318提升至0.467，局部畸变（local distortion，LD）从33.608降低至11.615，字符错误率（character error rate，CER）从0.570降低至0.273。相比于业内效果较好的DFE-FC（displacement flow estimation with fully convolutional network），AsymcNet的MS-SSIM提升了0.036，LD降低了2.193，CER降低了0.033，且AsymcNet处理单幅图像的平均耗时仅为DFE-FC的8.85%。结论 实验验证了本文所提出AsymcNet的有效性与先进性。;Objective Electronic entry of paper documents is normally based on optical character recognition（OCR）technology. A commonly-used OCR system consists of four sequential steps：image acquisition，image preprocessing，character recognition，and typesetting output. The acquired digital image will have a certain degree of geometric distortion because paper document may not be parallel to the plane where the image acquisition device is located. The lens of the image acquisition device may have its own problem of distortion，or its paper document may challenge for deformation. Image acquisition problems of interferences and distortions will be more severe when handheld image capture devices are used（e. g. ，mobile phone cameras）. Computer vision-oriented highly robust correction algorithms are focused on removing geometric distortions derived from imaging process of paper documents. Currrent researches are concerned about neural networks-based geometric correction of document images. Compared to traditional geometric correction algorithms，neural network-based document image correction algorithms have its potential ability in terms of both hardware requirements and algorithm implementation. However，it is still challenged for optimizing processing performance，especially for the contexts of offline and light weight. To improve the visual effect and OCR recognition accuracy of the original image，geometric correction of document graphics can be used to handle distortion，aberration，skew，and other related image-capturing geometric perturbations. Conventional image processing methods are required for such auxiliary hardware like laser scanners or multiple views-captured documents，and the algorithms can not be robusted. The emerging deep learning methods can be used to optimize traditional algorithms via modeling，but these models still have certain limitations. So，we develop a lightweight geometric correction network（AsymcNet），for which an integrated document region localisation and correction method can be oriented to implement geometric correction of document images end-to-end. Method AsymcNet is designed and dealt with possible geometric interference in image acquisition. It consists of document regions-located segmentation network and a grid regression-rectifying regression network，as well as two sub-networks in a cascade form. Segmentation network-based AsymcNet can achieve good correction results for document images in various fields of view. In the regression part of the network，the resolution of the output regression grid is down to shrink the memory consumption and duration of training and inference. The methodologies are illustrated as follows：1）Segmentation of the network：a simplified Unetbasd skip connection is set up between the encoder and decoder，in which lower layers-derived features can flow into higher layers directly and melt them into small resolution inputs and outputs. Considering the simplicity of the segmentation task， the segmentation network uses a small resolution（128×128 pixels）document image as input and outputs a small resolution segmentation result for the sake of lightweight and possible subsequent localization and mobile porting. 2）Regression network：compared to the segmentation task，the regression task of correcting the grid output is more complex. To capture more details from the image to be corrected for the final corrected grid regression，the regression network can be used to adapt a large resolution（512×512 pixels）document image as input with the segmentation result of the segmentation network output as a dot product，and outputs a small resolution（128×128 pixels）corrected grid. Result AsymcNet-relevant comparative analysis is carried out in relevance to 4 popular methods. The multi-scale structural similarity（MS-SSIM）of raw images can be improved from 0. 318 to 0. 467；The local distortion（LD）is improved from 33. 608 to 11. 615；and the character error rate（CER）is optimized from 0. 570 to 0. 273. Compared to displacement flow estimation with fully convolutional network（DFE-FC），AsymcNet’s MS-SSIM is improved by 0. 036，LD is lower by 2. 193，CER is shrinked by 0. 033，and AsymcNet’s average processing time for a single image is required for 8. 85% of DFE-FC’s only. The experimental results demonstrate that the proposed AsymcNet has certain advantages in comparison with other related correction algorithms. In particular，when the relative area occupied by document regions in the image to be processed is small，the advantage of AsymcNet is more significant due to the integration of sub-networks for document region segmentation in the structure of AsymcNet. Conclusion Our AsymcNet proposed has been validated for its effectiveness and generalization. Compared to existing methods，AsymcNet has its priorities in terms of correction accuracy，computational efficiency，and generalization. Furthermore，the design of AsymcNet is focused on“small resolution grid”as the regression target of the network，which can alleviate the convergence difficulty of the network and the memory consumption during training and inference. The generalizability of the network can be improved further.