Quick response barcode deblurring via <i>L</i> <sub>0</sub> regularisation based sparse optimisation

IET Image ProcessingVolume 13, Issue 8 p. 1254-1258 Research ArticleFree Access Quick response barcode deblurring via L0 regularisation based sparse optimisation Yan Zhang, Yan Zhang Department of Electrical Engineering and Information Technology, Shandong University of Science and Technology, Jinan, 250031 People's Republic of ChinaSearch for more papers by this authorHaitao Pu, Haitao Pu Department of Electrical Engineering and Information Technology, Shandong University of Science and Technology, Jinan, 250031 People's Republic of ChinaSearch for more papers by this authorJian Lian, Corresponding Author Jian Lian lianjianlianjian@163.com orcid.org/0000-0003-0305-8454 Department of Electrical Engineering and Information Technology, Shandong University of Science and Technology, Jinan, 250031 People's Republic of ChinaSearch for more papers by this author Yan Zhang, Yan Zhang Department of Electrical Engineering and Information Technology, Shandong University of Science and Technology, Jinan, 250031 People's Republic of ChinaSearch for more papers by this authorHaitao Pu, Haitao Pu Department of Electrical Engineering and Information Technology, Shandong University of Science and Technology, Jinan, 250031 People's Republic of ChinaSearch for more papers by this authorJian Lian, Corresponding Author Jian Lian lianjianlianjian@163.com orcid.org/0000-0003-0305-8454 Department of Electrical Engineering and Information Technology, Shandong University of Science and Technology, Jinan, 250031 People's Republic of ChinaSearch for more papers by this author First published: 10 May 2019 https://doi.org/10.1049/iet-ipr.2018.5738Citations: 1AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract In recent years, the two-dimensional barcode techniques have been widely applied in a large number of fields due to the requirement for the development of information acquisition technology. Also, the identification and verification of the static barcodes could be implemented flawlessly. However, the recognition of the motion blurred two-dimensional barcode images still remains a challenge to most of the current machine vision systems. To address the above-mentioned problem, the authors propose a regularisation-based approach to reverse the blurry barcode images while taking the number of black and white blocks in the two-dimensional barcode image as the prior. To evaluate the performance of the proposed approach, they conducted comparison experiments between state-of-the-art image deblurring methods and ours. Experimental results on synthetically blurry barcode images show that the proposed method outperforms the state-of-the-art image restoration techniques. 1 Introduction As one of the most popular two-dimensional (2D) barcodes, QR barcode [1], which was invented by the Toyota subsidiary Denso Wave in 1994 and at the very beginning it was exploited to monitor the vehicles in the automobile production line, plays a significant role in various fashionable applications, including e-payment [2], express delivery validation [3], train ticket [4], and the cell phone-based applications [5, 6] due to its capacity, accessibility, and stability (as shown Fig. 1). Fig. 1Open in figure viewerPowerPoint QR code Currently, the readability of static QR code has been solved almost thoroughly since the rapid development of the image processing algorithms and the high performance image acquisition hardware. However, the identification and verification of the blurry QR code images still remain a challenge, while the blurry effects might be caused by different types of degradations, including out-of-focus blur [7] or motion blur [8]. For instance, the blurry artefacts from high-speed mutual motion can defeat most of the image deblurring algorithms. To reverse the blurry QR barcode images, plenty of approaches have been proposed previously. For example, Xu et al. [9] presented a machine vision system for localising and deblurring the motion-blurred 2D barcodes. It is based on a coded exposure imaging equipment that could accommodate to a wide range of velocities, which both the quality of restoration and the accuracy of the metric for evaluating the quality of the deblurred 2D barcodes depend on. In [10], Liu et al. proposed a pipeline for 2D barcode reversion that is consisting of three sub-procedures, firstly the Fourier transformation is used to figure out the type of blurry effect; then the deblurring function is obtained; finally, the barcode image is restored with iterative computation. A great deal of regularisation models has been proposed to implement the image deblurring tasks in the literature. Aria et al. [11] proposed a deblurring algorithm that is based on a conjugate gradient technique and used the previously developed weighted -norm total variation (TV) regulariser to obtain the reasonable solution. In [12], Chen et al. presented a second-order approach for blind motion deblurring. Its main idea is to define an energy functional that is solved by the normalised norm regularisation term for the second-order gradient image. Very recently, the -norm has shown its effectiveness for image deblurring [13, 14]. However, none of the above-mentioned regularisation term-based TV methods took the unique characteristics of the QR barcode into account. Meanwhile, the deep learning-based approaches have shown their great performance in various applications, including image classification, object tracking, image segmentation, and image deblurring. However, as the most typical data-driven techniques, these algorithms significantly rely on the quality and large quantity of the image samples, which might not be applicable to the practical scenarios. Since each QR code image is unique as well as its blurry counterpart. Therefore, it is difficult to obtain enough training samples without data augmentation skills. Bearing the above-mentioned analysis in mind, we propose an efficient image deblurring approach to reverse the blurred QR barcodes. It is implemented by using a regularisation term and the adapting to the characteristics of the QR barcode, especially the number of black and white squares. Generally, the intensity, the gradient of the QR barcode images, and the blur kernels are taken as regularisation terms into the objective function corresponding to the image deblurring process. Also, the alternating minimisation is exploited to produce an accurate outcome from the objective function, iteratively. To testify the performance of the proposed method, we conducted comparison experiments between state-of-the-art image deblurring algorithms and ours. The experimental results on synthetically blurred QR barcode images show that the proposed approach outperforms state-of-the-art methods both in the accuracy and the visual effects. Our work offers at least five significant contributions as listed below: To our best knowledge, this is the first attempt to introduce regularisation term into the image deblurring of QR barcode. By introducing the characteristics of QR barcode into the framework of image deblurring, the proposed method could significantly enhance the image deblurring performance. Our approach performs with an impressive superiority compared with the state-of-the-art deblurring methods. The rest of this paper is organised as follows. In Section 2, we present our proposed QR barcode image deblurring approach. Section 3 contains experimental results and its corresponding analysis. In Section 4, we introduce the conclusion. 2 Our method Image restoration is one typical inverse problem, which is used to produce the latent image O from its blurry counterpart I. Mathematically, it can be formulated as follows: (1)where I denotes the blurred image (or specifically the QR barcode image), O stands for the corresponding latent image, is the convolution operator, K denotes the blur kernel, and N denotes the noise. Although the noise N is commonly neglected in the computation, it is still an ill-pose problem as there are two unknown variables in the formulation (2). Therefore, the prior-enforcing regularisation terms need to be incorporated to stabilise the recovery of latent image O. Therefore, the process of image recovery could be expressed as the following optimisation equation: (2)where L denotes the data fidelity or difference between and I, is the regularisation term on O, K, and the gradient of O etc. Various regularisation terms have been exploited, such as and norms. However, none of them is suitable for the characteristics of the QR barcode images. Therefore, we propose a -norm by constraining the number of blocks or the image gradient in the QR code images. Also, the details about the proposed method are described as follows. 2.1 regularised intensity and gradient prior The proposed regularisation term of intensity and gradient came from our observation about the pattern of QR barcode, including the uniform intensity of the black and white squares and the sharp edges in the latent images without blurry effects. Due to only black and white squares in QR barcode image, the intensity of the pixels in each QR barcode image is sparse. Inversely, the intensity of the blurry QR barcode image is denser than the corresponding latent image. Therefore, we use the intensity of QR barcode image as a regularisation term shown as (3)where x is the image and relates to the number of white blocks within this QR barcode image. Meanwhile, there are more non-zero values of gradients in blurry QR barcode image than latent QR barcode image, which can be used to differentiate the blurry image from the clear image. Thus, we present the gradient prior , and our proposed prior is shown as (4)where denotes the trade-off parameter used to balance the weight between and . 2.2 Image deblurring via regularised prior Based on the proposed regularisation priors in Section 2.1, the objective function of our method is shown as (5)where x and y denote the latent QR barcode image and blurry image, respectively, * denotes the convolution operator, k is the blur kernel, and and are the weighting parameters. The latent image corresponding to the blurry QR barcode image can be acquired by the following two sub-functions: (6)and (7)Then the two above functions can be solved iteratively with alternative minimisation. 2.2.1 Estimating x with k Based on the half-quadratic splitting minimisation method [15], we present a novel method to solve the two sub-problems as follows. Firstly, we introduce two auxiliary variables u and corresponding to the intensity of the QR barcode image and the gradient of the QR barcode image, the objective function can be rewritten as (8)where is the same parameter as (4). within this formulation can be computed through alternative minimisation by fixing two variables, iteratively. The initial values of u and g are zeros. The solution to x can be acquired by (9)We introduced a fast Fourier transform (FFT) to implement the time-frequency domain transform since the convolution operator is difficult to calculate in the time domain. Also, the closed-form solution for this least squares minimisation problem is (10)where and denote the FFT operator and inverse FFT, respectively, is the complex conjugate operator, and where and denote the horizontal and vertical differential operators, respectively. 2.2.2 Estimating k with x With the given x, kernel k can be calculated with the following function: (11)where the solution can also be computed by the alternative minimisation method. Due to the duality of the latent image and the blur kernel in convolution, the closed-form of blur kernel estimation in the frequency domain could be expressed as (12)Also, the proposed algorithm is shown in Fig. 2. Fig. 2Open in figure viewerPowerPoint Algorithm 1: Deblurring for QR barcode 3 Experimental results 3.1 Synthetically blurry effects In order to evaluate the performance of the proposed regularisation-based approach for recovering the blurry QR barcode images, we captured a large amount of QR barcode images from a digital device (Canon EOS 760D, Japan). To simulate the blurry effects from the practical scenarios, we synthetically added two different types of blur kernels on the latent QR barcode images, i.e. the out-of-focus blur kernels and the motion blur kernels. 3.1.1 Out-of focus blur kernels In this study, we choose several blur kernels to simulate the blurring effect that might occur in the practical QR barcode images. Firstly, the out-of-focus kernels are constructed by the following equation: (13)where , K denotes the out-of-focus kernel, denotes the radius of confusion circle [16] or blur radius, is the coordinate of each pixel in a QR barcode image. Totally, 20 out-of-focus blur kernels were generated as shown in Fig. 3a. Fig. 3Open in figure viewerPowerPoint The synthetic blur kernels (a) Synthetic out-of-focus blur kernels, (b) Synthetic motion blur kernels 3.1.2 Motion blur kernels To simulate the motion blurry effects, 91 different motion blur kernels were generated, which have 18 motion directions from to with interval 10 (i.e. , , , …, ), and five-motion lengths from 1 pixel to 10 pixels with an interval of 2. Mathematically, these motion blur kernels could be expressed as (14)where L denotes the variant motion lengths, is the coordinate of each pixel in a QR barcode image, and indicates each motion direction. Also, the motion blur kernels are shown in Fig. 3b. 3.2 Experimental results and discussion To demonstrate the performance of the proposed method, we conducted the comparison experiments between the state-of-the-art image deblurring techniques and ours, including the sparse prior deconvolution [17] and text deblurring algorithm [13] on synthetically blurred QR barcode images with the two types of blur kernels, respectively. Both the peak signal to noise ratio (PSNR) and mean squared error were used as the measurement metrics to demonstrate the effectiveness of the competing approaches. (15)and (16)where denotes the blur kernel space, represents the predictable blur kernel, is the ground truth of blur kernel, and denotes the maximum of all of the feasible blur kernel. The PSNR plots for recovering the synthetically blurry QR barcode image, including the out-of-focus blurred and motion blurred, are shown in Figs. 4a and b, respectively. They both demonstrate that the proposed approach outperforms state-of-the-art methods. Due to the employment of the sparse prior, the approach [13] achieves a better result than the other deblurring technique [17]. The average PSNR results are listed in Table 1, which also demonstrates the decent performance of the proposed approach. Table 1. Comparison of PSNRs with the competing methods PSNR Method 1 [17], dB Method 2 [13], dB Our method, dB out-of-focus 26.45 27.58 28.32 motion 27.61 30.92 32.29 Fig. 4Open in figure viewerPowerPoint PSNR of (a) Out-of-focus blur kernels, (b) Motion blur kernels The results showed that the proposed method is validated and has superior performance, and it can obtain much better deblurring outcome by increasing the iterative cycles of the proposed algorithm. We believe the experimental result should be ascribed to the appropriateness of the proposed regularisation term to the characteristics of the QR barcode images. Meanwhile, the model-based deconvolution methods might not be suitable for the QR barcode. For instance, the sparse prior deconvolution [17] assumes that the intensity of the image should satisfy the Gaussian distribution distribution, while it was designed for the natural image. Meanwhile, the technique presented in [13] is also built upon the introduction of regularisation term, while it differs from our approach mainly locating at the assumption of the number of non-zero values in text images and barcode images. Furthermore, different from baselines, the proposed approach takes the smoothness of the latent QR barcode image into consideration and it can obtain much better deblurring outcome by increasing the iterative cycles of the proposed algorithm. Meanwhile, we present the visual comparison between the state-of-the-art methods and the proposed approach on the out-of-focus and motion blurred QR barcode images (as shown in Figs. 5 and 6), respectively. It can be observed that our method outperforms state-of-the-art techniques with sharp edges and free from ringing artefacts. Fig. 5Open in figure viewerPowerPoint Motion blurred image and the corresponding deblurred image by state-of-the-art methods and our method (a) Blurred image (motion blurred, length = 21, angle = 0), (b) Deblurred image by Krishnan and Fergus [18], (c) Deblurred image by Krishnan et al. [19]), (d) Deblurring image by our method Fig. 6Open in figure viewerPowerPoint Out-of-focus blurred image and the corresponding deblurred image by state-of-the-art methods and our method (a) Blurred image (out-of-focus blurred, radius = 9), (b) Deblurred image by Krishnan and Fergus [18], (c) Deblurred image by Krishnan et al. [19]), (d) Deblurring image by our method Finally, to evaluate the readability of the proposed method for the practical applications, we conducted the comparing experiments between state-of-the-art methods [18, 19] and our method with Zbar [20] (as shown in Table 2). The hit rate of the proposed approach is much better than the state-of-the-art techniques. Table 2. Comparison of readability with the competing methods Readability Method 3 [18], % Method 4 [19], % Our method, % out-of-focus 67.22 75.18 88.83 motion 71.61 76.58 90.19 Furthermore, we changed the values of , , in (4) and (5). According to the results shown in Fig. 7, the proposed approach performs best when , , and , respectively. However, since there are many parameters in the proposed algorithm, it is difficult to find the best combination of the parameters. Fig. 7Open in figure viewerPowerPoint Readability of our algorithm with different values of the parameters 4 Conclusion and future work To recover the blurry effects in the QR barcode images, we propose an regularisation term-based image deblurring approach. Both the intensity of the QR barcode and the gradient of the QR barcode image are introduced as regularisation terms in the final objective function. To optimise the outcome of the objective function, the alternating minimisation strategy is exploited. The experimental results show that the proposed method outperforms the state-of-the-art image deblurring methods for QR barcode images. In the future, we would continue to research on the other appropriate sparseness priors as the regularisation terms in the proposed final objective function. Meanwhile, to extend the ability of the proposed method, the characteristics of different types of barcodes and different sources of blurry effects would also be taken into consideration. 5 Acknowledgment We would like to appreciate the reviewers and editors for their valuable comments. 6 References 1Ouaviani E., Pavan A., and Bottazzi M. et al: ‘ A common image processing framework for 2D barcode reading’, IEE 7th International Conference on Image Processing and its Applications, Manchester, UK, 1999, vol. 2, pp. 652– 655 2Ma T., Zhang H., and Qian J. et al: ‘ The design and implementation of an innovative mobile payment system based on QR barcode’. Int. Conf. on Network and Information Systems for Computers, Wuhan, Hubei, China, 2015, pp. 435– 440 3Liu Z., Zeng Q., and Wang C. et al: ‘Application research of QR code barcode in validation of express delivery’, Commun. Comput. Inf. Sci., 2011, 143, pp. 346– 351 4Yu X.: ‘Application of 2D QR code barcode in train tickets’, J. Wenzhou Vocational Tech. Coll., 2011, 11, pp. 53– 55 5Liu Y., Yang J., and Liu M.: ‘ Recognition of QR code with mobile phones’. Control and Decision Conf., 2008. CCDC, Yantai, Shandong, China, 2008, pp. 203– 206 6Walsh A.: ‘QR codes: using mobile phones to deliver library instruction and help at the point of need’, J. Inf. Literacy, 2010, 4, (1), pp. 55– 65 7Liu N., Zheng X., and Sun H. et al: ‘Two-dimensional barcode out-of-focus deblurring via the increment constrained least squares filter’, Pattern Recognit. Lett., 2013, 34, (2), pp. 124– 130 8Chu C.H., Yang D.N., and Pan Y.L. et al: ‘Stabilization and extraction of 2D barcodes for camera phones’, Multimedia Syst., 2011, 17, (2), pp. 113– 133 9Xu W., and McCloskey S.: ‘ 2D barcode localization and motion deblurring using a flutter shutter camera’. Applications of Computer Vision, Kona, Hawaii, USA, 2011, pp. 159– 165 10Liu Z., Chen X., and Wan Y.: ‘Two-dimensional barcode image deblurring algorithm’, Comput. Digit. Eng., 2014, 42, pp. 1038– 1041 11Abubakar A., and Berg P.M.V.D.: ‘ A multiplicative weighted L2-norm total variation regularization for deblurring algorithms’. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Prague, Czech Republic, 2011, pp. IV-3545– IV-3548 12Chen Z., Fang F., and Xu Y. et al: ‘ A second-order approach for blind motion deblurring by normalized L1 regularization’. Pacific Rim Conf. on Multimedia, Xi'an, China, 2016, pp. 296– 305 13Pan J., Hu Z., and Su Z. et al: ‘ Deblurring text images via L0-regularized intensity and gradient prior’. IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, Ohio, USA, 2014, pp. 2901– 2908 14Xu L., Zheng S., and Jia J.: ‘ Unnatural L0 sparse representation for natural image deblurring’. IEEE Conf. on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013, pp. 1107– 1114 15Xu L., Lu C., and Xu Y. et al: ‘ Image smoothing via l 0 gradient minimization’. Int. Conf. on Computer Graphics and Interactive Techniques, Vancouver, BC, Canada, 2011, vol. 30, no. 6, p. 174 16Battiato S.: ‘ Restoration of out-of-focus images based on circle of confusion estimate’, Proc. SPIE, 2002, 4790, (2), pp. 408– 416 17Krishnan D., Tay T., and Fergus R.: ‘ Blind deconvolution using a normalized sparsity measure’. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 2011, pp. 233– 240 18Krishnan D., and Fergus R.: ‘ Fast image deconvolution using hyper-Laplacian priors’. NIPS. Advances in Neural Information Processing Systems 22: Conf. on Neural Information Processing Systems 2009. Proc. of A Meeting Held 7–10 December 2009, Vancouver, British Columbia, Canada, 2009, pp. 1033– 1041 19Krishnan D., Bruna J., and Fergus R. et al: ‘ Blind deconvolution with non-local sparsity reweighting’. Computer Vision and Pattern Recognition, Portland, OR, USA, 2013 20Zbar barcode reader. Available at http://zbar.sourceforge.net Citing Literature Volume13, Issue8June 2019Pages 1254-1258 FiguresReferencesRelatedInformation