Fast bilateral filtering

Electronics LettersVolume 55, Issue 5 p. 258-260 Image and vision processing and display technologyFree Access Fast bilateral filtering Xueli Zhang, Xueli Zhang School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094 People's Republic of ChinaSearch for more papers by this authorLongquan Dai, Corresponding Author Longquan Dai dailongquan@njust.edu.cn School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094 People's Republic of ChinaSearch for more papers by this author Xueli Zhang, Xueli Zhang School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094 People's Republic of ChinaSearch for more papers by this authorLongquan Dai, Corresponding Author Longquan Dai dailongquan@njust.edu.cn School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094 People's Republic of ChinaSearch for more papers by this author First published: 01 March 2019 https://doi.org/10.1049/el.2018.7278Citations: 5AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract The bilateral filter is a fundamental smoothing tool in image processing and computer vision due to its outstanding edge-preserving ability. However, the computational complexity depends on the size of the support of the spatial kernel. This drawback makes bilateral filtering time-consuming and significantly limits its applications. A novel strategy to turn the range kernel of the bilateral filter into a sum of exponential functions is proposed. As the convolution with the exponential function can be accelerated by the box filtering, the computational complexity of the bilateral thus becomes . Experimental results disclose that the proposed algorithm is competitive with the state-of-the-art algorithms in terms of filtering accuracy and computational efficiency. Introduction The bilateral filter is a non-linear weighted averaging filter. Its bilateral weights are determined by two kernels. One is the spatial kernel measuring the spatial distance with respect to the centre pixel. The other is the range kernel defined by the intensity distance with respect to the centre pixel. The range kernel makes the bilateral filter different from other filters as it makes contributions to the edge-preserving ability. So the bilateral filter can alleviate the over and thus becomes a popular filter in various disciplines [1]. In addition, it can be used as spatial convolution layers in convolutional neural network (CNN) architectures, the CNN includes at least one bilateral convolutional layer called as bilateral neural network in [2]. Without loss of generality, we only discuss the standard form of the bilateral filter in this Letter, where the box or Gaussian function is used for spatial filtering, and the Gaussian function is used for range filtering. Let I: be a high-dimensional image, the output of the bilateral filter is given by (1)with (2)where d is the dimension of domain, n is the dimension of the range of the input image I, the domain centre at x, and present the range kernel and the spatial kernel, respectively. The brute-force implementation is time-consuming and thus makes the bilateral filter unusable in real-time applications. To deal with this problem, people proposed several schemes in the literature: (i) Intermediate filtered images based methods: The main idea of these kinds of methods is that the intermediate filtered images, sampled copies, are computed first, then do interpolate to form output at every pixel. Paris and Durand [3] figured out the intermediate images in 3D space, which is produced by adding the image intensity to the spatial dimension, and was called 'bilateral grid' by Chen et al. in [4]. (ii) Shiftable kernels based methods: The first kind of methods speed up the bilateral filter at the cost of low filtering accuracy due to interpolation. Instead, people take a different approach by employing shiftability function to transform the bilateral filter into computing the moving sum of a stack of images. Specifically, Porikli [5] exploited the polynomial kernel to approximate the Gaussian range kernel and Chaudhury [6] took the trigonometric to obtain an approximation. In addition, more examples can be found in [7, 8]. (iii) Others: The approaches based on the shiftable approximation have been restricted to Gaussian range kernels. Adams [9] proposed an efficient Gaussian kd-tree representation to eliminate the restriction. Later, Adams et al. [10] presented an alternative high-dimensional representation – permutohedral lattice representation to solve the dimensionality dependence by dividing position into simplices [9]. However, these methods are hard to understand and implement. Proposed method Inspired by works [11, 12], we present a novel fast algorithm to accelerate (1). To achieve higher accuracy, we factorise (2) into three terms and take mean square error to find the optimal approximation of one term of the three. Details of our method are listed in the following sections. (i) Factorisation: We factorise the range kernel into the following equation: (3)where and . We note that (3) is a function of for a fixed , and the first term is a scaling factor, whereas the second term is a Gaussian function, which contributes to the bell shape of (3). The third term is a monotonic exponential, which rides the bilateral filter non-linear and time-consuming. The target of our algorithm is to find the approximation of the last term. To achieve the approximation, we convert the last term into the following equation: (4)On the basis of the principle of parity, if is an even function, and in (4) are equal. That is (5)Then, we translate (1) into the following expansion: (6)with (7) (ii) Optimisation: We linearise the bilateral filter by turning (1) into (6) which is the linear combination of linear convolutions. As there is an unknown function in (6), we employ mean square error to find its optimal approximation. The objective function is defined as (8)Putting as the histogram of the input image , we can express (8) in the form of integral (9)Applying variational techniques to (9), we get the following equation: (10)Furthermore, we translate (10) into the basic calculus (11)To determine the minimal points, (11) equals to zero for arbitrary functions . Combining orthonormality condition in [12], we obtain the following equations: (12)where is the Kronecker delta. Let function be a cumulative histogram, (12) can be further simplified as (13)with (14) Employing (13) and (14), we can find the optimal approximation of by solving singular value decomposition instead of directly minimising (8). Above all, our algorithm is summarised as follows: (1) Computing histogram and cumulative histogram of the input image . (2) Sampling on a regular grid to get the matrix . (3) Calculating eigenvectors and eigenvalues of matrix . The eigenvalues correspond to the coefficients and the eigenvectors provide the values of the functions on sampled points. Other missing values can be obtained by interpolating. (4) Putting and into (6) to get the output of fast bilateral filter. Experiments In this section, we compare the filtering accuracy and computational efficiency of the proposed fast bilateral filter algorithm with the brute-force implementation for bilateral filter [13] and the state-of-the-art fast bilateral filters [4, 7, 9]. The filtering accuracy is measured by peak signal-to-noise ratio (PSNR) and structural similarity image measurement (SSIM), the computational efficiency is scaled by the running time when keeping parameters constant. For fair, all methods are run on MATLAB R2016a, and the experimental environment mounts on a notebook with an Intel i7 central processing unit and 8GB memory. Keeping parameters unchanged, we illustrate the outputs of the brute-force implementation, the state-of-the-art fast bilateral filters and the proposed method in Fig. 1. Note that our result is more akin to the brute-force implementation of the bilateral filter than other methods visually, especially the grass on the pictures. Fig 1Open in figure viewerPowerPoint Illustration of filtering results with parameters and . From left to right: input image; brute-force implementation of bilateral filter; fast bilateral filter proposed in Paris [3]; output of fast bilateral filter in Paris Chaudhury [6]; our result To demonstrate the efficiency advantage of our method, we keep the parameters and consistent and measure the running time on a series of test images and calculate their average to increase credibility. The test images come from the database [14]. Table 1 shows that the proposed method is distinguishable from other algorithms. For , our work outperforms other algorithms. Besides, we do an experiment to measure the PSNR and SSIM for filtering accuracy on a series of test images mentioned above. Similarly, we average the experimental results, and plot them in Fig. 2, which discloses that our method ranks one in terms of filtering accuracy. Table 1. Quantitative running time comparison, where and running time is measuredby second 40 20 10 5 3 Paris and Durand [3] 0.6392 0.5507 0.5488 0.5449 0.5490 Chaudhary and Dabhade [11] 1.5015 0.7780 0.4711 0.3597 0.3183 Papari et al. [12] 0.7455 0.3473 0.1952 0.1434 0.1244 Chaudhary [6] 1.1331 0.5561 0.3582 0.2613 0.2409 Ours 0.6250 0.3690 0.1694 0.1038 0.0934 Fig 2Open in figure viewerPowerPoint Comparison of filtering accuracy (PSNR and SSIM) of five different algorithms as function of parameter . Test images are the standard image, which is of size 512*512, from image database mentioned before. Parameters of both line charts are same with We note that the running time of the proposed method in Table 1 is slightly higher than [9] for with default acceleration configuration. This is because our method provides accurate filtering results. This can be verified in Fig. 2. If we decrease filtering accuracy, the running efficiency can be significantly boosted according to our experiments. Conclusion Our main contribution is to present an efficient algorithm to speed up the bilateral filter. Our algorithm no longer employs the shiftable kernels, which are usually used by the acceleration of bilateral filtering, and therefore results in better result than other algorithms. Experiments validate that our method gets state-of-the-art results in terms of accuracy and efficiency. Acknowledgments This work was supported by the National Natural Science Foundation of China Fund (Grant no. 61701235) and the Fundamental Research Funds (Grant no. 30917011323). References 1Das, A., Nair, P., Shylaja, S.S. et. al.,: ' A concise review of fast bilateral filtering'. Int. Conf. Image Information Processing, Shimla, India, December 2017, pp. 1– 6 2Jampani, V., Kiefel, M., Gehler, P.V.: ' Learning sparse high dimensional filters: image filtering, dense CRFs and bilateral neural networks'. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, June 2016, pp. 4452– 4461 3Paris, S., Durand, F.: 'A fast approximation of the bilateral filter using a signal processing approach', Int. J. Comput. Vis., 2009, 81, (1), pp. 24– 52 (https://doi/org/10.1007/s11263-007-0110-8) 4Chen, J., Paris, S., Durand, F.: 'Real-time edge-aware image processing with the bilateral grid', ACM Trans. Graph., 2007, 26, (3), p. 103 (https://doi/org/10.1145/1276377.1276506) 5Porikli, F.: ' Constant time O(1) bilateral filtering'. IEEE Conf. Computer Vision and Pattern Recognition, Anchorage, AK, USA, June 2008, pp. 1– 8 6Chaudhury, K.N.: 'Acceleration of the shiftable O(1), algorithm for bilateral filtering and nonlocal means', Trans. Image Process., 2013, 22, (4), pp. 1291– 1300 (https://doi/org/10.1109/TIP.2012.2222903) 7Ghosh, S., Chaudhury, K.N.: ' Fast and high-quality bilateral filtering using Gauss–Chebyshev approximation'. Int. Conf. Signal Processing and Communications., Bangalore, India, June 2016, pp. 1– 5 8Ghosh, S., Chaudhury, K.N.: 'On fast bilateral filtering using Fourier kernels', Signal Process. Lett., 2016, 23, (5), pp. 570– 573 (https://doi/org/10.1109/LSP.2016.2539982) 9Adams, A.: 'Gaussian KD-trees for fast high-dimensional filtering', ACM Trans. Graph., 2009, 28, (3), pp. 1– 12 (https://doi/org/10.1145/1531326.1531327) 10Adams, A, Baek, J., Davis, M.A.: 'Fast high-dimensional filtering using the permutohedral lattice', Comput. Graph. Forum, 2010, 29, (2), pp. 753– 762 (https://doi/org/10.1111/j.1467-8659.2009.01645.x) 11Chaudhury, K.N, Dabhade, S.D.: 'Fast and provably accurate bilateral filtering', Trans. Image Process., 2016, 25, (6), pp. 2519– 2528 (https://doi/org/10.1109/TIP.2016.2548363) 12Papari, G., Idowu, N., Varslot, T.: 'Fast bilateral filtering for denoising large 3D images', Trans. Image Process., 2017, 26, (1), pp. 251– 261 (https://doi/org/10.1109/TIP.2016.2624148) 13Tomasi, C., Manduchi, R.: ' Bilateral filtering for gray and color images'. 6th Int. Conf. Computer, Bombay, India, January 1998, pp. 839– 846 14 'ImageVision databases' http://www.imageprocessingplace.com/root_files_V3/image_databases.html, accessed May 2019 Citing Literature Volume55, Issue5March 2019Pages 258-260 FiguresReferencesRelatedInformation