As‐global‐as‐possible stereo matching with adaptive smoothness prior

IET Image ProcessingVolume 13, Issue 1 p. 98-107 Research ArticleFree Access As-global-as-possible stereo matching with adaptive smoothness prior Peng Yao, Peng Yao Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of ChinaSearch for more papers by this authorHua Zhang, Corresponding Author Hua Zhang hztjut@163.com Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Sino-German University of Applied Science, Tianjin, 300350 People's Republic of ChinaSearch for more papers by this authorYanbing Xue, Yanbing Xue Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of ChinaSearch for more papers by this authorShengyong Chen, Shengyong Chen Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of ChinaSearch for more papers by this author Peng Yao, Peng Yao Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of ChinaSearch for more papers by this authorHua Zhang, Corresponding Author Hua Zhang hztjut@163.com Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Sino-German University of Applied Science, Tianjin, 300350 People's Republic of ChinaSearch for more papers by this authorYanbing Xue, Yanbing Xue Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of ChinaSearch for more papers by this authorShengyong Chen, Shengyong Chen Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, 300384 People's Republic of China Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384 People's Republic of ChinaSearch for more papers by this author First published: 01 January 2019 https://doi.org/10.1049/iet-ipr.2018.5801Citations: 5AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract More global matching (MGM) overcomes the limitation of one-dimensional scanline optimisation in semi-global matching (SGM). Nevertheless, the possible weaknesses of the MGM algorithm are as follows: (i) only two directions are considered for each image traversal direction, which may lead to massive mismatches; (ii) disparity estimation around the object boundaries usually performs terrible since the smoothness term is designed independent of the image prior. In this research, the authors consider all of the four directions for each image traversal direction through a novel model. Besides utilising the prior of neighboured pixels' correlation, adaptive smoothness terms are modelled and augmented into the energy function. These contributions encourage ‘as-global-as-possible (AGAP)’. More importantly, different from the recent works in which the aggregated algorithms have been conducted as the data term of an energy function, conversely, the authors make the energy function as a part of cost aggregation framework. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed AGAP outperforms other four most challenging stereo matching algorithms, and also performs better on Microsoft i2i stereo videos. In addition, under various strategies of parallelisation, the presented AGAP shows a near real-time execution time. 1 Introduction Accurate and fast dense stereo matching is an important requirement for a lot of computer vision applications, such as 3D reconstruction, image refocusing, view interpolation etc. Generally speaking, stereo matching is the task of how to establish dense correspondence between two images at the same scene. Conventionally, according to the analyses in [1-3], stereo matching algorithms are divided into two categories: aggregated and global algorithms. With regard to both of them, four steps are usually required: matching cost computation, cost aggregation, disparity computation and disparity refinement (post-processing). Local window based stereo matching algorithms, such as adaptive supported weight [4], geodesic stereo [5], guided filter [6-8], histogram aggregation [9, 10], accelerating cost filter [11] and supported windows for slanted surfaces [12] are seemed to be an important part of aggregated ones. The fundamental behind them is the local disparity consistency assumption [13] that demonstrates that all of the pixels within a local supported window have constant disparities. To be a typical case of the rest part of aggregated ones, the non-local cost aggregation (NLCA) algorithm [14, 15] performs much better than the traditional local ones. In this research, cost volumes are aggregated on a minimum spanning tree. Soon afterwards, segmentation cue has been successfully integrated into the NLCA framework [16]. More recently, a novel one that aggregates cost volumes on a horizontal tree structure, named as weighted cost propagation with smoothness prior (WCPSP) [17], is also proposed. Based on these, other NLCA algorithms [18-22] have recently risen. Both the categories of aggregated algorithms enforce matching cost computation at first and then implement cost aggregation with a local or non-local framework. After that, a winner-takes-all (WTA) strategy is employed for disparity computation, which selects the corresponding disparity value of the minimum aggregated cost for each pixel. This strategy has been reported to provide fine and fast disparity estimation. Global ones, or called as energy minimisation algorithms, always omit the step of cost aggregation by directly formulating an explicit smoothness term and minimise a predefined energy function. Some of them produce much greater results that of than aggregated ones but may face the bottleneck of computational complexity. In fact, minimising an energy function is a non-deterministic polynomial hard problem, so former researchers have focused on various approximate optimisation approaches. The two most celebrated optimisers are belief propagation (BP) and graph cut (GC). In the light of above optimisers, some significant progresses have been made [23-28]. For all of the above-mentioned stereo matching algorithms, some disparity refinement strategies, such as left–right cross-check (LRC), occlusion filling (OF) and weighted median filter (WMF) are widely employed for attaining a final refined disparity map. Lately, some research works integrating aggregated algorithms into the energy function model have sprung up. The two-step global optimisation (TSGO) [29] algorithm integrates the NLCA algorithm into an energy function model. Then, Yao et al. improve TSGO with a more elegant formulation [30]. MeshStereo (MS) model [31] could produce a high-quality disparity map in the requirement of View Interpolation. It innovatively conducts the PatchMatch (PM) stereo [32] as the data term of an energy function. It is noteworthy that the computational complexity of the MS model is independent of the image pixels due to the normal-depth representation for each simple linear iterative cluster [33] segment's barycenter. Inspired by MS and cross-scale cost filtering 21, 22], a novel fused model, named as MeshStereo with cross-scale [34] is put forward later. The above-mentioned algorithms draw a closer connection between aggregated and global algorithms, and also provide more pleasant results. Unfortunately, they remain holding too much computational load. Deep learning (DL) is an accurate method for solving dense stereo correspondence problem. Actually, such DL-based approaches [35, 36] are used for computing matching cost volumes. Based on massive training data of Ground Truth (GT), they could provide more accurate volumes than classical methods. However, DL-based approaches might be invalid for any unknown environment and cannot be well transplanted to Robotics and Embedded Systems. Most recently, a synthesised algorithm called sampled photo-consistency stereo (SPS) [37] appeared. The authors claim that SPS implements fast without any training data and shows approximate accuracy compared to DL-based approaches. That is to say, SPS still not performs well. MGM [38] enlightens us an excellent way for fast solving dense stereo matching within a ‘more global’ opinion, which effectively overcomes the limitation of one-dimensional scanline optimisation in SGM [39, 40]. However, the performance of MGM possibly performs weakly due to the following reasons: (i) only two directions (current pixel's previous one's direction and its directly above direction, as shown in Fig. 1a) are considered for each image traversal direction, which may lead to massive mismatches; (ii) disparity estimation around the object boundaries usually performs terrible since the smoothness term is designed independent of the image's prior. In this paper, a novel model is proposed that considers all of the four directions (as shown in Fig. 1b, details in Section 3) for each image traversal direction, i.e. an ‘AGAP’ opinion. Then, we also take the neighboured pixels’ correlation into account. This correlation is seemed as adaptive smoothness prior and has been augmented into the initial energy function. In summary, the proposed AGAP constructs a form of energy function with adaptive smoothness prior at first, and in fact, it shows the matching costs that are derived from all of the four directions for each image traversal. Then, the matching costs of all of the eight image traversals are aggregated together. After that, using WTA and various disparity refinement strategies, a final disparity map can be attained. The experimental results show that our AGAP could provide more accurate disparity estimation and also speeds up 35–45 times than MGM. Fig. 1Open in figure viewerPowerPoint Recursive updating procedures of the current (red) pixel for all of the eight image traversals on (a) MGM and proposed (b) AGAP. Also, the initial disparity maps of Cones that are provided by two stereo matching algorithms, respectively On account of the contents we have mentioned above, some previously related research works are first reviewed in Section 2. Then, our proposed AGAP stereo matching algorithm is discussed in Section 3. After that, the implementation details with parallelisation considerations are described in Section 4. Performance evaluations are shown in Section 5. Finally, in Section 6, we draw a conclusion and plan the future work. 2 Related work Different from BP and GC, dynamic programming (DP) [41] perfectly divides the multi-step decision process into sequential single-step ones. While applying in stereo matching, this category of algorithms [42-44] innovatively transforms the two-dimensional optimisation problem into a series of one-dimensional ones. The same idea is also employed into the SGM algorithm. It constructs an energy function at first. Next, the energy function is conducted as one-dimensional matching cost computation on each scanline (usually 8 or 16 cardinal directions' scanlines centred at one pixel are adopted). Then, all of the one-dimensional matching cost volumes are aggregated together and selected the disparities that correspond to the minimum matching costs for all of the pixels with WTA. Theoretically, SGM demonstrates a rudiment of how to integrate energy function into a cost aggregation framework. However, the limitations are obvious. The same as other DP-based stereo matching algorithms, SGM also shows terrible streaking effects because of the one-dimensional scanline optimisation, and it is sensitive from peaks due to textureless regions, reflection, noise and so forth. As compensation, SGM utilises Mutual Information (MI) [45] for matching cost computation and various disparity refinement strategies. In fact, MI is a supervised learning approach, which means the referred disparity map heavily determines the final quality of stereo matching. In addition, disparity refinement could cover the true performance for a certain algorithm because it effectively handles mismatches, occlusions, streaks, peaks and so on. Before MGM, the first attempt to extend the one-dimensional DP approach to two-dimensional energy was done in [46], and lately improved in [47]. Based on the connection between SGM and mini-sum BP, which has been analysed in [48], the recently proposed MGM also overcomes the limitation of one-dimensional scanline optimisation in SGM through a two-dimensional opinion. As shown in Fig. 1a, MGM incorporates the messages (beliefs) from the pixels visited in the previous scanlines with two directions (current pixel's previous one's direction and its directly above direction) for each image traversal direction, which performs ‘more global’ than SGM. However, MGM still performs weakly because of the following reasons: (i) only two directions are considered for each image traversal direction, which inevitably leads to massive mismatches; (ii) the same as in SGM, disparity estimation around the object boundaries usually performs terrible since the smoothness term is modelled independent of the image's prior. It distinctly differs from the works that have conducted the aggregated algorithms into an energy function model [29-31, 34]. Our main contribution in this study is to propose an AGAP stereo matching algorithm with adaptive smoothness prior that effectively integrates the energy function into the cost aggregation framework. Specifically, the proposed AGAP constructs a form of energy function with adaptive smoothness prior at first, and in fact, it shows the matching costs from all of the four directions for each image traversal (as shown in Fig. 1b; more contents are described in Section 3). Then, the matching costs for all of the eight image traversals are aggregated together. After that, using WTA and disparity refinement, the final disparity map can be attained. Performance evaluations on Middlebury v.2 and v.3 stereo data sets [49-51] demonstrate that the proposed AGAP outperforms other four most challenging stereo matching algorithms, and also performs better on Microsoft i2i stereo videos [52]. In addition, under various strategies of parallelisation, the presented AGAP shows a near real-time execution time and also appears to have a relative lower energy than MGM. 3 AGAP stereo matching 3.1 Matching cost computation Different from original SGM that utilises supervised hierarchical mutual information (HMI), but the same as in MGM, a pixel-wise matching cost computation between images I1 and I2 can be computed as follows: (1)where , px is the x-axis location of pixel p on Cartesian coordinate, denotes a 5×5 census feature [53] around p for each corresponding image and is the Hamming Distance. 3.2 Matching cost with adaptive smoothness prior In SGM, the costs Li(p,d) on disparity level d are computed recursively from the pixels of the image along a one-dimensional path (performs as a scanline) on each cardinal direction i centred at p (totally performs 8 or 16 cardinal directions). In fact, it is constructed as a form of energy function. Then, the authors of MGM have interpreted the relationship between SGM and mini-sum BP. They clearly demonstrate that recursive updating Li(p,d) is computing the state belief of pixel p on each cardinal direction i. This interpretation guides MGM could directly receive as well as the beliefs from the pixels from two directions: current pixel p's previous pixel's direction p–r and its directly above direction p–r⊥ for each image traversal direction r. They also explain that compared to BP, in SGM, the propagated belief overcounts the data term (i.e. the matching cost) by a factor Ndir − 1 (Ndir = 8), which needs to subtract them in cost aggregation. Following the idea of MGM, we construct a more elegant form of energy function model of AGAP, which considers all of the four directions for each image traversal direction. While scanning on each image traversal direction r (as shown in Fig. 2) for current pixel p at disparity level d, the costs could be formulated recursively as follows: (2) Fig. 2Open in figure viewerPowerPoint Depiction of all of the eight image traversals and the corresponding recursive directions on the main directions and side directions in AGAP Here, , denotes (1) and it performs as the data term. In this model, we divide the four directions into two main and side directions, respectively, where , represent the two main directions and , represent the two side directions. The two main directions must perform in accordance with or perpendicular to current the image traversal direction, and the other two side ones' intersection angles with current image traversal direction must equal 45° or 135°. For each current pixel p (red pixels shown in Fig. 2), the matching costs are recursively updated from all of the four previous pixels' directions (the yellow, green, cyan and magenta pixels' directions shown in Fig. 2). For all of the eight image traversal directions, the procedures of each red pixel's recursive updating are shown in Fig. 1b. For the smoothness term of two main directions, Vm(d,d′) is formulated as (3)where the term represents the adaptive smoothness prior and it is computed as (4)Here, P1, P2, T and K are user-specified constants, and calculates the dissimilarity of neighbored pixels u and v. It shows definitely that this prior adaptively smooth the neighbored pixels based on their correlation, i.e. the dissimilarity in RGB colour space. When u and v are similar to each other, ; otherwise, . The, the smoothness term of two side directions Vs(d,d′) is formulated as (5)where P3 is a penalty constant while performing d = d′ on two side directions. The reason why this term be augmented is to alleviate the mismatching and propagate the error disparity estimation. For the rest constants and adaptive smoothness prior, are the same as in (3) and (4). Consequently, similar to SGM and MGM, (2) is finally expressed with a form of energy function as (6)where and . 3.3 Cost aggregation The same as other aggregated stereo matching algorithms, a cost aggregation step must be implemented. In AGAP, all of the matching costs (with a form of energy function) for all of the eight image traversals are aggregated together but subtract the excess data terms directly as in MGM (7)where Ndir = 8 and Lr(p,d) denotes (6). 3.4 Disparity computation A WTA strategy is widely used for selecting the disparity value d∈[0, max disp] for pixel p in cost aggregation results (8)After disparity computation, the initial disparity map is obtained. It shows the true performance of a stereo matching algorithm. 3.5 Disparity refinement For attaining the final refined disparity map, the strategies such as LRC, OF and WMF are also widely applied like other stereo matching algorithms. 4 Implementation details The proposed AGAP is described in Algorithm 1 (see Fig. 3) with pseudo code. We could easily observe that AGAP could implement recursively as other normal aggregated stereo matching algorithms. Fig. 3Open in figure viewerPowerPoint Algorithm 1: AGAP stereo matching Similar to MGM, AGAP also introduce dependency with multiple scanlines, which means it cannot be used for parallelised computing as SGM. Thus, parallelisation in AGAP remains keeping a diagonal-by-diagonal approach on multi-CPU processing, which means transforming other image traversal directions into a raster order (from left to right and top to bottom). In this circumstance, the pixels of the diagonal going from top-right to bottom-left can be processed in parallel, because they only depend on their four neighbours: top, left, upper-left and upper-right (except for the upper-right one, the rest of them are computed by previous diagonals, as shown in Fig. 4). Moreover, we also utilise Intel Threading Building Blocks [54] and Open Multi-Processing [55] for further acceleration. Using these strategies, AGAP performs a near real-time execution time, speeds up 35–45 times than MGM. Fig. 4Open in figure viewerPowerPoint Diagonal-by-diagonal parallelisation for AGAP on raster order. In this strategy, all of the image traversal directions have been transformed into this circumstance for acceleration As an example, the disparity maps of Cones (without disparity refinement) of original MGM and our proposed AGAP are shown in Fig. 1. It shows that the result of AGAP (Fig. 1b) greatly outperforms that of MGM (Fig. 1a). 5 Experimental results In this section, we are interested in comparing our AGAP with other four state-of-the-art stereo algorithms: MGM [38], WCPSP [17], SWSS [12] and SPS [37]. Except for the original MGM, WCPSP and SWSS could be seen as the most recently cost aggregation stereo matching algorithms, where WCPSP should be seen as a non-local algorithm and SWSS indicates a local one. Besides, SPS is seemed as a most challenging algorithm in which the accuracy is approximate to DL-based methods. All of the estimated algorithms are implemented on the same PC platform with 3.60 GHz, Octa-Core Intel Core i7 CPUs, 16 GB RAM and 64-bits OS. Also, they were completely done by C++ code. That means they could be fairly evaluated both in accuracy and time efficiency. For further proving the effectiveness, we estimate all of the five algorithms on Middlebury v.2 and& v.3 stereo data sets and Microsoft i2i stereo videos. Moreover, for the parameters P1, P2, P3, T and K, which appeared in (3)–(5), we set them as , respectively, for AGAP. 5.1 Energy evaluation From Section 3, we know that (1) is the initial matching cost computation. It is usually utilised as data term (unary potential) for a standard energy function. Furthermore, (2) performs as an approximation to the point-wise marginal, i.e. matching cost with adaptive smoothness prior, which means that after cost aggregation and WTA, the solution will be attained. Thus, the solutions of MGM and AGAP could be conducted as the data term for computing the energy of an energy function. We first formulate an energy function based on the solution of MGM, which can be expressed as (9)where the data term CMGM(p) denotes the solution of MGM, i.e. the cost volume for each pixel by WTA. is the edge set of the four connected image graphs. The smoothness term is the l1-norm truncated at 4, which amounts to taking , , and it is formulated as (10)Then, similar to (9), the energy function based on the solution of our proposed AGAP can be easily expressed as (11)Here, the data term CAGAP(p) denotes the solution of AGAP. To be fair, the smoothness term is set to be the same as (9). After constructing the energy functions for MGM and AGAP with different data terms and a same smoothness term, a comparison of energy evaluation between them could be depicted. Here, eight stereo pairs, Tsukuba, Teddy, Midd1, Reindeer (from Middlebury v.2 stereo data set), Adirondack, Classroom1, Playroom and Sword1 (from Middlebury v.3 stereo data set) are adopted to evaluate the two algorithms, respectively. This comparison is depicted as a bar chart in Fig. 5. It can be observed that the proposed AGAP (draws in green bars) shows a much lower energy than that of MGM (draws in red bars). Fig. 5Open in figure viewerPowerPoint Energy comparison between MGM (red bars) and APAP (green bars) on eight different stereo pairs, where Tsu., Reind., Adiron., Class. and Play. denote Tsukuba, Reindeer, Adirondack, Classroom1 and Playroom, respectively 5.2 Middlebury stereo data sets 5.2.1 Without disparity refinement We first compare our AGAP with MGM, WCPSP, SWSS and SPS on all 31 Middlebury v.2 stereo pairs. For fair evaluation, there are no disparity refinement strategies adopted and only the error rates in non-occluded regions are measured. The error threshold is set as 1.0 pixel. We reveal the quantitative evaluations in Table 1. The normal numbers are the percentages of error pixels of initial disparity maps and the subscript numbers are the relative rank in each row. It shows that AGAP ranks 1 on 20/31 stereo pairs (marked with bold in corresponding rows). The average rank and the average error rates are computed in the last but one and two rows. We can easily conclude that AGAP performs best overall accuracy and the best overall rank among all of the five estimated algorithms. For time efficiency, AGAP shows 152.56 times faster than SWSS, also speeds up 42.38 times than MGM. Although our AGAP performs a little slower than WCPSP and SPS, it is well worth the time and keeps the near real-time execution time. Corresponding to Table 1, the initial results of Cloth1 and Lampshade2 are shown in Fig. 6. For these two traditional stereo pairs, our AGAP shows marvelous improvement. Table 1. Quantitative evaluations on Middlebury v.2 stereo data set Stereo pairs MGM WCPSP SWSS SPS AGAP Tsukuba 9.964 3.012 2.261 10.935 5.373 Venus 6.744 0.861 3.933 7.455 1.552 Teddy 17.425 3.711 9.963 14.754 9.562 Cones 11.344 3.952 4.293 12.205 3.301 Aloe 7.214 4.042 4.563 19.145 3.691 Art 14.384 8.331 9.073 30.335 8.672 Baby1 4.423 3.342 4.954 9.335 2.301 Baby2 5.254 3.212 5.093 9.065 2.351 Baby3 5.133 2.581 11.304 11.925 3.522 Books 10.164 6.421 8.993 17.625 8.882 Bowling1 6.882 11.364 9.853 14.075 4.131 Bowling2 8.374 5.852 6.903 13.595 5.791 Cloth1 0.522 0.693 1.644 7.475 0.141 Cloth2 2.703 2.372 3.774 13.905 1.351 Cloth3 2.094 1.632 2.033 10.465 1.211 Cloth4 1.713 1.642 1.784 10.935 0.851 Dolls 8.344 4.061 5.633 23.395 4.832 Flowerpots 14.285 13.533 13.102 13.614 8.821 Lampshade1 9.972 10.313 10.544 21.575 4.861 Lamspshde2 9.653 7.242 15.345 14.744 4.231 Laundry 15.313 11.771 18.394 24.525 13.132 Midd1 44.245 28.722 34.283 41.504 24.331 Midd2 44.425 35.073 31.262 39.204 24.691 Moebius 12.264 10.423 10.212 19.905 7.601 Monopoly 29.405 28.164 22.932 26.233 14.831 Plastic 39.604 43.245 26.272 34.533 14.311 Reindeer 8.084 6.112 7.523 17.235 4.621 Rocks1 2.442 1.771 4.384 11.775 3.443 Rocks2 2.343 1.331 3.114 10.425 2.262 Wood1 5.354 2.842 4.023 7.045 1.991 Wood2 2.673 0.781 4.794 8.695 2.072 avg. error 11.704 8.662 9.743 17.025 6.411 avg. rank 3.654 2.062 3.163 4.715 1.421 avg. time (s) 94.504 0.411 340.205 1.942 2.233 Fig. 6Open in figure viewerPowerPoint Results of Cloth1 and Lampshde2 (Middlebury v.2). There are no disparity refinement strategies adopted and error threshold is 1.0 pixel. All of the error rates in non-occluded regions are indicated below for each disparity map Then, we also evaluate the same five stereo matching algorithms on 28 Middlebury v.3 stereo pairs (quarter size). For this new high-resolution data set, we also perform all of the implementations without disparity refinement strategies and only the error rates in non-occluded regions are measured. The error threshold is set as 2.0 pixel here. We reveal the quantitative evaluations of Middlebury v.3 stereo data set in Table 2. As described in Table 1, the normal numbers are the percentages of error pixels of initial disparity maps and the subscript numbers are the relative rank in each row. Even for this challenging stereo data set, AGAP outperforms other four algorithms on 23/28 stereo pairs and remains keeping the best overall accuracy. Even if the execution time has increased than that of Middlebury v.2 stereo pairs, AGAP also demonstrates a higher time efficiency. Especially, our presented AGAP speeds up 38.32 times than MGM on Middlebury v.3 stereo data set, and also implements 234.53 times faster than SWSS. We observe that WCPSP shows a shorter implementing time than that of AGAP, but with 3.61 times rocketing than on Middlebury v.2 stereo data set. Corresponding to Table 2, the initial disparity maps of Classroom1, Pipes and Storage are shown in Fig. 7. It