Learning Optimal  Combination Patterns for Lightweight  Stereo Image Super-Resolution

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-15 07:35:01

Abstract: Stereo image super-resolution (stereoSR) strives to improve the quality of super-resolution  by leveraging the auxiliary information provided by another perspective. Most approaches concentrate on  refining module design, and stacking massive network blocks to extract and integrate information. Although there have been advancements, the memory and computation costs are increasing as well. To tackle this issue, we propose a lattice structure that autonomously learns the optimal combination patterns of network blocks, which enables the efficient and precise acquisition of feature representations, and ultimately achieves lightweight stereoSR. Specifically, we draw inspiration from the lattice phase equalizer and design lattice stereo NAFBlock (LSNB) to bridge pairs of NAFBlocks using re-weight block (RWBlock) through a coupled butterfly-style topological structures.  RWBlock empowers LSNB with the capability to explore various combination patterns of pairwise NAFBlocks by adaptive re-weighting of feature. Moreover, we propose a lattice stereo attention module (LSAM) to search and transfer the most relevant features from another view. The resulting tightly interlinked architecture, named as LSSR, extensive experiments  demonstrate that our method performs competitively to the  state-of-the-art.

Main paper: 
 introduction: 
 Stereo super resolution (stereo SR) images have garnered significant interest owing to their significant utility in 3D applications, such as depth estimation for autonomous vehicles [20] and computerassisted surgery [21] . Nevertheless, in real-world scenarios, the generation of low-resolution (LR) image pairs is common due to acquisition limitations and the widespread use of cost-effective imaging systems. Consequently, the effective utilization of information from low resolution stereo images for stereo super resolution has emerged as a crucial undertaking.Unlike single image SR (SISR), stereoSR performance relies not only on the intra-view information within the left and right images but also on the cross-view information between them. Hence, utilizing established SISR methods such as [15-17, 40, 41, 43] for independent reconstruction of the left and right images faces limitations in terms of performance due to the lack of cross-view information.Recently, numerous deep learning approaches [3, 5, 13, 18, 19, 25, 36, 38] have been proposed for the stereoSR, yielding remarkable achievements. PASSRnet [33] first attempt to introduce a parallaxattention stereo super resolution network, which employs a global receptive field to effectively handle a diverse set of stereo images exhibiting substantial disparities. To enhance the effective integration of cross-view information, [1, 2, 34, 42] focus on the finely designing cross-attention module. Nevertheless, this results in an Figure 1 : The total number of parameters vs. PSNR of models for 4× stereo SR on KITTI 2015 [26] testset. Our LSSR achieve the SOFT performance with up to 89% of parameter reduction.increase in system complexity. To address this issue, [5] propose the stereo cross-attention modules (SCAM) between consecutive NAFBlocks [4] , which blends the simplicity and effectiveness of NAFNet [4] . Leveraging the remarkable capabilities of the Transformer [30] , [3, 9, 18, 38] devise Transformer-based models to reliably capture stereo correspondence and seamlessly integrate cross-view information for stereoSR. Despite the successive breakthroughs achieved by the aforementioned methods, they give rise to increased memory and computational demands, mainly as a consequence of refining modules and merely stacking multiple network blocks.Based on the information presented, a natural question that comes to mind is whether it is feasible to design a learnable combination pattern instead of merely stacking massive network blocks? To achieve this goal, we design a lattice stereo super resolution architecture, named as LSSR. Specifically, we utilize NAFBlock [4] as the base block and concentrate on designing a novel combination pattern. Drawing lessons from the butterfly-style topological structures of the lattice phase equaliser [14, 24] , we propose the lattice stereo NAFBlock (LSNB). LSNB is a versatile block that connects pairs of NAFBlocks using the re-weight block (RWBlock) within coupled butterfly-style topological structures. RWBlock adaptively reweighting input features empowers LSNB to explore the various linear combination patterns between pairwise NAFBlocks rather than merely stacking numerous network blocks, resulting in lightweight stereoSR through the utilization of the learned optimal combination pattern. What's more, we propose a lattice stereo attention module (LSAM) to enhance the effective information exchange between two views. It initially calculates bidirectional cross-attention from both left to right and right to left views and subsequently merges the mutually influenced cross-view features with intraview features. Figure . 1 shows our LSSR achieves state-of-the-art performance while also maintaining a lightweight parameter.The main contributions of this work are:(1) We introduce a lightweight lattice stereo super resolution approach named LSSR, which learns the optimal combination patterns of network 4) We design a lattice stereo attention module (LSAM) to enhance the effectiveness of information exchange between intra-view and cross-view features. 
related works 2.1 single image super-resolution: 
 Single image super-resolution (SISR) is a longstanding problem that has been under investigation for decades [15, 27, 37, 43] . Its objective is to generate high-resolution images solely based on intra-view information derived from low-resolution counterparts. SRCNN [7] makes the initial foray into applying deep learning to SISR by employing a three-layer convolutional neural network for the SR task. In order to enhance the model's representation capabilities, increasingly intricate models are being devised. VDSR [11] and EDSR [17] amplify the model's depth and width while implementing skip connections for residual information learning, thereby preventing gradient collapse. CBAM [35] utilizes channel and spatial attention blocks to extract the contextual relations as highly effective tools for addressing SISR. Transformers [16, 30, 39] have been applied to SISR, demonstrating commendable performance by effectively capturing non-local information. SwinIR [16] proposes an image restoration method based on the Swin Transformer [22] and achieves state-of-the-art performance on SISR.Stereo image super-resolution (stereoSR) is dedicated to restoring high-resolution details in both the left and right views of stereo image pairs by leveraging cross-view information [31, 32] . Stere-oSR [10] learns a parallax prior by jointly training two cascaded subnetworks for luminance and chrominance, integrating cross-view information by concatenating the left image with right images with predefined shifts. However, it has limitations in handling scenes with significant disparity variations due to fixed shift intervals. To handle this issue, PASSRnet [33] introduces a parallax attention module to acquire stereo correspondence. iPASSR [34] integrates a symmetric bi-directional parallax attention module (biPAM) and an inline occlusion handling scheme to effectively utilize symmetry cues. And NAFSSR [5] yields impressive results by interposing simple cross-view attention modules (SCAMs) between consecutive NAFBlocks [4] . Transformer-based models [3, 18, 38] are now being utilized in stereoSR because of their capacity to capture long-range dependencies within images. SIR-Former [38] is the first to introduce transformers into stereo image super-resolution, utilizing a cross-attention module to learn epipolar line relationships and a transformer-based fusion module for accurate cross-view feature integration. Then, SwinFSR [3] introduces an extended StereoSR method, building upon the SwinIR [16] foundation and leveraging frequency domain knowledge acquired through fast fourier convolution [29] . Futhermore, Steformer [18] leverages Transformer's self-attention for capturing both cross-view and intra-view information in stereo image, ensuring dependable stereo correspondence and cross-view integration. While all of the mentioned models have shown significant performance enhancements, they primarily rely on stacking numerous blocks and do not delve into the combination patterns among blocks.In this paper, we introduce a novel lattice stereo NAFBlock (LSNB) that leverages coupled butterfly-style topological structures. This design allows to learn the optimal combination patterns between pairwise NAFBlocks. Additionally, we have designed a lattice stereo attention module (LSAM) to effectively seamlessly integrate cross-view information for stereoSR. 
method: 
 Our primary goal is to investigate a adaptively regulated combination pattern instead of merely stacking massive network blocks in the model. With this goal in mind, we present a lattice stereo super resolution network (LSSR) shown in Figure . 2. LSSR employs two weight-sharing branches constructed with lattice stereo NAFBlock (LSNB) to separately extract intra-view features from the left and right images. Additionally, lattice stereo attention modules (LSAMs) are incorporated to combine cross-view features. Overall Pipeline. Given a pair of low resolution stereo imagesI L ∈ R 𝐻 ×𝑊 ×3 (left view) and I R ∈ R 𝐻 ×𝑊 ×3 (right view), LSSR first applies a 3 × 3 convolutional layer to extract shallow feature maps F S L ∈ R 𝐻 ×𝑊 ×𝐶 , F S R ∈ R 𝐻 ×𝑊 ×𝐶 (𝐻,𝑊, 𝐶 are the feature map height, width, and channel number, respectively). Next, these shallow features F S L , F S R pass through 𝑁 LSNBs to achieve deep intra-view feature extraction. To interact with cross-view information, we incorporate LSAM following each LSNB. After the aforementioned process, we acquire deep features denoted as F D L and F D R , encompassing both intra-view and cross-view information. Furthermore, we apply a 3 × 3 convolution layer followed by a pixel shuffle layer to upsample the deep feature by a scale factor of 𝑠, and generate R L ∈ 𝑅 𝐻 ×𝑊 ×3 , R R ∈ 𝑅 𝐻 ×𝑊 ×3 . Noted, to alleviate the burden of feature extraction, the R L ∈ 𝑅 𝐻 ×𝑊 ×3 and R R ∈ 𝑅 𝐻 ×𝑊 ×3 are the difference between the bilinearly upsampled low-resolution image and the high-resolution ground truth. Thus, the R L + I L , R R + I R are the high-resolution images of the left and right views, respectively. In pursuit of comprehensive and precise features, [3, 18, 34, 38] concentrates on refining module design and stacking massive network blocks, despite the resource-intensive nature of this approach. To address these limitations, inspired by the butterfly-style topological structures [24] , we design a lattice stereo NAFBlock (LSNB) as the basic building block from the perspective of network block combination patterns. As shown in Figure . 3(a), we use the nonlinear activation free block (NAFBlock) [4] as the base block and employ the re-weight block (RWBlock) adaptively reweighting input features empowers LSNB to learn the optimal combination pattern between pairwise NAFBlocks. To be specific, given a input featuresx, the first combination can be defined as :EQUATIONwhere 𝑁 𝐴𝐹 𝑖 (•) denotes the 𝑖-th NAFBlock, and 𝑅𝑊 𝑗 (•) represents the 𝑗-th RWBlock, which are described below. Then,𝑇 𝑖 -1 (x), 𝐵 𝑖 -1 (x) are used as input and a second combination is performed as:EQUATIONAfterwards, the results of the second combination 𝑇 𝑖 (x) and 𝐵 𝑖 (x) are combined in the channel dimension and subsequently subjected to a 1 × 1 convolution to transform them to their original dimensions.2024-04-12 14:14. Page 3 of 1-9. Based on the formula provided above, it becomes evident that the output feature result following RWBlock re-weighting has an influence on the composition of both 𝑇 𝑖 (x) and 𝐵 𝑖 (x). RWBlock empowers LSNB with the capability to explore various combination patterns of pairwise NAFBlocks. Next, we will provide a detailed explanation of several candidate structures for LSNB following the application of different re-weight results. In the following cases, the bold "0" denote vectors with all elements being 0:(1) If 𝑅𝑊 1 (x) = 𝑅𝑊 2 (x) = 𝑅𝑊 3 (x) = 𝑅𝑊 4 (x) = 0, then Eq. 2 can be reformulated as follows:EQUATIONIn this case,𝑇 𝑖 (x) ≠ 𝐵 𝑖 (x), the structure of LSNB is simplified as two concurrent NAFBlocks as shown in Figure . 4(a). ( 2) If 𝑅𝑊 1 (x) = 𝑅𝑊 2 (x) = 𝑅𝑊 3 (x) = 𝑅𝑊 4 (x) = x, then Eq. 2can be reformulated as follows:EQUATIONIn this case,𝑇 𝑖 (x) = 𝐵 𝑖 (x), the structure of LSNB is simplified as cascade connections of pair-wise NAFBlocks as shown in Figure . 4(b).(3) If 𝑅𝑊 1 (x) = 𝑅𝑊 2 (x) = x, 𝑅𝑊 3 (x) = 𝑅𝑊 4 (x) = 0, then Eq. 2can be reformulated as follows:EQUATIONIn this case, 𝑇 𝑖 (x) ≠ 𝐵 𝑖 (x), as shown in Figure . 4(c), the structure of LSNB is simplified as a NAFBlock following a NAFBlock, the final result is the output of each block like [4, 5] . (4) If 𝑅𝑊 1 (x) = 𝑅𝑊 2 (x) = 0, 𝑅𝑊 3 (x) = 𝑅𝑊 4 (x) = x, then Eq. 2 can be reformulated as follows:EQUATIONIn this case,𝑇 𝑖 (x) = 𝐵 𝑖 (x), the structure of LSNB is simplified as two parallel NAFBlocks as shown in Figure . 4(d) .Beyond the special cases mentioned earlier, the RWBlock can also dynamically re-weight features using the attention mechanism, leading to the inclusion of numerous other potential candidate structures within LSNB. To put it differently, the varied combination patterns provided by LSNB allow us to learn the optimal combination pattern for for designing lightweight models, avoiding the approach of simply stacking a large number of modules. As Figure . 3(b) shows, RWBlock consists of two branches: in the upper branch, we calculate the mean value of input features, while in the lower branch, we compute the standard deviation of input features. Following this, each branch are separately processed by two 1 × 1 convolution layers. Each of these convolution layers is succeeded by a RELU activation layer and a Sigmoid activation layer. Subsequently, the output from the two branches are averaged to obtain the reweight coefficients. Finally, these reweight coefficients are multiplied by the input features to obtain the re-weighted features. In this way, given an input feature 𝑋 𝑓 , the entire feature re-weight procedure of the developed RWBlock is formulated as:𝑋 𝑢𝑐 = 𝑆𝑖𝑔𝑚𝑜𝑖𝑑 (𝑓 𝑐 1×1 (𝑅𝐸𝐿𝑈 (𝑓 𝑐 1×1 (𝐺𝐴𝑃 (𝑋 𝑓 ))))) 𝑋 𝑙𝑐 = 𝑆𝑖𝑔𝑚𝑜𝑖𝑑 (𝑓 𝑐 1×1 (𝑅𝐸𝐿𝑈 (𝑓 𝑐 1×1 (𝑆𝑇 𝐷 (𝑋 𝑓 ))))) 𝑋 𝑐𝑐 = (𝑋 𝑢𝑐 + 𝑋 𝑙𝑐 ) × 0.5 𝑋 𝑟 𝑓 = 𝑋 𝑓 ⊗ 𝑋 𝑐𝑐 (7)where 𝑓 𝑐 1×1 represent 1 × 1 convolution, GAP is the global average pooling, STD is the standard deviation, 𝑋 𝑐𝑐 is the learned reweight coefficients, and 𝑋 𝑟 𝑓 is the re-weighted features.As previously mentioned in our introduction, we use the nonlinear activation free block (NAFBlock) [4] as the base block in our LSNB. Fig. 3(c ) illustrates the process of obtaining an output 𝑁 𝐴𝐹 (𝑋 ) from an input 𝑋 using Layer Normalization (LN), Convolution, Simple Gate (SG), and Simplified Channel Attention (SCA). Express as follows: where 𝑓 𝑑𝑤𝑐 3×3 is the 3 × 3 depth-wise convolution, and 𝑋 𝑓 1 , 𝑋 𝑓 2 ∈ 𝑅 𝐻 ×𝑊 × 𝐶 2 are obtained by dividing 𝑋 𝑓 0 into channel dimensions. For a more intuitive presentation, we show 𝑆𝐶𝐴(•) in Fig. 3(d ).EQUATIONWe reexamine all the prior cross-attention modules [3, 5, 34] , where we compute the dot products between the query 𝑄 ∈ R 𝐻 ×𝑊 ×𝐶 projected by the source intra-view feature (e.g., left-view), and the key,value 𝐾, 𝑉 ∈ R 𝐻 ×𝑊 ×𝐶 projected using the target intra-view feature (e.g., right-view). Followed by applying a softmax function to derive the weights assigned to the values:EQUATION)where 𝛽 is a learning scaling parameter used to adjust the magnitude of the dot product of 𝑄 and 𝐾 prior to the application of the softmax function defined by 𝛽 = √ 𝐶. Inspired by [3, 24] , we propose a lattice stereo attention module (LSAM) to effectively capture reliable stereo correspondences and seamlessly integrate cross-view information for stereoSR, as shown in Figure . 5. Specifically, given a pair of intra-view features 𝑋 𝐿 , 𝑋 𝑅 ∈ R 𝐻 ×𝑊 ×𝐶 , we begin by applying layer normalization, and subsequently derive the feature X𝐿 and X𝑅 using LSNB in conjunction with a 1 × 1 convolution layer. Then, we follow [3, 34] 𝐿 . Next, we generate the value matrix 𝑉 𝐿 and 𝑉 𝑅 by using a 1 × 1 convolution layer, respectively. Subsequently, we compute bidirectional crossattention between the left and right views as follows:EQUATIONFinally, the interacted cross-view information 𝐹 𝑅→𝐿 , 𝐹 𝐿→𝑅 and intra-view information 𝑋 𝐿 , 𝑋 𝑅 are fused by element-wise addition:EQUATION)where 𝜆 𝐿 and 𝜆 𝑅 are channel-wise scale parameters that are trainable and initialized with zeros to aid in stabilizing training.Data augmentation. In stereoSR tasks, it's common to apply random horizontal and vertical flips for dataset diversity. Furthermore, to enhance data utilization, we employ channel shuffling, randomly rearranging the RGB channels for color augmentation. Loss. Following [5] , we only use the pixel-wise L1 distance between the SR and ground-truth stereo images: where 𝐼 𝑆𝑅 𝐿 and 𝐼 𝑆𝑅 𝑅 represent the super-resolved left and right images, and 𝐼 𝐻𝑅 𝐿 and 𝐼 𝐻𝑅 𝑅 represent their ground-truth high-resolution images.EQUATION) 
experiments 4.1 implementation details: 
 Datasets. To train the proposed network, we utilize training data identical to that of [5, 34] . Specifically, we collected 800 images from the Flickr1024 dataset [33] and 60 images from the Middlebury dataset [28] for our training dataset. Since the images from the Middlebury dataset have a much higher spatial resolution than others, we perform bicubic downsampling with a scale factor of 2 to generate HR images from them. To generate LR images, we apply bicubic downsampling to HR images using specific scaling factors (i.e., 2× and 4×). The resulting LR images were then cropped into 30 × 90 patches with a stride of 20, and their HR counterparts were cropped accordingly. Finally, we obtained a total of 49, 020 patches for 4× SR training and 298, 143 patches for 2× SR training.Evaluation details. To assess the performance of the proposed network, we employed a test dataset consisting of 112 images from the Flickr1024 dataset [33] , 5 images from the Middlebury dataset [28] , 20 images from the KITTI 2012 dataset [8] , and 20 images from the KITTI 2015 dataset [26] . To achieve fair comparison with [5, 33, 34] , we report Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) on the left views while cropping their left boundaries (64 pixels), and the average scores on stereo image pairs (i.e., (𝐿𝑒 𝑓 𝑡 + 𝑅𝑖𝑔ℎ𝑡)/2) without any boundary cropping.Training details. We train our LSSR using the Adam [12] optimizer (𝛽 1 = 0.9, 𝛽 2 = 0.9) and the batch size was fixed at 32 for 1 × 10 5 iterations. We initiated training with a learning rate of We compare our LSSR with existing SR methods, encompassing both SISR methods [16, 17, 40, 41] and stereoSR methods [1, 5, 6, 18, 33, 34, 38] . Note that, for a fair comparison, we retrained all of these methods using our training dataset. Quantitative results. As the quantitative results shown in Table. 1, LSSR achieves considerable results on all datasets [8, 26, 28, 33] and upsampling factors (×2, ×4) with a lower number of parameters. More specifically, in the case of 4× stereo SR, with 29% fewer network parameters, our LSSR outperforms the previous state-of-the-art model NAFSSR [5] by 0.09 dB, 0.11 dB, 0.20 dB on KITTI 2012 [8] , KITTI 2015 [26] and Middlebury datasets [28] , which demonstrates the effectiveness of the proposed LSSR.Lightweight and Efficiency. We presented a visualization of the trade-off results between the total number of parameters and PSNR on the KITTI 2015 dataset [26] for 4× stereo SR. As shown in Figure . 1, it's evident that in comparison to SwinFSR [3] , our LSSR achieves a state-of-the-art result with an impressive 89% reduction in parameters. And as shown in Table . 2, our LSSR use fewer Flops. This indicates that our LSSR is lightweight network.We also provide the runtimes in Table . 2 (evaluated with 128 × 128 input on RTX 2080Ti GPU) to compare the computational complexity with SSRDE-FNet [6] and NAFSSR [5] . Our LSSR achieves the best performance with a speedup of up to 3.07×. This highlights the fast and efficient nature of our LSSR. Visual Comparison. We show the visual comparisons for ×2 stereo SR on Flickr1024 [33] , and ×4 on Middlebury and KITTI2012 in Figure. 6 7 8 . These figures showcase that our LSSR effectively generates high-quality super-resolution images with intricate details and well-defined edges. In contrast, the other methods we compared may exhibit undesirable artifacts. This solidifies the proof of the effectiveness of our LSSR.Lattice Stereo NAFBlock. To confirm the effectiveness and versatility of the Lattice Stereo NAFBlock (LSNB), we conducted experiments on both the NAFSSR [5] model and the LSSR model, using LSNB and NAFBlock [4] , respectively. Here, "w/o" denotes the usage of NAFBlock, while "w" signifies the utilization of LSNB. As shown in Table . Re-weight Block. To learn the optimal combination pattern, we obtain re-weighted features using the attention-based mechanism RWBlock, which consists of two branches: global average pooling (GAP) and standard deviation (STD). As illustrated in Table. 4, the combination of both ensembles yields the best results. Lattice Stereo Attention Module. To show the effectiveness of LSAM, we substitute the LSAM in LSSR with several SOTA approaches, including biPAM [34] , SCAM [5] , and RCAM [3] . As shown in Table . 5, compared with biPAM, SCAM, and RCAM, our LSAM achieves improvements of 0.24 dB, 0.11 dB, and 0.12 dB, respectively. When we examine LSSR without LSAM to assess the effect of the proposed LSAM on cross-view information, we find that our method achieves a 0.46 dB improvement with LSAM.Data augmentations. We trained our model using different data augmentations to validate their effectiveness. As shown in Table . 6, introducing data augmentations such as random horizontal flip, random vertical flip, and channel shuffle has a positive impact on LSSR's performance. By employing all three data augmentations simultaneously, LSSR's PSNR increases from 23.45 dB to 23.87 dB, which is 0.11 dB better than using random flip alone.Single Input vs. Stereo Input. StereoSR leverages supplementary data from cross-view images to significantly improve performance compared with SISR. To showcase the efficacy of stereo information in enhancing super-resolution performance, we conducted experiments using various input schemes. As indicated in Table 7 , utilizing individual images during training results in a PSNR decrease of 0.45 dB compared to the baseline network. Likewise, when employing duplicated left images as inputs, the performance 2024-04-12 14:14. Page 7 of 1-9. of this modified configuration notably falls short of our initial network. These trials underscore the efficacy of our LSSR in capturing information from various perspectives. 
conclusion: 
 In this paper, we propose a lattice structure that autonomously learn the optimal combination pattern of network blocks, avoiding the common practice of indiscriminately stacking numerous network blocks, ultimately presenting a lightweight model. Specifically, we design a lattice stereo NAFBlock (LSNB), which serves to bridge pairs of NAFBlocks by incorporating the re-weight block (RWBlock) through a coupled butterfly-style topological structure. RWBlock empowers LSNB with the capability to explore diverse combinations of pairwise NAFBlocks by utilizing the feature results obtained from RWBlock's adaptive re-weighting process. Furthermore, we propose a lattice stereo attention module (LSAM) facilitates the proper and accurate extraction of cross-view information.Extensive experimentation shows our LSSR surpass current models, establishing itself as a state-of-the-art performer in the field. 
 
 Appendix:  


Decision: The paper proposes a new stereo image super-resolution method. The main contribution is a lattice structure that autonomously learns the optimal combination patterns of network blocks.
All reviewers agree that the experimental results show the proposed method outperforms SOTA with more efficient models.

The authors have addressed most of the concerns raised in the initial review.

Reviews: review: paper_summary: paper_summary: This paper introduces a novel stereo image super-resolution (stereoSR) method, aimed at enhancing the quality of super-resolution by leveraging auxiliary information from another viewpoint. The paper points out that existing stereo image super-resolution methods mainly focus on improving module design and stacking a large number of network blocks to extract and integrate information, but this approach leads to increased memory and computational costs. To address this issue, the paper proposes a lattice structure that can autonomously learn the optimal combination pattern of network blocks to achieve efficient and accurate feature representation, ultimately achieving the goal of lightweight stereo image super-resolution.
strength_weakness: strength_weakness: 1.	By autonomously learning the optimal combination mode of network blocks, it avoids simply stacking a large number of network blocks, thereby reducing the number of parameters of the model and achieving lightweight.
2.	The proposed lattice structure design allows the network to process data more efficiently, reducing memory and computational costs, which is especially important for resource-constrained environments.
3.	Utilizing the Lattice Stereo NAFBlock (LSNB) and the Re-weight Block (RWBlock), the network can more accurately acquire and integrate feature representations, which is crucial for enhancing the quality of stereo image super-resolution.
limitations: limitations: 1.Lacking innovation.The main architecture of this paper is very similar to NAFSSR, with a simple modification of the module. In the comparative experiments with NAFSSR, the performance does not show a significant improvement.
2. Please specify the version of NAFSSR used in the paper. The quantitative and qualitative results of NAFSSR (B) are superior to the proposed method.
3. In runtime comparison，the results of SSRDEFNet experiments in this paper differ significantly from those in the NAFSSR paper. Please explain the reason.
paper_summary: paper_summary: The paper proposes a novel method for improving stereo image super-resolution (StereoSR) efficiency. It introduces a lattice structure to learn optimal combinations of network blocks, significantly reducing the computational load while maintaining competitive performance. The method leverages a new component, the lattice stereo NAFBlock (LSNB), which connects NAFBlocks using a re-weight block (RWBlock) in a butterfly-style topology to optimize the combination patterns dynamically.
strength_weakness: strength_weakness: The introduction of a lattice structure to autonomously learn the optimal combination of network blocks represents a significant shift from the traditional stacking of blocks, offering a fresh perspective on network design for StereoSR.

The proposed method uses butterfly-style topologies to enable efficient feature representation, which is theoretically sound and technically innovative. The lattice stereo attention module (LSAM) enhances cross-view feature integration, contributing to the model’s effectiveness.

Extensive experiments demonstrate that the LSSR model outperforms existing state-of-the-art methods with fewer parameters, providing robust evidence of its efficiency and effectiveness. The experiments cover various datasets and comparisons against multiple baseline models.

The paper is well-structured, with clear explanations of the novel components and their contributions to the overall system.
limitations: limitations: While the overall architecture is innovative, the individual components like NAFBlock and RWBlock are not new and have been used in similar contexts in previous works.

The method involves complex topological structures that might be challenging to implement and optimize in practice without significant expertise.

While the model is lightweight, the scalability of the approach when dealing with higher resolutions or significantly different stereo image configurations is not discussed.
paper_summary: paper_summary: This work proposes a Light Stereo Super-resolution Method (LSSR) that learns the optimal combination patterns of network blocks. This network is constructed based on the proposed LSNB and LSAM.
strength_weakness: strength_weakness: The comparison results show the LSSR model’s excellent performance.

The writing is clear and well-articulated, with detailed descriptions of LSNB, RWBlock, and LSAM.
limitations: limitations: 1. As described in the Methods section, the LSNB can vary by adding or eliminating the NAFBlock. It is unclear how an LSNB block should select which variant to use.

2. The specific design features that help the proposed cross-attention module (LSAM) outperform other methods, as shown in Table 5, are not well explained. Is the whiten block a significant factor?

3. In Table 3, the comparison of parameters and FLOPs should be shown to ensure fairness in the experiments.

4. In Table 4, what would be the result if the GAP and STD were replaced with a simple MLP layer?
paper_summary: paper_summary: This paper studies the problem of how to effectively learn combination pattern instead of stacking massive network blocks in the context of stereo image super-resolution. Latticed adaptive reweighting is used to explore linear combination patterns between pairwise NAFBlocks, and lattice stereo attention module is employed to exchange information between stereo views. This design leads to lightweight network with good performance.
strength_weakness: strength_weakness: + The idea of adaptive combination pattern learning and lightweight cross-view information exchange is interesting. 
+ The authors have made every effort to introduce and explain the roles of LSNB and LSAM in their LSSR framework.
+ Promising experimental results are reported.
limitations: limitations: - Even though the network design looks reasonable, the paper lacks convincing evidence to describe what, how, and why diverse combination patterns can be learned and good information exchange can be achieved between stereo views to guide super-resolution. 
- The experiments are not very convincing. Since the topic is about Stereo SR, an instance of clear improvement of the left image by some clue in the right image would be beneficial. But unfortunately such examples are missing. In addition, more details should be given to show why the Number of parameters is low in comparison to other methods (Fig. 1).
- Important ablation studies are missing. The parameter N (# of lattice layers) is closely related to the structure and performance of the network. The performance curve plot with respect to N would provide helpful understanding and judgement of the LSSR framework. However, even its value (N=?) is not given, let alone a definitely deserved ablation study.
- The presentation has some problems. In the “overall pipeline” paragraph, both low-resolution and high-resolution images have dimension HxWx3. In Figure 8, the hightlight block is missing.  In Table 4, only Middlebury results are given?