Serial section microscopy image inpainting guided by axial optical flow

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-05 09:11:28

Abstract: Volume electron microscopy (vEM) is becoming a prominent technique in three-dimensional (3D) cellular visualization. vEM collects a series of two-dimensional (2D) images and reconstructs ultra-structures at the nanometer scale by rational axial interpolation between neighboring sections. However, section damage inevitably occurs in the sample preparation and imaging process, suffering from manual operational errors or occasional mechanical failures. The damaged regions present blurry and contaminated structure information, even local blank holes. Despite significant progress in single-image inpainting, it is still a great challenge to recover missing biological structures, that satisfy 3D structural continuity among sections. In this paper, we propose an optical flow-based serial section inpainting architecture to effectively combine the 3D structure information from neighboring sections and 2D image features from surrounding regions. We design a two-stage reference generation strategy to predict a rational and detailed intermediate state image from coarse to fine. Then, a GAN-based inpainting network is adopted to integrate all reference information and guide the restoration of missing structures, while ensuring consistent distribution of pixel values across the 2D image. Extensive experimental results well demonstrate the superiority of our method over existing inpainting tools.

Main paper: 
 introduction: 
 Volume electron microscopy (vEM) plays a critical role in understanding biological structures across scales, revealing biological complexity from the arrangement of organelles within cells, through the tissues composed of cellular communities, to the structural components of organisms [5] . vEM builds connections between two-dimensional (2D) imaging series and three-dimensional (3D) ultra-structures at the nanometer scale, according to high-precision 2D section alignment among an image stack and subsequent rational axial interpolation between neighboring sections. Analyzing the reconstructed volume, vEM enables researchers to comprehensively interpret the intricate topology of organelles within the crowded environments of a cell. However, section damage often occurs during the preparation and imaging process inevitably, preventing researchers from obtaining consistently high-resolution volumes [22] . It is largely due to manual operational errors and mechanical precision limitations, resulting in information missing in local regions and further reducing the resolution of reconstructed biological structures. The existence of section damage runs through the entire data acquisition process, involving heavy-metal staining, serial sectioning, and imaging. Different processes suffer specific image degradation patterns, but they all lead to invalid structural information (shown in Figure 1 ). Uneven staining caused by imperfect sample preparation greatly reduces the overall signal-to-noise ratio, and accompanying cloud-like sediments cover most of the biological structural areas. The blurred image greatly affects the contrast and pixel intensities of the reconstructed volume. Low-precision sectioning technique possibly generates sections with varying thicknesses, and even "erases" some regions of abundant structural information. The loss of section content undermines the foundation of high-resolution detailed structure reconstruction. Moreover, the erroneous or unstable microscope settings, such as probe dwelling time, electron beam, and so on, similarly lead to discontinuous or missing structures in a circular region. Under the collective influence of inevitable section damages, the reconstructed 3D volume may experience an inconsistent resolution or visualization scale across different across-sectional regions, severely interfering with the subsequent analysis and modeling. Although the above damages introduce a relatively large range of unreliable structural signals, abundant biological information remains in the section images. To achieve higher resolution and present clearer structures, missing information restoration in the damaged areas is becoming an urgent demand in vEM reconstruction.Benefiting from coherent 3D attributes in serial sections, many works explore the utilization of information from neighbor sections to restore the damaged areas. CCPGAN [31] first proposes to "copy" the most relevant patches from both the damaged image and its neighboring image in the feature domain to recover the missing regions. It relies heavily on the search for structurally similar regions, limiting its application scope to the repetition structure restoration with high-quality neighbor sections. Huang et al. [11] utilizes the interpolation result between two neighboring sections to guide the restoration of missing contents. The introduction of interpolation between sections preserves the axial continuity of biological structures and extends the application range. However, the blurred interpolation patches neglect the 2D structural consistency between the recovered content and undamaged parts. ACCP-GAN [23] designs a missing structure detection module to automatically determine the inpainted regions with arbitrary shapes. Transformer-based network enables the ability to capture structural similarity across input image stacks. Deng et al. [6] introduces a unified image inpainting framework to restore the artifacts caused by different image degradation. The interpolation result simply estimated between neighboring sections cannot provide clear and acceptable guidance for the subsequent inpainting module. In serial section inpainting, the 3D structural continuity and 2D image structural consistency are both the keys to rational and seamless restoration. Unfortunately, the axial trends of biological structures are non-trivial, without realistic ground truth images. The reference of interpolation results between neighboring sections is inevitably inconsistent with the damaged section, especially around the edge of missing areas. This raises a great challenge to bridge the gap between the reference image derived from 3D structural continuity and the damaged section of realistic 2D biological structures.Here, we propose a GAN-based vEM section inpainting network called FlowInpaint, which estimates an intermediate state between neighboring sections to guide the restoration of damaged regions. Firstly, two neighboring images of the damaged section are fed into an initial reference module to derive a plausible intermediate state that satisfies the axial continuity of biological structures. Compared to raw neighboring images, this intermediate state exhibits more similar structures to those of the damaged section, thus significantly reducing the complexity of subsequent optical flow estimation. Then, the damaged section and the intermediate state are jointly input into the refined reference module, aiming to generate credible reference structures for the damaged regions. Finally, the GANbased guided inpainting module employs a Unet-like architecture to extract multi-scale 2D image features from the reference image and raw neighbor sections. The abundant 2D structural features guide the rational restoration of missing structures, adhering to the 2D structural consistency around the missing areas. Moreover, a discriminator is used to ensure uniform pixel and noise value distribution, providing seamless and clear structure recovery in the inpainting regions. In summary, our main contributions are:  
related work: 
 Video frame interpolation and image inpainting are the key techniques to exploring 3D structural continuity and 2D image structural consistency respectively.Video frame interpolation (VFI) aims to synthesize one or several frames in the middle of two adjacent frames of the original video.Traditional methods construct a series of explicit functions, such as polynomial functions [14] , B-spline functions [2] , and so on, to fit the changes of pixel values under the same plane coordinates. However, the biological structures perform elastic transformation in the axial direction, making the correspondences of plane coordinates unable to directly reflect the correspondences of structures.With the development of deep learning, several convolutional neural networks (CNN) based VFI methods are presented to capture structural changes in-between the original frames. Long et al. [16] first attempt to use encoder-decoder architecture for the direct intermediate frame generation. Niklaus et al. [17] propose spatially-adaptive interpolation kernels to expand the receptive field without the excessive introduction of parameters. Cheng et al. [4] propose deformable separable convolution (DSepConv) to learn deformable offsets and masks, which can further extract features beyond the receptive field. These methods effectively restore the overall contour structures but exhibit a limited ability to recover more detailed or subtle structures. To trace the plausible motion of detailed structures, optical flow-based approaches estimate displacement for each pixel under the anti-folding constraints. Liu et al. [15] predict the 3D voxel flow to warp the input frames based on a trilinear sampling. DAIN [1] estimates the intermediate flow as a weighted combination of bidirectional flow and then warps the input frames using the adaptive warping layer. The above methods successfully estimate accurate and acceptable motion states of two input frames even under large displacement. However, in the serial section image inpainting, the missing information areas greatly influence the estimation of optical flow, making it difficult to estimate a reasonable deformation field directly from neighboring sections to the damaged area.Existing image inpainting approaches can be classified into sequentialbased algorithms and deep-learning based algorithms. Sequentialbased algorithms [7] search the best-matching patches of damaged areas from the surrounding areas or given image database and "paste" them into the appropriate positions. By performing patch-wise computations, sequential-based approaches effectively Î𝑖 . In the training stage, a hybrid discriminator is adopted to ensure the consistency between inpainted content and known regions.transform the complex image inpainting task into a simple database retrieval problem, whose key is the construction of similarity measurement between patches. Different from sequential-based algorithms, deep learning-based methods have a strong ability to extract complex texture features and establish pixel-level correspondences in the feature domain [26] . Recently, the generative adversarial network (GAN) [9] has emerged as a promising paradigm for clear and rational image content recovery. The introduction of discriminators effectively ensures the consistent distribution of signal and noise values across the final restored image. Inspired by the attention mechanism, contextual attention [27] is proposed to depict the patch-based correlation between missing regions and background areas explicitly. Researchers explore designing different feature extraction architectures to provide comprehensive feature descriptions of image patches or pixels. AOT-GAN [29] adjusts the dilation rates of different dilated convolution layers to expand or shrink the receptive fields, offering both large-range and small-range structural features. Quan et al. [19] combine respective advantages of the convolution with small receptive fields and large-scale attention mechanism to globally and locally restore the missing information. Zhang et al. [32] introduce semantic priors from specific pretext tasks to image inpainting. Existing deep-learning based algorithms have exhibited powerful abilities to preserve 2D structural consistency between missing and background areas in natural image inpainting. However, in serial section image inpainting, a reliable and high-resolution reconstruction of local structures depends crucially on the 3D axial structural continuity. Wang et al. [25] fuse useful features of neighboring images to reconstruct the intermediate image. Without information from neighboring sections, the image inpainting from 2D images alone cannot meet the requirements of downstream analysis tasks. 
methodology: 
 In this section, we provide a detailed introduction to the FlowInpaint architecture as shown in Figure 2 , which consists of three main modules, i.e., (a) the initial reference module, (b) the refined reference module, and (c) the guided inpainting module. with the estimated pixel displacement field. Finally, the GAN-base guided inpainting module accepts the 𝐼 𝑟𝑒 𝑓 𝑖 , 𝐼 𝑖 -1 and 𝐼 𝑖+1 as guidance information to restore the missing structures in 𝐼 𝑖 . The discriminator forces the 2D structural consistency around the missing regions while the reference image provides 3D axial continuity constraints of the internal structures.The initial reference module accepts two neighboring images 𝐼 𝑖 -1 and 𝐼 𝑖+1 of damaged section 𝐼 𝑖 to generate a stable intermediate state 𝐼 𝑖𝑛𝑡𝑒𝑟 𝑖 . Classical interpolation techniques use functions to fit the changes of pixel values of corresponding positions, unable to trace the transformation of structures. Here, we adopt an optical flow-based method raised by González-Ruiz et al. [8] to depict the changes of axial structures between inputs 𝐼 𝑖 -1 and 𝐼 𝑖+1 . This module first estimates two optical flows→ 𝑓 (𝑖 -1, 𝑖 + 1) from 𝐼 𝑖 -1 to 𝐼 𝑖+1 and → 𝑓 (𝑖 + 1, 𝑖 -1) from 𝐼 𝑖+1 to 𝐼 𝑖 -1 .Considering the structural continuity, we assume that the biological structures are transformed smoothly along the axial direction. Therefore, we can acquire two candidate intermediate states by simply weighting on the estimated optical flows as follows:EQUATIONEQUATIONwhich W(𝑖, 𝑓 ) warps the image 𝑖 with a optical flow 𝑓 , and 𝛼 is a weighting coefficient representing the transformation degree. In this paper, we uniformly set the 𝛼 to 0.5, assuming that all sections have the same thickness. Each candidate image exhibits the trend of structure transformation from the source image effectively without folding phenomenon. Finally, we fuse the two candidates to generate a bidirectional accessible state using the following weighting strategy:EQUATIONThe EQUATION)where 𝑇 is the transpose operator, 𝑁 is the length of the feature vector 𝐼 𝑖 (𝑘 ) , and 𝑈 𝑝 (•) represents the ×2 upsampling. According to the identification of low feature correlation, the estimator forces the sub-network E 𝑘 (•) to pay more attention to unregistered areas, as shown in Figure 3(A) . The optical flow → 𝑓 (𝑘 ) (𝑖𝑛𝑡𝑒𝑟 𝑖 , 𝑖) at the 𝑘th level is estimated by integrating all extracted information:EQUATIONwhere the binary mask 𝑚 𝑘 is downsampled from 𝑚 to match the current feature size. Finally, we utilize an additional context net- (6)The guided inpainting module integrates all guidance information including the neighboring images 𝐼 𝑖 -1 , 𝐼 𝑖+1 , and the refined reference images 𝐼to recover the missing regions of damaged image 𝐼 𝑖 . To ensure the consistent distribution of pixel values, we adopt a GAN-based architecture consisting of an attention-based generator and a patch-based discriminator, as shown in Figure 2 .The generator first constructs two multi-scale feature pyramids {η 𝑙 1 } 𝐿 𝑙=1 and {η 𝑙 2 } 𝐿 𝑙=1 from the guidance images {𝐼 𝑖 -1 , 𝐼 𝑖+1 , 𝐼 𝑟𝑒 𝑓 𝑖 } and 𝐼 𝑖 using a series of convolution layers (shown in Figure 2 ). The accompanying binary mask m, equaling 1 -𝑚, helps the network concentrate on the location of missing information. Notably, compared to the pyramidal extraction branch on the damaged image 𝐼 𝑖 , we add several extra feature aggregation layers (FAL) in the guidance image branch to extract and aggregate richer contextual features. Inspired by the atrous spatial pyramid pooling (ASPP) strategy [3] , the FAL consists of four dilated convolutions g 𝑟 (•) (𝑟 = 1, 2, 4, 8) with different dilation rates, providing an extended characterization of local structures at different receptive fields. A large dilation rate enables the convolution layer to capture a wide range of pixel information, but also easily results in low resolution and content loss. To fully utilize the advantages of dilated convolutions, the FAL aggregates the results of four dilated convolutions EQUATION)where ⊕ is a concatenation operation, η 𝑙 1 represents the feature at the 𝑙th level. For the level 𝑙, the 𝑙th feature attention layer accepts the 1 , η 𝑙 2 and the attention feature Ψ 𝑙+1 from the former level 𝑙 + 1 to generate the fused feature. The feature attention layer consists of a feature fusion layer h(•), integrating guidance features, and a contextual attention layer 𝑎𝑡𝑡 (•), "copying" the similar patches in the feature domain. The contextual attention mechanism has been introduced to ensure better results in multi-scale [24] . The contextual attention layer 𝑎𝑡𝑡 (•) [27] is used to model the patch (3 × 3) correlation between known regions 𝐾 𝑙 and missing regions 𝑅 𝑙 in the fused feature at the 𝑙th level. Let 𝑘 𝑙 𝑖 denotes the 𝑖th patch in the known region and 𝑟 𝑙 𝑗 denotes the 𝑗th patch in the missing region, the patch affinity s𝑙 𝑖,𝑗 is measured by the cosine similarity:EQUATIONand the attention score 𝑆 𝑙 𝑖,𝑗 ∈ 𝑆 is obtained by the softmax operation:EQUATION)Based on the feature similarity, the unknown regions 𝑟 𝑙 𝑗 in the fused feature at the 𝑙th level are updated by weighting the content of known patches:EQUATION)So the attention feature Ψ 𝑙 is represented as:EQUATIONFinally, the attention feature Ψ 2 is processed through a convolution block 𝒅 (•) for the generation of the inpainting result Î𝑖 as follows:EQUATIONwhere ⊙ is pixel-wise multiplication.The convolution architecture prefers to generate blurred visual results. To improve the perceptual quality of the inpainted image Î𝑖 , we adopt PatchGAN [12] as the discriminator, to ensure the consistent distribution of pixel values and the continuity of structures around the missing areas. The discriminator consists of a global context discriminator network and a local context discriminator network, judging the rationality of restored information at different scales. The outputs of the two discriminators are concatenated together to finally predict a value within the range of [0, 1], representing the probability that the input image is real or fake.FlowInpaint is trained in a two-stage strategy with the same groundtruth image 𝑥, so as the loss functions.In the first stage, we train the refined reference network, and the loss function L 𝐶 is defined by collecting pyramidal losses L (𝑘 ) 𝐶 across all levels as follows:EQUATIONAt each level 𝑘, the loss L EQUATIONwhere L 𝑟 represents the reconstruction loss, L 𝑠𝑠𝑖𝑚 represents the structural similarity loss, L 𝑝𝑒𝑟 represents the perceptual loss. All 𝑤 𝑘 and 𝜆 𝑐 are constant coefficients. The detailed definition of each loss term is listed as follows:Reconstruct loss L 𝑟 is used to measure the L1 loss between the images 𝑥 and 𝑧 at the pixel level:EQUATIONStructural Similarity Loss L 𝑠𝑠𝑖𝑚 is used to calculate the difference in luminance, contrast, and structure of two images 𝑥 and 𝑧:EQUATIONwhere 𝜇 𝑥 and 𝜇 𝑧 are the average value of image 𝑥 and 𝑧, 𝜎 𝑥 and 𝜎 𝑧 are the standard deviation, 𝜎 𝑧 is the covariance, 𝐶 which is more fit for the human perception of image quality [13] :EQUATIONwhere 𝑝 𝑖 (•) represents the 𝑖th feature layer of the pre-trained VGG16 model.In the second stage, we train the guided inpainting network with the loss function L 𝐺 of the generator and the loss function L 𝐷 of the discriminator as follows:EQUATIONEQUATIONwhere L 𝐺_𝑎𝑑𝑣 and L 𝐷_𝑎𝑑𝑣 are adversarial loss, 𝜆 𝑟 and 𝜆 𝑎𝑑𝑣 are weighting coefficients. Adversarial Loss L 𝐺_𝑎𝑑𝑣 and L 𝐷_𝑎𝑑𝑣 are used for optimizing generator and discriminator. The input image consists of two regions: the initial known region and the generated inpainting region. To distinguish the generated patches in the missing region from the real patches in the context, we use the discriminator that is constructed based on PatchGAN [12] . The adversarial loss for the generator is:EQUATIONThe hinge version of adversarial loss for the discriminator is:EQUATION 
experiments: 
 FlowInpaint is compared with two single-image inpainting methods: GAN-based Unet-like network PEN-Net [28] , multiple contextual feature fusion framework AOT-GAN [29] , and two consecutiveimage inpainting methods: feature "copying and pasting" architecture CCPGAN [31] , frame interpolation based pipeline SSF-Restoration [11] .To comprehensively evaluate the network performance, we generate synthetic data based on the commonly used datasets, CREMI EQUATIONNotably, the mask is of irregular shape to simulate the realistic damaged regions.FlowInpaint is implemented by PyTorch [18] and all the experiments are trained on the NVIDIA A100 GPU. The input images are of 256 × 256 size. For the refined reference module, we extract two feature pyramids with 𝐾 = 6 and train the model with a learning rate schedule starting from 0.0001 and reducing the learning rate by half at 0.4M, 0.6M, 0.8M, and 1M iterations. For the guided inpainting module, we extract two multi-scale feature sets with 𝐿 = 6 and adopt the learning rate of 0.0001. Both modules are trained by the Adam optimizer, where 𝛽 1 = 0.5, 𝛽 2 = 0.999. We set the weights 𝑤 1 = 1, 𝑤 2 = 0.64, 𝑤 3 = 0.32, 𝑤 4 = 0.16, 𝑤 5 = 0.08, 𝑤 6 = 0.04, 𝜆 𝑐 = 5, 𝜆 𝑟 = 6, and 𝜆 𝑎𝑑𝑣 = 0.1 in the loss functions.generates completely nonsensical content while SSF-Restoration outputs severely blurry content. However, even in structure-dense areas, our method can still produce relatively coherent results.We conduct ablation experiments on the CREMI dataset to verify the effectiveness of each module in FlowInpaint.4.6.1 Reference Image Generation. FlowInpaint adopts a two-stage reference image generation strategy to introduce the 3D axial structural continuity into the reference image. First, we remove the reference image generation and directly input two neighboring images to the guided inpainting module (denoted as GuidedGAN in Table 2 ). Without reference images, GuidedGAN though has high enough numerical metrics, it prefers to generate false structures, which are obviously shown in the visual example (Figure 5 ).  
conclusion: 
 In this paper, we propose FlowInpaint, a vEM image inpainting network to recover the missing information at the damaged regions under the guidance of two neighboring sections. Previous singleimage inpainting methods only utilize 2D known cross-sectional information, and consecutive-image inpainting approaches adopt "copy-and-paste" strategy to fill similar structures from the neighboring images into the damaged region. These methods neglect the 3D structure changes along the axial direction and are quite sensitive to section thickness. On the contrary, FlowInpaint estimates optical flow from neighboring sections to construct 3D structural continuity. Meanwhile, the two-stage reference generation strategy effectively ensures strong stability under different thicknesses of samples. Compared to existing EM image inpainting networks, our method simultaneously considers the complexity of guaranteeing 3D structural continuity along the axial direction and preserving 2D structural consistency around the missing regions. Comprehensive experiments well demonstrate the superiority of FlowInpaint over existing methods in serial section inpainting. Further stability analysis reveals the applicability of FlowInpaint to different thicknesses of tissue sections and imaging situations, which are variable in real-world data. 
 
 Appendix: Dataset available at https://cremi.org/data/ 


Decision: After rebuttal, this paper received mixed ratings: 3 BAs and 1 BR. It appears that the rebuttal has done a good job of resolving reviewers' concerns. I would recommend acceptance, and ask the authors to consider improving their paper before the camera-ready due by incorporating the following comments: "I think more qualitative results about the impact of registration should be provided, and the generalization results are also important."

Reviews: review: paper_summary: paper_summary: This paper proposes an axial optical flow-guided microscopy image inpainting method, utilizing a GAN-based two-stage strategy. The structure is comprehensive, making it a commendable paper. However, improvements can be made in the experimental section, specifically in including comparative algorithms from the past two years and analyzing the loss functions.
strength_weakness: strength_weakness: The paper is well-structured, logically coherent, and rich in experiments. The proposed method demonstrates excellent qualitative and quantitative performance in the microscopy image inpainting problem.
limitations: limitations: 1. The text and images in Figure 2 overlap; the layout could be further optimized.
	2. There are several deficiencies in the experimental section:
		○ The necessity of using multiple loss functions, such as SSIM loss and perceptual loss, should be experimentally validated in the paper.
		○ The parameter count and computational load of different methods should be indicated. Additionally, more recent and excellent inpainting methods from the past two years should be included in the comparison.
		○ Since some methods are not specifically designed for microscopy image inpainting, direct comparisons may lead to unfairness.
		○ Using generative methods may harm pixel-level metrics such as PSNR and SSIM while benefiting perceptual metrics like FID, which is usually related to the weighting of loss functions. The authors should conduct relevant experiments to address this
paper_summary: paper_summary: 1. This paper proposes a novel EM image inpainting framework to recover damaged regions in serial sections.
2. This paper designs a two-stage reference image generation strategy,  to estimate biological structures in the inpainted regions in a multi-scale manner.
3. This paper adopts a GAN-based image inpainting sub-network to generate final results.
4. This paper conducts comprehensive experiments based on the CREMI datasets.
strength_weakness: strength_weakness: 1. This paper proposes a new idea to recover damaged regions in ssEM data, while considering both 3D structural continuity and 2D image structural consistency.
2. This paper's design is easy to understand, and details have been discussed comprehensively.
3. This paper's writing and figures are relatively clear.
limitations: limitations: 1. Relevance to multimedia is not clearly stated. I think the paper about electron microscopy image processing may be more suitable for MICCAI or ISBI conferences.
2. I think serial section image inpainting is something similar to video inpainting, but this paper misses the related discussion, can you describe the (dis)similarity between ssEM image inpainting and video inpainting? And can recent advanced video inpainting methods (such as "Inertia-guided flow completion and style fusion for video inpainting") be utilized to solve this problem?
3. This paper holds an assumption "we assume that the biological structures are transformed smoothly along the axial direction", but in real cases, there are many unregistered serial section EM datasets. Recent papers (such as "Fast and Accurate Electron Microscopy Image Registration with 3D Convolution") have discussed the learning-based registration methods for the ssEM dataset, and I think you should add the discussion with EM registration methods. In other words, you should consider the pre-registration operation before your methods, or you just combine the registration with your methods for the real cases. Otherwise, your method is very limited.
4. All your experiments are conducted on the CREMI datasets, but I think CREMI datasets are not enough. On the one hand, CREMI datasets do not involve unregistered cases, on the other hand, CREMI datasets do not have diverse corrupted distortions compared with the real cases. So I think you should add more experiments from other real ssEM datasets, which only contain the corruptions but do not have the ground-truth image, and you can give the visualization results for the comparison.
By the way, the experiments about "the isotropic EPFL dataset" mentioned in the paper do not seem to appear in the paper.
5. The PWC-Net optical flow structure adopted in the paper is not advanced now, can other advanced optical flow structures bring more improvements?
6. How about the generalization ability of your method, such as the trained model from CREMI dataset can directly apply to other (different) EM datasets?
paper_summary: paper_summary: The authors propose a deep-learning based vEM image inpainting framework named FlowInpaint to recover the missing information at the damaged regions under the guidance of two neighboring sections, which considers both 3D structural continuity and 2D image structural consistency. The framework applies a two-stage reference image generation strategy to estimate plausible biological structures in the inpainted regions from coarse to fine. Experiments results on the public CREMI datasets show the superiority of the proposed method among other existing methods.
strength_weakness: strength_weakness: The framework applies a two-stage reference image generation strategy to estimate plausible biological structures in the inpainted regions from coarse to fine. The initial reference is interpolated between neighboring sections to preserve the axial 3D continuity. The subsequent refined reference module warps it to fit the realistic structures in the undamaged areas. A GAN-based image inpainting sub-network is implemented to generate final seamless results.
Experiments results on the public CREMI datasets show the superiority of the proposed method among other existing methods.
limitations: limitations: 1. Missing some necessary experiments.
e.g., 
a) Have the authors ever tested the robustness of the proposed method when the distance between adjacent images is larger (e.g., 80nm?).
b) Have the authors tested the efficiency of the proposed method on much larger sections?
c) What if two adjacent images have artifacts at the same area?
d) The performance of the proposed method using different number of levels k is not tested.

2. Some details about the proposed method are not clearly described
a) What is the isotropic EPFL dataset mentioned in Line 670?
b) How to choose the number of levels (k)?
paper_summary: paper_summary: This paper presents an optical flow-based serial image inpainting approach, which is able to recover the missing biological structures caused by section damage during the vEM imaging of three-dimensional (3D) cellular tissues. Specifically, a two-stage reference generation strategy is proposed to predict the intermediate state image from coarse to fine. Then, a GAN-based inpainting network is used to reconstruct the final structures. Experiments are conducted on  three CREMI datasets and demonstrate the superiority of the proposed method over existing inpainting models by evaluating PSNR, SSIM, FSIM, and FID.
strength_weakness: strength_weakness: 1. The authors introduce a two-stage approach to recover damaged regions in serial sections, considering both 3D structural continuity and 2D image structural consistency. Incorporating optical flow estimation to generate intermediate images and to ultimately recover 3D image structures is an interesting design.
2. The proposed FlowInpaint exhibits superior performance in serial section inpainting compared to existing methods. Extensive ablation studies and analysis are also conducted that validates the effectiveness of different modules.
limitations: limitations: 1. Please clarify the fairness of comparisons.
In the evaluation of experimental outcomes, traditional image inpainting methods have predominantly served as the baselines for comparison. Given the disparate task settings and potential inconsistencies between training inputs and outputs.
It is imperative that the authors provide a more detailed exposition of these comparison methods, particularly in their adaptation to biological image datasets.
2. Please explain the adaptation of optical flow estimation to the domain of biological images
The manuscript lacks a thorough explanation of the adaptation of optical flow estimation to the domain of biological images, despite its common usage in video media. As a cross-domain application, it's important for the author to meticulously consider the correlation and distinctions between successive frames in videos and the upper and lower images of 3D cellular structures. A more detailed analysis is necessary to elucidate these aspects and validate the applicability of the proposed approach.
3. Please clarify the difference to related articles.
This paper needs to explain the difference and improvement between an optical flow-based method raised by Gonzalez-Ruiz and PWC-Net.