Title: SAM2Flow: Interactive Optical Flow Estimation with Dual Memory for in vivo Microcirculation Analysis
Venue: NeurIPS 2025
Accepted: yes
Keywords: Optical Flow Estimation

Microscopic Blood Cell Analysis

Segment Anything Model
Abstract: Analysis of noninvasive microvascular blood flow can improve the diagnosis, prognosis, and management of many medical conditions, including cardiovascular, peripheral vascular, and sickle cell disease. This paper introduces SAM2Flow, an interactive optical flow estimation model to analyze long Oblique Back-illumination Microscopy (OBM) videos of *in vivo* microvascular flow. Inspired by the Segment Anything Model (SAM2), SAM2Flow enables users to specify regions of interest through user prompts for focused flow estimation. 
SAM2Flow also incorporates a dual memory attention mechanism, comprising both motion and context memory, to achieve efficient and stable flow estimations over extended video sequences. According to our experiments, SAM2Flow achieves SOTA accuracy in foreground optical flow estimation on both microvascular flow and public datasets, with a fast inference speed of over $20$ fps on $512\times512$ inputs. Based on the temporally robust flow estimation, SAM2Flow demonstrated superior performance in downstream physiological applications compared to existing models. The code and dataset will be published with this paper.


Peer reviews:
comments: SUMMARY:
This paper proposed a SAM-enhanced model to estimate optical flows of blood in microcirculation analysis. The key contributions are the dataset and dual-memory modules (motion and context memories). On one in-house dataset and another public dataset, the performance is good to estimate the optical flows.

STRENGTHS AND WEAKNESSES:
Strengths:

1. The studied problem is important, and sharing the dataset is a good contribution.

2. Extensive experiments have shown that the model achieves good performance.


Weaknesses:

1. The NO-ROI setting can achieve better performance in Table 3. More justifications are needed.

2. [50] can achieve comparable and even better performance. The authors are suggested to clearly discuss its superiority and contributions.

3. More prompts can be explored to validate its generalizability.

4. The authors are suggested to analyze failure cases, especially for some subtle objects which cannot be segmented well by SAM2.

QUESTIONS:
1. Please include the complexity comparisons in the tables.

2. How is the performance when using bboxes as prompts? Here, the point prompts are a bit time-consuming.

3. Discussion of failure cases

LIMITATIONS:
There is a limitation discussion about the imperfect prompts in the model. No failure cases.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-06-16

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SAM2Flow, an interactive optical flow estimation model designed for analyzing long in vivo microvascular flow videos captured by Oblique Back-illumination Microscopy (OBM). Inspired by the Segment Anything Model (SAM2), the model enables users to specify regions of interest (ROIs) through prompts, focusing flow estimation on targeted vascular segments. A key innovation is the dual memory attention mechanism, combining motion and context memory, which ensures stable and efficient flow estimation over extended video sequences. SAM2Flow integrates a feature encoder from SEA-RAFT and a context encoder from SAM2, adapted via an adapter to the OBM domain, and employs ROI-guided correlation lookup to optimize computational efficiency. The model is designed to address the limitations of existing methods in handling long sequences and complex vascular structures. Experimental results demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\n[Strengths]\n\n1. This paper proposes a specialized dataset tailored for optical flow estimation in in vivo microvascular studies, comprising 75 long-form videos and 306,800 paired ground-truth flow. The dataset can support downstream research in this area.\n2. This paper Introduces interactive ROI-guided flow estimation via user prompts, enhancing clinical efficiency by focusing on relevant vascular regions and reducing unnecessary calculations.\n3. The proposed methods employs a dual memory module with motion and context memories, improving long-sequence flow estimation stability for sustained microvascular dynamics.\n\n\n[Weaknesses]\n\nThe primary limitation of the proposed method is its dependency on the context branch for ROI (region of interest) detection to guide flow estimation. In complex scenarios with intersecting vascular structures or turbulent blood flow, failed or incomplete ROI detection can directly degrade the accuracy of flow estimation.\n\nQUESTIONS:\n[Questions]\n\nOverall, the method proposed in this paper is novel, but there are still some confusing minor issues:\n\n1. How reliable is the ROI detection in the proposed method?\n2. When ROI detection is not fully reliable, how does it affect the estimation accuracy of the proposed method?\n3. For the Spring dataset, how are foreground regions defined and generated? Could you provide a more detailed explanation?\n4. In the studied scenarios, vascular overlapping in images seems to exist. How does the proposed method perform in such scenarios? Can the proposed strategies handle such scenarios effectively?\n5. Open discussion: The paper mentions multiple microvascular imaging techniques, but it mainly focuses on OBM. Does the proposed method have generalizability to other microvascular imaging techniques? If its generalizability is limited, what are the reasons?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a noel approach for optic flow estimation. Uniquely to the work, it focuses primarily on in vivo microcirculation analysis. Given a collection of images, the paper allows the user to select which area needs to have its optic flow computed via a prompting mechanism similar to the one in SAM. The rest of the work follows a relatively standard flow estimation pipeline, with the addition of a memory based mechanism for improving results on long video sequences.\n\nSTRENGTHS AND WEAKNESSES:\nPositive:\n- The paper is fairly well written and easy to understand.\n- I am not an expert in the field, but according to the authors this is one of the first approaches to produce optic flow for this type application.\n- The authors also introduce a novel dataset specific to microcirculation analysis.\n- I very much appreciate that the authors included an application specific result section, commenting on the usefulness of the estimated flow. \n- The paper also appears to produce good results for a more standard (i.e. non-medical) dataset.\n\nNegative:\n- The method section of the paper reads somewhat disconnected from the introduction and experimental section. The latter sections very much link to the target application domain, whereas the former seems quite generic i.e. not specific to microcirculation flow estimation.\n- I am not an expert in the field so do not know if the methods evaluated are adequate for the specific task.\n\nQUESTIONS:\nOverall I do not have significant suggestions for improvement, as I am very much a non expert in the specific application domain. From a pure CV optic flow estimation paper, the paper is not overwhelmingly novel, using concepts from SAM and relatively standard optic flow formulations. The application and dataset however appears to bring sufficient novelty to warrant publication.\n\nIf I had one request, it would be to anchor the design of the approach a bit more into the specific application. The main mention of multivascular flow estimation within the method appears to be on line 227, and this could be expanded.\n\nLIMITATIONS:\nYes, these are discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
For microvascular blood flow, this work proposes an optical flow estimation method based on SAM2 to confirm the RoI (Region of Interest) area according to user prompts, and introduces a new microvascular blood flow dataset. The method achieves leading results on both standard optical flow benchmarks and this dataset.

STRENGTHS AND WEAKNESSES:
Strengths:  

1. The method proposes an effective optical flow estimation approach for the microvascular blood flow task.  

2. It provides a novel microvascular blood flow dataset.  

3. The paper features good visualizations for demonstration.  


Weaknesses:  
1. Lack of comparison with more state-of-the-art optical flow estimation methods, such as the SAM-based method SAMFlow [1] and the multi-frame method StreamFlow [2].

2. Absence of results on more standard optical flow benchmarks (e.g., Sintel). Although the task focuses on microvascular blood flow, since there was no large-scale dataset for this task previously, experiments on more public benchmarks (whether fine-tuned results or 0-shot results) could more effectively validate the method’s effectiveness. After all, the method itself is not specifically designed for microvascular tasks, and the same architecture can be directly applied to optical flow estimation in other scenarios. More comparisons would better demonstrate its validity.

3. The experiments on the Spring dataset should present more metrics such as Fl.

[1] SAMFlow: Eliminating Any Fragmentation in Optical Flow with Segment Anything Model

[2] StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences

QUESTIONS:
Please refer to the weaknesses.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02