# STAVOS: A Medaka Larval Cardiac Video Segmentation Method Based on Deep Learning
###### Abstract  
Medaka (_Oryzaias latipes_), as a crucial model organism in biomedical research, holds significant importance in fields such as cardiovascular diseases. Currently, the analysis of the medaka ventricle relies primarily on visual observation under a microscope, involving labor-intensive manual operations and visual assessments that are cumbersome and inefficient for biologists. Despite attempts by some scholars to employ machine learning methods, limited datasets and challenges posed by the blurred edges of the medaka ventricle have constrained research to relatively simple tasks such as ventricle localization and heart rate statistics, lacking precise segmentation of the medaka ventricle edges. To address these issues, we initially constructed a video object segmentation dataset comprising over 7000 microscopic images of medaka ventricles. Subsequently, we proposed a semi-supervised video object segmentation model named STAVOS, incorporating a spatial-temporal attention mechanism. Additionally, we developed an automated system capable of calculating various parameters and visualizing results for a medaka ventricle using the provided video. The experimental results demonstrate that STAVOS has successfully achieved precise segmentation of medaka ventricle contours. In comparison to the conventional U-Net model, where a mean accuracy improvement of 0.392 was achieved, our model demonstrates significant progress. Furthermore, when compared to the state-of-the-art Tackling Background Distraction (TBD) model, there is an additional enhancement of 0.038.  
video object segmentation; ventricular segmentation; cardiac parameters; medaka; deep learning +
Footnote †: journal: Medical Image Analysis  
the periodic bright intensity variations in video frames associated with heartbeats and employed the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to cluster pixels based on \"pixel density\", calculating the heart rate. [PERSON] et al. (2018) designed a platform based on a resonant laser scanning confocal microscope for analyzing the heart rate of fish embryos. However, this equipment is quite expensive for some laboratories. Traditional computer vision methods can only reduce a small number of manual operations. Therefore, some scholars have begun to use deep learning methods for research. However, the research is mainly limited to heart rate estimation ([PERSON] and [PERSON], 2018) or the segmentation of the entire fish body ([PERSON] and [PERSON], 2018), and there has been little progress in the precise segmentation of internal organs such as the heart or ventricle. This is mainly due to the small size of medaka fish larvae, which require video capture through microscopy. After multiple layers of refraction, the video data exhibits low pixel density, resulting in severe dynamic blurring and prominent artifacts. Additionally, the medaka fish body is not entirely transparent, featuring various degrees of pigmentation that cause different levels of occlusion in the collected ventricle videos. Moreover, the heart atrium and ventricle share a high degree of morphological similarity, lacking clear boundaries, making the contour edges almost indistinguishable. These factors make the precise segmentation of the ventricle a significant challenge. In response to these challenges, there are currently two main solutions: The first approach involves using fluorescence labeling to enhance the visibility of the ventricle. For example, [PERSON] et al. (2018) improved the accuracy of heart segmentation through fluorescence microscopy techniques, and [PERSON] et al. (2018) achieved overall imaging of the heart using fluorescence labeling. The second approach is to enhance the morphological features of the heart through transgenic cultivation. For instance, [PERSON] et al. (2018) implemented automatic segmentation of the heart based on U-Net. [PERSON] et al.