# STAVOS: A Medaka Larval Cardiac Video Segmentation Method Based on Deep Learning
###### Abstract
(2018), even based on SegNet, proposed a method named Cardiac Functional Imaging Network (CFIN), capable of automatically assessing cardiac volume. Regardless of whether it is transgenic cultivation or fluorescence labeling, both methods significantly enhance the segmentation effectiveness of the ventricle. However, these approaches still require a substantial number of biological experiments, deviating from the original intention of achieving automation. Therefore, researchers in this field eagerly hope to find an approach that demands the least additional manual operations, truly realizing automation. Although the ventricular image data of medaka fish has various limitations, cardiac pulsation is a continuous process, and the ventricle undergoes significant changes in volume and shape during pulsation, exhibiting considerable consistency between consecutive frames. To leverage visual information between adjacent frames, this study adopts the Video Object Segmentation (VOS) method (2018). The purpose of VOS is to maintain pixel-level tracking of a specific target object throughout the entire given video, and there is a significant demand for datasets in this field. However, there is no open and authoritative dataset for medaka larval ventricular beating. Therefore, we utilized pixel-level annotation tools to annotate the video data of medaka larval heartbeats, creating a novel dataset specifically designed for VOS tasks. Due to the issues of occlusion and severe background interference in medaka ventricular video images, our model is based on the state-of-the-art video segmentation model TBD (Tackling Background Distraction) (2018). Additionally, to address the problem of edge blurring, we propose a spatiotemporal attention module for video object segmentation (STAVOS), which better captures the dynamic features of heartbeats in the video, achieving precise segmentation of the medaka ventricle. Moreover, we have designed an automated system. By providing a video capturing the heartbeat of medaka larvae and a mask for one frame, the system can perform ventricular segmentation. Based on the precise segmentation results from STAVOS, the system automates the computation of various cardiac parameters of the medaka ventricle, such as heart rate (HR), stroke volume (SV), fractional shortening (FS), ejection fraction (EF), etc. Some of these parameters will undergo visualization to facilitate further research by relevant biologists into medaka. The main contributions of the work in this paper are as follows:  
1. For the first time, a medaka ventricular video image dataset was established, which was created in the format of a VOS task dataset with over 7000 labeled images. It can be used for precise segmentation of the medaka ventricle and provides an effective basis for calculating ventricular parameters. 2. We propose a spatiotemporal attention module for VOS, aiming to address the issue of edge blurring in the segmentation of moving targets in videos and thereby enhance the performance of semi-supervised VOS models. 3. We developed a deep learning-based automated system that achieves precise segmentation of the medaka ventricle. This system is capable of computing various parameters for the medaka heart and producing visualizations for some of these parameters.