# Surgical Phase Recognition: From Public Datasets to Real-World Data
## 4 Discussion  
In this work we thoroughly (i) analyzed the distribution of laparoscopic cholecystectomy video data recorded in a given medical center over time, and (ii) tested existing surgical phase recognition models on such video data. We introduced a new large-scale private dataset which consisted of 384 complete laparoscopic cholecystectomy videos. Unlike curated public datasets, our dataset contained all videos, including emergency surgeries and teaching cases, recorded in a continuous time frame of five years. Consequently, we observed strong differences with the most commonly used public dataset for phase recognition: Cholec80. In particular, we observed much longer videos and the presence of additional surgical phases such as _gallbladder drainage_, _liver biopsy_, and _intraoperative cholangography_. Note that our dataset was gathered from a single medical center. While we considered all videos being recorded over a given timeframe, the exact distribution of the videos (e.g., the presence of specific surgical phases) may differ across other medical centers. To our knowledge, there is no study detailing the distribution of phases observed across medical centers and surgeons. As usual in healthcare, we could not publicly share our dataset for legal reasons. We compensated for this lack of public access through an exhaustive analysis of our dataset and a thorough comparison with an existing public dataset. To our knowledge, such analysis of closer to real-world data distribution was lacking in the current literature on surgical phase recognition. Previous studies were performed on different surgery types (e.g., sleeve gastrectomy [24, 26, 27], right hemicotomy [27], and appendectomy [27]) and different scale of laparoscopic cholecystectomy datasets [13, 18]. The two latter studies [13, 18] used private datasets larger than Cholec80, which might have included the phases presented in this article. Nevertheless, the authors focused on the model performance without detailing the difference in data distribution observed between their dataset and public ones. Figure 7: Illustration of segmental performance metrics of the online TeCNO model on our dataset, on a video basis. (**a**) Frame-wise f1-score compared to segmental f1-score and edit score. The edit score is encoded by the color bar. F1@50 displays the individual performance in a wider range compared to the frame-wise f1-score. (**b**) Timeline plot of the ground truth (top) and predictions (bottom) for _video12_, illustrating the over-segmentation errors. Each phase is encoded by a different color explained in the legend. We studied how existing surgical phase detection models developed on Cholec80 performed when trained on our more complex dataset. We considered two different temporal model architectures out of numerous modeling approaches for surgical phase recognition [12]. Our aim was not to make an exhaustive comparison between many models. Instead, we explored the potential impact of video data distribution on model selection. We opted for MSTCN- [16; 24; 28] and transformer-based temporal models for analysis, since these state-of-the-art approaches for surgical phase recognition achieved good results on Cholec80 [15]. The performances of models trained on our dataset were lower than the performance reported on Cholec80, with a margin of 7% for accuracy, and almost 20% for precision and recall. This was expected given our higher number of phases and the higher complexity of our phase distribution. Comparing Trans-SVNet and MSTCN-based models on our dataset, the performance gap was not as significant as it was reported on the Cholec80 dataset [15]. For example, the gap between TeCNO and Trans-SVNet-WCE was less than 1% for accuracy, precision, and recall on our dataset (Table 3), compared to the 4% gap for precision and 2% gap for accuracy on Cholec80 [15]. This difference may be explained by the longer tail in our phase distribution compared to Cholec80 (Supplementary Figures S1 and S2).