P-197 Successful implementation of an end-to-end artificial intelligence (AI) platform in a busy IVF clinic: A prospective observational study
Authors: Papatheodorou A , Gilboa D , Seidman D , Oraiopoulou C , Karagianni M , Papadopoulou M.I , Tsarfati M , Kubany A , Christoforidis N , Chatziparasidou A 
Published: June 2022

Abstract

Study question
How demanding in terms of time and resource allocation is the full integration of an AI platform to the routine operation of an IVF clinic?

Summary answer
The rapid and effective implementation, and continuous performance, of EMA was qualitatively and quantitatively demonstrated for the first time in a real-world clinical setting.

What is known already
The role of AI-based embryo predictive analysis tools is often hailed as one of the most important recent developments in the IVF clinic. The high precision of such systems has been reported. For instance, the AI-system implemented here, EMATM(AiVF), employs a convolutional neural network architecture, providing an area-under-the-curve (AUC) of 0.95 with 83% accuracy. However, little was reported to date on the full implementation of these advanced systems in the active IVF lab. This study prospectively evaluated the clinical implementation and process-of-use in an IVF clinic of EMA, the first end-to-end AI-driven platform designed to algorithmically aid in evaluating embryos.

Study design, size, duration
A prospective observational single center study. The study was performed in two phases. Phase I: EMA was integrated into standard workflow and qualitatively evaluated over the course of one month by five embryologists in a series of twice-daily qualitative checks and questionaries. Phase II: The rate of agreement between EMA and embryologists were benchmarked and compared to evaluate how the model aids embryologists in efficiently assessing embryos.

Participants/materials, setting, methods
Phase I: Five senior embryologists completed electronic questionaries to qualitatively report on the ease-of-use, functionality, and performance of EMA after using the platform as adjunctive information on 588 embryos; ICSI was performed on all treatment cycles. Phase I was completed within 2 weeks. Phase II: The rate of agreement between five senior embryologists and EMA was calculated for the accuracy in ranking embryo(s) for transfer/freeze (146 treatment cycles). Phase II was undertaken in 4 weeks.

Main results and the role of chance
EMA was effectively incorporated into a busy IVF laboratory for routine daily use to algorithmically assess all embryos at 105 hours post-fertilization prior to vitrification. All embryos were cultured in a time-lapse incubator and successfully evaluated by both EMA and embryologists in parallel to conventional morphologic embryo evaluation. In Phase I, all five embryologists qualitatively approved of EMA’s integration and clinical utility inside their workflow and reported enhanced efficiency when EMA was used per its intended use. In Phase II we demonstrated a 86% agreement rate between embryologists and EMA. Of all embryos selected for transfer by embryologists 100% were also identified by EMA as having the highest potential for implantation. Of all embryos selected for vitrification by embryologists, 85%were identified by EMA as top-quality (Gardner criteria: A/B grade) embryos. Among embryos that were graded C/D (Gardner criteria) by embryologists, 89% were identified by EMA as “low grade” as well. Pregnancies were shown to be highly associated with EMA's embryo selection. The final stage of our implementation analysis of EMA is currently ongoing; the association between the algorithmic outputs of EMA and clinical implantation rates are being investigated in a prospective double-blinded, observation cohort study and results will be presented.

Limitations, reasons for caution
This is a single center study, based on relatively homogenous patient population. Nevertheless, given the impressive results reported herein, we conclude that this single case-study is sufficient for demonstrating rapid and successful implementation and process-validation of EMA for routine use by the clinic.

Wider implications of the findings
AI-based decision support systems like EMA have the potential to increase rapid and objective standardization inside the clinic, thereby improving accurate decision making and saving time and resources without interfering with the busy workflow of an IVF setting. Routine use of EMA in IVF should be prioritized for further evaluation.

Trial registration number
N/A