<figure><img src="image_1.png" /></figure>

HAL
open science

# echOpen-Cardio: Low-cost Open POCUS & Robust AI for Cardiac Image Analysis

Yingyu Yang, Mehdi Benchoufi, Pamela Moceri, Maxime Sermesant

► To cite this version:

Yingyu Yang, Mehdi Benchoufi, Pamela Moceri, Maxime Sermesant. echOpen-Cardio: Low-cost Open POCUS & Robust AI for Cardiac Image Analysis. MICCAI FAIR workshop, Sep 2021, Strasbourg (100% virtuel), France. hal-04187523

HAL Id: hal-04187523

https://hal.science/hal-04187523v1

Submitted on 18 Oct 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
echOpen-Cardio: Low-cost Open POCUS &
Robust AI for Cardiac Image Analysis

Yingyu Yang<fnref n="1" />, Mehdi Benchoufi<fnref n="2" />,<fnref n="3" />,<fnref n="4" />, Pamela Moceri<fnref n="1" />,<fnref n="5" />, and Maxime
Sermesant<fnref n="1" />

**Abstract.** In this paper, we are going to present the echOpen-Cardio project, which aims at developing low-cost open-source point-of-care ultrasound devices combined with robust artificial intelligence (AI) tools for cardiac disease diagnosis and therapy. We will first review the current market of ultrasound and the role of point of care ultrasound (POCUS), then present the echOpen project and its achievements by now. Finally, we will present an example of robust AI tools for cardiac ultrasound segmentation that leverage shape priors. This has great potential to be applied for the echOpen-Cardio platform.

# 1 Introduction

Medical ultrasound is known with its popularity, non-invasiveness and real-time imaging. The actual perspective of growth for the market between 2021 and 2023 is 30% per year in the impulsion of new drivers of the market. The key market trends are mobility, ultra-portability, point-of-care, development of new clinical segments, growing offer of low-cost handheld ultrasounds, critical role of user experience in the adoption of an emerging technology, integration with artificial intelligence (AI) technologies and the extension of handheld ultrasounds use from specialist toward generalist (general, internal medicine, urban primary care).

Among the new trends, handheld ultrasound, also know as POCUS (point-of-care ultrasound), allows better, faster and closer diagnosis to anyone, anywhere and at any moment (fig.1). Top-ranked reviews such as JAMA, LANCET also announce handheld ultrasounds as a key tool to complete the clinical examination, and expect it as the stethoscope of the 19th century or the fifth pillar of clinical examination. Its handheld characteristics open the path to operating the examination at patient's bedside. This can be reflected in the fact that the ultrasound market is shifting more toward emergency screening globally. Currently, 4% of emergency screenings use ultrasound for diagnosis. This figure is expected to increase to 20% by 2023. Cost-effective ultrasound focusing on automation and real-time and definitive diagnosis provide impetus for growth.

<footnote n="1">¹ Université Côte d'Azur, Inria Epione Team, Sophia Antipolis, France</footnote>

<footnote n="2">² Centre d'Epidémiologie Clinique, AP-HP, Paris, France</footnote>

<footnote n="3">³ Centre de Recherche Épidémiologie et Statistiques (CRESS-UMR1153) Université de Paris, Paris, France</footnote>

<footnote n="4">⁴ Inserm, Paris, France</footnote>

<footnote n="5">⁵ UR2CA, Université Côte d'Azur, Faculté de Médecine, Nice, France</footnote>
<table><thead><tr><th>From complex cart-based to handheld connected solution</th><th></th><th></th></tr></thead><tbody><tr><td>Healthcare professionals</td><td>Experts only</td><td>All HCP</td></tr><tr><td>Costs</td><td>$$$</td><td>Affordable</td></tr><tr><td>Ability to share</td><td>Physical support</td><td>Cloud</td></tr><tr><td>Usability</td><td>Heavy, complex</td><td>Handheld, easy</td></tr></tbody></table>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 1: The comparison between traditional ultrasound and POCUS devices. (copyright: echOpen)</figcaption></figure>

In addition, the advent of AI-driven algorithms endows POCUS with the right advantages to be one of the key technology benefiting of AI impact potential: namely improving patient care, facilitating patient journey and more prospectively deploying the technology as a common good in the hand of the patients themselves. Integrating AI solutions will simplify the adoption of the solution by simplifying the training needed to deploy usage at scale.

<figure><img src="image_5.png" /><figcaption>Fig. 2: The 2-dimensional scatter of current POCUS market in terms of mobility and affordability. (copyright: echOpen)</figcaption></figure>

There are already many market players who are trying to develop POCUS for efficient diagnosis (see fig.2). However, their high cost may be a burden for public use at large scale. The need of echOpen device enhanced by AI, which is open-source and low-cost, will increase in areas such as nursing home for elderly, health centres, at home and performed by non medical doctors such as nurses or even social workers especially in a under-served area.
## 2 EchOpen Project

EchOpen has been launched as a community project from the year 2014. Up to now, it has achieved the goal of a fully industrialised device, respecting the main target of affordability.

- One prototype displaying medical quality ultrasound images on smartphones (iOS, Android) and fully documented under open licence

- Off-line echo-stethoscope 48h training module

CE mark is expected by the upcoming months. Corresponding hardware and software materials are released on Github<fnref n="6" /> for communities to invest in this innovation space. echOpen fixed an interesting ecosystem, which is the hybrid fruit of formal and informal structures:

- Medical, Assistance Publique des Hôpitaux de Paris is endowed with one of the largest data warehouse in the world, top level academics as Inria, EPFL with advanced skills in AI, large foundations as Foundation Sanofi ou Foundation Pierre Fabre promoting development of affordable POCUS in under-developed countries, international industrial groups as Altran Cap Gemini, in charge with regulatory, market access and deployment.

- At the grass-root level: a core community of thirty healthcare professionals, engineers, makers, designers working at developing an affordable technology along with open source materials aiming to accelerate innovation in the acoustics technology field. echOpen foundation animates a community of hundreds of people supporting the project.

The shared vision of this ecosystem is working at universal access to ultrasound imaging by leveraging an affordable POCUS together with AI tools, not as standalone applications, but with the dedicated device to deploy them.

## 3 echOpen Cardio & AI: Robust Cardiac Segmentation

We developed deep learning tools for robust cardiac segmentation and demonstrated its performance on two large open-source ultrasound datasets: CAMUS<fnref n="7" /> and EchoNet<fnref n="8" />.

**CAMUS dataset** consists of 2D apical 4-chambers (A4C) and 2-chambers (A2C) echocardiographies of 500 patients and the manual annotations of cardiac structures (left endocardium, left epicardium and left atrium) acquired by expert cardiologists for each patient in each view, at end-diastole (ED) and end-systole (ES) [2]. In particular, the annotation of 450 patients are publicly accessible. The last 50 patients evaluation can be performed on the platform<fnref n="9" />.

<footnote n="6">⁶ https://github.com/echopen</footnote>

<footnote n="7">⁷ https://www.creatis.insa-lyon.fr/Challenge/camus/databases.html</footnote>

<footnote n="8">⁸ https://echonet.github.io/dynamic/</footnote>

<footnote n="9">⁹ http://camus.creatis.insa-lyon.fr/challenge/</footnote>
**EchoNet dataset** contains 10 030 2D apical 4-chambers (A4C) echocardiography videos as part of routine clinical care at Stanford University Hospital [3]. Segmentation measurements (left endocardium) at end-diastole and end-systole are available for all videos.

## 3.1 Method

We used an adapted U-Net model whose encoder consists of 5 downsampling (MaxPool + Conv) blocks with ReLU activation after the 3x3 convolution. The corresponding decoder has 5 upsampling (UpSample + Conv) blocks and is skip-connected with the encoder.

**Loss function:** in addition to the standard Dice loss:

$$ loss_{Dice} = 1 - \frac{2 \sum P * M}{\sum P + \sum M} \qquad (1) $$

where *P* is the predicted mask and *M* the ground truth mask, we also added a multi-class contour loss[1],

$$ loss_{contour} = \sum (D(M) \circ contour(B(P))) \qquad (2) $$

where *D(M)* is calculated from the ground truth mask and represents the shortest euclidean distance of each pixel to the closest ground-truth border, and *B(P)* represents a differentiable thresholded sigmoid for binarisation *B(P)* = 1/(1+exp[-(γ(P-T))]) with γ = 20 and T = 0.5. The contour of the binarised mask is extracted with a Sobel filter:

$$ contour(P) = |G_x * P| + |G_y * P| \qquad (3) $$

where *\* denotes 2D convolution and $G_x, G_y$ are 2D Sobel kernels in x- and y-dimensions:

$$ G_x = \begin{bmatrix} 1 & 0 & -1 \\ 2 & 0 & -2 \\ 1 & 0 & -1 \end{bmatrix}, G_y = \begin{bmatrix} 1 & 2 & 1 \\ 0 & 0 & 0 \\ -1 & -2 & -1 \end{bmatrix}. $$

The final loss function is

$$ loss = \alpha loss_{Dice} + \beta loss_{contour} \qquad (4) $$

**Model implementation:** we trained the model (implemented with Pytorch) using only the 450 training data of the CAMUS dataset (both 2-chambers and 4 chambers ED/ES frames). The 450 patients are randomly split into 10 folds, balanced with similar distribution of image quality and LVef distribution. Every turn, we used 8 folds data for training, 1 fold for validation and 1 fold for testing. An Adam optimiser was applied with a learning rate of 1e-3. The parameters of the loss function were set to $\alpha = 100, \beta = 1e-4$. Training was early stopped
when the Dice loss on validation data showed no increase for more than 5 epochs. We also applied data augmentation at training phase: random rotation, crop, brightness adjustment, contrast change, sharpening, blurring and speckle noise addition. Each of the augmentation technique is applied with a probability of 0.5 for every input image.

## 3.2 Results

We show the cross-validation results of 450 patients from CAMUS, the result of 50 patients (test data) from CAMUS, and that of 1277 patients (test data) from EchoNet (table.1). First, we can see that our AI model demonstrates very good performance on CAMUS dataset, which indicates that our model learns well the segmentation style of CAMUS. Second, compared with the Dice coefficient of the model trained on EchoNet data, which is 0.92% [3], our model which was trained on a totally different dataset showed a quite good generalisation performance on the same test data (0.895% Dice). What's more, based on the segmentation, we are able to extract useful cardiac indexes, such as ejection fraction.

## 3.3 Prospects

These encouraging results imply that our AI model generalises well to different image qualities and has the potential to be applied to images from echOpen, that demands a stable and robust algorithm for automatic image processing.

However, CAMUS and EchoNet datasets were collected from commercial ultrasound devices, which differ in image quality and frequency from the images acquired with the echOpen device. Appropriate transfer learning experiments will be conducted on echOpen cardiac images to take advantage of existing learned segmentation parameters.

<table><caption>Table 1: **Segmentation Metric.** Endo.: endocardium, Epi: epicardium, EF: ejection fraction, HD: Hausdorff distance, MSD: mean surface distance, MAE: mean absolute error, Corr.: correlation. HD and MSD are in mm unit For CAMUS dataset, and in pixels for EchoNet (256x256 size).</caption><thead><tr><th rowspan="2">Dataset</th><th rowspan="2">Dice</th><th colspan="2">Endo</th><th colspan="2">Epi</th><th colspan="2">EF(%)</th></tr><tr><th>HD (mm)</th><th>MSD (mm)</th><th>Dice</th><th>HD (mm)</th><th>MSD (mm)</th><th>MAE (%)</th><th>Corr.</th></tr></thead><tbody><tr><td>CAMUS (Train)</td><td>0.931</td><td>4.99</td><td>1.50</td><td>0.952</td><td>5.63</td><td>1.67</td><td>5.06</td><td>0.84</td><td>Bias(%)<br>± std</td></tr><tr><td>CAMUS (Test)</td><td>0.940</td><td>4.75</td><td>1.40</td><td>0.958</td><td>5.05</td><td>1.60</td><td>4.86</td><td>0.91</td><td>0.7<br>± 7.50</td></tr><tr><td>EchoNet (Test)</td><td>0.895<br>± 0.057</td><td>12.56<br>± 6.34</td><td>3.58<br>± 2.16</td><td>NaN</td><td>NaN</td><td>NaN</td><td>8.39</td><td>0.69</td><td>2<br>± 7.50<br>5.03<br>± 13.21</td></tr></tbody></table>
<figure><img src="image_6.png" /><figcaption>Fig. 3: **Segmentation examples** (good/medium/bad in terms of HD from left to right). First row: CAMUS examples. Second row: EchoNet examples. Colourful regions represent predicted masks and contours/marker lines represent the ground truth annotations.</figcaption></figure>

## 4 Conclusion

EchOpen is a prospective solution for fair and efficient medical imaging. Its application for cardiac imaging opens up possibilities in diagnosis and therapy planning. With 7 years of development, echOpen has started industrialisation and achieved encouraging cooperation results both in industry and academia. With the integration of such AI tools, we are aiming to propose automated workflows in order to enable a wider diffusion toward less expert users. The prospective application of the project is to use echopen device for signal acquisition, the images can be either treated locally with proper deployment of AI algorithm in the smartphone, or transferred to cloud cluster for computation to facilitate general usage especially in developing regions.

## References

1. Jia, S., Despinasse, A., et al.: Automatically segmenting the left atrium from cardiac images using successive 3d u-nets and a contour loss. In: Pop, M., Sermesant, M., et al. (eds.) Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges. pp. 221–229. Springer International Publishing, Cham (2019)

2. Leclerc, S., Smistad, E., et al.: Deep learning for segmentation using an open large-scale dataset in 2d echocardiography. IEEE Transactions on Medical Imaging 38(9), 2198–2210 (2019)

3. Ouyang, D., He, B., et al.: Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020)