# Echocardiographic view and feature selection for the estimation of the response to CRT

Alban Gallard, Elena Galli, Arnaud Hubert, Auriane Bidaut, Virginie Le Rolle, Otto Smiseth, Jens-Uwe Voigt, Erwan Donal, Alfredo I Hernández

<figure><img src="image_1.png" /></figure>

## To cite this version:

Alban Gallard, Elena Galli, Arnaud Hubert, Auriane Bidaut, Virginie Le Rolle, et al., Echocardiographic view and feature selection for the estimation of the response to CRT. PLoS ONE, 2021, 16(6), pp.e0252857. 10.1371/journal.pone.0252857. hal-03281735

HAL Id: hal-03281735

https://hal.science/hal-03281735

Submitted on 8 Jul 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RESEARCH ARTICLE

# Echocardiographic view and feature selection for the estimation of the response to CRT

Alban Gallard<sup>1</sup>, Elena Galli<sup>1*</sup>, Arnaud Hubert<sup>1</sup>, Auriane Bidaut<sup>1</sup>, Virginie Le Rolle<sup>1</sup>, Otto Smiseth<sup>2</sup>, Jens-Uwe Voigt<sup>3</sup>, Erwan Donal<sup>1**</sup>, Alfredo I. Hernández<sup>1‡</sup>

1 University of Rennes, CHU Rennes, Inserm, LTSI UMR 1099, Rennes, France, 2 Center for Cardiological Innovation and Department of Cardiology, Oslo University Hospital, Oslo, Norway, 3 Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium

© These authors contributed equally to this work.

‡ These authors also contributed equally to this work.

* erwan.donal@chu-rennes.fr

<figure><img src="image_3.png" /></figure>

## Abstract

Cardiac resynchronization therapy (CRT) is an implant-based therapy applied to patients with a specific heart failure (HF) profile. The identification of patients that may benefit from CRT is a challenging task and the application of current guidelines still induce a non-responder rate of about 30%. Several studies have shown that the assessment of left ventricular (LV) mechanics by speckle tracking echocardiography can provide useful information for CRT patient selection. A comprehensive evaluation of LV mechanics is normally performed using three different echocardioraphic views: 4, 3 or 2-chamber views. The aim of this study is to estimate the relative importance of strain-based features extracted from these three views, for the estimation of CRT response. Several features were extracted from the longitudinal strain curves of 130 patients and different methods of feature selection (out-of-bag random forest, wrapping and filtering) have been applied. Results show that more than 50% of the 20 most important features are calculated from the 4-chamber view. Although features from the 2- and 3-chamber views are less represented in the most important features, some of the former have been identified to provide complementary information. A thorough analysis and interpretation of the most informative features is also provided, as a first step towards the construction of a machine-learning chain for an improved selection of CRT candidates.

## Introduction

Cardiac resynchronization therapy (CRT) is one of the most effective therapies for patients suffering from heart failure with reduced ejection fraction. CRT leads to improved quality of life and significant reductions in heart failure hospitalization rates and all-cause mortality [1, 2]. 20 to 30% of heart failure patients with reduced ejection fraction show a left bundle branch block (LBBB). These patients might benefit from a CRT implantation, with a reverse remodeling of the left ventricle (LV), an improvement in systolic function, a better control of symptoms and, finally, an improved quality of life and life expectancy [3]. Despite its

<figure><img src="image_4.png" /></figure>

### OPEN ACCESS

**Citation:** Gallard A, Galli E, Hubert A, Bidaut A, Le Rolle V, Smiseth O, et al. (2021) Echocardiographic view and feature selection for the estimation of the response to CRT. PLoS ONE 16(6): e0252857.
https://doi.org/10.1371/journal.pone.0252857

**Editor:** Giuseppe Coppola, University Hospital
*Paolo Giaccone*, ITALY

**Received:** March 31, 2021

**Accepted:** May 25, 2021

**Published:** June 10, 2021

**Peer Review History:** PLOS recognizes the benefits of transparency in the peer review process; therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. The editorial history of this article is available here: https://doi.org/10.1371/journal.pone.0252857

**Copyright:** © 2021 Gallard et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data Availability Statement:** All relevant data are within the paper and its Supporting information files.

**Funding:** The author(s) received no specific funding for this work.
**Competing interests:** The authors have declared that no competing interests exist.

well-established clinical benefits and cost-effectiveness, it remains a widely underutilized treat-
ment option. The topic of “non-response” to CRT has received large research attention. How-
ever, recommendations remain unchanged during the past years, mainly based on cardiac
electrical activity markers, without the integration of novel markers related to the assessment
of cardiac mechanical dyssynchrony and electro-mechanical couplingss [4, 5].

To overcome this important issue, a variety of methods have been proposed. One solution is to improve CRT implantation [6]. The optimal implantation of the LV lead is mainly based on the possibility to get an appropriate vein and good stimulation parameters. Therefore, most of the gain we could expect from CRT is expected to be in the selection process. Tremendous efforts have been made and a best understanding of mechanical dyssynchrony and electro-mechanical coupling have been achieved this recent years using imaging tools that are much more robust than in the past [7, 8]. Another solution aims at improving the identification and characterization of patients that are probably good responders to CRT. Among many methods, some authors have suggested that the analysis of speckle tracking echocardiography could improve the selection of CRT candidates [9–11]. The aim is to better characterize the regional mechanical function of the LV through this technique. In particular, some markers derived from LV wall motion estimation (peaks and timings) have been associated with CRT response. The analysis of LV deformation (strain) has also shown to be useful to understand LV mechanics and to provide information for CRT response estimation [5, 12, 13].

The best practice to assess overall LV mechanical deformation by bidimensional echocardi-
ography is to acquire 3 different views (Fig 1) [14]: i) the 4 chambers view (4ch) which shows
the two atria and the two ventricles; ii) the 2 chambers view (2ch) only focused on the left

<figure><img src="image_5.png" /><figcaption>Fig 1. Bull's-eye representation of myocardial LV segments and their relation to the different echocardiographic views (4ch: 4-chamber, 2ch: 2-chamber, 3ch: apical long axis). In each label, the first letter corresponds to B: Basal, M: Mid, and A: Apical. The second letter correspond to S: Septal and L: Lateral for the 4ch, I: Inferior and A: Anterior for the 2ch, P: Posterior and As: Anteroseptal for the 3ch.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0252857.g001
atrium and the left ventricle and iii) the apical long axis view (3ch) focused on the left part of the heart with the aorta.

Data acquired using these views can be processed to extract a wide variety of features. Machine Learning (ML) methods can then be applied to process these features in order to estimate the response to CRT, as shown in a recent work from our group, that was initially limited to the 4ch view [15]. Other works have also applied ML to predict the response to CRT or outcomes after CRT implantation [16, 17]. But a critical step in this approach is an appropriate feature analysis study, which implies the estimation of the relative importance of each input feature to the global decision output. Moreover, this feature analysis may lead to a feature selection step in which the number of features used is reduced while maintaining an acceptable performance level. By reducing the dimension of the problem in such a way, the convergence and generalization properties of the ML method are improved, while minimizing overfitting [18, 19].

Concerning the estimation of CRT response, a recent work by Cikes et al. applied an unsupervised method based on Multiple Kernel Learning and K-means clustering to a set of 1106 HF patients [13]. After the clustering step, the authors identified four natural groups and characterized these groups through expert knowledge according to their phenotypes. This data mining approach allowed for the identification of two phenogroups presenting a higher proportion of some clinical characteristics that are known from the literature to be predictive of CRT response. In this work, we propose another view to the problem, directly focused on the quantitative analysis of myocardial deformation for the selection of the most informative echocardiographic views and features for the estimation of CRT response. To this end, we apply a supervised approach, based on the Random Forest (RF) method and focused on a set of advanced, quantitative features extracted from strain curves, using three different echocardiographic views.

# Methods

## Population

161 patients undergoing CRT according to current recommendations [20] were included in this multicenter study. Patients were enrolled at the University Hospital of Rennes, at the University Hospital of Oslo, and the Universitair Ziekenhuis Leuven. All patients underwent 2D- standard transthoracic echocardiography before CRT implantation and at 6-month follow-up. Because of the poor-quality of some strain signals and/or irregular heart rhythm, 31 patients were withdrawn from the study, so the final study population was composed of 130 patients. The study was reviewed by an independent ethics committee “Ouest V” ethical committee validation number: 35RC14-9767) and conducted in accordance with the ‘Good Clinical Practice’ Guidelines in the Declaration of Helsinki. All patients provided written informed consent.

Each patient had the implantation performed during the month following echocardiography. When required, patients received an implantable cardiac defibrillator. Responders were defined as having a > 15% decrease in LV end-systolic volume at the 6-month follow-up, as compared with baseline [21, 22].

## Echocardiographic study

All patients had a complete baseline echocardiography before implantation (GE, Vingmed System 7, Ve9, Ve95, Horten, Norway) equipped with a 3S or M5S 3.5-mHz transducer. Two-dimensional, color Doppler, pulsed-wave, and continuous-wave Doppler data were stored on a dedicated workstation (BT12-EchoPAC PC V202.0.0, GE Healthcare, Horten, Norway) and
analyzed according to current guidelines [8] by a certified senior echocardiographer. LV volumes and ejection fraction were calculated using the biplane modified Simpson method. Systemic ejection time was measured by recording aortic flow with pulsed-wave Doppler imaging from the QRS onset to the aortic valve closure.

In order to obtain LV strain curves, two-dimensional gray-scale images were acquired in the standard apical 4-chamber (4ch), 3-chamber (3ch), and 2-chamber (2ch) views, at a frame rate of at least 60 frames/s. Fig 1 shows a schematic representation of such 3 views, as well as the corresponding position and labeling of the 18 myocardial segments. For each view, offline analysis was performed using the software pack described before. A line was traced along the endocardium's inner border in each of the three apical views on an end-systolic frame, and a region of interest was automatically defined between the endocardial and epicardial borders, with global longitudinal strain (GLS) then automatically calculated from the strain in the three apical views [23].

Close attention was paid to the placement of timing markers (onset of the QRS and aortic valve closure), as previously described by our group [12]. The calculated longitudinal strain signals for each segment were exported from the BT10-EchoPAC software. Each file is composed of longitudinal strain time series corresponding to 6 myocardial segments. These files were processed through a custom-made Python script in order to extract a set of features for further processing.

## Extracted features

Several features were extracted from the strain time series, as depicted in Fig 2. The process of obtaining these features is completely automatic, after the application of a standard cardiac strain study. The first phase consists in obtaining the strain curves, which implies the manual segmentation of the LV with the help of BT10-EchoPAC software. Strain curves are automatically calculated and exported. In our method, we also require a manual verification of the instant of aortic valve closure, which is classically performed in echo analyses. The calculation of the strain features from the obtained strain curves is performed in a completely automatic fashion by a custom Python program developed in our team. This automatic feature extraction aspect increases the reproducibility of the results, allows for a massive application on strain datasets and eases the future translation to the clinics.

In order to minimize the estimation error of these features, each strain curve was upsampled to 500 Hz. As performed in previous works, strain values between -5% and 5% were ignored from all calculations [12]. The onset of the QRS ($T_{QRS}$) is used as reference for the calculation of all features.

The first set of features is obtained from standard amplitude and time-domain analysis of the ECG and the available strain time series. Firstly, the aortic valve closure (AVC) instant ($T_{vw,avc}$) was manually annotated for each view vw. Fig 2A shows this feature as a black vertical line on top of the strain signals observed from an example 4ch view.

The peak strain value was automatically identified for each segment of each view. It corresponds to the maximum percentage of contraction and was termed $P_{vw}^{sg}$, for view vw and segment sg. The instant at which this peak strain value is identified is represented as $T_{vw,peak}^{sg}$. In Fig 2A, an example of these features for the Basal Septal (BS) segment is presented in red. The mean and standard deviation (Std) of the peak values and their corresponding time instants were calculated for each view and noted $P_{vw}^{Mean}$, $P_{vw}^{Std}$, $T_{vw,peak}^{Mean}$, and $T_{vw,peak}^{Std}$, respectively. The difference between the maximum and the minimum for the two types of features was also calculated
<figure><img src="image_6.png" /><figcaption>**Fig 2.** Feature extraction from longitudinal strain time series in an example case of a 4ch view. A) Longitudinal strain signals for all segments, observed from a 4ch view, showing standard amplitude and time-domain features extracted from these signals. Black vertical lines correspond to the manually identified onset of the QRS ($T_{QRS}$) and the aortic valve closure instant ($T_{4ch,avc}$). The color dots correspond to the peak strain value ($P_{4ch}^{sg}$) of each segment, arriving at time $T_{4ch,peak}^{sg}$. B) Calculation of three strain integral features from the longitudinal strain signal of segment BL of the 4ch view. C) Identification of the different segments on the 4ch view, showing how the sum of integrals for different segments are calculated (B: basal, M: Mid, and A: Apical, S: Septal and L: Lateral). Note that corresponding colors for each segment were used for panels A and C but these colors are different from those used in Fig 1.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0252857.g002

as follows (see Fig 2A):

$$ \Delta P_{vw} = \max_{sg}(P_{vw}^{sg}) - \min_{sg}(P_{vw}^{sg}) \qquad (1) $$

$$ \Delta T_{vw,peak} = \max_{sg}(T_{vw,peak}^{sg}) - \min_{sg}(T_{vw,peak}^{sg}) \qquad (2) $$

$\Delta P_{vw}$ is a marker of the heterogeneity of contraction between the segments of one view, while $\Delta T_{vw,peak}$ is a marker of the dyssynchrony between the segments.

The second set of features is based on the estimation of the integral of strain signals, as proposed in our previous works [12]. For each segment, different integrals of the negative part of the strain signal were calculated. Fig 2B shows an example of such extracted integral features for the Basal Lateral segment (BL) of a 4ch view. One integral feature ($I_{vw,avc}^{sg}$) is calculated from $T_{QRS}$ to $T_{vw,avc}$ (upper panel of Fig 2B). It represents the cumulative myocardial strain that is useful to eject blood. A second integral ($I_{vw,peak}^{sg}$) is calculated from $T_{QRS}$ to $T_{vw,peak}$ (middle panel of Fig 2B). It represents the total cumulative myocardial strain deployed by the segment. The third integral is calculated as follows:

$$ E_{vw}^{sg} = I_{vw,peak}^{sg} - I_{vw,avc}^{sg} \qquad (3) $$
and corresponds thus to the integral between the strain peak and the aortic valve closure (lower panel of Fig 2B). This is a marker of the mechanical efficiency of the segment. Positive values of this marker reflect an inefficient (or wasted) cumulative strain, acting after the closure of the aortic valve, when blood is no longer ejected.

This procedure was applied to all segments and to all views. Then, the mean and the standard deviation (Std) of these different integrals were calculated for each view: $I_{vw,avc}^{Mean}$, $I_{vw,avc}^{Std}$, $I_{vw,peak}^{Mean}$, $I_{vw,peak}^{Std}$, $E_{vw}^{Mean}$, and $E_{vw}^{Std}$.

The last set of features was based on sums of integrals [12]. For the two sides sd of each view and for the three different types of integrals, the sums of integrals of the three segments were calculated as follows:

$$I_{vw,avc}^{sd} = \sum_{sg \in sd} I_{vw,avc}^{sg} \qquad (4)$$

$$I_{vw,peak}^{sd} = \sum_{sg \in sd} I_{vw,peak}^{sg} \qquad (5)$$

$$E_{vw}^{sd} = \sum_{sg \in sd} E_{vw}^{sg} \qquad (6)$$

They represent the cumulative strain from the totality of side sd. The two possible opposing sides are S = Septal and L = Lateral for 4ch, I = Inferior and A = Anterior for 2ch, As = Anteroseptal and P = Posterior for 3ch. Fig 2C shows, as an example, the two opposing sides in the case of a 4ch view. Finally, the differences of the cumulative strain of the two sides were calculated as follows:

$$I_{vw,avc}^{sd1-sd2} = I_{vw,avc}^{sd1} - I_{vw,avc}^{sd2} \qquad (7)$$

$$I_{vw,peak}^{sd1-sd2} = I_{vw,peak}^{sd1} - I_{vw,peak}^{sd2} \qquad (8)$$

$$E_{vw}^{sd1-sd2} = E_{vw}^{sd1} - E_{vw}^{sd2} \qquad (9)$$

In addition to the above-mentioned features, two additional features that are manually identified by the clinicians have also been studied: the QRS duration and the LVEF. The final set of features extracted from data acquired from a given patient is thus constituted of 158 elements, represented in Table 1.

## Estimation of feature importance

The objective here is to analyze the relative importance of the extracted 158 features for the estimation of the response to CRT. Different methods have been applied and compared in order to estimate this feature importance:

**Random forest-based feature importance estimation.** Feature importance estimation can be performed within the ML chain by comparing the sensitivity of classification performance when applying the selected ML model, using different feature sets [24]. In this paper we used a ML approach based on random forests (RF), a versatile supervised ensemble learner method.

We have chosen to apply RF in this case for two main reasons. Firstly, RF can easily cope with unbalanced classes through a class weighting approach. Secondly, our dataset is of
<table><caption>Table 1. Number of features per view and in total.</caption><thead><tr><td>Feature</td><td>Number per view</td><td>Total number</td><td>Automatic or manual extraction</td></tr></thead><tbody><tr><td>T<sub>vw,avc</sub></td><td>1</td><td>3</td><td>Manual</td></tr><tr><td>P<sup>v</sup><sub>vw</sub></td><td>8</td><td>24</td><td>Automatic</td></tr><tr><td>T<sup>v</sup><sub>vw,peak</sub></td><td>8</td><td>24</td><td>Automatic</td></tr><tr><td>ΔP<sub>vw</sub></td><td>1</td><td>3</td><td>Automatic</td></tr><tr><td>ΔT<sub>vw,peak</sub></td><td>1</td><td>3</td><td>Automatic</td></tr><tr><td>I<sup>v</sup><sub>vw,avc</sub></td><td>11</td><td>33</td><td>Automatic</td></tr><tr><td>I<sup>v</sup><sub>vw,peak</sub></td><td>11</td><td>33</td><td>Automatic</td></tr><tr><td>E<sup>v</sup><sub>vw</sub></td><td>11</td><td>33</td><td>Automatic</td></tr><tr><td>QRS duration</td><td></td><td>1</td><td>Manual</td></tr><tr><td>LVEF</td><td></td><td>1</td><td>Manual</td></tr></tbody></table>

The last column shows if the subset is automatically or manually obtained. Symbols $v \in \{sg, Mean, Std\}$ and $\zeta \in \{sg, Mean, Std, sd1, sd2, sd1 - sd2\}$.

https://doi.org/10.1371/journal.pone.0252857.t001

relatively limited size. RF, with reduced degrees of freedom with respect to other ML methods, makes it an appropriate method for datasets such as the one studied in this work.

In this work, a random forest composed of *M* decision trees has been retained. Trees are configured for classification of each patient in one of two classes: “non-responder” or “responder” and the majority rule between trees is used for the final classification decision. All the available features from all patients were used as input to the RF-based feature importance analysis, which is based on an “out-of-bag” (OOB) approach (Fig 3). In OOB, a bootstrap

<figure><img src="image_7.png" /><figcaption>**Fig 3.** Out-of-bag process. The procedure inside the dotted rectangle is performed for each one of the *M* trees on the ensemble.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0252857.g003
procedure based on uniform random selection with replacement is applied to create a different learning set for each tree of the RF. Features are also chosen randomly for the construction of each tree. The data that was excluded from the learning set of each tree (out-of-bag data), is used to estimate the performance of the RF. Since the set of features used for the construction of each tree is different, due to the random selection process, the relative sensitivity of each feature to the final decision can also be estimated from this OOB approach.

**Correlation.** The linear correlation between each pair of features has been analyzed by estimating the covariance matrix. The absolute value of the correlation was analyzed for the most important features obtained from the previous analysis step. Only a correlation higher than 0.5 has been taken into account for the analysis.

**Wrapping and filtering methods.** As a complement to the OOB method described above, wrapper and filter methods [18] have also been applied. Filtering approaches use a ranking criterion to remove features below a suitable threshold. For example, the “correlation with the target” filter removes the features for which the correlation with the response to CRT is the lowest. Wrapper approaches evaluate the features’ performance to estimate the response to CRT.

We used five wrapping and filtering methods: i) correlation with target, ii) Welch’s t-test, iii) K-Best, iv) Recursive Feature Elimination (RFE), and v) Relief. Results from these five methods were combined to produce a single list of relative feature importance. The combination of these results, named CWF in this paper was performed as follows. We firstly created a list of features appearing in the 20 most important features of at least three wrapping or filtering methods. Then, for each feature in this list, the relative importance rank was calculated as the mean importance rank obtained for this feature in the 20 most important feature of the five wrapping and filtering methods. We compared the feature importance of the CWF method and the ML approach.

# Results

## Characteristics of the clinical data

Table 2 shows the baseline characteristics of the population included in the study. The number of women is expressed in number and in percentage according to CRT response. The other characteristics are given in mean and standard deviation. Continuous variables are compared using the Wilcoxon test, categorical data are compared by the $\chi^2$ test. A value of $p < 0.05$ was considered statistically significant.

## Feature importance

After a first stage of local sensitivity analysis, the random forest used for OOB feature importance analysis was configured with $M = 700$ trees, using the ‘gini’ criteria, with a maximum number of features per tree of 10 and unlimited tree depth [25, 26].

<table><caption>Table 2. Clinical features.</caption><thead><tr><td></td><td>Responders 83 (64%)</td><td>Non-Responders 47 (36%)</td><td>P Value</td></tr></thead><tbody><tr><td>Women, n (%)</td><td>29 (35%)</td><td>6 (13%)</td><td>0.036</td></tr><tr><td>Age (years)</td><td>67.1±10.5</td><td>65.6±12.3</td><td>0.576</td></tr><tr><td>LVEF (%)</td><td>29.3±7.2</td><td>28.4±9.7</td><td>0.434</td></tr><tr><td>QRS (ms)</td><td>168.4±19.2</td><td>165.4±22.0</td><td>0.293</td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0252857.t002
<figure><img src="image_8.png" /><figcaption>Fig 4. 20 most important features obtained from the OOB analysis (blue: 4ch, orange: 3ch, and green: 2ch).</figcaption></figure>

https://doi.org/10.1371/journal.pone.0252857.g004

Fig 4 shows the 20 most important features obtained from the OOB analysis. Results were normalized to have the most important feature equal to 1. Most of these significant features (12/20) are obtained from 4ch view, 5/20 from 2ch view, and 3/20 from 3ch view.

All data are available in S1 Data.

## Correlation

Fig 5 indicates the correlation between the 20 most important features. Two given features with an absolute correlation higher than 0.5 are linked by a line. The thickness of the line is calculated as $Thickness = 10*Correlation - 5$. Therefore, lines corresponding to a correlation equal or less than 0.5 will not be shown. A correlation of 1 will produce a line of thickness of 5 points.

## Wrapping/filtering methods and CWF approach

Table 3 shows the 20 most important features obtained from the OOB and CWF methods. In this table, the features are sorted by importance for each method. The features in bold are the common features from both methods. 10 of the 20 most important features of the OOB analysis are also present in the 20 most important ones of the CWF method.

## Discussion

The main objective of this work was to identify the most informative echocardiographic features and views for the estimation of CRT response in terms of LV remodeling. Different feature selection methods were applied and combined in order to obtain a quantitative
<figure><img src="image_9.png" /><figcaption>**Fig 5.** Correlation between the 20 most important features (blue: 4ch, orange: 3ch, and green: 2ch). The thickness of the line represents the importance of the correlation between the two features. Only correlation coefficients higher than 0.5 are displayed.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0252857.g005

estimation of the relative importance of each feature and view. To our knowledge, this is the first quantitative analysis of the contribution of multi-view echocardiographic features to the estimation of CRT response. This could be a step forward for CRT that remains under-used, partly because of the problem of lack of response of certain patients and the imperfect selection criteria that are recommended, up to now, by current guidelines [3].

<table><caption>**Table 3.** List of the 20 most important features estimated using OOB and CWF.</caption><thead><tr><th>Rank</th><th>OOB</th><th>CWF</th></tr></thead><tbody><tr><td>1 to 3</td><td>I<sup>BI</sup><sub>2ch,avc</sub>, I<sup>Std</sup><sub>2ch,avc</sub>, E<sup>BS</sup><sub>4ch</sub></td><td>I<sup>BI</sup><sub>2ch,avc</sub>, P<sup>AAs</sup><sub>3ch</sub>, E<sup>AA</sup><sub>2ch</sub></td></tr><tr><td>4 to 6</td><td>T<sup>AAs</sup><sub>3ch,peak</sub>, I<sup>Std</sup><sub>4ch,peak</sub>, T<sup>MS</sup><sub>4ch,peak</sub></td><td>T<sup>Mean</sup><sub>4ch,peak</sub>, E<sup>MS</sup><sub>4ch</sub>, E<sup>BS</sup><sub>4ch</sub></td></tr><tr><td>7 to 9</td><td>T<sup>MI</sup><sub>2ch,peak</sub>, I<sup>AS</sup><sub>4ch,peak</sub>, E<sup>AAs</sup><sub>3ch</sub></td><td>E<sup>Mean</sup><sub>2ch</sub>, P<sup>AS</sup><sub>4ch</sub>, T<sup>MI</sup><sub>2ch,peak</sub></td></tr><tr><td>10 to 12</td><td>P<sup>MAs</sup><sub>3ch</sub>, E<sup>AS</sup><sub>4ch</sub>, I<sup>BL</sup><sub>4ch,peak</sub></td><td>I<sup>AA</sup><sub>2ch,peak</sub>, I<sup>L</sup><sub>4ch,avc</sub>, I<sup>AS</sup><sub>4ch,avc</sub></td></tr><tr><td>13 to 15</td><td>E<sup>MS</sup><sub>4ch</sub>, E<sup>BL</sup><sub>4ch</sub>, T<sup>Mean</sup><sub>4ch,peak</sub></td><td>I<sup>L</sup><sub>4ch,peak</sub>, E<sup>BI</sup><sub>2ch</sub>, T<sup>AAs</sup><sub>3ch,peak</sub></td></tr><tr><td>16 to 18</td><td>I<sup>L</sup><sub>4ch,peak</sub>, I<sup>BL</sup><sub>4ch,avc</sub>, T<sup>BS</sup><sub>4ch,peak</sub></td><td>E<sup>L</sup><sub>2ch</sub>, E<sup>MA</sup><sub>2ch</sub>, P<sup>MAs</sup><sub>3ch</sub></td></tr><tr><td>19 and 20</td><td>E<sup>Mean</sup><sub>2ch</sub>, T<sub>2ch,avc</sub></td><td>T<sup>MS</sup><sub>4ch,peak</sub>, E<sup>MI</sup><sub>2ch</sub></td></tr></tbody></table>

Features are sorted by importance. Common features from both methods are marked in bold.

https://doi.org/10.1371/journal.pone.0252857.t003
This paper was focused on a data processing pipeline, based on the explicit extraction of a set of physiologically interpretable features, and the application of Random Forest as the underlying model. Another possible approach would have been to apply Deep Learning (DL) to the raw image data. We have clearly avoided such a DL approach because of several reasons [27, 28]. DL lacks interpretability and this aspect seems to us incompatible with the targeted clinical application. Moreover, DL, characterized by a particularly high number of degrees of freedom, is highly sensitive to noise, to overfitting and requires massive datasets. Our choice for a features-based approach was considered more adapted to this problem.

In the OOB analysis, the 4ch view provides the highest proportion of important features, with 60% of them in the 20 most important features. When we consider the 20 most important features, half of the uncorrelated features (3/6) are obtained from the 4ch view and we observe a low correlation between features within the 4ch view (between 0.5 and 0.6). Results from the CWF method also show many features from the 4ch view (40%). We can also notice that 50% of the significant features that have been selected through both feature selection methods are observed from the 4ch view. These results suggest that the 4ch view provides the highest pro- portion of information for the prediction of CRT response and should be considered as the main view to prioritize if only one view can be analyzed. Studies have already demonstrated that the value of 4ch view is of primary importance in this context [29].

Nevertheless, from the OOB analysis, 40% of the 20 most important features are obtained from the two other views. Features from the 2ch view represent 25% of the most important fea- tures and, moreover, the two most important features overall are obtained from the 2ch view ($I_{2ch,avc}^{BI}$ and $I_{2ch,avc}^{Std}$). However, these two features are correlated (0.63). The correlation analysis shows that only one feature of the 2ch view, $T_{2ch,peak}^{MI}$, is correlated with two other time domain features from the 4ch view. Features from the 2ch view are also very important according to the CWF method (45%). These facts show that the 2ch view can provide important comple- mentary information for the prediction of the CRT response and should be the second view to prioritize. The 3ch view shows the lowest proportion of important features (15%) and the selected features from this view were usually correlated with features from the 4ch view. The 3ch view can thus provide rather low additional information and should be considered as the lesser priority.

The 20 most important features from the OOB and CWF analyses include features of differ- ent subsets, mainly time-domain and strain integrals. Correlation often appears for features from the same subset and rarely between different subsets. The regional heterogeneity of the features extracted from strain integrals appears as a particularly useful marker for the predic- tion of CRT response. Almost every cardiac segment, except those from the 3ch, appear in at least one of the selected features.

In our analysis combining OOB and CWR results, we identified 10 main features that are particularly informative to predict CRT response. 8 of these features are extracted from the 4ch view, and 2 from the 2ch view. $I_{2ch,avc}^{BI}$, and $I_{2ch,avc}^{Std}$ obtained from the 2ch view are two most important features. $I_{2ch,avc}^{BI}$ represents a quantification of the cumulative strain developed by the infero-basal segment, which effectively contributes to LV ejection, whereas $I_{2ch,avc}^{Std}$ estimates the heterogeneity of the cumulative strain developed by the anterior and inferior LV wall during contraction. To our knowledge, the specific role of 2ch-derived features in the prediction of CRT response has never been described before. Nevertheless, the cumulative strain observed during LV contraction, which is an expression of myocardial viability, is known to be directly correlated to CRT response and survival after CRT [30, 31] and might contribute to explain the effect of $I_{2ch,avc}^{BI}$ on CRT efficacy. Also, no previous publication has addressed the role of opposite wall activation in 2ch view for the prediction of CRT response. Yu et al. have shown
that the standard deviation of opposite wall delay measured in short axis view, which includes
the assessment of the anterior and inferior segments, is an interesting marker to predict CRT
response [32]. The mechanical dyssynchrony of the anterior and posterior LV and the subse-
quent heterogeneity of strain signals can explain the importance of I<sup>Std</sup><sub>2ch,avc</sub> in CRT.

$E_{4ch}^{BS}$, $E_{4ch}^{AS}$ and $E_{4ch}^{MS}$ are respectively the third, the eleventh and the thirteenth most important features. They are the expression of the mechanical efficiency of the septal segments. In normal subjects, all LV segments contract almost simultaneously. In patients with left bundle branch block, which are typical candidates to CRT, the ventricular conduction delay causes the early activation of the septum and a delayed activation of the lateral wall. In these patients, the early septal activation is energetically inefficient because the LV pressure is still low at this early instant of the cardiac cycle and the aortic valve is still closed. Thus, this mechanical activation does not contribute to LV ejection [33]. Previous studies have shown that the septal wasted energy is increased in CRT candidates and is useful to predict CRT response [34, 35].

$I_{4ch,peak}^{AS}$ is the eighth most important feature in the OOB analysis. This feature represents the total strain deployed by the AS segment. According to Cikes et al. [13], strain is low in the apical region for non-responders to CRT. This makes sense, since AS is often one of the septal segments where the rebound stretch is the highest. In other words, this is one of the segments where the lack of efficiency of myocardial systolic strain is the greatest.

$I_{4ch,peak}^L$ is the sixteenth most important feature in the OOB classification and thirteenth in the CWF. In patients undergoing CRT, the delayed activation of the lateral wall lasts after aortic valve closure. This means that only a part of the cumulative strain developed by the lateral wall will contribute to LV ejection (Fig 2B). $I_{4ch,peak}^L$ corresponds to the global cumulative strain developed by the contraction of the lateral wall and is an indirect marker of lateral wall viability. Because the electrical stimulation of the lateral wall is a target of CRT therapy, the amount of mechanical energy which can be developed by this wall is known to be associated to CRT response [36, 37].

Other well-ranked 4ch-derived variables include $T_{4ch,peak}^{Mean}$, $T_{4ch,peak}^{MS}$ and $T_{4ch,peak}^{BS}$. $T_{4ch,peak}^{MS}$ and $T_{4ch,peak}^{BS}$ represent the time necessary to the maximal mechanical activation of the corresponding septal segments. The true LBBB mechanics causes a significant activation delay between the septal and the lateral wall. The presence of this typical pattern, with early septal activation and delayed lateral wall contraction [38] can be reversed by CRT and can be associated with significant reverse LV remodeling. It might explain the predictive role of septal activation timing. $T_{4ch,peak}^{Mean}$ is a measure of the global LV mechanical discoordination, which can be assessed in 4-chamber view. Previous studies have demonstrated that the heterogeneity of activation timings of opposite LV walls [39, 40], or the standard deviation of opposite walls delay are interesting predictors of CRT response. Nevertheless, these parameters are plagued by a low reproducibility when calculated manually and their utility has never been demonstrated in large multicenter trials, which can explain the lower predictive value of these variables [41].

From this discussion, it appears that a multivariate analysis of regional myocardial strain information is an added value for the prediction of the response to CRT. Cikes et al. have shown some similar results [13] through unsupervised machine learning. Although only qualitative strain analyses were performed in their work, they have pointed out the importance of the strain shape on segments AS, BS, and BI for the prediction of CRT outcome. In our study, quantitative strain analysis yields results supporting this fact, since features $I_{4ch,peak}^{AS}$, $E_{4ch}^{AS}$, $E_{4ch}^{BS}$, and $I_{2ch,peak}^{BI}$ are considered informative. We find the same results for the AA segment in the CWF method ($E_{2ch}^{AA}$) but not in the OOB method. Nevertheless, we did not find the same results for the AL segment, except in the sum of integrals ($I_{4ch,peak}^L$). More importantly, as described
above, we have identified a number of other particularly informative features that should be considered for the estimation of CRT response and the proposed method is based on a quantitative approach that may be easily included into a machine learning pipeline.

## Limitations

Despite its multicenter design, this study included a limited number of patients. Some of them had to be withdrawn because of atrial fibrillation or poor quality strain signals. The quality of images and the presence of sinus rhythm have a pivotal importance for the assessment of strain dynamics, but cannot be obtained for all patients in clinical practice.

Moreover, this study focuses on patients that are supposed to be implanted according to guidelines (I-A). Patients with irregular heart rhythm were excluded of this study. In a future work, these results could be extended to those patients.

A higher number of patients is necessary to study the performance of the RF to predict CRT response. The performance could be studied for the best features per view and for all views.

## Conclusion

In the present study, we applied different feature analysis methods to characterize a pool of quantitative features extracted from different views and methods from cardiac echocardiographic analyses. We found that most of the significant strain-derived features for the prediction of CRT response are obtained from the 4ch view. According to our data, the assessment of strain dynamics in 4ch view provide the most important information to predict CRT response. The 2ch view, might provide additional, complementary information on LV deformation. This analysis is a first step towards an improved development of multivariate machine learning methods for CRT prediction.

## Supporting information

**S1 Data. Data disclosure.** Each line corresponds to a different patient and each column to a feature.
(XLSX)

## Acknowledgments

The authors would like to thank Valerie Le Moal and Isabau Labrousse for their skillful assistance in the collection of data.

## Author Contributions

**Conceptualization:** Alban Gallard, Erwan Donal, Alfredo I. Hernández.

**Data curation:** Alban Gallard, Elena Galli, Arnaud Hubert, Auriane Bidaut, Virginie Le Rolle, Otto Smiseth, Jens-Uwe Voigt, Erwan Donal, Alfredo I. Hernández.

**Formal analysis:** Alban Gallard, Alfredo I. Hernández.

**Investigation:** Elena Galli, Arnaud Hubert, Auriane Bidaut, Otto Smiseth, Jens-Uwe Voigt, Erwan Donal.

**Methodology:** Alban Gallard, Virginie Le Rolle, Alfredo I. Hernández.

**Project administration:** Erwan Donal, Alfredo I. Hernández.
**Resources:** Elena Galli, Arnaud Hubert, Auriane Bidaut, Otto Smiseth, Jens-Uwe Voigt, Erwan Donal.

**Software:** Alban Gallard, Virginie Le Rolle, Alfredo I. Hernández.

**Supervision:** Erwan Donal, Alfredo I. Hernández.

**Validation:** Alban Gallard, Virginie Le Rolle, Erwan Donal, Alfredo I. Hernández.

**Visualization:** Alban Gallard.

**Writing – original draft:** Alban Gallard.

**Writing – review & editing:** Alban Gallard, Elena Galli, Arnaud Hubert, Erwan Donal, Alfredo I. Hernández.

## References

1. Bleumink GS, Knetsch AM, Sturkenboom MCJM, et al. Quantifying the heart failure epidemic: prevalence, incidence rate, lifetime risk and prognosis of heart failure The Rotterdam Study. Eur Heart J, 2004. 25 (18): 1614–9. https://doi.org/10.1016/j.ehj.2004.06.038 PMID: 15351160

2. Zareba W, Klein H, Cygankiewicz I, et al. Effectiveness of cardiac resynchronization therapy by QRS morphology in the multicenter automatic defibrillator implantation trial-cardiac resynchronization therapy (MADIT-CRT). Circulation, 2011. 123: 1061–72. https://doi.org/10.1161/CIRCULATIONAHA.110.960898 PMID: 21357819

3. Mullens W, Auricchio A, Martens P, Witte K, Cowie MR, Delgado V, et al. (2020), Optimized implementation of cardiac resynchronization therapy: a call for action for referral and optimization of care. Eur J Heart Fail, 22: 2349–2369. https://doi.org/10.1002/ejhf.2046 PMID: 33136300

4. Coats AJS, Pieske B, Linde C, Jankowska EA, Ruschitzka F, Rutten FH, et al, 2016 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: The Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC)Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. European Heart Journal, 2016. 37 (27): 2129–2200. PMID: 27206819

5. Lim P, Buakhamsri A, Popovic ZB, Greenberg NL, Patel D, Thomas JD, et al, Longitudinal Strain Delay Index by Speckle Tracking Imaging. Circulation, 2008. 118 (11): 1130–1137. https://doi.org/10.1161/CIRCULATIONAHA.107.750190 PMID: 18725491

6. Lecoq G, Leclercq C, Leray E, Crocq C, Alonso C, de Place C, et al. Clinical and electrocardiographic predictors of a positive response to cardiac resynchronization therapy in advanced heart failure. Eur Heart J. 2005 Jun; 26(11):1094–100. https://doi.org/10.1093/eurheartj/ehi146 PMID: 15728648

7. Lumens J, Vernooy K. Better understanding before implanting. International Journal of Cardiology. 2015 Apr; 184:6–8. https://doi.org/10.1016/j.ijcard.2015.01.090 PMID: 25705002

8. Morgan JM, Delgado V. Lead positioning for cardiac resynchronization therapy: techniques and priorities. Europace. 2009 Nov; 11 Suppl 5:v22–8. https://doi.org/10.1093/europace/eup306 PMID: 19861387

9. Carità P, Corrado E, Pontone G, Curnis A, Bontempi L, Novo G, et al. Non-responders to cardiac resynchronization therapy: Insights from multimodality imaging and electrocardiography. A brief review. Int J Cardiol. 2016 Dec 15; 225:402–407. https://doi.org/10.1016/j.ijcard.2016.09.037 PMID: 27776243

10. Bonadei I, Vizzardi E, Gorga E, Carubelli V, Pagnoni M, Sciatti E, et al. Role of the old and new echocardiographic technologies in cardiac resynchronization therapy. Minerva Cardioangiol. 2016 Oct; 64 (5):572–80. Epub 2015 Jun 23. PMID: 26099222

11. Gorcsan J 3rd, Oyenuga O, Habib PJ, Tanaka H, Adelstein EC, Hara H, et al. Relationship of echocardiographic dyssynchrony to long-term survival after cardiac resynchronization therapy. Circulation. 2010 Nov 9; 122(19):1910–8. https://doi.org/10.1161/CIRCULATIONAHA.110.954768 PMID: 20975000

12. Bernard A, Donal E, Leclercq C, Schnell F, Fournet M, Reynaud A, et al, Impact of Cardiac Resynchronization Therapy on Left Ventricular Mechanics: Understanding the Response through a New Quantitative Approach Based on Longitudinal Strain Integrals. Journal of the American Society of Echocardiography, 2015. 28 (6): 700–708. https://doi.org/10.1016/j.echo.2015.02.017 PMID: 25819341
13. Cikes M, Sanchez-Martinez S, Claggett B, Duchateau N, Piella G, Butakoff C, et al, Machine learning-based phenogrouping in heart failure to identify responders to cardiac resynchronization therapy. European Journal of Heart Failure, 2019. 21 (1): 74–85. https://doi.org/10.1002/ejhf.1333 PMID: 30328654

14. Lang RM, Badano LP, Mor-Avi V, Afilalo J, Armstrong A, Ernande L, et al, Recommendations for Cardiac Chamber Quantification by Echocardiography in Adults: An Update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. Journal of the American Society of Echocardiography, 2015. 28 (1): 1–39. https://doi.org/10.1016/j.echo.2014.10.003 PMID: 25559473

15. Donal E, Hubert A, Le Rolle V, Leclercq C, Martins R, Mabo P, et al, New Multiparametric Analysis of Cardiac Dyssynchrony: Machine Learning and Prediction of Response to CRT. JACC: Cardiovascular Imaging, 2019. PMID: 31005538

16. Hu SY, Santus E, Forsyth AW, Malhotra D, Haimson J, Chatterjee NA, et al, Can machine learning improve patient selection for cardiac resynchronization therapy?. PloS one, 2019. https://doi.org/10.1371/journal.pone.0222397 PMID: 31581234

17. Kalscheur MM, Kipp RT, Tattersall MC, Mei C, Buhr KA, DeMets D, et al, Machine Learning Algorithm Predicts Cardiac Resynchronization Therapy Outcomes: Lessons from the COMPANION Trial. Circ Arrhythm Electrophysiol, 2019.

18. Chandrashekar G, Sahin F, A survey on feature selection methods. Computers and Electrical Engineering, 2014. 40: 16–28. https://doi.org/10.1016/j.compeleceng.2013.11.024

19. Bron EE, Smits M, Niessen WJ, Klein S. Feature Selection Based on the SVM Weight Vector for Classification of Dementia. IEEE Journal of Biomedical and Health Informatics. 19(5): 1617–1626. https://doi.org/10.1109/JBHI.2015.2432832 PMID: 25974958

20. Vahanian A, Auricchio A, Hoes AW, Merkely B, Popescu BA, Israel CW, et al., 2010 Focused Update of ESC Guidelines on device therapy in heart failure: An update of the 2008 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure and the 2007 ESC guidelines for cardiac and resynchronization therapy Developed with the special contribution of the Heart Failure Association and the European Heart Rhythm Association. European Heart Journal, 2010. 31 (21): 2677–2687.

21. Doltra A, Bijnens B, Tolosana JM, Borràs R, Khatib M, Penela D, et al. Mechanical abnormalities detected with conventional echocardiography are associated with response and midterm survival in CRT. JACC Cardiovasc. Imaging. 2014; 7(10):969–979. https://doi.org/10.1016/j.jcmg.2014.03.022 PMID: 25240452

22. Rickard J, Baranowski B, Wilson Tang WH, Grimm RA, Niebauer M, Cantillion D, et al. Echocardiographic Predictors of Long-Term Survival in Patients Undergoing Cardiac Resynchronization Therapy: What Is the Optimal Metric? J. Cardiovasc. Electrophysiol. 2017; 28(4):410–415. https://doi.org/10.1111/jce.13175 PMID: 28176410

23. Voigt JU, Pedrizzetti G, Lysansky P, et al. Definitions for a common standard for 2D speckle tracking echocardiography: consensus document of the EACVI/ASE/Industry Task Force to standardize deformation imaging. European Heart Journal—Cardiovascular Imaging, 2015. 16 (1): 1–11. https://doi.org/10.1093/ehjci/jeu184 PMID: 25525063

24. Salem Omar AM, Shameer K, Narula S, Abdel Rahman MA, Rifaie O, Narula J, et al, Artificial Intelligence-Based Assessment of Left Ventricular Filling Pressures From 2-Dimensional Cardiac Ultrasound Images. JACC: Cardiovascular Imaging, 2018. 11 (3): 509–510. PMID: 28734920

25. Probst P, Wright MN, Boulesteix AL. Hyperparameters and tuning strategies for random forest. WIREs Data Mining Knowl Discov. 2019; 9:e1301. https://doi.org/10.1002/widm.1301

26. Genuer R, Poggi JM, and Tuleau-Malot C. 2010. Variable selection using random forests. Pattern Recogn. Lett. 31, 14 (October, 2010), 2225–2236. https://doi.org/10.1016/j.patrec.2010.03.014

27. Waldrop MM. News Feature: What are the limits of deep learning?. Proc Natl Acad Sci U S A. 2019; 116 (4):1074–1077. https://doi.org/10.1073/pnas.1821594116 PMID: 30670601

28. Akhtar N and Mian A. Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey. IEEE Access, vol. 6, pp. 14410–14430, 2018. https://doi.org/10.1109/ACCESS.2018.2807385

29. Salaun E, Casalta AC, Donal E, et al. Apical four-chamber longitudinal left ventricular strain in patients with aortic stenosis and preserved left ventricular ejection fraction: analysis related with flow/gradient pattern and association with outcome. Eur Heart J Cardiovasc Imaging. 2018; 19(8):868–878. https://doi.org/10.1093/ehjci/jex203 PMID: 28950336

30. Galli E, Leclercq C, Hubert A, Bernard A, Smiseth OA, Mabo P, et al. Role of myocardial constructive work in the identification of responders to CRT. European Heart Journal—Cardiovascular Imaging, 2017. 19 (9): 1010–1018. https://doi.org/10.1093/ehjci/jex191

31. Delgado V, van Bommel RJ, Bertini M, Jan Willem Borleffs C, Ajmone Marsan N, Ng ACT, et al. Relative merits of left ventricular dyssynchrony, left ventricular lead position, and myocardial scar to predict long-
term survival of ischemic heart failure patients undergoing cardiac resynchronization therapy. Circulation, 2011. 123 (1): 70–78. https://doi.org/10.1161/CIRCULATIONAHA.110.945345 PMID: 21173353

32. Yu CM, Fung WH, Lin H, Zhang Q, Sanderson JE, Lau CP. Predictors of left ventricular reverse remodeling after cardiac resynchronization therapy for heart failure secondary to idiopathic dilated or ischemic cardiomyopathy. The American Journal of Cardiology, 2003. 91 (6): 684–688. https://doi.org/10.1016/S0002-9149(02)03404-5 PMID: 12633798

33. Sweeney MO, Prinzen FW. Ventricular Pump Function and Pacing. Circulation: Arrhythmia and Electro-physiology, 2008. 1 (2): 127–139. PMID: 19808403

34. Linde C, Curtis AB, Fonarow GC, Lee K, Little W, Tang A, et al, Cardiac resynchronization therapy in chronic heart failure with moderately reduced left ventricular ejection fraction: Lessons from the Multi-center InSync Randomized Clinical Evaluation MIRACLE EF study. International Journal of Cardiology, 2016. 202: 349–355. https://doi.org/10.1016/j.ijcard.2015.09.023 PMID: 26426276

35. Galli E, Leclercq C, Fournet M, Hubert A, Bernard A, Smiseth OA, et al. Value of myocardial work estimation in the prediction of response to cardiac resynchronization therapy. Journal of the American Society of Echocardiography, 2018. 31: 220–230. https://doi.org/10.1016/j.echo.2017.10.009 PMID: 29246513

36. Khan FZ, Virdee MS, Palmer CR, Pugh PJ, O'Halloran D, Elsik M, et al. Targeted Left Ventricular Lead Placement to Guide Cardiac Resynchronization Therapy: The TARGET Study: A Randomized, Controlled Trial. Journal of the American College of Cardiology, 2012. 59 (17): 1509–1518. https://doi.org/10.1016/j.jacc.2011.12.030 PMID: 22405632

37. Delgado-Montero A, Tayal B, Goda A, Ryo K, Marek JJ, Sugahara M, et al. Additive Prognostic Value of Echocardiographic Global Longitudinal and Global Circumferential Strain to Electrocardiographic Criteria in Patients With Heart Failure Undergoing Cardiac Resynchronization Therapy. Circulation: Cardiovascular Imaging, 2016. 9 (6): e004241. PMID: 27252359

38. Risum N, Tayal B, Hansen TF, Bruun NE, Jensen MT, Lauridsen TK, et al. Identification of Typical Left Bundle Branch Block Contraction by Strain Echocardiography Is Additive to Electrocardiography in Prediction of Long-Term Outcome After Cardiac Resynchronization Therapy. Journal of the American College of Cardiology, 2015. 66 (6): 631–641. https://doi.org/10.1016/j.jacc.2015.06.020 PMID: 26248989

39. Pitzalis MV, Iacoviello M, Romito R, Massari F, Rizzon B, Luzzi G, et al. Cardiac resynchronization therapy tailored by echocardiographic evaluation of ventricular asynchrony. Journal of the American College of Cardiology, 2002. 40 (9): 1615–1622. https://doi.org/10.1016/S0735-1097(02)02337-9 PMID: 12427414

40. Bax JJ, Bleeker GB, Marwick TH, Molhoek SG, Boersma E, Steendijk P, et al. Left ventricular dyssynchrony predicts response and prognosis after cardiac resynchronization therapy. Journal of the American College of Cardiology, 2004. 44 (9): 1834–1840. https://doi.org/10.1016/j.jacc.2004.08.016 PMID: 15519016

41. Chung ES, Leon AR, Tavazzi L, Sun JP, Nihoyannopoulos P, Merlino J, et al. Results of the Predictors of Response to CRT (PROSPECT) Trial. Circulation, 2008. 117 (20): 2608–2616. https://doi.org/10.1161/CIRCULATIONAHA.107.743120 PMID: 18458170