Causal Analysis of the TOPCAT Trial: Spironolactone for Preserved Cardiac Function Heart Failure
Abstract
We describe the results of applying causal discovery methods on the data from a multi-site clinical trial, on the Treatment of Preserved Cardiac Function Heart
Failure with an Aldosterone Antagonist (TOPCAT).
The trial was inconclusive, with no clear benefits consistently shown for the whole cohort.
However, there were questions regarding the reliability of the diagnosis and treatment protocol for a geographic subgroup of the cohort.
With the inclusion of medical context in the form of domain knowledge, causal discovery is used to demonstrate regional discrepancies and to frame the regional transportability of the results.
Furthermore, we show that, globally and especially for some subgroups, the treatment has significant causal effects, thus offering a more refined view of the trial results.
Introduction
Complex trials with heterogeneities in the data pose a significant challenge for traditional techniques of statistical analysis.
Suboptimal patient cohort selection and recruitment, paired with ineffective monitoring of patients, are among the main causes for high trial failure rates: only 10% of the compounds entering a clinical trial reaches the market [1].
Phase 2 clinical studies, albeit a poor predictor of drug success ( > 30% and > 58% trial failure rates in phase 2 and 3 , respectively) are critical in determining drug costs [2].
Results of multi-site clinical trials often show significant differences between the trial sites.
There can be a variety of reasons for this, including different treatment benefits among different types of participants or site anomalies that could invalidate the study, such as suspected failures to assign the correct diagnosis and to adhere to treatment protocols.
The TOPCAT clinical trial is a randomized, double-blind trial: 3445 patients with symptomatic Heart
Failure and a preserved left ventricular Ejection Fraction (HFpEF) of 45% or more were assigned to receive either spironolactone 1 ( 15 mg to 45 mg daily) or placebo, with a mean follow-up of 3 . 3 years.
Eligible patients had a history of hospitalization within the previous 12 months, with management of heart failure a major component of the care provided, or an elevated natriuretic peptide level within 60 days before randomization 2 .
Trial randomization was stratified according to whether patients were [Table] enrolled on the basis of the former criterion (i.e. hospitalization stratum) or the latter criterion (i.e. the BNP stratum).
The TOPCAT clinical trial was funded by the National Heart, Lung, and Blood
Institute [4], [5].
Details of the trial (inclusion and exclusion criteria and covariates) and its first analysis can be found in [3].
The goal is an evaluation of the effects of spironolactone in patients with
HFpEF: the primary outcome is a composite of death from cardiovascular causes, aborted cardiac arrest, or hospitalization for the management of heart failure.
[-Brain Natriuretic Peptide (BNP) ≥ 
-]
The trial was carried out on a geographically diverse populations with different demographic characteristics (Western vs Eastern regions) [6].
Moreover, the clinical diagnostic criteria were not uniformly interpreted or applied [7].
While country-specific and regional heterogeneity could be viewed as expected statistical variation in a large multinational trial, the differences in patient characteristics, lower event rates, lack of certain drug side effects, and complete lack of treatment effect in Russia and Georgia compared with the Americas lead to a strong suspicion that this data is unreliable in both diagnosis and adherence to treatment [8].
Transportability methods detect significant regional differences and find that measured participant characteristics do not explain the discrepancies between the sites [9].
These methods show promise in detecting anomalies within multi-center Randomized
Controlled Trials (RCTs) and in assisting data monitoring to attribute important treatment-effect heterogeneities to participant differences or to site performance differences [10].
Traditional methods based on correlation and survival analysis are prone to confounding and struggle to capture complex interactions: causal discovery is a necessary step in overcoming these issues.
Causality can offer new insights into clinical trials: the causal methods described in [11]–[14] show promise for analyzing results from RCTs, via target trials, transportability, and prediction invariance.
This paper shows that causal discovery allows us to understand the causal pathways of the main outcomes of the TOPCAT trial, by deriving a causal diagram that provides insight into the regional transportability of the trial results.
We were able to demonstrate regional differences of the causal mechanisms between the two main geographical subsets of the trial, offering evidence of the controversial discrepancy of the original study (Western vs Eastern countries).
Finally we show that the identified causal models can be linked to a more traditional survival analysis of the trial data, achieving better explanatory performance.
The rest of the paper is organized as follows: the data preparation and the causal discovery are detailed in § 2, and the modelling of clinical outcomes following a survival analysis with the newly found causal relationships is detailed in § 3.
A list of acronyms can be found on page 9.
Causal Discovery
A total of 3445 participants were recruited at 233 sites in 6 countries ( 1151 participants in the United
States, 326 in Canada, 167 in Brazil, 123 in Argentina, 1066 in Russia, and 612 in Georgia).
The original data has a number of time varying variables based on each patient’s hospital visits.
The original trial data consist of 55 separate datasets with a total of 759 columns.
A number of these columns are identifiers such as patient ID, date of observation, etc., that are repeated across datasets.
The time and patient coverage of these datasets is varied and we excluded echo-gram data due to its low coverage.
There are 7 measured outcomes and the primary outcome is a composite of 3 of them.
The outcomes are presented in a time to event format consistent with the survival analysis approach taken in the original paper [3].
For our analysis we converted this data to a time series format to better capture the time varying nature of the data.
We aligned the data in time and by patient to create a time series per individual for each variable present in the data.
The time resolution was set to 6 months based on a similar average frequency of hospital visits.
In each 6 month period starting from initial randomization to 6 years, for each patient we recorded the latest observed data and generated binary variables indicating whether a measured outcome had occurred in the given period or previously.
Baseline or static covariates were forward filled from initial randomization.
Censored data is automatically handled by this representation, as any period of survival offers a contribution to the data and the time resolution is sufficiently granular to make intra-period dropout approximately equivalent to exact time censoring.
By weighting each individual equally as opposed to each row of data we also can avoid underweighting individuals with shorter survival times.
The goal of causal discovery is to identify potential causal drivers from observational data.
These methods distinguish correlated variables into causal drivers and confounded variables: the latter are correlated to the target, but only because there exists a third variable, which influences both.
Directed
Acyclic Graphs (DAGs) are used to map all a priori assumptions surrounding a causal question and to graphically describe the underlying data generating process.
Each node represents a random variable, and directed causal paths are represented by directed edges.
The causal graph structure thus provides qualitative information about the conditional independence of the variables of interest, and illustrates potential sources of confounding and selection bias.
Given the complexity of the TOPCAT dataset and the importance of a few clinical outcomes, it would be inefficient to estimate the entire underlying DAG.
Instead, we chose a feature selection strategy aimed at deriving the smallest subset of variables, excluding those that do not provide additional information on the outcomes of interest.
If the lack of additional information is assessed through conditional independence tests, the problem of feature selection is analogous to identifying the
Markov Blanket (MB) of the outcome variable.
Definition 1 The Markov Blanket (MB) of the outcome Y , MB ( Y ) , is the minimal subset of X , conditioned on which, all other variables of X not included in MB ( Y ) are independent of Y :
[Equation]
If the prediction model for Y can estimate the underlying true probabilities P ( Y | MB ( Y )) , then the variables included in the MB of the outcome Y are the only variables needed for an optimal prediction in terms of calibration [14].
In order to derive the MB of the primary outcomes, we used PCMCI, specifically designed for time series data, which tests the variables for conditional independence for identified conditioning sets [15].
This method can work directly on the trial data converted to the time series format.
PCMCI performs two steps, consisting of an implementation of the PC algorithm, followed by the MCI algorithm.
The PC Algorithm identifies a set of potential parents for both each feature and the target.
After the potential parents for each variable and the target have been identified, the MCI Algorithm calculates conditional independencies conditioned on those parents.
This implementation of PCMCI uses time in order to identify the causal direction.
Our causal discovery is enhanced by the integration of medical context as a constraint structure that our algorithms have to respect.
This guarantees both a better performance and time complexity.
We specified 8 tiers of variables, i.e. the structure of directionality that we want to enforce in our causal discovery in the form of domain knowledge for groups of variables (cf.
§ A).
The method used for conditional independence tests is the Spearman rank-order correlation coefficient, i.e. a nonparametric measure of the monotonicity of the relationship between two datasets.
Unlike the Pearson correlation, the Spearman correlation does not assume that both datasets are normally distributed and that their relationship is linear 3 .
We set the threshold of significance to 0 .
01 for the PC step and to 0 .
005 for the
MCI step of the algorithm (i.e. the threshold for accepting a variable as causal in the resulting DAG).
2.1 Causal graphs with domain knowledge
We applied PCMCI to achieve the causal discovery of the MB of the outcomes at three scales:
Global level: the whole dataset (cf.
Figures 2 and 3).
Regional level: the Western countries (US, Canada, Brazil, Argentina) (cf.
Figures 4 and 5).
Regional level: the Eastern countries (Russia, Georgia) (cf.
Figures 6 and 7).
We present those DAGs in the supplementary material (cf.
§ A).
We also show simplified causal graphs for each scale for visualisation purposes, allowing us to focus on the identified relationships of most interest and impact by specifying a restrictive causality threshold to enforce sparsity.
These diagrams visualize the causal pathways through which covariates affect outcomes through time.
We introduced additional domain knowledge of known causal pathways as edges from treatment to the known side effects and from the constituents of the primary outcome to the primary outcome event itself.
In order to explain the treatment effect, we consider the p-values of the edges that are outgoing from treatment and incoming to the outcomes (cf.
Table 1).
We notice a correspondence between the global and the Western tables, both in terms of significance and of the score signs, as opposed to the
Eastern table.
A negative score means that the treatment has the effect of reducing the probability of the outcomes occurring in a time period. GLOBAL WEST EAST
[Table] Using Cox regression [Table] the treatment was only significantly associated with Outcome hfhosp in the global dataset (for results of the original study see § A.2).
By modelling the causal dynamics and adjusting for the broader MB set, our global analysis has mitigated the regional discrepancies to a greater degree, allowing us to identify significant causal effects of the treatment on the main outcomes as well as on additional relevant outcomes on the global data.
(cf.
Table 1).
In the global dataset we detected a positive causal effect of treatment on the risk of Outcome cvd death.
We noticed a negative causal effect of treatment on the other primary outcomes (Outcome hfhosp and Outcome abortedca).
This discrepancy with the Western subset might be a consequence of the conditioning set in the causal discovery that needs to be investigated and addressed.
We detected a positive effect of treatment on most secondary outcomes.
In the Western subset we detected a positive causal effect of treatment on almost all the considered primary and secondary outcomes.
In the Eastern subset we noticed fewer significant causal drivers in general, and more discrepancies in the signs of the interactions.
2.2 Bootstrap analysis
We propose a bootstrap analysis as a baseline for evaluating the regional heterogeneities, by comparison with random partitions of the data.
When examining the observed differences in causal scores and graph structures across regions, it is important to contextualize these numbers.
With random variation in the data some level of difference would be expected and we can quantify this expected variation using bootstrap sampling.
By drawing random samples of data partitions that are uncorrelated to Region or other covariates, we can observe the expected distribution of the differences due to randomness and compare them to the observed regional behavior.
We drew 100 of these bootstrap samples, randomly partitioning the data into two parts.
After running causal discovery on all the sets of subsections, we calculated a number of metrics to capture the variation in the identified causal structures in Table 2.
Histograms of SHD and the number of contradictions can be found in Figure 1, and details of the differences among single outcomes can be found in Table 3.
Looking at a broader causal picture has allowed us to capture regional differences very strongly when a univariate survival analysis could not.
We can see in Tables 2 and 3 that stratifying the data by region led to significantly higher differences in results compared to random partitions.
Differences in score (SMAPE) were significant with regard to treatment ( 86% quantile) and extreme when taking into account other covariates ( 100% quantile).
Measurements around graph structure (SHD) also gave a clear picture ( 97% quantile).
The number of contradictory causal conclusions between regions ( 26 ) was higher than any observed bootstrap sample with the highest observed being 19 ( 100% Symmetric Mean Absolute Percentage Error (SMAPE) [Table] [Table]
[Figure] [Figure]
Histogram
[Figure]
SHD
[Figure]
Causal Contradictions
Histogram of Bootstrap Contradictions vs Observed bootstrapobserved
[Table] quantile).
This comparison strongly supports the claim of regional treatment discrepancy in terms of the difference between the corresponding causal structures.
Survival Analysis
We propose the use of causal discovery as an automatic tool for feature selection of the most influential covariates for a traditional survival analysis.
Generally, covariates are selected manually or using correlation methods, without taking into account the causal relationship to the outcomes of interest.
In RCTs with a survival outcome, treatment effects are generally estimated with the Cox hazard ratio, often without adjusting for known prognostic variables.
When interpreting Cox regression through the lens of causality, omission of confounding terms creates problems, despite randomization of treatment at the origin of time ( t = 0 ).
Therefore, the hazard ratio does not admit a causal interpretation in the case of unknown heterogeneity [16], as the risk sets beyond the first event time are comprised of the subset of individuals who have not previously failed.
Despite randomization at t = 0 , the contributions to the Cox partial likelihood following the first failure will not remain randomized.
On the other hand, the effect of treatment in terms of an Accelerated Failure Time (AFT) model, does yield a causal measure of treatment effects as the hazard function has an additive form in the covariates [16] (cf. § A.3 for more details).
The coefficient of the treatment variable in the AFT model can be interpreted as slowing down or speeding up the survival function.
A coefficient b changes the expected survival time by a factor of exp( b ) .
We can see in the tables below that the Global and
Western AFT models identify a significant effect of the treatment on heart failure hospitalization.
This coefficient of 0 .
28 is interpreted as meaning that the expected survival time before heart failure hospitalization is exp(0 .
28) − 1 = 32 .
3% longer given the treatment than without.
We present results of goodness of fit for the AFT regression of the outcomes in the form of Bayesian
Information Criterion (BIC).
For large datasets, the model with minimum Bayesian Information
Criterion (BIC) ideally corresponds to the candidate model which is a posteriori most probable [17].
Results of the AFT regression model for the global dataset and the Western and Eastern subsets can be found in Table 4.
The rows correspond to the particular predicted outcome, whereas the columns correspond to two feature selection strategies before AFT modeling: (1) Features corresponding to the MB of the outcomes (cf.
Section 2); (2) Features used in the primary study in [3].
We can see that the MB yielded by causal discovery in all cases leads to a lower BIC and correspondingly a better model of the data.
This accounts for increased complexity in the model with the introduction of the new MB covariates.
The absolute scale of the BIC for each model is a function of the size of the data so does not allow for relative comparison across regions but by looking at the baseline Original model on each region we can see relatively how well we capture the survival process in this data.
[Table]
Conclusion and Perspectives
Complex trials with heterogeneities in the data pose a significant challenge for traditional techniques of statistical analysis.
Suboptimal patient cohort selection and recruitment, paired with ineffective monitoring of patients, are among the main causes for high trial failure rates in phases 2 and 3 [1], [2].
Our ambition is to use causality in order to provide earlier determination of drug futility, inform adaptive trial design [Table] and shorten overall drug development time.
In this paper we show that cutting edge techniques of causal discovery augmented with human domain knowledge can offer novel insights and achieve results at greater level of significance.
Traditional correlational methods are limited by confounding and multiple pathways and traditional survival modelling is restricted to a univariate outcome, necessitating the definition of a single primary outcome.
Causal discovery allows us to understand the interaction between outcomes and covariates leading to a diagram describing the causal process of the TOPCAT dataset.
Our causal analysis has mitigated the regional discrepancies, allowing us to identify significant global causal effects of the treatment on the main outcomes as well as on additional relevant outcomes.
After derivation of the regional causal diagrams, our bootstrap analysis provided evidence for a different causal effect of the treatment by region.
We also applied an AFT modelling to the data and were able to achieve lower
BIC after MB feature selection.
As a future extension of our work, we intend to achieve better causal effect estimation and design of clinical trials, as secondary relationships are learned from this dataset via full causal discovery.
A combination of more in depth domain knowledge and data driven causal inference methods could lead to better survival models.
Additional covariates could be added thus accounting for further potential confounding.
The causal framework also allows conclusions from trials to go beyond binary outcomes and further refine our analysis (e.g. including multiple levels/doses of treatment).
These conclusions could be continuously derived during a clinical trial, potentially leading to changes in its framework or to an early termination.
Transportability of causal graphs [18]–[22] refers to the problem of extrapolating experimental findings across domains (i.e., settings, populations, environments) that differ both in their distributions and in their inherent causal characteristics [23], [24].
For instance, this can allow us to infer the causal effect at one population from experiments conducted in a different population after noticing that some of the covariate distributions are different.
The subject of transportability in epidemiology is also related to data fusion, i.e. the combination of compatible graphs according to their degree of confidence, similarly to [25].
Since the TOPCAT study combined data from multiple geographical sources that had different underlying dynamics leading to mixed results, we could combine compatible graphs from different sub-populations according to their degree of confidence.
We could similarly validate data sources and our confidence in their adherence to the study design by comparing their causal dynamics to known ground truths.
A Supplementary Material Acronyms
AFT Accelerated Failure Time. 6, [-
-]7,
BIC Bayesian Information Criterion.
DAG Directed Acyclic Graph. 3,[-
-]
HFpEF Heart Failure and a [Table] Ejection Fraction. 1,
MB Markov Blanket.
[Table] abortedca Aborted [Table]
outcome [Table]
Hospitalisation
[Table] [Table] [Table] [Table]
[Table]
[Table] infarction [Table] [Table]
[Table] [Table] [Table]
Randomized Controlled Trial
SHD Structural Hamming Distance. 4,
SMAPE Symmetric Mean Absolute Percentage Error
TOPCAT Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone [-Antagonist.1, 2, 
-]Antagonist.
[ [ ‘Americas’, ‘Region interaction’, ‘treatment’, ‘age entry’, ], [ ‘blood potassium’, ‘systolic blood pressure’, ‘diastolic blood pressure’, ‘Left ventricular ejection fraction’, ‘Heart Failure History’, ‘Previous Atrial Fibrillation’, ‘Known chronic hepatic disease’, ‘Heart
Failure hospitalization’, ‘Angina’, ‘Chronic Obstructive Pulmonary
Disease’, ‘Asthma’, ‘Peripheral Arterial Disease’, ‘Dyslipidemia’, ‘Implanted cardioverter defibrillator’, ‘Pacemaker implanted’, ‘Atrial fibrillation’, ‘Coronary artery bypass graft surgery’, ‘Percutaneous
Coronary Revascularization’, ‘Stroke’, ‘myocardial infarction’, ‘smoker’, ‘years smoker’, ‘previous smoker’, ‘cigarettes smoke a day’, ‘sodium intake score’, ‘alcohol per week’, ‘activity level’, ‘weight’, ‘waist circumference’, ‘heart rate’, ‘Use of blood pressure lowering medication’, ‘Use of Non-cardiovascular medication’, ‘Use of cardiac medications’, ‘aspirin use’, ‘nitrate use’, ‘statin use’, ‘warfarin use’, ‘diuretic use’, ‘Use of Angiotensin receptor blocker’, ‘Use of Angiotensin converting enzyme inhibitor’, ‘white blood cell count’, ‘hemoglobin’, ‘Platelet’, ‘Albumin’, ‘Total Bilirubin’, ‘blood glucose’, ‘Alkaline Phosphatase’, ‘Aspartate
Aminotransferase’, ‘blood sodium’, ‘blood chloride’, ‘creatinine result’, ‘gfr’, ‘Insulin treated’, ‘Not Insulin treated’, ‘IN HOSP’, ‘left atrial volume 4’, ‘left atrial volume 2’, ], [ ‘serum potassium high’, ‘Systolic blood pressure (SBP) > 160 mm Hg’, ], [ ‘NYHA class’, ‘hyperkalemia’, ‘Hypertension’, ], [ ‘creatinine doubled’, ‘hyperkalemia hospitalization’, ], [ ‘Outcome anyhosp’, ‘Outcome mi’, ‘Outcome stroke’, ‘Outcome abortedca’, ‘Outcome hfhosp’, ], [ ‘Outcome death’, ‘Outcome cvd death’, ‘hyperkalemia side effect discontinued’, ‘potassium side effect discontinued’, ‘renal function side effect discontinued’, ], [‘Outcome primary ep’, ‘target’], ]
A.2 Comparison with the original study
For the sake of comparison with the primary study in [3], we reproduce their original table of results:
Outcome
[Table]
The original study in [3] concluded that adding spironolactone to existing therapy in patients with heart failure and a preserved ejection fraction did not significantly reduce the incidence of the primary outcome (cf.
Table 5).
As for the single Primary outcomes, spironolactone did not significantly reduce the composite primary end point of death from cardiovascular causes, aborted cardiac arrest, or hospitalization for heart failure.
Also, neither the time to first hospitalization for any reason nor the time to death from any cause was significantly altered by random assignment to spironolactone.
These two inclusive outcome measures are important for assessing overall risk versus benefit of the drug.
Post hoc analysis indicated marked regional differences in event rates in the Supplementary Appendix of [3] (cf Table 6).
In the Americas (the United States, Canada, Brazil, and Argentina), the primary outcome occurred in 242 patients in the spironolactone group ( 27 . 3% ) and 280 patients in the placebo group ( 31 . 8% ).
In Russia and Georgia, the primary outcome occurred in 78 patients in the spironolactone group ( 9 . 3% ) and 71 patients in the placebo group ( 8 . 4% ).
However, the pre-specified test for interaction between region and study group was not significant ( p = 0 . 12 )
[Table]
A.3 Accelerated Failure Time (AFT)
AFT Models assume the following form:
[Equation]
where T denotes the time to event, ϵ is distributed according to the underlying parametric assumption, and θ denotes the joint effect of covariates x i (typically θ = β T x = exp ( (cid:80) i β i x i ) ).
This reduces
AFT modelling to a regression where we estimate the fixed effect of our covariates with the chosen parametric distribution modelling the noise.
We used a Weibull distribution for our analysis.
We can write Equation 2 as
[Equation]
where x is the set of covariates and W has extreme value distribution.
Given transformations
[Equation]
we obtain a Weibull model with proportional basal hazard
[Equation]
Notice that the logarithm of the hazard in Equation 5 is additive in the different covariates:
[Equation]