Estimation of treatment effects in observational stroke care data: comparison of statistical approaches
Authors: Amini Marzyeh , van Leeuwen Nikki , Eijkenaar Frank , van de Graaf Rob , Samuels Noor , van Oostenbrugge Robert , van den Wijngaard Ido R. , van Doormaal Pieter Jan , Roos Yvo B. W. E. M. , Majoie Charles , Roozenbeek Bob , Dippel Diederik , Burke James , Lingsma Hester F. ,   on behalf of the, MR CLEAN Registry Investigators, Dippel Diederik W. J. , van der Lugt Aad , Majoie Charles B. L. M. , Roos Yvo B. W. E. M. , van Oostenbrugge Robert J. , van Zwam Wim H. , Boiten Jelis , Vos Jan Albert , Brouwer Josje , den Hartog Sanne J. , Hinsenveld Wouter H. , Kappelhof Manon , Compagne Kars C. J. , Goldhoorn Robert-Jan B. , Mulder Maxim J. H. L. , Jansen Ivo G. H. , Dippel Diederik W. J. , Roozenbeek Bob , van der Lugt Aad , van Es Adriaan C. G. M. , Majoie Charles B. L. M. , Roos Yvo B. W. E. M. , Emmer Bart J. , Coutinho Jonathan M. , Schonewille Wouter J. , Vos Jan Albert , Wermer Marieke J. H. , van Walderveen Marianne A. A. , Staals Julie , van Oostenbrugge Robert J. , van Zwam Wim H. , Hofmeijer Jeannette , Martens Jasper M. , Lycklama à Nijeholt Geert J. , Boiten Jelis , de Bruijn Sebastiaan F. , van Dijk Lukas C. , van der Worp H. Bart , Lo Rob H. , van Dijk Ewoud J. , Boogaarts Hieronymus D. , de Vries J. , de Kort Paul L. M. , van Tuijl Julia , Peluso Jo Jo P. , Fransen Puck , van den Berg Jan S. P. , van Hasselt Boudewijn A. A. M. , Aerden Leo A. M. , Dallinga René J. , Uyttenboogaart Maarten , Eschgi Omid , Bokkers Reinoud P. H. , Schreuder Tobien H. C. M. L. , Heijboer Roel J. J. , Keizer Koos , Yo Lonneke S. F. , den Hertog Heleen M. , Sturm Emiel J. C. , Brouwers Paul , Majoie Charles B. L. M. , van Zwam Wim H. , van der Lugt Aad , Lycklama à Nijeholt Geert J. , van Walderveen Marianne A. A. , Sprengers Marieke E. S. , Jenniskens Sjoerd F. M. , van den Berg René , Yoo Albert J. , Beenen Ludo F. M. , Postma Alida A. , Roosendaal Stefan D. , van der Kallen Bas F. W. , van den Wijngaard Ido R. , van Es Adriaan C. G. M. , Emmer Bart J. , Martens Jasper M. , Yo Lonneke S. F. , Vos Jan Albert , Bot Joost , van Doormaal Pieter-Jan , Meijer Anton , Ghariq Elyas , Bokkers Reinoud P. H. , van Proosdij Marc P. , Krietemeijer G. Menno , Peluso Jo P. , Boogaarts Hieronymus D. , Lo Rob , Gerrits Dick , Dinkelaar Wouter , Appelman Auke P. A. , Hammer Bas , Pegge Sjoert , van der Hoorn Anouk , Vinke Saman , Dippel Diederik W. J. , van der Lugt Aad , Majoie Charles B. L. M. , Roos Yvo B. W. E. M. , van Oostenbrugge Robert J. , van Zwam Wim H. , Lycklama à Nijeholt Geert J. , Boiten Jelis , Vos Jan Albert , Schonewille Wouter J. , Hofmeijer Jeannette , Martens Jasper M. , van der Worp H. Bart , Lo Rob H. , van Oostenbrugge Robert J. , Hofmeijer Jeannette , Flach H. Zwenneke , Lingsma Hester F. , el Ghannouti Naziha , Sterrenberg Martin , Puppels Corina , Pellikaan Wilma , Sprengers Rita , Elfrink Marjan , Simons Michelle , Vossers Marjolein , de Meris Joke , Vermeulen Tamara , Geerlings Annet , van Vemde Gina , Simons Tiny , van Rijswijk Cathelijn , Messchendorp Gert , Nicolaij Nynke , Bongenaar Hester , Bodde Karin , Kleijn Sandra , Lodico Jasmijn , Droste Hanneke , Wollaert Maureen , Verheesen Sabrina , Jeurrissen D. , Bos Erna , Drabbe Yvonne , Sandiman Michelle , Elfrink Marjan , Aaldering Nicoline , Zweedijk Berber , Khalilzada Mostafa , Vervoort Jocova , Droste Hanneke , Nicolaij Nynke , Simons Michelle , Ponjee Eva , Romviel Sharon , Kanselaar Karin , Bos Erna , Barning Denn , Venema Esmee , Chalos Vicky , Geuskens Ralph R. , van Straaten Tim , Ergezen Saliha , Harmsma Roger R. M. , Muijres Daan , de Jong Anouk , Berkhemer Olvert A. , Boers Anna M. M. , Huguet J. , Groot P. F. C. , Mens Marieke A. , van Kranendonk Katinka R. , Treurniet Kilian M. , Jansen Ivo G. H. , Tolhuisen Manon L. , Alves Heitor , Weterings Annick J. , Kirkels Eleonora L. F. , Voogd Eva J. H. F. , Schupp Lieve M. , Collette Sabine , Groot Adrien E. D. , LeCouffe Natalie E. , Konduri Praneeta R. , Prasetya Haryadi , Arrarte-Terreros Nerea , Ramos Lucas A. 
Published: April 2022

Introduction

Various statistical approaches can be used to deal with unmeasured confounding when estimating treatment effects in observational studies, each with its own pros and cons. This study aimed to compare treatment effects as estimated by different statistical approaches for two interventions in observational stroke care data.

Patients and methods

We used prospectively collected data from the MR CLEAN registry including all patients (n = 3279) with ischemic stroke who underwent endovascular treatment (EVT) from 2014 to 2017 in 17 Dutch hospitals. Treatment effects of two interventions – i.e., receiving an intravenous thrombolytic (IVT) and undergoing general anesthesia (GA) before EVT – on good functional outcome (modified Rankin Scale ≤2) were estimated. We used three statistical regression-based approaches that vary in assumptions regarding the source of unmeasured confounding: individual-level (two subtypes), ecological, and instrumental variable analyses. In the latter, the preference for using the interventions in each hospital was used as an instrument.

Results

Use of IVT (range 66–87%) and GA (range 0–93%) varied substantially between hospitals. For IVT, the individual-level (OR ~ 1.33) resulted in significant positive effect estimates whereas in instrumental variable analysis no significant treatment effect was found (OR 1.11; 95% CI 0.58–1.56). The ecological analysis indicated no statistically significant different likelihood (β = − 0.002%; P = 0.99) of good functional outcome at hospitals using IVT 1% more frequently. For GA, we found non-significant opposite directions of points estimates the treatment effect in the individual-level (ORs ~ 0.60) versus the instrumental variable approach (OR = 1.04). The ecological analysis also resulted in a non-significant negative association (0.03% lower probability).

Discussion and conclusion

Both magnitude and direction of the estimated treatment effects for both interventions depend strongly on the statistical approach and thus on the source of (unmeasured) confounding. These issues should be understood concerning the specific characteristics of data, before applying an approach and interpreting the results. Instrumental variable analysis might be considered when unobserved confounding and practice variation is expected in observational multicenter studies.