Large Database and Registry Research in Joint Arthroplasty and Orthopaedics (2022)

The digital revolution has made possible the proliferation of large databases and the mining of “big data” from many sources in orthopaedic surgery. This revolution in the compiling of, and access to, huge amounts of data has in turn led to dramatic advances in the types of studies and research that can be done in our field. A symposium/workshop was held on October 14 and 15, 2021, in Chicago, Illinois, to explore the power and potential of large databases; consider the weaknesses and risks of analyses using these databases; provide information on methods, specific attributes, and best uses of the most commonly used databases; and provide guidance on how best to select and use these databases for orthopaedic research. The JBJS Supplement on Large Database and Registry Research in Joint Arthroplasty and Orthopaedics is the product of that symposium. The idea behind the symposium was that a systematic evaluation of the state of large-database research in orthopaedic surgery could provide much valuable information to the many stakeholders who interact with these databases. These stakeholders include orthopaedic researchers, orthopaedic surgeons who read orthopaedic research, and journal reviewers and editors who evaluate research. The symposium concentrated on databases commonly used in joint arthroplasty, but much of the information presented in the JBJS Supplement has applicability across other orthopaedic disciplines. The symposium was sponsored by a National Institutes of Health/National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIH/NIAMS) P30 Center Grant (Principal Investigator, Daniel Berry, MD, and Co-Principal Investigator, David Lewallen, MD), The Journal of Bone and Joint Surgery, and the Orthopaedic Research and Education Foundation (OREF). Important input in the genesis of the symposium also came from the Editor of the Journal of Arthroplasty (John Callaghan, MD). Speakers, and authors of articles in the JBJS Supplement, were recruited for their expertise in each subject area. Many thanks to them and to the other workshop participants who gave of their time and expertise to participate in the presentations and discussions. Large databases in orthopaedics include national registries, state and regional registries, institutional registries, large payor administrative databases, and specialty or procedure-specific databases1,2. A partial list of the broad variety of research questions that can be addressed with large databases includes associations between variables such as a patient characteristic, a diagnosis, a specific procedure, a category of orthopaedic implant, or a pharmacologic agent and the relationship of each variable to the subsequent outcome of surgical or nonsurgical treatment; cost-effectiveness of treatments; and descriptive research documenting trends in orthopaedic practice. While large databases and “big data” provide “real-world evidence” to aid research that can improve the care of orthopaedic patients, these data sources also have important limitations, not all of which are intuitively obvious. One of the most important is that, although large observational databases are powerful tools to identify associations between variables (and thus are good tools to form hypotheses that can be subsequently tested with more rigorous study designs), they can rarely prove a causal relationship between an exposure and an outcome due to the potential for sampling bias and unmeasured confounders. A second important reality is that studies using large databases may show significant associations between variables—because of the power of big numbers—but these differences can be of limited clinical relevance due to the small magnitude of the effect. Some limitations are highly specific to the individual database or data source. Moreover, the data contained in specific databases may differ with respect to criteria for and selectiveness of patient inclusion; availability of specific data points with respect to patient characteristics, comorbidities, and surgical and implant-specific factors; accuracy of each data point collected; duration, frequency, and completeness of follow-up; and the outcomes collected at the time of follow-up. Also, some of these factors may change over time, even in the same database. Many of these same factors can introduce selection and reporting bias and distort findings. The databases also vary with respect to access criteria and expense to gain access. The strengths, weaknesses, limitations, best uses, and inappropriate uses of many commonly utilized orthopaedic databases at a granular level were explored at the symposium. The first group of papers in the Supplement evaluate commonly used databases in orthopaedic surgery, and for each one provide information on patient inclusion criteria, main data elements collected, completeness of follow-up, and data accessibility (Table I)3-7. Subsequent papers evaluate what to look for in a big database; best and poor uses of large databases8; how to use and not use “big data” in cost-effectiveness research9; the strengths and important weaknesses of large databases (especially administrative databases)9 particularly in evaluating new technology10; linkage between databases11; and how natural language processing, machine learning, and artificial intelligence12 may be used to mine data from formal and less formal data sources (such as electronic health records) now and in the near future. TABLE I - Databases and Topics Addressed in JBJS Supplement on Large Database and Registry Research in Joint Arthroplasty and Orthopaedics Description Databases State and government administrative databases Components of Medicare data (Standard Analytical Files, Master Beneficiary Summary File), National Inpatient Sample (NIS), and State Inpatient Databases (SIDs) Strengths: Good for evaluation of performance and quality improvement programs, cost/utilization studies, studies requiring large representative samples Limitations: Reliance on billing coding; inconsistencies in coding, data quality, and expertise in using the data; lack of granularity; limited clinical information Large databases with direct data abstraction Description & data content of American College of Surgeons National Surgical Quality Improvement Program (NSQIP) and Veterans Health Administration (VHA) databases Strengths: Inclusion of both inpatient and outpatient data, good data quality, complete 30-day follow-up, easily accessible, free resource for orthopaedic investigators at participating sites Limitations: No data on facility characteristics/geography, no trauma data, large but only nationally representative, only selected postoperative diagnoses VHA Databases: Nearly complete electronic health record (EHR) information, patient population >90% adult males, loss to follow-up, access only by VHA investigators Commercial claims databases & PearlDiver Databases through individual commercial insurers not easily accessible but others compile data from multiple sources (IBM MarketScan, OptumLabs, PearlDiver) Strengths: PearlDiver available for purchase, simplified use, large samples, includes both commercial and government claims, good for analysis of short-term outcomes Limitations: Not good for long-term outcomes due to interruptions in continuity of coverage, limited medical history data, some data not included, no operative or implant data, not all payors included State-based and national U.S. registries History, data elements, strengths, and limitations; best current uses of Michigan Arthroplasty Registry Collaborative Quality Initiative (MARCQI), California Joint Replacement Registry (CJRR), and American Joint Replacement Registry (AJRR) International registries Characteristics and ongoing data linkage efforts by Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR), registries in Nordic countries, and National Joint Registry (NJR) for England and Wales Database-related topics Optimizing use of large databases Assessment of large databases using the “5 Ws” of information gathering: who, what, where, when, why. Best current uses, common errors, and distinction between hypothesis generation and hypothesis testing studies, importance of adhering to STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) and CONSORT (Consolidated Standards of Reporting Trials) criteria Linkage between databases Advantages of data linkages, ongoing/planned AJRR data linkage efforts with Medicare data and other private payor data Administrative data use in registry efforts and cost-effectiveness research 4 pitfalls to avoid when using administrative claims data: overreliance on a single code, ignoring fluidity of coding, trying to get too granular, not ensuring lateralityCost-effectiveness research: Lack of data on direct non-medical costs, patient-reported outcomes for cost-effectiveness research, important to distinguish costs, charges, payments Analysis of new technologies in large databases Historical perspectives, challenges, and opportunities in using large databases for evaluation of effectiveness of new orthopaedic technologies (robotics, computer-assisted surgery) highlighting issues related to patient selection, confounding, and measurement bias Getting more out of large databases and EHRs Emerging opportunities in using artificial intelligence (AI) and specifically natural language processing (NLP) technologies in extracting orthopaedic data elements from unstructured sources in EHRs Readers of the Supplement may use the material to guide their approach to orthopaedic studies that rely on large databases. The symposium was not designed to answer the important question of the best methodologies to use in the analysis of data. A separate series of papers, underwritten by the same NIH/NIAMS P30 Center Grant, is being published along with video tutorials to help address those questions in a user-friendly, detailed manner centered around orthopaedic surgery13. The Supplement will answer many key questions related to large databases: Can I answer my question based on existing data in a large database? Which databases are available? Which one is best suited to answering the desired questions? What limitations, drawbacks, or caveats are associated with using such a database as the foundation for my study? Finally, by bringing all of this information together in a single volume, we hope that the value of the amalgamated information will be greater than the sum of its parts.