bluuebunny/crossref_metadata_2025_split huggingface dataset:
Designing a utility evaluation framework for synthetic health data
Authors: Marashi Amir , Dohadwalla Sadiq , Browne Patrick , Hachey Ben 
Published: September 2024

ObjectivesSynthetic data (SD) promises to unlock health data for training, research, and innovation. However, where utility evaluation is performed, it is applied ad-hoc for a single task of interest. We produce an initial design for a robust benchmark across a range of tasks.
ApproachWe undertook several projects as a prototyping experiment to gather requirements. These projects replicate previous studies performed on the Medical Information Mart for Intensive Care — a dataset used in more than 4,000 studies. We refine definitions, identify personas, draft a user statement, and collect requirements.
ResultsDefinitions: We define utility as an extrinsic measure of SD on a larger system, most often through comparison to system performance on real data. This contrasts with fidelity, which measures the accuracy of SD through direct comparison to real data.
Personas: Data custodian, User of SD, SD researcher.
User statement: As a technical stakeholder, I need a reliable way to measure the utility of datasets and a benchmark to compare generation techniques.
Requirements: SD researchers can focus on generation not evaluation; Supports comparison and leaderboards; Based on relevant and applications; Comprehensive across study types and applications; Future proof for population research requiring linking.
ConclusionWe propose the following design:

Data pipelines follow an extract-generate-evaluate workflow.
Study types include cross-sectional and longitudinal.
Applications include predictive modelling and clinical research.

This results in a comprehensive utility benchmarking suite that complements current frameworks for fidelity and privacy of SD.
Improving Electronic Health Record (EHR) data linkages for identifying Aged Care Residents through publicly available data
Authors: Ravipati Tanya , Beare Richard , Srikanth Velandai , Carver Alison , Huang Lei , Dhillon Rashwinder , Andrew Nadine E 
Published: September 2024

IntroductionAccurate identification of Aged Care residents accessing hospital services is challenging due to multiple hospital Electronic Health Record (EHR) systems and entry points resulting in inconsistent addresses and coding. Improving accuracy and completeness of addresses is crucial for cohort identification and data linkage.
ObjectiveDevelop methods to validate cross-sectoral data linkages to improve identification of aged care residents within EHR systems.
ApproachAmong individuals aged ≥65 years (or ≥50 identifying as Aboriginal or Torres Strait Islander) between January 2013 to December 2023 within our Data Platform, we first used data items mandated by the Department of Health to identify aged care residency. Data quality was enhanced using a residential address index derived from Australian Geocoded National Address File (G-NAF). We evaluated sensitivity, specificity, and Positive Predictive Values (PPVs) of recorded addresses against the Australian Institute of Health and Welfare (AIHW) aged care services list.
ResultsAmong 409,677 addresses extracted, 55.4% (raw) had sufficient data for validation. This increased to 89.9% (enhanced) with G-NAF. Among 79,028 admitted and 88,876 emergency patients, 22.7% and 18.4% respectively were aged care residents. We compared performance of raw vs enhanced comparisons were: sensitivity (admission: 34.1% vs 73.8%), (emergency: 42.6% vs 66.2%); specificity (admission: 96.2% vs 90.9%), emergency (96.8% vs 95.2%); Positive Predictive Value (admission: 61.3% vs 58.8%), (emergency: 64.2% vs 65.3%).
Conclusions and ImplicationsOur approach significantly increased sensitivity without sacrificing specificity, enhancing Aged Care resident identification in EHR data. It also improved linkage with external datasets and Statistical Area Level 1 (SA1) mapping.
Comparing the linkage performance of fastLink, Match*Pro and Splink at the Florida Cancer Registry using simulated pseudopeople data
Authors: Alexandersson Anders 
Published: September 2024

ObjectivesThe objective was to compare the record linkage performance of fastLink, Match*Pro, and Splink using realistic simulated "Rhode Island" data from the Python package pseudopeople. A requirement was no expected wrong matches (false positives, FP) after a limited manual review.
ApproachWe incorporated custom noise on (a) Social Security Number (SSN) and (b) address. We first used high-quality data where SSN was available and then low-quality data where SSN was unavailable. Missed matches (false negatives, FN) were acceptable. We used an iterative process of determining the “appropriate” match threshold and the correct matches (true positives, TP).
ResultsThe two analysis datasets, df1 and df2, resulted in 660,227 actual matches and 9,987 actual non-matches. The specified match threshold probability was 0.995 for fastLink, 0.999999999 for Splink, and about 0.99 for Match*Pro. When SSN was available, the (rounded) results were: Splink 627,000 TP (4 FP), fastLink 577,000 TP (15 FP), and Match*Pro 436,000 TP (4 FP) in the default frequency-based EpiLink-like option; the EM option crashed. When SSN was unavailable, the results were: Splink 579,000 TP (3 FP), fastLink 582,000 TP (137 FP), and Match*Pro 442,000 TP (3 FP); the EM option had 516,000 TP (442 FP). The fastLink developers are working on adding Active Learning.
ConclusionOverall, Splink and fastLink performed better than Match*Pro. A likely explanation is that only Splink and fastLink provide match probability and are open source. Splink (Python) and fastLink (R) require basic user programming skills. The results will be updated once the final report has been approved.
Transparent reporting on multi-site studies about algorithms for linked health data: A systematic review of Health Data Research Network Canada’s Algorithms Inventory
Authors: Lix Lisa M , Ahmed Nasiba , Fung Daryl , Al-Azazi Saeed , Hamm Naomi C 
Published: September 2024

ObjectiveMulti-site studies that use routinely-collected, linked health data often rely on validated algorithms to harmonize definitions of health conditions or other population characteristics across sites. If algorithm validation is not practicable, feasibility studies that assess whether an algorithm can be consistently and accurately implemented across sites, are recommended. We investigated the transparency of reporting on multi-site algorithm feasibility studies found in Health Data Research Network (HDRN) Canada’s Algorithms Inventory.
ApproachPublished, multi-site Canadian feasibility studies about chronic or infectious disease algorithms were assessed using an adaptation of the RECORD (REporting of studies Conducted using Observational Routinely collected health Data) and STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) guidelines. Reviewers were trained on a subset of studies. The remainder were independently assessed by two reviewers and quality checks were conducted. Results were descriptively summarized.
ResultsThirty published multi-site algorithm feasibility studies were reviewed; the majority were conducted in four or more sites, and 83% assessed chronic disease algorithms. Almost one-fifth (17%) of the studies did not justify the methods used to assess algorithm feasibility. However, the majority (>95%) addressed potential sources of bias and discussed algorithm generalizability.
ConclusionsWe observed accurate and complete reporting of most elements of multi-site algorithm feasibility studies conducted in Canada. These studies provided important information about availability of data elements, generalizability, and potential algorithm uses.
ImplicationsTransparent reporting of algorithm feasibility studies facilitates algorithm reuse, enhances the credibility and reproducibility of study findings, and promotes collaboration within the data linkage community.
CURL Extensions – Weaving a Unique Property Reference Number into Address Histories in the Scottish Community Health Index
Authors: Clark David 
Published: September 2024

The Community Health Index (CHI) is the patient registration system for the National Health Service (NHS) in Scotland, covering the entire population. CHI has long been used as a population Indexing Spine to facilitate health-based data linkage for both operational and research purposes in Scotland.
Changes in patient’s residential address since 2000 have been captured in a CHI Address History File.
Addresses in CHI are recorded in unstructured free-text format, so property-based analyses and linkage is problematic. In August 2020, 90% of CHI addresses for currently registered patients, and people who had died that year, were seeded with a Unique Property Reference Number (UPRN). This product, called CHI-UPRN Residential Linkage (CURL), enabled NHS Scotland to harness important information on the patient population (5.8 million) and their place of residence, facilitating several data products related to COVID-19.
The current project is extending CURL to UPRN-seed the 31 million address histories of everyone registered with a family doctor in Scotland since January 2000.
The project, due for completion in Summer 2024, will look at alternative address-matching algorithms with the aim of achieving a 95% UPRN-match rate across the longitudinal CURL file.
Quality assurance exercises help us to understand the precision and sensitivity of matching across time and geography. Analysis of how the longitudinal file is constructed across the person and property-level indices permits a deeper understanding of the capabilities and limitations of the CURL product when used to deliver innovative analyses or modelling that link people and places over time.
Implementing a streamlined data access governance process for the release of routinely collected electronic health record data for research
Authors: Parker Emily , Kim Naude , Ravipati Tanya , Beare Richard , Clavarino Lee-Anne , Srikanth Velandai , Andrew Nadine 
Published: September 2024

ObjectivesIn Australia, there are no standardised processes for the release of Electronic Health Record (EHR) data for research. We describe our streamlined ethics and data access governance framework for the release of EHR data for research.
ApproachIn consultation with our academic and health service partner organisations and with reference to national and international standards, we developed a process to safely and efficiently release EHR data for research. The Five Safes Framework guided our processes ensuring: safe people; safe projects; safe settings; safe data; and safe outputs. Piloting of processes has commenced.
ResultsStreamlined data access and release processes acceptable to all stakeholders were implemented using an online system. Independent researcher and consumer review of an Expression of Interest, followed by ethics review/approval from the health service research office ensures projects are methodologically and ethically sound. Completion of a detailed technical assessment form, implementation of a signed data use agreement and release of de-identified data to a secure e-research environment promotes safe data and settings. Constraints around publication are enforced via pre-publication review ensuring safe outputs. Nine projects are in process using our system with an average of 4 days for EOI review and 6 days for TAF review.
ConclusionWe have established a streamlined data access pathway to minimise duplication of effort and allow timely and secure access to EHR research data. We have built in efficiencies and promoted a cultural shift for clinicians who have previously not had data access processes governed in this way.
Developing an integrated HIV clinical and population database in rural south Africa
Authors: Gareta Dickman , Lauren Evelyn , Shumba Khumbo , Herbst Kobus , Onoya Dorina , Bor Jacob 
Published: September 2024

BackgroundHIV cohort data are crucial for understanding the effects of HIV infection and treatment. Linkage between HIV clinical and population records unlocks potential to understand the cause and consequences of social-economic factors on HIV-care seeking behaviors and mortality outcomes. We created HIV data infrastructure by linking HIV treatment records with routinely collected records.
MethodsWe linked HIV cohort data(n=119,434) to Health and Demographic Surveillance(n=256,425), laboratory tests(n=617,314), clinic visits and hospital admissions(n=170,325) from KwaZulu-Natal, an HIV-endemic setting in rural South Africa. We used graph-based probabilistic record linkage algorithm to link the data for HIV patients accessing care at 17 clinics within the Hlabisa sub-district. Africa Health Research Institute (AHRI) has collected health and demographic surveillance data since 2000. AHRI has set up a hospital information system in the district to capture ICD-10 code admissions and discharges.
ResultsFrom the 5% sample of 462,524 linkage clusters, the computed F-score was 0.88, sensitivity was 88% and positive predictive value was 91%. 38.5% of the patient ID clusters contained records linked across multiple databases. 36% of the HIV patient ID clusters contained records linked with laboratory tests. 19.5% of the HIV patient ID clusters contained data on demographic surveillance. 12.5% of the HIV patient ID clusters contained records linked with clinic visits and hospital admissions. 4% of patient IDs contained records across all four databases.
ConclusionWe have successfully created a cross-sectorial HIV cohort through probabilistic record linkage methods. The cohort provides a platform to answer policy-relevant questions.
Prevalence of a dual diagnosis of mental illness and substance use disorder in Ontario, Canada: a retrospective cohort study using linked administrative data
Authors: Young Jesse , Tanner Bryan , Crocker Matthew , Sharpe Isobel , Lu Hong , Kurdyak Paul 
Published: September 2024

ObjectivePopulation-based prevalence estimates of co-occurring mental illness and substance use disorder (herein dual diagnosis) are scarce and derived from a single source (e.g., survey data) which may lead to underestimation of prevalence. We linked administrative data to a representative mental health survey in Ontario, Canada to estimate dual diagnosis prevalence using a data triangulation method.
Approach We retrospectively linked the 2002 and 2012 Canadian Community Health Survey on Mental Health (CCHS-MH) to emergency department, inpatient hospital, and outpatient physician records in Ontario, Canada. Mental illness and substance use disorder were ascertained though self-report, Composite International Diagnostic Interview screening, and linked administrative health records. We estimated 1-year, 5-year, and lifetime prevalence of dual diagnosis.
ResultsOf the CCHS-MH survey participants, 14,790 (99.8%) were included in the study. The 1-year, 5-year, and lifetime prevalence of dual diagnosis was 2.3%, 4.8%, and 9.8%, respectively, which attenuated to 2.1%, 4.1%, and 8.4%, respectively, when tobacco use disorder was removed from substance use disorder ascertainment.
ConclusionDual diagnosis is more common in the general population than previously estimated. Considering barriers to accessing care for both mental illness and substance use disorders among people with dual diagnosis annually, our 5-year prevalence estimate is likely most informative for health system planning.
ImplicationsIn the context of increased health burden, barriers to care access, and challenges to effective treatment associated with dual diagnosis, our findings present a case for increased investment in integrated models of mental healthcare and addiction medicine.
Model-based algorithms to ascertain smoking in administrative health data: a registry-based validation study
Authors: Haque Md Ashiqul , Nickel Nathan C , Turgeon Maxime , Lix Lisa M 
Published: September 2024

Objectives We developed a machine-learning model-based algorithm (MBA) for smoking in Administrative Health Data (AHD). The validity of this MBA was compared to a rule-based algorithm (RBA).
Approaches The study included adults (≥18 years) from a clinical registry containing self-reported current smoking from 2017 to 2020 in Manitoba, Canada. Clinical data were linked to up to five years of hospitalization, physician billing claims, and prescription medication records. The RBA was based on diagnosis codes for tobacco use and nicotine dependence medication. MBAs, constructed using random forest (RF) models, included these indicators in addition to comorbid condition diagnoses and sociodemographic factors. Sensitivity, specificity, positive and negative predictive values (PPV, NPV), and 95% confidence intervals (CIs) were estimated.
Results The cohort comprised 24,718 individuals (88.6% female); prevalence of current smokers was 10.0%. The RBA had sensitivity of 27.3% (95% CI: 24.2-30.7), specificity of 96.6% (95% CI: 96.1-97.0), and PPV of 47.2% (95% CI: 42.9-51.5). The MBA had sensitivity of 68.6% (95% CI: 65.1-71.9), specificity of 76.3% (95% CI: 75.2-77.3), and PPV of 24.3% (95% CI: 23.2-25.6). NPV was high irrespective of algorithms. Stratified analyses revealed similar estimates for males and females, and the number of years of AHD did not affect the MBA results.
ConclusionsAn RF-based MBA for smoking ascertainment in linked AHD sources improved sensitivity compared to the RBA. However, the RBA excelled in specificity and PPV.
ImplicationBalancing accurate smoker identification with the risk of false positives is crucial when choosing an algorithm to ascertain current smokers using AHD.
A proof-of-concept minimum data set for older care home residents: what value does this add?
Authors: Tracey Freya , Crellin Elizabeth , De Corte Kaat , Lloyd Therese , Brine Richard 
Published: September 2024

ObjectiveDeveloping resources And minimum data set for Care Homes’ Adoption (DACHA) is a four-year NIHR-funded project to establish what data need to be in place to support research, service development and innovation uptake in care homes for older people. An aspirational minimum data set (MDS) specification was identified, which included data from several sources and pilot variables.
Approach767 care home residents (with a valid NHS number recorded) across three integrated care systems consented to linkage of their care home record with administrative health and social care records from NHS England. Data were linked using pseudonymised NHS number. Variables were derived to summarise healthcare utilisation.
Where variables are sourced from more than one data set, expert opinion on quality across these data sets informed hierarchical selection. We reported descriptive statistics including missingness across each of the >300 variables in the MDS, with information on how these were accessed and derived, where appropriate. Descriptive summaries were informed by stakeholder engagement and cover ambulance and emergency department utilisation.
An ongoing DACHA PPIE panel contributed to MDS development through sharing opinions on data access and linkage, and best sources of information across different domains.
ResultsAnalysis is in progress. All results will be in pre-print publication by 31 April 2024.
ConclusionsThe MDS constructed gives rich insights on healthcare utilisation and resident needs, with opportunity to understand how these interrelate.
ImplicationsAs a proof-of-concept study, the findings of this research will inform national decision making on future linked care home data sets.
Building access to linked data for program evaluation: lessons and opportunities from the evaluation of a pan-Canadian skills training initiative
Authors: Palamar Max , Nkusi Sandra 
Published: September 2024

In recent years, Canada’s statistical agency (Statistics Canada) has developed sophisticated linkage infrastructure through their Social Data Linkage Environment (SDLE), linking internal data sets held by Statistics Canada and external data held by researchers. While the SDLE presents an opportunity for better measurement of the effectiveness of policies and programs, to date it has not been used for this purpose outside of government, instead largely supporting academic research projects. This session presents a novel approach to program evaluation in the Canadian context, using the SDLE Statistics Canada administrative data alongside independently collected program data from multiple service providers to evaluate the outcomes and effectiveness of a portfolio of skills training programs.
Our approach aims to both rigorously evaluate the long-term outcomes of participants in this portfolio of public skills training programs, and develop a replicable proof-of-concept for using linked data infrastructure in Canada to support program evaluations. Through this initiative, we identify the key design features needed to support this use case for linked data, including study design parameters, necessary datasets and linkage processes, and privacy and data governance policies. Finally, we identify opportunities for future replication of this approach in Canada, including strategies for expedited linkage and analysis of evaluation results that maintain privacy requirements.
We find that using the SDLE infrastructure for program evaluation is both feasible and desirable, however barriers exist to replicating this approach, including sectoral capacity for collecting linkable data, and transposing linkage and confidentiality rules from an academic research to a program evaluation context.
Barriers and enablers to secondary use of multi-regional linked administrative health data for three distinct user groups: academic researchers, health-system knowledge users, and private sector researchers
Authors: Duchen Raquel , Curtis Maillet Donna , McDonald Ted , Gately Jodi , Gale Keltie , Ho Minnie , Schull Michael , Caetano Trish , Hayes Anne 
Published: September 2024

Objective and ApproachWe analyzed barriers and enablers to accessing pan-Canadian health sector data and data analytic services for three distinct user groups: 1) academic researchers, 2) health-system knowledge users (KUs), and 3) private sector researchers. Specifically, we a) conducted a legislative and policy scan regarding use of health administrative data for research, quality improvement, and health system management and b) systematically consulted with data centre directors and staff, privacy and legal experts, knowledge users (KUs), private sector entities, and university and government officials.
ResultsKUs and private sector organizations face unique barriers because structures and regulations related to data access and use are set at multiple university and government institutions and are primarily designed for academic research. Additionally, data repositories often have specific policy/legislative barriers to working with KUs and/or private sector researchers. However, all groups face common issues including variations in regional regulations and policies making harmonized and streamlined processes challenging; lengthy data access and ethics review processes; and regional variations in data availability. Enablers included diverse expertise and initiatives across Canada such as: initiatives to streamline ethics reviews for multi-regional research, expand data holdings, increase standardization and harmonization, and implement systems for federated analyses.
ConclusionsAccess to health system data and analytics in Canada varies by user group and across regions. Although each user group faces unique challenges, they also all have many barriers and enablers in common.
ImplicationsImproving health system data access for one use group has the potential to benefit the others.