SlowGuess/NeurIPS_2023_OCR huggingface dataset:
MEDSAT: A Public Health Dataset for England Featuring Medical Prescriptions and Satellite Imagery
Sanja Šćepanović<sup>1,2†*</sup> Ivica Obadic<sup>2,3*</sup> Sagar Joglekar<sup>1,6‡</sup> Laura Giustarini<sup>5</sup> Cristiano Nattero<sup>5</sup> Daniele Quercia<sup>1,4</sup> Xiao Xiang Zhu<sup>2,3</sup>
$^{1}$ Nokia Bell Labs  $^{2}$ Technical University of Munich  $^{3}$ Munich Center for Machine Learning  $^{4}$ Centre for Urban Science and Progress, King's College London  $^{5}$ WASDI  $^{6}$ Intercom
As extreme weather events become more frequent, understanding their impact on human health becomes increasingly crucial. However, the utilization of Earth Observation to effectively analyze the environmental context in relation to health remains limited. This limitation is primarily due to the lack of fine-grained spatial and temporal data in public and population health studies, hindering a comprehensive understanding of health outcomes. Additionally, obtaining appropriate environmental indices across different geographical levels and timeframes poses a challenge. For the years 2019 (pre-COVID) and 2020 (COVID), we collected spatio-temporal indicators for all Lower Layer Super Output Areas in England. These indicators included: i) 111 sociodemographic features linked to health in existing literature, ii) 43 environmental point features (e.g., greenery and air pollution levels), iii) 4 seasonal composite satellite images each with 11 bands, and iv) prescription prevalence associated with five medical conditions (depression, anxiety, diabetes, hypertension, and asthma), opioids and total prescriptions. We combined these indicators into a single MEDSAT dataset, the availability of which presents an opportunity for the machine learning community to develop new techniques specific to public health. These techniques would address challenges such as handling large and complex data volumes, performing effective feature engineering on environmental and sociodemographic factors, capturing spatial and temporal dependencies in the models, addressing imbalanced data distributions, developing novel computer vision methods for health modeling based on satellite imagery, ensuring model explainability, and achieving generalization beyond the specific geographical region.
The impact of environmental factors on human health has gained significant attention in recent years, particularly in the face of increasing environmental pressures caused by extreme weather events. Understanding these impacts is crucial for effective public and population health interventions. However, existing studies often face challenges in obtaining appropriate health and environmental data.
Fine-grained and comprehensive data on the prevalence of medical conditions is often scarce in public health studies. Traditional approaches rely on infrequent surveys or cohort studies, such as NHANES [23], HSE [46], BRFSS [52], The Swiss National Cohort [60], or the UK Biobank [61]. However,
\*Shared first authorship. The work was done during the author's AI4EO Beyond Fellowship at the Technical University of Munich.  $\ddagger$ The work was done prior to the author's tenure at Intercom.
image features
environmental features
outcomes
Figure 1: Structure of MEDSAT dataset (single year): This figure illustrates the four data components constituting MEDSAT: i) sociodemographic features (111), ii) environmental point features (43), iii) image features (4 seasonal Sentinel-2 composite tiles  $\times$  11 bands each), and iv) prescription outcomes (prevalence scores for 7 medical prescription types). The distributions of example variables are shown. Sociodemographic variables are presented as percentages ranging from 0 to 1, while environmental variables have varying ranges (e.g., NO2:  $(0 - 3.1\times 10^{-3})mol / m^2,\mu = 2.43\times 10^{-6},$ $\sigma = 2.0\times 10^{-4}$ ). Outcome variables represent yearly prescription quantities per capita and are mostly normally distributed (except opioids and total prescriptions). For instance, diabetes prescriptions range from 0.02 to  $104.84$  ( $\mu = 38.76$ ,  $\sigma = 16.35$ ). Each Sentinel-2 composite image consists of 11 spectral bands. MEDSAT offers two such yearly snapshots, for 2019 and 2020. A comprehensive description can be found in the Appendix.
survey methods suffer from biases related to sampling, non-response, recall, and question wording. Cohort studies, while aiming to mitigate these biases, are limited in size, expensive, time-consuming, and prone to participant dropout. The All of Us Research Program [58] is an ambitious initiative recruiting over 1 million participants, but its representativeness and long-term engagement remain to be seen. In summary, existing health outcomes data are often limited in scope, granularity, and subject to various biases. Furthermore, despite the increasing availability of finer-resolution measurements for crucial environmental indicators relevant to public and population health, such as greenery, sun radiation, and air pollution, challenges persist in obtaining comprehensive and suitable indices that cover diverse geographical levels and timeframes. While the Earth Observation (EO) community has made significant efforts in capturing detailed satellite imagery with improved resolutions, frequencies, and accessibility, there remains a gap in transforming this vast amount of data into user-friendly indices that can be effectively utilized by non-technical stakeholders and the wider community unfamiliar with EO methods. Even when institutions provide data, such as air quality data from DEFRA [20] in the UK, individuals interested in compiling various environmental information often need to collect it from multiple sources, and there are spatial and temporal limitations to the available data.
In this paper, we present the MEDSAT dataset (Figure 1) consisting of four complementary components and covering two years (2019 and 2020), specifically designed for studying the effects of the environment on population health in small administrative areas in England. Our approach involves an open-source framework that utilizes National Health Services (NHS) practice-level prescription data to extract medical prescription prevalences at fine spatial (i.e., Lower Layer Super Output Area (LSOA)) and temporal (seasonal) scales for the entire population of 57 million. We derived environmental indicators from satellite products, such as Sentinel-5 and OMI, using Google Earth Engine [30]. Additionally, we created cloud-corrected seasonal composite images for 11 spectral bands of multispectral instrument (MSI) Sentinel-2 images in the WASDI platform [67], covering the whole of England, i.e.,  $130,279\mathrm{km}^2$ . Our dataset also includes 111 socioeconomic indicators
Table 1: A comparison of MEDSAT with similar datasets, some of which are not publicly available. NHS corresponds to the UK health care system and DHS to the Dutch healthcare system.
dataset	health indicator(s)	indicator source	imagery	env.	soc.	spatial unit	public
SustainBench [69]	BMI, child mortality, water quality, sanitation	surveys	Landsat street view	X	X	village (59km2)	✓
Landscape Aesthetics [39]	environment scenicness	crowdsourcing	Sentinel-2	✓	X	1.6km2	✓
COVID-19 [62]	COVID-19 cases and deaths	WHO	X	✓	✓	city	X
Greenery & mortality [7]	mortality	Eurostat	X	✓	✓	city	X
Greenery & prescribing [31]	antidepressants prescriptions	DHS	X	✓	✓	municipality (up to 506km2)	X
Nat. env. & prescribing [27]	mortality, prescriptions: cardiovascular antidepressants	NHS	X	✓	✓	LSOA (up to 18km2)	X
MEDSAT	prescriptions: respiratory (asthma) metabolic (diabetes, hypertension) mental (depression, anxiety), opioids, & total	NHS	Sentinel-2	✓	✓	LSOA (up to 18 km2)	✓
obtained mostly from the UK census. By integrating these diverse datasets for the years 2019 and 2020, we provide researchers with a comprehensive resource for studying spatial and temporal health attributes and identifying regional health disparities.
Environmental conditions such as air or noise pollution are relevant indicators for various health issues like asthma or heart diseases [22]. However, the benefits of using EO data to monitor the impact of environmental conditions on human health are still limited. This limitation stems from the existing datasets in the literature listed in Table 1 that either focus on a narrow set of conditions derived from surveys that might not be representative of the entire population or do not provide detailed medical prevalence data on a fine-grained spatial level. Often these datasets are not publicly available and fail to include environmental and sociodemographic features relevant to health studies. For example, the SustainBench dataset [69] presents the problem of predicting 4 different health indicators derived from surveys based on Landsat satellite imagery and street-view images. The spatial unit of this study corresponds to a village or a local community covering an area of  $\approx 58\mathrm{km}^2$  and this dataset does not contain additional environmental and sociodemographic features. By considering a more abstract health indicator, Levering et al. [39] introduced a dataset that associates Sentinel-2 images with crowdsourced data for landscape scenicsness used as a proxy for human health and well-being. By analyzing fine-grained geographical regions of  $1.6\mathrm{km}^2$ , the authors discovered plausible associations between a landscape's beauty and its land cover distribution. Targeting a specific condition across the population, Temenos et al. [62] assembles a dataset that relates COVID-19 cases with point features describing environmental data for urban greenness, air quality, sociodemographic features, and health factors. The authors reveal that temperature and mobility trends are among the most important features for predicting COVID-19 cases. Yet, this dataset is not publicly available, does not contain any imagery and the point features represent entire cities, thus describing very coarse spatial units. Further, the impact of the natural environment on mortality rates and prescriptions for various conditions is investigated in [7, 31, 27]. However, the datasets used in these studies are also not publicly available, include only environmental variables related to greenery, and do not contain any imagery. Moreover, the analyses in [7, 31] are performed on larger spatial units like municipalities and cities.
In comparison to these works, our dataset, MEDSAT, enables comprehensive modeling of the population health on a very-fine-grained spatial level as it jointly offers environmental and sociodemographic features as well as EO imagery at LSOA level. Further, it covers prescriptions associated with 5 medical conditions, as well as opioids and total, which allow a detailed understanding of specific condition-related factors, and shed light on the overall population health and well-being.
3 The MEDSAT Dataset
The MEDSAT dataset serves as a comprehensive resource for public and population health studies, encompassing medical prescription quantity per capita as outcomes and a wide array of sociodemographic, environmental and image features across 33K LSOAs in England (Figure 1). In this release,
we provide data snapshots for the years 2019 (pre-COVID) and 2020 (COVID). Sociodemographic variables align with the latest UK census from 2021. Figure 2 visualizes examples of variables present in MEDSAT.
Access the code at https://github.com/sanja7s/MedSat, and the dataset at https://doi.org/10.14459/2023mp1714817. The dataset is released under the CC BY-SA 4.0 license.
3.1 Sociodemographic Features
Our dataset comprises sociodemographic variables sourced from the latest UK Census in 2021 [1]. These variables encompass essential indicators employed in public and population health research, such as gender (percentage of males), age distribution (percentage within 5-year age groups up to 85 and above), deprivation scores (percentage of deprived households in 1-4 dimensions), self-reported health (percentage reporting health on a five-point scale), ethnicity (percentage of individuals with White, Asian, Black, or Mixed backgrounds), and English proficiency (percentage reporting English as their main language). Additional variables indirectly related to health outcomes were incorporated, covering religion, commute means and distance to work, residence and housing, profession, and marital status. Our sociodemographic data do not contain any personally identifiable information because census implements stringent privacy protection measures, including targeted record swapping and cell key perturbation, to ensure confidentiality without compromising aggregated statistics [1].
3.2 Environmental Features
We obtained environmental point features for the MEDSAT dataset using various satellite data products on Google Earth Engine (GEE) [30]. For air quality, we used satellite data products such as Sentinel-5P NRTI to derive nitrogen dioxide (NO2) [32], TOMS&OMI for ozone [4], and CAMS for total aerosols and PM2.5 [25]. Greenery variables were derived from Sentinel-2 MSI for NDVI and ERA5-ECMWF product for high/low vegetation greenery indices. Climate variables, including wind components, air temperature, soil temperature, atmospheric pressure, and incoming solar radiation, were obtained from ERA5-ECMWF. All land cover variables were sourced from Google Dynamics World product [11]. Importantly, our open-source code can be easily adapted to extract other similar indices, and at different spatial scales (e.g., wards or other countries) and temporal scales (e.g., monthly or different years). More information can be found in Appendix section D.2.
3.3 Image Features
In addition to the above-described approach for deriving preprocessed environmental point features, in our dataset we also included the spectral bands provided by the Sentinel-2 mission [3], thus resulting in a more comprehensive set of environmental image features. Concretely, we processed Sentinel-2 images for the years 2019 and 2020 through the WASDI [67] platform by calculating average values for each of the four meteorological seasons [49]. We focused on 11 specific bands (Figure 1) capturing relevant environmental factors, excluding the band B10 that is primarily used for cloud detection and water vapor mapping. To ensure data consistency and quality, we resampled the bands to a uniform resolution of  $10\mathrm{m}$ , applied cloud masks to exclude affected pixels, and computed pixel-per-pixel averages over time under cloud-free conditions. The resulting composite images represent the typical environmental characteristics for each season (over 500 GB per year in total).
To explore the potential of the high-resolution Sentinel-2 imagery for modeling population health on a fine-grained spatiotemporal level, we extracted image features per LSOA for each meteorological season with the procedure depicted in Figure 4 in Appendix. Concretely, we used the shapefiles that describe LSOA's geographical location to crop the LSOA pixels from a seasonal Sentinel-2 image. Next, we extracted basic image features from the cropped LSOA pixels by performing the following 5 aggregations per Sentinel-2 band: mean, stdev, min, max, and median, thus resulting in 55 image features per LSOA per season.
3.4 Prescription Outcomes
Extracting Prescriptions on an LSOA Level from NHS data. The monthly practice-level prescribing data in England, provided by the National Health Services (NHS) since July 2010 [47],
Figure 2: Visualization of the MEDSAT point features. The example distributions for the year 2020 of three health outcomes (diabetes, anxiety, and total prescriptions), three environmental variables (NO2, north-south wind component, and incoming thermal radiation), and three sociodemographic variables (percentage of White population, professional occupation, and work-from-home). The missing values are highlighted in black. Depending on the specific analyses intended, the missing value rate will be constrained by the outcomes, standing at  $5.7\%$ , and ranging up to  $15.2\%$  if all the features are to be used. Notably, we possess sociodemographic and image features data for all LSOAs. Please refer to Appendix for the details.
constitutes the foundation of our analysis. It offers anonymized information about monthly prescriptions across General Practitioner (GP) practices and patient membership to GP practices on an LSOA level. We parse this data to extract the prescribed drugs and summarize the total number of patients per LSOA and compute the fraction of a GP practice's patients associated with a specific LSOA. Further details about the NHS prescriptions data and the applied procedure for calculating the number of patients per LSOA can be found in the Appendix, Section D.4.
Associating Prescriptions with a Condition. To determine prescriptions related to specific medical conditions, our framework utilizes curated lists of drugs, such as the one collated for opioids by previous works [59, 18], or it leverages DrugBank [37] to automatically identify drugs associated with a given condition (see the Appendix DrugBank section D.4.3). DrugBank is an online database that provides comprehensive information on active pharmacological ingredients (APIs) and their corresponding conditions. Each drug name is associated with one or more conditions (i.e., symptoms and diseases), drug categories, and an Anatomical Therapeutic Chemical code assigned by the World Health Organization (WHO) for unique identification purposes. For instance, the drug name Citalopram (https://go.drugbank.com/drugs/DB00215) is linked to a range of diseases, including Depression, Anorexia Nervosa, Generalized Anxiety Disorder, and Post Traumatic Stress Disorder. During our crawl, we obtained data on 9,105 drug names from the website, and by filtering out drug names that were not linked to any drug categories, symptoms, or conditions we were left with 3,013 drug names. Next, we generated a curated list of drugs associated with a specific condition by selecting drugs from DrugBank that were linked to that condition (e.g., we associated Citalopram with depression, anxiety, and the other conditions mentioned above).
Estimating the Prescription Prevalence. To estimate the number of prescriptions associated with a specific condition  $c$ , we first matched the condition-specific drugs from DrugBank with their British National Formulary (BNF) codes from the NHS prescribing dataset. Next, the number of prescriptions for a specific condition  $c$  in area  $a$  is computed using the following formula:
$$
N _ {c} (a) = \sum_ {G P \in a} N _ {c} (G P) \cdot f (G P, a), \tag {1}
$$
Here,  $N_{c}(GP)$  represents the total number of prescriptions per GP for drugs associated with the curated list for the condition, and  $f(GP, a)$  denotes the fraction of patients of the GP who reside in the area  $a$ . To ensure comparability across areas with varying population densities, we computed the metric of "prescriptions quantity per capita," commonly used in medical studies [19, 18], as follows:
$$
\tilde {N} _ {c} (a) = \frac {N _ {c} (a)}{n _ {p a t} (a)}, \tag {2}
$$
where  $n_{pat}(a)$  corresponds to the total number of patients residing in the area  $a$ .
For MEDSAT we calculated medical prescriptions associated with three classes of conditions: i) metabolic (diabetes and hypertension), ii) mental (depression, and anxiety), and iii) respiratory (asthma); as well as opioids prescriptions, (which are predominantly prescribed for pain management, but they do have other applications, and have been associated with a consumption crisis in the UK [53, 56]), and total prescriptions, as a proxy for general health and well-being. We highlight that, for simplicity, we use condition names to refer to related prescriptions, however we cannot know for each individual prescription what was the exact cause for which it was prescribed. E.g., "depression prescriptions" means antidepressants and "anxiety prescriptions" means anxiolytics, regardless of actual use. Co-prescriptions across conditions may arise from this method, as it does not ascertain specific prescription reasons, as such details are absent in the NHS dataset. However, the multitude of studies examining prescriptions [59, 18, 8, 41, 34, 33, 31, 44, 35, 63, 12, 66, 29], akin to our approach, attests to its significance as a public health outcome.
Our prescription dataset does not contain any personally identifiable information, as it is derived from publicly available monthly prescription data provided at the level of practices, each serving numerous patients.
4.1 Revealing Health Inequalities
In Figure 8 in Appendix, we present the healthcare accessibility disparities across regions. First, interestingly, we find a prevailing pattern where the number of registered patients exceeds the census population in most areas of the country. This aligns with previous investigations by UK authorities [64]. Second, although the correlation  $(r = .87, p \approx 0)$  is strong, certain LSOAs exhibit disproportionate patient-to-population ratios.
Additionally, our analysis highlights broader factors contributing to healthcare inequalities. The residual values, representing deviations from the linear fit of the patient to the population numbers, correlate with deprivation levels ( $r = .16$ ,  $p \approx 0$  for mid-deprived areas;  $r = .22$ ,  $p \approx 0$  for highly-deprived areas), suggesting a greater burden on healthcare access in socioeconomically disadvantaged regions. Moreover, the residual values exhibit a negative correlation ( $r = -.40$ ,  $p \approx 0$ ) with the percentage of White population, indicating disparities associated with ethnic backgrounds. For more details on this analysis, please refer to Section E.1 in Appendix.
4.2 Predicting Prescriptions
To evaluate the plausibility of our dataset for modeling population health, we applied a classical geostatistical method called Spatial Lag Model (SLM) [5] as well as trained the LightGBM machine learning model [36] to predict the medical prescriptions based on the point features, including image-derived ones, in our dataset. Both models were applied separately for every condition and a combination of the environmental, sociodemographic and image features, to better understand the contribution of different input features in modeling population health.
Table 6 in Appendix displays the SLM results. Collectively, the input features account for a variance ranging from  $38\%$  (for diabetes and total prescriptions) up to  $63\%$  (for opioids). Individually, sociodemographic features lead the way, explaining between  $31\%$  (total) and  $52\%$  (opioids) of the variance. They are followed by environmental point features, which account for variances from  $13\%$  (diabetes) to  $49\%$  (opioids). Image features, though least impactful, still cover a variance from  $4\%$  (diabetes) to  $28\%$  (opioids).