SlowGuess/NeurIPS_2023_OCR huggingface dataset:
While the models such as SLM account for the well-known spatial autocorrelation effects [5] present in spatial analysis and modelling (referring to the process that creates clusters of values), machine learning models, such as LightGBM, require a special type of cross-validation that is adapted to account for these effects [55]. A block-buffered cross-validation is a common approach [38], and it is implemented in an R package called blockCV [65]. We employed this package to calculate spatial folds on the input of our LSOA shapefiles. For the details, please refer to Appendix E.2. The results obtained through spatial cross-validation using LightGBM are detailed in Table 2. In the initial row, it is evident that even fundamental image features exhibit predictive capability, albeit to a limited extent. Conversely, the subsequent two rows highlight that environmental and sociodemographic features offer improved explanatory power for the observed variances, outperforming image-based features. Furthermore, these feature categories display varying degrees of importance across different conditions. Environmental attributes notably enhance the predictive accuracy of depression, opioid prescriptions, asthma, and total prescriptions. Conversely, sociodemographic features prove more effective in accurately forecasting prescriptions for other medical conditions. Notably, the integration of both environmental and sociodemographic characteristics becomes pivotal for a holistic model of population health. This is exemplified by the last row, indicating that using both, the environmental and the sociodemographic features results in the best  $R^2$  scores for all conditions under consideration. Moreover, for both the SLM and the LightGBM model, we observe a consistent pattern that the prescriptions for the mental conditions are predicted with higher accuracy than the ones for the other conditions. In Appendix Section E.2, we present a detailed overview of this experiment's setup and provide a comparison of the LightGBM model with a Feed-Forward Neural Network (FNN), which shows that the LightGBM model consistently outperforms the FNN.
Table 2: The average  $R^2$  scores resulting from the 5-fold spatial cross-validation of LightGBM. These scores are computed across various prescription types and combinations of dataset features specifically for the year 2020.
input	metabolic	mental	respiratory	total
diabetes	hypertension	depression	anxiety	asthma	opiods
Image	0.02 ±0.07	0.15 ±0.12	0.15 ±0.15	0.14 ±0.15	0.14 ±0.13	0.19 ±0.14	0.07 ±0.1
Env.	0.19 ±0.08	0.33 ±0.13	0.42 ±0.15	0.41 ±0.13	0.37 ±0.11	0.52 ±0.12	0.26 ±0.1
Soc.	0.26 ±0.1	0.37 ±0.11	0.41 ±0.15	0.39 ±0.12	0.32 ±0.13	0.47 ±0.14	0.22 ±0.11
Env. + Soc.	0.35 ±0.08	0.44 ±0.1	0.50 ±0.13	0.48 ±0.12	0.43 ±0.1	0.6 ±0.1	0.31 ±0.1
Uncovering Health Factors Understanding the impact of the environment and sociodemographic conditions on human health has been a focus of many studies [14, 2]. Our dataset offers new perspectives on such studies, as it allows the investigation of these relationships on a fine-grained spatial level, and for many conditions simultaneously. To shed light on these perspectives, we apply the SHAP [42] approach on the LightGBM models trained for prescription prediction. In Figure 3 we show the 10 most important features estimated by the SHAP algorithm for the models used to predict diabetes and total prescriptions. The left plot shows that the sociodemographic indicators describing the occupation, commute habits, migration, and ethnicity are highly relevant for predicting diabetes prescriptions. Specifically, the model establishes a linkage between lower prescription rates and LSOAs characterized by a substantial prevalence of professional occupations, a high number of people working from home, an active bicycle commuting trend, and a prominent student population. Conversely, increased prescription levels are associated with LSOAs having a high proportion of individuals of Asian ethnicity. When it comes to the environmental features, the east-west wind component and PM2.5 appear to be relevant indicators as higher diabetes prescriptions align with LSOAs characterized by an eastward wind pattern and heightened air pollution levels, as indicated by the PM2.5 metric. Additionally, bare soil evaporation and ozone stand out as noteworthy contributors to diabetes prescriptions even though these features exhibit no straightforward linear association with their corresponding SHAP values. On the other hand, in the right plot, we note that the environmental features describing canopy evaporation, NO2, east-west wind, and thermal radiation rank among the
Figure 3: SHAP summary plots for diabetes (left) and total prescriptions (right). The SHAP value for a feature indicates its contribution towards the difference between the prediction for an instance and the average model prediction. These plots reveal the 10 most important features for both conditions and the association between the feature values and their importance. In line with the results shown in Table 2, the sociodemographic indicators appear to be more relevant for modeling diabetes prescriptions than the environmental features while the opposite is observed for the total prescriptions. Although a similar set of features are ranked among the most important for both conditions, some features are particularly relevant for specific conditions, such as work from home and Asian ethnicity for diabetes prescriptions and thermal radiation and Mixed ethnicity for the total prescriptions.
top-5 most relevant features for estimating total prescriptions. Concretely, high values for canopy evaporation, NO2, and thermal radiation are negatively correlated with the total prescriptions while east-west wind displays a similar association as for the diabetes prescriptions. With respect to sociodemographic features, LSOAs characterized by mixed ethnicity are associated with a lower number of total prescriptions compared to the ones where the majority of the people are born in the UK. Moreover, we also see that low prescriptions are again associated with a high percentage of students and professional occupations. Finally, for both conditions, we notice that the positive self-assessment of health is linked to lower prescription values. Examples of SHAP values for the other conditions as well as dependence plots describing the feature interactions are provided in Sections E.2 in the Appendix.
Describing Health of Environment Using Visual Concepts. To shed light on the potential benefits of using the Sentinel-2 imagery for modeling population health, we reveal the learned visual features patterns in Figure 4 by visualizing examples of LSOA Sentinel-2 images for which the LightGBM model trained on the simple image features closely approximates the actual opioids prescriptions. The LSOAs were visualized with the band combination (B11, B06 and B01) as these bands appeared among the most salient image bands according to the SHAP values for the LightGBM model shown in Appendix, Figure 13. First, we note that although the simple image features do not encode the size of an LSOA, they still enable the LightGBM model to associate higher opioid prescriptions with LSOAs covering larger geographical areas. Larger-area LSOAs are rural (because these administrative units are designed to have roughly equal populations), and it is known from previous research that opioid consumption is higher in rural areas [16]. Equally important, we also note that the LSOAs with high prescription values in the first row are characterized by a stronger presence of blue and pink colors occurring near traffic roads than those LSOAs with low opioid prescriptions in the bottom row. Due to the chosen band combination that displays the aerosols (B01) band in blue, this finding points out that the model can relate increased opioid prescriptions for LSOAs exposed to air pollutants. In conclusion, as observed in Table 2 and Table 6 in the Appendix, the environmental and sociodemographic features explain a higher percentage of the variance for prescription predictions. However, this analysis underscores that even basic image features can offer valuable insights into the intricate task of population health modeling. This suggests that leveraging the deep learning methodologies for processing the Sentinel-2 images at the LSOA level has great potential to improve
Figure 4: Visualizing LSOA Instances in the (B11, B06, B01) band combination. The short-wave infrared band (B11) is shown in red, the near-infrared band (B06) is shown in green and the aerosols band (B01) is shown in blue. The LSOAs with high and low opioid prescriptions are shown in the first and the second row, respectively. Remarkably, LSOAs with high opioid prescriptions cover a larger geographical area than those with low prescriptions (notice the higher zoom-in level), and have greater presence of aerosols (band B01) depicted in blue and purple colors.
prescription predictions by capturing the spatial dependencies inside an LSOA. Moreover, using the recent works in eXplainable Artificial Intelligence (xAI) such as [26] can portray environmental health through rich visual concepts, thus opening possibilities for novel insights about the relevant urban and rural structures influencing population health.
4.3 Temporal Analyses
Using MEDSAT, we analyzed temporal differences in outcome and environmental features between 2019 and 2020. Appendix Figures 15 and 16 illustrate varied distributions for both prescription quantities and environmental features. Notably, in the first COVID year (2020), there was a rise in prescriptions for anxiety and depression (in line with reports that the pandemic presented enormous challenges to mental health services in UK [13]) and diabetes medications in England, while asthma and hypertension prescriptions decreased [13].
The environmental shifts during the initial COVID year are evident from altered air pollutant distributions. Satellite data showed decreased levels of NO2, ozone, and PM2.5 in England for 2020 (as reported erlier in [57]). Land cover changes saw increased built areas, likely due to heightened construction activity in the latter half of the year [24, 21], and reduced trees cover. Furthermore, 2020 witnessed elevated temperatures, solar radiation, and both components of wind compared to 2019.
5 Impact, Limitations, and Perspectives
We introduced MEDSAT, a unique dataset providing a comprehensive view of medical prescriptions, average yearly environmental indicators, image features, and sociodemographic factors across England for 2019 (pre-COVID) and 2020 (COVID). This resource enables a thorough assessment of health status for various conditions and exploration of their relationships with sociodemographic and environmental factors.
MEDSAT has three significant impacts. First, it has the potential to empower the development of novel machine learning (ML) approaches tailored for spatially-autocorrelated public health data [45], that can augment still predominant traditional statistical models like spatial linear regression [6] and BYM [9], as recent work indicates for xAI models [40]. MEDSAT enables ML research with large and complex data, effective feature engineering, capturing temporal dependencies, addressing
imbalanced data, ensuring interpretability, and achieving generalization across diverse regions. Secondly, MEDSAT can facilitate novel discoveries in public health by revealing influential factors that profoundly affect health outcomes. Through SHAP analyses, we confirmed the established link between diabetes and ethnicity, with higher prevalence among people of Asian descent [15, 28], and the preventive effects of biking and active commuting against diabetes and metabolic conditions [54]. Notably, our data from the initial year of the COVID-19 pandemic highlights the impact of socioeconomic factors. Higher percentages of professional occupations and individuals working from home are associated with lower prevalence of diabetes and total prescriptions, underscoring the influence of deprivation on health outcomes [10, 51, 68, 43]. Our findings not only confirm existing knowledge but also expose less-explored connections between the environment and human health. For instance, our SHAP results demonstrated associations between ozone exposure and mental health prescriptions, as well as between total aerosols and metabolic condition prescriptions. Furthermore, a north-sound wind is linked to a decrease in both types of prescriptions. Thirdly, MEDSAT enables groundbreaking discoveries in population health, particularly regarding health inequalities. Our preliminary analysis uncovers disparities in health accessibility among different economic and ethnic groups. By examining deprivation dimensions such as income, education, and occupational factors, across prescriptions of different types, we can gain a deeper understanding of their contributions to health disparities.
Although MEDSAT is among the most comprehensive publicly-available public and population health datasets to date, it is not without limitations. First, prescription prevalence may not always reflect the true prevalence of the medical condition itself. That is because disparities in healthcare access, privilege, knowledge, and stigmatization can influence prescription rates for certain conditions among different populations [50, 48, 17]. However, it is crucial to note that despite this limitation, our dataset offers a unique opportunity to disentangle these effects, especially when combined with other types of health outcome indicators. Compared to surveys and population samples, which come with their own set of biases, MEDSAT provides a more comprehensive health outcome perspective. Moreover, our method of estimating prescriptions using a probabilistic framework, particularly for the four conditions for which we associated drugs using DrugBank, is imperfect. There exists a possibility that we missed certain drug names, or that medications designed for alternate conditions could potentially be inaccurately included. This limitation of our study arises from our labeling method for prescribed drugs. Drugs are labeled according to associated conditions as sourced from the DrugBank database, without claiming any specific intent behind the prescription from the GP. While we can ascertain that a drug is likely prescribed for a given condition, it is worth noting that drugs can be associated with multiple conditions, both as per DrugBank, and in prescriptions by a GP. This multi-condition association increases the chances of co-prescriptions in our dataset. However, numerous studies on prescription patterns, from antihypertensive [34], to antidepressants [44] to anxiolytics [35] highlight the importance of prescriptions as a health outcome per se and its significance in the field of public health outcomes. We also emphasize that the initial drug list output by DrugBank can be augmented with human expert knowledge in a mixed-method approach to ensure the most accurate results. Our analyses show that the correlations between prescription prevalence scores derived with the automatic and manual methods range from .94 (for anxiety) to .99 for diabetes (see Appendix section D.4.3). Second, our dataset exclusively covers England, representing a single developed country. It is noteworthy that England provides high-quality data on both prescriptions and auxiliary census information, and the methods and insights derived from MEDSAT can serve as a foundation for the development of ML approaches that can subsequently be applied to developing countries once high-quality data becomes available, fostering progress towards tracking SDG about health. Third, there is the risk of stigmatizing certain communities based on our dataset. We believe that listed benefits and opportunities offered by MEDSAT outweigh this risk, and we invite ML and health research communities to employing ethical considerations, fostering inclusivity, and ensuring that the insights gained from MEDSAT are used to enact positive change, promote equity, and reduce health disparities.
Project supported by AI4EO Beyond Fellowship at The International AI for Earth Observation Future Lab at TUM/DLR, ESA Network of Resources Initiative, German Federal Ministry for Economic Affairs and Climate Action in the framework of the "national center of excellence ML4Earth" (grant number: 50EE2201C), Munich Center for Machine Learning, Nokia Bell Labs, and European Union's
Horizon 2020 research and innovation programme under grant agreement No. 869764, awarded to the GoGreenRoutes project.
The authors also thank Sinziana Oncioiu, and Dario Augusto Borges Oliveira for helpful discussions, and Paolo Campanella and Bertrand Robert for their support during the dataset creation.
[1] UK Office for National Statistics. https://www.nomisweb.co.uk/sources/census_2021, 2022. [Online; accessed 26-February-2022].
[2] Nancy E Adler and Joan M Ostrove. Socioeconomic status and health: what we know and what we don't. Annals of the New York academy of Sciences, 896(1):3-15, 1999.
[3] The European Space Agency. Sentinel-2 mission guide. https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-2, 2023.
[4] Suraiya P Ahmad, Omar Torres, Pawan K Bhartia, Gregory Leptoukh, and SJ Kempler. Aerosol index from toms and omi measurements. In Proc. of the 86th AMS annual meeting, 2006.
[5] Luc Anselin. *Spatial econometrics: Methods and Models*, volume 4. Springer Science & Business Media, 1988.
[6] Luc Anselin. Spatial regression. The SAGE handbook of spatial analysis, 1:255-276, 2009.
[7] Evelise Pereira Barboza, Marta Cirach, Sasha Khomenko, Tamara Iungman, Natalie Mueller, Jose Barrera-Gomez, David Rojas-Rueda, Michelle Kondo, and Mark Nieuwenhuijsen. Green space and mortality in european cities: a health impact assessment study. The Lancet Planetary Health, 5(10):e718-e730, 2021.
[8] Douglas A Becker, Matthew HEM Browning, Olivia McAnirlin, Shuai Yuan, and Marco Helbich. Is green space associated with opioid-related mortality? an ecological study at the us county level. Urban Forestry & Urban Greening, 70:127529, 2022.
[9] Julian Besag, Jeremy York, and Annie Mollier. Bayesian image restoration, with two applications in spatial statistics. Annals of the institute of statistical mathematics, 43:1-20, 1991.
[10] Luca Bonacini, Giovanni Gallo, and Sergio Scicshitano. Working from home and income inequality: risks of a 'new normal' with Covid-19. Journal of population economics, 34(1):303–360, 2021.
[11] Christopher F Brown, Steven P Brumby, Brooke Guzder-Williams, Tanya Birch, Samantha Brooks Hyde, Joseph Mazzariello, Wanda Czerwinski, Valerie J Pasquarella, Robert Haertel, Simon Ilyushchenko, et al. Dynamic world, near real-time global  $10\mathrm{m}$  land use land cover mapping. Scientific Data, 9(1):251, 2022.
[12] Nicholas A Buckley and Peter R McManus. Changes in fatalities due to overdose of anxiolytic and sedative drugs in the uk (1983-1999). Drug safety, 27:135-141, 2004.
[13] Wendy Burn and Santosh Mudholkar. Impact of Covid-19 on mental health: Update from the united kingdom. Indian journal of psychiatry, 62(Suppl 3):S365, 2020.
[14] Bingheng Chen and Haidong Kan. Air pollution and population health: a global challenge. Environmental health and preventive medicine, 13:94-101, 2008.
[15] Yiling J Cheng, Alka M Kanaya, Maria Rosario G Araneta, Sharon H Saydah, Henry S Kahn, Edward W Gregg, Wilfred Y Fujimoto, and Giuseppina Imperatore. Prevalence of diabetes by race and ethnicity in the united states, 2011-2016. Jama, 322(24):2389-2398, 2019.
[16] Theodore J Cicero, James A Inciardi, and Alvaro Muñoz. Trends in abuse of oxycontin® and other opioid analgesics in the united states: 2002-2004. The Journal of Pain, 6(10):662-672, 2005.
[64] Parliament UK. Population estimates HP registers: why the difference? https://commonslibrary.parliament.uk/population-estimates-gp-registers-why-the-difference, 2016. [Online; accessed 3-June-2023].
[65] Roozbeh Valavi, Jane Elith, José J Lahoz-Monfort, and Gurutzeta Guillera-Arroita. block cv: An r package for generating spatially or environmentally separated folds for k-fold cross-validation of species distribution models. Methods in Ecology and Evolution, 10(2):225-232, 2019.
[66] Alex J Walker, Helen J Curtis, Richard Croker, Seb Bacon, and Ben Goldacre. Measuring the impact of an open web-based prescribing data analysis service on clinical practice: cohort study on nhs england data. Journal of Medical Internet Research, 21(1):e10929, 2019.
[67] WASDI platform. Earth Observation tech for everyone. https://www.wasdi.cloud, 2023. [Online; accessed 3-June-2023].
[68] Liam Wright, Andrew Steptoe, and Daisy Fancourt. Are we all in this together? longitudinal assessment of cumulative adversities by socioeconomic position in the first 3 weeks of lockdown in the uk. J Epidemiol Community Health, 74(9):683-688, 2020.
[69] Christopher Yeh, Chenlin Meng, Sherrie Wang, Anne Driscoll, Erik Rozi, Patrick Liu, Ji-hyeon Lee, Marshall Burke, David B Lobell, and Stefano Ermon. Sustainbench: Benchmarks for monitoring the sustainable development goals with machine learning. arXiv preprint arXiv:2111.04724, 2021.
Appendix MEDSAT: A Public Health Dataset for England Featuring Medical Prescriptions and Satellite Imagery
A Dataset Licenses 1
B Dataset Storage and Maintenance Plans 2
C Datasheet 2
D Dataset Details 11
D.1 Sociodemographic Features 11
D.2 Environmental Features 11
D.3 Image Features 11
D.3.1 Extracting Image Features per LSOA 11
D.4 Prescription Outcomes 12
D.4.1 Existing Public and Population Health Datasets 12
D.4.2 NHS Medical Prescriptions 14
D.4.3 DrugBank Network 15
E Benchmarks Results 16
E.1 Detailed Health Inequalities 16
E.2 Predicting Prescriptions 17
E.2.1 Spatial Lag Model (SLM) 17
E.2.2 Machine Learning Models 18
E.3 Temporal Analyses 20
A Dataset Licenses
The dataset is released under the CC BY-SA 4.0 license.
B Dataset Storage and Maintenance Plans
The dataset is available and will be maintained on TUMMedia, a data sharing service from Technical University Munich. It can be accessed via the following link: https://doi.org/10.14459/2023mp1714817. The code for datasets creation and experiments can be accessed on https://github.com/sanja7s/MedSat.
Understanding the impact of the environment on human health is crucial for informing policymaking, promoting human well-being, and implementing timely health interventions, particularly in the context of increasing climate change-induced extreme weather events. Earth Observation data has emerged as a valuable resource for addressing critical challenges related to climate change, food security, and poverty [6, 15]. However, current applications that monitor population health using such data have limitations. Health indicators are often derived from surveys [46] and crowdsourcing [24], which may not be representative of the entire population. Additionally, these applications rely on a narrow set of environmental and sociodemographic indicators, hindering a comprehensive understanding of the relevant health factors [5, 17, 13]. Moreover, many of these datasets are not publicly available.
In this study, we aim to address these challenges by providing a publicly available dataset that harnesses the potential of earth observation data to monitor and comprehend the environmental influence on health outcomes. Our dataset combines four complementary data sources to enable comprehensive health modeling across the entire population of England. Specifically, for the years 2019 and 2020, we integrated medical prescription data at the practice level from the NHS, satellite-derived environmental features, Sentinel-2 satellite imagery, and sociodemographic indicators obtained from the latest UK census in 2021. These data sources were combined at the level of Lower Layer Super Output Areas (Lower Layer Super Output Area (LSOA)), which represent small administrative units ranging in size from  $1\mathrm{km}^2$  to  $18\mathrm{km}^2$ . Our objectives are to: