SlowGuess/NeurIPS_2023_OCR huggingface dataset:
AIRDELHI: Fine-Grained Spatio-Temporal Particulate Matter Dataset From Delhi For ML based Modeling
Sachin Kumar Chauhan, Sayan Ranu, Rijurekha Sen
IIT Delhi
{cz188012, sayanranu, riju}@cse.iitd.ac.in
Zeel B Patel, Nipun Batra
IIT Gandhinagar
{patel_zeel, nipun.batra}@iitgn.ac.in
Air pollution poses serious health concerns in developing countries, such as India, necessitating large-scale measurement for correlation analysis, policy recommendations, and informed decision-making. However, fine-grained data collection is costly. Specifically, static sensors for pollution measurement cost several thousand dollars per unit, leading to inadequate deployment and coverage. To complement the existing sparse static sensor network, we propose a mobile sensor network utilizing lower-cost  $\mathrm{PM}_{2.5}$  sensors mounted on public buses in the Delhi-NCR region of India. Through this exercise, we introduce a novel dataset AIRDELHI comprising  $\mathrm{PM}_{2.5}$  and  $\mathrm{PM}_{10}$  measurements. This dataset is made publicly available at https://www.cse.iitd.ac.in/pollutiondata, serving as a valuable resource for machine learning (ML) researchers and environmentalists. We present three key contributions with the release of this dataset. Firstly, through in-depth statistical analysis, we demonstrate that the released dataset significantly differs from existing pollution datasets, highlighting its uniqueness and potential for new insights. Secondly, the dataset quality has been validated against existing expensive sensors. Thirdly, we conduct a benchmarking exercise (https://github.com/sachin-iitd/DelhiPMDatasetBenchmark), evaluating state-of-the-art methods for interpolation, feature imputation, and forecasting on this dataset, which is the largest publicly available PM dataset to date. The results of the benchmarking exercise underscore the substantial disparities in accuracy between the proposed dataset and other publicly available datasets. This finding highlights the complexity and richness of our dataset, emphasizing its value for advancing research in the field of air pollution.
United Nations Sustainable Development Goals [UN, 2015], especially the Goal-11 Sustainable Cities and Communities, is a primary research focus in institutions of developing countries like India [IIITD, 2023], with related research works in sustainable transport [Chauhan et al., 2020] and pollution [Shukla et al., 2020; Bikkina et al., 2019; Iyer et al., 2022; Abidi et al., 2022].
Air pollution has now reached life-threatening levels in Delhi-National Capital Region (NCR), India [Tripathi et al., 2019; Mannucci and Franchini, 2017], which is one of the most densely populated urban centers. The population of Delhi-NCR exceeds 46 million people [Nagar et al., 2017] and it has been reported that  $50\%$  of all children staying in this region suffer from irreversible lung damage [Chatterji, 2021; ORF, 2021]. Particulate Matter (PM) is especially dangerous, since
our breathing cannot filter out the ultra-fine particles. To mitigate the effects of air pollution, there is an urgent need to identify causes of pollution and strategies to curb its spread. [Sahu et al., 2020; Sutaria, 2022] suggests to use one sensor per  $\mathrm{km}^2$  for better pollution analysis. The Central Pollution Control Board (CPCB) and Delhi Pollution Control Committee (DPCC) have only 81 realtime air pollution measurement centers in Delhi-NCR Sutaria [2022] with 65 manually monitored centers, which are thoroughly inadequate [Guttikunda et al., 2023; ET, 2022] to cover the vast geography of  $55,000\mathrm{km}^2$  [NCRPB, 2018].
In the literature, several models have been proposed for predicting pollution levels at same/future time points [Patel et al., 2022; Gao and Li, 2021; Kurt et al., 2008; Tsai et al., 2018; Le et al., 2020], and identifying factors affecting pollution [Apte et al., 2011; Google, 2014; Messier et al., 2018; Apte et al., 2017; Alexeeff et al., 2018]. There exists interpolation models [Qiao et al., 2019; Ras and Williams, 2005; Hamilton et al., 2017; Patel et al., 2022] to reliably predict pollution levels at unseen locations based on a sufficient number of pre-installed sensors. These models can improve with fine-grained pollution data. The interpolation and forecasting models are supervised in nature and hence can do better with more training data. Unfortunately, collecting pollution data using realtime centers is highly expensive as each instrument costs thousands of US Dollars.
In this work, we aim to mitigate the problem of lack of sufficient data in a cost-effective manner. We design a low-cost sensing mechanism (thoroughly compared in quality against high cost sensors) that allows us to collect PM data over a subset of the Delhi-NCR region at a fine spatio-temporal granularity. The key highlights and contributions of our work are:
1. Quality dataset: As it is not cost-effective to repeat even the low cost sensors per  $\mathrm{km}^2$ , we establish a low-cost vehicle-mounted PM sensing network and release the largest  $\mathrm{PM}_{2.5}$  dataset from one of the most polluted regions in the world. This dataset is shown to be as good as the data collected from the few high-cost static-sensor deployed in the same region. As it is very challenging to collect such dataset in a developing country due to constraints in infrastructure and government permissions, we document our data collection experience briefly in the paper. (§ 3.2).
2. Unique dataset: This dataset complements the static sensor data available from the government deployed instruments in important ways. The static sensors are located at the top of high towers to get precise recordings of ambient pollution values, not affected by local sources. Our mobile sensors, on the other hand, are installed in the bus driver's cabin to measures the ground level pollution that daily commuters breathe in. We also perform a thorough comparison with PM datasets available from other parts of the world and establish that the released dataset is unique in terms of scale and statistical characteristics. Hence, it can be of immense value to environmental think tanks. (§ 3.3).
3. Utility for ML modeling: Through extensive benchmarking using state-of-the-art Machine Learning (ML) algorithms, we demonstrate the utility of this new dataset for modeling problems using ML, like spatio-temporal interpolation, missing data imputation and forecasting. The dataset is shown to be more challenging to model with ML algorithms, compared to previously available datasets, as Delhi has much higher variance in PM across space and time. This dataset, therefore opens opportunities for ML researchers for designing and benchmarking new ML algorithms, to reduce the interpolation, missing data imputation or forecasting errors. (§ 4).
A primer for Air Pollution Monitoring is available at Urbanemissions [2023]. Spatio-temporal (ST) interpolation involves predicting air quality at unmonitored locations in the past and/or present time using training data observed from the sensors during the past and present time. Zheng et al. [2013] developed a co-training-based approach for ST interpolation using  $\mathrm{PM}_{2.5}$  values captured every hour from ground stations of 4 cities in China which are converted to AQI (Air Quality Index), along with meteorological and traffic data. Cheng et al. [2018] proposed an attention-based hybrid model involving LSTM and dense layers and Patel et al. [2022] proposed a domain-inspired non-stationary Gaussian process model for ST interpolation which can also be used for ST forecasting. The two used 36 monitoring stations in Beijing with the collection time interval of 1 hour (with the latter additionally using London data), alongside meteorological data.
Missing data imputation problem can be considered a variation of spatio-temporal interpolation where observations on the spatio-temporal cube are missing at random and we want to impute the missing data. Models that work for ST interpolation can mostly be adapted readily for this problem.
Spatio-temporal forecasting aims to predict air quality at a particular location in future using the past and current data available at all the installed sensors. Kurt et al. [2008] developed an online neural network based approach to predict air quality maximum 3 days ahead in time using 1 year  $\mathrm{PM}_{10}$  data
for 1 region in Turkey. Zheng et al. [2015] develop and deploy a machine learning based air quality forecasting system with the Chinese Ministry of Environmental Protection. Yi et al. [2018] develop a deep learning based approach to provide short-term and long-term air quality forecasts. The two used meteorological data along with pollution data generated every hour from 2,296 stations in 302 Chinese cities, and converted these concentrations into corresponding (individual) AQIs according to Chinese AQI standards. Air quality forecasting was posed as a challenge in KDD2018, where Luo et al. [2019] presented a winning solution based on a combination of classical machine learning and deep learning models using the provided data from stations in Beijing and London. Gao and Li [2021] propose a graph-based LSTM model for air quality forecasting and evaluate on Northwest China hourly data from 32 china stations. All these prior arts utilize the static ground stations Air Quality data for the analysis, which enforces a restricted spatial coverage. They also use meteorological data from the respective regions. Bhattacharyya et al. [2022] provides a similar USA AQI dataset collected from Air Quality Open Data Platform.
There also have been studies on low cost sensors available in market for developed (EU) regions Karagulian et al. [2019]. Also, projects about installing low cost sensors at different roadside locations, like Schneider et al. [2023] and Iyer et al. [2022], to complement the existing expensive static sensor network are done recently, but they kept the sensors at fixed locations. Vehicle mounted air pollution sensing has also been conducted [Apte et al., 2011; Google, 2014; Apte et al., 2017; Alexeeff et al., 2018; Guo et al., 2016; Adams and Corr, 2019; Li et al., 2012], with certain limitations. Abidi et al. [2022] used similar low-cost sensors to analyze the relation of PM with static (green cover, buildup, commercial, residential) and dynamic (meteorological, traffic) factors, particularly at traffic intersections with odd-even policy in Delhi. As per Yi et al. [2015]; Pavani and Rao [2017], static sensors are better in data quality, endurance and temporal resolution, a low-cost static sensor is better in temporal resolution which can provide (static) pollution maps with better temporal resolution for the installed locations. But the effective static pollution maps require careful placement of sensor nodes in large quantity leading to resource wastage. Mobile sensors are better in mobility, geographic coverage, maintenance, cost-efficiency which can provide (dynamic) pollution maps with greater geographic coverage, with limitation of communication overheads and redundant sampling. In contrast, we are working with the PM data, collected with mobile sensors deployed on frequent bus routes, which is fine-grained and provides better spatio-temporal coverage, and our benchmarked models do not rely on meteorological factors.
3 Dataset Description
3.1 Dataset Collection Challenges
Creating the mobile PM dataset (as a replacement for low cost static PM dataset and high-cost ground station PM dataset) required us to design and implement our own embedded platform, choosing and calibrating appropriate sensors for maximum accuracy at low cost. The complete design is presented in Goyal et al. [2018]. We opted to install our device in public buses, to utilize their pre-defined/fixed and frequent routes of travel. Packaging was challenging to securely mount the instruments in the public buses, avoiding theft and ensuring enough ambient air to measure PM. Cellular connectivity was intermittent as the buses traversed the city, requiring us to augment real time data transfer when signal was present, with local storage to save data when signal strength dropped. Finally, getting permissions from different government entities to instrument the public bus fleet needed strict safety certifications that our devices do not interfere with the electrical and mechanical functioning of bus.
We mounted pollution tracking sensors on the permissible 13 public buses in Delhi for 3 months (Nov 1, 2020 to Jan 31, 2021), in collaboration with Delhi Integrated Multimodal Transport System, after rigorous tests for automotive safety certification and appropriate permissions and letters of support from the Delhi Ministry of Transport and Delhi Pollution Control Committee (provided in Appendix I). The Covid'19 restrictions were relaxed by this time, limiting to containment zones at local level only [MHA, 2020]. So, the impact of the COVID'19 on the collected dataset is expected to be minimal, if not none.
As discussed in Goyal et al. [2018], the inside of our custom-made instrument comprising (a) PM sensor measuring  $\mathrm{PM}_{2.5}$ ,  $\mathrm{PM}_{10}$  and  $\mathrm{PM}_1$ , (b) GPS sensor to locate the bus, (c) 4G radio to communicate data from bus to server, (d) SD card for locally storing data when 4G signal is unavailable, (e) BME sensor [BME, 2023], a sensor especially developed for mobile applications and wearables, to record temperature and relative humidity and  $(f)$  micro-controller to orchestrate the sense-store-communicate software (See Fig. 1a). The mounting location in the bus driver's cabin, next to two open windows to allow enough air-flow (Fig. 1b-lc). Each bus commutes for
(a) Measuring device
(b) Mounting location
Figure 1: (a) Inside of our PM measuring IoT unit. (b) Mounting location in bus driver's cabin in non air-conditioned public bus (below the existing white box). (c) Mounted IoT unit in the bus (below the existing white box). (d) Government deployed static sensors installed in and around our bus trajectories, as location icons.
(c) Mounted device
(d) Bus trajectories
16-20 hours per day, and our instruments collect data at a fine granularity of 20 samples per minute. Overall, the bus trajectories cover  $559~\mathrm{km}^2$ , along the main arterial roads in North-West, North, North-East and South-East Delhi (Fig. 1d). The dataset, having 3 pollution parameters:  $\mathrm{PM}_1$ ,  $\mathrm{PM}_{2.5}$  and  $\mathrm{PM}_{10}$  with 7 non-pollution parameters: latitude, longitude, time, deviceId, pressure, temperature, relative humidity, has been made available at https://www.cse.iitd.ac.in/pollutiondata/ and https://huggingface.co/sachin-iitd/DelhiPollDataset with proper documentation, under a Creative Commons Attribution 4.0 International License [CC-by4, 2013].
3.2 Data Quality Analysis
Fig. 2a plots  $\mathrm{PM}_{2.5}$  values measured by two low cost PM sensors built by us (cost USD 30), and the same measured by an industry grade reference instrument TSI DustTrak (cost USD 9500), while all three instruments are placed close to each other. The plot shows hours of the day along  $x$ -axis and sensed  $\mathrm{PM}_{2.5}$  values along  $y$ -axis, for 10 sample days Jul 21-31, 2021. This is after the deployment of the low cost sensors in the buses is over, and the sensors have been brought back to the lab.
(a) Time-series comparison
(b) Time-series difference (Dustrak vs Mobile1)
(c) Histogram of  $\mathrm{PM}_{2.5}$  values (Dustrak vs Mobile)
Figure 2: (a)  $\mathrm{PM}_{2.5}$  values measured by our low-cost mobile PM sensors (USD 30) vs. TSI DustTrak (USD 9500) between Jul 21-31, 2021. (b) Mean Difference and (d) Histogram of pointwise differences of  $\mathrm{PM}_{2.5}$  values measured by DustTrak and low cost mobile PM sensors. (c) Histogram of  $\mathrm{PM}_{2.5}$  in the intervals on x-axis. The values are almost identical.
(d) Histogram of differences
Fig. 2b shows the difference of hourly mean  $\mathrm{PM}_{2.5}$  between DustTrak and one mobile sensor, the mean difference is  $6.16\%$ . Also, Fig. 2c shows the histograms of hourly mean  $\mathrm{PM}_{2.5}$  for the shown  $\mathrm{PM}_{2.5}$  intervals for the three devices. Fig. 2d shows the histogram of difference of hourly mean  $\mathrm{PM}_{2.5}$  between DustTrak and one mobile sensor, and two mobile sensors, for the same 10 days. Also, Fig. 9 in Appendix C shows the similar mean and standard deviation between the 3 devices. While the cost gap between the instruments is huge, the gap between their sensed  $\mathrm{PM}_{2.5}$  values, as seen in the plots, is negligible. This pattern has been observed consistently by us and other researchers [Zheng et al., 2018; Cheng et al., 2014; Gao et al., 2015; Rai et al., 2017; Jiao et al., 2016; Zheng et al., 2019].
We additionally compare the distribution of  $\mathrm{PM}_{2.5}$  values recorded by our mobile sensors vs. those by the high-cost static sensors, deployed at sparse locations by CPCB and DPCC in Delhi-NCR. Fig. 3a(Left) shows hours of day along x-axis and average  $\mathrm{PM}_{2.5}$  for that hour, as measured by reference grade static monitors, with standard-deviation bars along y-axis. Fig. 3a(Right) shows the same averaged over all bus mounted sensors. We select the static sensors that are within  $1\mathrm{km}$  of mobile sensor trajectory for each hour, and plot for 7 sample days. Fig. 3a reveal that both static and bus mounted sensors show similar PM distributions for each day, in spite of the difference in heights they have been installed at, and the difference in PM measurement technique. We see this agreement for the entire 3 months deployment period. The agreement between low cost mobile sensors, and a co-located high cost TSI Dusttrak, as well as reference grade static monitors, give us confidence to release the dataset to the research community.
(a) Static v/s Mobile Sensors Data
(b) Heat Map
Figure 3: (a) Distribution of  $\mathrm{PM}_{2.5}$  collected by our low-cost sensor and gold standard sensor over 7 random days. The distributions are similar across the two sets of instruments. (b) Heat Map for all bus routes for Dec 15, 2020 (darker locations contain more samples).
Heat Map: During our analysis, we discovered variations in data availability across different timestamps and spatial locations. It was evident that certain timestamps were not available at all spatial locations. Furthermore, some spatial locations, which were situated along routes with fewer bus visits throughout the day, exhibited limited temporal samples. As illustrated in Fig. 3b, a typical day (Dec 15, 2020) demonstrated this pattern, where the outermost locations (depicted in light/pink color) contained samples from 4 hours duration within the 16.5-hour effective temporal window. Conversely, the darker/brown locations near the bottom right of the figure displayed a higher number of samples, ranging from 14 to 16.5 hours. These locations are associated with common bus routes that connect with the depot.
3.3 Dataset Novelty
Tables 1 and 2 summarize the statistics of the dataset. While vehicle mounted air pollution sensing has been conducted [Apte et al., 2011; Google, 2014; Apte et al., 2017; Alexeeff et al., 2018; Guo et al., 2016; Adams and Corr, 2019; Li et al., 2012], our dataset is unique in characteristics and scale. Specifically, only two studies from Ontario, Canada [Adams and Corr, 2019] and Zurich, Switzerland [Li et al., 2012] have made their datasets publicly available. The Zurich dataset does not include PM values. Compared to the Canada dataset, our dataset is 1000 times larger and has a significantly different distribution of PM values (See Tables 1 and 2).
This is understandable as Delhi-NCR is an air pollution hotspot, whereas Zurich and Ontario have negligible PM levels. We also compare our dataset with a recent USA AQI dataset Bhattacharyya et al. [2022] collected from Air Quality Open Data Platform. We also purchased Wind Speed (WS) data
Table 1: Details of Delhi, India and Hamilton, Ontario, Canada and USA datasets.
Metric	Delhi-NCR	Canada	USA
Total area	559 km2	1138 km2	54 cities
Total samples	12,542,183	46,080	35,596
Samples with PM2.5	12,542,183	12,154	35,134
Pollutants covered	PM1, PM2.5 and PM10	CO, NO, NO2, SO2, O3, PM1, PM2.5 and PM10	CO, NO2, SO2, O3, PM2.5 and PM10
Meteorological	Temp, RH, Pressure, Wind Speed *	-	Temp, RH, Pressure, Dew, Wind Speed, Wind Gust
Sensor source	Public bus	Commercial van	OpenDataPlatform
Monitoring days	91	114	668
Table 2: Statistical comparison of PM values in Delhi, Canada and USA datasets.
Metric	Delhi-NCR	Canada	USA
PM1	PM2.5	PM10	PM1	PM2.5	PM10	PM2.5 AQI	PM10 AQI
Mean	120.35	207.92	226.11	12.15	15.08	46.45	31.15	17.67
Std-dev	57.27	114.36	123.86	9.02	12.87	97.36	17.11	11.00
Missing %	0	0	0	71.71	73.62	72.24	1.30	52.34
for Nov 2020 - Jan 2021 from www. windfinder. com to complement our Delhi-NCR dataset for the meteorological analysis performed in Appendix E and Fig. 15, showing pollutants correlation with different meteorological factors and peculiar situations due to external factors (like impact of stubble burning episodes). Due to this behaviour, interpolation and forecasting are hard problems. Still, in Appendix D and Fig. 14, we observed covariance among some spatial locations, and in Fig. 11, we observed some autocorrelation over the entire dataset. Similar trends were observed with the bus route analysis shown in Appendix F. Hence we decided to formulate an interpolation problem and a 24 hour forecasting problem to analyze the model performance, next.
4 ML Modeling Benchmarks
In this section, we benchmark the machine learning problems of (1) spatio-temporal interpolation, (2) spatio-temporal data imputation and (3) spatio-temporal forecasting on the proposed, Canada and USA datasets. This benchmarking study serves two roles. First, it allows us to compare the complexities of the three datasets beyond just statistical characterization. Secondly, spatio-temporal interpolations, data imputations, and forecasting methods are crucial for environmental research, policy-making, and individual decision-making. They empower various stakeholders to gain a comprehensive understanding of air pollution, proactively address potential increases in pollution levels, and make informed choices to reduce personal exposure. In order to harness the full potential of spatio-temporal forecasting, interpolations, and data imputations, it is crucial to benchmark and evaluate the performance of algorithms designed to tackle these problems.
4.1 Formulation of different ML Prediction Problems
(a) Spatio-temporal Interpolation: Given set of visible/available locations with input features (latitude, longitude and time) and  $\mathrm{PM}_{2.5}$  available for  $\mathrm{T} + 1$  days, we wish to estimate  $\mathrm{PM}_{2.5}$  for a set of held-out locations for the  $\mathrm{T} + 1^{th}$  day using the input features (latitude, longitude and time). This approach is compatible to the scenario where we have data for some locations and we use interpolation algorithms to know the  $\mathrm{PM}_{2.5}$  values at new locations.
(b) Spatio-temporal Missing Data Imputation: Given set of locations with input features (latitude, longitude and time) and  $\mathrm{PM}_{2.5}$  available for T days and a set of visible locations with input features (latitude, longitude and time) and  $\mathrm{PM}_{2.5}$  available for the  $T + 1^{th}$  day, we wish to estimate  $\mathrm{PM}_{2.5}$  for a set of held-out locations for the  $T + 1^{th}$  day using the input features (latitude, longitude and time). This setting is compatible to the scenario where we have intermittent data missing throughout the day and we use interpolation algorithms to predict the missing points taking past & present data as input.
(c) Spatio-temporal Forecasting: Given a set of all locations with input features (latitude, longitude and time) and  $\mathrm{PM}_{2.5}$  available for T days, we wish to estimate  $\mathrm{PM}_{2.5}$  for a set of all locations for the  $T + 1^{th}$  day using the input features (latitude, longitude and time).
4.2 ML Algorithms Benchmarked in this Paper