DOI: 10.5281/zenodo.16989885
title: Supporting data repository for "A Network Calibration Approach Improves the Accuracy and Long-Term Stability of a Low-Cost Air Quality Mesonet in New York City"

description: Repository Overview 

By: Ellie Hojeily, updated 20 December 2025. 

Important! If you have any issues accessing or using the data from this repository or are interested in additional data or code, please contact ehojeily@albany.edu or send me a message via LinkedIn. If you do not get a response within 1-2 weeks, please contact Scott Miller (smiller@albany.edu) as an alternative. 

 

This repository contains supporting calibration files for Hojeily et al., 2025: "A Network Calibration Approach Improves the Accuracy and Long-Term Stability of a Low-Cost Air Quality Mesonet in New York City" (https://doi.org/10.1021/acsestair.5c00205) and Miller et al. 2025: "Combining Low-Cost Sensors with the New York State Mesonet for  Continuous Fine-Scale Air Quality Monitoring in the New York City Metropolitan Area" (https://doi.org/10.1021/acsestair.5c00200). This repository supports version 1.3 of the New York State Mesonet air quality network. 

 

Computing Information

Python version 3.10 was used during this project via VSCode and JupyterHub. The following packages and version numbers were used during this project: 




Package
Version
Use


Jupyter
1.1.1
Python intepretor 


joblib
1.3.2
Handling random forest files


matplotlib
3.8.3
Visualization


numpy
1.26.4
Data processing


pandas
2.2.1
Data processing


scikit-learn
1.0.2
Model development & verification


scipy
1.11.4
Model development & verification


statsmodels
0.14.1
Model development & verification


pytz
2023.3.post1
Data processing




If you run into any issues using the provided code in supporting_functions.py, please check that you are running the correct versions of these core Python packages.

Available Files and Corresponding Metadata



'slr_transfer_function_stats.csv' hosts the performance of the SLR transfer functions verified on the keystone package. The model_eval function from supporting_functions.py was used to score models. 





Column name
Information


Package
Package serial number, all start with AQA20


Sensor
Pollutant/sensor that is being adjusted (i.e., NO, NO2, O3NO2, etc.) 


Calibration_Number
Co-location in which package is being corrected. C1 = co-location 1, C2 = co-location 2


R2
Coefficient of determination w/ keystone package after Transfer function is applied. 


MAE
Mean absolute error w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. 


RMSE
Root-mean-square error w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. 


m
Slope w/ keystone package after Transfer function is applied. 


y-int
Offset w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. 


mbe
Mean bias error w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. 






'_network_SLR_coefficients' are the Transfer Function slope and offset values trained during co-location at Queens College w/ the Keystone package. These files are organized by pollutant. 





Column name
Information


Offset
y-int term. Units are in volts from the trace gases while pm2.5 is in µg/m³.


Slope
coefficient term. 


Sensor 
Package serial number, all start with AQA20 


Co-location number
Co-location in which package is being corrected. C1 = co-location 1, C2 = co-location 2






'aqa2020_keystone_data.csv' is the calibrated data from the keystone package, AQA2020. 





Column name
Information


Datetime (UTC)
Hourly datetime in UTC


_MLR
Calibrated using MLR. Units are in ppb for the trace gases while pm2.5 is in µg/m³. 


_RF
Calibrated using RF. Units are in ppb for the trace gases while pm2.5 is in µg/m³. 


_Hybrid
Calibrated using Hybrid. Units are in ppb for the trace gases while pm2.5 is in µg/m³. 


T_ambient
Ambient temperature measured by package in degrees celsius 


RH_ambient
Ambient relative humidity measured by package as a percent


Period
Indicates if a row was used for training or testing






'keystone_package_raw.csv' is the raw data from the keystone package, AQA2020. The file includes meteorological variables measured by the package. No data has been qc'd from this file outside of a simple despiking algorithm and resampling from 5-second data to hourly data.





Column name
Information


Datetime 
Datetime in UTC


_aux
Auxilary voltage from electrochemical sensors


_we
Working voltage from electrochemical sensors


_sensor
Estimated voltage concentration (working-auxilary) from electrochemical sensors in units of Volts, copy of PM2.5 as reported by the Plantower PMS5003 in units of µg/m³


T_ambient
Ambient temperature measured by package in degrees celsius 


RH_ambient
Ambient relative humidity measured by package as a percent


T_board
Internal sensor package temperature in degrees celsius, this is can occasionally deviate from T_ambient and T_manifold  


T_manifold
Manifold temperature measured by package in degrees celsius, this is typically identical to T_ambient


RH_manifold
Manifold relative humidity measured by package as a percent, this is typically identical to RH_ambient


pres_board
Internal sensor package pressure in millibars


Tdew_manifold
Manifold dewpoint in degrees celsius 


pm25_env, pm25_std
PM2.5 as reported by the Plantower PMS5003 in units of µg/m³






'supporting_functions.py' are some example python functions that were used in this work, including functions to train & verify models

pollutant+'_colocation_networkstats.csv' host the performance of each package, organized by colocation (1, 2, and wildfire for PM2.5) & pollutant, when verified on regulatory instruments from the NYSDEC Queens College site. The model_eval function from supporting_functions.py was used to score models. 





Column name
Information


Package ID
Package serial number, all start with AQA20


r2
Coefficient of determination w/ reference instrument after calibration algorithm is applied. 


rmse
Root-mean-square error w/ reference instrument after calibration algorithm is applied. Units are in ppb from the trace gases while pm2.5 is in µg/m³. 


pbias
The percent bias w/ reference instrument after calibration algorithm is applied. 


mae
Mean absolute error w/ reference instrument after calibration algorithm is applied. Units are in ppb from the trace gases while pm2.5 is in µg/m³. 


mbe
Mean bias error w/ reference instrument after calibration algorithm is applied. Units are in ppb from the trace gases while pm2.5 is in µg/m³. 


nmae
The normalized mean absolute error w/ reference instrument after calibration algorithm is applied. Normalized using the mean reference concentration. 


nrmse
The normalized root-mean-square error w/ reference instrument after calibration algorithm is applied. Normalized using the mean reference concentration. 


nmbe
The normalized mean bias error w/ reference instrument after calibration algorithm is applied. Normalized using the mean reference concentration. 


nhours
The number of co-location hours.


ref_avg
The average reference concentration of the target gas during the respective co-location. 






pollutant+'_NCM_MLR.txt' are files containing the MLR NCM models. A custom Python function was used to generate (generate_coefficients) and apply the coefficients (calibrate) and is included in the repository in supporting_functions.py.





Column name
Information


model_type
'net' indicating the Network Calibration Algorithm 


coef_const
Y-intercept


coef_+variable
Coefficient for specific variable. For example, 'coef_pm25_sensor' represents the coefficient for the PM2.5 sensor in the PM2.5 MLR model. 






pollutant+'_NCM_RF.joblib' are files containing the RF NCM models. The RF models are stored as joblib files, requiring the joblib Python package to access. 


Other Relevant Datasets: 

Reference/Regulatory Instrument Data

Reference instrument data should be retrieved from the EPA via their AQS repository (https://www.epa.gov/aqs/obtaining-aqs-data). The supplemental information in Hojeily et al. (2025) details the specific site ids used for nearby field evaluations. Reference data from the calibration site, the NYSDEC Queens College site, can be retrieved via the following site id: 36-081-0124. Relevant AQS parameter codes for the EPA AQS data can be found below. 




Pollutant
AQS Parameter Code/ID


PM2.5 
88101 (PM2.5 Local Conditions) 


O3
44201


NO2
42602


NO
42603


CO
42101




 

New York State Mesonet Meteorological and Air Quality Data 

Requests for New York State Mesonet meteorological data must be made via the mesonet website (https://www2.nysmesonet.org/weather/requestdata). The air quality data from the mesonet field sites will be made available on the mesonet website. 

Acknowledgements:

This research was supported by the New York State Energy Research and Development Authority (NYSERDA, https://www.nyserda.ny.gov/) award 156228. The opinions expressed in this report do not necessarily reflect those of NYSERDA or the State of New York.