Senseable City Lab :.:: Massachusetts Institute of Technology
This paper might be a pre-copy-editing or a post-print
author-produced .pdf of an article accepted for publication. For
the definitive publisher-authenticated version, please referdirectly to publishing house’s archive system

SENSEABLE CITY LAB

Atmospheric Environment 301 (2023) 119692

Contents lists available at ScienceDirect

Atmospheric Environment
journal homepage: www.elsevier.com/locate/atmosenv

Leveraging machine learning algorithms to advance low-cost air sensor
calibration in stationary and mobile settings
An Wang a, Yuki Machida a, Priyanka deSouza b, Simone Mora a, c, *, Tiffany Duhl d,Neelakshi Hudda d, John L. Durant d, Fábio Duarte a, Carlo Ratti aaSenseable City Lab, Department of Urban Studies and Planning, Massachusetts Institute of Technology, USA
Department of Urban and Regional Planning, University of Colorado Denver, USA
Department of Computer Science, Norwegian University of Science and Technology, NorwaydDepartment of Civil and Environmental Engineering, Tufts University, USAbc

H I G H L I G H T S

G R A P H I C A L A B S T R A C T• Our work pilots the standardized EPA
sensorperformance
evaluation
guidelines.
• We examined factors that affect low-cost
sensor and calibration algorithm
performance.
• We observed better transferraiblity ofmachine learning algorithm in a sta­tionary setting.
• We demonstrated the risk of using lowcost sensors without a local calibration.

A R T I C L E I N F O

A B S T R A C T

Keywords:
Low-cost sensor calibration
PM2.5
NO2
Machine learning
Mobile monitoring
Environmental justiceLow-cost air sensing is changing the paradigm of ambient air quality management research and practices.
However, consensus on a structured low-cost sensor calibration and performance evaluation framework islacking. Our study aims to devise a standardized low-cost sensor calibration protocol and evaluate the perfor­
mance of various calibration algorithms. Extensive collocation data were collected in stationary and mobile
settings in two American cities, New York and Boston. We trained the calibration models using stationary data
aggregated at various intervals to examine the performance of several commonly used calibration algorithms
described in the literature. Linear models provide consistently satisfactory calibration results, indicating linear
responses from the low-cost sensors in our stationary test environment. Its simplicity is recommended for citizen
science and education usages. Models that can account for non-linear relationships, especially random forest,
perform well and transfer between sensors better than generalized linear regression models for PM2.5 calibration,
which should be adopted for regulatory and scientific purposes. Data collected in a mobile validation campaign
in Boston were passed through the best-performing calibration models to assess their transferability. The results
indicate that models trained with data from a different urban environment and season in the stationary setting
did not transfer well to a mobile setting. It is recommended that low-cost sensors should be calibrated more often
than suggested in Environmental Protection Agency’s air sensor performance evaluation guidelines and used in
an environment that is as similar as possible to the calibration environment.

* Corresponding author. Building 9-25, 77, Massachusetts Ave., Cambridge, MA, 02142, USA.
E-mail address: moras@mit.edu (S. Mora).
https://doi.org/10.1016/j.atmosenv.2023.119692
Received 7 October 2022; Received in revised form 24 February 2023; Accepted 28 February 2023
Available online 1 March 2023
1352-2310/© 2023 Elsevier Ltd. All rights reserved.

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

Fig. 1. Methodology flow chart.

2

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

The three main categories of low-cost sensor calibration algorithms
currently in use include physical mechanism-based models, parametricmodels such as linear and non-linear regression models, and machinelearning models (Liang, 2021). To date, physical mechanism-basedmodels have targeted the problem that relative humidity can change
PM hygroscopic size and refractive index, influencing conversion from
PM number count to mass concentrations. Crilley et al. (2018) and
Zheng et al. (2018) applied κ-Kӧhler theory and scattering efficiency,
respectively, to adjust relative humidity correction factors for better
low-cost PM sensor calibration. Parametric models, including linear andnon-linear regression models, are more widely used in the literature, as
shown in Table S1. They can consider more features affecting low-cost
sensor readings, while physical mechanism-based methods mainly
calibrate for relative humidity. As state-of-the-art calibration algo­rithms, machine learning approaches can provide higher accuracy indata-rich environments (Liang, 2021; Malings et al., 2019; Wang et al.,2021). Nonetheless, some machine learning algorithms lack interpret­ability and transferability, which hinders their broader implementation.
Given that there is no all-in-one solution to low-cost sensor calibra­
tion, it is necessary to explore various calibration approaches in different
applications and compare their performance. To this end, we tested
multiple commonly used low-cost sensor calibration algorithms in sta­tionary and mobile settings. The calibration models trained with sta­tionary collocation data were further examined with mobile collected
data for transferability tests. We were motivated to simulate a common
scenario deploying sensors calibrated in one location in a different
location that does not have high-quality air instruments to enable local
colocation and calibration. This is especially useful for sensor applica­
tions in low-income regions and countries to address the existing air
quality data gap and related environmental justice issues. This study isamong the first to incorporate the standardized EPA air sensor evalua­tion frameworks and the effects of data aggregation in stationary andmobile settings. Beyond simply comparing model performance byinvestigating algorithm behaviors and the impact factors of sensor per­formance, this study explores the appropriate application scenarios for
multiple calibration algorithms to account for non-linearity in low-cost
sensor performance. Finally, our study provides valuable information on
algorithm transferability by exploiting two large air monitoring datasetscollected in different locations under stationary and mobile conditions.

1. Introduction
Lacking the resources to deploy high-quality monitors, low- andmiddle-income countries are held back from making effective mea­
surements to inform air pollution management. The rapid developmentof low-cost sensors (<$2500(US EPA, 2014)) in recent years provides aunique opportunity to shift the current air quality monitoring paradigm.
While low-cost sensors have been widely adopted in high-income re­gions to supplement traditional air quality monitoring and mapping of
local air quality (Castell et al., 2017; Crawford et al., 2021; Gressent
et al., 2020; Miskell et al., 2018), their applications in low-income
countries are limited (Brauer et al., 2019; deSouza et al., 2020; SM
et al., 2019). The benefits of incorporating low-cost sensors in theexisting monitoring network are multifold, including improving publicawareness of air pollution, pushing for environmentally justdecision-making and reducing data colonialism in air quality monitoring,where government agencies and high-income countries claim ownershipof collected air quality data (Duarte and deSouza, 2020).
Despite the advantages in cost efficiency, flexibility, and ease of use,
low-cost sensors suffer from constant data quality and stability issues;
therefore, sensor collocation and calibration are of utmost importanceprior to field deployment. Collocation is the process of deploying sensors
side-by-side with reference monitors, and calibration involves adjusting
raw sensor readings using collocation data and mathematical methods.
We summarized seven issues that affect low-cost sensor performancefrom low-cost sensor calibration literature within the last five years:
inter-sensor variability, intra-sensor variability, drift, aging, response
time, cross-sensitivity, and sensitivity to environmental factors. Table S1
presents detailed findings from each study.
Inter-sensor variability refers to the variability in measurements
using multiple identical sensors under the same testing environment.
The calibration for inter-sensor variability is crucial for low-cost sensors
as it is the foundation of large-scale sensor deployment and data trans­
ferability. Intra-sensor variability describes the variability in consecu­
tive measurements made by a given sensor under a similar testing
environment. Drift is the gradual change in sensor response over time.
Aging refers to the continuous deterioration of sensor performance overtime. Unlike reference sensors, low-cost sensors are prone to drifting andhave a much shorter lifetime. Thus, they require routine sensor cali­
bration and replacement. Response time reflects the lag before sensors
reach stable readings in the test environment, parity between responsetime and temporal scale of change in signal is critical for time-resolved
sampling. Cross-sensitivity, an issue exclusive to gas sensors, denotes asensor’s false response to gases other than the target gas. Finally,
sensitivity to environmental factors, including temperature, humidity,
wind, barometric pressure, and particle composition, is ubiquitous inboth low-cost and reference sensors. These environmental factors arecommonly identified as the main explanatory features in low-cost sensor
calibration models. Acknowledging that there might be no single solu­
tion to all seven issues in given applications, it is essential to characterize
the performance of various calibration algorithms and identify theproper context for their usage.
In light of low-cost sensor performance issues and the surging in­terest in using sensors for air quality monitoring in areas where regu­
latory monitors are absent, the US Environmental Protection Agency(EPA) has been engaging local communities, subject matter experts, andair sensor manufacturers to develop guidelines for low-cost sensor
usage. EPA published two performance target reports for O3 and par­ticulate matter (PM) sensors to support low-cost sensor non-regulatory
supplemental and informational monitoring (NSIM) applications (USEPA, 2021a, US EPA, 2021b). These reports provide low-cost sensor
collocation, calibration, performance evaluation, and deployment pro­
tocols. The reports suggest using linear regression to calibrate low-cost
sensor data against the reference station for ease of use; however,linear calibration may not meet performance targets in many cases(Kelly et al., 2017; Malings et al., 2019).

2. Methodology
Our study included two main components: developing calibration
models and evaluating the transferability of the models. As shown in
Fig. 1, reference station data, raw low-cost sensing data, and meteoro­
logical data during the collocation were first aggregated at various
temporal aggregations to test the effects of data aggregation on the al­
gorithms. We employed a k-fold time series cross-validation to avoid
overfitting and performance overestimation, where past observations
were used to predict future observations but not the other way around.
Two major categories of calibration algorithms were tested: the gener­
alized linear regressions (yellow) and the machine learning algorithms(red). The best-performing model of each calibration algorithm waspassed to the transferability test using mobile data collected withresearch-grade mobile sensors.
2.1. Low-cost sensors
The mobile low-cost sensing platform we evaluate is developed aspart of the City Scanner initiative at the Senseable City Lab, Massa­
chusetts Institute of Technology. It is designed to address big data needs
for urban development and planning while helping advance environ­mental justice in global cities. For simplicity, the sensing platform isreferred to as the ‘City Scanner’ or ‘CS’ hereafter. Each CS unit isequipped with an Alphasense OPC-N3 and an Alphasense NO2-A43F thatmeasure particle counts and nitrogen dioxide, respectively. The
3

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

Alphasense OPC-N3 is an optical particle counter with a nominal par­ticle monitoring size range from 0.35 to 40 μm, which has been widelyadopted in multiple previous studies (Bezantakos et al., 2018; Crilley
et al., 2018, 2020; Sousan et al., 2016). It provides particle counts in 24
size bins and estimates PM1, PM2.5, and PM10 mass concentrations basedon particle shape and density presumptions. Alphasense NO2-A43F is anelectro-chemical gas sensor that can operate in a variety of ambientenvironments but is known for its cross-sensitivity issue with NO and
CO2, while its cross-sensitivity with ozone is minimized with an ozone
scrubber. The NO2 readings are given as electric signals (millivoltages)
and need to be converted to parts per billion by volume. The low-cost
sensors are integrated into the CS, which is a standalone sensing plat­form with data storage and remote monitoring capability. The platform
is powered by a battery that can be charged through a power supply or asolar panel. The data from the sensors are stored on an onboard SD card,
which can be accessed remotely through an LTE connection for routine
checks on the data from sensors and the device’s status.

mobile validation in Boston. The mobile validation campaign occurredin neighborhoods north of Boston Logan International Airport and lasted
from February 11 to April 8, 2022, with data collected on 16 different
days. It is worth noticing that between the New York collocation andBoston mobile validation with TAPL, from September 13 to December
16, we continuously used these CS units for opportunistic mobile air
quality for four months (unattended in an open environment) on
municipal service vehicles. This study does not report these data as they
are irrelevant to sensor calibration.
2.3. Data processing and calibration
2.3.1. Data quality assurance and aggregation
In total, during the stationary collocation, 1.6 million unique 5-s
resolution data points were collected by the five CS units in New York.
During the mobile validation, 130,000 unique 5-s-interval data points
were recorded by two CS units in Boston. The CS data were then filteredby two criteria before they were aggregated. Fifteen percent of raw CS
data from stationary collocation were excluded in the process, where
12% percent were removed due to high relative humidity (>90% orraining), while the other 3% due to extreme readings (<1 μg/m3 or>1000 μg/m3 for PM2.5, <200 mv or > 900 mv for NO2 electro-signal).
The filtered raw CS data were aggregated into five temporal reso­
lutions, including minute-level, 5-min-level, 10-min-level, 30-min-level,
and hour-level. Minute-by-minute reference air quality data were
aggregated in the same intervals. Hourly meteorological data weregathered from the nearest LaGuardia Airport, about 4 km away. Raw CS
data, reference data, and meteorology were synchronized and matched,yielding five datasets of various temporal aggregations to explore data
aggregation’s effect on sensor calibration. Similarly, mobile validation
CS data were cleaned using the same criteria as in the stationary collo­
cation. The filtered CS and TAPL data were matched and aggregated to
the five temporal resolutions. Hourly meteorological data were obtained
from the Boston Logan International Airport (within 5 km of the drivingarea) and matched with air quality data.

2.2. Stationary and mobile data collection
As per EPA’s air sensor calibration guidelines (US EPA, 2021a,
2021b), the stationary collocation lasted four weeks, from August 13 to
September 10, 2021. With permission from the New York State Depart­
ment of Environmental Conservation, five CS units were placed next to areference station collecting PM and NO2 data every 5 s. The reference
station (Site ID: IS 52/MS 302) is located on the roof of New York City
Department of Education Public School 52/Middle School 302 in SouthBronx. The station measures criteria pollutants, particulate matter
speciation, hydrocarbons, and volatile organic chemicals continuously orintermittently. We obtained continuous minute-by-minute measure­ments of PM2.5 and NO2 at the station from NYSDEC, measured by Tel­edyne T640 at 5.0 L per minute (Federal Equivalent Method 236) and
Thermo Environmental Instruments 42C (Federal Reference Method074), respectively.
In addition to the stationary collocation, a mobile validation com­
parison campaign was carried out using the Tufts Air Pollution Labo­ratory (TAPL), which measured real-time particulate matter and NOx
concentrations (Hudda et al., 2020; Padró-Martínez et al., 2012). TAPL
is housed in an electric vehicle to avoid self-pollution and equipped withresearch-grade instruments, including a DustTrak™ DRX Aerosol
Monitor (Model 8533, TSI, Shoreview, MN) for size-resolved massfraction concentrations and a NO–NO2-NOx Analyzer (Model 42i,
Thermo Fisher, Waltham, MA) for NO/NO2/NOx concentrations. TAPL
instruments were synchronized with CS units using GPS time, reporting
PM2.5 and NO2 readings every second and every 10 s, respectively. TAPL
Dusttrak was calibrated against a regulatory site in Boston managed by
the Massachusetts Department of Environmental Protection (Nubian
Square, site ID 250250042). Detailed information on this calibration is
documented in the SI. Two CS units were mounted on the TAPL for

2.3.2. Development of calibration models
Our study evaluated seven distinct algorithms in two categories, thegeneralized linear regressions and machine learning algorithms, which
frequently appear in the existing literature. The algorithms wereselected in light of a study that assessed the air quality prediction per­formance by Kerckhoffs et al. (2019). Ordinary least squares (OLS) is themost widely adopted and straightforward algorithm in sensor calibra­
tion, which is also recommended by the EPA air sensor performance
evaluation guidelines. It is an effective tool for sensor calibration
providing intuitive results and straightforward interpretation. Elastic
net is essentially an extension of linear regression with penalty terms(Lasso and/or Ridge regularization) for correlated explanatory features.
While we acknowledge that there is great variability and capability inlinear regression algorithms, we only adopted the simplest form of thesealgorithms as recommended by the EPA guidelines for citizen scienceand educational purposes without considering feature transformations.
In addition, feature transformation and smoothing are taken into ac­
count in the generalized additive model (GAM), which estimates thetarget feature with linear combinations of smooth functions of the fea­tures. GAM models in this study did not consider interactions between
the features for better interpretability.
Regarding machine learning algorithms, supporter vector regression(SVR) is a commonly used regression method that uses a kernel function
to transform the data into a higher dimension and finds the hyperplanethat fits the data with a desirable error margin. Random forest and Light
Gradient Boosting Machine (LightGBM) belong to the bigger ensemble
modeling family but employ bagging and boosting techniques, respec­tively. Specifically, LightGBM is considered the state-of-the-art gradientboosting framework originally developed by Microsoft, which has fastertraining speed, higher efficiency, and lower memory usage while

Fig. 2. Illustration of k-fold time series cross-validation splitCalibration Algo­rithm Performance Evaluation and Transferability tests.
4

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

Table 1
Raw PM2.5 levels in μg/m3 collected by City Scanner and reference monitors during stationary collocation in New York City and mobile comparison in Boston.
New York City (μg/m3)
CS Unit 1
CS Unit 2
CS Unit 3
CS Unit 4
CS Unit 5
Reference

Boston (μg/m3)

25th percentile

50th percentile

75th percentile

25th percentile

50th percentile

75th percentile

1.6
2.8
1.7
2.3
2.2
5.6

2.6
4.1
2.8
3.6
3.6
7.3

4.1
6.2
4.4
5.6
5.5
10.3

–
–
3.9
–
0.2
1.2

–
–
5.4
–
0.5
2.2

–
–
8.1
–
1.1
3.0

yielding better accuracy (Ke et al., 2017). Artificial neural network(ANN) is popular in regression for prediction purposes, yet its black box
nature hinders its wider adoption in explanatory tasks.
For each temporally aggregated dataset, we adopted a k-fold timeseries cross-validation approach to evaluate calibration models’ per­formance (Pedregosa et al., 2011). The approach is designed to avoid
predicting past observations from future ones, limiting data leakage andperformance overestimation. Each dataset is first ranked temporally anddivided into five sequential sections with the same number of observa­
tions. In this case, the first k folds are used to train the models, and thek+1 fold split is used to validate and test the model. The train-test split isillustrated in Fig. 2. In the training and test sets, the CS readings forPM2.5 and NO2, and meteorological factors, including temperature in
Celsius, relative humidity in percentage, dew point in Celsius, and airpressure in kilopascal, were treated as explanatory features, and thereference pollutant readings were the target feature. Other meteoro­
logical factors, such as wind speed, wind direction, and feel tempera­ture, were also tested in the models but excluded due to statisticalinsignificance. Detailed feature descriptions and model-tuning infor­
mation are documented in the SI. All features were tested for theirnormality and transformed if not following a normal distribution. It is
worth noting that unlike the PM sensor, which provides mass concen­
tration, the NO2 sensor yields electric signals instead of parts per billion
by volume as the raw readings. Conventionally, one converts
electro-signals to gas concentrations using sensitivity factors provided
by the manufacturer from factory calibration, which is conducted forevery batch of sensors. Then, the converted gas concentrations arecalibrated with reference concentrations collected in the field. Our
approach directly mapped the electric signals to reference concentra­
tions rather than using factory-calibrated sensitivity factors, which canbe different between sensor batches and taking average within eachbatch.

though we train models with log-transformed CS and reference air
pollutant readings, both performance measures are calculated using
back-transformed predictions and observations.
⎛⎞2∑ny i − ̂y )⎟⎜2
i=1 (yi − y)(̂̅⎠
Eq.1
r = ⎝√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
∑n
2 ∑n
2
y i − ̂y )i=1 (yi − y)i=1 (̂
√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅∑n̂i )2i=1 (yi − y
RMSE =
n

Where:
yi and ̂y i are observed and predicted target features in the test
dataset;y and ̂y are mean values of observed and predicted target features inthe test dataset;n is the size of the test dataset.
The best-performing models and temporally aggregated dataset wereselected for each algorithm (7 algorithms × 5 temporal resolutions = 35
models). Mobile CS data collected by TAPL were passed to the models,
whose transferability was evaluated using the same performancemeasures.
2.3.4. Model interpretation
In addition to performance evaluation, we explored the impact of
meteorological factors on sensor and model performance. We focus themodel interpretation on the ensemble algorithms, as they are widelyadopted in the existing calibration literature to account for non-linear
relationships with an established global interpretation method. We
employed the Shapley Additive exPlanations (SHAP) method (Lundberg
et al., 2018; Lundberg and Lee, 2017) to interpret the trained random
forest and LightGBM model. The method is based on Shapley values, acommonly used metric to measure contributions from players in coop­erative game theory. It treats each explanatory feature as a player andcalculates its importance by comparing the model performance with andwithout the feature. Another useful visualization, the SHAP summary
plot, sorts the features by their global impact and calculates each fea­ture’s impact on the model output of each test sample, illustratingcomplex associations between the target and explanatory features.

2.3.3. Performance measures
In each fold of the time series cross-validation, the trained model wasassessed against the test dataset using two performance measures, r2 andthe root mean squared error (RMSE) as shown in Eqn 1 and Eqn 2. r2
simply represents the square of the Pearson correlation coefficient be­
tween predictions and observations. It measures the correlation between
calibrated CS readings and reference readings, while RMSE quantifies
the absolute difference between them. It is worth noting that even

3. Results

Table 2
Meteorological conditions during the stationary collocation and mobile
validation.
Duration andmeteorology

New York average (Standard
deviation)

Boston average (Standard
deviation)

Duration

August 13 - September 10,
2021
25.6 (±3.5)

February 11 - April 8,
2021
4.1 (±6.4)64.5 (±14.6)

58.7 (±21.8)

17.9 (±3.5)
101.5 (±0.5)− 4.1 (±8.6)
101.6 (±1.0)

Temperature(Celsius)
Relative humidity(%)
Dew point (Celsius)
Air pressure (kPa)

Eq.2

3.1. Descriptive analysis of stationary collocation and mobile validation
We first examined the raw data collected by CS units compared to thereference. As shown in Table 1, the average PM2.5 concentrations during
the sampling periods were 7.9 and 4.7 μg/m3, respectively, in New York
City and Boston. All five CS units underestimated PM2.5 concentrations
in stationary collocation by 62.1% on average in New York City. During
the mobile validation in Boston, CS Unit 3 overestimated PM2.5 con­centrations, while CS Unit 5 still underestimated PM2.5. The discrepancy
is mainly caused by the different particle characteristics in New Yorkand Boston and inter-sensor sensing variability for these particles. Theinter-sensor variability of CS units is exhibited, which indicates the5

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

Fig. 3. K-fold cross validation model performances (a) PM2.5 and (b) NO2 in a stationary setting for City Scanner Unit 1.

importance of developing unit-specific calibration models. The specif­
ically high inter-sensor variability in the Boston campaign is mainly
attributable to the continuous deployment in an unattended and open
environment in NYC. The average reference NO2 concentrations were
10.2 and 14.3 μg/m3, respectively, in New York City and Boston, which
is not included in Table 1 as CS provides only raw millivoltage readings.
The weather conditions were different during the NYC stationary
collocation and Boston mobile validation. Table 2 demonstrates themeteorological factors that were employed in the calibration algorithms.
While relative humidity and air pressure were relatively consistentregionally across the two monitoring campaigns, air temperature anddew point were about 20 ◦ C lower during the Boston mobile campaign
than NYC.

values reach over 0.8, compared to those of NO2 models around 0.7.
Still, it indicates the feasibility of calibrating CS NO2 readings directlyfrom millivoltage signals to reference concentrations. Among all algo­rithms, ordinary least squares and elastic net have demonstrated
consistently satisfactory performance in PM2.5 calibration, r2 ranging
from 0.6 to 0.8. This phenomenon is also observed in NO2 calibration,
indicating good linear responses between the low-cost sensors andreference sensors. RMSE values are very low after calibration, mostly
below 0.5 μg/m3 and 0.8 ppb for PM2.5 and NO2, respectively, consid­
ering that the median concentrations for PM2.5 and NO2 are 7.3 μg/m3
and 9.0 ppb in the stationary collocation. Machine learning models canalso deliver moderate calibration results but sometimes fail due tooverfitting the training data. Across all units and algorithms, we do not
observe an obvious effect of temporal aggregation interval on model
performance.
Fig. 4 tests the inter-sensor algorithm transferability for the bestperforming PM2.5 and NO2 models. Test data from a CS unit werepassed through models developed using data from another CS unit,
which process was repeated for all five CS units and all calibration al­
gorithms. 1-min aggregated data and models were used as they provide
the worst-case scenario in transferability and are most prone to over­fitting data from a specific sensor. It is hypothesized that linear re­
gressions should have the best transferability across sensors as they are

3.2. Algorithm performances in a stationary setting
We summarized the calibration algorithm performance (r2 andRMSE) of CS Unit 1 for PM2.5 and NO2 in Fig. 3. Performance summary
plots for other CS units can be found in the Supporting Information.
After calibration, all five CS units present good agreement with thereference data. In general, the algorithms provide good calibration re­sults for both PM2.5 and NO2, while PM calibration models have slightly
better performance than NO2 ones across all CS units. Best PM models’ r2
6

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

Fig. 4. Inter-sensor variability tests of best-performing 1-min (a) PM2.5 and (b) NO2 models. Values in the tables are r2 values calculated from passing sensor-specificdata (y-axis) to sensor-specific models (x-axis); darker color represents higher values.

7

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

attributed to a difference in particle hygroscopicity from city to city,
especially in the collocation region. We observe that the relationshipbetween air pressure and model output is non-linear. It is worth notingthat air pressure can be merely a good predictor rather than a good
explainer for sensor readings. It is in line with our main purpose todevelop high-accuracy calibration models. Dew point in the PM2.5 modeland temperature in the NO2 model demonstrate similar behaviors,proving the necessity of using non-linear models.
3.3. Algorithm transferability in a mobile setting
We further transfer the best-performing models trained with sta­tionary collocation data in NYC to predict mobile validation observa­
tions in Boston. We observe in Fig. 6 significantly deteriorated
performance across all algorithms for both CS units used in Boston(Units 3 and 5). The r2 values are low in the range of 0.1–0.3. Multiple
possible reasons lead to unsatisfactory model transferability. On the one
hand, mobile validation has brought extra challenges to sensor perfor­
mance. In mobile validation, the sensors can be exposed to a variety ofemission sources at different distances in a short amount of time. Suchdrastic changes in particle properties and pollutant concentrations are
difficult to account for in stationary calibration. Also, the sensors being
mobile validated were used for another mobile measurement campaign,
which has led to accelerated drifting and aging due to vibration thanpromised in the manual. On the other hand, some problems exist ubiq­
uitously for low-cost sensor calibration models in both stationary andmobile settings. First, the weather in NYC during stationary collocationand in Boston during the mobile validation are drastically different,
especially the temperature and dew points, which play important rolesin the PM2.5 and NO2 models, respectively. Some weather feature values
are unseen in the training data, thus, hard to extrapolate when trans­ferred to mobile data prediction. Second, while we use reference sta­tionary data to train our model, the target data to predict during the
Boston mobile validation are from research-grade instruments rather
than reference-grade ones. Third, it is common that the performance of
low-cost sensors degrades, and their behaviors change after long-time
outdoor usage, given that the stationary collocation and mobile vali­
dation were conducted more than six months apart.

Fig. 5. SHAP summary plots of best performing 1-min random forest models on
a City Scanner unit for (a) PM2.5 and (b) NO2. Dots represent explanatory
feature values and color-coded from low to high values in blue to red; data
points with the same SHAP value are stacked vertically; all features are ranked
from high to low feature importance on the y-axis.

the least likely to overfit. Nevertheless, for PM2.5 models, the general­ized additive model and random forest are observed to have the bestinter-sensor transferability, while the transferability of linear re­
gressions is moderate. For NO2, ordinary least squares and elastic net
models perform, on average, as well as random forest and LightGBM
models. But the latter two’s transferability still outperforms generalized
linear models developed for CS Unit 1. It is worth noting that models
developed for CS Unit 2 have the worst transferability, and thosedeveloped for CS Unit 1 have the best transferability for both pollutants
and all other algorithms.
Finally, we examine the model behaviors in the stationary settingfrom the best-performing random forest models for PM2.5 and NO2. The
results are visualized using SHAP summary plots, which consist of
marginal contributions of all data points of each explanatory feature,
represented by their SHAP values. Positive values denote positive ef­
fects, and higher absolute values denote larger effects on the output andvice versa. Data points are color-coded from blue to red, indicating low
to high values. Data points of each feature are stacked vertically if
different values have the same impact as in a violin plot. Features areranked by the absolute average SHAP values of all observations fromhigh to low, indicating high to low feature importance. In Fig. 5, logtransformed PM2.5 readings from CS units have the highest featureimportance. Moreover, its impact on the predictions is monotonic andpositive as the SHAP value (x-axis) increases as log PM2.5 values increase
with a smooth color change from blue to red (low to high feature
values). A similar phenomenon is found in the NO2 calibration model,
even though we mapped the electro-signal directly to reference read­ings. It is desirable that low-cost sensor readings can reflect air qualitychanges. Other than CS unit readings, the temperature is the second
most important that positively correlates with reference PM2.5 readings,
while in the NO2 model, the dew point feature plays the same role. The
effect of relative humidity is not significant in the PM2.5 and NO2
models, where it is positively correlated with NO2 readings but nega­tively correlated with PM2.5. This observation is opposite to previous
literature (Crilley et al., 2018; di Antonio et al., 2018), which can be

4. Discussion
This study systematically evaluated the effects of temporal aggre­
gation, calibration algorithms, and meteorological conditions on lowcost sensor calibration in both stationary and mobile settings.
We observed that linear regression models, including ordinary leastsquares and elastic net, perform consistently well in both PM2.5 and NO2
calibrations. Meanwhile, algorithms that consider non-linear relation­
ships, including random forest and GAM, provide good prediction ac­
curacy and better transferability between CS units than those that do notwhen calibrating PM2.5. This phenomenon is not found in NO2 calibra­tion models, where linear regression transfers as well as machinelearning models. It indicates that Alphasense NO2-A43F sensor provides
consistent linear responses in most circumstances with little inter-sensor
variability. It is recommended to employ algorithms that can account for
non-linear interactions between meteorology and low-cost sensor be­
haviors, especially for PM calibration. For most educational and citizen
science projects, generalized linear regressions are sufficient for lowcost sensor calibration as they are easy to apply and interpret. Robust
results can be reached if we properly follow the procedures recom­
mended in the EPA guidelines for field tests. We also observed that
random forest and GAM models are highly transferable between PM2.5
sensors, second by linear regression models. It supports adopting lowcost air sensors at a large scale to obtain big air quality data in urban
areas.
By interpreting the best PM2.5 and NO2 model behavior using SHAP
plots, we confirmed the non-linear relationships between
8

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

Fig. 6. Model performances in a mobile setting for two City Scanner units and five data aggregations. Box colors illustrate 1 min–60 min temporal aggregations fromdark to light.

meteorological features and model outputs, especially for PM calibra­tion. Moreover, we developed all models using only the CS readings andfour meteorological features without further feature engineering. It in­
dicates that capturing non-linearity and feature interactions might bemore effective in improving calibration models than incorporating andcompositing more features.
We observed that calibration model performance deteriorates whentransferred from stationary to mobile use. This is mainly caused byseveral reasons, some of which are ubiquitous to all low-cost sensors,
while others are unique to mobile measurement. The ubiquitous ones
include weather differences between stationary collocation and mobile
validation periods, using research-grade instead of reference-grade
monitors in the mobile validation, and performance degradation of the
low-cost sensors, including drifting and aging. Drifting and aging arecommon among all types of low-cost sensors (Kim et al., 2018; Malings
et al., 2019) and can happen after 2–6 months of usage (van Zoest et al.,2019). Moreover, mobile deployment between the stationary colloca­
tion and mobile validation reported in this study has led to accelerated
aging and drifting. It is suggested to calibrate low-cost sensors before
and validate after each deployment in an environment as similar to theactual monitoring as possible if it lasts more than six months and isunsupervised, as in our case. Regular collocations and calibrations arerecommended for low-cost sensors performed no more than six months
apart. Our work also demonstrates the risk of deploying low-cost sensors
in an environment different from the calibration environment. The lackof local reference instruments to perform calibration can constrain
low-cost sensor applications in low-income regions and countries;
alternatively, the viability of calibration (spatial and temporal differ­ences in calibration and deployment locations) should be acknowledged
in data analysis.
We acknowledge several limitations in our study. First, the PM2.5 andNO2 levels were low during the stationary collocation and mobile
9

A. Wang et al.

Atmospheric Environment 301 (2023) 119692

validation periods (<20 μg/m3 and <50 ppb, respectively). Ideally, an
as wide as possible air quality range should be included in the calibra­tion process to account for the variability of air pollution in mobile onroad deployments. Kelly et al. (2017) reported that some low-cost sen­sors begin to exhibit a non-linear response only when in high PM con­centrations (>40 μg/m3). Second, not all CS units are tested in themobile comparison due to limited space and resources. Lastly, while it ismore rigorous to calibrate low-cost sensors against Federal Reference
Methods instruments, our study used data from Federal Equivalent
Methods instruments as they are available minute-by-minute, which canbetter serve the purpose of our study.

Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.atmosenv.2023.119692.
References
Bezantakos, S., Schmidt-Ott, F., Biskos, G., 2018. Performance evaluation of the costeffective and lightweight Alphasense optical particle counter for use onboard
unmanned aerial vehicles. Aerosol. Sci. Technol. 52 (4), 385–392. https://doi.org/
10.1080/02786826.2017.1412394/SUPPL_FILE/UAST_A_1412394_SM0236.ZIP.
Brauer, M., Guttikunda, S.K., K A, N., Dey, S., Tripathi, S.N., Weagle, C., Martin, R.v.,
2019. Examination of monitoring approaches for ambient air pollution: a case study
for India. Atmos. Environ. 216, 116940 https://doi.org/10.1016/J.
ATMOSENV.2019.116940.
Castell, N., Dauge, F.R., Schneider, P., Vogt, M., Lerner, U., Fishbain, B., Broday, D.,
Bartonova, A., 2017. Can commercial low-cost sensor platforms contribute to air
quality monitoring and exposure estimates? Environ. Int. 99, 293–302.
Crawford, B., Hagan, D.H., Grossman, I., Cole, E., Holland, L., Heald, C.L., Kroll, J.H.,
2021. Mapping pollution exposure and chemistry during an extreme air quality
event (the 2018 Kı̄lauea eruption) using a low-cost sensor network. Proc. Natl. Acad.
Sci. USA 118 (27). https://doi.org/10.1073/PNAS.2025540118.
Crilley, L.R., Shaw, M., Pound, R., Kramer, L.J., Price, R., Young, S., Lewis, A.C., Pope, F.
D., 2018. Evaluation of a low-cost optical particle counter (Alphasense OPC-N2) for
ambient air monitoring. Atmos. Meas. Tech. 11 (2), 709–720. https://doi.org/
10.5194/AMT-11-709-2018.
Crilley, L.R., Singh, A., Kramer, L.J., Shaw, M.D., Alam, M.S., Apte, J.S., Bloss, W.J.,
Hildebrandt Ruiz, L., Fu, P., Fu, W., Gani, S., Gatari, M., Ilyinskaya, E., Lewis, A.C.,
Ng’ang’a, D., Sun, Y., Whitty, R.C.W., Yue, S., Young, S., Pope, F.D., 2020. Effect ofaerosol composition on the performance of low-cost optical particle countercorrection factors. Atmos. Meas. Tech. 13 (3), 1181–1193. https://doi.org/10.5194/
AMT-13-1181-2020.
di Antonio, A., Popoola, O.A.M., Ouyang, B., Saffell, J., Jones, R.L., 2018. Developing a
relative humidity correction for low-cost sensors measuring ambient particulate
Matter. Sensors 18 (9), 2790. https://doi.org/10.3390/S18092790.
deSouza, P., Kahn, R.A., Limbacher, J.A., Marais, E.A., Duarte, F., Ratti, C., 2020.
Combining low-cost, surface-based aerosol monitors with size-resolved satellite data
for air quality applications. Atmos. Meas. Tech. 13 (10), 5319–5334. https://doi.
org/10.5194/AMT-13-5319-2020.
Duarte, F., deSouza, P., 2020. Data science and cities: a critical approach. Harvard Data
Sci. Rev. 2 (3) https://doi.org/10.1162/99608F92.B3FC5CC8.
Gressent, A., Malherbe, L., Colette, A., Rollin, H., Scimia, R., 2020. Data fusion for air
quality mapping using low-cost sensor observations: feasibility and added-value.
Environ. Int. 143, 105965 https://doi.org/10.1016/J.ENVINT.2020.105965.
Hudda, N., Simon, M.C., Patton, A.P., Durant, J.L., 2020. Reductions in traffic-related
black carbon and ultrafine particle number concentrations in an urban neighborhood
during the COVID-19 pandemic. Sci. Total Environ. 742, 140931 https://doi.org/
10.1016/J.SCITOTENV.2020.140931.
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., Liu, T.-Y., 2017.
LightGBM: A Highly Efficient Gradient Boosting Decision Tree. 31st Conference on
Neural Information Processing Systems (NIPS 2017). In: https://proceedings.neur
ips.cc/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf.
Kelly, K.E., Whitaker, J., Petty, A., Widmer, C., Dybwad, A., Sleeth, D., Martin, R.,
Butterfield, A., 2017. Ambient and laboratory evaluation of a low-cost particulatematter sensor. Environ. Pollut. 221, 491–500. https://doi.org/10.1016/J.
ENVPOL.2016.12.039.
Kerckhoffs, J., Hoek, G., Portengen, L., Brunekreef, B., Vermeulen, R.C.H., 2019.
Performance of prediction algorithms for modeling outdoor air pollution spatial
surfaces. Environ. Sci. Technol. 53 (3), 1413–1421. https://doi.org/10.1021/acs.
est.8b06038.
Kim, J., Shusterman, A.A., Lieschke, K.J., Newman, C., Cohen, R.C., 2018. The Berkeley
Atmospheric CO2 Observation Network: field calibration and evaluation of low-cost
air quality sensors. Atmos. Measure. Tech. 11 (4), 1937–1946. https://doi.org/
10.5194/AMT-11-1937-2018.
Liang, L., 2021. Calibrating low-cost sensors for ambient air monitoring: techniques,trends, and challenges. Environ. Res. 197, 111163 https://doi.org/10.1016/J.
ENVRES.2021.111163.
Lundberg, S.M., Erion, G.G., Lee, S.-I., 2018. Consistent Individualized Feature
Attribution for Tree Ensembles.
Lundberg, S.M., Lee, S.-I., 2017. A unified approach to interpreting model predictions.
Section 2, 1–10.
Malings, C., Tanzer, R., Hauryliuk, A., Kumar, S.P.N., Zimmerman, N., Kara, L.B.,
Presto, A.A., Subramanian, R., 2019. Development of a general calibration modeland long-term performance evaluation of low-cost sensors for air pollutant gas
monitoring. Atmos. Meas. Tech. 12 (2), 903–920. https://doi.org/10.5194/AMT-12903-2019.
Miskell, G., Salmond, J.A., Williams, D.E., 2018. Use of a handheld low-cost sensor to
explore the effect of urban design features on local-scale spatial and temporal air
quality variability. Sci. Total Environ. 619, 480–490. https://doi.org/10.1016/J.
SCITOTENV.2017.11.024. –620.
Padró-Martínez, L.T., Patton, A.P., Trull, J.B., Zamore, W., Brugge, D., Durant, J.L., 2012.
Mobile monitoring of particle number concentration and other traffic-related air
pollutants in a near-highway neighborhood over the course of a year. Atmos.
Environ. 61, 253–264. https://doi.org/10.1016/J.ATMOSENV.2012.06.088.

5. Concluding remarks
Our work presents two major takeaways regarding effective androbust low-cost calibration with a special focus on mobile air quality
monitoring. We first demonstrated that PM calibration should consider
the complex relationship between sensor responses, pollutant concen­
trations, and meteorological factors. Therefore, algorithms that consider
non-linear relationships should be adopted in this case.
Models trained in the stationary setting can hardly be transferred tothe mobile setting in a different urban environment and climate; addi­
tionally, sensors degrade after long-term outdoor mobile deployment. It
is necessary to calibrate low-cost sensors in an environment that issimilar to real-world deployment. It is recommended to calibrate orassess comparability both before and after mobile deployments of du­rations exceeding six months. Our recommendation exceeds existing
EPA guidelines that are primarily focused on stationary monitoring.
The findings in our study are important to citizen scientists, air
quality researchers, and practitioners interested in advancing low-cost
sensor applications, such as raising awareness and community engage­
ment in scientific analysis. Further research is needed on othercommonly used low-cost particulate matter and gas sensors and low-cost
sensor performance in temperature, humidity, and concentration range
not included in this study. Finally, there is a need to assess low-cost
sensor performance in mobile settings given the rapidly increasing useof and demands for mobile environmental sensing.
Author credit statement
First author An Wang was responsible for study conceptualization,
data curation, formal analysis, methodology as well as visualization,
writing original draft, manuscript review, and editing. Yuki Machidaresponsible for data curation, investigation, and writing, reviewing, andediting the manuscript review. Priyanka deSouza was responsible forstudy conceptualization, and manuscript review and edit. Simone Mora
was responsible for project administration, resources, and manuscript
review and editing. Tiffany Duhl was responsible for data curation andmanuscript review and edit. Neelakshi Hudda was responsible for data
curation and manuscript review and eidt. John L. Durant was respon­sible for resources and manuscript review and edit. Fabio Durate was
responsible for supervision, resources, and manuscript review and edit.
Carlo Ratti was responsible for funding acquisition. All authors reviewed
the results and approved the final version of this manuscript.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Data availability
Data will be made available on request.

10

A. Wang et al.

Atmospheric Environment 301 (2023) 119692
US EPA, 2021b. Performance Testing Protocols, Metrics, and Target Values for Ozone Air
Sensors: Use in Ambient, Outdoor, Fixed Site, Non-regulatory and Informational
Monitoring Applications.
van Zoest, V., Osei, F.B., Stein, A., Hoek, G., 2019. Calibration of low-cost NO2 sensors inan urban air quality network. Atmos. Environ. 210, 66–75. https://doi.org/10.1016/
J.ATMOSENV.2019.04.048.
Wang, S., Ma, Y., Wang, Z., Wang, L., Chi, X., Ding, A., Yao, M., Li, Y., Li, Q., Wu, M.,
Zhang, L., Xiao, Y., Zhang, Y., 2021. Mobile monitoring of urban air quality at highspatial resolution by low-cost sensors: impacts of COVID-19 pandemic lockdown.
Atmos. Chem. Phys. 21 (9), 7199–7215. https://doi.org/10.5194/acp-2020-1169.
Zheng, T., Bergin, M.H., Johnson, K.K., Tripathi, S.N., Shirodkar, S., Landis, M.S.,
Sutaria, R., Carlson, D.E., 2018. Field evaluation of low-cost particulate mattersensors in high-and low-concentration environments. Atmos. Meas. Tech. 11 (8),4823–4846. https://doi.org/10.5194/AMT-11-4823-2018.

Pedregosa, F., Michel, V., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R.,
Vanderplas, J., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, É., 2011. Scikitlearn: machine learning in Python. J. Mach. Learn. Res. 12 (85), 2825–2830. http://jmlr.org/papers/v12/pedregosa11a.html.
Sm, S.N., Reddy Yasa, P., Mv, N., Khadirnaikar, S., Rani, Pooja, 2019. Mobile monitoringof air pollution using low cost sensors to visualize spatio-temporal variation ofpollutants at urban hotspots. Sustain. Cities Soc. 44, 520–535.
Sousan, S., Koehler, K., Hallett, L., Peters, T.M., 2016. Evaluation of the Alphasense
optical particle counter (OPC-N2) and the Grimm portable aerosol spectrometer(PAS-1.108). Aerosol. Sci. Technol. 50 (12), 1352–1365. https://doi.org/10.1080/
02786826.2016.1232859/SUPPL_FILE/UAST_A_1232859_SM0343.ZIP.
US EPA, 2014. Air Sensor Guidebook. https://cfpub.epa.gov/si/si_public_record_Report.
cfm?Lab=NERL&dirEntryId=277996.
US EPA, 2021a. Performance Testing Protocols, Metrics, and Target Values for Fine
Particulate Matter Air Sensors: Use in Ambient, Outdoor, Fixed Sites, Non-regulatory
Supplemental and Informational Monitoring Applications.

11