A machine learning paradigm for necessary observations to reduce uncertainties in aerosol climate forcing

Domain: earth_science
Published: 2024-09-27
Editorial outcome: accept

Abstract
Uncertainties in estimates of climate cooling by anthropogenic aerosols have not decreased significantly in the last two decades, partly because observational constraints on crucial aerosol properties simulated in Earth System Models are insufficient. To help address this insufficiency in aerosol observations, we describe a paradigm for deriving higher-level aerosol properties with machine learning algorithms that use only lidar observations and reanalysis data as predictors. Our paradigm employs high-accuracy suborbital lidar and collocated in situ measurements to train and test two fully-connected neural network algorithms. We use two lidar datasets as input to our machine learning algorithms. The first dataset consists of suborbital lidar observations not previously used in the training of the machine learning algorithms. The second dataset consists of simulated UV-only observations to preview the algorithms’ predictive capabilities in anticipation of data from the ATmospheric LIDar system on the Earth CARE satellite, which was launched in May 2024. Here we show that our algorithms predict two crucial aerosol properties, aerosol light absorption and cloud condensation nuclei concentrations with unprecedented accuracy, yielding mean relative errors of 21% and 13%, respectively, when suborbital lidar data are used as predictors. These errors represent significant improvements over conventional aerosol retrievals. Applied to future satellite missions, the paradigm presented here has great potential for constraining Earth System Models and reducing uncertainties in their estimates of aerosol climate forcing and future global warming.

Introduction
Anthropogenically produced atmospheric aerosols are estimated to exert an average cooling of Earth’s climate that offsets between one-fifth and one-half of the warming induced by greenhouse gasses1. While our understanding of the role of atmospheric aerosols in the Earth system has evolved steadily over the past two decades, uncertainties in aerosol forcing of climate have not decreased commensurately between recent IPCC (Intergovernmental Panel on Climate Change) reports2,3. This is partly due to continuous, new discoveries of previously unknown, complex interactions of aerosols with clouds that drive the aerosol indirect radiative forcing. Arguably the most significant hindrance to the reduction of uncertainties in aerosol forcing of climate though is the limitation of observational constraints on crucial aerosol properties simulated in Earth System Models (ESMs), e.g., refs. 1, 4. We note that past satellite observations have yielded essential global-scale insights into the distribution of atmospheric aerosols5–8. Likewise, suborbital measurements have provided insights into aerosol processes by providing high-accuracy measurements not obtainable from space. However, neither type of measurement has yielded the data to constrain aerosol properties and processes in global-scale ESMs to result in notable reductions in uncertainties of aerosol climate forcing, primarily because observations are often limited in accuracy and spatiotemporal resolution and coverage.

Specifically, the vertical distribution of key aerosol properties, particularly in the remote atmosphere and near those types of cloud that are highly susceptible to aerosol-induced modifications, has proven an elusive measurement challenge for the observational community9. In this paper, we describe an ML-based paradigm for observations of crucial aerosol properties, specifically aerosol light absorption (ABS) and cloud condensation nuclei (CCN) concentrations, that can be applied to satellite lidar measurements in the next decade, but also to past suborbital lidar observations. These lidar-based ML predictions yield aerosol data in closer proximity to clouds because the active lidar measurements are not subject to some of the artifacts that afflict passive remote sensing of aerosols in the immediate vicinity of clouds10. ABS and CCN are aerosol properties that have not been traditionally derived from lidar measurements – we refer to these aerosol properties as higher-level aerosol properties hereafter. We note that our paper intends to present the paradigm of estimating higher-level aerosol properties with ML algorithms that use only lidar observations and reanalysis data as predictors. These ML algorithms are trained with collocated lidar measurements and in situ observations, the latter representing the truth regarding the higher-level aerosol properties. The paradigm is therefore only possible because of the recent collection of carefully coordinated lidar and in situ aerosol observations. We fully expect that future ML models that use lidar observations as input can be further optimized with evolving ML techniques and additional training data.

New satellite mission concepts are being developed to address some of the observational gaps in aerosol properties that determine aerosol climate forcing. They typically include multi-angle polarization-sensitive passive remote sensing of column-integrated aerosol quantities and active remote sensing of the vertical distributions of aerosol backscattering, extinction, and depolarization with lidars, e.g., NASA’s Atmosphere Observing System (AOS)11, and ESA/JAXA’s EarthCARE (Earth Cloud, Aerosol and Radiation Explorer)12. These aerosol retrievals are typically complemented with observations of crucial cloud properties (e.g., cloud droplet number concentration and effective radius, liquid and ice water path) and meteorological conditions (e.g., updraft velocity) to elucidate aerosol-induced changes in cloud properties and lifetimes (e.g., ref. 13). We note that only the lidar observations provide the vertically resolved aerosol information that is crucial for addressing the most uncertain aerosol climate interactions. Hence, the paradigm developed here is primarily aimed at retrieving essential aerosol properties from spaceborne lidar observations expected to be available in the future, either from the recently launched (EarthCARE) or from the AOS project that is currently in development. Because EarthCARE and its lidar system ATLID (Atmospheric LIDar) were launched recently (May 2024) and their Level 2 data is expected to be released publicly in the spring of 2025, we include ML predictions that use only UV observables in the training of the ML models and then test the models by using noise-added UV lidar observations as input. The actual error characteristics of the ATLID system will ultimately determine the errors in ML predictions that use ATLID observations as input. Nonetheless, we believe that our analysis provides a meaningful bounding exercise for the expected ML predictions based on ATLID observations.

Most remote sensing of aerosol properties relies on physics-based retrievals in an optimal estimation framework14. Physics-based refers to the fact that these retrievals use forward models that describe the complex interactions of electromagnetic radiation with the aerosol itself, but also the Earth’s surface and the atmosphere15. More recently, such retrieval approaches have been applied to joint observations from lidars and polarimeters, with great promise for the increased information content from the combined observations16. The grand challenges for physics-based aerosol remote sensing include: (a) some details of the underlying electromagnetic scattering processes in forward models (e.g., scattering of polarized radiation with non-spherical aerosol particles and polarized surface reflectance) are unknown; (b) uncertainties in a priori retrieval assumptions propagate more or less directly into the retrieved quantities; and (c) the optimal estimation techniques are computationally expensive.

The paradigm shift presented here involves the training of ML models with lidar data and in situ measurements and then applying the models to an arbitrary lidar dataset for the retrieval of vertically resolved higher-level aerosol properties. The ML models are trained using high-accuracy lidar measurements and collocated in situ observations of key aerosol properties that are not traditionally derived from lidar observations alone. Atmospheric reanalysis data are used to constrain the ML predictions further. Because lidar systems anticipated for space deployment in the future will include only a subset of observations of the airborne lidar system we use in the training of our ML models presented here, our models can be applied directly to future satellite missions after retraining them with the available subset of observables for these missions. We illustrate this paradigm by using airborne lidar observations and collocated in situ measurements from four recent airborne field campaigns to train two sets of ML algorithms to predict aerosol light absorption (ABS) and Cloud Condensation Nuclei (CCN), respectively. These higher-level aerosol quantities are arguably the most important aerosol properties for determining aerosol effects on clouds and climate1,17. The majority of CCN data (>77%) available to this study was measured at supersaturations in the 0.35-0.4% SS range, hence our choice to focus the ML retrievals on CCN~0.4%SS, although we omit the subscript in the remainder of the manuscript for readability. ABS determines the direct interactions of aerosols with solar radiation and it affects clouds by changing local atmospheric heating rates and atmospheric stability18. CCN is the fraction of aerosols capable of forming cloud droplets. Although a fraction of them are too small to be optically active (e.g., ref. 19), they are the key ingredient for understanding how atmospheric aerosols affect cloud droplet number concentrations, and hence the radiative properties, precipitation, and lifetimes of clouds20,21.

Our work builds on research in which we have shown that linear regression models between lidar-derived aerosol extinction or backscattering and CCN are aerosol-type specific and only work for a limited range of relative humidities (RH)22. This is caused by the non-linear response of aerosol optical properties to RH increases, while CCN concentrations remain constant. The ML models we developed can predict this exact behavior from the in situ observations. Both ABS and CCN are difficult to constrain with current satellite measurements and physics-based retrieval approaches. However, global observations of their vertical distribution are urgently needed to constrain physical processes and aerosol climate-forcing estimates in ESMs. In the remainder of this paper, we use the term retrieval as a short-hand term for the predictions of the ML models, despite our awareness of the fundamental differences to the physics-based retrievals discussed above.

Results
In this section, we discuss the performance of the ML algorithms for CCN and ABS predictions jointly, because of the general similarities in the algorithms’ training and validation. We present results that use the full suite of HSRL-2 observables to train our ML models and as input to the ML predictions in the “ML predictions using HSRL-2 data as input” section. In the “ML predictions using simulated EarthCARE/ATLID observations as input” section, we discuss the performance of ML models trained with the lidar observables available for the EarthCARE lidar system ATLID (i.e., aerosol backscatter, depolarization, and extinction at only one wavelength, 355 nm). For this configuration, we provide predictions of CCN and ABS using the HSRL-2 UV observations as input to the model after applying random noise to each observation. The noise generation is described in the “Data availability” section.

ML Predictions using HSRL-2 data as input
Using the HSRL-2 data as input to the ML predictions, Fig. 2 indicates that the ML predictions yield correlations of 0.93 (0.80) for CCN (ABS) when lidar observables are used to train the ML models; these correlations increase to 0.97 (0.90) for CCN (ABS) when the T and RH reanalysis data is used as additional constraints. The density plot function indicating data density in Fig. 2 is adapted from the original dscatter function provided by MATLAB. First, the natural logarithms of the original X and Y variables are calculated. These values are then divided into 200 equal logarithmic bins to create a 2D histogram for density calculation. Finally, the individual data point densities are scaled by the maximum density to a range of 0–1.

For CCN (ABS), the ML predictions based on lidar observations alone [panels a(c), also Fig. 3, and Table 1] have mean relative errors of 22% (25%). These errors are reduced to 13% (21%) when reanalysis products are used as additional predictors [panels b(d)]. Other performance metrics, including the predictions’ mean absolute error (MAE), the correlation coefficient R, and the number and fraction of predictions within either 30% or 50% error are summarized in Table 1.

ML predictions using simulated EarthCARE/ATLID observations as input
In analogy to Figs. 2, 4 shows the ML predictions for ML models trained with EarthCARE/ATLID observables only and using noise-added HSRL-2 UV observations as input to simulate an application to satellite measurements anticipated to be released publicly in the spring of 2025, as described above. Figure 4 indicates that ML predictions thus produced yield correlations with the test data of 0.57 for CCN and 0.55 for ABS, respectively, when only the lidar observables are used to train the ML models (panels a and c); these correlations increase to 0.88 and 0.74, respectively, when the reanalysis data is incorporated as additional constraints.

For CCN (ABS), the ML predictions based on lidar observations alone and applied to simulated ATLID measurements [panels a(c), also Fig. 5, and Table 1] have mean relative errors of 51% (40%). These errors are reduced to 23% and 28%, respectively, when reanalysis products are used as additional predictors [panels b(d)]. Other performance metrics, including the predictions’ mean absolute error (MAE), and the number and fraction of predictions within either 30% or 50% error are summarized in Table 1.

Methods
Details of the suborbital data used for the training and testing of ML models
For the training and testing of our ML algorithms, we used lidar data that was spatially collocated with in situ observations within horizontal distances of 1100 m and vertical distances of 45 m, and coincident within 30 min. These collocation criteria yielded 9873 lidar measurements paired with in situ CCN measurements in the supersaturation range of 0.35%–0.4% at which the majority of in situ CCN data were measured, and 2516 lidar measurements coincident with ABS measurements to train our ML models. The in situ observations of ABS and CCN were taken from a second aircraft that flew in close spatio-temporal coordination below the HSRL platform. For a small fraction of data (<2%), the in situ observations were collected from the same aircraft as the HSRL after maneuvering the aircraft into the column previously observed below the aircraft by the HSRL.

The archived horizontal resolution of HSRL-2 aerosol backscatter, depolarization, and extinction measurements depends on aircraft speed and is on average 1500–2000 m corresponding to a 10-s average of the native lidar observations; the vertical resolution is 15 m. Our spatial averaging and collocation scheme is intended to find all in situ observations within an 1100 m horizontal distance of the central lidar data location, and it allows for vertical variations of the in situ aircraft inside a horizontal HSRL bin. The vertical averaging is set to 45 m, so that a maximum of 3 vertical lidar measurement bins and 20 in situ measurement bins may be located inside each collocation bin, using the in situ data sampling of 1 Hz and an assumed average aircraft speed ~100 m/s for the in situ aircraft. These collocation criteria are adapted from our recent work on simple correlation studies22.

We used CCN data from continuous flow CCN counters and absorption data from Particle Soot Absorption Photometers (PSAP). CCN counters measure CCN concentrations at different levels of supersaturation (SS) in standard mode and scanning-flow mode32. The CCN counters in each of the four experiments were operated in similar modes, typically alternating between periods of constant supersaturation (SS) or scanning through SS ranges to create CCN spectra. The majority of CCN data (>77%) available to this study was measured in the 0.35%–0.4% SS range, hence our choice to focus the ML retrievals on CCN~0.4%SS, although we omit the subscript in the remainder of the manuscript for readability. The uncertainty associated with the CCN number concentration is typically 10% at high signal-to-noise ratios. The PSAP is a filter-based instrument that collects aerosol particles on a substrate and measures the change in light transmission at three wavelengths (467, 530, and 660 nm) relative to a reference filter. Bulk aerosol absorption is derived after correcting for scattering effects. PSAP uncertainty is an ongoing research topic and depends on filter loading, aerosol type, and corrections applied to data. Under relatively clean conditions, a recent study33 estimated PSAP ABS uncertainties of 40%. In aircraft operations, PSAP uncertainty is highly sensitive to pressure changes. Thus, we carefully filtered the PSAP data and discarded measurements when consecutive aircraft altitude variations exceeded 5 meters vertically. Both instrument types were operated in all four campaigns, providing data at a temporal resolution of 1 s. Before collocating with HSRL lidar observations, CCN concentrations below 10 cm−3 and ABS below 0.1 Mm−1 were discarded to account for the low sensitivity and relatively larger uncertainty of both measurements in clean conditions.

We note that in our methodology the lidar data used as input to the ML algorithms are independent of the lidar data used to train the algorithms in the first place unless the dataset shows strong spatial autocorrelation at the horizontal sampling scales (2.2 km) we applied. We tested the potential spatial autocorrelation in our dataset by removing any data points that were located within a horizontal distance of ±5 km from any other data points in the dataset. The results of this exercise show a slight decrease in the predictive capability of our algorithms (see Fig. S2), for example from 12.9% (see Table 1) mean relative error in CCN to 16.9% (see Fig. S2a). However, even this slight decrease in predictive capability cannot be attributed to spatial autocorrelation of the original dataset, because the simple reduction of the number of data points in the reduced dataset may have a similarly negative impact on the algorithms’ predictive performance. To further test the fidelity of our ML retrieval results, we investigated the impact of the completeness of the training dataset in terms of the range and continuity of the suborbital in situ measurements of ABS and CCN. These tests indicate sufficient sampling completeness for the ABS and CCN datasets used in the training of our models and are described in the supplementary materials, specifically Figs. S3 and S4 and the accompanying text.

Reanalysis data used as additional predictors in the ML models
In addition to using the lidar observables as predictors in our ML algorithms, we explored the use of reanalysis data, in particular temperature (T) and relative humidity (RH), as additional predictors. They were added in the input layer at the same level as the lidar observables and their addition did not change the architecture of our ML algorithms. The reanalysis data, sourced from the ERA5 (ECMWF Reanalysis v5) product34, were spatiotemporally interpolated to match the lidar data locations. ERA5 reanalysis data has a spatial resolution of 0.25°×0.25° in the horizontal, 37 pressure levels in the vertical, and a temporal resolution of 1 h. The product is spatially and temporally interpolated to the exact time and location of the airborne data using linear interpolation.

Figure S6 shows the frequency distribution of RH and T in the reanalysis dataset collocated with our lidar and in situ observations in each campaign, similar to the frequency distributions of CCN and ABS in the inset of Fig. 1. We chose T and RH as additional predictors because they describe the thermodynamic state of the atmosphere in which the lidar observations were collected. RH, in particular, determines the size and refractive index of atmospheric aerosols under ambient conditions and thereby the lidar-observed aerosol optical properties, i.e., extinction and backscattering35. We note that the frequency distributions of RH and T between the four airborne campaigns in Fig. S6 showed many similarities and significant overlap in RH frequency distributions in particular. Therefore, it seems unlikely that the Fully Connected Neural Networks (FCNN) that we employ inadvertently distinguish and preferentially select training data from any one of the four flight campaigns. The addition of reanalysis data improves the ML retrieval capabilities significantly, as we show above.

Features of the ML architecture
Our ML algorithms are based on Deep Learning (DL) with FCNN. We trained a feedforward neural network regression model using the Levenberg-Marquardt algorithm36,37. While such ML algorithms have been used extensively in the atmospheric sciences and specifically in Meteorology38, their use in retrievals of geophysical variables from remote sensing observations is relatively new (e.g., ref. 39). We explored the use of FCNN and Random Forest algorithms for our application. Both methods demonstrated comparable performance in predicting CCN and ABS. However, the training time for the Random Forest algorithm was nearly twice as long as that for the FCNN. Consequently, FCNNs were chosen for the final model training in this study. For future applications of this proposed paradigm, it is possible to implement other training algorithms within this framework.

In our DL architecture, the lidar observables and reanalysis data constitute the predictors, while the in situ measured CCN and ABS constitute the algorithm responses. For each algorithm, we use 70% of the available collocated lidar and in situ observations as the training dataset, 15% as the validation dataset, and 15% as the test dataset. The models’ hyperparameters are tuned iteratively using Bayesian optimization. We use 10-fold cross-validation, which means that we train the ML models on subsets of the training data and validate them with the complementary subsets ten times. The training of neural networks largely depends on the selection of optimal hyperparameters (including the number of layers, number of neurons per hidden layer, activation functions, etc.). Therefore, we have included the optimal sets of these important hyperparameters specific to the models we trained for this study in Table S2. Further details of our data selection, along with a description of the FCNN and its setup can be found in Fig. S1, and Tables S1 and S2 in the supplementary information.

During the review of our manuscript, several reviewers commented on the relatively larger uncertainties in our ML predictions of CCN and ABS in clean conditions (i.e., CCN < 100/cm3; ABS < 1 Mm−1). In response, we pursued a technique that is commonly used in ML algorithms and entails an extra weighting of data points in data-sparse regions, in this case, the clean conditions. Specifically, we explored the retraining of our ML models for CCN/ABS with a training dataset that assigned extra weighting to the data in clean conditions (CCN < 100/cm3; ABS < 1 Mm−1). A risk to such weighting of data in specific data ranges is that it can result in less good fitting of the data outside of the extra-weighted data range. We investigated additional weighting of data in clean conditions by up to a factor of 10 and found that the mean relative uncertainty (MRE) in CCN could be reduced significantly with an increased weighting of data in clean conditions by a factor of 3, with relatively little impact on the MRE in overall conditions (i.e., all predictions). The improvement in ML predictions at the low CCN range between equal weighting of all data versus triple weighting of the data in clean conditions is illustrated in Fig. S5. As a result of this analysis, all ML model predictions of CCN in this study use a triple weighting of training data in the clean CCN conditions.

An analogous analysis for extra weighting of ABS training data in clean conditions indicated that extra weighting of ABS training data in clean conditions resulted in significantly increased MRE in ABS predictions for all conditions. The reason for the greater impact of the extra weighting of the ABS training data in clean conditions on the overall ABS prediction performance (by comparison to CCN retrievals) is that a relatively larger fraction of ABS data resides in clean conditions (more than 60% of overall ABS data). Unlike the CCN predictions, the ABS predictions in clean conditions are climatically less important than the predictions at higher absolute ABS. In combination with the increased penalties, i.e., the higher MRE for overall ABS conditions, we decided to use equal weighting of all ABS training data.

Simulating EarthCARE/ATLID error characteristics
In the “ML predictions using simulated EarthCARE/ATLID observations as input” section, we discussed the performance of ML algorithms trained with the set of lidar observables that is anticipated to be available for the EarthCARE lidar system ATLID (i.e., aerosol backscatter, depolarization, and extinction at only one wavelength, 355 nm). The uncertainties we applied are derived from actual comparisons between airborne HSRL-2 observations and collocated spaceborne CALIOP (Cloud-Aerosol LIdar with Orthogonal Polarization) extinction and backscatter retrievals. For the HSRL backscatter data used as input to the ML models, we assigned random noise by assuming a Gaussian distribution, such that the mean of the noise added equals the mean relative difference between the collocated CALIOP and HSRL data. We then propagated the noise in the backscatter data to the extinction observations using uncertainties in lidar ratios published in the literature40. In this manner, we created a UV-only dataset of aerosol backscatter, extinction, and depolarization from airborne HSRL-2 measurements that is noisier than the HSRL-2 airborne observations themselves, in an attempt to simulate the error characteristics of the EarthCARE/ATLID system. Such a noise envelope may well be an overestimate of the uncertainties that we can expect from future spaceborne HSRL and night-time Raman observations because the CALIOP extinction retrievals are subject to systematic errors in the assumptions of lidar ratios, which are necessary in the inversion of elastic backscatter lidar measurements as is the case for CALIOP.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Comments on manuscript: NCOMMS-24-11301A-Z\nImproving our knowledge about aerosol cooling is crucially important to reduce uncertainties in\nclimate assessment. Large-scale observations of aerosol properties are needed to constrain Earth\nSystem Models (ESMs). For examples, aerosol light absorption (ABS) and cloud condensation\nnuclei (CCN) play key roles in estimating aerosol direct radiation and aerosol-cloud interactions\neffects, respectively, but large-scale observations of these two crucial properties with good\naccuracy are still lacking. The manuscript of Redemann et al. addresses this problem by combining\nmachine learning algorithm and satellite/in-situ/reanalysis data to predict ABS and CCN. The\npredictions are significantly improved over conventional aerosol retrievals, with mean relative\nerrors of 19% for ABS and 13% for CCN. These results will help ESMs improve their aerosol\nsimulations therefore aerosol cooling assessments.\nGeneral comments:\nThe study is well written and structured. Some details about the methodology and its validation are\ncurrently missing or need to be better justified. If the conclusions remain qualitatively similar after\nthe issues presented below have been addressed appropriately this study will provide valuable\ninsights into the aerosol simulations in ESMs therefore aerosol cooling in climate assessments.\nSpecific Comments:\nMethodology\n1) Some key descriptions of input dataset for machine learning and ML algorithm itself are missing,\nwhich is important for readers to understand the design of study and use it if they are interested.\nYou could consider putting them in the supplement considering the world limit in main context.\nPlease clarify (not limited to):\n-Measurements:\na. temporal and spatial resolution of HSRL-2 measurements\nb. regarding the CCN measurement, do they run the CCN counter in the mode of certain\nsupersaturations with scanning aerosol sizes or certain sizes with scanning supersaturations? Time\nresolution? What are the typical size distribution and number concentrations of aerosol particles in\nthis study? I am asking these because the supersaturation (0.35-0.4%) looks very narrow and\nrelatively high, which may not cover the full meteorological regimes and lead to inaccurate\nextrapolation of CCN predictions in Fig 6. How do you cope with this situation?\nc. how do you derive aerosol types shown in Fig. 1\nd. how do you consider the representativeness of the point in-situ measurement compared to the\nHSRL grid box?\n-Reanalysis data:\na. describes the reanalysis data that you used, including the temporal and spatial resolution, the\ninterpolation method to match observation data. Do you use surface level of RH and T to match the\nin-situ observations?\n-Machine learning algorithm:\na. a full list of input parameters and hyperparameters.\nb. reason to choose FCNN (not other DL algorithms). Is the input sample size sufficient for ML\ntraining in this study? What is the optimised hyperparameter used in this study?\nc. page 4, line 17, clarify UV HSRL-2 observables.\nd. page 4, line 14-18, clarify reasons to perform two analysis and add noise to UV HSRL-2 in the\nsecond analysis. The first paragraph of results section explained the reasons well, suggest moving\nsome sentences to the methodology.\nMachine learning validation and analysis\n2) Based on Fig. 2 the validation via the linear regression fit is done with a zero-intercept. This\nintroduces a bias to the slope and correlation coefficient.\n3) In Fig. 2, the errors become larger (even exceeding 50%) when the in-situ CCN and ABS are low.\nWhat are the possible reasons for this? (i.e., out of ML prediction range? Measurement\nuncertainties?)\n4) Compared to RH and T, atmospheric instability is a more direct indicator of updraft determining\nCCN activation. It could be helpful to improve prediction accuracy if available.\n5) Clarify your key conclusion of Fig.6.\nMinor/editorial comments:\n6) Page 1, line 8-9, “…decreased significantly in the last two decades, largely because observational\nconstraints on crucial aerosol properties simulated in Earth System Models (ESMs) are\ninsufficient”. This statement is not accurate, insufficient aerosol measurements and constrains\npartly induce the large uncertainty in aerosol cooling as demonstrated in the introduction, but\ncannot claim as “largely because..”. Cloud processes and feedbacks are important too.\n7) Page 2, line 25, “They also yield aerosol properties that have not been traditionally derived from\nlidar measurements”. Which properties do you mean? Clarify it.\n8) Page 3, line 18-20, “Because lidar systems anticipated for space deployment in the future will\ninclude only a subset of observations of the airborne lidar system we use in the training of our ML\nmodels presented here, our models are directly applicable to future satellite missions”. A bit\nconfused here, if you have incomplete input parameters for ML model from observables in lidar\nsystems on space deployment, how can you predict the anticipated properties?\n9) Page 5, line 3-4, how do you get ATLID data for ML training since it has not launched yet?\n10) Page 6, line 25, clarify the uncertainty range of lidar ratios in the literature.\n11) Page 7, line 16, “relative errors of 20%”, clarify “mean relative errors”.\nReviewer #1 (Remarks on code availability):\nThe attached code contains a README file and a zip file including code and data to generate the\nfigures in the main context and supplement. The instructions are clear to reproduce the figures. The\nmachine learning algorithm and validation code are not available yet if it is requested to be shared\ntoo.", "reviewer_2": "Review of Redemann and Gao\nThe authors present work using machine learning methods to prediction aerosol light absorption\nand CCN concentrations from remote sensing observations. This is a highly important research\narea, and the results the authors present are promising. While the work is very interesting, there are\nseveral major issues with the manuscript that are described below.\nData and Methodology\nMethodological Baselines\nThe authors use a feed forward neural network. This method is not justified, beyond the fact that it\napparently works. An appropriate baseline (e.g., linear model) is necessary to determine if the\ncomplexity of the machine learning approach is justified.\nData Sampling\nThe authors use data from a variety of field campaigns, and then apply various filtering\nrequirements. It is not clear (e.g., Figure 1) how much of the data used here is from any particular\ncampaign. My assumption is that the data are not evenly distributed across the campaigns, which\ncould very well bias the conclusions here.\nMERRA-2\nIt is unclear why the authors use MERRA-2 data over another reanalysis product that potentially has\nhigher resolution. Further, the use of temperature and relative humidity alone from these products\nis not well justified. Why not include other variables?\nAnalytical Approach\nThe final architecture of the neural network should be included in supplemental material in\naddition to the hyperparameter search.\nClarity and Context\nMost of the in-situ observations appear to be over the ocean, do the authors believe their findings\ngeneralizable to land surface as well? If so, why?\nMinor Comments\nP5 L24: It’s not clear how the reanalysis product data was implemented into the ML model e.g. via\ntwo new nodes in the input layer or another avenue. It should also be stated if the model was\nretrained after this addition and whether or not the architecture was changed or optimized with\nrespect to the altered dataset.\nP5 L44: Horizontal spatial auto-correlation was discussed and considered but the vertical spatial\nauto-correlation was not.\nIt would be helpful to have color bars for the density of points in Figures 2 and 4. Table 1 is\nadditionally very difficult to read (what do the arrows mean?).", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part\nof the Nature Communications initiative to facilitate training in peer review and to provide\nappropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_4": "The manuscript addresses the urgent need to improve constraints on observed aerosol properties\nfor understanding aerosol direct and indirect effects on climate, both from the pre-industrial period\nuntil now and for future climate projections. Rather than using a traditional physics-based forward\nmodel to perform the retrieval of aerosol properties, the manuscript proposes to use a feedforward,\nfully connected neural network (NN), a novel application in this specific domain. Specifically, the\nNN uses plane-based lidar observations (supplemented with reanalysis products) to predict CCN\nconcentration and aerosol absorption (ABS). The manuscript argues this machine learning model\ncan be applied to future satellite lidar observations with appropriate adjustments to input data\n(restricting wavelengths of lidar data used as model inputs and adding random noise).\nThis work demonstrates the major potential of machine learning and lidar observations to\nrevolutionize retrievals of vertically resolved aerosol properties. However, the results currently\npresented seem to be a stepping stone toward revolutionary aerosol retrievals rather than a fait\naccompli. The manuscript at present does not include enough information about data\nmeasurement and processing and associated variability/uncertainty to access whether this\nrepresents an significant additional source of uncertainty presently unaccounted for (major issue\n1). The machine learning models show substantial biases in their predictions of aerosol properties\nunder clean conditions, which are important for estimating climate impacts (major issue 2). Finally,\nthe manuscript does not convincingly demonstrate that the models trained on plane lidar data are\ntransferable to satellite lidar data with only restrictions in inputs and additional input noise (major\nissue 3). The combination of the issues makes it difficult to evaluate the manuscript’s claim that\nthey demonstrate machine learning can substantially improve observationally inferred aerosol\nproperties relative to present physics-based retrievals.\nMajor issue 1: Data processing and its associated uncertainties are not adequately described.\nThe manuscript does not appear to include information on the instruments and processing\nperformed to produce the predicted aerosol properties of CCN and ABS. The co-location procedure\nused to combine in-situ aerosol measurements and lidar profiles is also very sparsely described.\nWhy were the distance and time values used chosen? How sensitive are the distributions of CCN\nand ABS data to these choices? (Independent of the ML predictions, the distribution of in-situ\npoints in Figure 2 vs Figure S2 look non-trivially different.) How are multiple observations satisfying\nco-location requirements combined? What kind variability is present among multiple\nobservations? The text should at least mention and ideally estimate the magnitude of\nuncertainty/variability introduced by the co-location and data processing algorithms as well as\ntypical instrument errors and counting uncertainties.\nMajor issue 2: The models presented show strong biases as a function of aerosol absorption and\nconcentration.\nFigures 2 and 4 show clear biases in overestimating both CCN and ABS at low values. These biases\nare not mentioned in the manuscript text and seem to have limited effect on the metrics used to\nquantify the performance of the NN models because most data points appear to be clustered at\nintermediate values of CCN and ABS. (This is difficult to say quantitatively because density of\npoints in Figure 2 and 4 are not given and Figure 1 only includes distributions of occurrence\nfrequency of CCN and ABS in each campaign, not the full dataset.) This bias is particularly\nconcerning because better constraints on aerosol properties in clean conditions are needed to\nreduce uncertainty in aerosol-cloud indirect effects important for climate (Gryspeerdt et al., 2023,\nhttps://doi.org/10.5194/acp-23-4115-2023).\nMajor issue 3: The manuscript does not convincingly demonstrate the transferability of the\npresented models based on aircraft observations to satellite observations.\nSensitivity test 2 (Figure S3) suggests (unsurprisingly) that the NN models have no capacity to\nextrapolate. The manuscript does not address the extrapolations likely to be encountered when\napplying four limited-area field campaigns to global data. While sensitivity test 3 (Figure S4)\ndemonstrates that the NN models can interpolate between values under conditions seen in the\nfield campaigns, it does not show the transferability of the NN models to different meteorological or\naerosol conditions outside what was seen in the field campaigns. Additionally, while manuscript\nconsiders restricting lidar input data and random noise to address differences between airborne\nand satellite measurements, the manuscript does not address differences in airborne and satellite\nlidar instrument systematics as well as their observing profiles, resolutions, and coverages.\nminor comments:\nFigure 1: An additional row of histograms with data from all campaigns would be helpful. The\nreanalysis products (RH, T) used in some NNs should also be included in these histograms.\nFigures 2 and 4: Missing colorbars to quantify densities. I find the scatter point-based density\ndifficult to interpret. What range of values does each point encompass?\nFigure 6: Missing description of white points (appears to be some retrieval of cloud top). Missing\ndescription of how blacked out portions determined. I would recommend using a perceptually\nuniform colormap over jet.\nText does not explain selection of T and RH as the reanalysis quantities chosen to include to\nimprove predictions. References should be included to justify why these properties are physically\npredictive. Ideally some analysis would be included to show that T and RH are not being using by\nthe NN to distinguish different campaigns, leading to spurious relationships over a broader range of\nmeasurements.\nWhile the NN hyper parameter ranges searched are presented in the Table S2, I don’t see the hyper\nparameters used for the presented results anywhere. These should be included somewhere to\ninform (among other things) the number of trainable parameters vs the number of training points.\nCode repository does not appear to include data processing and NN training, validation, and hyper\nparameter tuning.\nReviewer #4 (Remarks on code availability):\nThe code to generate figures is present in the listed repository, but the repository does not appear to\ninclude co-location data processing and NN training, validation, and hyper parameter tuning."}, "author_response": "GENERAL RESPONSE TO REVIEWER COMMENTS - NCOMMS-24-11301A-Z\nWe would like to thank the four reviewers who took the time to provide such detailed, thoughtful, and\nconstructive comments to our manuscript. The consistency between reviewer comments speaks to\nthe care taken in providing this feedback and the need for revisions to our paper.\nWe note that the majority of comments refer to omissions of study details. Given that this is our first\nsubmission to the Nature family of journals, we admit that we have apparently overemphasized\nbrevity. Of course, this error occurred in our attempt to observe the word limits on the main paper\nand to not overwhelm the reader with too much technical detail. Thankfully, we had already\nconducted many of the studies necessary to address the reviewers' comments. In most cases, we\nhave decided to include the additional information in the methodology section of the paper or the\nsupplements, as advised by the handling editor, Dr. Sebastian Müller.\nThe most substantive change to our analysis is the retraining of the ML models to predict CCN\nconcentrations. As correctly pointed out by two reviewers, the previous model showed poor\nperformance for relatively low CCN concentrations (below 300/cm3), yet in those conditions clouds\nare highly susceptible to aerosol contamination. In response, we retrained the models with a training\ndata set that used triple weighting to the data points in clean conditions (below 100/cm3), which\nresulted in improved predictive capability in clean conditions without causing a significant loss of\npredictive capabilities for the overall conditions. In addition, we have added results that showed the\ngood performance of our ML models in moderately clean conditions (100/cm3<CCN<300/cm3). The\nmean relative errors for the clean conditions are still relatively large, but they provide a useful\nconstraint beyond any currently available remote sensing observations.\nAnother significant change in our study is the replacement of Merra-2 reanalysis data of T and RH\nwith ERA5 reanalysis data. We compared both reanalysis data sets against the airborne in situ data\nand found slightly better agreement in RH between ERA5 and the in situ data, likely due to the higher\nspatial resolution of the ERA5 data by comparison to Merra-2. The use of the ERA5 data as predictors\ninstead of Merra-2 resulted in a modest improvement in predictive performance in most ML models\n(1-2% lower mean relative uncertainties).\nAnother important change in our manuscript is the clarification of the purpose of our paper, i.e., to\npresent a paradigm that uses ML algorithms to estimate higher-level aerosol properties using only\nlidar observables and reanalysis data as predictors. These ML models are based on collocated\naircraft data for training and testing. The paradigm can be applied to future satellite (or suborbital)\nobservations after training the models to use the available sets of lidar observables as predictors.\nThis purpose is different from presenting a final set of ML algorithms to predict ABS and CCN. As a\nmatter of fact, we fully expect that future ML models that use lidar observations as input can be\nfurther optimized with evolving ML techniques and additional training data.\nWe investigated several potential issues that were raised by the reviewers regarding the robustness\nof our ML predictions with respect to the selection of a specific reanalysis data set, the collocation\ncriteria and potential spatial auto-correlation of our training data, details of the lidar and in situ\nmeasurements that constitute our training data set, and the transferability of our ML models that are\ntrained with four limited-area field campaigns to global data.\nBelow, we have enumerated our responses, to make it easier to refer to responses raised by other\nreviewers. Our responses appear in italic font and text that was revised in the main manuscript\nappears in blue font. Page and line numbers refer to the revised manuscript when viewed in track-\nchanges mode and “All Markup” visible.\nWe hope that we have addressed the reviewers' concerns to their satisfaction and that the paper is\nnow acceptable for publication in Nature Communications. Allow us to express our gratitude to the\nreviewers one more time - we believe that addressing their comments significantly improved the\nclarity of our manuscript and the merit of our study."}]

Decision letter