Spatio-Temporal Weathering Predictions in the Sparse Data Regime with Gaussian Processes
Anonymous Author(s)
AffiliationAddress email
Abstract
We investigate the problem of predicting the expected lifetime of a material indifferent climatic conditions from a few observations in sparsely located testing facilities.
We propose a Spatio-Temporal adaptation of Gaussian Process Regressionthat takes full advantage of high-quality satellite data by performing an interpolation directly in the space of climatological time-series.
We illustrate our approachby predicting gloss retention of industrial paint formulations.
Furthermore, ourmodel provides uncertainty that can guide decision-making and is applicable to awide range of problems.
Introduction
Weathering refers to the in-service degradation of materials from the natural environment [1].
Beingable to predict the service-life of materials under different climatic conditions is of critical importanceto guarantee long-term safety, improve cost efficiency of design decisions during the constructionphase and shorten development cycles for new products.
However, it is usually a challenging problem,since several chemical degradation processes can occur simultaneously and are triggered by differentenvironmental conditions.
While there are research avenues focussing on chemical understanding[2, 3, 4] and empirical fitting [5, 6, 7], Machine Learning (ML) methods have gained popularity inthe last decade, showing very promising effectiveness in modeling complex non-linear behaviourswithout the need to unravel the underlying physical processes [8, 9].
Within this context, in order topredict the material behaviour in untested locations, we seek a data-driven approach to model therelation between degradation over time and exposure location, i.e. a Spatio-Temporal (ST) problem.
The problem is quite unique and differs from common ST forecasting [10] as no past observation isavailable at the new location.
Furthermore, ST methods commonly rely on large datasets and performinterpolation between densely allocated spatial points.
However, in weathering, this is hardly thecase, since testing facilities tend to be few and isolated around the globe.
In such cases of very sparsesampling, the spatial correlation is lost and the use of spatial interpolation or Deep Learning based
ST methods is not possible [11, 12].
While geographical coordinates become uninformative, localclimatic conditions still carry most of the information needed to identify material degradation at mostlocations [13].
This leads to our focus on climate representation.
Climate is itself a dynamical system and measurements mainly come as time-series (TS) observations.
While previous works have relied on extracted features, such as mean values or fitted parameters[14, 15], we propose to retain the TS format to avoid any loss of information.
The choice is alsohighly motivated by the recent developments in satellite data acquisitions which make high-qualityclimatology TS data available on a global scale.
We then propose to represent each climate with a
Multivariate Time Series (MTS), containing the time evolution of any relevant climatic variable.
Topredict degradation in new locations, we perform the interpolation in this highly informative climatespace by adopting an ST adaptation of a Gaussian Process model (ST-GP).
The proposed framework benefits from great flexibility and leads us to the following main results:
• by pairing weathering data from different testing facilities with climatology satellite data,the model can predict performances in untested locations.
This severely reduces the needfor additional testing sites;• by providing uncertainty estimates, the model also gives insight on where it is optimal togather additional data;• the model allows for extensions to incorporate other sources of variability such as chemicalproperties and can then support material discovery.
We begin with a presentation of the framework illustrated in Fig.
1, followed by quantitative andqualitative experimental evaluation.
1.1 Notation
Through the article, we indicate variables as lowercase ( x ); vectors and TS as bold lowercase ( x ) ;matrices and MTS as bold uppercase X .
An index between square parenthesis x [ n ] | Nn =1 indicate then-th sample of a dataset and a subscript x t | Tt =1 the time stamp t .
Methodology
Consider a dataset of N loc entities, where each ( i ) is a TS of length T i which samples the degradationof the same quality variable y in a different location: y [ i ] ∈ R T i | N loc i =1 .
In our framework, everydegradation data entry y t [ i ] is associated with a time stamp ( t ) , marking the exposure duration, andwith an MTS of spatial descriptors ( C [ i ] ), representing the climate at the exposure site:
[Equation]
What remains is to model the dependence between the spatio-temporal inputs ( t, C ) and the responsey .
We accomplish that by training a Gaussian Process (GP) [16], mainly motivated by the followingconsiderations:
• GPs have remarkable modeling performances in the little data regime;
• GPs have been used in ST regression problems [17, 18] and geostatistics ( kriging )
• GPs are flexible with any kind of input, as long as a kernel function is provided for the dataat hand.
Furthermore, the closure of kernels under multiplication allows us to decouple thetemporal and spatial dependence.
Following a common practice, we adopt a zero mean prior and do not introduce any prior knowledgeinto the model.
Regarding the covariance function, we employ a decoupled spatio-temporal kernelcomposed of an RBF over time and the Global Alignment Kernel (GAK)
[20] over the MTS spatialinputs:
[Equation]
where the hyperparameters are the variances θ scale and θ noise , the time-bandwidth σ time and the
GAK multiplicative factor parameter.
The GAK was originally proposed for univariate series and canbe extended to the multivariate case by averaging GAK on individual dimensions (independent GAK)or by considering alignments of vector-valued entries (dependent GAK)
As our main claimswill not be significantly affected by this choice, we present the results by adopting the independent
GAK and show some comparisons with the alternative in Appendix B.
After having trained the ST-GP model, it is finally possible to predict the degradation y ( t, ¯ C ) for anyt , by providing ¯ C for a new location.
Application: weathering of paint formulations
In this section, we apply the developed framework to service-life predictions of paint formulations inuntested locations.
A toy-experiment with more abundant and publicly available data is presented in
Appendix A.
3.1 Datasets
Climate data The data were obtained from the NASA Langley Research Center (LaRC) POWER
Project’s Climatology on 2022/01/18.
The database contains data about surface solar energy fluxesand other meteorological quantities obtained through satellite systems and further reanalysis.
Foreach point on a dense global-scale grid, climate data are provided in TS format with customizabletime resolution.
Further information, together with data and API, is available from the project website[22].
We choose the climatology TS format, consisting of 12 time-stamps with monthly valuesaveraged over up to 30 years.
We select the following 10 variables: mean temperature ( C ), relativehumidity ( % ), wind speed ( m/s ), dew/frost point ( C ), maximum temperature ( C ), cloud amount ( % ),temperature range ( C ), precipitation ( mm/ day), all-sky surface shortwave and longwave irradiance( W/m 2 ).
Each MTS then has the form C ∈ R 12 × 10 .
Paints degradation data The degradation data have been privately provided by the Beckers group.
They consist of TS data about the changes over time in a quality parameter, i.e. Gloss Retention (GR),for 37 different polyester-based paints systems (13 out-of-market qualities in 3 colour pigments:
White, Red-Brown and Dark-Blue).
Every GR value is obtained as the ratio between measuredand initial gloss at t =
(details about gloss measures are available in [23]).
For each one of the37 formulations, the data come in two forms: TS obtained from long natural exposures (up to 139months) in 10 different sites on a global scale and TS from 3 different controlled accelerated tests.
The number of available time-stamps in each TS depends on the exposure location, ranging from3 to 19, not necessarily equally spaced.
A table with the availability in each location is shown in
Appendix B.
3.2 Training on the same formulation
Consider one fixed formulation and the task of predicting its degradation curve in a new climate fromexposure data of the same formulation in other locations.
As the chemical variability is removed, wejust need to take into account the climatic variability.
For each fixed formulation, our data consistof N loc = 10 degradation TS ( y
[ i ] ∈ R T i ), each one paired with a climate
To internally test the interpolation performances, we withheld one degradation TS from one locationand try to recover it by training the ST-GP model on the remaining N loc − 1 .
To report the accuracy,we use the Mean Average Error (MAE) and the Mean Standardized Log Loss (MSLL), a modeluncertainty aware score that computes a standardized negative probability of the target under themodel; therefore is approximately zero for simple methods and negative for better methods.
Both are computed at each withheld location, averaged across different fixed formulations andreported on the left-side in Tab. 1.
Results are shown using the independent version of the GAK,with the bandwidth multiplicative factor set to σ mul = 2 .
More details about the training and moreinsights on the variability with these choices are given in Appendix B.
Some prediction examples areshown in Fig.2.
[Table]
training data is expanded with similar formulations in the same climate. Given the complexity of the problem, it is a success to find most average errors around ∆ GR ∼ 0 .
1and negative MSLL.
Of particular interest is the interpolation accuracy in the arid climate of South Africa, climatologically far from all the other locations.
On the contrary, we would expect smallerrors at the cities of Singapore and Kuala Lumpur, as one should be very informative in predictingthe other.
We observe instead a high and positive MSLL.
In this case, the gloss data in the city of Kuala Lumpur are mainly influenced by the heavy accumulation of dust and dirt (as opposed to
Singapore), for which appropriate descriptors have not been included.
These results indicate thatclimatic agents do not always constitute the only source of degradation and different effects might beimportant too.
These can easily be incorporated into the framework by expanding C and reserved fora future study.
As a parallel experiment, we redirect predictions to new locations where we do not have data.
Insteadof the whole service-life of the material, we focus now on one particular time stamp, i.e. 10 years ofexposure, but extend the predictions to a dense grid in a geographical region (which is possible thanksto the high resolution of the satellite climate data).
Even if we do not have testing data, this represents a qualitative validation as we can check if the predictions reflect our intuition.
In general terms, onecan expect harsher degradation in hot and aggressive climates, while steadier performances in morerigid ones.
As it can be seen in Fig.
3, the degradation map in Italy indeed reflects many features of
134 the corresponding Köppen Geiger map [25].
This is a stunning result, considering that the trainingset is composed of only 10 sparse climates.
On top of that, only a few locations contain data for upto 10 years.
Of particular interest is the relatively harsh degradation predicted on the Alps, which,despite the rigid climate, can be expected as a consequence of the high altitude and the high level of
UV radiation.
Additional studies will be conducted to validate such considerations.
As we show in Fig.
4, it is also possible to observe the geographical dependence of the modelreliability by plotting the uncertainties of the ST-GP on a global scale.
This comes with manyadvantages, such as guiding decision making, optimal gathering of additional data and supportwarranties.
3.3 Training on more formulations
During the previous sections, we have restricted our focus to climatic variability only.
In a moregeneral case, the flexibility of composing different kernels allows for extending the variability overdifferent entities.
In our case, it is possible to extend the variability over different chemicals byconsidering the 3 accelerated test data. We use them as MTS descriptors that capture dynamicproperties of each formulation rather than the location they are exposed at.
In practice, eachformulation exposed in a specific climate now corresponds to an unique pairing of climate MTS( C clm ) and accelerated MTS ( C acc ).
The independent GAK averaged across all variables containedin C clm and C acc is finally able to distinguish between all 37 × 10 instances.
To demonstrate that chemical similarity can be captured in such a way, we consider a similar crossvalidation experiment to Sec.
3.2 but expand the training data to all other formulations exposed inall climates, including the target one.
Since similar materials degrade similarly in the same climate,the model may now have access to formulations with similar behaviour to what we try to predict.
What we are testing here is the ability of the model to give more importance to those data.
If thesimilarity is captured correctly, we expect this task to be easier, with higher overall accuracy.
Theexperimental results are reported in the last column of Tab. 1 and confirm this hypothesis.
Notethat the missing descriptors for Kuala Lumpur and Singapore are much less relevant in this scenario.
We also report that the accuracies can be further refined by considering an additional kernel termthat takes into account the variability over static chemical properties, e.g. the T g of the formulation.
However, discussing specific properties goes beyond the scope of this note and the relative resultswill then not be deepened.
Conclusions and future directions
In this work, we have introduced a Spatio-Temporal adaptation of Gaussian Process Regression topredict the in-service degradation of materials under different weathering conditions.
Our approachfinds its most optimal applicability in the gap between advanced ST-ML methods, of which theusage is discouraged because of data sparsity, and knowledge-based methods that cannot capture thecomplex dependence between climate and chemical degradation.
Both quantitative and qualitativevalidations have been presented.
Regarding future works, we consider of critical importance the inclusion of additional validationarguments, either by collecting more data or by studying the relevance of each climate variable inthe predictions.
As an additional important follow-up to this work, we will also consider includingmore domain knowledge into the framework, e.g. additional spatial descriptors (air quality, chemicalproperties, etc.) or monotonicity constraints in the GP.
As opposed to climatological monthly values,daily data collected during the exposure (also easily available through [22]) might constitute evenmore informative climatic descriptors.
To deal with the poor scalability of the GAK, a novel kernelshould be paired with this choice to process very long TS.
Acknowledgments and Disclosure of Funding
We would like to acknowledge useful discussions with James Maxted from the Beckers Group.
Thisresearch is supported by the Beckers Group: https://www.beckers-group.com .
Climate datawere obtained from the NASA Langley Research Center (LaRC) POWER Project funded through the
NASA Earth Science/Applied Science Program
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] framework for spatio-temporal predictions with sparsedata, application to weathering, uncertainty maps to guide decision making.
(b) Did you describe the limitations of your work?
[Yes] Failures have been observedwhen the dominant factors are not included in the spatial MTS descriptors.
This requiredomain knowledge in choosing the right variables.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] The degradationdata and code are proprietary.
Appendix A provides a toy-experiment employing theframework with a publicly available dataset.
Link to the climate database has beengiven in Sec. 3.1.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?
[Yes] Details for reproducibility have been included in the supplementary
material for both experiments.
(c) Did you report error bars (e.g., with respect to the random seed after running exper-
iments multiple times)?
[No] In appendix, we report different results for differentchoices of the only unoptimized hyperparameter.
(d) Did you include the total amount of computing and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] Execution time can be found in the supplementary material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] All assets arereferenced in Appendix A.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
We do not propose new assets
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] Consent of data usage have been given by the Beckers group.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Toy-experiment: climate of world capitals
We present here a test of the ST-GP in a controlled experiment with climate data only.
This serves asa way of supporting the weathering predictions in Sec. 3.2, furthermore, it provides results in moredata abundant context and all within a fully publicly available dataset.
For each of the 235 capitals in the world, we interrogate the NASA climate dataset and collect thetemporal evolution of the same 10 variables as in Sec. 3.1.
The dataset we have built then consists of235 MTS ∈ R 12 × 10 distributed across the globe.
We also standardize to zero mean and unit standarddeviation within each dimension.
To simulate our regression framework, we choose one variable asresponse
y and the remaining as spatial descriptors C .
Because of its correlation with other variables,its relatively smooth evolution and variety of shapes, we choose ’mean temperature’ as our responsevariable.
If we now remove the response from one instance, the problem of retrieving its evolutionfrom observations in other locations is equivalent to the regression problem of Sec. 3.2.
Predictionsof this kind can be iterated by hiding the response from one location at a time.
For the GAK bandwidth, we set its value to eight times the median distance of the MTS in the dataset,scaled by the square root of the median length.
We have observed that equally high multiples worksimilarly well (e.g. 5 or 10 ) and better than lower multiples.
Regarding all the other parameters,we set them to the average result of a preliminary Leave One Out Cross Validation (LOOCV) cycleby training on the 20 nearest locations in the kernel space and minimizing the negative Type-II
MLE (marginal log-likelihood).
A second cycle, with all N loc − 1 instances included in the training,concludes with an average R 2 = 0 .
962 and average MAE = 0 . 020 .
The experiment runs in ∼ 2 .
5 hwith an AMD Ryzen™ 5 3600X processor.
Some examples of predictions are shown in Fig.
With very few exceptions, the model was always able to predict the correct shape.
Less accurate in correspondence of withheld testing data for 2 different capitals.
319 predictions are always paired with higher uncertainties.
A similar but more ambitious experiment can be set in the case of predicting daily climatic data.
Forthe first 50 days of 2018, we performed a LOOCV cycle and report a lower average accuracy of
MAE = 0 .
This is expected, as the higher noise level decorrelates nearby locations in the climatespace.
One successful example, for the first 200 days in the city of London, is shown in Fig.
B Paint formulations weathering
[Figure] B.1 Data table [Figure]
Fig.
7 shows the availability of data within each location.
The letter ’G’ indicates that a GR measureis available for the corresponding location and time-stamp (top row).
B.2 Training details
For what regards the GAK bandwidth, we set its value to σ mul times the median distance of the MTSin the dataset, scaled by the square root of the median length.
Regarding all the other parameters,we set them by minimizing the negative Type-II MLE (marginal log-likelihood) in each iteration ofthe LOOCV cycle.
The only bounded parameter in the minimization is the bandwidth of the timekernel: σ time ∈ [20 , 150] .
The multiplicative factor σ mul is then our only fixed hyperparameter.
Tab.2 contains several experiments with the two different GAK variants and three values of σ mul .
For thedependent GAK, the variability is negligible, except in Bohus Malmon.
For the independent GAK,better accuracies tend to be found in the correspondence of lower multiples.
In general, the claims inthe article are not significantly affected by the choice of this parameter.
GA dep
GA ind
Withheld location σ mul =
σ mul = σ mul =
σ mul =
σ mul =σ mul =
Allunga 0.
Miami 0.
Hainan 0.
Goa 0.
Kuala Lumpur 0.
Singapore 0.
Vereeniging 0.
[Table]
Each experiment for a given configuration runs in less than 10 minutes with an AMD Ryzen™ 53600X processor.
B.3 Implementation details
This experiment, as well as the one in Appendix A, has been implemented in Python 3.9; for the
GAK we use the implementation provided by the package tslearn [26] while the GP training has beendone both within GPy [27] and GPyTorch [28], producing compatible results.