Explainable Spatio-Temporal Forecasting with Shape Functions
Anonymous Author(s)
AffiliationAddress email
Abstract
Spatio-temporal modeling and forecasting are challenging due to their complicatedspatial dependence, temporal dynamics, and scenarios.
Many statistical models,such as Spatial Auto-regression Model (SAR) and Spatial Dynamic Panel Data
Model (SDPD), are restricted by a pre-specified spatial weight matrix and thusare limited to reflect its flexibility.
Graph-based or convolution-based methodscan learn more flexible representations, but they fail to show the exact interactionsbetween locations due to the lack of explainability.
This paper proposes a spatial regression model with shape functions to address the limitations of existing methods.
Our method learns the shape functions by incorporating shape constraints, whichare able to capture spatial variability or distance-based effects over distance.
Therefore, our approach enjoys a learnable spatial weight matrix with a distance-basedexplanation.
We demonstrate our method’s efficiency and forecasting performanceon synthetic and real data.
Introduction
Spatio-temporal data is widely observed in many areas, such as transportation (33; 27), climatology(2) and environmental research(19).
The popularity of spatio-temporal data brings varieties of tasksfor researchers and one of the key tasks is forecasting.
Spatio-temporal data has some inherentcharacteristics, namely, spatial dependence and temporal dynamics, which need to be considered formodeling and forecasting.
Spatial dependence means that the observations at different locations are not independent, andobservations at closer locations often have a stronger correlation.
In the statistics community,extensive research has been conducted to model spatial dependence and various spatial models havebeen proposed.
For example, in the spatial autoregressive (SAR) models, the spatial dependence ismodeled by a product of an unknown parameter and a pre-specified spatial weight matrix (4; 1; 11; 12).
Combined with the panel data, various types of spatial panel data models have been used to analyzespatio-temporal data (35; 13; 7; 22).
One limitation of the autoregressive models is that the elementsof the spatial weight matrix are pre-specified, such as an inverse distance.
Although these prespecified spatial weight matrices are applied to capture decreased distance-based effects, they fail tocapture complex distance relations in real-world applications.
Researchers in the computer science community have developed various methods modeling spatiotemporal data using deep neural networks.
Various neural network architectures have been proposedand applied to spatio-temporal forecasting, for example, spatio-temporal LSTM (31), fully connectedgated graph architecture (20), Convolutional LSTM (23) and etc.
One advantage of these methodsis that they can incorporate unstructured data and rely on a high-performance computing platformto learn complicated representations for spatio-temporal problems.
However, a critical limitation ofthese methods is that they fail to explain how the spatial interaction works explicitly.
The lack of interpretability restricts its reliability and deep insights into the underlying spatio-temporal process.
The explanation can be obtained if we can estimate the coefficient matrix that intuitively explainsspatio-temporal interactions.
In this paper, we propose an Explainable Spatio-Temporal Forecasting (ESTF) model, which utilizesa spatial autoregressive model with shape functions to address the current limitations.
Our methodextends the vector autoregressive (VAR) model (24) by incorporating distance information into thetemporal coefficient matrix using shape functions (3).
The shape constraints are designed to beconsistent with the common fact that observations from neighbours have stronger spatial dependenceversus long-distance pairs.
It is known as Tobler’s First Law, which is "Everything is related toeverything else, but near things are more related than distant things"(26; 18).
Unlike the pre-specifiedspatial weight matrix, this coefficient matrix is learnable and is thus more flexible in capturingreal-world complex spatial relations.
Moreover, the shape functions are represented as a combinationof basis functions, and thus a smaller number of parameters needs to be estimated.
Finally, ESTF canbe easily extended to forecasting in non-stationary scenarios using a dynamic spatial weight matrix.
We conduct experiments on both simulated and real data, and the results demonstrate that our methodachieves better forecast accuracy and is computationally efficient and more explainable.
Related work
Statistical models Several works focus on temporal dynamics when considering spatio-temporalforecasting problems.
The classical time series models, such as VAR, and ARIMA models, are appliedto spatio-temporal process modeling(21; 38).
Besides, a spatial weight matrix is also introduced to the ARIMA model to capture spatial dependence (28).
The non-stationarity, particularly unit-root nonstationarity, is mainly modeled by ARIMA or Co-integration models.
In addition, spatial regressionmodels or panel data are classical models in econometrics and can also be applied to model spatiotemporal problems.
These models, for example, spatial auto-regression models, take spatial weightmatrix into consideration and estimate parameters in the framework of regression.
However, thecommon characteristics of these models need a pre-specified spatial weight matrix(35; 6).
Elementsin the matrices are generally an inverse distance of corresponding locations.
Meanwhile, thesespatial models focus on statistical inference on the scalar parameters placed before the spatial weightmatrix(25).
Although there are many choices for the spatial weight matrix, such as inverse distance,adjacency relationships, and K-nearest neighbors, there is a lack of research on estimating the spatialweight matrix.
The pre-specified spatial weight matrix restricts models’ application and fails tocapture more complicated underlying spatial dependence.
Some researchers developed a sparsespatio-temporal model that can estimate a sparse spatial weight matrix (17).
The strict sparse settingalso restricts the wide application of the spatial weight matrix.
Graph-based methods Graph-based methods are widely applied for a non-Euclidean domain.
Some types of spatio-temporal data, for example, traffic flow data or brain network data, can berepresented as graphs.
The graph structures well model the complicated spatial dependence.
Thus,the definition or pre-specified graphs structure is normally required when developing a graph-basedmodel.
Related works can be found in (30; 14).
The common typical method is GraphCNN, which isto apply a convolutional transformation to the neighbors of each node (29; 34).
The graph convolutioncan capture patterns and features in the spatial domain.
Graph-based methods have been proposedand widely applied to lots of real cases.
Traffic flow data modeling and forecasting is a popular topicin this area (30; 20).
Other topics, for example, climate sensor data (16), video (10) and etc, are alsoapplied by variant graph-based models.
RNN or LSTM combined with graphs, i.e., a sequence ofgraphs, are also considered in spatio-temporal forecasting problems (10).
CNN-based methods Unlike graph-based methods, CNN-based methods are more suitable formodeling spatio-temporal data collected in regular grid locations.
It applies filters to find relationshipsbetween neighboring inputs.
Although some works (32) applied convolution neural networks tomodel non-grid traffic data, it is more common to see CNN-based methods process grid structures,e.g., images, video rather than a general domain.
As some spatio-temporal data are collected from aregular grid in the Euclidean space (29), they thus can be viewed as a kind of special image.
The CNNstructure combined with RNN or LSTM has been developed to make forecasting for spatio-temporaldata, for example, diffusion convolutional RNN (15), Convolutional LSTM networks (23; 36)and etc.
Proposed method3.1 Problem formulation and notation
We use a n × 1 vector X t = { x 1t , x 2t , · · · , x nt } to denote observations at time t , where n isthe number of locations.
At each location i , S i =
( c xi , c yi ) is the coordinates of the location i .
The distance between location S i and S j is d ij = (cid:113) ( d xij )
+ ( d yij ) 2 , where d xij =
| c xi − c xj | andd yij = | c yi − c yj | .
Our goal is to make forecasting for spatio-temporal data: given training data set
X 1 , X 2 , · · · , X T , we would like to make forecasting for the next h , ˆ X T +1 , · · · , ˆ X T + h .
3.2 The stationary spatio-temporal model with shape functions
We first consider the stationary case.
To model the spatio-temporal stationary process, we considerthe following model
[Equation]
where W k is a spatial weight matrix for capturing the spatial dependence at lag k , and ϵ t is whitenoise.
Moreover, we assume the ( i, j ) th element of W k , w ( k ) ij , depends on the distance d ij .
That is,w ( k ) ij depends on a function f k ( d ij ) .
For spatio-temporal data, the spatial dependence, represented by w ( k ) ij , between locations decreasesas the distance between two locations increases.
In other words, there is a shape constraint forthe function f k ( d ) , such as a decreasing function.
In order to estimate the shape function, wemodel f k ( d ) as a linear combination of basis functions g i ( d ) , i = 1 , 2 , · · · , m .
More specifically,the shape function f k ( d ) is a linear combination of basis functions and coefficients with positivevalue f k ( d )
= a 21 ,k g 1 ( d ) + · · · + a 2 m,k g m ( d ) , where a 1 ,k , · · · , a m,k are parameters to be estimated.
The constraint of decrease needs parameters non-negative and thus each parameters squared.
Thespatial weight matrix can take the value of decreased shape function directly.
The element of W kis w ( k ) ij = f k ( d ij ) .
The details of the shape function and the corresponding basis functions can befound in Section 3.
The parameters in shape functions can be estimated from the neural network illustrated in Figure 1.
The neural network can be trained from the following criterion:
[Equation]
3.3 The non-stationary spatio-temporal model with time-variant shape functions
The static spatial weight matrix W k can reflect spatial dependence and thus can be applied tostationary scenarios.
Next, we consider the nonstationary case.
Therefore, we extend the stationarymodel to non-stationary cases.
The spatial weight matrices only reflect static relationships across timelags in the static model.
Unlike these settings, we change spatial weight matrices to be time-variant.
The spatial weight matrices formed by time-variant shape functions can thus capture non-stationarydynamic spatial dependence.
The non-stationary model has the form below,
[Equation]
where ϵ t is white noise, and W t , k relies on shape function f t,k ( d ) .
Similar with stationary settings, the time-variant shape functions are still represented as a linear combination of basis functions
g i ( d ) , i = 1 , 2 , · · · , m .
The coefficients are therefore time-variant.
The shape function at time t has the form below f t,k ( d ) =
a 2 1 ,t,k g 1 ( d ) + · · · + a 2 m,t,k g m ( d ) .
Unlike stationary setting, the coefficients of nonstationary setting, { a i,t,k } mi =1 , depend on the time t .
The non-stationary model can be trained from the criterion by minimizing
[Equation]
The networks for the stationary model as well as the non-stationary model are presented in the Figure1.𝑓𝑊 1 𝑋 𝑡− 𝑋 𝑡− 𝑋 𝑡−𝑝 ⋮ ෠𝑋 𝑡 [𝑔 1 ,𝑔 2 ,⋯,𝑔 𝑚 ] [𝑔 1 ,𝑔 2 ,⋯,𝑔 𝑚 ] [𝑔 1 ,𝑔 2 ,⋯,𝑔 𝑚 ] ⋰ 𝑓𝑊𝑓 𝑝 𝑊 𝑝 𝑓 𝑡, 𝑊 𝑡,1 𝑋 𝑡− 𝑋 𝑡− 𝑋 𝑡−𝑝 ⋮ ෠𝑋 𝑡 [𝑔 1 ,𝑔 2 ,⋯,𝑔 𝑚 ] [𝑔 1 ,𝑔 2 ,⋯,𝑔 𝑚 ] [𝑔 1 ,𝑔 2 ,⋯,𝑔 𝑚 ] ⋰ 𝑓 𝑡, 𝑊 𝑡, 𝑓 𝑡,𝑝
𝑊 𝑡,𝑝
3.4 The basis functions for shape functions
The shape functions are integrated into our model to obtain distance-based explanations in stationaryand non-stationary scenarios.
The motivation of the proposed shape functions is that as the distancebetween two observations increases, the effects between these two locations decreases.
These distancebased effects can be reflected in spatial weight matrix W and each element in the matrix can measurehow the corresponding locations interact.
The shape function is represented as a linear combination ofbasis functions.
The basis functions, satisfying shape constraint, rely on the corresponding definitionof basis functions.
Definition of basis functions for various shape constraints.
We list the definition of basis functionsfor increased and decreased shape (3).
The distance quantile among { d i 1 ,j 1 , d i 2 ,j 2 , , · · · , d i N ,j N } at 131 quantile level q 1 , q 2 , · · · , q m is denoted by { d (1) , d (2) , · · · , d ( m ) } , where 0 ≤ q 1 < q 2 < · · · < q m ≤1 and { q 1 , q 2 , · · · , q m } = { 1 m , 2 m , · · · , 1 } .
Here, we can set the number of m << n 2 , and thus, thenumber of parameters is significantly reduced.
For the constraint of monotone decreasing function, the basis function is defined as g i ( d ) =
1 { d < d ( i ) } .
The basis function for the shape function with the constraint of concave decrease is defined asg
i ( d ) =
( d ( i ) − d ) 1 { d ( i ) ≤ d } and convex decrease is defined as g i ( d ) =
( d ( i ) − d ) 1 { d ≤ d ( i ) } , for1 ≤ i ≤ m .
Figure 2 shows the definition of basis functions for monotone decreased and increasedshape functions, respectively.
We only present four basis functions for each shape and each of themis related to four quantile levels.
The dashed lines indicate the turning points for each basis functionand they equal one or zero at the beginning and turn to zero or one at turning points.
[Figure]
3.5 Model forecasting
The stationary model requires fixed shape functions and related spatial weight matrix are timeinvariant.
Given training data set X 1 , X 2 , · · · , X T , we can estimate spatial weight matrixˆ W 1 , ˆ W 2 , · · · , ˆ W p and make forecasting iteratively.
That is ˆ X T
{++
+}
=
(cid:80) pk
=1 ˆ W k X T + 1 − k , ˆ X T +
[Equation]
[Equation]
k . The non-stationary model incorporate time-variant spatial weight matrix ˆ W t, · .
Given the training data set X 1 , X 2 , · · · , X T , we can obtain corresponding shape functions ˆ f 1 , · , ˆ f 2 , · , · · · , ˆ f T, · , where · denotes time lag.
For lag p = 1 , we can use { ˆ f t } Tt =1 to represent time-variant shape functions for convenience.
We can make dynamic forecasts for the next h windows.
One simple forecasting method is to use ˆ W T,k to make forecast for ˆ X T
+ h , that is
[Equation]
The alternative method is to retrain the new forecast to obtain the latest shape functions as well as spatial weight matrix.
Given long-term forecast window L , we first make short-term forecast for h steps
[Equation]
where h = 1 , 2 , · · · and ˆ W T + h,k is estimated by training forecast value of ˆ X T
+ h − k .
We repeat theprocess until L steps in total have been predicted.
We summarize the whole process of our model when making spatio-temporal forecasts.
Step 1 Given the observation { X t } Tt = 1 and its coordinates, calculate all distance pairs among alllocations, denoted by { d i 1 ,j i , · · · , d i N ,j N } .
Step 2 Calculate { 1 m , 2 m , · · · , 1 } quantile levels and obtain corresponding distance quantile value{ d (1) , d (2) , · · · , d ( m ) } .Step
3 Determine the shape constraints and construct corresponding basis functions.
Specify thetime lag p .
Step 4 Train the model according to the illustration of Figure 1.
Experiment
In order to assess our model in stationary and non-stationary scenarios, we synthesize data.
Then,we apply our model to make some comparisons.
On the one hand, we need to evaluate howthe estimated shape functions look and assess their similarity and accuracy.
On the other hand,our model can make spatio-temporal forecasting after estimating for spatial weight matrix.
Thebasic idea for completing the two goals is to set up the expected shape function and compareestimated parameters with the real one.
Next, we assess the forecasting performance with baselinemodels.
Codes and data for replicating our experiments are anonymously published at https://anonymous.4open.science/r/STVAR-F16E/ .
4.1 Simulation for stationary model
Here, we synthesize 100 stationary spatio-temporal data sets.
The spatial domain consists of 30 locations and their coordinates can be found at https://anonymous.4open.science/r/
STVAR-F16E/ .
For each location, we observe 500 values.
The observation is generated from the stationary model X t = (cid:80) pk =1 W k X t − k + ϵ t , where ϵ t is randomly generated from the standard normal distribution.
The next step is to construct random spatial weight matrices for each synthesized data set.
The shape functions are set to be decreasing, and we set them as a logarithmic function: [Equation] where α is randomly generated from uniform distribution [0.05,0.06] but kept to be fixed for eachsimulated data set.
We use d + 1 to avoid zero value.
This setting can make the real shape function decrease and make it equal to zero when d = 169 .
The stationary model can iteratively generate the
X t given initial value X 0 , where X 0 is randomly generated from a uniform distribution with bounds[-0.01,0.01].
The time lags are set as p = 1 .
[Figure]
Estimation for shape functions.
In Figure 3,the estimated shape function is presented in red,while the real shape function is presented
It can be seen that the estimated shape functioncan capture the trend of the real shape function.
Training details.
The first 300 steps are used astraining data, saving the last 200 steps for evaluation.
We train all models for 100 epochs with
Adam optimizer (5) and a learning rate of 0 . 01 .
The process involves parallel training across
CPUs.
We select 100 quantile levels, and thus100 basis functions g i ( d ) were generated as theinputs for the model.
Assessment for forecasting.
We assess the forecasting performance for the stationary modelwith baseline models.
As introduced in the literature review, the baseline models are selected from the VAR model(21), the spatial panel data(SPE)model that applied pre-specified spatial weigh matrix (28), graph-based models (20; 37) andconvolution-based models (15; 23).
The error metrics are mean absolute error and root mean squarederror defined by 1 Nn (cid:80)
[Equation]
|
[Equation]
h
[Equation]
[Equation]
) it ) 2 ,respectively.
The Table 1 shows the six baseline models with the proposed model.
As totally wehave 100 synthesised data sets, X ( j ) it and ˆ X ( j ) it denote i − th variable in j − th data sets.
n = 30 isthe number of locations and N = 100 is the number of synthesised data.
We conducted one-stepforecasting for the next 200 observations.
Compared with baseline models, the proposed model performs better under the metric MAE and
RMSE.
The proposed method outperforms the closest competing method, DC-RNN, by 10%.
4.2 Experiments for non-stationary model
We conduct a simulation for the non-stationary model with time lag p = 1 and synthesize 100 data sets using a similar approach to the stationary model simulation.
The initial value X 0 and ϵ t are generated from a uniform and normal distribution respectively.
The locations of observations are the same as those in the stationary model simulation.
In order to construct W t , the time-varying shape functions are created under the decreased constraint.
The shape function at time t is constructed as [Equation] , where α t controls the level of value at each time t .
ϵ t is generated from a normal distribution.
X 0 isgenerated from a uniform distribution with bound [-0.001,0.001].
Shape functions settings and estimation.
The shape functions are set as time-variant, as they cansimulate the non-stationary process across time.
We specified α 0 at t = 0 from uniform distribution[ 1 × 10 − 4 , 2 × 10 − 4 ] and then make an interpolation from α 0 to α 500 .
The total length for everylocation is
and we set α 500 = 10 × α 0 .
For example, generally if α 0 = 0 .
, we haveα t = 0 .
0001(1 − tT )
+ 0 .
001 tT , where T = 500 .
This setting guarantee that shape functions varyfrom lower level to higher level.
The larger α t is, the more larger distance-based effects they have.
Thus, the corresponding spatial weight matrix consists of dynamic shape functions and can reflect thenon-stationary dependence among each site.
We present the estimated shape functions in Figure 4and compare them with the real ones.
[Figure]
Training details.
Similar to the stationary simulation, the train-test split is 300 − 200 over the datasize of 500 .
However, we train all models for 100 epochs with Adam optimizer (5) at a learning rateof 0 .
We train models in parallel across 10 CPUs.
Forecasting performance.
The forecasting performance is assessed by the same metrics used in theprevious simulation for the stationary case.
We made a one-step forecast by our model.
As for thebaseline models, we adjusted their published code accordingly.
The results show that the proposedmodel can still capture non-stationary processes compared with baseline models.
The proposedmethod outperforms the other competing methods.
The error metric is shown in Table 1.
[Table]
Air quality data.
We apply our model to air quality data, which records air quality in California over2021 1 .
The daily mean of PM 2.5 is recorded across 172 sites.
We obtain the first 200 steps for training and perform forecasting for the next 165 steps.
All modelsare trained for 100 epochs using Adam optimizer (5), at a learning rate of 0 .
01 and batch size of50 .
We present the estimated time-variant shape functions in supplemental file.
The value of shapefunctions decays to zero at around 5.926, which is 80% quantile in the sample of distance pairs.
Inother words, the distance-based effects decay to zero at a distance equal or larger than 5.926.
Our model has ideal performance with low time consummation compared with baseline models.
We putdetailed forecasting results of simulation and real cases in a supplemental file.
[Figure]
[Figure]
The result is shown in Table 2.
The ESTF performs best in terms of RMSE, while the DC-RNNmethod performs best in terms of MAE.
For the computational time, the ESTF method is significantlyfaster than most machine learning methods, and only takes around 1 / 10 time of DC-RNN.
In Figure 5,
MAE, RMSE, and time are presented with different numbers of m .
As m increases, the computationaltime increases while both MAE and RMSE decrease.
There is a significant increase in the forecastingperformance when m increases from 10 to 50 .
For m > 50 , the forecasting performance does notincrease much as m increases.
One key advantage of the ESTF method is that we can make an explicit distance-based explanationfor our dataset.
Figure 6 shows the distance-based effects at time t = 9 .
We only present the effectsusing a threshold to obtain a more concise visualization.
The estimated shape function ˆ f 9 rangesfrom 0 to 9.8 and we set 5 as the threshold.
The red line indicates the value of the shape functionlarger than 7, while the gray line indicates the value between 5 and 7.
Figure 6 shows how any twolocations interact and measure the distance-based effects quantitatively.
For example, air qualitymonitoring sites around the Greater Los Angeles(red circle in Figure 6) area have a strong spatialinteraction with each other, such as node 7 and node 8.
4.4 SO 2 data
Texas is the second largest manufacturing state in the USA and prediction for SO 2 is critical task forresearchers. The data 2 records daily SO 2 at 31 locations in 2021.
More detailed spatial informationcan be found in the supplemental file.
The numeric result is listed in Table 2.
[Table]
Similar conclusions can be drawn in SO 2 data as that of air quality data.
The ESTF model performsbest under the RMSE metric, while DC-RNN is best in the MAE metric.
In terms of training time, the proposed ESTF method costs around 10% of that of DC-RNN.
For the inference time, ESTF and
DC-RNN are comparable.
[Figure]
[Figure]
E
[Figure]
V a l u
[Figure]
RMSE
CPU time (s)
C P U t i m e s ( s )
36 Latitude
L o n g i t u
{+d e 2 
+}
Spatial Network Effects At Time =
The efficiency analysis and performance at different quantiles are shown in Figure 7.
Together with
Figure 5, we can see that the increasing number of basis functions does not have much improvementwhen the number of basis functions is larger than 50, while the training time increases as the numberof basis functions increases.
The spatial distribution at time t = 90 is presented in Figure 8 wherecoordinates are denoted by latitude and longitude.
Two significant clusters, representing Houstonand Dallas respectively, have the strongest distance-based effect.
It quantitatively shows how theseneighbors affect each other.
Counties around Dallas-Fort Worth metropolitan area show stronginteraction, which should be noted by environmental policy-makers.
More detailed results arepresented in the supplemental file.
Discussion
This paper applies learnable shape functions to capture distance-based effects.
It can model dynamicspatial dependence for stationary and non-stationary spatio-temporal data based on their distance.
The model does not have the limitations of classical statistical spatial models and provides a moreexplanatory model than usual deep learning methods.
Furthermore, some spatio-temporal data, suchas temperature for sea surface and air quality monitoring data, usually viewed as collected from thecontinuous field, are more suitable for the proposed models since these kinds of data follow the basicrule that variability between two locations is significantly affected by their distance.
However, somespatio-temporal data, such as traffic flow or some biology data, do not follow the rule.
As a result, thespatial dependence may rely on road structure or biological mechanisms instead of distance.
It isworth researching such data by considering graph structure when estimating spatial weight matrix.
Inaddition, we can develop spatio-temporal causal inference based on the ESTF model.
Grander causalanalysis can be done by fitting the first-order VAR model (24).
The estimation of the coefficientsmatrix of the VAR model attracts researchers’ interest as it can be treated as a causal transitionmatrix.
In the causal inference community, lots of work have been conducted on the VAR model(8; 9).
However, there is a lack of research on causal inference under the spatio-temporal process.
The quantitative distance-based effects in ESTF can be further researched and extended to develop aspatio-temporal causal model.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We describe limitations in discussion section
(c) Did you discuss any potential negative societal impacts of your work?
[No](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] We upload data
and code to github
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] Please check training details in simulation and real case section
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g.,type of GPUs, internal cluster, or cloud provider)?
[Yes] They are included in training details
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A](b)
Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A](c)
Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]