Dynamic Sparse Network for Time Series Classification: Learning What to “See”
Anonymous Author(s)
AffiliationAddress email
Abstract
The receptive field (RF), which determines what hidden signals can be “seen”in a time series model, is critical to improve the performance for time seriesclassification (TSC).
However, the variation of signal scales across and within timeseries data, makes it challenging to decide proper RF sizes for TSC.
In this paper,we propose a dynamic sparse network (DSN) with sparse connections for TSC,which can learn to cover various RF without cumbersome hyper-parameters tuning.
The kernels in each sparse layer are sparse and can be explored under the constraintregions by dynamic sparse training, which makes it possible to reduce the resourcecost.
The experiment results show that the proposed DSN model can achieve thestate-of-art performance on both univariate and multivariate TSC datasets with lessthan 50% computational cost compared with recently baseline methods, openingthe path towards more accurate resource-aware methods for time series analyses.
Our code is provided in the supplementary material, and it will be made availableonline.
Introduction
Time series classification (TSC) as an important research topic in the data mining communities [19],has a wide range of applications from health care [20, 12] and social security [51], to grid energy[34] and remote sensing [36].
In the last decade, neural networks, especially deep neural networks,have achieved competitive or even better performance than traditional TSC approaches (e g. DTW[39], BOSS [40], COTE [1]) in many cases [49, 19].
However, how to discover and exploit the various scaled signals hidden in time series (TS) isstill a significant challenge for TSC tasks [45, 31, 17].
One main reason lies in several variances,like sampling rate and record length, which are natural during time-series collection [40, 27, 4].
Furthermore, the amplitude offset, warping and occlusion of data points are unavoidable (see Figure 1).
So determining the optimal scales for feature extraction is difficult but significant in the TSC task.
One main solution is to cover as many receptive field (RF) scales as possible, so it does not ignoreany useful signals from the time series inputs [45, 5, 6, 21].
Grounding on the motivation that multiple kernels in combination can capture discriminative patternsdespite complex warping, Fawaz et al.
[11] aimed to ensemble convolutional neural networks witha large number and variety of kernels.
Tang et al.
[46] have stacked several OS-CNN blocks, eachof which consisted of a list of kernel sizes to cover all scales of RF and achieve better performanceacross several TSC tasks.
In Multi-scale Convolutional Neural Network (MCNN) [3], a grid searchwas applied to find suitable kernel sizes.
Without elaborate setting and fine-grid search, Dempster etal.
[5] transformed time series by using random convolutional kernels and then trained a time seriesclassifier.
Even though the use of kernels with various sizes can help to extract latent hierarchicalfeatures of multiple resolutions, it tends to cause a huge computational problem and overfits the datasets when a limited amount of training data is available [21].
To address this problem, dilatedconvolution [52] has been used to reduce model size while keeping the receptive field size.
Franceschiet al.
[13] presented an unsupervised learning model with convolutional kernels for time series featuretransformation, in which the dilation factors of kernels increased exponentially layer by layer.
To date,there is still an unresolved challenge with hyperparameter selections (e.g. kernel sizes and dilatedfactors) to capture patterns at different scales.
This raises a foundational question: Can we designa scalable method for TSC that achieves a trade-off between the computation and performancewithout cumbersome hyperparameter tuning?
[Figure]
Inspired by sparse neural network models which can match the performance of dense counterpartswith fewer connections [33, 30, 28, 53], we propose a dynamic sparse network (DSN) that is composedof CNN layers with large but dynamic sparse kernel for TSC tasks, which can automatically learnsparse connections in terms of covering various RF.
The proposed DSN is trained by a dynamic sparsetraining (DST) strategy, which makes it possible to reduce the computational cost (e.g. floating-pointoperations (FLOPs)) for training and inference as well.
Different from the traditional dynamic sparsetraining methods, which explore the connections in a layer-wise manner [33, 29, 10] and can hardlycover smaller RF, we propose a more fine-grained sparse training strategy for TSC.
Specifically, the
CNN kernels in each layer are split into several groups, each of which is sparse and can be exploredunder a constraint region during the course of sparse training.
In this way, each layer can extractfeatures in a more scalable and diverse way, and then by cascading, the model can cover more variousscales of RF.
To sum up, our main contributions are:
• We propose a novel dynamic sparse CNN layer with various effective RF, which can learn
to “see” selectively and adaptively for more accurate TSC.
• By stacking the above dynamic sparse CNN layers, we introduce a novel DSN model
natively for TSC, which is trained by DST to reduce computational and memory cost.
• To bridge gap between resource awareness and accuracy, we propose a more fine-grainedtraining strategy for DSN, which can more easily cover diverse effective RF for TSC.•
We perform extensive empirical evaluation on both univariate and multivariate TSC tasks,showing that DSN achieves superior performance in terms of accuracy and resource cost.
Related Work2.1 Time Series Classification
In the last decade, the success of deep learning encourages researchers to explore and extend itsapplication on TSC tasks [24, 19, 49, 18, 25].
For univariate TSC, deep learning-based modelsattempt to directly transform raw time series data into low-dimensional feature representation viaconvolutional or recurrent networks [21, 38, 35].
For multivariate TSC, LSTM and attention layersare often stacked with CNN layers to extract complementary features from TS data [22, 54].
Recently, many works attempt to extract features in a more scalable manner as TS data is naturallycomposed of multiple signals on different scales [40, 4, 45].
One main solution is to configure suitable kernels of various sizes such that they increase the probability to capture the proper scales[45, 46, 21].
Rocket-based methods [5, 6, 7] aim to use random kernels with several sizes and dilationfactors to cover diverse RF for univariate TSC.
Different from these works, our proposed dynamicsparse CNN layer can adaptively learn the various RF with deformable dilation factors and trade-offboth computation and performance without cumbersome hyperparameter tuning.
2.2 Sparse Training
Recently, the lottery ticket hypothesis has shown the possibility of training a sparse sub-network tomatch the performance of the dense network with less computational cost [14].
Rather than iterativelypruning from a dense network [15, 55, 14], recent works try to find an initial mask with one-shotpruning based on gradient information during training [26, 48].
After pruning, the topology of theneural network will be fixed during training.
However, this kind of models can hardly match theaccuracy achieved by their dense counterparts [32, 48, 44].
Introduced as a new training paradigm before the lottery ticket hypothesis, DST starts from a sparseneural network and allows the sparse connectivity to evolve dynamically with a fixed number ofparameters throughout the training [33, 10, 30, 53].
Nowadays, DST attracts increasing attentionsfrom other research fields such as reinforcement learning [42] and continual learning [41], due to itspotential of outperforming dense neural networks training [33, 53, 30].
Different from the traditional
DST methods, our proposed DSN is trained with a fine-grained sparse training strategy rather than atraditional layer-wise manner to capture more diverse RF for TSC.
2.3 Adaptive Receptive Field
The RF, which can be adaptively changed during the training, has proven to be effective in manydomains [47, 37, 50].
The adaptive RF can be usually captured by learning the optimal kernel sizes orthe masks of kernels during the training stage [43, 16, 50].
However, neither the kernels nor the masksare sparse, which may cause the huge computational problem when larger RF needed.
Different fromthese methods, our dynamic sparse CNN layers in DSN can be trained with DST, which can learn tocapture variable RF with deformable dilation factors.
What’s more, the kernels are always sparseduring training and inference, which makes it possible to reduce the computational cost.
The Proposed Model
3.1 Problem Definition
Definition 1.
( Time Series Classification (TSC) ).
Given a TS instance X
= { X 1 , . .
X n } ∈ R n × m ,where m denotes the number of variates and n denotes the number of time steps, TSC aims toaccurately predict the class label y ∈ { 1 , . .
. , c } from c classes.
When m equals 1, TSC is univariateand otherwise it is multivariate.
Definition 2.
( Time Series (TS) Training Set ).
A training set D
= { (cid:0) X (1) , y (1) (cid:1) , . . .
,(cid:0) X ( N ) , y ( N ) (cid:1) } consists of N time series instances,
where X ( i ) ∈ R n × m could either be a univariate or multivariate time series instance with its corresponding label y
( i ) ∈ { 1 , . .
, c } .
Note that, in our case, all instances have the same number of time steps in a TS dataset.
Without theloss of generality, given a training set, we aim to train a CNN classifier with adaptive and various RFwith a low resource cost (e.g. memory and computation) for the TSC task.
3.2 Dynamic Sparse CNN Layer with Adaptive Receptive Field
A straightforward strategy to cover various RF is to apply multi-sized kernels in each CNN layer, butit has several limitations.
Firstly, TS instances from different TS datasets do not have the same lengthand cycles 1 in most cases, making it difficult to set a fixed kernel configuration for all the datasetseven with prior knowledge.
Secondly, obtaining a large receptive field commonly requires a largekernel or stacking more layers, which introduces more parameters and thus increases storage andcomputational costs.
To tackle those challenges, the proposed dynamic sparse CNN layer possesses large but sparse kernels,which are learnable to capture adaptive RF.
Specifically, given the input feature map x l ∈ R c l − 1 × h × win the l th layer ( h equals 1 for univariate TSC and c l − 1 is the number of input channels) , andkernel weights Θ l ∈ R c l − 1 × c l × 1 × k ( k is the kernel size and c l is the number of output channels), theconvolution with stride 1 and padding in our proposed dynamic sparse CNN layer is formulated as
[Equation]
where O j ∈ R h × w denotes the output feature representation at the j -th output channel, Z denotesthe set of integers, I l ( · ) : R c l − 1 × c l × 1 × k
→ { 0 , 1 } c l − 1 × c l × 1 × k is an indicator function,
I l
(Θ l ) i,jindicates activated weights for Θ li,j ∈ R 1 × k
which is the kernel in the ( i, j ) -th channel, ⊙ denotesthe element-wise product, and · denotes the convolution operator.
The indicator function I l ( · ) , whichis learned during the training of the proposed DSN, satisfies that ∥
I l ( · ) ∥ 0 ≤ (1 − S ) N l where0 ≤ S < 1 is the sparsity ratio, ∥ .
∥ 0 denotes the L 0 norm and N l = c l − 1 ∗ c l ∗ 1 ∗ k .
When S > 0 ,the kernel is sparse, and we can use a large k in the dynamic sparse CNN layer to obtain a large RFwith reduced computational cost.
Remark 1.
( Effective Receptive Field (eRF) size in the dynamic sparse CNN layer ).
In thestandard CNN layer, the receptive field is defined as the input region that is used for generatingevery pixel feature.
Thus, the RF size is equivalent to the kernel size.
Differently, the eRF size of theproposed dynamic sparse CNN layer is smaller than the kernel size, when the first or last weight in akernel is not activated, e.g., ∃ i ∈ { 1 , . . .
, c l − 1 } and j ∈ { 1 , . .
c l } , such that I l (Θ l ) i,j, 1 , 1 = 0 or
I l
(Θ l ) i,j, 1 ,k = 0 .
As illustrated in Figure 2, we defined the eRF size f li,j of the kernel Θ li,j as thedistance between the first activated weight and last activated weight in the l -th CNN layer as
[Equation]
otherwise
[Equation]
) where Ind li,j denotes the set of indices corresponding to non-zero weights in the kernel Θ li,j .
Obviouslywe have 0 ≤ f li,j ≤ k .......
= =
= = =
= =
= 3 =
The eRF size set of l -th dynamic sparse CNN layeris denoted by F ( l ) , which satisfies that 0 ≤ min( F ( l ) )and max( F ( l ) ) ≤ k .
Take the case in the Figure 2as an simple example for l -th layer, then F ( l ) ={ 0 , 1 , 2 , 3 , 4 , 5 , 6 } .
Each dynamic sparse CNN layeris likely to cover various eRF sizes ranging from 1 tok .
When global information is expected, I l ( · ) couldactivate weights more dispersed to get a larger eRFand exploit the input features selectively.
To capturethe local context, a smaller eRF is expected, thus theactivated weights tend to be concentrated.
By takingk = 5 as an example, I l (Θ l ) i,j may be [1, 0, 1, 0, 1] for global context, while it may be [0, 0, 1, 1, 0]for local context.
In this way, the eRF can be adaptively adjusted.
3.3 Architecture in DSN
[Figure]
R eL U
R eL U
A dap t i v e A v g P oo l 1d
R eL U
C [Figure] v 1 x
B N
A dap t i v e A v g P oo l 1d
S
[Figure]
r s e C
N N
[Figure] e
[Figure] The proposed DSN model consists of three sparse CNN modules, each of which composes of adynamic sparse CNN layer and a 1 × 1 CNN layer.
Following the stacked sparse CNN modules, there is an additional dynamic sparse CNN layer, two adaptive average pooling layers, and one 1 × 1convolution layer acting as the classifier in the DSN model. The overall architecture is shown in
Figure 3.
The set of eRF sizes S ( l ) in the l th sparse CNN module is equal to that in the dynamic sparse CNNlayer, since the eRF size of 1 × 1 convolution is constantly equal to 1 .
S ( l ) satisfies that 0 ≤ min( S ( l ) )and max( S ( l ) ) ≤ k .
Then, the set of eRF sizes for the three successively stacked sparse CNNmodules can be described by RF as
[Equation]
[Figure]
According to Eq.
(3), we can see that stacking multiple sparse CNN modules can increase the size of linearly, where each dynamic sparse CNN layer increases by the size of S ( l ) .
For simplicity, thekernel size in each dynamic sparse CNN layer is consistently set to k in our study.
Then, RF satisfiesthat max( RF ) ≤ 3 k − 2 and 0 ≤ min( RF ) .
Thus, a large k in sparse CNN modules can increase therange of the eRF sizes for stacked sparse CNN modules.
3.4 Dynamic Sparse Training for DSN Model
Input: Dataset D , Network f Θ , Sparsity ratio S Exploration regions in layer l : {R l 1 , ..., R lN }
Exploration schedule: T , ∆ T , α , f decay 1: θ li ← initialize activated weights in Θ li using S and R li 2: for t = 1 to T do 3:
Sample a batch B t ∼ D L t = (cid:80) i ∼ B t L (( f θ ( x i ) , y i ) 4: if ( t mod ∆ T ) =
=
then 5: for each layer l = 1 to L do 6: for each group i = 1 to N do 7: u = n ( R li )
f decay ( t ; α, T ) (1 − S ) 8:
I prune = ArgTopK (cid:0) − (cid:12)(cid:12)
θ li (cid:12)(cid:12) , u (cid:1) 9:
I grow =
RandomK (cid:0) R li \ θ li , u (cid:1) 10:
I l ( . )
← Update I l ( . )
using I prune
and I grow 11: θ li ← Update activated weights
I l (Θ li ) Θ li 12: end for 13: end for 14: else θ = θ − α ∇ θ L t 15: end if 16: end for
In this section, we present the trainingstrategy to discovery the weights requiredto be activated in order to ensure wellperforming eRF.
That is, we need to studyhow to update the indicator function
I l ( · )during the training of the proposed DSN.
We sparsely train the proposed DSN modelfrom scratch to keep kernel sparse following the main idea of DST methods [33].
Bydesign, during the training phase, the total number of activated weights must notexceed N l (1 − S ) .
However, we observethat the tiny eRF is hard to capture by DSTmethods with a layer-wise exploration manner, namely activated weights are discovered layer by layer, especially when thesparsity ratio S is undersized (more analysis in Section 4.5).
Motivated by thisobservation, the kernels in each dynamicsparse CNN layer are divided into differentgroups, whose corresponding explorationregions are of different sizes as shown in
Figure 4.
Contrary to the DST methods, in the DSN the exploration of activated weights is separatelyperformed in different kernel groups, which is a more fine-grained strategy.
Specifically, the kernelweights Θ l ∈ R c l − 1 × c l × 1 × k in the l th layer are split into N groups along the output channel, that is,Θ l 1 , . . .
, Θ lN ∈ R c
l − 1 × clN × 1 × k with corresponding exploration regions R l 1 , . . .
, R lN .
The exploration region R li in the i th group of the l th layer is defined as the first i × kN positions of each kernel in thisgroup.
By taking k = 6 and N = 3 for example, activated weights in the first group are only withinthe first two positions of each kernel in Θ l 1 , while they are within the whole kernel for the last group(as shown in Figure 4).
In this way, the various eRF can be covered, and exploration space can bereduced (more details can be found in Appendix A.1) to improve exploration efficiency.
Given the weight exploration regions R l 1 , . . .
, R lN and the sparsity ratio S , we train
the proposed
DSN model as shown in Algorithm 1.
The activated weights are explored within the explorationregions and updated every ∆ t iterations.
The fraction of updated weight decays over time accordingthe function f decay ( t ; α, T ) , which follows the cosine annealing [8] as
[Equation]
where α is the initial fraction of activated weights updated, t is the current training iteration, and Tis the number of training iterations.
Thus, during the t th iteration, the number of updated activatedweights in the i th group of the l th layer is n ( R li ) f decay ( t ; α, T ) (1 − S ) , where n ( R li ) is the numberof weights can be explored in region R li .
During the update of activated weights, we firstly prunethe activated weights determined by ArgTopK (cid:0) − (cid:12)(cid:12) θ li (cid:12)(cid:12) , u (cid:1) and then randomly grow new activatedweights by RandomK (cid:0) R li \ θ li , u (cid:1) , where ArgTopK( v , u ) gives the indices of top u elements withlarger values in a vector v , RandomK( v , u ) outputs the indices of random u elements in a vector v ,and R li \ θ li denotes the weights within R li but except θ li .
The exploration for activated weights is straightforward.
Firstly, pruning weights with small magnitudes is intuitive, because the contribution of weights with smaller magnitudes is insignificant oreven negligible.
Considering the pruning recoverability, we randomly regrow new weights with thesame number as the pruned weights to achieve better activated weight exploration.
In this way, theexploration of weights is dynamic and of plasticity.
Experiments
4.1 Datasets and Baselines
We evaluate the proposed method on both univariate and multivariate TSC datasets.
The details ofeach kind of dataset are as follows.
• Univariate TS dataset from UCR 85 archive [2] : This archive consists of 85 univariate TSdatasets, which are collected from various domains (e.g. health care and remote sensing),have distinguishable characteristics, and various levels of complexity.
The number ofinstances in the training set varies from 16 to 8926, while their time step resolutions rangefrom 24 to 2709.• Three multivariate TS datasets from UCI [9] : The EEG2 dataset contains 1200 instancesof 2 categories and 64 variates.
The Human Activity Recognition (HAR) dataset consists of10,299 instances with 6 categories, and the number of variates is 9.
The Daily Sport datasetcontains 9,120 instances of 19 categories, and its number of variates is 45.
The processingof these dataset is the same as in [22].
Univariate TSC baselines: The performance of the following three baseline methods are reported,
OS-CNN [45], InceptionTime [21], and ResNet[49], as they are widely used for univariate TSC.
Multivariate TSC baselines: Similar with [54], the following three multivariate baselines areselected: OS-CNN, TapNet [54], MLSTM-FCN [22].
To avoid an unfair comparison, we let outsidethe ensemble methods.
However, based on our best knowledge, our performance analysis accountsfor most of state-of-the-art methods.
4.2 Experimental Settings and Implementation Details
For optimization, we use the Adam optimizer with an initial learning rate of 3 × 10 − 4 , cosine decayedto 10 − 4 .
Our model is trained for 1,000 epochs with a mini-batch size of 16.
As in [21, 22], the bestmodel, which corresponds to the minimum training loss, is used to evaluate performance over thetesting sets.
Inspired by [21], the kernel sizes k in each dynamic sparse CNN layer are set to 39.
The number of kernel groups N in Algorithm 1 is set to 3, which helps to cover the small, medium andlarge eRF.
Each setting is repeated for five times and the average results are reported.
4.3 Results and Analysis
We show the performance on both univariate and multivariate TSC benchmarks in Table 1-2.
Forunivariate TSC, we can see that our method outperforms the baseline methods in most cases with asmaller number of parameters (e.g. Params 2 ) and less computation cost (e.g. FLOPs) on UCR 85archive datasets.
For multivariate TSC, our proposed DSN method achieves better performance on
EEG2 and HAR datasets, while for Daily Sport, MLSTM-FCN was with 0.46% more accurate.
Itis worth highlighting that the resource cost (i.e. Params and FLOPs) of our proposed DSN is muchsmaller than that of baseline methods.
Due to the page limitation, the detailed results including testaccuracy and resource cost on each dataset of UCR 85 archive can be found in Appendix C.
[Table] [Table]
4.4 Sensitivity Analysis
Effect of Sparsity Ratio: From Table 3, we can find out how the sparsity ratio of dynamic sparse
CNN layer affects the final test accuracy and the resource cost in multivariate TS datasets.
Here,we analyze the trade-off between test accuracy and the resource cost under various sparsity ratios(i.e.
S ∈
[50% , 80% , 90%] ) and a dense DSN model.
Note that the dense model is defined exactlyas our DSN model with dense connections along entire kernels.
We can see that it is not that themore parameters of the model, the better performance it will achieve.
This is mainly because moreparameters will lead to overfitting and the proper sparse ratio will easily cover the desired receptivefield (more demonstrations can be seen in Appendix B.1).
Taking EEG2 data set as an example,with the decrease of sparse rate, the performance on it drops sharply.
We hypothesise that this dataset has more expectations for features from small receptive fields (the validation can be found in
Appendix B.2).
According to the experiments, a 80% sparsity ratio reflects a good trade-off betweenaccuracy and resource cost, which is the default setting for the dynamic sparse CNN layer in our
DSN model.
Effect of Architecture: To show how the architecture of DSN affects the final performance, weadopt the critical difference diagram for a detailed comparison, which is usually used for evalution in
TSC [45, 21].
From Figure 5, we can find that the DSN model with 4 dynamic sparse CNN layersand 177 output channels in each layer performs consistently better than the other structures.
Theproposed model with more output channels seems to be crucial for preserving the capacity of thenetwork, while requires more parameters and FLOPs compared to other counterparts. This highlights [Table]
an interesting trade-off between accuracy and computational efficiency.
According to the experiments,the DSN model with 4 dynamic sparse CNN layers and 141 output channels in each layer is thetrade-off between the performance and resource cost, which is the default setting for our DSN model.
[Figure]
4.5 Ablation Study
[Table]
Effect of Kernel Group: As shown in
Figure 6, training DSN without kernelgroup hampers the capture of small RF, which can not guarantee satisfactory performance on the datasets suchas EEG2 expecting more local information.
With the decrease in sparsityratio S , the phenomenon that large RFoccupies the majority becomes moresevere (more demonstrations see in Appendix B.1), which increases theperformance gap between training with group and without it as shown in Table 4.
Grouping thekernels can achieve various RF coverage and adorable accuracy for all datasets.
Effect of Dynamic Sparse Training:
Furthermore, we study the effect of the dynamic sparse trainingin our DSN method.
We conduct ablation studies to compare DSN with its static variant (i.e. DSN fix ),which fixes its topology during training after activated weights initialization.
In other words, theindicator function
I l ( · ) in Eq.
(1) will not update during the training process.
According to the results shown in Figure 7 (a), we can find that the DSN model mostly outperforms the DSN fix on UCR 85archive, indicating that what are learned by dynamic sparse training is the suitable activated weightstogether with their values.is better here is better here is better here is better here (a) (b) is better here is better here (c)
Case Study on other DST
[Figure]
Likewise, the effectiveness of the proposed DSN method is also analyzedin the presence of other DST methods,such as RigL [10].
Rather than growing weights randomly (as in the line9 of Algorithm 1), RigL grows themwith the highest magnitude gradients.
From Figure 7 (c), we can see thatgrowing weights according to gradients (i.e. DSN grad ) can outperformsthe OS-CNN method in most cases on
UCR 85 archive.
Furthermore, as shown in Figure 7 (b), we can find that DSN grad has a similarperformance as growing weights randomly (i.e. DSN rand ).
Note that, DSN rand is the main settingof the proposed method in this paper with the reason that we don’t need to compute the full gradientsat any moment in time.
In summary, the proposed DSN with fine-grained sparse strategy can beeasily combined with the existing DST techniques, showing the potential for further improvementsby merging it with other advanced DST methods in the future.
4.7 Visualization of Activated Weights
[Figure] Figure 8 shows the normalized activated weights of the DSN model after sparse training on HARdataset.
(a), (b) and (c) correspond to the activated weights of the three kernel groups in the firstdynamic sparse CNN layer, and each row of the figure represents one kernel.
We can see thatthe weights in each kernel are sparse, and can be activated within the constraint region defined in
Section 3.4.
Conclusions
This paper proposes dynamic sparse network (DSN) to cover diverse effective receptive field (eRF)for time series classification (TSC) without the cumbersome hyper-parameters tuning.
The proposed
DSN can achieve about 2 × reduction in memory and computational cost, while achieving superiorperformance in terms of accuracy on both univariate and multivariate TSC datasets.
Furthermore,
DSN explores the activated weights in a more fine-grained strategy, which can be easily mergedwith existing DST techniques, showing a potential for further improvements when combined withadvanced technology.
DSN provides a feasible solution to bridge gap between resource awareness andvarious eRF coverage for TSC, hoping to inspire other researches in different fields.
It is worthwhileextending DSN to other research domains (e.g. time series forecasting and computer vision).
Our
DSN model has not yet been executed on real edge devices, and the performance in terms of accuracyand memory footprint on edge devices is not investigated.
In the future, we hope to extend our workto other fields and decentralise training on edge devices.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix D
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes](d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using curating contains personally identifiable
information or offensive content?
N /A]
If you used crowdsourcing or conducted research with human subjects.
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]