T-W AVE N ET : A T REE -S TRUCTURED W AVELET N EURAL N ETWORK FOR T IME S ERIES S IGNAL A NALYSIS
A BSTRACT
Time series signal analysis plays an essential role in many applications, e.g., activity recognition and healthcare monitoring.
Recently, features extracted with deep neural networks (DNNs) have shown to be more effective than conventional hand-crafted ones.
However, most existing solutions rely solely on the network to extract information carried in the raw signal, regardless of its inherent physical and statistical properties, leading to sub-optimal performance particularly under a limited amount of training data.
In this work, we propose a novel tree-structured wavelet neural network for time series signal analysis, namely T-WaveNet , by taking advantage of an inherent property of various types of signals, known as the dominant frequency range .
Speciﬁcally, with T-WaveNet , we ﬁrst conduct frequency spectrum energy analysis of the signals to get a set of dominant frequency subbands.
Then, we construct a tree-structured network that iteratively decomposes the input signal into various frequency subbands with similar energies.
Each node on the tree is built with an invertible neural network (INN) based wavelet transform unit.
Such a disentangled representation learning method facilitates a more effective extraction of the discriminative features, as demonstrated by our extensive experiments on various real-life time series classiﬁcation datasets.
I NTRODUCTION
Time-varying signal analysis plays a crucial role in various applications.
For example, smartwatches utilize inertial signals for human activity logging (Khan et al., 2019); brain-computer interfaces employ electroencephalography (EEG) signals to identify user intentions (Zhang et al., 2020; Autthasan et al., 2021); clinical diagnosis systems use surface electromyography signals (sEMG) for neuromuscular pathological analysis (Duan et al., 2015), and develop muscle-computer interfaces to control external devices (Pancholi et al., 2021).
The signals mentioned above are all typical time series data, i.e., a set of observations collected and ordered chronologically, and expressing information at speciﬁc frequency ranges.
Generally speaking, time series signal analysis consists of three steps: (i) data segmentation , wherein the continuous signals are partitioned into segments using ﬁxed- or variable-sized windows; (ii) feature extraction , wherein various techniques are applied on each segment to extract distinguishing features; and (iii) downstream tasks , which generate the desired outputs for certain tasks (e.g., classiﬁcation and anomaly detection) with the extracted features.
Among the three steps, feature extraction is usually the most critical one.
Traditional feature extraction approaches for time series signals can be broadly categorized as statistical and structural methods (Lara et al., 2012).
The former utilizes statistical measurements in time- or frequencydomain to ﬁgure out discriminative features, where exemplar time-domain measurements are mean and variance (Kao et al., 2009; Vepakomma et al., 2015), and typical frequency-domain measurements include short-time Fourier transform (STFT) and discrete Wavelet transform (DWT)
(Jiang & Yin, 2015; Duan et al., 2015)).
On the other hand, structural methods aim at describing the morphological interrelationship among the data via polynomial and/or exponential analysis (Olszewski, 2001).
While the above hand-crafted features are efﬁcient for some simple signal analysis tasks, their limited representation capability makes them incapable of dealing with complicated signals contaminated by noise or artifacts, and hence less competitive in real-world tasks (Chen et al., 2021).
Recently, deep neural networks (DNNs) have become the mainstream approaches for feature extraction in time series signal analysis.
Various DNN models, including CNN-based solutions (Xi et al., 2018; Lawhern et al., 2018; Lin et al., 2020; Amin et al., 2019), CNN-LSTM combined models (Wang et al., 2020;
Xu et al., 2019; Yuki et al., 2018), and Transformer-based techniques (Song et al., 2021; Li et al., 2021), are proposed in the literature, achieving promising performance in many tasks.
Despite of the remarkable success of these deep models, most of them place heavy demands on a large amount of labeled data.
However, in many real-world applications, it is difﬁcult and/or expensive to acquire sufﬁcient labeled data to train these models.
In addition, noise and artifacts that commonly exist in the time series signals make it even harder to obtain discriminative and robust representations.
To the end, it may not fully reveal the superiority of deep models when solely relying on the network to directly extract information from raw signals.
To solve this problem, some works attempt to use prior knowledge or hand-crafted features to guide the training of the deep learning models, aiming at extracting more effective features with a limited amount of training data (Ito et al., 2018; Wei et al., 2019; Laput & Harrison, 2019).
However, these methods either simply leverage prior knowledge as gates to select features or intuitively add some statistical measurements as regularization terms to the losses, which fail to bring signiﬁcant improvement.
Moreover, designing tailored feature sets for different kinds of signals is tedious and time-consuming, which severely limits the generalization capability of such methods.
In this regard, a promising way to acquire more discriminative features is to explore approaches to deeply integrating the inherent properties of the signals into the training process of the deep learning models.
An inherent property termed as the dominant frequency range has been evidenced in many time series signals, which is a small subset of the frequency components that carries the primary information of the signal (Telgarsky, 2013).
For instance, more than 95 % of human body motion energy exists in the frequency components below 15 Hz (Karantonis et al., 2006); the informative frequency ranges in brain signals (EEG) have been discovered and named as δ (0.1-3 Hz) , θ (4-7 Hz), α (8–13Hz), β (14-30 Hz) and γ (31-100Hz) (CR & MP, 2011).
However, most existing deep learning models are unaware of this essential property, and do not fully consider the different roles of various frequency components, leading to sub-optimal solutions.
Motivated by the above, we propose a novel tree-structured wavelet neural network, namely T-WaveNet , to extract more effective features from time-series signals by seamlessly and effectively integrating this property into a deep model.
Different from previous models, the proposed T-WaveNet adaptively represents the dominant energy range of the input signal with more discriminative features, which can be naturally and easily generalized and applied to different kinds of time series signals.
The main contributions of this work are threefold:
• We perform frequency spectrum energy analysis for signal decomposition in T-WaveNet , wherein the frequency ranges with more energies are divided into ﬁner-grained subbands and are thus represented with more dimensions compared with other low-energy frequency ranges in the feature vector, which facilitates effective learning of informative and discriminative features with a limited number of potentially noisy training samples.
• To extract the features effectively, we propose a novel invertible neural network (INN) based wavelet transform as the tree node in the T-WaveNet .
Compared to ﬁxed-basis wavelet Haar or lifting scheme-based wavelet, INN-based wavelet provides better representation capacity owing to its entirely data-driven characteristic.
To the best of our knowledge, this is the ﬁrst attempt to model wavelet transform using INN.
• Finally, inspired by the self-attention mechanism in Transformer, we propose an instrumental feature fusion module which considers the feature dependencies across different frequency components, thus effectively enhances the robustness of the model by mitigating the impact of the heterogeneity exists in sensor signals recorded from different subjects.
Extensive experiments on four popular sensor signal datasets, namely UCI-HAR for activity recognition, OPPORTUNITY for gesture recognition, BCICIV2a for intention recognition, and NinaPro
DB1 for muscular movement recognition, show that our T-WaveNet consistently outperforms state-ofthe-art solutions.
R ELATED W ORK
In this section, we review related work on feature extractions for time series analysis and wavelet transform modeling using deep learning techniques.
2. F EATURE E XTRACTION FOR S IGNAL A NALYSIS
Existing feature extraction methods for time-varying signal analysis can be broadly classiﬁed into hand-crafted and deep learning-based methods, and the former can be further divided into statistical and structural approaches.
Hand-crafted statistical and structural approaches are widely used in early studies.
For instance,
Kao et al.
(2009) applies statistical features such as the mean and the mean absolute deviation (MAD) for online activity detection from a portable device.
Duan et al.
(2015) performs discrete wavelet transform on surface-Electromyography (sEMG) signal for hand motion classiﬁcation.
While relatively easy to calculate, these hand-crafted features are not effective for complicated tasks.
In recent years, DNNs have become the mainstream approaches for signal feature extraction.
CNNbased models are widely used to extract local temporal correlations of time series data.
For example, Lee et al.
(2017) combines multiple CNN layers with different kernel sizes to obtain the temporal dependencies at various time scales.
Lawhern et al.
(2018), Lin et al.
(2020), and Amin et al.
(2019) utilize CNNs in BCI applications to establish end-to-end EEG decoding models and achieve promising performance.
As CNN-based models are often insufﬁcient for extracting long-term temporal features,
Ord ´ o ˜ nez & Roggen (2016), Yuki et al.
(2018), and Chambers & Yoder (2020) propose to combine
CNNs with LSTMs to extract both short- and long-term temporal features.
Xu et al.
(2019) bring out the Inception CNN structures to extract local temporal features at various time scales and utilize gated recurrent units (GRUs) to obtain global temporal representations.
Recently, Transformer-based methods utilize the self-attention mechanism to model the global temporal dependencies and shows superior performance in various tasks.
For example, Song et al.
(2021) construct a simple yet effective
Transformer-based model to obtain discriminative representations for EEG signal classiﬁcation.
Li et al.
(2021) design a two-stream convolution-augmented transformer to extract features for human activity recognition, which considers both time-over-channel and channel-over-time dependencies.
The above deep learning based solutions try to extract features directly from the raw signal, which ignore the unique characteristics of each type of signals (e.g., the frequency spectrum information) and usually become less effective when training on scarce signal data.
Notably, some works try to alleviate this issue by guiding the DNNs with the traditional time-frequency features such that more effective deep representations can be learned.
For instance, Ito et al.
(2018) feed the spectrogram images constructed from the temporal features of the inertial signals into CNN models to learn inter-modality features.
Laput & Harrison (2019) train a CNN on the time-frequency-spectral features of the input sensor signal to obtain a ﬁne-grained hand activity sensing system.
Furthermore, S et al.
(2019) integrates the Short-Time Fourier Transform into the DNNs to directly learn frequency-domain features.
However, the above solutions often need to design tailored features for different kinds of signals, which is tedious and time-consuming, and severely limits the application scenarios.
2. W AVELET T RANSFORM M ODELING
Frequency-domain feature extraction methods such as Fourier or Wavelet transform are more preferable to time-domain ones, since it is usually easier to extract discriminative features in the frequency domain than directly from the raw input signal.
Compared with Fourier transform, which decomposes a signal into ﬁxed frequency components, Wavelet transform is shown to have excellent properties for transient signal analysis, thanks to its capability to analyze signals at different frequencies with various time resolutions.
In this section, we focus on Wavelet transform.
Recently, integrating wavelet transform with deep learning techniques is shown to be effective in signal and image processing.
Early attempts directly replace certain layers in the DNNs with traditional wavelet transform to reduce the number of parameters as well as improve the interpretability.
For instance, Williams & Li (2018) replace the max-pooling layer with a wavelet pooling algorithm to address the overﬁtting problem.
Fujieda et al.
(2017) propose to replace the pooling and convolution layers with wavelet transform.
Feature fusion module
Sub-band features
Multi-ResolutionDecomposition
[0~12] 푓 ￿
[12~1] 푓 ￿ [0~14] 푓 ￿ [14~12] 푓 ￿ [12~34] 푓 ￿ [34~1] 푓 ￿
[0~ 116] 푓 ￿ [ 116~18] 푓 ￿ [18~ 316] 푓 ￿ [ 316~14] 푓 ￿
[0~ 132] 푓 ￿ [ 132~ 116] 푓 ￿
c
휙 d
X 휓 휂 휌 X 1 S p li t + -
Linear q 푘 ￿ v × ×
Softmax
Attention score ℎ ￿ ℎ ￿ h
H even odd ℎ ￿ ℎ ￿
Linear
Linear
Input ： X (b) T-WaveNet (c) Frequency Bisection Operator (d) Feature Fusion Module [0~1] 푓 ￿ [0~18] 푓 ￿ [18~14] 푓 ￿ [14~38] 푓 ￿ [38~12] 푓 ￿ (a) Frequency Spectrum Energy Analysis
Frequency (Hz)
[Figure]
lit
[Figure]
e
[Figure]
input, we utilize a (d) Feature Fusion Module to fuse the subband features { h i } according to the feature dependencies across various frequency components.
Finally, the enhanced feature vectors are supervised by
Cross-Entropy loss for classiﬁcation.
See Section 3 for more details.
The above methods typically utilize a ﬁxed wavelet basis (such as Haar (Haar, 1910)), which tend to be sub-optimal as it is less ﬂexible and expressive for complex data.
Recently, Rodriguez et al.
(2020) propose to replace the ﬁxed wavelet basis in the lifting scheme with deep learnable modules to realize an adaptive wavelet transform unit for image classiﬁcation, aiming at learning more discriminative frequency features from images.
In this work, to better handle the complex spatial-temporal time series data, we build a novel deep wavelet transform unit with powerful representation capability, named frequency bisection operator .
It decomposes the input signal into frequency subbands of various sizes and models the wavelet basis with an INN (Dinh et al., 2014).
To the best of our knowledge, this is the ﬁrst attempt to model the wavelet transform using INN.
We detail the proposed solution in Section 3.2.
M ETHODS
Instead of solely relying on the neural network for information extraction from raw signals, T-WaveNet takes into consideration the uneven information distribution among different frequency ranges of signals and constructs a tree-structured network for feature learning, where those frequency ranges with more energy are divided into ﬁner-grained subbands and represented with more dimensions in the feature vector, leading to more powerful representations embedding more desired information extracted from time series signals.
An overview of the T-WaveNet is shown in Fig.
We ﬁrst perform frequency spectrum energy analysis (Fig. 1 (a)) to obtain the set of signal frequency subbands with various formants and energy envelops.
Then, we construct T-WaveNet , a tree-structured network (Fig. 1 (b)), which iteratively decomposes the input signal into frequency subbands of various sizes according to frequency spectrum energy analysis .
Each node on the tree is a frequency bisection operator built with an INN-based wavelet transform unit (Fig.
1 (c)).
The operator is conditioned on a binary gate.
It outputs the high-frequency and low-frequency components of the input if the gate value is “1”, and otherwise bypass the input.
In addition, different subjects may have some unique characteristics, resulting in distribution divergence.
To deal with such individualized heterogeneity, after obtaining the set of features from each frequency subband, we utilize an effective feature fusion module to enhance the task-oriented subband features (Fig. 1 (d)).
3. F REQUENCY S PECTRUM E NERGY A NALYSIS
To quantitatively evaluate the information within the frequency range and perform the subband splitting accordingly for constructing the T-WaveNet , we perform frequency spectrum energy analysis ( FSEA ) to obtain a set of frequency subbands with roughly equivalent energy.
In this way, the informative frequency range would be divided into ﬁner subbands and be represented with more feature dimensions, easing the representation learning of signal data.
FSEA consists of two phases: a formants-guidance frequency band splitting phase to obtain the initial subbands set, and an energy-guidance advanced subband splitting to further balance the energy of each subband.
In the ﬁrst phase, given the input signal X ∈ R N , we ﬁrst calculate the spectrum with the Fourier transform on the entire signal (Fig. 1 (a)) so as to obtain the overall frequency distribution of the signal.
Next, we obtain the set of formants P = { f p } p , where f p is a local maximum ( i.e. , formant) of the envelope of spectrum 1 .
Such formants represent the most direct source of the signal information.
Then, we recursively bisect the frequency band until there is at most one formant that falls in each frequency subband [ f i , f j ] , where f i and f j are the starting and ending frequency, respectively.
All these subbands are collected in set Q .
In the second phase, we calculate the energy of each subband in Q with Eq.
(1) (Boashash, 2003) and further bisect the subband whose energy exceeds ζ times of E min , where E min is the the minimum subband energy for subbands in Q , and ζ is a scaling factor.
This subband splitting phase further ensures that each resulted subband in the new set Q (cid:48) contains similar amount of energy.
Here, Amp ( f ) is the amplitude of the frequency f .
[Equation]
The FSEA , in principle, can be regarded as a perquisite for T-WaveNet construction.
Based on Q (cid:48) obtained from FSEA , we construct the proposed T-WaveNet in two steps: (i) bottom-up marking, and (ii) pruning .
In the bottom-up marking process, given a full binary tree with some height (Fig. 1 (b)), we locate each subband in Q (cid:48) and set its binary gate as “0”, which serves as the leaves.
Next, we set the binary gates of all other nodes on the path from the leaves to the root as ”1”.
In the pruning process, the nodes without gate settings are removed from the tree.
The leaves of resulted sub-tree cover all the frequency subbands in Q (cid:48) .
With the above, FSEA divides the frequency ranges with more energy into ﬁner-grained subbands.
3. F REQUENCY B ISECTION O PERATOR
Each node in the T-WaveNet is a frequency bisection operator built with an INN-based secondgeneration wavelet transform.
The second-generation wavelets, also known as Lifting Scheme theory (Sweldens, 1998), is a simple yet powerful approach to construct different wavelets such as Haar .
The main idea is to utilize the strong correlation among the neighboring samples in the signal to separate the low-frequency (approximation) and high-frequency (details) subbands.
The
Lifting Scheme separates the input vector X =
( x [0] , x [1] , ..., x [2 k − 1]) , k ∈ N , into low- and high-frequency subbands in three steps.
• Splitting .
The signal is split into two non-overlapping partitions as shown in Eq.(2).
The general partition method is dividing the signal to even part X even =
( x [0] , x [2] , ..., x [2 k − 2]) and odd part X odd = ( x [1] , x [3] , ..., x [2 k − 1]) .
The spliting operater is: • Predictor .
The two partition sets are distributed alternatively in the original signal.
Based on the signal correlation, it is possible to build a good predictor P to predict one partition set from the other.
One example to predict X odd from X even is shown in Eq.
(3) , where d (details) denotes the difference between X odd and the prediction set P ( X even ) .
• Updator .
As shown in Eq.
(4) , the details d is further used to update the even part with an updator U to preserve some consistent characteristics of the original signal, such as mean and higher moments.
Here, c is also called “approximation”.
[Equation]
In traditional wavelet transform, the choice of the wavelet basis is important yet difﬁcult, which would affect the analysis results signiﬁcantly.
Traditional wavelet construction framework with ﬁxed coefﬁcients ( P, U ) usually lacks of adaptability to extract more informative features from complex signals.
Therefore, we take advantage of the INN (Dinh et al., 2014), a bijective transformation, to build the wavelet transform unit, which can effectively model the correlations between the inputs and the outputs with learnable structures.
To combine the Lifting Scheme with INN, the improvements include: (i) adapting Eq.
(3) , (4) to an afﬁne function, as shown in Eq.
(5) , (6) , which improves the transformation ability; (ii) learning the wavelet coefﬁcients ( φ , ψ , ρ and η ) in Eq.
(5) , (6) using separate CNN modules.
[Equation]
Here, exp( φ ( · )) , exp( ψ ( · )) stand for scale and ρ , η stand for translation.
(cid:12) is the element-wise production.
The exp( · ) is to introduce non-linearity into the function and is omitted in Fig.
1 (c).
Note that, if the values of φ and ψ are set to zero, Eq.
(5) , (6) will degenerate to the standard Lifting
Scheme equations (Eq.
The proposed INN-based wavelet transform has two advantages.
First, being an invertible architecture,
INN avoids information loss during feature extraction and, hence, inherits the conventional wavelet transform property.
Second, the wavelet basis is entirely data-driven, and learning high-dimensional coefﬁcients can be regarded as implementing various wavelet basis to the input simultaneously according to the requirements of the downstream task, which is more effective than conventional ﬁxed-basis wavelets.
3. F EATURE F USION M ODULE
There inevitably exists personalized heterogeneity in the signal data collected on different subjects.
Thus, the features extracted from a certain frequency subband may contribute differently to different subjects.
To alleviate the effect of personalized heterogeneity and achieve better task performance, we build an efﬁcient feature fusion module inspired by the well-known Transformer (Vaswani et al., 2017).
The feature fusion module is built on the self-attention mechanism, which learns the personalized feature dependencies across different frequency subbands, and adaptively enhances the task-related subband features with higher weights during fusion, as illustrated in Fig.
1 (d).
To calculate the weights of the features h ∈ R n × m extracted from the frequency subband set Q (cid:48) , we ﬁrst calculate the query q , keys k , and values v from subband features h :
[Equation]
where W ∗ ∈ R r × n and b ∗ ∈ R r are the weight matrix and bias, respectively, m is the number of features, n is the feature dimension of h , and r is the feature dimension of q , k and v .
Then, we compute the attention weights for each value as the compatibility score of the query with Eq.(8).
Then the output of the feature fusion module is calculated as the weighted sum of the values in Eq.(9).
[Equation]
The ﬂattened attentive frequency subband representation H ﬂatten ∈ R ( mr ) is ﬁnally fed into a standard softmax classiﬁer to produce the prediction results:
[Equation]
where W ∈ R C × ( mr ) and b ∈ R C are the weight matrix and bias, respectively, p i is the i -th element of p , which denotes the predicted probability for class i = 1 , . . .
, C , and C is the number of class.
3. L OSS F UNCTION
Our loss function is shown in Eq.
The ﬁrst term is the cross-entropy loss for classiﬁcation, where C denotes the number of classes, and y i is the binary ground-truth.
Besides this term, we also explore incorporating an additional term to regularize the wavelet decomposition during training.
The second loss term ensures that, for each frequency bisection operator, the mean value of the decomposition output
c j is close to that of the input x j .
Here, M is the total number of operators in the tree-structured network, and λ tunes the strength of the regularization.
[Equation]
[Table]
E XPERIMENTS
In this section, we evaluate the performance of T-WaveNet on four datasets, namely OPPORTUNITY (OPPOR)
(Ord´o˜nez & Roggen, 2016),
UCI-HAR (Davide et al., 2013), BCICIV2a (Brunner et al., 2008), and NinaPro DB1 (Manfredo et al., 2012).
A brief description of these datasets is listed in
Table 1.
More details on experimental settings, additional experimental results and discussions (e.g., performance on other datasets, hyperparameter analysis, robustness, and computational time) are presented in the supplementary materials.
4. C OMPARISON WITH S TATE - OF - THE -A RT M ETHODS
We compare T-WaveNet with state-of-the-art methods on various signal classiﬁcation tasks and datasets, and the experimental results are shown in Tables 2-5.
As can be observed, the performances of previous state-of-the-art methods vary with different types of signals, while the proposed T-WaveNet consistently achieves better performance across all these datasets.
We attribute it to the fact that the proposed method seamlessly and effectively integrates the inherent dominant energy range property of signals into our deep model.
In the following, we present detailed analysis of comparison on benchmarks of each task.
[Table] [Table]
Human activity recognition (UCI-HAR and OPPOR)
As can be observed from Table
[Table]
T-WaveNet outperforms both the recent end-to-end deep models (Harmonic (Hu et al., 2020) ,
LSTM-CNN (Xia et al., 2020), MI-CNN-GRU (Dua et al., 2021)) and a hybrid model (STFNet (S et al., 2019)) on the UCI-HAR dataset.
To be speciﬁc, compared with
MI-CNN-GRU , T-WaveNet yields 2 .
18% and 1 .
35% relative improvements in terms of Accuracy and F w score, respectively.
This is because, MI-CNN-GRU solely relies on combining the CNN and
GRU model for feature extraction, without considering the essential information in the frequency domain.
In contrast, STFNet integrates the STFT (Short-Time Fourier Transform) into the neural network to learn features directly in the frequency domain.
However, it does not distinguish the roles of various frequency components.
T-WaveNet decomposes the most informative frequency range into ﬁner-grained subbands, which increases the representation capacity for feature learning under limited training samples.
Similarly, for the OPPOR dataset in Table 3, compared with the popular
CNN-LSTM structure used in FilterNet (Chambers & Yoder, 2020) , T-WaveNet clearly demonstrates better feature learning capability with a 2 .
66% improvement in F m score.
As for, and 2 . 66% improvement of F m for OPPOR.
Such improvements clearly demonstrate the feature learning ability of the proposed method.
Moreover, T-WaveNet can better capture the inter-class differences than existing solutions, because it learns the feature dependencies across different frequency subbands.
Brain intention recognition (BCICIV2a)
It is challenging to learn effective features for brain intention recognition since EEG signals are usually quite noisy.
As shown in Table 4, T-WaveNet achieves superior performance on BCICIV2a compared with standard CNN classiﬁers (e.g., CTCNN (Schirrmeister et al., 2017) and EEGNet (Lawhern et al., 2018)).
The EEG-Image (Bashivan et al., 2018) model tries to apply all sorts of handcrafted features (i.e., spectral, spatial, and temporal features) into the CNN-RNN architecture, but it actually mislead the neural network in feature learning, resulting in poor performance.
The graph neural network-based classiﬁer NG-CRAM (Zhang et al., 2020) is effective in modeling the spatial correlations among signals, but it also suffers from noises during message passing among neighbouring nodes.
Finally, MIN2Net (Autthasan et al., 2021) utilizes auto-encoder to extract features from the input signals, but it may lose some essential information during this procedure.
Also, the implementation of the Conv2D may restrict the model’s ability in learning long-term temporal dependencies.
In contrast, T-WaveNet concentrates on the features extracted from the dominant energy range with more discriminative power, and the proposed INN-based wavelet can avoid the information loss during feature extraction, thereby yielding more robust representations.
[Table] [Table]
Muscular movement recognition (NinaPro DB1)
As shown in Table 5, T-WaveNet achieves a relative 6.65 % improvement over WeiNet (Wei et al., 2019) that trains a CNN classiﬁer on hand-crafted features, and 1.52 % improvement over XceptionTime (Rahimian et al., 2020) that solely relies on time domain features.
DLPR (Pancholi et al., 2021) combines both the time- (peaks and zero-crossing) and frequency- (power spectral) domain features with CNNs and achieves remarkable performance for this task.
However, this solution requires lots of domain knowledge and much efforts to design signal-speciﬁc features, which is challenging to be generalized to other signal types.
In contrast, the proposed INN-based wavelet transform in T-WaveNet is adaptive for learning information from both time- and frequency-domain, which facilitates to extract more discriminative features for muscular movement recognition.
[Figure]
4. A BLATION S TUDY
Here, we study the impact of each component in T-WaveNet by performing a detailed ablation study on the challenging OPPOR dataset and the corresponding frequency spectrum is shown in Fig.1 (a).
Impact of frequency spectrum energy analysis .
To verify the effectiveness of FSEA , we train two other variants of the T-WaveNet with different feature vector distributions (Fig. 2).
Fig.
2 (a) shows that the tree structure constructed with the dominant energy range concept outperforms other variants, indicating that dominant frequency bands represented with more dimensions in feature vectors indeed contain more discriminative information for classiﬁcation.
The impact of the two-phases subband splitting scheme is shown in the supplementary.
Impact of frequency bisection operator .
To verify the effectiveness of our INN-based wavelet transform in the frequency bisection operator, we also experiment with two variants: (i) T-WaveNetHaar , which replaces the INN-based wavelet transform with the traditional Haar wavelet basis; (ii)
T-WaveNet-LS , which implements the deep version of the Lifting Scheme, in which the P and U in Eq.
(3)(4) are realized using the same deep modules as φ , ψ , ρ and η (Fig.1 (c)).
The results in
Table 6 show that our T-WaveNet achieves 18 .
6% and 3 .
8% improvements on F m score in OPPOR dataset over T-WaveNet-Haar and T-WaveNet-LS , respectively.
We attribute the improvements to the fact that our high-dimensional deep wavelet coefﬁcients are learned from data.
Thus, our INN-based wavelet is more adaptable to various signals compared with the ﬁxed Haar wavelets, and has higher representation capacity than the deep Lifting Scheme.
See the supplementary for more details.
[Table]
Feature fusion module .
To demonstrate the effectiveness of the proposed adaptive feature fusion module for handling the personalized heterogeneity of the signal data, we remove the fusion module from T-WaveNet and fuse the leaf features { h i } with equivalent weights ( TWaveNet-noFusion ).
As shown in Table 6, the performance of the resulted model decreases by 2% - 3% , which indicates that the proposed feature fusion module learns the feature dependencies across various frequency components and enhances task-related features for classiﬁcation.
More details are presented in the supplementary.
4. D ISSCUSSION AND L IMITATION
Our approach is currently more suitable for the time-series data with some speciﬁc patterns in their frequency spectrum across subjects/cases.
Biological signals often have such characteristics and in this regard, we extensively evaluate the proposed network on biological signals.
However, if other time-series signals have similar properties, our network can also been applied to analyze them and achieves performance gains (see the supplementary for more experimental results).
On the contrary, if the time-series data does not have this characteristic, especially when the frequency energy spectrum is time-variant (e.g., some ﬁnancial data), the proposed network cannot achieve remarkable performance improvements.
C ONCLUSION
We propose T-WaveNet , a novel tree-structured wavelet neural network for time series signal analysis, which exploits an inherent property of various types of signals, i.e., the dominant frequency range , to more efﬁciently extract informative representations from input signals than most existing solutions that depend only on the raw data.
We ﬁrst conduct frequency spectrum energy analysis of the signals to get a set of dominant frequency subbands.
We then construct a tree-structured network with
INN-based wavelet transform units to iteratively decomposes the input signal into various frequency subbands with similar energies.
Finally, all the leaf node features are adaptively fused to mitigate the impact of the individualized heterogeneity of the signals.
Experimental results on four datasets clearly show the superiority of T-WaveNet over other state-of-the-art solutions for time series classiﬁcation task.
This work, in general, offers insights on how to leverage both data and empirical knowledge to drive a learning model towards more powerful representations for time series signals, which is of great signiﬁcance for many practical applications with a limited amount of training data.