T-WaveNet: A Tree-Structured Wavelet Neural Network for Time Series Signal Analysis

1 INTRODUCTION . Time-varying signal analysis plays a crucial role in various applications . For example , smartwatches utilize inertial signals for human activity logging ( Khan et al. , 2019 ) ; brain-computer interfaces employ electroencephalography ( EEG ) signals to identify user intentions ( Zhang et al. , 2020 ; Autthasan et al. , 2021 ) ; clinical diagnosis systems use surface electromyography signals ( sEMG ) for neuromuscular pathological analysis ( Duan et al. , 2015 ) , and develop muscle-computer interfaces to control external devices ( Pancholi et al. , 2021 ) . The signals mentioned above are all typical time series data , i.e. , a set of observations collected and ordered chronologically , and expressing information at specific frequency ranges . Generally speaking , time series signal analysis consists of three steps : ( i ) data segmentation , wherein the continuous signals are partitioned into segments using fixed- or variable-sized windows ; ( ii ) feature extraction , wherein various techniques are applied on each segment to extract distinguishing features ; and ( iii ) downstream tasks , which generate the desired outputs for certain tasks ( e.g. , classification and anomaly detection ) with the extracted features . Among the three steps , feature extraction is usually the most critical one . Traditional feature extraction approaches for time series signals can be broadly categorized as statistical and structural methods ( Lara et al. , 2012 ) . The former utilizes statistical measurements in time- or frequencydomain to figure out discriminative features , where exemplar time-domain measurements are mean and variance ( Kao et al. , 2009 ; Vepakomma et al. , 2015 ) , and typical frequency-domain measurements include short-time Fourier transform ( STFT ) and discrete Wavelet transform ( DWT ) ( Jiang & Yin , 2015 ; Duan et al. , 2015 ) ) . On the other hand , structural methods aim at describing the morphological interrelationship among the data via polynomial and/or exponential analysis ( Olszewski , 2001 ) . While the above hand-crafted features are efficient for some simple signal analysis tasks , their limited representation capability makes them incapable of dealing with complicated signals contaminated by noise or artifacts , and hence less competitive in real-world tasks ( Chen et al. , 2021 ) . Recently , deep neural networks ( DNNs ) have become the mainstream approaches for feature extraction in time series signal analysis . Various DNN models , including CNN-based solutions ( Xi et al. , 2018 ; Lawhern et al. , 2018 ; Lin et al. , 2020 ; Amin et al. , 2019 ) , CNN-LSTM combined models ( Wang et al. , 2020 ; Xu et al. , 2019 ; Yuki et al. , 2018 ) , and Transformer-based techniques ( Song et al. , 2021 ; Li et al. , 2021 ) , are proposed in the literature , achieving promising performance in many tasks . Despite of the remarkable success of these deep models , most of them place heavy demands on a large amount of labeled data . However , in many real-world applications , it is difficult and/or expensive to acquire sufficient labeled data to train these models . In addition , noise and artifacts that commonly exist in the time series signals make it even harder to obtain discriminative and robust representations . To the end , it may not fully reveal the superiority of deep models when solely relying on the network to directly extract information from raw signals . To solve this problem , some works attempt to use prior knowledge or hand-crafted features to guide the training of the deep learning models , aiming at extracting more effective features with a limited amount of training data ( Ito et al. , 2018 ; Wei et al. , 2019 ; Laput & Harrison , 2019 ) . However , these methods either simply leverage prior knowledge as gates to select features or intuitively add some statistical measurements as regularization terms to the losses , which fail to bring significant improvement . Moreover , designing tailored feature sets for different kinds of signals is tedious and time-consuming , which severely limits the generalization capability of such methods . In this regard , a promising way to acquire more discriminative features is to explore approaches to deeply integrating the inherent properties of the signals into the training process of the deep learning models . An inherent property termed as the dominant frequency range has been evidenced in many time series signals , which is a small subset of the frequency components that carries the primary information of the signal ( Telgarsky , 2013 ) . For instance , more than 95 % of human body motion energy exists in the frequency components below 15 Hz ( Karantonis et al. , 2006 ) ; the informative frequency ranges in brain signals ( EEG ) have been discovered and named as δ ( 0.1-3 Hz ) , θ ( 4-7 Hz ) , α ( 8–13Hz ) , β ( 14-30 Hz ) and γ ( 31-100Hz ) ( CR & MP , 2011 ) . However , most existing deep learning models are unaware of this essential property , and do not fully consider the different roles of various frequency components , leading to sub-optimal solutions . Motivated by the above , we propose a novel tree-structured wavelet neural network , namely T-WaveNet , to extract more effective features from time-series signals by seamlessly and effectively integrating this property into a deep model . Different from previous models , the proposed T-WaveNet adaptively represents the dominant energy range of the input signal with more discriminative features , which can be naturally and easily generalized and applied to different kinds of time series signals . The main contributions of this work are threefold : • We perform frequency spectrum energy analysis for signal decomposition in T-WaveNet , wherein the frequency ranges with more energies are divided into finer-grained subbands and are thus represented with more dimensions compared with other low-energy frequency ranges in the feature vector , which facilitates effective learning of informative and discriminative features with a limited number of potentially noisy training samples . • To extract the features effectively , we propose a novel invertible neural network ( INN ) based wavelet transform as the tree node in the T-WaveNet . Compared to fixed-basis wavelet Haar or lifting scheme-based wavelet , INN-based wavelet provides better representation capacity owing to its entirely data-driven characteristic . To the best of our knowledge , this is the first attempt to model wavelet transform using INN . • Finally , inspired by the self-attention mechanism in Transformer , we propose an instrumental feature fusion module which considers the feature dependencies across different frequency components , thus effectively enhances the robustness of the model by mitigating the impact of the heterogeneity exists in sensor signals recorded from different subjects . Extensive experiments on four popular sensor signal datasets , namely UCI-HAR for activity recognition , OPPORTUNITY for gesture recognition , BCICIV2a for intention recognition , and NinaPro DB1 for muscular movement recognition , show that our T-WaveNet consistently outperforms state-ofthe-art solutions . 2 RELATED WORK . In this section , we review related work on feature extractions for time series analysis and wavelet transform modeling using deep learning techniques . 2.1 FEATURE EXTRACTION FOR SIGNAL ANALYSIS . Existing feature extraction methods for time-varying signal analysis can be broadly classified into hand-crafted and deep learning-based methods , and the former can be further divided into statistical and structural approaches . Hand-crafted statistical and structural approaches are widely used in early studies . For instance , Kao et al . ( 2009 ) applies statistical features such as the mean and the mean absolute deviation ( MAD ) for online activity detection from a portable device . Duan et al . ( 2015 ) performs discrete wavelet transform on surface-Electromyography ( sEMG ) signal for hand motion classification . While relatively easy to calculate , these hand-crafted features are not effective for complicated tasks . In recent years , DNNs have become the mainstream approaches for signal feature extraction . CNNbased models are widely used to extract local temporal correlations of time series data . For example , Lee et al . ( 2017 ) combines multiple CNN layers with different kernel sizes to obtain the temporal dependencies at various time scales . Lawhern et al . ( 2018 ) , Lin et al . ( 2020 ) , and Amin et al . ( 2019 ) utilize CNNs in BCI applications to establish end-to-end EEG decoding models and achieve promising performance . As CNN-based models are often insufficient for extracting long-term temporal features , Ordóñez & Roggen ( 2016 ) , Yuki et al . ( 2018 ) , and Chambers & Yoder ( 2020 ) propose to combine CNNs with LSTMs to extract both short- and long-term temporal features . Xu et al . ( 2019 ) bring out the Inception CNN structures to extract local temporal features at various time scales and utilize gated recurrent units ( GRUs ) to obtain global temporal representations . Recently , Transformer-based methods utilize the self-attention mechanism to model the global temporal dependencies and shows superior performance in various tasks . For example , Song et al . ( 2021 ) construct a simple yet effective Transformer-based model to obtain discriminative representations for EEG signal classification . Li et al . ( 2021 ) design a two-stream convolution-augmented transformer to extract features for human activity recognition , which considers both time-over-channel and channel-over-time dependencies . The above deep learning based solutions try to extract features directly from the raw signal , which ignore the unique characteristics of each type of signals ( e.g. , the frequency spectrum information ) and usually become less effective when training on scarce signal data . Notably , some works try to alleviate this issue by guiding the DNNs with the traditional time-frequency features such that more effective deep representations can be learned . For instance , Ito et al . ( 2018 ) feed the spectrogram images constructed from the temporal features of the inertial signals into CNN models to learn inter-modality features . Laput & Harrison ( 2019 ) train a CNN on the time-frequency-spectral features of the input sensor signal to obtain a fine-grained hand activity sensing system . Furthermore , S et al . ( 2019 ) integrates the Short-Time Fourier Transform into the DNNs to directly learn frequency-domain features . However , the above solutions often need to design tailored features for different kinds of signals , which is tedious and time-consuming , and severely limits the application scenarios . 2.2 WAVELET TRANSFORM MODELING . Frequency-domain feature extraction methods such as Fourier or Wavelet transform are more preferable to time-domain ones , since it is usually easier to extract discriminative features in the frequency domain than directly from the raw input signal . Compared with Fourier transform , which decomposes a signal into fixed frequency components , Wavelet transform is shown to have excellent properties for transient signal analysis , thanks to its capability to analyze signals at different frequencies with various time resolutions . In this section , we focus on Wavelet transform . Recently , integrating wavelet transform with deep learning techniques is shown to be effective in signal and image processing . Early attempts directly replace certain layers in the DNNs with traditional wavelet transform to reduce the number of parameters as well as improve the interpretability . For instance , Williams & Li ( 2018 ) replace the max-pooling layer with a wavelet pooling algorithm to address the overfitting problem . Fujieda et al . ( 2017 ) propose to replace the pooling and convolution layers with wavelet transform . The above methods typically utilize a fixed wavelet basis ( such as Haar ( Haar , 1910 ) ) , which tend to be sub-optimal as it is less flexible and expressive for complex data . Recently , Rodriguez et al . ( 2020 ) propose to replace the fixed wavelet basis in the lifting scheme with deep learnable modules to realize an adaptive wavelet transform unit for image classification , aiming at learning more discriminative frequency features from images . In this work , to better handle the complex spatial-temporal time series data , we build a novel deep wavelet transform unit with powerful representation capability , named frequency bisection operator . It decomposes the input signal into frequency subbands of various sizes and models the wavelet basis with an INN ( Dinh et al. , 2014 ) . To the best of our knowledge , this is the first attempt to model the wavelet transform using INN . We detail the proposed solution in Section 3.2 .

Summary: The authors introduce a deep learning approach to classify time series. In particular, a wavelet-based algorithm is carried out using invertible neural networks and a formant-based strategy. Besides, self-attention mechanisms are utilized to code short and long time-series correlations. Finally, different databases are tested, showing an interesting performance for non-stationary pattern coding.