T-WaveNet: A Tree-Structured Wavelet Neural Network for Time Series Signal Analysis

1 INTRODUCTION . Time-varying signal analysis plays a crucial role in various applications . For example , smartwatches utilize inertial signals for human activity logging ( Khan et al. , 2019 ) ; brain-computer interfaces employ electroencephalography ( EEG ) signals to identify user intentions ( Zhang et al. , 2020 ; Autthasan et al. , 2021 ) ; clinical diagnosis systems use surface electromyography signals ( sEMG ) for neuromuscular pathological analysis ( Duan et al. , 2015 ) , and develop muscle-computer interfaces to control external devices ( Pancholi et al. , 2021 ) . The signals mentioned above are all typical time series data , i.e. , a set of observations collected and ordered chronologically , and expressing information at specific frequency ranges . Generally speaking , time series signal analysis consists of three steps : ( i ) data segmentation , wherein the continuous signals are partitioned into segments using fixed- or variable-sized windows ; ( ii ) feature extraction , wherein various techniques are applied on each segment to extract distinguishing features ; and ( iii ) downstream tasks , which generate the desired outputs for certain tasks ( e.g. , classification and anomaly detection ) with the extracted features . Among the three steps , feature extraction is usually the most critical one . Traditional feature extraction approaches for time series signals can be broadly categorized as statistical and structural methods ( Lara et al. , 2012 ) . The former utilizes statistical measurements in time- or frequencydomain to figure out discriminative features , where exemplar time-domain measurements are mean and variance ( Kao et al. , 2009 ; Vepakomma et al. , 2015 ) , and typical frequency-domain measurements include short-time Fourier transform ( STFT ) and discrete Wavelet transform ( DWT ) ( Jiang & Yin , 2015 ; Duan et al. , 2015 ) ) . On the other hand , structural methods aim at describing the morphological interrelationship among the data via polynomial and/or exponential analysis ( Olszewski , 2001 ) . While the above hand-crafted features are efficient for some simple signal analysis tasks , their limited representation capability makes them incapable of dealing with complicated signals contaminated by noise or artifacts , and hence less competitive in real-world tasks ( Chen et al. , 2021 ) . Recently , deep neural networks ( DNNs ) have become the mainstream approaches for feature extraction in time series signal analysis . Various DNN models , including CNN-based solutions ( Xi et al. , 2018 ; Lawhern et al. , 2018 ; Lin et al. , 2020 ; Amin et al. , 2019 ) , CNN-LSTM combined models ( Wang et al. , 2020 ; Xu et al. , 2019 ; Yuki et al. , 2018 ) , and Transformer-based techniques ( Song et al. , 2021 ; Li et al. , 2021 ) , are proposed in the literature , achieving promising performance in many tasks . Despite of the remarkable success of these deep models , most of them place heavy demands on a large amount of labeled data . However , in many real-world applications , it is difficult and/or expensive to acquire sufficient labeled data to train these models . In addition , noise and artifacts that commonly exist in the time series signals make it even harder to obtain discriminative and robust representations . To the end , it may not fully reveal the superiority of deep models when solely relying on the network to directly extract information from raw signals . To solve this problem , some works attempt to use prior knowledge or hand-crafted features to guide the training of the deep learning models , aiming at extracting more effective features with a limited amount of training data ( Ito et al. , 2018 ; Wei et al. , 2019 ; Laput & Harrison , 2019 ) . However , these methods either simply leverage prior knowledge as gates to select features or intuitively add some statistical measurements as regularization terms to the losses , which fail to bring significant improvement . Moreover , designing tailored feature sets for different kinds of signals is tedious and time-consuming , which severely limits the generalization capability of such methods . In this regard , a promising way to acquire more discriminative features is to explore approaches to deeply integrating the inherent properties of the signals into the training process of the deep learning models . An inherent property termed as the dominant frequency range has been evidenced in many time series signals , which is a small subset of the frequency components that carries the primary information of the signal ( Telgarsky , 2013 ) . For instance , more than 95 % of human body motion energy exists in the frequency components below 15 Hz ( Karantonis et al. , 2006 ) ; the informative frequency ranges in brain signals ( EEG ) have been discovered and named as δ ( 0.1-3 Hz ) , θ ( 4-7 Hz ) , α ( 8–13Hz ) , β ( 14-30 Hz ) and γ ( 31-100Hz ) ( CR & MP , 2011 ) . However , most existing deep learning models are unaware of this essential property , and do not fully consider the different roles of various frequency components , leading to sub-optimal solutions . Motivated by the above , we propose a novel tree-structured wavelet neural network , namely T-WaveNet , to extract more effective features from time-series signals by seamlessly and effectively integrating this property into a deep model . Different from previous models , the proposed T-WaveNet adaptively represents the dominant energy range of the input signal with more discriminative features , which can be naturally and easily generalized and applied to different kinds of time series signals . The main contributions of this work are threefold : • We perform frequency spectrum energy analysis for signal decomposition in T-WaveNet , wherein the frequency ranges with more energies are divided into finer-grained subbands and are thus represented with more dimensions compared with other low-energy frequency ranges in the feature vector , which facilitates effective learning of informative and discriminative features with a limited number of potentially noisy training samples . • To extract the features effectively , we propose a novel invertible neural network ( INN ) based wavelet transform as the tree node in the T-WaveNet . Compared to fixed-basis wavelet Haar or lifting scheme-based wavelet , INN-based wavelet provides better representation capacity owing to its entirely data-driven characteristic . To the best of our knowledge , this is the first attempt to model wavelet transform using INN . • Finally , inspired by the self-attention mechanism in Transformer , we propose an instrumental feature fusion module which considers the feature dependencies across different frequency components , thus effectively enhances the robustness of the model by mitigating the impact of the heterogeneity exists in sensor signals recorded from different subjects . Extensive experiments on four popular sensor signal datasets , namely UCI-HAR for activity recognition , OPPORTUNITY for gesture recognition , BCICIV2a for intention recognition , and NinaPro DB1 for muscular movement recognition , show that our T-WaveNet consistently outperforms state-ofthe-art solutions . 2 RELATED WORK . In this section , we review related work on feature extractions for time series analysis and wavelet transform modeling using deep learning techniques . 2.1 FEATURE EXTRACTION FOR SIGNAL ANALYSIS . Existing feature extraction methods for time-varying signal analysis can be broadly classified into hand-crafted and deep learning-based methods , and the former can be further divided into statistical and structural approaches . Hand-crafted statistical and structural approaches are widely used in early studies . For instance , Kao et al . ( 2009 ) applies statistical features such as the mean and the mean absolute deviation ( MAD ) for online activity detection from a portable device . Duan et al . ( 2015 ) performs discrete wavelet transform on surface-Electromyography ( sEMG ) signal for hand motion classification . While relatively easy to calculate , these hand-crafted features are not effective for complicated tasks . In recent years , DNNs have become the mainstream approaches for signal feature extraction . CNNbased models are widely used to extract local temporal correlations of time series data . For example , Lee et al . ( 2017 ) combines multiple CNN layers with different kernel sizes to obtain the temporal dependencies at various time scales . Lawhern et al . ( 2018 ) , Lin et al . ( 2020 ) , and Amin et al . ( 2019 ) utilize CNNs in BCI applications to establish end-to-end EEG decoding models and achieve promising performance . As CNN-based models are often insufficient for extracting long-term temporal features , Ordóñez & Roggen ( 2016 ) , Yuki et al . ( 2018 ) , and Chambers & Yoder ( 2020 ) propose to combine CNNs with LSTMs to extract both short- and long-term temporal features . Xu et al . ( 2019 ) bring out the Inception CNN structures to extract local temporal features at various time scales and utilize gated recurrent units ( GRUs ) to obtain global temporal representations . Recently , Transformer-based methods utilize the self-attention mechanism to model the global temporal dependencies and shows superior performance in various tasks . For example , Song et al . ( 2021 ) construct a simple yet effective Transformer-based model to obtain discriminative representations for EEG signal classification . Li et al . ( 2021 ) design a two-stream convolution-augmented transformer to extract features for human activity recognition , which considers both time-over-channel and channel-over-time dependencies . The above deep learning based solutions try to extract features directly from the raw signal , which ignore the unique characteristics of each type of signals ( e.g. , the frequency spectrum information ) and usually become less effective when training on scarce signal data . Notably , some works try to alleviate this issue by guiding the DNNs with the traditional time-frequency features such that more effective deep representations can be learned . For instance , Ito et al . ( 2018 ) feed the spectrogram images constructed from the temporal features of the inertial signals into CNN models to learn inter-modality features . Laput & Harrison ( 2019 ) train a CNN on the time-frequency-spectral features of the input sensor signal to obtain a fine-grained hand activity sensing system . Furthermore , S et al . ( 2019 ) integrates the Short-Time Fourier Transform into the DNNs to directly learn frequency-domain features . However , the above solutions often need to design tailored features for different kinds of signals , which is tedious and time-consuming , and severely limits the application scenarios . 2.2 WAVELET TRANSFORM MODELING . Frequency-domain feature extraction methods such as Fourier or Wavelet transform are more preferable to time-domain ones , since it is usually easier to extract discriminative features in the frequency domain than directly from the raw input signal . Compared with Fourier transform , which decomposes a signal into fixed frequency components , Wavelet transform is shown to have excellent properties for transient signal analysis , thanks to its capability to analyze signals at different frequencies with various time resolutions . In this section , we focus on Wavelet transform . Recently , integrating wavelet transform with deep learning techniques is shown to be effective in signal and image processing . Early attempts directly replace certain layers in the DNNs with traditional wavelet transform to reduce the number of parameters as well as improve the interpretability . For instance , Williams & Li ( 2018 ) replace the max-pooling layer with a wavelet pooling algorithm to address the overfitting problem . Fujieda et al . ( 2017 ) propose to replace the pooling and convolution layers with wavelet transform . The above methods typically utilize a fixed wavelet basis ( such as Haar ( Haar , 1910 ) ) , which tend to be sub-optimal as it is less flexible and expressive for complex data . Recently , Rodriguez et al . ( 2020 ) propose to replace the fixed wavelet basis in the lifting scheme with deep learnable modules to realize an adaptive wavelet transform unit for image classification , aiming at learning more discriminative frequency features from images . In this work , to better handle the complex spatial-temporal time series data , we build a novel deep wavelet transform unit with powerful representation capability , named frequency bisection operator . It decomposes the input signal into frequency subbands of various sizes and models the wavelet basis with an INN ( Dinh et al. , 2014 ) . To the best of our knowledge , this is the first attempt to model the wavelet transform using INN . We detail the proposed solution in Section 3.2 .

Summary: This paper introduces a novel technique for the analysis of biomedical signals. The technique is inspired from Wavelet transform and consists in analyzing the original signal in different frequency subband. The technique differs from wavelet transform by the fact that instead of using a fixe wavelet function, a dedicated invertible neural network is learnt for each frequency subband. Finally, features from each subbands are fused using a Transformer like model. The proposed techniques has been thoroughly tested on separate datasets and separate tasks, and compared to various state-of-the-art techniques, and the proposed technique has consistently outperformed other techniques on these various tasks. Finally, an ablation study has been performed in order to demonstrate the benefits of the different choices of the model architecture.