MultiWave: Multiresolution Deep Architectures through Wavelet Decomposition for Multivariate Timeseries Forecasting and Prediction
Authors: Iman Deznabi
Madalina Fiterau
Venue: Submitted to ICLR 2023 (2023)
Keywords: Time series
Wavelets
Wavelet decomposition
Recurrent Neural Networks
Deep Learning

Abstract: One of the challenges in multivariate time series modeling is that changes in signals occur with different frequencies, even when the sampling rate is consistent across signals. In the case of multivariate time series prediction, the outcome is also determined by patterns of different frequencies. These encapsulate both long-term and short-term effects, which have so far not been sufficiently leveraged by deep learning time series models. We fill this gap by introducing a framework, called MultiWave, which augments any deep learning time series model with components operating at the intrinsic frequencies of the signals. MultiWave applies wavelet decomposition on each signal to obtain subsignals of different frequencies and groups all subsignals in the same frequency band together to train a component. The output of the components is combined through a gating mechanism that removes irrelevant frequencies for the given predictive task. We show that MultiWave accurately determines the informative frequency bands and that the augmented models including components trained to operate on those bands outperform the original models. We further show that applying MultiWave on top of different deep learning models improves their performance in several real-world applications.

Under review as a conference paper at ICLR 2023
MULTIWAVE: MULTIRESOLUTION DEEP ARCHITEC- TURES THROUGH WAVELET DECOMPOSITION FOR MUL- TIVARIATE TIMESERIES FORECASTING AND PREDIC- TION
Anonymous authors authors Paper under double-blind review
ABSTRACT
One of the challenges in multivariate time series modeling is that changes in signals occur with different frequencies, even when the sampling rate consistent across signals. In the case of multivariate time series prediction, the outcome is also determined by patterns different frequencies. These encapsulate both long-term and short-term effects, which have so far not been sufficiently leveraged by deep learning time series models. We fill this gap by introducing framework, called MultiWave, which augments any deep learning time series model with components operating at the intrinsic frequencies of the signals. MultiWave applies wavelet decomposition on each signal to obtain subsignals of different frequencies and groups all subsignals in the same frequency band together to train a component. The output of the components is combined through a gating mechanism that removes irrelevant frequencies for the given predictive task. We show that MultiWave accurately determines the informative frequency bands and that the augmented models including components trained to operate on those bands outperform the original models. We further show that applying Multi Wave on top of different deep learning models improves their performance in several real-world applications.
INTRODUCTION
Multivariate time series prediction has long been a crucial task in machine learning, as has important applications in many fields such as healthcare, traffic flow, and economic forecasting. However, the final prediction these applications can depend on many factors, such as information at different frequencies, long-term and short-term changes in input signals. Moreover, in many tasks, observations come from multiple sources and are often collected at various sampling rates. Here, we propose a approach that can leverage temporal dependencies different frequencies and scales in multivariate time series data that might be collected with multiple sampling rates (multirate time series data) using multilevel discrete wavelet decomposition.
There are two important categories of methods for time series analysis: Time-domain methods that consider the time series sequence of ordered points in time and frequency-doman methods that use transform algorithms such as Fourier transform and Z-transform to analyze the original sequence in the frequency spectrum. Deep learning-based methods that are introduced into time series analysis, such as recurrent neural networks (Williams & Zipser 1989). Convolutional Neural Networks (CNN) (Zheng al. 2016) and more recently transformers (Wen al. 2022) achieve state-of-the -iee---f-heearrr results in many applications (Lai et al. [202 Tipirneni & Reddy 2021 Huang et al.) 2022). However, they have two notable shortcomings handling multivariate time series data, First, most of these methods only use information available in the time domain and cannot leverage information present in the frequency domain of the signals. Additionally, these methods cannot directly model signals that are collected with different frequencies (multirate signals), and upsampling downsampling these time series to a single rate can artificially introduce or remove some important temporal dependencies Chel et al. 2018a) Tipirneni & Reddy (2021): Che et al. (2018b).
To overcome these deficiencies, we propose a novel model-agnostic framework, which uses discrete wavelet decomposition to break signals into different frequency components (subsignals), group

---

Under review as a conference paper at ICLR 2023
MM
Signal
lowpass
highpass
A1
D1
lowpass
highpass
Figure I: Multi level Discrete Wavelet Decomposition, image on the left shows how lowpass and highpass filters are used to decompose signals in multi level discrete wavelet decomposition and the image on the right shows a signal getting decomposed by Haar wavelet. As you can see the resulted signals are zero except the signal matching the true frequency of the original signal.
subsignals with similar frequencies into separate time series models, and then combine the output of the models to make a prediction. This framework brings the following improvements to multivariate time series modeling: 1) Model-agnostic, MultiWave time series model. 2) Uses the information available in both the time and frequency domains. 3) Reduces the amount of variation between the sampling frequency of multiresolution signals that modeled together. 4) Provides unique insight into which frequencies of the signals are important for given task.
be applied to any neural network-
BACKGROUND
2.1 NOTATION
We denote multivariate and multirate time series data with m signals collected before time T set of signals X1:I {-t ,42 ...,,"hhh where each signal is collected at initial rates R (ri, "......mm/// The length of each signal proportional its collected rate Len, T. The r; problem is given X1:7 1.T we want to predict a label y. which can continuous (regression) or discrete (classification). In the rest of the paper, will remove the time indication for the signals and show the set of signals as X and the signal as v4. We show the sampling rate of signal x as fs (x)
2.2 MULTILEVEL DISCRETE WAVELET DECOMPOSITION
We use wavelet decomposition to break down the signals into different frequencies. Wavelet decom- positions (Daubechies) 1992) are well-known methods for capturing information time series in both the time and frequency domains. They have been used successfully
preprocessing step for neural networks (Liu et al. 2013 Wang et al., (2020a) and as an integral part of them (Subasi et al,) 2006 Zhang al. 1995| Wang et al. 20188 Kumar al.) 2021). Multilevel discrete wavelet decomposition can extract multilevel frequenyy features from a time series by iteratively applying low-pass and high-pass filters derived from wavelets the signal. The formula for this decomposition is given below:
xxt)= =
Σ ALL + Σ DL.AWLL
&
Σ De-IAV +...+ DI.NVI,
V..r is the mother wavelet with scale and time and ø is the father wavelet. This multilevel wavelet decomposition converts the input signal ztt) into signals A₁ = U₁ AL.k, which is a coarse general approximation of the signal (low frequency) and the detail coefficients D2 U2 DL.k- Dz-1 D.. = UR Di, that influence the function on various scales. Figure D depicts this decomposition. To simplify the notation, show the decomposition of signal
set
2

---

Under review as a conference paper at ICLR 2023
S(z) {D1, D2, Dz....., AL} that includes signals retrieved when decomposing z(t) at L levels. This notation allows us to denote D1 as Sixx), D2 as S2(z). Dr as SL()), and As as (L+1(z).
There many different wavelets introduced in the literature, such as Haar, Daubechies, and Biorthogonal (Haar 1909; Cohen et al.] 1992, Daubechies 1992) that can be used here. Our framework is independent of the type of wavelet used.
2.3 RELATED WORK
Multirate time series classification Deep learning-basee based methods that are used for time series analy- sis, such recurrent neural networks (Williams & Zipser 1989). Convolutional Neural Networks (CNN) (Zheng et al. 2016) and more recently transformers Wen et al. 2022) achieve state-of-the-ar results in many applications (Lai et al. [2022] Huang et al. (022)) However, these approaches cannot be used directly for multirate time series data. To be able to apply these models in these settings, signals should be aligned by upsampling lower frequency signals or downsampling higher frequency signals, leading to emrrrs in prediction and loss of information Che et al. (2018a); Tipirneni & Reddy (202T). These models fall short in two important ways when handling multivariate time series data. First, most of these methods only use information available in the time domain and are not designed to take advantage of the information present in the frequency domain of signals. Furthermore, these methods cannot be easily used on signals that are collected with different frequencies (multiresolution signals). To be able to apply these models, the signals should be aligned either by upsampling the lower frequency signals, prediction and loss of information Che et al. (2018a); Tipireni & Reddy (202T). There are several models proposed to solve this problem, such as Che et al. (201 8a) and Tipirneni & Reddy (2021) which can inherently model irregularly sampled time series data and thus can model multirate time series data without aligning the signals. There are other methods, such as Che et al. (2018b); Armesto eet al.] (2008); Safari et al. (2014) that use architectures specifically developed for multirate time series data. All these approaches only consider the information available in the time domain of the time series data, while our model is specifically developed for multirate data and is able to leverage the information in the frequency domain.
downsampling the higher frequency signals that lead to errors in
F Frequency analysis of time series Frequency analysis of time series is an extensively studied subject in the signal processing community. Methods such as the discrete Fourier transform (Bracewell & Bracewell, 1986). the discrete wavelet transform (Daubechies] 1992). and Z-Transform (Foster) 1996) have been used to analyze time series. For deep learning models, similar methods have been used in the preprocessing steps (Cui et al. [016)] Yuan et al., 2017: [Song et al., 2021) or part of neural networks (Koutnik et al. 2014 Lee- Thorp et al.) 2021). l). Most of these models focus on univariate time series data and cannot be directly used on multivariate and multirate time series data.
Wavelet decomposition Wavelet decompositions (Daubechies) 1992) are well-known methods for capturing information in time series in both the time and frequency domains. They have been used successfully as a preprocessing step for neural networks (Liu et al.] [2013] Wang et al. 2020a Allhaity
et al. 2021 Kim et al.) 2021 Althelaya al. 2021 Zucatelli al. 2021) and an integral part of
them (Subasi et al.) 2006)/Zhang et al. 1995, Wang et al.] 2018: Guo et al.) 2022 Li et al. 202T). Wang al. (2018) proposes the methodology closest to our method, implementing trainable wavelet decomposition framework, which can be trained with the rest of the network. Although similar to our method, this paper uses wavelet decomposition to extract frequency- -based information from datasets and modeling them using different model components; it cannot be applied to multivariate and multirate time series data and does not use feature masks to remove useless frequency components of the signals from the framework. Furthermore, they propose model to be used, while our architecture is model-agnostic and can be applied to any time series model.
MULTIWAVE
3.1 SIGNAL DECOMPOSITION AND FREQUENCY GROUPING
Figure 3 shows the overall structure of the framework for two signals when the signals xn and I2 collected with frequencies of 64Hz and 32 respectively. We will use discrete wavelet decomposition to decompose each signal into different frequency groups, so for m input signals, = (x1, { T2,.... -......,,. )m) we will have set of decomposed signals, S(X) = (S(x)) S(x2)... ..... ((m)))
3

---

Under review as a conference paper at ICLR 2023
X1 64
X2: 30 Hz
12
32 64
5,(%)
Sa(ka)
(xaa))
16 Hz 0 15
16 32 Hz
15 30 Hz
Figure 2: The decomposition and grouping of two signals sampled at 64Hz (21) and the other one 30Hz (I2).
In the case where all signals are sampled at the same rate, all elements of S(X) will have the same number of levels and the frequencies each level will be the same. Therefore, group all frequency components at the same level into a model component. Therefore, the component i, denoted as 4,, receives the input I; {SS )....... (xm)) The number of components is equal the level of wavelet decomposition I.. Then we concatenate the outputs of all models and use a fully connected layer to obtain the output.
FC(# (11) @ (2(12) ..... (n.)))
If the signals are collected with different sampling rates, will group the subsignals with frequencies that are close to each other together into a model component. In this case, the number of model com- ponents equal the maximum level of signal decomposition Lynex max(L L2...., L. } determined by the signal with the highest sampling rate. To better illustrate this, assume, without loss of generality, that we have two signals m where signal ze is collected twice as often as signal æ1. fa (22) level of wavelet decomposition, we have:
x (ar)). ). In this case, L22 T, + 1 since the frequencies are reduced by half at each
(=(S)(r))) (=1) S (1a())))) (S( (x1)) fS( (,()))))))..,,((((((())))))))))) = (5 (12))
So, to model signals with the same rates in each component, this case, the inputs of the model would be:
1 = (5)()))) 12
I1, (1.
If the rates of signals for one component are not equal, we will over-sample the shorter signal to match the longer signal. The difference between the number of levels between the decomposition of two signals is log( Jz(zz) ), so if this proportion is not a power of 2, the shorter signal at each component level should be oversampled to the closest power of 2 to match the corresponding level of other signal. For example, if fa (12) x fa(x1) in the above example, the input will be:
1 (S,))) 12 (S2(22)) 13 (5.())), ...,, It, {St, (x1). St, (x2))
since for the component 33, (x(s((z)) 1.5, (x1) should be oversampled by a proportion of
(a:))
1.5. Note that the oversampling proportion will always less than 2. Figure 2 shows how this decomposition and grouping would work for two signals with different sampling rates.
Using this approach, the components trained on lower frequencies learn long-term changes in the data, while the faster frequency components learn short-term fast-changing trends in the data. Furthermore, since the signals are grouped with respect to their sampling rates, the signals that are input into each component have similar frequencies, which significantly reduces the amount of oversampling.
3.2 MASKING FREQUENCY COMPONENTS
Not all the frequency components of all signals are important for the prediction of the final label. To filter these frequencies, we introduce a learnable mask for each frequency component of the signals. We use ReI.U activations (Agarapl 2018) the weights for the model to be able to mask the
4

---

Under review as a conference paper at ICLR 2023
64 Hz
W..
ReLU
Model
HPF
X2: Hz
32 64 Hz
WLs
ReLU
Fusion
3.0R)
Model
LPF
HPF
LPF
HPF
Wax
RetU
0 0-116 Hz
16-32 Hz
166 Hz
Hz
5,0%)
W22
ReLU
9,6%)
Model
W ReLU
Figure 3: Structure of the model when there are two signal inputs: One that is sampled 64Hz (x1) and the other one at 32Hz. (I2). This figure shows how wavelet decomposition decomposes these signals into different frequencies with low pass filters (I.PF) and high pass filters (HPF) and how we group these signals into different model components. This is model-agnostic architecture any model which works on multivariate time series data can utilized.
uninformative components by setting the mask to zero so the input to component in the model is defined
(sr))) "c (W. (s)()))
=
In order to force the model to use sparse mask for the feature components we add the l norm of the weights the final loss of the model:
Loss C al1 (Re((RUU()))
where the normal loss of the model which is usually defined by Mean- Mean-Squared--Frrr (MSE) for regression models and Cross- Entropy-Loss for classification models. is hyperparameter that determines the weight of the regularization term and sets a trade-off between minimizing the mask weights and the model's loss.
3.3 FINAL FUSION OF COMPONENTS
We tried many different fusion techniques, such as attention, mean, weighted average, ensemble methods, transformer fusion, as well as GradBlend (Wang al. 2020b), Efficient Low-rank Multi- modal Fusion with Modality-Specific Factors (Liu et al. 2018). 8). Du et al.] (2018) but concatenating and passing through a fully connected layer worked best in our experiments. This considering the unique structure of the inputs to the fusion layer, as each input will have some partial information about the target and should be combined to get the output. Note that Multi Muve is agnostic to the choice of fusion technique, and while in our experiments concatenating and passing through a fully connected layer works best for other datasets, different fusion techniques can be utilized.
surprising
3.4 MODEL TRAINING
Algorithm DaoummSshoooe shows the MultiWave training procedure. We first train the model with feature masks included, and then remove the features that have mask value of zero they do not contribute to the model output anymore. We also remove all the components for which all the feature masks have values of zero. Then add the bascline model with input of the original signals x1:T {x] x1 ...., ,,::7' as an additional component to the fusion model, and then train the model without the included masks. In this way, MultiWave can fall back the baseline model (using early fusion of frequencies) if the addition of frequency components does not help, while also being able to frequency components to improve the performance.
We used Weights & Biases Biewald (2020) for tracking and logging the experiments and Pytorch Paszke al. 2019 implement and train models.
5

---

Under review as a conference paper at ICLR 2023
Algorithm 1 MultiWave training procedure
Input n samples with signals X1:T" {x] 22
Apply wavelet decomposition and obtain 1, 3.... ) with rates R = fri, 22, "2,.,.TTmm
1.2
S, (,(m)) for i E {1 (1....).
Train fusion model with components (,(4) and weights W, for E {1... L}, where each W; is made up of weights W: corresponding to subsignal S: (14), i.e. the part of the signal æy for j E {1...m m provided to component i. The model is trained with ReLU masks, as shown in Figure 3 which denote M (J),, and which are 0 if the ReLU filtered out the corresponding subsignal.
for Each component P,, i E (1... [) do for Each mask M(); do
if M(); 0 then
Set w = 0 (remove corresponding frequency)
end if end for
(j); = for all j then
Set Q; = 0 (remove component i)
end end for if AddBaseline is True then
Add the baseline model with original signals input X1.77 additional component
fx1:T 202
x..TT as
Continue train the model without training the mask weights end if
4 EXPERIMENTS
In this section, we evaluate the performance of MultiWave in synthetically generated and datasets.
4.1 SYNTHETICALLY GENERATED DATA
To determine the effectiveness of MultiWave in handling signals with different frequencies and sampling rates, we generated synthetic data. The generated data consist of multiple square signals, each with different frequency and amplitude. The amplitude of each signal is randomly selected from a uniform distribution ranging from 0-10. We then add uniform noise to the input data with an amplitude of 3. The label y is the sum of the amplitudes of the generated signals. The task is to predict the label y from given time series data. In Figure results of 2 experiments are shown. For the first experiment, we start with two square wave signals with frequencies of I and Hz. sampled at 128 Hz. for 1 second. Then we iteratively add two signals to the input with the same sampling rate but frequencies of (4, 4, 8, 8, 16, 16, 32,32) 32). respectively. The performance of the LSTM and Transformer model with the addition of MultiWave are shown in the first column of Figure 4 MultiWave delivers consistent and robust improvements with the addition of new signals. In the second experiment, we generate two square-wave signals with frequencies of 2Hz and 4Hz sampled, respectively, at 64 and 128 Hz., for I second. Then we reduced the sampling rate of the first signal from 64 Hz. to 32, 16, 8, and 4 Hz. The results for the LSTM and Transformer models with and without the addition of MultiWave are shown in the second column of Figure Here again, Wave consistently improves the performance of the baseline model.
4.2 THE WEARABLE STRESS AND AFFECT DETECTION (WESAD)
Wearable Stress and Affect Detection (WESAD) is a publicly available multimodal data set for stress and affect detection. WESAD contains physiological response data from 15 subjects during three sessions of baseline, amusement, and stress. The baseline session is 20 minutes, where the subject doing neutral reading task, the amusement session is watching set of funny videos for 392 seconds, and the stress session is when the subject is exposed to the Trier Social Stress Test Kirschbaum et al. (1993) for 10 minutes. During these sessions, physiological measurements such as

---

Under review as a conference paper at ICLR 2023
1.4
Model
Model
MultWWave
LSTM MultWWave
1.0
1.5
0.8
MSE
0.5
0.2
10
2
2
2
2
2
Number of Signals
Sampling Rate
1.4
Model
1.2
Transformer MultiWave
2.0
Transformer
Transformet
Transforme
1.0
MSF
MSE
0.6
0.4
0.5
0.2
2
10
2
2"
2
Number of Signals
Sampling Rate
Figure 4: The Mean Squared Error (MSE) results on synthetic dataset, when we increase the number of signals (left column) and change the difference between the signal sampling rates (right column). First row describs the results for LSTM model while second row shows the results for Transformers.
Dataset
Model
AUC without MultiWave
AUC with MultiWave
LSTM
0.822 0.04
0.828 + 0.04
WESAD
CNN-Attn
0.831 = 0.03
0.877 1 0.03
CNN-LSTM
0.807 = 0.04
0.839 + 0.04
FCN
0.805 = 0.04
0.833 1 0.05
LSTM
0.983 + 0.008
0.989 + 0.004
COVID-19
CNN-Attn
0.978 = 0.012
0.979 1 0.02
0 days ahead
CNN-LSTM
0.979 = 0.010
0.981 + 0.012
Transformer
0.980 = 0.007
0.984 +: 0.008
LSTM
0.977 + 0.007
0.979 + 0.006
COVID-19
CNN-Attn
0.967 = 0.008
0.967 + 0.013
12 days ahead
CNN-LSTM
0.961 + 0.012
0.962 + 0.009
Transformer
0.969 = 0.01
0.972 + 0.01
Table 1: The AUC results on real-world datasets
blood volume pulse (BVP), electrocardiogram (ECG), electrodermal activity (EDA), electromyogram
(EMG), respiration (RESP), temperature (TEMP), and accelerometer (ACC) are collected using a chest-worn device and wrist-worn device. The chest-worn device collects data at 700 Hz. while the wrist-worn device collects data at 64, 32 and Hz. We followed Dziežye et al. (020) to preprocess the data. Unlike Dziezyc et al. (2020), however, more consistent signals. More details on the preprocessing of the signals in this data set are given in the Supplementary Section A.
sampling rates that powers of to obtain
7

---

Under review as a conference paper at ICLR 2023
Frequency component
Features
Ie days
High sensitivity C-reactive protein, Glucose
I g days
Lactate dehydrogenase
g I days
1
D-D dimer
4 2 days
2 I days
(% Hymphhocye High sensitivity C-reactive protein
Table 2: The features with nonzero mask weights in COVID-19 dataset at the end of the training. These features were consistently selected in 5 different runs of the training procedure, indicating that they are informative for the predictive task, which is in concordance with domain expertise. MultiWave was able to automatically determine these dependencies, as well as determine which of the subsignals of different frequencies are relevant, as shown in Figure
The results achieved in this data set are shown in Table 1 Consistent with the results reported in Dziežyc et al. (2020), CNN models achieve the best performance in this data set, and the Convolutional Neural Network with Attention (CNN-Attn) was presented as achieving the best performance in this data set We could not apply basic Transformers to this dataset, as the length of sequences leads to out-of out-of-memory errors. MultiWave significantly improves the performance of all baseline models, since it allows the model components to learn short-term and long-term changes and can combine these multirate signals without the need for alignment and imputation. More details about these experiments are given in the Supplementary Section A
4.3 COVID-19
The COVID-19 19 dataset Yan et al. (2020) is publicly available data set that contains 74 indicators of 375 patient blood samples from 10 January to February 2020 at Tongji Hospital, Wuhan, China. These indicators are collected in irregular time intervals and sampling rates range from 0 to 6 per day. We are interested in the task of predicting in-hospital mortality given the time series of biomarkers. The duration of hospital stays for patients varies from 2 hours to 35 days.
To process this data set, we sampled the features with different rates ranging from I to 8 day intervals (more details in the Supplementary Section BB. If multiple values are recorded for feature in the determined rate, we use the last recorded value. We fill in the missing values by linear interpolation. If a feature is completely missing for patient, we use the mean value of that feature in all patients in the training set to fill these values. To evaluate the capability to select features in our model, we used all 74 features. Because of this, we were unable to use the original test set provided with the dataset as it only contains 3 features. Therefore, to evaluate our models, we separated 100 patients from the original data set and used 50 patients for validation and 50 patients test our models.
We perform two groups of experiments first to predict mortality when all patient data leading to hospital discharge are included days ahead prediction) and predicting mortality 12 days ahead of hospital discharge is included (12 days ahead prediction). The average AUC results for runs of each experiment are included in T For this data set, we used the LSTM, CNN-Attn, CNN-LSTM, and Transformer models, and the results are reported with and without inclusion of the Multi Wave framework. Consistent with the results reported in Sun et al. (202I). LSTM- models achieve the best results, and MultiWave brings consistent improvements to all baseline models.
Yan et al. (2020) reports three features of lactic dehydrogenase, lymphocytes, and high-sensitivity C-reactive protein are the most important characteristics in the prediction of hospital mortality in this data set. To determine whether Multi Wave can recognize the correct features in predicting the target, looked non-zero masked values in each model component for five different runs and we show the common ones in Table As can be seen, these three features are consistently selected by model, and the important frequency of them is shown. In Figure we show how the mask weights for these features across the components are changed over the training epochs of the model for one run.
Note that the results are not directly comparable to this paper we use different sampling rates and we multi channel inputs to FCN rather than single channel used in this paper
8

---

Under review as a conference paper at ICLR 2023
High sensitivly Conactave protein
Lactate
(hllymphecyte
days
01
328
1/16- 1/8 days
533
012
500
1/8 1/4 days
40
99
523
1/4 1/2 days
20
1/2 1/1 days
Figure 5: The mask weights over time for the three most important features for different frequency components in COVID-19 dataset. The masks that are not zeroed out are highlighted.
CONCLUSIONS
In this paper, we proposed new framework called MultiWave that augments any deep learning time series model with components that operate at different frequencies of signals using wavelet decomposition. We further improved this model by introducing frequency masks, which remove non-informative frequency components of signals from the component inputs. We show that this framework improves the performance of time series models in synthetic datasets, well as two real-world datasets for stress detection and COVID-19 prediction.
REFERENCES
Abien Fred Agarap. Deep learning using rectified linear units (relu), arXiv preprint crXiv:1803.08777 2018.
Bashar Alhnaity, Stefanos Kollias, Georgios Leontidis, Shouyong Jiang, Bert Schamp, and Simon Pearson. An autoencoder wavelet based deep neural network with attention mechanism for multi-step prediction of plant growth. Information Sciences, 560:35-50, 2021.
Khaled Althelaya, Salahadin Mohammed, and El-Sayed El-Alfy. Combining deep learning and multiresolution analysis for stock market forecasting. JEEE Access, 2213099-13111, 2021.
Leopoldo Anmesto, Josep Toreroo, and Markus Vincze. On multi-rate fusion for non-linear sampled- data systems: Application 6d tracking system. Robotics and Autonomous Systems, 56(8): 706-715, 2008.
Lukas Biewald. Experiment tracking with weights and biases, 2020. URL httph:t///// wwndw com/] Software available from wandb.com.
Ronald Newbold Bracewell and Ronald Bracewell. The Fourier transform and its applications, volume 31999. McGraw-Hill New York, 1986.

---

Under review as a conference paper at ICLR 2023
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):1-12, 2018a.
Zhengping Che, Sanjay Purushotham, Guangyu Li, Bo Jiang, and Yan Liu. Hierarchical deep generative models for multi-rate multivariate time series. In International Conference on Machine Learning, pp. 784-793. PMLR, 2018b.
Albert Cohen, Ingrid Daubechies, and J-C Feauveau. Biorthogonal bases of compactly supported wavelets. Communications on pure and applied mathematics, 45(5):885-560 1992.
Zhicheng Cui, Wenlin Chen, and Yixin Chen. Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv: 16Xx::600330995, 2016.
Ingrid Daubechies. Ten lectures on wavelets. SIAM, 1992.
Yunshu Du, Wojciech M Czarnecki, Siddhant M Jayakumar, Mehrdad Farajtabar, Razvan Pascanu, and Balaji Lakshminarayanan, Adapting auxiliary losses using gradient similarity, arXiv preprint arXiv:1812.022244, 2018.
Maciej Dziežyc, Martin Gjoreski, Przemystaw Kazienko, Stanistaw Saganowski, and Matjaž Gams. Can we ditch feature engineering? end-to-end deep learning for affect recognition from physi- ological sensor data. Sensors, 20(22), 2020. ISSN 1424-8220. doi: 10.3390/s202265355 URL
https: wwww. wwwpwwwwww.....mm.. com 4224 /220/20/22/ 6535
Grant Foster. Wavelets for period analysis of unevenly sampled time series. The Astronomical Journal, 11::1709-172, 1996.
Wenhui Guo, Guixun Xu, Baodi Liu, and Yanjiang Wang. Hyperspectral image classification using cnn-enhanced multi-level haar wavelet features fusion network. IEEE Geoscience and Remote Sensing Letters, 19:1-5, 2022.
Alfred Haar. Zur theorie der baahogonnenn.funktiinnnnnnnnnnne 1909.
Gottingen.,
Lei Huang, Feng Mao, Kai Zhang, and Zhiheng Li. Spatial-temporal convolutional transformer network for multivariate time series forecasting. Sensors, 22(3):841, 2022.
Jusong Kim, Xiaoli Wang, Chollyong Kang, Jinwon Yu, and Penghui Li. Forecasting air pollutant concentration using novel spatiotemporal deep learning model based clustering, feature selection and empirical wavelet transform. Science of The Total Environment, 801: 801:149654, 2021.
Clemens Kirschbaum, Karl-Martin Pirke, and Dirk H Hellhammer. The 'trier social stress test' tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology 28 (1-2):76 81, 1993.
Jan Koutnik, Klaus Greff, Faustino Gomez, and Juergen Schmidhuber. A clockwork mn. In International Conference on Machine Learning, pp. 1863-1871. PMLR, 2014.
Peeyush Kumar, Ranveer Chandra, Chetan Bansal, Shivkumar Kalyanaraman, Tanuja Ganu, and Michael Grant. Micro-climate prediction-muil scale encoder-decoder based deep learning frame- work. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 3128-3138, 2021.
Chi Qin Lai, Haidi Ibrahim, Aini Ismafairus Abd Hamid, and Jafri Malin Abdullah. Lstm network as a screening tool to detect moderate traumatic brain injury from resting-state electroenceephllog.... Expert Systems with Applications, pp. 116761, 2022.
James Lee- Thopph Joshua Ainslie, 1lya Eckstein, and Santiago Ontanon. Fnet: Mixing tokens with fourier transforms. arXiv preprint arXiv: 105.03824 2021.
Yiqun Li, Songjian Chai, Zhengwei Ma. and Guibin Wang. A hybrid deep learning framework for long-term traffic flow prediction. IEEE Access, 2211264-1127 2021.
10

---

Under review as a conference paper at ICLR 2023
Hui Liu, Hong qi Tian, Di fu Pan, and Yan fei Li. Forecasting models for wind speed using wavelet, wavelet packet, time series and artificial neural networks. Applied Energy. 107:191-208 208. 2013. ISSN 0306-2619. doi: http://www....eeeeeeee.a.eaaeaaa
URL https:
// /www. www.sciencedii
encedi rect. www.ttcooccc.teeoeeeceeecccccooe
com sci htte:/cco/cceeeeee/eeeeiooe
Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, and Louis PhillpPhil Morency. Efficient low-rank multimodal fusion with modality-specine factors. arXiv preprint arXiv: 1806.000644 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8033 Curran Associates, Inc., 2019. U http::://pppers.eerris....oo
neur
9015 Sess-pytorch--hee an anttorchaa--imeratttee imperativeeeeeeeeeeee styeesssleeeeee high per.opppeerommeeee-eeeee deep-e-eeeeeeeeeeee pepp-oooaaaaeeeee--e-eeaaiinnnna library. pdf
Sajjad Safari, Faridoon Shabani, and Dan Simon. Multirate multisensor data fusion for linear systems using kalman filters and a neural network. Aerospace Science and Technology, 39:465-471, 2014.
Donghwan Song, Adrian Matias Chung Back, and Namhun Kim. Forecasting stock market indices using padding- based fourier transform denoising and time series deep learning models. IEEE Access, 9:83786-83796. 2021.
Abdulhamit Subasi, Mustafa Yilmaz, and Hasan Riza Ozcalik. Classification of emg signals using wavelet neural network. Journal of Neuroscience Methods, 156(1)) (66(1))360-36)) 2006. ISSN 0165-0270. doi: https://doi.org/10..o
00.0044 URL https: / /ww
sclencedirech wttp://c...ccccceceeeeeeeeee
Chenxi Sun, Shenda Hong. Moxian Song, Yanxiu Zhou, Yongyue Sun, Derun Cai, and Hongyan Li. Te-esn: Time encoding echo state network for prediction based irregularly sampled time series data. arXiv preprint arXiv:2 104,0 2021.
Sindhu Tipirneni and Chandan Reddy. Self-supervised transformer for multivariate clinical time-series with missing values. arXiv preprint arXiv:2107.14293 2021.
Hongyu Wang, Wenrui Ding, Duona Zhang, and Baochang Zhang. Deep convolutional neural network with wavelet decomposition for automatic modulation classification. 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 1566-1571. IEEE, 2020a.
Jingyuan Wang, 7e Wang, Jianfeng Li, and Junjie Wu. Multilevel wavelet decomposition network for interpretable time series analysis. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 2437-2446, 2018.
Weiyao Wang, Du Tran, and Matt Feiszli. What makes training multi- classification networks hard? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12695-12705, 2020b.
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125, 2022.
Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2):270-280, 1989.
Li Yan, Hai-Tao Zhang, Jorge Goncalves, Yang Xiao, Maolin Wang. Yuqi Guo, Chuan Sun, Xiuchuan Tang, Liang Jing, Mingyang Zhang, et al. An interpretable mortality prediction model for covid-19 patients. Nature machine intelligence, 2(5):283-288) 2020.
Ye Yuan, Guangxu Xun, Kebin Jia, and Aidong Zhang. A multi-view deep learning method for epileptic seizure detection using short-tima fourier transform. In Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, pp. 213-222, 2017.
11

---

Under review as a conference paper at ICLR 2023
Jun Zhang, Gilbert G Walter, Yubo Miao, and Wan Ngai Wayne Lee. Wavelet neural networks for function learning. IEEE transactions on Signal Processing, 43(6):1485- -4947 1995.
Yi Zheng, Qi Liu, Enhong Chen, Yong Ge, and J Leon Zhao. Exploiting multi-channels deep convolutional neural networks for multivariate time series classification. Frontiers of Computer Science, 10(1):6-111,, 2016.
PJ Zucatelli, EGS Nascimento, AÁB Santos, AMG Arce, and DM Moreira. An investigation on deep learning and wavelet transform to nowcast wind power and wind power ramp: A case study in brazil and uruguay. Energy, 208:2 2021.
12

---

Under review as a conference paper at ICLR 2023
Signal
Original Sampling
Downsampled
ECG RespiBAN
700 Hz
64 Hz
ACC RespiBAN
700 Hz
Hz
EMG RespiBAN
700 Hz
Hz
EDA RespiBAN
700 Hz
4 Hz
TEMP RespiBAN
700 Hz
4 Hz
Respiration RespiBAN
700 Hz
Hz
BVP Empatica
64 Hz
64 Hz
ACC Empatica
32 Hz
8 Hz
EDA Empatica
4 Hz
Hz
TEMP Empatica
4 Hz
4 Hz
Table 3: WESAD dataset feature frequencies
A WESAD
A.1 PREPROCESSING
We follow Dziežyc al. (2020) preprocess the data and use similar technique to downsample the signals. However, instead of the frequencies used the article, we use frequencies that are powers of 2 given in the table We split data into five folds and for each fold train the data 3 folds, validate on 1 fold, and test on the last remaining fold. We do this 5 times and the mean and standard deviation are reported in the paper.
A.2 HYYPEPPARAMETER
We select hyperparameters for the baseline models based on the validation set using grid search for each model and select hyperparameters for the components of MultiWave such that the number of parameters is approximately equal between the two models. For other MultiWave hyperparameters, we select them using grid search on the validation set as well, The hyperparameters selected for the WESAD data set are given in Table 6
B COVID-19 DATASET
B. PREPROCESSING
This is an irregularly sampled dataset, we resample features based on their overall rate all training data according to T If feature is missing in the time of sampling, used the closest available value, if feature is completely missing for a patient, we fill the values with average of that feature among all the training samples. We also normalize the features values between 0 and using min-max scaler.
B.2 HYPERPARAMETERS
We use the validation data to select the hyperparameters for the baseline model using grid search, and then for components of MultiWave,
set the model sizes such that the number of parameters is approximately the same between the two models. We use data from 50 subjects for validation. The hyperparameters selected for MultiWav in this dataset are given in Table
13

---

Under review as a conference paper at ICLR 2023
Model
Hyper parameter
Value
LSTM cell size
28
Initial Mask Weight values
0.5
Mask norm weight
0.05
LSTM
Number of Layers
Learning rate
0.001
Batch size
16
pattence
15
Number of Components
AddBaseline
True
CNN Kernel size
Initial Mask Weight values
0.5
Mask norm weight
0.1
CNN-Attn
Number of Layers
2
Learning rate
0.001
Batch size
16
pattence
15
Number of Components
AddBaseline
True
CNN Kernel size
Initial Mask Weight values
0.5
Mask norm weight
0.1
CNN-LSTM
Number of Layers
2
Learning rate
0.001
Batch size
16
patience
15
Number of Components
AddBaseline
False
CNN Kernel size
Initial Mask Weight values
0.5
Mask norm weight
0.1
FCN
Number of Layers
2
Learning rate
0.001
Batch size
16
patience
15
AddBaseline
False
Table 4: The selected MultiWave hyperparameters for WESAD dataset
14

---

Under review as a conference paper at ICLR 2023
Signals
Sampled
Rates
hemoglobin, Serum chloride, Prothrombin samneprocalcitonnneooiiii http::aseeaabuuibbaasaassooo
every
Total Setirubin,Platekee bilirubin, Red blood cell distribution width, neutrophils(%) (,), total protein, Prothrombin activity, mean corpuscular volume, hematocrit, White blood cell count, mean corpus- cular hemoglobin concentration, fibrinogen, Urea, lymphocyte count, Red blood cell count, Eosinophil count, Corrected calcium, Serum potassium, glucose, neutrophils count, irect bilirubin, Mean platelet volume, RBC distribution width SD), Thrombin time, (0),,% (m)yyythoy D-D dimer, Total cholesterol, aspartate aminotransferase, Uric acid, HCOS-callcimmaacteee Lactate dehydrogenase, platelet large cell ratio, monocytes count, PLT distribution width, globuli..7-glutamyl transpeptidase, International standard tio,basophi count(#) (amt(())meaa mean corpuscular hemoglobin ,Activation of partial thromboplas- tin High sensitivity C-reactive protein,serum sodium, thromoooyttoroiigluuaatii pyruvic transaminase,eGii creatiniee
wtw.mmooootteee).innnooe. indirect
day
antithrombin, Quantification of Treponema pallidum amtibodie..HBsAg.HCCV antibody www.aattiiiinninteeeeeeeeee brain natriuretic peptide precussor(TTTpooNP).FFiy degradation products, antibody quantification,ESR
every 2
days
PH value
every 3
days
Interleukin receptor, receptor, Interleukin Intereerlii 8, T,Tum necrosis factorra,
every 5
I.t.enterleukin 6
days
2019-nCoV nucleic acid detection
every 7
days
ferritin
every 8
days
Table 5: COVID dataset feature frequencies
15

---

Under review as a conference paper at ICLR 2023
Model
Hyper parameter
Value
LSTM cell size
19
Initial Mask Weight values
0.3
Mask norm weight
0.1
LSTM
Number of Layers
Learning rate
0.001
Batch size
16
patience
25
Number of Components
Reset Model
False
CNN Kernel size
7
Initial Mask Weight values
0.3
Mask norm weight
0.1
CNN-Attn
Number of Layers
Learning rate
0.001
Batch size
16
patience
25
Number of Components
Reset Model
False
CNN Kernel size
7
Initial Mask Weight values
0.3
Mask norm weight
0.1
CNN-LSTM
Number of Layers
Learning rate
0.001
Batch size
16
patience
25
Number of Components
Reset Model
False
Hidden Embedding Size
13
Initial Mask Weight values
0.3
Mask norm weight
0.1
Transformer
Number of Layers
Number of Heads
3
Learning rate
0.001
Batch size
16
patience
25
Reset Model
False
Table 6: The selected MultiWave hyperparameters for COVID dataset
16


Reviews:
[{"review_id": "zawG-0uyJeK", "reviewer": "Reviewer_6oFs", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a framework, named MultiWave, which augments any deep learning time series model with components operating at the intrinsic frequencies of the signals. It applies discrete wavelet decomposition on each signal to obtain subsignals of different frequencies and groups all subsignals in the same frequency band together to train a component. Besides, it removes the irrelevant frequency components for the given predictive task. Experiments have been evaluated on three deep learning framework and three time series data sets.", "summary_of_the_review": "This work proposed MultiWave, which augments any deep learning time series model with components operating at the intrinsic frequencies of the signals. It applies discrete wavelet decomposition on each signal to obtain subsignals of different frequencies and groups all subsignals in the same frequency band together to train a component. Besides, it removes the irrelevant frequency components for the given predictive task. However, it is not novel enough to apply to analyse time-series data sets. The contributions of this work should be emphasized.", "strength_and_weaknesses": "Strength. This paper proposes a framework that applies the wavelet transform to obtain different frequency time series signals and removes the irrelevant frequency for the given predictive task. Experiments have been evaluated on benchmark \ndeep learning framework and the benchmark data sets, which validates the effectiveness of the proposed methods.\n\nWeaknesses.\n\tApplying the wavelet transform to analyse the time-series data set is not novel enough. \n\tThe contributions should be emphasized. \n\tBesides, the novelty and the differences of this work should be stated in the text.\n\tSome description in the text should be improved. I.e. : and more recently transformers\n\tWhy there is something missing about the features about the 1/4-2/1 days frequency components in the  Table 2.  \n\tThere are some errors about the punctuation and format should be corrected.\n I.e. ：Multirate time series classification;Frequency analysis of time series ...-->add the “:” or “.” behind the sentences.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ipUy5YKpOZ", "reviewer": "Reviewer_BvMP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a new framework MultiWave for multivariate time series forecasting using Wavelet decomposition. Key contributions of the proposed MultiWave is that it can handle input signals with different frequencies and select most informative frequencies after the Wavelet decomposition using feature masks. Experimental results show efficacy of the proposed method on boosting existing time series forecasting methods.", "summary_of_the_review": "1. For Algorithm 1, my understanding is that learning of the feature mask M is done in sequential iterations during the supervised learning of the forecasting task. All components with zero masks will be removed after the training. Please confirm the process above.\n2. What happens if the training set contains samples with non-zero Wavelet decomposition coefficients at all levels? Feature mask will not kick in in this case and the proposed method will degenerate to feature extractor using the Wavelet decomposition.\n3. Instead of oversampling signals with lower sampling rates, how about just concatenate all signals/features after the Wavelet decomposition and use them for the subsequent learning task? Another option is to pad signal with the low sampling rates with all zeros.\n4. Experiments look a little bit weak. Can the proposed framework improve other state-of-the-art algorithms for time series forecasting? For example, deepar, N-beats, informer and state-space based forecasting models, to name a few.\n5. One minor issue: in page 3, sec. 3.1, is it 30 HZ for signal x2?\n", "strength_and_weaknesses": "Strength:\nUsing feature mask to learn the best Wavelet decomposition coefficients is interesting.\n\nWeaknesses\nMy concern is on the novelty of the proposed method. The framework is close to the work as the authors referred: “Multilevel wavelet decomposition network for interpretable time series analysis”. Also experiment section could be improved.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VCkH4i1dGBr", "reviewer": "Reviewer_1QUg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper \"MultiWave: Multiresolution Deep Architectures through Wavelet Decomposition for Multivariate Timeseries Forecasting and Prediction\" combines wavelet decomposition with learnable time-series models. In addition to performing wavelet analysis to split the data into frequency components, a masking approach is proposed to select a sparse set of frequencies. The framework is shown to perform better on synthetic data, as well as two real-world datasets (WESAD & Covid19). \n", "summary_of_the_review": "The paper is clearly written and can be nicely followed. However, the paper needs more work to carve out evidence for the fundamental design decisions (e.g. the masking method).", "strength_and_weaknesses": "Strengths.\n- the idea of the paper is clear and the paper can be nicely followed.\n- frequency decomposition is a standard tool in signal processing, and hence combination with deep learning tools is promising\n- learning the masking to prune uninformative frequencies is very promising\n\nWeaknesses.\n- Some parameter investigations that would provide insight into the method are missing: what is the impact of the mask sparsification parameter alpha on the performance? Is alpha=0 maybe the best case? Or where is the optimum? Also: The model likely performs best if the input signals are sampled at integer multiples - does performance suffer when this is not the case? By how much?\n- Table 1 shows that the results are better, but only marginally so, hence I'd expect moderate impact (e.g. in the first row, I'd argue that it might not even be significant due to overlapping error-bars - if this is true, please bold all methods in these cases).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IoFC_XeeCMR", "reviewer": "Reviewer_jcQW", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper \"MULTIWAVE: MULTIRESOLUTION DEEP ARCHITECTURES THROUGH WAVELET DECOMPOSITION FOR MULTIVARIATE TIME SERIES FORECASTING AND PREDICTION\" proposes to fuse time series data with different sampling rates via concatenation of transformed features at matching scales in the wavelet domain.\nAfter motivating and explaining the approach the paper proceeds to test their approach on the \"Wearable Stress and Affect Detection\" dataset and a COVID-19 mortality prediction problem. \n\n", "summary_of_the_review": "This paper is not properly embedded into the existing literature in its current form. Unfortunately, I am no expert in the field of medical time series processing, so I can't suggest the leading papers in the field for comparison. However, I am confident in my assessment, that no comparison to existing work is not enough.", "strength_and_weaknesses": "Strengths:\n- Fusing signals measured at different sampling rates is an important part of every time-series processing pipeline and therefore important.\n- Using the wavelet transform for fusing purposes is an interesting approach.\n\nWeaknesses:\n- The experimental part makes no comparisons to existing work in related papers.\n- Time-series processing is a long-established field.\n   - See: i.e. https://arxiv.org/pdf/2010.12493.pdf for a review which also studied the covid-19 data set.\n   - Or https://www.mdpi.com/2227-7390/8/12/2125 for a paper which is also using wavelets to process medical time-series data.\n- Comparing the numbers from this study to the results from other papers is an important part of the process of assessing if this work is suitable for ICLR. I am not claiming that comparison to precisely the two papers above is necessary, but a future version of this draft should include leading papers in the field and compare to them.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject