Conditional Generation of Periodic Signals with Fourier-Based Decoder
Abstract
Periodic signals play an important role in daily lives.
Although conventional sequential models have shown remarkable success in various ﬁelds, they still come short in modeling periodicity; they either collapse, diverge or ignore details.
In this paper, we introduce a novel framework inspired by Fourier series to generate periodic signals.
We ﬁrst decompose the given signals into multiple sines and cosines and then conditionally generate periodic signals with the output components.
We have shown our model efﬁcacy on three tasks: reconstruction, imputation and conditional generation.
Our model outperforms baselines in all tasks and shows more stable and reﬁned results.
Introduction
Periodic signals exist in daily lives.
In biomedical domain, electrocardiogram (ECG) [18] and body temperature [16, 25] are critical periodic signals in examining patients health status.
ECG, in particular, is an important measure to diagnose patient heart diseases [14] such as Myocardial
Infarction, AV Block and Ventricular Tachycardia.
Despite its signiﬁcance, most publicly opensourced ECG datasets [20, 2] are small in size and they all tend to have a severe data imbalance problem [27]–normal or common disease ECG records make up the majority while rare diseases barely exist.
Therefore, generating ECG records conditioned on diagnosis can be beneﬁcial in solving data imbalance issue and further developing ECG deep learning models.
Sequential models, such as recurrent neural networks (RNNs) [15, 6], transformer decoder [31], neural ordinary differential equations (NODEs) [4] and neural processes (NPs) [12, 13, 17] have shown excellent results in various ﬁelds [7, 3, 22].
However, as shown in section 4, there is room for improvement when it comes to generating periodic signals.
These models either collapse, diverge or ignore subtle periodic details.
In theory, discrete Fourier series (DFS) or discrete Fourier transform (DFT) can handle sampled periodic signals.
In practice, however, DFS and DFT perform poorly when signals are obtained in irregular timesteps or contain noise [28, 33], and are difﬁcult to serve as (conditional) generative models.
In this paper, we introduce a novel architecture that directly utilizes Fourier series for generating periodic signals.
By the deﬁnition of Fourier series, periodic signals are composed of sinusoids.
Built upon conditional variational autoencoder [29], we ﬁrst obtain latent representation vector of the signal with the encoder.
The decoder then outputs Fourier coefﬁcients, which represent how much each sinusoid contributes to compose the original signal, using the sampled latent vector.
We further
0 𝑡ℎ 𝑒𝑚𝑏𝑒𝑑𝑑𝑖𝑛𝑔 (𝐵 𝜓0 ) 1 𝑠𝑡 𝑒𝑚𝑏𝑒𝑑𝑑𝑖𝑛𝑔 (𝐵 𝜓1 ) 2 𝑛𝑑 𝑒𝑚𝑏𝑒𝑑𝑑𝑖𝑛𝑔 (𝐵 𝜓2 ) 𝑛 𝑡ℎ 𝑒𝑚𝑏𝑒𝑑𝑑𝑖𝑛𝑔 (𝐵 𝜓 𝑛 ) ...
Basis Embedding 𝐵 𝜓 . . .
𝐵 𝜓 𝐵 𝜓𝐵 𝜓𝑛
Decoder 𝐷 𝜙 𝑎𝑏
Decoder 𝐷 𝜙 𝑎𝑏
Decoder 𝐷 𝜙 𝑎 𝑛 𝑏 𝑛 parametersharing . . .
+ + + ෍ 𝑘= 𝑛 𝑎 𝑘 cos 2𝜋𝑘𝑡 + 𝑏 𝑘 sin(2𝜋𝑘𝑡) . . .
+𝒄
Encoder 𝐸 𝜃 𝑡 1 𝑥 𝑡𝑡 2 𝑥 𝑡𝑡 𝐿 𝑥 𝑡 𝐿 ... 𝒄 𝒄 𝒄
Input 𝝁 𝒛 𝝈 𝒛 𝒛 𝑥 𝑡 1 ∗ 𝑥 𝑡 2 ∗ 𝑥 𝑡 𝐿 ∗ ... 𝒛 ∗
We sample z from N ( µ z , σ z ) , and concatenate with label c , producing z ∗ .
Then z ∗ is added with the basis embeddings B kψ and passed through the decoder D φ , which produces a k and b k , namely the k-th approximate Fourier coefﬁcients.
With a set of Fourier coefﬁcients A = { a 0 , . . .
, a n }
and
B = { b 0 , . . .
, b n } , we build an approximated Fourier series.
generate the label conditioned periodic signals by adding the class information to the sampled latent vector.
We evaluate our model on three tasks: reconstruction, imputation, and conditional generation to demonstrate its usefulness.
Our model outperforms baseline models in all tasks, suggesting a new direction for modeling periodic data.
Related Work Modeling Periodic Signals
Previous research that tried to model periodic signals replaces nonlinear activation functions by sinusoids such as sin ( x ) , cos ( x ) or the linear combination of the two [28, 34, 21, 35].
However, due to the activations’ non-monotonicity , the periodic activation functions induce numerous local minima, causing troubles in model optimization [21].
Fourier Neural Networks (FNNs) FNNs are neural networks that resemble Fourier series.
Gallant and White [11] suggest cosine squasher as an activation function.
Silvescu [28] use cos ( x ) as an activation function to mimic Fourier series.
Liu [19] proposes a combination of cos ( x ) and sin ( x ) as activation function, and showed comparable results on various datasets empirically.
FNN had a superior performance in practical tasks such as aircraft engine fault diagnostics [30], and control of a class of uncertain nonlinear systems [36–38].
Conditional Time-Series Generation
Previous conditional time-series generative work employ convolutional neural networks (CNNs) [1, 5] and RNNs [10].
They often make use of generative adversarial networks (GANs) to achieve a realistic-looking data [23].
Esteban et al.
[10] achieve conditional generation by setting RNNs in both generator and discriminator and concatenating a class label for every time steps.
Proposed Methods 3.1 Problem Deﬁnition
In this section, we will clarify notations to be used throughout the paper and deﬁne our task.
Let
T = { t 1 , t 2 , . . .
, t L } be a set of L ordered timesteps with t i ∈ R , and X T
= { x t 1 , x t 2 , . .
, x t L } be a corresponding input sequence, where x t i ∈ R .
We note that the intervals among T can be irregular.
Additionally, the input sequence X T has its own one-hot vector label c ∈ R n c , where n c is the number of unique labels.
Given a label c , the goal of conditional generation is to generate X S with another ordered timesteps S = { s 1 , s 2 , . . .
, s M } that maximizes P ( X S | c ) .
3.2 Background: Fourier Series
Fourier series approximates any periodic function f ( t ) as an inﬁnite sum of sines and cosines with increasing frequency as written in eq.
Here, P is the period of function f ( t ) .
[Equation]
A k ∈ R , B k ∈ R are Fourier coefﬁcients .
They represent how much each sine and cosine is contributing to formulate the given function.
Hereafter, we will name the set of sines and cosines sharing the same frequency as basis and we will assume all signals have a period of 1.
Originally,
Fourier coefﬁcients are acquired by integrating the original function f ( t ) with corresponding sines or cosines.
However, this calculation is infeasible when the data points are discrete and irregular.
For the most of the real signals, ﬁnite numbers of basis are sufﬁcient to model a function f ( t ) .
As such, we will predeﬁne the number of basis, denoted as n .
In our model, given an input sequence X T , we approximate the true value of A i and B i .
We denote the approximations as A = { a 0 , . . .
, a n } and B = { b 1 , . . .
b n } : a i and b i approximate A i and B i , respectively.
3.3 Model Architecture
Our model is built upon conditional variational autoencoder [29].
The overall architecture is illustrated in ﬁg. 1.
We construct the input sequence X ∗T =
[ x ∗ t 1 , x ∗ t 2 , . . .
, x ∗ t L ] where x ∗ t i is a concatenation of x t i ∈ X T , t i ∈ T , and label c as follows:
[Equation]
Encoder
E θ produces µ z ∈ R z and σ z ∈ R z from the input sequence X ∗T and we sample the latent variable z ∈ R z from q ( z |X ∗T ) :
= N ( µ z , diag ( σ z )) .
E θ can be any type of model ( e.g., 1-D
CNNs, RNNs, transformer encoder) capable of producing an informative representation of the input sequence.
Decoder z ∈ R z is sampled from q ( z |X ∗T ) and concatenated with c ∈ R n c , producing z ∗ ∈ R z + n c as z ∗ =
[ z ; c ] .
We have a basis embedding lookup table denoted as B ψ ∈ R n × ( z + n c ) , where n is the predeﬁned number of bases.
The k-th row of B ψ , B kψ , represents the embedding vector for the k-th frequency basis.
In order to obtain the k-th Fourier coefﬁcient, a k and b k , we add B kψ to the latent variable z ∗ .
Note that this addition is done in parallel, so we can compute it efﬁciently even when n is large.
The decoder D φ produces a k and b k as follows:
[Equation]
With A = { a 0 , . . .
, a n } and B = { b 1 , . . .
b n } , we build an approximated Fourier series in eq.
[Equation]
Finally, we construct a sequence of predictions as (cid:98) X T =
[ˆ x t 1 , ˆ x t 2 , . .
, ˆ x t L ]
where ˆ x t i =
ˆ f ( t i ) .
We optimize our model by the summation of reconstruction loss between the true observations X T and the predicted observations (cid:98) X T , and the Kullback–Leibler divergence loss ( D KL ) between the posterior distribution q ( z |X ∗T ) and the prior distribution p ( z ) =
N ( 0 , I ) .
The overall loss is:
[Equation]
where β > 0 is a hyperparameter.
After training, to generate X S conditioned on label c , we sample z from the prior distribution p ( z ) and concatenate with c .
Then we can generate X S by decoding the concatenated vector.
Currently, our model assumes all input signals have the period of 1.
Extension of this model to dynamically deal with signals of varying periods is our primary future research interest.
Experiments 4.1 Experimental Setup
We conduct experiments with two periodic datasets: toy sinusoid dataset and Physionet2021 [26].
Toy dataset is a simple mixture of three sine and cosine functions: (cid:80)
i =1 m 2 i − 1 cos (2 πd 2 i − 1 t ) + m 2 i sin (2 πd 2 i ) .
We put four class conditions for the toy dataset based on amplitudes M = { m 1 , . . .
, m 6 } and frequencies D
= { d 1 , . . .
, d 6 } , resulting in four amplitude-frequency class labels: ‘Low Amp. & Low Freq.’, ‘Low Amp. & High Freq.’ , ‘High Amp. & Low Freq.’, and ‘High
Amp. & High Freq.’.
‘Low Amp.’ classes sample m from a uniform distribution U (1 , 4) whereas ‘High Amp.’ classes sample m from U (6 , 9) .
‘Low Freq.’ classes sample d from U (1 , 4) , whereas ‘High Freq.’ classes sample d from U (8 , 11) .
Each signal has a total of 500 timesteps.
Physionet2021 [26] contains 12-lead ECG recordings collected from six separate datasets.
We cropped each record into one second consisting of 500 timesteps and extracted samples with three diagnoses, namely ‘Right Bundle Branch Block’ (RBBB), ‘Left Bundle Branch Block’ (LBBB), and ‘Atrial
Fibrillation’ (AF) from the V6 lead.
These diagnoses are selected because they can be examined within one second record [9].
After preprocessing, there were total 36,110 cropped ECG samples.
We split the data into train, validation and test sets with the ratio of 8:1:1.
More details on preprocessing are in appendix A.
We sampled 20% of the timesteps during training to insure the irregularity of time.
For all experiments, we employ 5-layer 1D CNN as an encoder E θ and 6-layer MLP as a decoder D φ .
We evaluate our model on three tasks: (1) reconstruction (for the sampled 20%), (2) imputation for missing timesteps (for the non-sampled 80%) and (3) conditional generation.
For the tasks (1) and (2), we use sampled time points and make the model perform both reconstruction and imputation in parallel by generating (cid:98) X T for all timesteps.
For the task (3), we sample z from the prior distribution N ( 0 , I ) and pass through the decoder D φ .
With the same 5-layer 1D CNN encoder, we compare our model with four different baseline decoders: Gated Recurrent Units (GRU), Transformer decoder, Neural
ODE (NODE) and Neural Processes (NP).
Further model implementation details are explained in appendix B.
4.2 Experiment Results of Toy Dataset
[Table]
We report reconstruction and imputation results in table 1.
Our model shows the lowest MSE on both reconstruction and imputation compared to other baseline models.
We illustrate the reconstruction results in ﬁg. 5 in the appendix C.
Based on ﬁg. 5, we found both GRU and Transformer to perform very poorly, though GRU was able to capture partial periodicity.
NODE and
NP showed better performance on low frequency samples, but they were not able to reproduce subtle details for high frequency samples.
In contrast, our model showed superior performance across all amplitude-frequency classes.
We conditionally generate 2,000 samples from the sampled z for each amplitude-frequency class, and for each decoder.
As visualized in ﬁg. 2, NODE produced ﬂattened samples and NP generated non-periodic signals with a large amplitude regardless of the class conditions.
GRU and Transformer were able to produce periodic signals, but all 2,000 samples were nearly identical with minimal sample diversity.
In contrast, our model was able to generate diverse periodic signals.
In order to verify whether the generated samples correctly reﬂect the class conditions, we conduct
Fourier series analysis, which decomposes a given signal into multiple sines and cosines as expressed in eq.
From the analysis, we can calculate which frequency is used to compose the signal and its corresponding coefﬁcients ( i.e. amplitude).
We plot histograms on both amplitude and frequency for the toy dataset, and compare the baselines with our model.
The results are shown in ﬁg. 3 for ‘Low Amplitude & High Frequency’ and ‘High Amplitude & High Frequency’, the rest two class conditions are described in ﬁg. 6 in the appendix C. In all baseline models, their amplitude and frequency histograms are similarly shaped across the two classes, which implies that those models fail to reﬂect the class conditions when generating samples. Our model overlaps with the dataset [Figure]
distribution most precisely compared to the other baselines, suggesting that the periodic signals generated by our model are properly conditioned on the class.
4.3 Experiment Results of Electrocardiogram
[Table]
We report reconstruction and imputation results in table 2.
Our model shows the lowest MSE in both reconstruction and imputation.
The results are visualized by ﬁg. 7 in appendix C. Based on ﬁg. 7, GRU and Transformer cannot capture the peak of given ECG records.
Although NODE and NP could grasp the peak point, they disregard details such as subtle waves in the isoelectric line, the straight line on the ECG.
Our model can both capture the peak and the subtle ﬂuctuations clearly.
We conditionally generate 3,000 samples for each diagnosis and decoder.
As visualized in ﬁg. 4, our model generates samples that are highly similar to the real dataset.
According to Rawshani [24],
RBBB diagnostic characteristics include a deep and broad depression after the peak while LBBB has a wide peak and a shallow depression after the peak.
AF has a f-wave, a ﬁbrillatory wave in the isoelectric line, and does not have a P-wave, a little uprising before the peak.
As shown in ﬁg. 4, our model captures the necessary characteristics of each diagnosis, and in the ﬁgure, its signiﬁcance is highlighted in red.
GRU generates similar samples regardless of the given [Figure] diagnosis, whereas Transformer draws ﬂat lines after a certain time point.
NODE and NP tend to generate smooth ECG signals ignoring all ﬂuctuations in ECG.
Also, they could not synthesize necessary characteristics of the given diagnosis.
We quantitatively evaluate the generated samples by using a pre-trained ECG classiﬁer.
The classiﬁer is trained beforehand on the real dataset to classify the three diagnosis.
We use total of ﬁve classiﬁers trained with a different parameter initialization seed.
We run the classiﬁer on our generated samples in order to conﬁrm whether our samples are classiﬁed to their given diagnosis.
We report our results in table 3.
Our model outperforms all other baselines for diagnosis-averaged overall scores, showing notable performance in AF.
We speculate the reason for this improved performance is that our model is the only model that can synthesize f-wave, a main feature of AF, while other models fail to generate such ﬁne oscillations.
Conclusion
In this work, we introduce a new Fourier-based model architecture that can generate periodic signals.
Conventional sequential models collapse, diverge, or ignore subtle details even if they modeled the periodicity successfully.
In contrast, our model outperforms all baseline models in all tasks across [Figure]
Atrial [Figure] [Table] NP 0.
0.725 (0.004) (0.004) (0.003) (0.004) (0.015) (0.009) (0.007) (0.013) (0.008) (0.005) (0.011) (0.010) (0.004) Fourier (Ours) 0.
Real Dataset 0.
[Table]
two datasets, showing stable and reﬁned results.
We plan to extend out work to more diverse signals with various periods, since we only examined the samples with a period of 1.
We believe the periodic structure inherited by the Fourier series will enable new possibilities to model various periodic signals even beyond bio-signals in the future.
A Dataset A.1 Toy Dataset
The number of samples in each class is 20,000 in train set and 5,000 for validation and test sets.
We have total 80,000 samples in train set and 20,000 for validation and test sets.
Each sample lasts for three seconds and has total of 500 time steps.
We added Gaussian noise sampled from
N ( 0 , diag (0 . 3)) in all samples.
A.2 Physionet
We ﬁltered out those which sampling rate is not 500Hz and those that last less than 10 seconds, or longer than 20 seconds.
We utilized data between 5 to 10 seconds because the majority of samples include extreme noise up to 5 second and stabilize afterward.
With using ECG R-peak detectors[8], we maintain samples that have one or two detected R peaks within one second.
This process ﬁlters out extremely noisy samples, such as samples with all zero values or samples that do not show any dominant QRS peaks but rather a random noise.
The total number of diagnosis label in ﬁnal preprocessed dataset is shown in Table 4.
[Table]
B Experiment Details B.1 Model Architecture
[Table]
Conv(# of output channels=256, Kernel=3, Stride=1), MaxPool(2), SiLU
Layer
Conv(# of output channels=256, Kernel=3, Stride=1), MaxPool(2), SiLU
Layer
Conv(# of output channels=256, Kernel=3, Stride=1), MaxPool(2), SiLU
Layer
Conv(# of output channels=256, Kernel=3, Stride=1), MaxPool(2), SiLU
Layer
Conv(# of output channels=128, Kernel=3, Stride=1)
[Table]
[Table]
[Table]
B.2 Hyperparameters
We share a ﬁxed hyperparameter set for Toy Datset and Physionet2021 as follows.
• dropout = 0.
• lr = 1e- • batch size =• β
= 1 for Toy Dataset, β = 30 for Physionet
B.3 Experiment environments
We train our models on NVIDIA GeForce RTX 3090.
Also, CUDA version is 11.1 and torch version is 1.8.1.
C Additional Experiment Results Low Amplitude Low Frequency
Low Amplitude High Frequency High Amplitude Low Frequency
High Amplitude High Frequency
[Figure]
[Figure] [Figure]
D Experiment Result without Sampling
This section describes the results from the experiments without sampling.
That is, the model receives all available input sequences.
D.1 Experiment Results of Toy Dataset
Generated samples on each amplitude-frequency classes are visualized in ﬁg. 8.
The overall characteristics are similar to ﬁg. 2 except that GRU can model more diverse samples.
we report histograms on amplitude and frequency based on Fourier series analysis in ﬁg. 9.
The overall distributions are alike to ﬁg. 3 but GRU covers more wide space in frequency
[Figure]
D.2 Experiment Results of Electrocardiogram
[Figure]
NP 0.
Fourier(Ours) 0.
RealDataset 0.
[Table]
Generated ECG samples from each model are visualized in ﬁg. 10.
GRU produces the same samples across all diagnoses.
We report the mean and standard deviation of classiﬁer performance on generated
ECG samples in table 7.
Similar to table 3, our model shows decent performance in diagnosis-averaged overall score and impressive performance in AF.
Since GRU generates the same samples regardless of diagnosis, recall in RBBB is 1 and all scores in other diagnoses scores are 0.
Likewise, Transformer synthesize AF samples as RBBB, causing recall in RBBB 1 and scores in AF nearly 0.
[Figure] [Figure]