Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN)
Authors: Ali Seyfi
Jean-Francois Rajotte
Raymond T. Ng
Venue: NeurIPS 2022 Accept (2022)
Keywords: Synthetic Data
Multivariate Time Series
Generative Adversarial Networks
Generative Modelling
Data Augmentation

Abstract: Generating multivariate time series is a promising approach for sharing sensitive data in many medical, financial, and IoT applications. A common type of multivariate time series originates from a single source such as the biometric measurements from a medical patient. This leads to complex dynamical patterns between individual time series that are hard to learn by typical generation models such as GANs. There is valuable information in those patterns that machine learning models can use to better classify, predict or perform other downstream tasks. We propose a novel framework that takes time series’ common origin into account and favors channel/feature relationships preservation. The two key points of our method are: 1) the individual time series are generated from a common point in latent space and 2) a central discriminator favors the preservation of inter-channel/feature dynamics. We demonstrate empirically that our method helps preserve channel/feature correlations and that our synthetic data performs very well in downstream tasks with medical and financial data.

Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN)
Ali Seyfi Department of Computer Science University of British Columbia Vancouver, BC aliseyfi@cs. ubc.
Jean-Francois Rajotte
Data Science Institute University of British Columbia Vancouver, BC Afrajomail ubc.
Raymond T. Ng
Department of Computer Science University of British Columbia Vancouver, BC rngScs ubc.ca
Abstract
Generating multivariate time series a promising approach for sharing sensitive data in many medical, financial, and IoT applications. A common type of multivari- ate time series originates from a single source such as the biometric measurements from medical patient. This leads complex dynamical patterns between individ- ual time series that are hard to learn by typical generation models such as GANs. There is valuable information in those patterns that machine learning models use to better classify, predict or perform other downstream tasks. We propose a novel framework that takes time series' common origin into account and favors channel/feature relationships preservation. The two key points of our method are: 1) the individual time series are generated from a common point in latent space and 2) a central discriminator favors the preservation of inter-channelfieature dynamics. We demonstrate empirically that our method helps preserve channel/feature corre- lations and that our synthetic data performs very well in downstream tasks with medical and financial data.
Introduction
Multivariate Time Series (MTS) are composed of individual time series (TS) sharing the same time reference. In some cases, the individual time series further share common source such as the biometric values from a medical patient, the stock prices from economic events or geographically separated seismic measurements from single earthquake. This leads to specific correlation patterns and time dynamics across the time series. Such complex patterns can crucial when model is trained on MTS and might need huge amount of training samples to be captured by machine learning algorithm. For many applications, however, there may not be enough high-quality training data. For example, for many biomedical and health care applications, data scarcity is common and data sharing to build a larger training set is challenging due to regulatory requirements or ethical concerns (1I, (2)). Such concerns are justified; it is well known that sharing data associated with single individual, even anonymized, can lead to unexpected privacy breaches (B). [4, [5] and [6].. Synthetic MTS could be attractive alternative to share the patterns and statistical information of an MTS dataset. done properly, synthetic MTS should not have a one-to-one mapping to the original data, although it can come with its own privacy and quality challenges (see [ for example).
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Beyond data sharing, synthetic MTS can be used for augmenting a dataset to improve the performance of a trained model [8] or increase the contribution of underrepresented sub-populations D For example, in the health domain, a patient can generate multiple TS from biometric measurements, wearable and IoT sensors, but the collection of such data may not have representative coverage of full population. Synthetic MTS can help to augment datasets for improving downstream analysis, such as for forecasting and classification tasks. In this work, we are interested in generating synthetic MTS that both preserve utility and statistical properties of the original data. By utility we mean the ability to support a specific downstream task such as the performance of a classifier when trained on synthetic data. Preserving statistical properties such channel or feature correlations increase the potential benefits to an unforeseen downstream task, exploratory data analysis or educational purposes when sharing data is not possible. We observe that current methods often struggle at correlation preservation needed for downstream task and our method addresses these limitations.
In this paper, we propose novel method for generating MTS that comes from a common source by defining an architecture that explicitly takes inter-channel correlations into account. While MTS can be created with typical generative deep learning architecture (such as VAE or GAN) with multiple chamnels output, we will demonstrate that our method not only preserves the quality each individual TS by generating each individual TS separately from a common noise but also preserves the relationships between TS by having a central discriminator receiving all the generated individual TS single input. We consider noise to be a point of random sampling in the latent space that represents a patient's 's "whole biological environment." " By common noise, we refer to a common point in the latent space, i.c., that latent space is the patient space, and we sample patient by sampling noise for the generator input.
We perform extensive empirical evaluation on MTS datasets where individual TS originate from single source. We evaluate the resulting synthetic MTS by comparing their statistical features with the real MTS and their utility on downstream classification task. We pick classification to measure the utility of the synthetic data because classification one of the most popular analytic tasks in machine learning. We also compare the real and synthetic MTS visually in embedding spaces, and evaluate our method against state-of saate-of-the-ar bascline methods. Our contributions can be summarized as follows:
To our knowledge, this is the first study to analyse how to generate multivariate time series with individual channel generation originating from common noise while inter-channel correlations preservation is forced with a central discriminator.
Demonstrating that COSCI-GAN compares favourably with state-of ff-thhean art algorithms in downstream tasks on an Electroencephalography (EEG) eye state time series dataset.
Demonstrating that COSCI-GAN results compares favourably with state-of-the-art algo- rithms in preservation statistical properties of on the EEG dataset.
Open sourcing the implementation of our methods and experiments
The rest of the paper is arranged as follows: he problem deseription. Section presents our COSCI-GAN model architecture and gives implemen- tation details. Section 5 presents extensive empirical evaluation. Section 6 discusses the benefits, limitations and potential negative societal impacts.
Section 2 discusses related work. Section 3 formalizes
2 Related Work
Synthetic data is often proposed to provide or to increase privacy protection [10]. The only privacy protection framework with predictable effect on privacy is to incorporate differential privacy (DP) into the learning algorithm ((II) by adding calíbrated noise the parameter updates. For every DP generation method, the privacy must be assessed empirically with privacy attacks ((1))
'Note that we use channel and feature interchangeably here since the term channel is commonly used for EEG and other medical time series that partly motivate this work. From typical machine learning context, a channel is basically feature. Generally in this paper, feature and channel corresponds to a collection of recorded data point from given sensor.
https:
/ / thub. www.coom/aaaaaeeaa commaalisey175//0000000

---

Generative Adversarial Networks [3] are popular method to generate synthetic data. Since their inception, they have expanded include time-series data production, see [4] for comprehensive overview. Mogren introduced the C-RNN-GAN approach, which employs RNNs as both the generator and discriminator order synthesise time series from a random vector [5 Esteban al. later proposed RCGAN to create medical data using a similar architecture [16]. These frameworks have been applied to a wide range of application domains, including biosignals [7]. finance [8]. sensor [1). text [20], and smart grid data (21). However, the typical framework and Ioss function of GANs are insufficient for the production multivariate time series, especially if we want to preserve the will demonstrate below. Xu al. developed COT-GAN based on ideas of optimal transport theory [22] and more recently, Li et al. developed TTS-GAN which is capable of generating realistic synthetic time series of any length 23 both work focusing solely statistical evaluation of the data such as correlations.
correlation among the channels
Yoon et al. developped TimeGANS based recurrent conditional GAN for capturing the temporal dynamies of data throughout time [24]. It entails training supervised and unsupervised targets concurrently using learnt embedding space, It creates time series data by learning embedding space and optimising it via binary adversarial feedback and stepwise supervised loss. Most recently, Fourier Flows 2 was proposed method based Fourier transform layer followed by a chain of spectral filters leading to an exact likelihood optimization.
3 Problem Formulation
Let X be an MTS dataset of N instances, each composed of C channels X,, where i E I,...,C. C. An instance of X can be described as x. {<=":-... :.:1:...;22)) ....,,)); We want find distribution Xc). the real distribution of our dataset, as possible.
((),...., ..) that is as close to In the typical GAN framework, it may be difficult to find the best optimization solution for such a complex goal, which depends on the number of channels, duration, and distribution of the data, This is why we use separate generators G, to learn the marginal distribution of each channel, p p(..). separately, and then use a central discriminator to force preserving the real correlation between the channels by focusing on the conditional distributions pl X, (iit)) where Xizt refers all the channels excluding channel i. Figure [ depicts this architecture, with more details discussed in the next section. Essentially, we have two objectives: local and central one.
Local objective: In the local objective, the goal is to estimate the marginal distribution of each channel, p(X,): which means for each channel we should optimize:
min D(p((x))la(X.))
where D is any suitable measure of the distance between distributions.
Central objective: In the central objective, the goal is to estimate the conditional distribution of a channel given all the other ones, p(X,) XX,x)))). In order illustrate this objective, demonstrate in special case where the channels are independent of one another, which means we will have:
C
min D I
P p
All II (X.1)))))))
Our approach is that each generator G; share the same initial (noise) source z, such that Equation becomes
C
min D
p X, II
(x.,=))
1=1
We define the global loss to be linear combination of the loss of local objective and loss of central objective.
3

---

Channel GAN
Generator
Discriminator
Noise
Central Discriminator
Real Dataset
Generator
Discriminator
Channel GAN C
Figure I: Overall structure of COSCI-GAN, Each Channel GAN is dedicated to one channel/feature with input either real one fake one with their discriminator being binary classifier. The central discriminator, a binary classifier, receives full instances (i.c. all channel) either all real or all fake.
COSCI-GAN
As shown in Figure ! COSCI-GAN made up of two main parts: 1) Channel GANs, which contain dedicated to a single channel (univariate TS), and 2) the Central
pairs of manerator-discriiing Discrininator, dedicated to all channels at once. Each of these parts is responsible for a specific task. In channel GANs, the generators responsible for producing realistic 'S and the discriminators responsible for distinguishing between real and synthetic TS. The central discriminator is responsible for enforcing that all the generated TS of given instance have the same correlation those from real MTS. Producing all channels simultancously necessitates that the generative model learn the joint distribution of all TSs, a difficult task that requires a substantial amount of data and time. In contrast, learning the marginal distribution of a single channel is a significantly simpler task. Consequently, the primary purpose of employing channel GANs opposed a single multichannel pair is to assist each individual TS generator in synthesizing more accurate TSs from its own channel's data distribution. By including the central discriminator, we aim to enforce realistic correlations between the channels as much as possible.
4.1 Algorithm
Let multivariate time series have dimensions of N I. C, where N is the number of instances in the dataset, L is the length of each time series, and C is the number of channels. As shown in Figure there
pairs of Generator-Dite Diterator-Dicrriiiattorr shared noise vector to begin the generation process. Each generator in a channel GAN will synthesize TS, and both the generated TS and the corresponding channel of real TS will be passed their paired discriminator, which determines whether the generated TS is from the same distribution as the real ones. A pseudo-code the COSCI-GAN algorithm is provided in Algorithm
channel GANs. All generators are fed
Algorithm 1 COSCI-GAN
for epoch in epochs do for batch in training set do Create a noise vector, Z.
for i 1 to Rehanneis do Extract signal, from the batch. Generate Fake signals generated, from Generator,
Train Discriminator, by feeding signal, and generated,. end for
Train Central Discriminator by feeding ((generated),... genere.....eeerredddm...m.
mohamee)s) )
for i to Nehannets
Train Generalor, with Lossp, and LossCn end for
end for end for

---

As mentioned earlier, the central discriminator's ' role is to preserve the inter-channel correlations. The TS synthesized by all channel generators will be concatenated as MTS and fed to the central discriminator, which aims to determine whether the MTS is real or fake. We hypothesize that this will penalize unrealistic (un)correlation patterns hetween channels, and we have provided some evidences in the Supplementary Materials that other State-Of Oftte-Of-theArt (SOTA) methods often exaggerate even create unrealistic correlations when compared to real data correlation.
4.2 Training
During training, the discriminators in the channel GANs (which we will refer to as channel Dis- criminators from now on), their paired generators, and the Central Discriminator will engage in three-player game. The saree-player objective of a given channel GAN combined with the central discriminator is:
min max max V(Gi,,, Di, CDa) = -arraasn [log (. + + log((CDa(()) 91
+ Eenp, [log(1 Di.. (Gr,,
(4)
1
(2), (z)((
where Gs,o, is the i-th generator with parameters 8,, Di is the i-th channel discriminator with parameters Φi, CDa is the central discriminator with the parameters a, Pdota is the distribution of the real time series, 2; is the i-th channel of time series x, Gjzt are all the other generators with fix parameters for the optimization step of Gi is a hyper-parameter that control the trade-off between well-preserving the correlation among the channels versus generating better quality signal within each channel, and is the shared noise vector sampled from Pz distribution. The objective for all channels 2C + player game adding the terms with subscript in Equation
In each epoch, we divide the MTS training dataset into a number batches. If the batch size is assumed to be m, then each batch contains ,:(2),... (m) D. each batch, we sampled as many noise vector
P= as the batch size. All channels' generators synthesize signals, and a gradient ascent step on the discriminators' parameters 9i,8 are taken. Then we concatenate the synthetic signals of all channels and use them in addition to the real data to take a gradient ascent step the central discriminator parameters a. In the end, we take gradient descent step on the generators parameters 0,s.
Rea Fave kre
Simmad
Real
LLD Modale
Module
Fale
Droport
(a) Structure of Central Discriminator.
(b) Inside the L.I module.
Figure 2: Structure an MLP-based Central Discriminator.
4.3 Key Implementation Details
There many possible choices of network types for channel GANs. We show in the Supplementary Materials that networks based on Long short-term memory (LSTM) generate signals of higher quality than the networks based on Multi-layer perceptron (MLP).
We have investigated LSTM and MLP based networks for the central discriminator. Although L-TT-- based networks perform better than MLP-based networks in Channel GANs, MLP-based networks perform better in the Central Discriminator. We hypothesize that if the central discriminator is too powerful, the results will be of lower quality, the generators will strive to make the signals more correlated at the expense realistic individual TS.
Figure 2a depicts the structure of an MLP-based central discriminator. It consists of three Linear- LeakyRel Be--ReeelU-Droou (LLD) modules, a linear module, and a sigmoid function. Figure 20 demon- strates the structure of an LLD module.
5

---

5 Empirical Evaluation
5.1 Toy Sine Datasets:
Diversity vs Correlation Preservation
To be able to investigate the empirical behavior of COSCI-GAN and, in particular, the effect of the central discriminator, we need to have complete control over the nature of the datasets, particularly the ground truth of our desired tasks. Thus, we simulated three "toy medical" datasets with two channels and used them as "real" datasets to generate synthetic data. For all these datasets, we assumed that each instance correspond to different patient, and each patient produce measurements for two channels (c) and 62). make the datasets a bit more realistic, we also assumed that there are two types of patients (ply and pl2), as in "healthy" vs "condition". The three datasets are:
Simple Sine is derived from the formula: x = A (2n f() + e. The difference between the signals is that the amplitude (A) for patient type comes from (0.4, 0.05). whereas the amplitude for patient type 2 comes (rromA((...) 0.05). The other difference is that Channel has a frequency (f) of 0.01, while channel 2 has a frequency of 0.005. In addition, the noise comes from (O, 0.05).
Sine with frequency changes contains the same signals as Simple Sine, except that the frequency of all sine functions doubles exactly the middle of the time series. This allows to examine the situation with varying frequencies.
Anomalies is created by replacing the middle of the time series with Gaussian noise, thus allowing us to examine the impact of anomalies.
The visualization of all three toy datasets are available in the Supplementary Materials.
We assessed the behaviour of COSCI-GAN, particularly the central discriminator, by two criteria:
(1) Diversity: requiring that the generators should synthesize from both patients' distributions and there should be no mode collapse, a common failure of GANs, which occurs when the generator fails to produce results as diverse as the real data. We measured the diversity by comparing the distribution of amplitude of patients' signals in the real dataset, which a bimodal Gaussian distribution, with the distributions of amplitude of the generated samples using Wasserstein Distance (WD). We took the WDs average (AWD) to aggregate across the channels. A lower AWD indicates a closer similarity to the real distributions. The first column of
the AWD for the various cases.
(2) Correlation Preservation: requiring that the amplitudes of channels for each patient types should be equal to each other as much as possible. We measured the amplitudes of channel and 2 in all signals and verified their similarity. We defined our correlation metric as the average euclidean distance (AED) between the amplitude mapped on 2D plane (Channel vs Channel 2) and a line with slope The resulting plot provided in the Supplementary Materials, and the numeric values summarized in Table
A lower AED indicates stronger preservation of correlation.
Table 1: Results of Diversity (AWD) V.S.
Correlation Analysis (AED)
Dataset
Method
AWD
AED
Simple Sine
Without CD
0.0472
0.1326
With CD
0.0800
0.0177
Freq changes
Without CD
0.0397
0.0769
With CD
0.0679
0.0242
Anomalies
Without CD
0.0540
0.0766
With CD
0.0726
0.0161
As clearly shown in T there is a trade-off between diversity and correlation preservation. That is when we used the CD, the correlation was better preserved, but the expense diversity (similarity between the generated time series' marginal distribution of amplitudes and the toy Simple Sine time series' marginal distribution of amplitudes). Conversely, not using the CD would allow the generated sample distributions to be closer to the real data, but the generated channels were less correlated. The strength of the CD is controlled by the parameter ? as shown in Equation 4 As we conducted experiments to tune the hyper- hyper-paramett parameter 7 observed that there is stable range for 7. We ended up setting 7 to 5, which provides stable results for all the experiments presented in this paper, whether the datasets are the toy medical ones the real ones to be discussed later.

---

5.2 Toy Sine Datasets Feature-based Correlation Analysis
The catch22 feature set has been introduced to capture 22 CAnonical Time series CHaracteristics commonly seen in diverse time series data mining tasks [26]. Using this feature how correlation between any pair of catch 22 features was preserved in synthetic time series data generation. In other words, if pair is strongly correlated in the real dataset between the two channels, we would like to see that preserved in the synthetic dataset. Similarly, if two features are not correlated in the real dataset, they should remain uncorrelated in the synthetic dataset.
assessed
Figure 3 shows three heatmaps for those pairwise correlations between the two channels simple sine dataset (The same figure for other toy datasets provided in the Supplementary Materials). To simplify the heatmaps, we removed 7 features that were constant among the real datasets, and kept the remaining 15 features that varied. The left heatmap shows the pairwise correlations of the 15 features for the real dataset. The centre heatmap the one for the synthetic dataset generated by COSCI-GAN without CD, whereas the right heatmap is the generated by COSCI-GAT GAN having the CD. Clearly, the right heatmap resembles the left heatmap much more closely. The centre heatmap shows that without the CD, almost all the correlation relationships of the 15 features were destroyed.
While the heatmaps are useful for visualization, we also compared quantitatively the correlation matrices between the two channels using various metrics: (1) Mean Absolute Error (MAE), (2) Frobenius norm, (3) Spearman's
and (4) Kendall's For MAE and the Frobenius norm, smaller value indicates greater similarity between the correlation matrices the real and synthetic datasets. For Spearman's coefficient and Kendall's coefficient, the closer the value is to I, the higher is the similarity. Results shown Table provide convincing evidence the effectiveness the CD in synthetic MTS generation.
Real Data
Without CD
With CD
Feature Channel 2
0.5
-0.5
Feature Channel 1
Feature Channel
Feature Channel
Figure 3: Heatmaps for Catch22 Pairwise Correlations.
Table 2: Similarity between Correlation Matrices
Dataset
Method
MAE
Frobenius norm Spearman's p Kendall's
Simple Sine
Without CD
0.671
11.295
-0.761
-0.569
With CD
0.298
5.666
0.848
0.700
Freq changes
Without CD
0.268
7.889
0.259
0.174
With CD
0.131
3,413
0.834
0.661
Anomalies
Without CD
0.289
8.113
0.428
0.297
With CD
0.199
5.362
0.786
0.612
5.3 EEG Eye State Dataset and Downstream Classification
We selected a 14-channel EEG eye state dataset to measure the effectiveness COSCI-GAN on real signals [27 This dataset contains label indicating whether the patient's 's eyes were open or closed indicates closed, and indicates open). Each time series is 117 seconds long, resulting in 14980 samples at sampling rate of 128 per second. To remove outliers from the dataset, we experimented with various z-score values and determined that value of optimal for our dataset, so we removed points with z-scores greater than 3, the same value as other EEG studies (c.g. (28)). The label of the dataset allows us to create a downstream eye blink classification task as follows.
http:///ttiiaa.....ccccaaaawasee with license details therein

---

We extracted window of 800 samples in length containing an eye blink, with a margin of 200 samples at the beginning and end of each eye blink, and labelled those frames with 1. We also extracted 800 samples that did not contain an eye blink, with a margin of 200 samples between the beginning and end of an eye blink, and labelled them with O. We now have 1024 frames of each label for classification. Then we performed forward feature selection, and chose top 5 channels regarding the accuracy of our classification task.
To measure the effectiveness of COSCI-GAN for classification, we used the approach of train-on-real and test-on-fake, and the opposite approach of train-on-fake and test-on-real. We measured the accuracy of an LSTMM-bsee classifier, which is described in details in the Supplementary Materials, on a dataset that contains two channels. Bable3]show the results of this comparison. We compared the accuracy of the classifier when the synthetic data were generated with and without the CD. Once again, the CD brings significant value to downstream classification tasks.
Table 3: Accuracy in Classification Task
Experiment
COSCI-GAN with CD
COSCI-GAN without CD
Train-on-real, Test-on-fake
0.790
0,644
Train-on-fake Test-on-real
0.634
0.561
Next compared COSCI-GAN against a baseline method for generating MTS data. The baseline method an LSTM-based GAN that simultaneously generated all channels. The same LSTM network was utilized in the baseline method and the COSCI-GAN modules. The only difference is that the output layer of networks in the baseline method must generate all channels rather than just one. Below show the results of two experiments: (1) the All-synthetic experiment, and (2) the Augmentation experiment.
(1) All-synthetic experiment In this experiment, we assessed how well COSCI-GAN performed in classification task when compared with the baseline method and the actual dataset. We performed
cross cross-validen validation by using 80% of our real dataset for training the GANs. Then only the synthetic data were used to train the classifiers, which were then tested on the hold-out 20% of the real dataset. We
investigate the utility of the synthetic data for different number of channels in Figure 4a We repeated each experiment 30 times with different random seeds for each setting, and statistical significance tests were done the boxplots. A further comparison including COSCI-GAN without the CD is provided in the Supplementary Materials.
Figure 4a demonstrates that as the number of channels is increased, the accuracy of a classifier trained on data generated by COSCI-GAN and evaluated on real datasets increases. In contrast, the baseline method went the opposite way and demonstrated that the performance of MTS generation with all channels together degrades as the number of channels increases. As the number of channels grew, synthetic data generated by COSCI-GAN gave similar average performance the real dataset.
(2) Augmentation experiment This experiment was set up exactly like the previous one, but instead of using only synthetic time series to train the classifiers, an equal number of synthetic training samples. In Figure 46 the boxplots for the real datasets across the different number of channels are exactly the same those in Figure for the real datasets.
augmented the real dataset with
Between Figure 4a and Figure 4b COSCI-GAN shows significant improvements in accuracy. The difference was that the synthetic data generated were added to the real data. COSCI-GAN still outperformed the baseline method both in terms of the median and the variations accuracy.
Figure 4b was based on an augmentation ratio of 1:1. Figure 4c shows a scatter plot of comparing the accuracy of COSCI-GAN (y-axis) against the baseline method (x-axis) across six different augmentation ratios: 1 : 1. 1: 2, 1 : 4. 1: 6, 1 8, : 10, i.c. augmenting with up to 10 times more synthetic data than real training data. We repeated the experiment five times with different random seeds for each of these settings; the accuracy of each run is plotted in Figure Thus, any point above the diagonal red line indicates that COSCI-GAN outperformed the baseline method. The points in the based on the number of channels. It is obvious that COSCI-GAN dominated
figure are colour-coded the baseline method when there is more than two channels. To further quantify the differences in accuracy, we add dotted diagonal lines in 4c representing difference in accuracy with increments of 0.1. For instance, the lowest dotted diagonal line represents the cases when the accuracy of COSCI-GAN is 0.1 below that of the baseline method. Conversely, the other three dotted diagonal
8

---

0.95
0.95
Model
baseline
0.9
0.9
COSCI-GAN
0.85
0.85
Accuracy
0.8
0.8
0.75
0.75
0.7
0.65
0.65
0.55
0.55
0.5
0.5
0.45
0.45
Number of channels
Number of channels
(a) Accuracy of All-Synthetic Experiment.
(b) Accuracy of Augmentation Experiment.
# channels
0.9
0.85
COSCI-GAN
0.756
0.656
0.6
0.5
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95
Baseline
(c) Accuracy Augmentation Experiment: each point of the same color corresponds to various augmentation ratios, from 1: to 10, see text for details.
Figure 4: Experiment three results. In figure (a) and (b), T-tests were applied results order to show whether there is a statistically significant difference between the distribution of the results. ns means no significant difference, ** means 0.001 < p-value 0.01, and *** means p-value < 0.001.
lines represent the situations when the accuracy of COSCI-GAN is 0.1, 0.2 or 0.3 better than that of the baseline. Figure 4c) shows that COSCI-GAN almost always performed better than the baseline method, up to 0.3 higher accuracy, and most exceptions are from the channel dataset where the accuracy at most 0.1 lower.
5.4 Comparing with State-of-the-art methods EEG Classification
In the final experiment, compared COSCI-GAN with two state-of-the-art (SOTA) methods: TimeGAN 24 and the most recent Fourier Flows 25 discussed previously. In these papers, the downstream task predicting the next time point of the time series; i.e., forecasting. Because our focus is on classification, we used TimeGAN and Fourier Flows' code to generate the five EEG channels that we chose in the previous experiment and performed the augmentation experiment that we described in 5.3 As shown Figure [ouure]]COSss COSCI-GAN leads to the best classification accuracy and there is statistically significant difference between the results of COSCI-GAN and two other methods.
To compare the statistical properties of COSCI-GAN with SOTA methods, repeated the correlation analysis from 5.2 Figures similar to Figure S for TimeGAN and Fourier Flows are provided in the Supplementary Materials. We computed the MAE between the real correlation matrix and each method's 's correlation matrix for each pair of channels in the EEG dataset. Table T Ihowws that COSCI-GAN, in addition to giving better classification accuracy, provides the closest similarity in inter-channel correlations to the real dataset.
http://gghhaacmmmmmyyommm//////n with license details therein https//ghhhb..oomm/mmcccmllaaaat with license details therein

---

0.95
0.85
Table 4: Similarity between
0.8
Correlation Matrices EEG dataset mean + standard deviation
0.75
0.7
Method
MAE
0.65
COSCI-GAN
0.111 + 0.005
TimeGAN
0.257 + 0.008
Fourier Flows
0.146 0.006
0.55
COSCI-GAN
Fourier Flows
Time GAN
Real
Figure 5: Accuracy for the Augmentation Experiment
Comparison with SOTA methods
In TimeGAN and Fourier Flows papers, daily historical Google stocks dataset from 2004 to 2019 was used, and PCA[29] and t-SNE(30) plots were shown to compare their diversity in compare with the real data. We repeated this experiment using data from TimeGAN repository [ and showed that COSCI-GAN samples were more diverse and distributed in a way that was more similar to the real dataset distribution. The figures of this experiment provided in the Supplementary Materials.
6 Discussion
In this paper, we introduced COSCI-GAN, a novel framework for multivariate time-series generation that delivers more correlated channels. By preserving the correlation between channels, COSCI- GAN is able to generate time-series that are more similar to the real seressries and achieve better performance in downstream classification tasks than other state of the methods.
We have shown that our framework relevant for generating MTS from a common source and we argue that it is particularly suited for human-based biometric measurements. In our experiments, we have never had performance limitations, we foresee however, that COSCI-GAN will not scale to very large number of channels as dedicated GAN for each channel is needed. However, this issue is not exclusive to the COSCI-GAN method. is a "computing resources" limitation for the COSCI-GAN method, whereas it is fundamentally intractable problem for the baseline and many other methods. In addition, COSCI-GAN has the benefit of being parallelizable, which makes faster. On the other extreme, we have shown that COSCI-GAS is not competitive for two channels where it often performed worse than a simple baseline.
It is worth re-iterating that synthetic data generation does not guarantee privacy and similarly, there no way to know in advance the performance on a downstream task so both characteristics should be empirically evaluated post-generation. Outliers and minorities leaks as they do not get protected by a large number of similar data samples. We also acknowledge that synthetic data generation can cause harm propagating and even magnifying bias from the data it based on.
often affected most by privacy
As future work, our method could be extended to more practical use cases where the various channels corresponds to different types of time series, e.g. heartbeats, temperature, respiration and wearable measurements and so on. As another extension, (corresponding to several initial noises) that are all originated from a single source, order to have more control on each channel's distribution. On the technical side, our framework can be implemented with a wide variety of GANs chosen based on the data type including modern architectures such as transformers. Additionally, as stated previously, Channel GANs could train in parallel, which would accelerate the training process. The code we have provided modular and its parallelized version can be implemented in future.
could consider having an initial noise embedding
10

---

References
[1] Centers for Medicare & Medicaid Services. The Health Insurance Portability and Accountability Act of 1996 (HIPAA). Online at http://www.cms.ll hhs. www.ggo/hipaa. 1996.
[2] 2018 reform of eu data protection rules. htR..httts:///pprraa
eu/
[3] Latanya Sweeney. Simple demographics often identiyy people uniquely. 2000.
[4] Kenneth Mandl and Eric D. Perakslis. Hipaa and the leak "deidentified" ehr data. The New England journal of medicine, 384 23:2171-2111 2021.
[5] Arvind Narayanan and Vitaly Shmatikov. Robust de- an-nymizaaiionn of large sparse datasets. In 2008 IEEE Symposium on Security and Privacy (sp 2008), pages 111-125, 2008. doi: 10. 10.1109/SPP2008.3..
[6] Maoguo Gong, Yu Xic, Ke Pan, Kaiyuan Feng, and A.K. Qin. survey on differentially private machine learning [review article]. IEEE Computational Intelligence Magazine, 15(2)49-64, 2020. doi: 10.1 MCL.2020.2976185.
[7] Theresa Stadler, Bristena Oprisanu, and Carmela Troncoso. Synthetic data anonymisation groundhog day. 2020.
[8] Jakub Nalepa, Michal Mareinkiewicz. and Michal Kawulok. Data augmentation brain-tumor segmenta- tion: A review. Frontiers in Computational Neuroscience, 13, 2019.
[9] lean-François Rajotte, Sumit Mukherjee, Caleb Robinson, Anthony Ortiz, Christopher West, Juan M. Lav- ista Ferres, and Raymond T. Ng. Reducing bias and increasing utility by federated generative modeling of medical images using a centralized adversary. Proceedings of the Conference on Information Technology for Social Good, 2021.
[10] Khaled El Emam. Lucy Mosquera, and Jason Bass. Evaluating identity disclosure risk in fully synthetic health data: Model development and validation. J Med Internet Res, 22(1 (2(11)c 132393,, Nov 2020. ISSN 1438-8871. doi: 10.2196/23139. URL sattps://dol org/ 10. 2196/23139
[11] Martin Ahadi, Andy Chu, Ian J. Goodfellow, H. B. McMahan, Tlya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference Computer and Communications Security, 2016.
[12] Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song. A. Terzis, and Florian Tramèr. Membership inference attacks from first principles. ArXiv, 1bs/2111.00350. 2021.
[13] Ian J. Goodfellow, Jcan Pouget- Abadic, Mehdi Mirza, Bing Xu, David Warde-Farlcy, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, 2014.
[14] Eoin Brophy, Zhengwei Wang, Qi She, and Tomas E. Ward. Generative adversarial networks in time series: A survey and taxonomy. ArXiv, abs/2107. 1098, 2021.
[15] Olof Mogren. C-rnn-gan gan: Continuous recurrent neural networks with adversarial training. ArXiv,
abs/1611.00 1,0990 2016.
[16] Cristóbal Esteban, Stephanic L. Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. ArXiv, abs/1706.00633. 2017.
[17] Shota Harada, Hideaki Hayashi, and Seiichi Uchida. Biosignal data augmentation based on generative adversarial networks. 2018 40th Annual International Conference of the IEEF Engineering in Medicine and Biology Suciety (EMBC). pages 368-371, 2018.
[18] Luca Simonetto. Generating spiking time series with generative adversarial networks : an application on banking transactions. 2018.
[19] Moustafa Alzantot, Supriyo Chakraborty, and Mani B. Srivastava. Sensegen: A deep learning architecture for synthetic sensor data generation. 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pages 188-193, 2017.
[20] Yizhe Zhang, Zhe Gan, and Lawrence Carin. Generating text via adversarial training. 2016.
[21] Chi Zhang, Sanmukh Rao Kuppannagari, Rajgopal Kannan, and Viktor K. Prasanna. Generative adversarial network for synthetic time series data generation in smart grids. 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmarfGridCommm) pages 1-6. 2018.
11

---

[22] Tianlin Xu, Li Kevin Wenliang, Michael Munn. and Beatrice Acciaio. Cot-gan: Generating sequential data via causal optimal transport. ArXiv, abs/2006.08571, 2020.
[23] Xiaomin Li, Vangelis Metsis, Huangyingrui Wang. and Anne H. H. Ngu. Tis-gan: A transformer-based time-series generative adversarial network. ArXiv, abs/2202.02691, 2022.
[24] Jinsung Yoon, Daniel Jarrett, and Mihaela van der Schaar. Time-series generative adversarial networks. In NeurIPS, 2019.
[25] Ahmed M. Alaa, Alex J. Chan, and Mihacla van der Schaar. Generative time-series modeling with fourier flows. In ICLR, 2021.
[26] Carl Henning Lubba, Sarab S. Sethi, Philip Knaute, Simon R. Schultz, Ben D. Fulcher, and Nick S. Jones. catch22: Canonical time-series characteristics. Data Mining and Knowledge Discovery, 33:1821 1852, 2019.
[27] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http::/archiiee...... uci edu/ml
[28] Mohit Agarwal and Raghupathy Sivakumar. Blink: fully automated unsupervised algorithm for eye- blink detection in eeg signals. In 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1113-1121, 2019. doi: LLL.://LLERTON.21999911111
[29] Bryant Paul R. Yarnold. Prineipal-components analysis and exploratory and confirmatory factor analysis. 1995.
[30] Laurens van Maaten and Geoffrey Hinton, Visualizing data using I-SNE, Journal of Machine Learning Research, 9:2579-2605, 2008. http::ttp//eet jmlr. wwg/papers//aae wwtp://aaadaeaaeenaaaa... html
12


Reviews:
[{"review_id": "KLo1s7po_Ad", "reviewer": "Reviewer_oPqB", "summary": "This paper proposes a generative method for multivariate time series. The method consists of individual GANs for each channel of time series with a shared initial noise, and a global discriminator for preserving the correlations between different channels. The design is to model the common factors of different channels.", "questions": "1. In Eq 2, why to assume the conditional distributions are independent in the conditions (channels)?\n2. Will the shared initial noise in Eq 3 and the modeling of the joint condition in Eq 2 have similar effects? How could they both be indispensable in the method?\n3. Is it reasonable to add several initial noises, instead of one, for improving the flexibility?\n", "limitations": "The authors addressed the potential negative societal impact well.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. The problem of synthetic time series generation is important in many tasks, e.g., data sharing, data augmentation.\n2. The proposed method uses a reasonable approach to enforce common factors of multivariate time series and preserve correlations.\n3. The experiments on synthetic and real datasets indicate the effectiveness of the proposed method to some extent.\n\nWeakness:\n\n1. The method is sort of incremental on GAN for time series generation. Preserving correlations using a central discriminator is similar to enforce a regularization of different channels (Fig 1), which is a common way of coordinating multivariate time series.\n2. In Eq 2, the condition conditional distributions are assumed independent. That is, the equivalence between the joint condition and the product of individual condition. This is not clearly justified.\n3. Enforcing a shared initial noise in Eq 3 and modeling the joint condition in Eq 2 seem have similar effects of regularizing different channels of time series. The description is not clear on how could these two parts both be important.\n4. Enforcing a single initial noise may be a strong constraint. Adding K initial noises, where K is smaller than the number of channels, could make the model more flexible and fit training data better.\n5. In the experiments, mode time generative model could be compared, on both synthetic and real datasets for comprehensiveness and consistency.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GxWL9jSn2Et", "reviewer": "Reviewer_zeED", "summary": "The authors present GroupGAN for synthesis of multivariate time-series data.  In contrast to previous approaches where features are jointly learned, GroupGAN trains a separate GAN while also training a central discriminator which classifies feature vectors as either real or fake.  The intuition behind the latter is to preserve correlations among the individually trained generators.  Synthetically trained data are evaluated for low-dimensional synthetic time-series data and supervised EEG data.", "questions": "The following is a major contribution of the paper: \"this is the first study to analyse how to generate multivariate time series\nwith individual channel generation originating from a common noise while inter-channel\ncorrelation preservation is forced with a central discriminator.\" <- How realistic is this scenario?  E.g., for the listed examples \"the biometric values from a medical patient, the stock prices from economic events or geographically separated seismic measurements from a single earthquake,\" do these assumptions hold?  E.g., while biometric values are collected from a single medical patient, different signals may have different noise properties (e.g., ECG, blood pressure, etc.).\n\nWe hypothesize that this will penalize unrealistic (un)correlation patterns between channels.\"  <- Can you supply evidence to support\n this, or point out where in the text evidence will be demonstrated?\n\nSuggestions:\n\" it is well known that sharing data associated with 26 single individuals, even anonymized, can lead to unexpected privacy breaches\" <- Well known ML examples:\n          -Narayanan, Arvind, and Vitaly Shmatikov. \"Robust\n          de-anonymization of large sparse datasets.\" 2008 IEEE\n          Symposium on Security and Privacy (sp 2008). IEEE, 2008.\n\n          -Gong, Maoguo, et al. \"A survey on differentially private\n          machine learning.\" IEEE computational intelligence magazine\n          15.2 (2020): 49-64.", "limitations": "A discussion of the tradeoff between learning a single multivariate GAN vs multiple GANs would help strengthen the paper.  While the authors have mentioned learning a large number of individual GANs can be prohibitive for high-dimensional data, in situations where jointly learning the distribution of all features is intractable, the presented architecture is highly parallelizable while maintaining a mechanism to encourage the separately trained GANs generate correlated data.  ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The proposed architecture is novel and interesting.  The central disciminator, which enforces preservation of correlations among independent GANs, is intuitive and could be helpful regularizing different GAN instances to promote consistency/correlations.  While the authors claim: \"The main reason for having channel GANs as opposed to a single multichannel generator discriminator pair is to make each generator powerful in its own channel distribution, and by including the central discriminator, we enforce realistic correlations between the\nchannels as much as possible.\"\nIt seems like another major advantage of this approach is tractability; jointly modeling all TS at once may lead to a computational explosion, but the current setup allows the computationally efficient benefits of training individual generators while also ensuring correlations are learned among the individual TS through the central discriminator.\n\nWeaknesses: The writing and description of the model could be much improved to align with time-series literature.  In particular, the description and focus on channels both confuses and limits the paper to vision data.  Channels typically refers to vision data, but moving the model description and text to describing time signals as comprised of features broadens the work and its applicability (i.e., a large number of time series work are not vision data).  Furthermore, the description in terms of channels breaks down for different applications (i.e., MFCCs in speech recognition).  The clearest/most general description in time-series analysis is a (possibly multirate as well as multivariate) process is composed of various features collected at different times.  Furthermore, there seems to be a general misconeption on what a time series is in the paper; the EGG eye state data is a single time series, each time-instance of which consists of 14 features (or measurements) and a binary label.\n\nEvaluation of the proposed method requires significant improvement.  The synthetic data is extremely low-dimensional (only 2 dimensions) and resulting hyperparameter tuning, performance, an(d (particularly) catch22 results on this data are thus unconvincing.  Furthermore, the real data is preprocessed in an extremely ad-hoc way (lines 229-234) without any explanation.  E.g., the EEG data is supervised, why are labels re-estimated during preprocessing?  How many outliers were present in the data, and what observed effects on GroupGAN and competitors warranted removal?\n\nNecessary details are also lacking from the paper.  In particular: \"The baseline method is an LSTM-based GAN that generated all of the\nchannels simultaneously\" <- this competitor requires much more explanation.  What were the exact details of the architecture and learning schedule?  Why were they chosen?  Why were existing methods (i.e., TimeGAN)  not compared to?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "G8WqI_b5fMt", "reviewer": "Reviewer_NYDe", "summary": "This paper introduces GroupGAN a method for generating multivariate time series data that share a single source or cause. GroupGAN uses a separate generator and discriminator for generating each data channel and a central discriminator for forcing the model to keep the correlation between different channels the same as the real data. Synthetic data as well as EEG eye state dataset is used to evaluate GroupGAN compared to other models in this area.", "questions": "- On lines 151-154 you mention that for central discriminator MLP networks work better than LSTM networks because \"if the central discriminator is too powerful, the results will be of lower quality, as it will strive to make the signals more correlated at the expense of realistic individual TS.\" can't this be controlled with the gamma weight?\n- Is the \"All-synthetic experiment\" described on lines 243-255 the same as the \"Train-on-fake, Test-on-real\" experiment mentioned in table 3?\n- In Figure 4, I wonder why GroupGAN does so much better than baseline on the All-Synthetic experiment while on the Augmentation experiment it is not doing as well? Is it because of a lower diversity of samples generated compared to the baseline?\n- In Figure 4c, does augmenting more synthetic data reduce the performance of GroupGAN with respect to the baseline model, or does it improve it? It is not obvious from this figure.\n- What is your hypothesis on why GroupGAN does not do as well on 2 channel datasets?", "limitations": "The limitations are discussed adequately.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nI believe the generation of multi-variate time-series data is a significant and very challenging problem. As mentioned by the authors it is urgently needed in many fields. The idea of using a central discriminator and separate generator discriminator pairs for each channel is very interesting and the authors show through experiments that this idea improves the data generation and downstream tasks.\n\nWeaknesses:\n- In the related works section it would be good to mention the weaknesses of the existing methods that the current model is trying to fix\n- I think some additional comparisons would've strengthened the claim of the paper that GroupGAN works better than the baseline model (a single multi-channel GAN model for generating the data) for data generation (on lines 116 - 118), for example in tables 1 and 2 I expected to see a comparison with the baseline model.\n- On line 155 the authors used LLD for the first time and then define them later it is better to define a term before using it.\n- The paper can be improved by changes in language and writing style. There are typos and grammar problems that if fixed can improve the clarity of the paper. I bring some examples below:\n- - on line 13 it should be \"performs very well in downstream tasks\"\n- - on line 95 \"channels\" should be \"channel\"\n- - on line 223 \"EGG\" should be \"EEG\"\n-- on line 292 \"compare\" should be \"comparison\"", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "FHVMN-fEybu", "reviewer": "Reviewer_sRJW", "summary": "In this paper, the authors propose Group GANs to generate Multivariate Time Series (MTS) data. MTS can be used for augmentation and privacy protection. To generate realistic MTS, the model should (1) generate realistic single-channel data (2) preserve the correlation between channels. So this paper uses a paired generator and discriminator to generate single-channel data and uses a Central Discriminator to force correlation preservation. They verify the results on synthetic and EEG datasets.", "questions": "Q1:\n\nEq (4) is the objective for “a given channel” i. What is the objective for all channels (for i generators + i discriminators + one central discriminator)?\n\nQ2: \n\nIn eq (4), when updating channel i, does the Central Discriminator treat the generated samples in  other channels (j != i) as real ones? It seems the case because in line 100 you say “the goal is to estimate the conditional distribution”.\n\nQ3:\n\nWhat is the output of the central discriminator? I still don’t get it after reading Algorithm 1. The central discriminator takes in data of all the channels, but does it output a single score for all the channels combined or does it output a score for every channel (i scores in total)? And how do you update the central discriminator exactly (Describe the loss)?", "limitations": "More useful datasets and advanced model architectures can be used.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths:\n* The paper is well structured and nicely written. The authors provide a clear motivation for why generating MTS is useful and explain the key challenges.\n* The authors tear down the problem of generating MTS into (1) generating realistic single-channel TS (2) preserving correlation. The authors propose Group GANs (many channel GANS + Central Discriminator) to address these problems. The method is straightforward and intuitive. I believe the proposed method will provide insight and inspire people working on related problems.\nIt’s good to see the authors submitted detailed supplementary materials.\n* The authors use many metrics and methods (eg, Feature-based Analysis) to show how Diversity and Correlation are preserved. They also discussed the implementation details and trade-offs of their model, which can be valuable for the community.\n\n## Weaknesses:\n* I think the algorithm and training details are very important for understanding the paper. Maybe you can consider moving them (mainly Algorithm 1) from the Supplementary Materials to the body part.\n* Some questions regarding the objectives remain. See Questions.\n* The datasets are small, and they don’t look very practical. (As the authors state “our method could be extended to more practical use cases” in line 312).\n* Whether the proposed model can scale very well remains a question. Performance limitations can be an issue. Another issue is that the authors only test MLP and LSTM, which is not a full guarantee that they can generalize to complex models like Transformers.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept