GlucoSynth: Generating Differentially-Private Synthetic Glucose Traces
Authors: Josephine Lamp
Mark Derdzinski
Christopher Hannemann
Joost Van der Linden
Lu Feng
Tianhao Wang
David Evans
Venue: NeurIPS 2023 poster (2023)
Keywords: Synthetic Data
Time Series
Generative Adversarial Networks
Differential Privacy
Glucose
Diabetes
TL;DR: Generation of synthetic time series glucose traces using a novel differentially private generative adversarial network framework

Abstract: We focus on the problem of generating high-quality, private synthetic glucose traces, a task generalizable to many other time series sources. Existing methods for time series data synthesis, such as those using Generative Adversarial Networks (GANs), are not able to capture the innate characteristics of glucose data and cannot provide any formal privacy guarantees without severely degrading the utility of the synthetic data. In this paper we present GlucoSynth, a novel privacy-preserving GAN framework to generate synthetic glucose traces. The core intuition behind our approach is to conserve relationships amongst motifs (glucose events) within the traces, in addition to temporal dynamics. Our framework incorporates differential privacy mechanisms to provide strong formal privacy guarantees. We provide a comprehensive evaluation on the real-world utility of the data using 1.2 million glucose traces; GlucoSynth outperforms all previous methods in its ability to generate high-quality synthetic glucose traces with strong privacy guarantees.

GlucoSynth: Generating Differentially-Private Synthetic Glucose Traces
Josephine Lamp Mark Derdzinski Christopher Hannemann"
Joost van der Linden Lu Feng' Tianhao Wang' David Evans!
University of Virginia, Charlottesvilll, VA, USA; Dexcom, USA
j14rj@virginin edu; {mark. ; christopher hannemann; joost. (annderlinden@@deee com; {lu.fengs tianhao; evanginvi edu
Abstract
We focus on the problem of generating high-quality, private synthetic glucose traces, task generalizable to many other time series sources. Existing methods for time series data synthesis, such as those using Generative Adversarial Networks (GANs), are not able to capture the innate characteristics of glucose data and cannot provide any formal privacy guarantees without severely degrading the utility of the synthetic data. In this paper present GlucoSynth, novel privacy-pree preserving GAN framework to generate synthetic glucose traces. The core intuition behind our approach is to conserve relationships amongst motifs (glucose events) within the traces, in addition to temporal dynamics. Our framework incorporates differential privacy mechanisms provide strong formal privacy guarantees. We provide comprehensive evaluation on the real-world utility of the data using 1.2 million glucose traces; GlucoSynth outperforms all previous methods in its ability generate high-quality synthetic glucose traces with strong privacy guarantees.
Introduction
The sharing of medical time series data can facilitate therapy development. As motivating example, sharing glucose traces contribute the understanding of diabetes disease mechanisms and the development of artificial insulin delivery systems that improve people with diabetes' quality of life. Unsurprisingly, there serious legal and privacy concerns (e.g.. HIPAA, GDPR) with the sharing of such granular, longitudinal time series data in a medical context (1. One solution is to generate a set of synthetic traces from the original traces. In this way, the synthetic data may be shared publicly in place of the real ones with significantly reduced privacy and legal concerns.
This paper focuses on the problem of generating high-quality, privacy-preserving synthetic glucose traces, a task which generalizes to other time series sources and application domains, including activity sequences, inpatient events, hormone traces and syber-physiee systems. Specifically, focus on long (over 200 timesteps), bounded, univariate time series glucose traces. We assume that available data does not have any labels or extra information including features metadata, which is quite common, especially in diabetes. Continuous Glucose Monitors (CGMs) easily and automatically send glucose measurements taken subcutaneously fixed intervals (e.g., every minutes) to data storage facilities, but tracking other sources of diabetes-related data is challenging (2). We characterize the quality of the generated traces based on three criteria- synthetic traces should (1) conserve characteristics of the real data, i.e., glucose dynamics and control-related metrics (fideiity;; (2) contain representation of diverse types of realistic traces, without the introduction of anomalous patterns that do not occur in real traces (breadth); and (3) be usable in place of the original data for real-world use cases (utility).
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Sample Glucose Traces
Motif High Peak
Motif 2: Deep Trough
Matif 3: Stable Line
400
Glucose (migoo))
Glucose (mo/dt))
Glucose (mg/dl)
Glucose (moloo))
300
2000
200
100-
2:4 Timestops
12
12
Timestops
Timestops
Timestops
Figure 1: Example Real Glucose Traces and Glucose Motifs from Dataset.
Generative Adversarial Networks (GANs) J have shown promise in the generation of time series data. However, previous methods for time series synthesis, e.g.. [ 15 6 suffer from one more of the following issues when applied to glucose traces: 1) surprisingly, they do not generate realistic synthetic glucose traces in particular, they produce human physiologically impossible phenomenon in the traces; 2) they require additional information (features, metadata or labels) to guide the model learning which not available for traces; 3) they do not include any privacy guarantees, or, in order to uphold a strong formal privacy guarantee, severely degrade the utility of the synthetic data.
Generating high-quality synthetic glucose traces is difficult task due to the innate characteristics of glucose data. Glucose traces can be best understood as sequences of events, which we call motifs, shown in Figure I, and they are more event-driven than many other types of time series. As such, current g glucose value may be more influenced by an event that occurred in the far past compared to values from immediate previous timesteps. For example, large meal caten carlier in the day (30-90 minutes ago) may influence a patient's 's glucose more than the glucose values from the past 15 minutes. As result, although there is some degree of temporal dependence within the traces, only conserving the immediate temporal relationships amongst values at previous timesteps does not adequately capture the dynamics of this type of data. In particular, we find that the main reason previous methods fail is because they may not sufficiently learn wvent-related characteristics of glucose traces.
sewvacy-preserving GAN framework to generate synthetic glucose traces. The core intuition behind our approach conserve relationships amongst motifs (events) within the traces, in addition to the typical temporal dynamics contained within time series. We formalize the concept of motifs and define a notion of motif causality, inspired from Granger causality 77]. which characterizes relationships amongst sequences of motifs within time series traces (Section 4). We define a local motif loss to first train a motif causality block that learns the motif causal relationships amongst the sequences of motifs in the real traces. The block outputs motif causality matrix, that quantifies the causal value of seeing one particular motif after some other motif. Unrealistic motif sequences (such peak to immediate drop in glucose values) will have causal relationships close to 0 in the causality matrix. We build a novel GAN framework that is trained to optimize motif causality within the traces in addition to temporal dynamics and distributional characteristics of the data (Section 5). Explicitly, the generator computes a motif causality matrix from each batch of synthetic data generates, and compares it with the real causality matrix. As such, as the generator learns to generate synthetic data that yields a realistic causal matrix (thereby identifying appropriate causal relationships from the motifs), it implicitly learns not to generate unrealistic motif sequences. We also integrate differential privacy (DP) [8] into the framework (Section 6), which provides an intuitive bound on how much information may be disclosed about any individual in the dataset, allowing the GlucoSynth model to be trained with privacy guarantees. Finally, in |Sectiio 7, we present a comprehensive evaluation using 1.2 million glucose traces from individuals with diabetes collected across 2022, showcasing the suitability of our model to outperform all previous models and generate high-quality synthetic glucose traces with strong privacy guarantees.
Contributions. We present GlucoSynth,
Related Work
We focus the scope of our comparison current state-of-the-art methods for synthetic time se- ries which build upon Generative Adversarial Networks (GANs) [3] and transformation-baat
approaches [9]. An extended related work is in Appendix A

---

00000
19120
15
1000
10.55
00
1190
00
$2.00
1400
14000
20000
1653
1200
(a) Glucose Motif 1
(b) Glucose Motif 2
(c) Temporal Motif
(d) Temporal Motif 2
Figure 2: Temporal Distributions of Sample Motifs. Each radial graph displays the temporal distribution of a motif; there are 24 radial bars from 00:00 to 23:00, and each segment displays the of motif occurrences by each hour. Glucose motifs and 2 are from Fig. | they are not temporally- dependent and show up across the day. Temporal motifs and 2 are from a cardiology dataset [15]
Time Series. Brophy et al. 10 provides a survey of GANs for time series synthesis. TimeGan is popular benchmark that jointly learns an embedding space using supervised and adversarial objectives in order to capture the temporal dynamics amongst traces. Esteban al. [T]] develops two time series GAN models (RGAN/RCGAN) with RNN architectures, conditioned on auxiliary information provided each timestep during training. encoding architecture in order to best preserve temporal dynamics. Transformation-
trains a GAN model that uses a transformer
approaches
such real-valued non-volume preserving transformations (NVP) (9] and Fourier Flows (FF) [12], have also had success for time series data. These methods model the underlying distribution of the real data to transform the input traces into a synthetic data set. Methods that only focus on learning temporal or distributional dynamics in time series are not sufficient for generating realistic synthetic glucose traces due to the lack temporal dependence within sequences of glucose motifs.
Differentially-PPe Private GANs. To protect sensitive data, several GAN architectures have been designed incorporate privacy-preserving noise needed satisfy differential privacy guarantees [13) Frigerio et al. [4] extends simple differentially-priin
architecture (dpGAN) to time-series data and RDP-CGAN 6 develops a convolutional GAN architecture specifically for medical data. These methods find large gaps in performance between the non- non-private and private models. Providing strong theoretical DP guarantees using these methods often results in synthetic data with little fidelity for use in real-world scenarios. Our framework carefully integrates DP into the motif causality block and each network of the GAN, resulting in a better utility-prine -privacy tradeoff than previous methods.
Preliminaries
3.1 Motifs
Glucose (and many other) traces can be best understood as sequences of events or motifs. Motifs characterize phenomenon in the traces, such as peaks or troughs. We define motif, μt, as short, ordered sequence of values (v) of specified length т,
[U;, U++l..... Ui4r] and or is a tolerance value allow approximate matching (within for each value). Some examples of glucose traces and motif's are shown in Figure I We denote a set of n time series traces as X [_.,,.., Zn.. Each time series may be represented as sequence of motifs: x,
( where each i, gives the index of the motif in the set that matches Zi.......BB, 1 Given the motif length т, the motif set is the union of all size-T chunks in the traces. This definition is chosen for straightforward implementation but motifs can be generated in other ways, such as through the use of rolling windows or signal processing techniques [16 [7]. Motifs are pulled from the data such that there always match from a trace motif to motif from the set (if multiple matches, the closest one is chosen).
3.2 Glucose Dynamics (Why Standard Approaches Fail)
We first present a study of the characteristics of glucose data order to motivate the development of our framework. Although there are general patterns in sequences of glucose motifs (e.g., motif patterns corresponding to patients that eat vs. 3x a day), individual glucose motifs are typically not time-dependent, illustrated in Figure 2 The radial graphs display the temporal distribution of
3

---

the first two glucose motifs from Figure 1 and two cesporally-dependde motifs from a cardiology dataset [T5]. There are 24 radial bars from 00:00 23:00 for each hour of the day, and the bar value is the percentage of total motif occurrences at that hour across the entire dataset (i.e., value of 10 would indicate that 10% of the time that motif occurs during that hour). Note that the glucose motifs show up fairly evenly across all hours of the day whereas the motifs from the cardiology dataset have shifts in their distribution and show up frequently at specific hours of the day. The lack of temporal dependence in glucose motifs is likely due to the diverse patient behaviors within a patient population. Glucose in particular is highly variable and influenced by many factors including eating, exercise, stress levels, and sleep patterns. Moreover, due to innate variability within human physiology, motif occurrences can differ even for the same patient across weeks or months. These findings indicate that only conserving the temporal relationships within glucose traces (as many previous methods do) may not be sufficient to properly learn glucose dynamics and output realistic synthetic traces.
3.3 Granger Causality
Granger causality 77] is commonly used to quantify relationships amongst time series without limiting the degree to which temporal relationships may be understood done in other time series models, e.g., pure autoregressive ones. In this framework, an entire system (set of traces) is studied together, allowing for broader characterization of their relationships, which may be advantageous, especially for long time series. We define Ie E IR" as an n-dimensional vector of time series observed across n traces and 7' timesteps. To study causality, vector autoregressive model (VAR) (18) may be used. A set of traces at time is represented as a linear combination of the previous K lags in the series: J4 = -E x( + ez where each A is a x n dimensional matrix that describes how lag k affects the future timepoints in the series' and et is zero mean noise. Given this framework, we state that time series q does not cranger-cause time series p, if and only if for all k. A/10) = 0. To better represent nonlinear dynamics amongst traces, a nonlinear autoregressive model (NAR) [99] g. may (Zice) ...; Inge) + e where Jpct (Tpp.... .... Tp--1 Tp.) describes the past of series p. The NAR nonlinear functions are commonly modeled jointly using neural networks.
be defined, in which
4 Motif Causality
Using Granger causality as defined would overwhelm the generator with too much information, resulting in convergence issues for the GAN. Instead of looking at traces comprehensively, we need way to scope how the generator understands relationships between time series. To this end, we aim to use the same intuition developed from Granger causality, namely developing an understanding of relationships comprehensively using less stringent temporal constraints, but scope these relationships specifically in terms of motifs. Therefore, we develop a concept of motif causality which, by learning causal relationships amongst sequences of motifs, allows the generator to learn realistic motif sequences and produce high quality synthetic traces as a result.
4.1 Extending Granger Causality to Motifs
In order to quantify the relationships amongst sequences of motifs to best capture glucose dynamics, we extend the idea of Granger causality to work with motifs. Given a motif set with m motifs, we build a separate (component) model, called a motif network in our method, for each motif, resulting in m motif networks. For a single motif μ4 at time L, pi, we define a function y, specifying how motifs in previous timesteps are mapped to that motif: fs, = 9i (41 ...y (mmz.)) + es, where Uje, = (M) .... 1 )..) describes the past of motif μy. The output of g, is a vector, which is added to the noise vector e,, Essentially, we define motif μ4 in terms of its relationship to past motifs. The Si function takes in some mapping that describes how motifs in previous timesteps are mapped to the current motif pa. The mapping is not specified in this notation, and could be defined in many different ways. In our case, we instantiate 9i using a single-layer LSTM, described next.
A 9: function for each motif μ4 in the motif set is modeled using a motif network with single- layer RNN architecture. For a RNN predicting a single component motif, let hr E Rm represent the m-dimensional hidden state at time l. This represents the historical context of the motifs in the series for predicting a component motif at time I, Mi. At time I, the hidden state updated: h g. (h,-)) + ei, 9i here is the function describing how motifs in previous timesteps are mapped

---

Motif Networks
Hotif Net O
WY
Matric
Data Partition
Weights
M
Motif Net
Aggi egate Matrices
Real Glacase Traces
Matric
Aggregate Causality
Data Partitio
Motif Networks
Matric
Data Partition
Molif Networks
Figure 3: Motif Causality Block.
to the current motif, and is modeled (instantiated) as a single-layer LSTM as they are good at modeling long, nonlinear dependencies amongst traces [20]. The output for motif μ4 at time t, pi, can be obtained by linear decoding of the hidden state, Mi, W - ci where W is a matrix of the output weights. These weights control the update the hidden state and thereby control the influence of past motifs on this component motif. Essentially, this function learns a weighting that quantifies how helpful motifs in previous timesteps are for predicting the specified motif μ4 time t. We note that we define causality in this way based on how Granger causality models such relationships, which different from traditional causality models.
all elements in the jth column of W are zero (W3 0), this is a sufficient condition for an input motif μú being motif non-causal on an output μt, Therefore, can find the motifs that motif-causal for motif μi using a group lasso penalty optimization across the columns of Wo
T
min E(e. 91 (:0 Um ))² + W
Σ 11 J=1
1=2
We define this the local motif loss, Lmt, which is optimized in each motif network using proximal gradient descent.
4.2 Training the Motif Causality Block
We next describe how the motif causality block is trained to learn motif causal relationships amongst traces, displayed in Figure 3 The block is structured in this way to accommodate the privacy integration (Section 6. here, we present its implementation without any privacy noise.
Partition data. First, the data is partitioned into partitions (Step 1, Figure 3) such that models are trained on overlapping data. The number of partitions, r, is a user-specified hyperparameter.
Build motif network for each motif. Next, within each data partition set of motif networks is trained. As a prepprocssin step, we assume each trace has been chunked into a sequence of motifs of size (Section 3.1). is a hyperparameter, which we suggest chosen based on the longest effect time of a trace event. We use t = 48, corresponding to hours of time, because large glucose events (from behaviors like eating) are encompassed within that time frame; see Appendix B for more details.
assume tolerance glucose. To model motif causality for entire set of data, 9: function is implemented for each motif via a separate RNN motif net following the description provided previously, resulting in m total networks (Step 2a, Figure 3). If all the motifs were trained together using single motif network. would not be possible to quantify the exact causal effects hetween each individual motif as we would not know which exact motifs contributed of unknown motifs that contribute to an accurate prediction for a particular motif). By training each motif network separately, are able quantify the exact effect each motif has each other, without any confounding effects from other motifs.
mg/dL, chosen to allow for reasonable variations in
prediction (only that there is some combination
Combine outputs of individual motif networks. Each motif network outputs vector of weights To of dimensionality m, corresponding to the learned causal relationships (Step 2b, Figure 3). Values in the vector are between 0 (no causal relationship) and (strongest causal relationship) and
5

---

Motif Causality
Generator
Discriminator
Guecan Fades
Caunsilty Marrs
Real Glucose Traces
Disrtiiiattr
Guesses Rea
Set
Mosi Nets
EEE Symthet
Embedddr
Recovery
Autoencoder
Figure Overview of GlucoSynth Architecture.
give the degree which every other motif is motif causal of the particular motif μr the RNN was specialized for. To return complete matrix that summarizes causal relationships amongst all motifs, we stack the weights (Step 2c). The output each data partition complete motif causality matrix, resulting in total matrices, each of dimensionality m x m.
Aggregate matrices and integrate with GAN. After motif causality matrices have been outputted from each data partition, the weights in the matrices are aggregated (Step 3. Figure 3) to return the final aggregate causality matrix, M (Step 4). In the nonprivate version, the weights are averaged. Finally, M is sent to the generator to help it learn how to conserve motif relationships within sequences of motifs in the synthetically generated data. Details are described next in the subsequent section.
GlucoSynth
The complete GlucoSynth framework, shown in Figure 4 comprises four key blocks: the motif causality block (explained previously in Section an autoencoder, a generator and discriminator. We walk through the remaining components the framework surrounding the GAN next.
5.1 GAN Architecture Components
Autencoder. We use an autoencoder (AE) with an RNN architecture learn lower dimensional representation of the traces, allowing the generator to better preserve underlying temporal dynamics of the traces. The autoencoder consists of two networks: an embedder and recovery network. The embedder uses an encoding function to map the real data into a lower dimensional space: Enc(x) E I > Je R while the recovery network reverses this process, mapping the embedded data back to the original dimensional space: Dec(x.) 3e E R - f E IR. A foolproof autoencoder perfectly reconstructs the original input data, such that x = I = Dee Enc(x)). This process yields the Reconstruction Loss, CR. the Mean Square Error (MSE) between the original data x and the recovered data, z: MSE(x, ())...
We implement the generator via an RNN or LSTM. Importantly, the generator works in the embedded space, by receiving the input traces passed through the embedder (xz). To generate synthetic data, a random vector of noise, 2 is passed through the generator and then the recovery network to return the synthetic traces in the original dimensional space. To learn how produce high-quality synthetic data, the generator receives three key pieces of information:
Generator.
-- Stepwise. The generator receives batches of real data to guide the generation of realistic next step vectors. To do this, Stepwise Loss, Ls, is computed at time using the MSE between the batch of embedded real data, Fetz and the batch of embedded synthetic data, Zer: MSE( ftr). This allows the generator to compare (and learn to correct) the discrepancies in stepwise data distributions.
2 Motif Causality. The generator needs to preserve sequences of motifs in addition temporal dynamies. Using the aggregate causality matrix M returned from the Motif Causality Block, the generator computes motif causality matrix, Mg. on the set of synthetic data å. Because the original

---

causality matrix was not trained on data in the embedded space, we first run the set of embedded synthetic data through the recovery network a. + From there, the Motif Causality Loss, C.. is computed as the MSE error between the two matrices: MSE(M, Mz). These matrices give a causal value of seeing motif ju; in the future after some motif رز unrealistic motif sequences will have causal values close to O. As the generator learns to generate synthetic data that yields realistic causal matrix (thereby identifying appropriate causal relationships from the motifs), it implicitly learns to not generate unrealistic motif sequences.
3 Distributional. To guide the generator to produce a diverse set of traces, the generator computes Distributional Loss, Lp the moments loss (MML), between the overall distribution of the real data He and the distribution of the synthetic data dz: (M The MML is the difference in the mean and variance of two matrices.
Discriminator. The discriminator is traditional discriminator model using an RNN, the only change being it also works the embedded space. The discriminator yields the Adversarial Loss Real, Lar- the Binary Cross Entropy (BCE) between the discriminator guesses on the real data Ye. and the ground truth y, a vector of O's, (CE( y) and the Adversarial Loss Fake, Las. the BCE between the discriminator guesses the fake data ys. and the ground truth y. vector of 1 I's, BCE (ya, y).
5.2 Training Procedure
First, the motif causality block is trained following the procedure described in Section 4.2 and then the rest of the GAN is trained. The autoencoder is optimized to minimize LR + aLs. where a is hyperparameter that balances the two loss functions. If the AE only receives Ln (as is typically done), it becomes overspecialized, i.e., it becomes too good at learning the best lower dimensional representation of the data such that the embedded data are longer helpful to the generator. For this reason, the AE also receives Ls, enabling the dual training of the generator and embedder. The generator is optimized using min - Car) + ( + LD) LM, where 7) is hyperparameter that balances the effect of the stepwise and distributional loss. Finally the discriminator is optimized using the traditional adversarial feedback min min f + Ar The networks are trained in sequence (within each epoch) in the following order: autoencoder, generator, then discriminator. In our experiments we set a 0.1 and 17 = 10 as they enable GlucoSynth converge fastest, i.e., in the fewest epochs.
6 Providing Differential Privacy
There are two components to our privacy architecture, described in the following two subsections: (1) each network in the GAN (Embedder, Recovery, Generator and Discriminator networks) is trained in a differentially private manner using the Differentially-Priien Privvate Stochastic Gradient Descent (DP-SGD) algorithm from Abadi al. [21]; and (2) the motif causality block is trained using the PATE framework from Papernot et al. (22). Importantly, two completely separate datasets are used for the training of the motif causality block (dataset B in Figure 4) and the GAN (dataset A in Figure 4). We structure the privacy integration in this way to allow for better privacy-utility trade-offs. Our design satisfies the formal differential privacy notion introduced by Dwork al. [23]. Differrntial Privacy (DP) provides an intuitive bound on the amount of information that can be learned about any individual in a dataset. A randomized algorithm M satisfies (e. 8) -differrrntia privacy if, for all datasets Dj and D2 differing by at most single unit, and all S C Range( S] + 8. The parameters c and 5 determine the privacy loss budget, which provide way to tradeoff privacy and utility; smaller values have stronger privacy. Importantly, privacy is provisioned at the trace level, and we assume each individual has only one trace in the dataset.
(r,) E S] S e Pr[M(D2)
6.1 Training the GAN Networks with DP
To add privacy to the GAN components, each of the networks (Embedder, Recovery, Generator and Discriminator) is trained in differentially private manner using DP-SGD [2].. Although the overall
GAN framework is complicated, the individual networks all use simple RNN or LSTM architectures with Adam optimizers. As such, adding DP noise to their network weights is straightforward. We employ the following procedure using Tensorflow Privacy functions 24
Since there are four
7

---

networks being trained with DP, we divide the privacy loss budget evenly to get the budget per network, Ceet = c/4. Then, we use Tensorflow's s built-in DP accountant determine how much noise must be added to the weights of each network based on the number of epochs, batch size, number of traces and Eee-- This function returns a noise multiplier, which we use when we instantiate a Tensorflow DP Keras Adam Optimizer for each network. Finally, we train each of the networks using their respective DP Keras Adam Optimizer, which automatically trains the network using DP-SGD.
6.2 Training the Motif Causality Block with DP
We train the motif causality block using the PATE framework [22]. PATE provides way to return aggregated votes about the class a data point belongs to. First, the data partitioned into partitions, where r is determined based on the size of the dataset and the privacy loss budget. Then, class membership model is trained independently for each partition. The class membership votes from each partition are aggregated by adding noise to the vote matrix and the noisiest votes are returned using the max-of-Laplacian mechanism (LNMax), tuned based on the privacy budget and r.
We use PATE to train the motif causality block: instead of predicting the degree of class membership we predict causal membership, e.g., does motif 14, have a causal relationship to Hy.- The motif causality block is trained in the same procedure described in Section 4.2 with two changes: (1) the number of data partitions, r. is determined based on the privacy budget, instead of a user-specified value; (2) the final causality matrix M is aggregated using DP across the partitions. In normal PATE, carefully calibrated noise is added to a matrix of votes for each class, such that the classes with the noisiest votes are outputted. In our use, each value in motif causality matrix may be likened to a class (i.e., causal "class" " prediction between motif (1; and (ı;). Thus, we use the LNMax mechanism (from predefined Tensorflow Privacy functions (24) to aggregate the matrices weights and return M.
We use PATE instead of training each motif network using DP-SGD for better privacy-utility trade- offs. With DP-SGD, we would need to add noise to every motif net, eating up our privacy budget quickly and severely impacting the quality of the returned casuality matrices. PATE allows us train each of the motif networks without any noise on the gradients, but then aggregates their returned manner, resulting in a better privacy-utility trade-off.
causality matrices
7 Evaluation
Evaluating synthetic data is notoriously difficult (25). so we provide an extensive evaluation across three criteria. Synthetic data should: 1) conserve characteristics of the real data (fidelity, (ecctinn 7.I) 2) contain diverse patterns from the real data without the introduction of anomalous patterns (breadih, Section 7.2 and 3) be usable in place of the original for real-world use cases (utility, Section 7.3).
Data and Benchmarks. We use 100,000 single-day glucose traces randomly sampled across each month from January to December 2022, for a total of 1.2 million traces, collected from Dexcom's G6 Continuous Glucose Monitors (CGMs) [26]. Data was recorded every 5 minutes (T' = 288) and each trace was aligned temporally from 00:00 to 23:59. We restrict our comparison to the five most closely related state-of-the-ar art models for generating synthetic univariate time series with no labels or auxiliary data: Three conprivate-TimeGANN -TimeGAN 4. Fourier Flows (FF) [12, non-volume preserving
transformations (NVP) 19; and two private- RGAN [II and dpGAN [1). We refer the reader to Appendix B for additional experimental details and all hyperparameter settings, including reasoning behind the choice of motif size 7 = 48.
7.1 Fidelity
Visualization. We provide visualizations of sample real and synthetic glucose traces from all models. Although this is not comprehensive way to evaluate trace quality, it does give a snapshot view about what synthetic traces may look like. We provide heatmap visualizations, where each heatmap contains 100 randomly sampled glucose traces. Each row is single trace from timestep 0 to 288. The values in each row indicate the glucose value (between 40 mg/dL. and 400 mg/dL.. Figure 5 shows the nonprivate models, and Figures 8 10 in Appendix C.1 show the private models with different privacy budgets. Upon examinging the heatmaps, we notice that GlucoSynth consistently generates realistic looking glucose traces, even at very small privacy budgets.
8

---

Real
GynchSytth
TimeGAN
NPV
103
39
350
80
300
70
Indiviiull
60
250
50
40
38
150
100
10
Time
Time
Time
Time
Time
Figure 5: Heatmaps for Nonprivate Models
Population Statisties. To evaluate fidelity on population scale, we compute a common set of glucose metrics and test if the difference between the synthetic and real data is statistically significant. Table provides an abbreviated summary of the results; Appendix C.2 has complete results, GlucoSynth performs the best, with few statistical differences between the real and synthetic data for 0.1.
Distributional Comparisons. We visualize differences in distributions between the real and synthetic data by plotting the distribution of variances and using PCA (27]. Figure 0 shows the variance distribution for the nonprivate models. Additional comparisons across privacy budgets are available in Appendix C.3 In both nonprivate and private settings, GlucoSynth produces synthetic distributions closest the real ones, better than all other models.
7.2 Breadth
We quantify breadth terms of glucose motifs. For each model's synthetic traces, we build a motif set (see Reetion 3.Tp. Given real motif set from the validation traces Sz, for each synthetic motif set Sg. we compute "Validation Motifs". (VM), the fraction motifs found in the validation motif set that are present in the synthetic motif set, VM/lSt This metric quantifies how good our synthetic motif set is (e.g., are its motifs mostly similar to motif' found in real traces). We also compute metrics related coverage, the fraction of motif' in the validation motif that are found in our synthetic data, defined as VM/|Sz|. This gives a sense of the breadth in a more traditional manner. To compare actual distributions of motifs (not just counts), we compute the MSE between the distribution of real motifs Sg and the distribution of synthetic motifs Sg. This gives a measure about how close the synthetic motif distribution
real one. We want high VM and coverage, low MSE. Results are in Table I with additional analysis in Appendix D overall our model provides the best breadth.
7.3 Utility
We evaluate synthetic glucose traces for use in glucose forecasting task using the common paradigm TSTR (Train on Synthetic, Test on Real), in which the synthetic data is used to train the model and then tested on the real validation data. We train an LSTM network optimized for glucose forecasting tasks (28) and report the Root Mean Square Error (RMSE) in Table [] We run the experiment 10 times and train the LSTM for 10,000 epochs. We have also tested with other models including RNNs, sirention-based models and other LSTM architectures (such as bidirectional LSTMs) but show the results for the best performing model. the LSTM optimized for glucose forecasting. Since RMSE provides a limited view about the model's predictions, we also plot the Clarke Error Grid (29, which visualizes the differences between predictive and reference measurement, and is basis for evaluating the safety of diabetes-related medical devices. More details are in Appendix F GlucoSynth provides the best forecasting results compared to all other models across all privacy budgets.

---

Table 1: Fidelity, Breadth and Utility Evaluation. Fidelity: bolded values do not have a statistically significant difference from the real data (what we want). Breadth and Utility: VM = fraction found validation motifs; We want high VM, Coverage and low MSE, RMSE; Bolded values indicate the best ones at each privacy budget (nonprivate compared with private models when o0).
Fidelity (metric. p-val) Time-in-Range
Breadth Coverage
Utility RMSE
Model
Varionce
VM
MSE
0.01
2576, <le-5
61.8, 2c-5
1,000
0.010
99.0
0.038 + 3e 4
0.1
2809, 0,356
60.1, 0.532
1.000
0.083
11.2
0.036 = 3e 1
GlucoSynth
2761, 0.022
60.6, 0,410
0.992
0.145
6.7
0.030 1e-4
10
2801, 0,316
60.2, 0.845
1.000
0.167
5.0
0.029 + 1e 4
2812, 0.503
60.2. 0.682
0.987
0.534
1.6
7e-
TimeGAN
2235. Se-3
62.3, 0.420
0.625
6e-3
107.7
0.061 1 Se-4
FF
2836, 0.902
46.6, <le-5
0.642
0.405
2.0
0.038 1 3e-4
NVP
00
1789, <le-5
65.3, <1e-5
0.482
0.328
1.9
0.029 + 36-5
0.01
57, <1e-5
78.8, <1e-5
0.013
1e-3
108.6
0.819 = 0.010
0.1
53, <le-5
71,6. 3e-5
0.015
0.031
107.3
0,688 1 Ge-3
RGAN
67, <le-5
78.2, <1e-5
0.015
0.033
103.3
0.651 + 0.018
10
77, <le-5
83.7, <1c-5
0.017
0.053
100.3
0.619 + 0.016
90, <le-5
78.0, <1e-5
0.026
0.091
79.6
0.460 1: 0.013
0.01
451, <1e-5
95.3.<1e <1e.5
0.094
0.054
180.1
0.205 1 5
0.1
1057, <1e-5
86.4, <1e-5
0.390
0.195
28.9
0.015 + 2
dpGAN
1
875, <le-5
86.6, <le-5
0.480
0.239
23.2
0.030 1 2--5
5
88.1,
0.743
16.1
5
1121, <
81.8, <
0.855
0.293
10.9
0.028 + 5
0.0006
0.0006
Diarbbutinn Value
Diatrbution Valuee
0.0044
Synthetic
0,0004
Syntheti
0.002
0.0002
0.0000
0.0000
5000
10000
15000
20000
5000
10000
15000
20000
Veriance
Veriance
(a) GlucoSynth
(b) TimeGAN
0.0006
0.0006
Distrbution Value
Real
Distrbution Value
Real Syrthelic
9 9.0044
Syrthalic
0.0004
00002
0.0002
0.0000
0.0000
500G
10000 Variance
20000
5000
10000 Variance
20300
(C) FF
(d) NPV
Figure Distributional Variance for Nonprivate Models
8 Limitations & Conclusion
Limitations. In order to train on a huge set of glucose traces, we used a private dataset, not publicly available (one of the motivations for this project was actually to share synthetic version of these traces). That being said, smaller samples
glucose traces with similar patient populations are available at OpenHumans (30] and TID Exchange Registry BT. In addition,
of the reasons privacy results perform well is because we use two separate datasets for the training of the motif causality block and the GAN. However, this may be limiting factor for others that do not have large enough set of traces available to be able to train adequately on partitioned data.
Conclusion. In this paper we have presented GlucoSynth, a novel GAN framework with integrated differential privacy to generate synthetic glucose traces. GlucoSynth conserves motif relationships within the traces, in addition to the typical temporal dynamics contained within time series. We pre- sented a comprehensive evaluation using 1.2 million glucose traces wherein our model outperformed all previous models across three criteria of fidelity, breadth and utility.
10

---

Acknowledgments and Disclosure of Funding
This work was supported in part by the U.S. National Science Foundation under Grant CCF-1942833 CNS-22 BNS-2213700, CNS- ENS-2220033, OAC-2319988 and by a Graduate Research Fellowship under Grant No. 1842490. This work was also supported by a Dexcom Graduate Fellowship.
References
[1] Katherine Britton and Jennifer D Britton-Colonnese. Privacy and security issues surrounding the protection of data generated by continuous glucose monitors. Journal of Diabetes Science and Technology, 11(2):216 219, 2017.
[2] Deborah Young-Hyman, Mary De Groot, Felicia Hill-Briggs, Jeffrey S Gonzalez, Korey Hood, and Mark Peyrot. Psychosocial care for people with diabetes: position statement of the American Diabetes Association. Diabetes Care, 39(12):2126-2144 2016.
[3] Ian Goodfellow, Jean Pouget-Abadie Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(1 1):1391144, 2020.
[4] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. Advances in Neural Information Processing Systems, 2019.
[5] Xiaomin Li, Vangelis Metsis, Huangyingrui Wang, and Anne Hee Hiong Ngu. TTS-GAN: A Transfomerrbasedd Sime-Series Generative Adversarial Network. In Martin Michalowski. Syed Sibte Raza Abidi, and Samina Abidi, e Intelligence in Medicine, pages 133-143, Cham, 2022. Springer International Publishing.
[6] Amirsina Torfi, Edward Fox, and Chandan K Reddy. Differentially private synthetic medical data generation using convolutional GANs. Information Sciences, 586:485-500, 2022.
[7] Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, pages 424-438, 1969.
[8] Cynthia Dwork. Differential privacy: A survey of results. In International Conference on Theory and Applications of Models of Computation. Springer, 2008
[9] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real NVP. arXiv preprint arXiv: 1605.0880 2016.
[10] Eoin Brophy, Zhengwei Wang, Qi She, and Tomas Ward. Generative adversarial networks in time series: survey and taxonomy. arXiv preprint arXiv:2107.1 1098, 2021.
[11] Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional GANs. arXiv preprint arXiv:1706.02633. 2017.
[12] Ahmed Alaa, Alex James Chan, and Mihaela van der Schaar. Generative time-series modeling with fourier flows. In International Conference on Learning Representations, 2021.
[13] Liyang Xie, Kaixiang Lin, Shu Wang, Fei Wang, and Jiayu Zhou. Differentially private generative adversarial network. arXiv preprint 10Xiv:1802.066.39, 2018
[14] Lorenzo Frigerio. Anderson Santana de Oliveira, Laurent Gomez, and Patrick Duverger. Differ- entially private Generative Adversarial Networks for time series, continuous, and discrete open data. In IFIP International Conference on ICT Systems Security and Privacy Protection, 2019.
[15) Cynthia Binanay, Robert Califf, Vic Hasselblad, Christopher M O'Connor, Monica R Shah, George Sopko, Lynne W Stevenson, Gary S Francis, Carl V Leier, Leslic W Miller, al. Evaluation study of congestive heart failure and pulmonary artery catheterization effectiveness: the ESCAPE trial. JAMA, 294(13): 1625-16333, 2005.
[16] George EP Box, Gwilym Jenkins, Gregory C Reinsel, and Greta M Ijung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.

---

[17] Lexiang Ye and Eamonn Keogh. Time series shapelets: new primitive for data mining. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2009.
[18] Helmut Lütkepohl. New introduction to multiple time series analysis. Springer Science & Business Media, 2005.
[19] Stephen A Billings. Nonlinear system identification: NARMAX methods in the time, frequency, and spatio-temporal domains. John Wiley & Sons, 2013.
[20] Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. A review of Recurrent Neural Networks: LSTM cells and network architectures. Neural computation, 31(7):1235-1270) 2019.
[21] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In ACM SIGSAC Conference Computer and Communications Security, 2016.
[22] Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. Scalable private learning with PATE. arXiv preprint arXiv:1802.08908 2018.
[23] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography Conference, 2006.
[24] Martín Abadi et al. TensorFlow: Large-scale machine learning on heterogeneous systems. atttps://wrr http://wwwwtttsollooo..ooo org/
2015.
[25] Ali Borji. Pros and cons of GAN evaluation measures; New developments. Computer Vision and Image Understanding, 215: 103329,,,, 2022.
[26] Halis Kaan Akturk, Robert Dowd, Kaushik Shankar, and Mark Derdzinski. Real-world evidence and glycemic improvement using Dexcom G6 features. Diabetes Technology & Therapeutics, 23(S1):S 21, 2021.
[27] Fred B Bryant and Paul R Yarnold. Principal-components analysis and exploratory and confir- matory factor analysis. American Psychological Association, 1995.
(28] Taisa Kushner, Marc D Breton, and Sriram Sankaranarayanan. Multi-hour blood glucose prediction in Type diabetes: A patient-specific approach using shallow neural network models. Diabetes Technology & Therapeutics, 22(12):883-81,, 2020.
[29] William L Clarke The original Clarke error grid analysis (EGA). Diabetes technology & therapeutics, 7(5):776-779, 2005.
[30] Open humans.
org/ 2023.
[31] T1D Exchange Registry.
org/registry/ 2023.
[32] Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. arXiv preprint arXiv:2208.09:3 2022.
[33] Hao Ni, Lukasz Szpruch, Magnus Wiese, Shujian Liao, and Baoren Xiao. Conditional Sig- Wasserstein GANs for time series generation. arXiv preprint 2rXiv:2006.0542), 2020.
[34] Zinan Lin, Alankar Jain, Chen Wang, Giulia Fanti, and Vyas Sekar. Using GANs for sharing networked time series data: Challenges, initial promise, and open questions. ACM Internet Measurement Conference, 2020.
[35] Mihai Dogariu, Liviu-Daniel Ştefan, Bogdan Andrei Boteanu, Claudiu Lamba, Bomi Kim, and Bogdan lonescu. Generation of realistic synthetic financial time-series. ACM Transactions Multimedia Computing. Communications, and Applications, 18(4):1-27, 2022.
[36] Mina Razghandi, Hao Zhou, Melike Erol-Kantarci, and Damla Turgut. Variational autoencoder generative adversarial network for synthetic data generation in smart home. arXiv preprint arXiv:220.003387, 2022.
12

---

[37] Debapriya Hazra and Yung- Tong-Cheol Byun. SynSigGAN: Generative Adversarial Networks for synthetic biomedical signal generation. Biology. (12):444, 2020.
[38] James Jordon, Jinsung Yoon, and Mihaela Van Der Schaar. PATE-GAN: Generating synthetic
data with differential privacy guarantees. In International Conference on Learning Representa- tions, 2018.
13

---

A Extended Related Work
We overview related work in three lines of research: time series, conditional time series, and time series methods that employ differential privacy. Iublaiiis previous time series synthesis methods. We note that there have been exciting developments for adjacent research tasks (data augmentation, forecasting) such as diffusion models [32], but there not yet any publicly available models specifically for the generation of complete synthetic time series datasets. As such, we focus the scope of our comparison on the current state-of-the-ee methods for synthetic time series which all build upon Generative Adversarial Networks (GANs) [3] and transformation-bee based approaches [9]. In particular TimeGAN [4) RGAN [11] and dpGAN [44 are most similar ours and used as benchmarks in the evaluation in Section
Table Summary of Previous Methods for Time Series Synthesis. *CI = conditional information or extra features
Name TimeGAN
Private?
No Labels Required?
No CI*?
Length
24 58
TTS-GAN
5
x
V
24 - 150
SigCWGAN
X
80,000
RGAN
V
16- 30
RCGAN
X
16-30
dpGAN [4
V
96
RDP-CGAN
x
4097
DoppelGANger gger
34
x
50 600
GlucoSynth (Ours)
288
Time Series. There have been promising models generate synthetic time series across variety of domains such as financial data (35. sybber-physical systems (c.g.. smart homes (36)), and medical signals [37]. Brophy al. [10] provides survey of GANs for time series synthesis. TimeGan is popular benchmark that jointly learns an embedding space using supervised and adversarial objectives order to capture the temporal dynamics amongst traces. TTS-GAN [. trains GAN model that uses transformer encoding architecture in order to best preserve temporal dynamics. Transformation- based approaches have also had success for time series data. Real-valued non- volume preserving transformations (NVP) [9 model the underlying distribution of the real data using generative probabilistic modeling and use this model to output a set of synthetic data. Similarly, Fourier Flows (FF) 12 transform input traces into the frequency domain and output a set of synthetic data from the learned spectral representation
the original data. Methods that only focus on learning the temporal or distributional dynamics in time series are not sufficient for generating realistic synthetic glucose traces due to the lack of temporal dependence within sequences of glucose motifs.
Conditional Time Series. Many works have developed time series models that supplement their training using extra features or conditional data. Esteban al. [1 develops two GAN models (RGAN/RCGAN) with RNN architectures, conditioned on auxiliary information provided at each timestep during training. SigCWGAN [33] uses a mathematical conditional metric (Sig - Wi) characterizing the signature of a path to capture temporal dependence of joint probability distributions long time series data. However, our g glucose traces do not have any additional information available so these methods cannot be !
GANs. To protect sensitive data, several GAN architectures (DP GANs) have been designed to incorporate crivacy-rreservig noise needed to satisfy differential privacy guarantees 13 Although DP GANs such as PateGAN 38 have had great success for other data types and learning tasks (e.g., tabular data, supervised classification tasks), results have been less satisfactory in DP GANs developed for time series.
Differentially-Priie
RGAN/RCGAN II also includes a DP implementation, but the authors find large gaps in perfor- mance between the nonprivate and private models. Frigerio et al. [14] extends simple DP GAN architecture (denoted dpGAN) to to time-series data. The synthetic data from their private model
'There is caveat here that RGAN does not use auxillary information, hence why we compare with it in our benchmarks.
14

---

Figure 7: Example motif causality matrix for small motif set (m 10). Each value in the grid is between and I. O indicates no motif-causal relationship, and indicates the strongest motif causal relationship.
conserves the distribution of the real data but loses some of the variability (diversity) from the original samples. RDP-CGAN [ develops convolutional GAN architecture that uses Rényi differential privacy specifically for medical data. Across different datasets, they find that reasonable privacy budgets result in major drops in the performance of the synthetic data. Finally, DoppelGANger [34] develops a temporal GAN framework for time series with metadata and perform an in-depth privacy evaluation. Notably, they find that providing strong theoretical DP guarantees results in destroying the fidelity of the synthetic data, beyond anything feasible for use in real-world scenarios. Each of these methods touches on the innate challenge of generating DP synthetic time series due to very high tradeoff's between utility and privacy. Our DP framework uses two different methods to integrate privacy into GAN architecture, resulting in better atility-privacy trade-off than previous methods.
B Additional Experimental Details
Note on Data Use. As explained in the approach (Section 5). our model uses two separate datasets for the training of the motif causality block and the rest of the GAN. As such, used two different samples of glucose traces with no overlap between patients for the training of each section (meaning actually used a total of 2.4 million traces across the entire model). We also note that have received the proper ethical and legal consent from the individuals to use their data in this way (and for this purpose).
Hyperparameters. Our experiments were completed the Google Cloud platform on an Intel Skylake 96-core cpu with 360 GB of memory. We use separate validation dataset (not the set of original training traces) for all experimental results. Throughout all our experiments we use GlucoSynth model paraneters of a 0.1 and 77 10 and motif tolerance of a 2 mg/dL and motif length 7 = 48. Motif length of 48 timesteps is equivalent to hours of time and represents a clinically significant threshold. This threshold was chosen because the effect of any behaviors glucose occur within 4 hours of the event (e.g.. the effect from eating a meal a rise in glucose - will occur within 4 hours after eating.) We note that other choices for 7 could be used, based what types of phenomenon the users wish to replicate; for example, to capture day/night glucose rhythm effects, suggest of 144, corresponding to 12 hours of time.
vary our privacy experiments, but keep 8 the same at 5e-4. Importantly, order to meet our privacy guarantees, we assume that privacy is provisioned at the trace level and each individual has only one trace in the dataset. The motif set is derived separately from the training data (cither from public dataset generated based on knowledge about the underlying data, e.g., the possible glucose motif combinations), so as not to effect the differential privacy guarantees or use up any privacy budget. In our case, we assume the motif set is all-encompassing and generated from the universe possible motifs, resulting in m = 5, 977, 610 total motifs in the motif set.
4 is implemented from www setthbb.com//ssyyo002333 12 are implemented from www www.github.. cww www.aammmmmaaaaaaaa.a Fourier-fll floss; RGAN [11 is implemented from www.github wwtt://///accccaa///a
Benchmark Details. TimeGAN
TimeCAN Fourier Flows (FF)
and DPGAN [14] is adapted from www. www.biithub... www/SAP-
All the benchmarks were trained according to
15

---

their suggested parameters, with most models trained for 10,000 epochs. We note that we trained for more than the suggested epochs (50,000 instead of 10,000) and tried many additional hyperparameter settings for RGAN to attempt to improve its performance and provide the fairest comparison possible.
Epsion:
Epston: 10
Epsilo:
Epsilon: 0.01
100
400
50
350
80
300
(T) SRIS) asconig
70
Indiived aM
68
250
50
200
40
80
20
Figure Heatmaps GlucoSynth Across Different Privacy Budgets
Epsllon: Epellon: 10
Epsilon:
Epsilon:
100
90
350
300
70
250
50
200
30
150
20
100
10
08388328888
0 0*888228888
Time
Time
Time
Time
Time
Figure 9: Heatmaps for RGAN Across Different Privacy Budgets
Real
Epsilon: in!
Epalion 10
Epsilo:
Epaiion: 0.01
160
400
90
350
80
(Te socomn
70
Indiv
60
250
200
403
203
150
20
100
10
Time
Time
Time
Time
Time
Figure 10: Heatmaps for dpGAN Across Different Privacy Budgets
C Additional Evaluation: Fidelity
C.1 Visualizations
We provide heatmap visualizations of sample real and synthetic glucose traces from all the models. Although this not comprehensive way to evaluate trace quality, does give snapshot view about how the synthetic traces compare to the real ones. Each heatmap contains 100 randomly sampled
16

---

glucose traces. Each row is a single trace from timestep 0 to 288. The values (coloring) in each row indicate the glucose value (between 40 mg/dL and 400 mg/dL). Figure[Jshows the nonprivate models, and Figures 8 TO show the private models with different privacy budgets. Upon examinging the heatmaps, we notice that GlucoSynth consistently generates realistic looking glucose traces, even at very small privacy budgets.
C.2 Population Statisties
In order evaluate fidelity on population scale, we compute a common set of glucose metrics used to evaluate patient glycemic control on the real and synthetic data, including average trace variability (VAR). Time in Range (TIR), the percentage of time glucose within the clinical guided range of 70- 70-180mg/dl and time hypo- and hyper- glycemic (time below and above range, respectively) in We test if the difference in metrics between the synthetic and real data is statistically significant, using a p-value of 0.05. A p-value <0.05 indicates the difference is statistically significant. We want synthetic data that has similar population statistics to the real data: p-values > 0.05 such that the differences in statistics between real and synthetic data are not significant. GlucoSynth outperforms
Table More details on each of the metrics are included in Table 3
other models, with no statistically significant difference in all metrics for privacy budgets of c > 100 and only one metric 10.
with a statistically significant difference for budgets
Table 3: Glycemic Metric Explanations
Metric
Name
Explanation
VAR TIR
Signal Variance
average trace variability % of time glucose > 70 & < 180 % of time glucose < 70
Time in Range
Hypo
Time Hypoglycemic
Hyper
Time Hyperglycemic
% of time glucose > 180
GVI
Glycaemic Variability Index
more detailed measure glucose variability
PGS
Patient Glycaemic Status
metric combining GVI and TIR
Table 4: Population Data Statistics. Each cell value for the synthetic data shows the (metric, p-value) using a 0.05 testing threshold. Bolded values do not have statistically significant difference from the real data (what we want).
Model
VAR
TIR
Hypo
Hyper
GVI
PGS
Real
1.58
4.03
0.01
2575.501, 0.0
61.759. 2.0c-5
1.331, 0.0
36.91, 5.66c
4,002, 0.085
323,056, 0.0
0.1
2803.513, 0.356
60.088, 0.532
1,264,
38.648, 0.137
3,969, 2.74e-
347.562, 0.712
GlucoSyath
2730.853 0.022
60,597, 0.41
1.512, 0.163
37,892, 0.537
4.019, 0.577
345.159, 0.368
2800.805, 0.316
60.24. 0.845
1,538, 0.395
38.222, 0.76
3,963, 6.
344.376, 0.28
100
279,.424, 0.244
60.138, 0.625
1,567, 0.808
38,295, 0.609
4,044, 0.32
352,679, 0.449
1.622, 0.503
60.165, 0.682
1.54,
38.295, 0.61
4.056,
353.584, 0.339
TimeGAN
2234,576, 8.08e-3
62.315, 0.42
0.657, 8.2336-3
37,028, 0.669
5.482,
503.148, 0.2c-5
FF
00
2836.066,, 0.902
46.378, 0.0
5.627, 0.0
47.795, 0.0
4,931, 0.0
$28.775 773, 0.0
NVP
00
1789.430,
65.499, 0.0
1,507, 0.154
32.994, 0.0
6.607,
589.473, 0.0
0.01
56.96, 0.0
78.756, 0.0
1.78e-4
21.244, 0.0
2.52, 0.0
93.409, 0.0
0.1
52.553, 0.0
71.617. 3.7m-5
0.0, 1.78e-4
25.715, 0.0
2.208. 0.0
98 944, 0.0
RGAN
67.346,
78.154, 0.0
1.78c-4
21,846. 0.0
2,251, 0.0
85.417, 0.0
76.632.0
83.681, 0.0
0.0, 1.78c-1
16,319, 0.0
2,23, 0.0
64.562.
100
84,918,
74.285, 0.0
1.78e-4
25.715, 0.6e-5
2,208, 0.0
98 944. 0.0
00
89,702,
78.044, 0.0
0.0, 1.78e-1
21.956, 0.0
2.184.0.0 0.0
82.923,
0.01
451.098. 0.0
95.275. 0.0
4.60. 0.0
0.124,
7.718.0.0 0.0
41.549, 0.0
0.1
1057.208 207. 0.0
86443, 0.0
0.837, 0.0
12.732, 0.0
6.349, 0.0
148.412, 0.0
dpGAN
874.663, 0.0
86.631. 0.0
1.135, 0.0
12.234,01
4.794, 0.0
118.286, 0.0
1029.971. 0.0
88.122, 0.0
2.002, 0.0
9.876.
4.759.0.0
93.632.
100
821.636.0 0.0
89.354, 0.0
0.664, 0.0
9,982.
4.613,
82.561,
0C
1120.553, 0.0
81.773, 0.0
1.359, 0.3e-5
16.868, 0.0
6.248, 0.0
188.991. 0.0
17

---

10.00
1000
800
-300
130
1968
-030
2000
1008
1900
2009
680
400
1300
Component
a GlucoSynth
(b) TimeGAN
Epelber In
Epiiion
1200
50
-200
100
ComponinL
1300
2000
2008
-330
-100
1X3
Compannn
2000
3002
4000
1000
(c) FF
(d) NPV
Figure 11: PCA Comparison for Nonprivate Models
C.3 Distributional Comparisons
We visualize differences in distributions between the real and synthetic data by plotting the distribution of variances and using PCA (27). Figure 6 and Figure II show the variance distribution and PCA plots, respectively for the nonprivate models. We also compare distributional changes across privacy budgets: Figures 12 and show GlucoSynth, Figures 14 and 15 show RGAN and Figures 16 and 17 show dpGAN
Looking at the figures, GlucoSynth better captures the distribution of the real data compared to all of the nonprivate models. As evidenced in the PCA plot, (Fig. []]. FF comes the closest to capturing the real distribution in its synthetic data, but ours does a better job of representing the more rare types of traces. GlucoSynth also outperforms all of the private models across all privacy budgets. Even at small budgets (e < 1), the general shape of the overall distribution is conserved (c.g., see Figure 12
D Additional Evaluation: Breadth
Compared to all other models across all privacy budgets, our model has the best ratio of found validation motifs, with close to 1.0 for VM and the lowest MSEs. It also has the best coverage for nonprivate settings and an of 100. Interestingly, dpGAN has the best coverage compared to all other models for privacy budgets c < 10 but worse MSES across all budgets than GlucoSynth. This means that although it finds broader number of motifs contained in the real data, the overall distributions of motifs creates the synthetic data have much higher error rates. We argue that the tradeoff found by our model is better because although it does miss some of the types of motifs from the real data (misses some breadth), from the ones it does find it constructs realistic distributions of the motifs and generates very few anomalous ones.
18

---

sonde
Pystrat
00000
10:00
17h
0:0000
30:30
000000
2000
30000
soves
000000
30000
1000
0:0000
20:00
10:30
0800
10:00
0.0000
020000
0.0000
030015
0.0015
0.30000
0.000
0:300
030000
1200
1:00
12000
0.0000
5000
Figure 12: GlucoSynth Distributional Variance Comparison Across Privacy Budgets
Comprner
Figure 13: GlucoSynth PCA Comparison Across Privacy Budgets
19

---

Pystrat
8.0
FR0D
1000
20:8
1900
12
07
3038
FRO
1900
9000
20:8
1900
1900
03
3006
7900
1000
17900
1390
20:0
1100
Figure 14: RGAN distributional Variance Comparison Across Privacy Budgets
Spaics
Figure 15: RGAN PCA Comparison Across Privacy Budgets
20

---

Fpeiber:
0300
Symmanc
0300
0300
0:000
0305
0308
6300
0000
2030
2013
1900
0.000
4010
9100
Vode
030/
0.000
0000
0.000
0000
0.00
16:00
1200
2004
12:00
250.8
2400
2000
Epoller:
0000
6:00
0208
12500
1730
Figure 16: dpGAN distributional Variance Comparison Across Privacy Budgets
Comprnst
Traler
Eysion
200
Figure 17: dpGAN PCA Comparison Across Privacy Budgets
21

---

(a) GlucoSynth
(b) TimeGAN
(c) FF
(d) NPV
(e) RGAN
(I) dpGAN
Figure 18: Clarke Error Zone Figures for All Models
E
Additional Evaluation: Utility
Since RMSE may provide a limited view about the predictions from the glucose forecasting model, also plot the Clarke Error Grid (29, which visualizes the differences between predictive measurement and reference measurement, and is the basis used for evaluation of the safety of related medical devices (for example, used for evaluating glucose outputs from predictive models integrated into artificial insulin delivery systems). The Clarke Error Grid is implemented using www. git.ub........... com/ suet weaddiee/Clakeeeee (Claarerrrrrrric The grids shown in Figure 18
the figures, the x-axis is the reference value and the y-axis is the prediction. A diagonal line means the predicted value is exactly the same the reference value (the best case). There are 5 total zones that make up the grid, listed order from best to worst:
Zone A Clinically Accurate: Predictions differ from actual values by no more than 20% and lead to clinically correct treatment decisions.
Zone B - Clinically Acceptable: Predictions differ from actual values by more than 20% but would not lead to any treatment decisions.
Zone Overcorrections: Acceptable glucose levels would be corrected (overcorrection).
Zone Failure to Detect: Predictions within the acceptable range but the actual values are outside the acceptable range, resulting in failure to detect and treat errors in glucose.
Zone E Erroneous Treatment: Predictions opposite the actual values, resulting in erroneous treatment, opposite of what clinically recommended.
We show Clarke Error grids for all models (and the private models with no privacy included, c = oo). This is because comparing the models at different privacy budgets is not very informative - can be hard to tell exactly where changes between different budgets may occur. We also present table with the percentages of predicted datapoints in each category in T This table includes a comparison among different privacy budgets for the private models (much more effective than the figures by themselves.)
22

---

Table 5: Clarke Error Grid Zones. Value is the percentage of predicted datapoints. Categories go from E, best worst. Bolded rows indicate the best results on the synthetic data each privacy budget (nonprivate models compared with private models when 00)
Model
A: Accurate
B: Acceptable
C: Overcorrection
D: Failure to Detect
E: Error
0.858
1.077-2
0.131
1.172e-3
3.271e-3-
0.017 1.158e--
5.79e-6i +
0.1
0.863
6.917e-3 3
0.126
7,526e-4
3.05de-3 2 1545--5
0.018
4.3ee-5
5.78e-8 0.0
GheoSynth
1
0.862
1.578e-3
0.128
1 250e-3
3
3 3.945e-3 # 45e-5
0.016 2 S 329929-44
5.79e-6 : 0.0
10
0.864 6.947e-3
0.125
6.513e-
4
3.039e-3 5.79--5
0.017 4.34e-5
8.68e-6 2.89e-5
0.864 1.74e
1.447e
0.017
0.964 8
= 1.201--3
0.035
1.158e-8
3 3.039e-4
89e-6
1 1.732e-4 :: 1.158e-4
8. 8.688--6 6 1.45e-6
TimeGAN
0.741 0.012
0.233 0.012
2.240l--
9.8e-5
0.024 8.44e-4
2.19c-4 4: 1.9e-5
00
0.824
6.624-33
0.136 6:188--
8.547--3 9.0e-5
0.017 : 3.940e4
3.57e-4 8.0e-6
NVP
o0
0.79
3.00c
0.186 a 3.87e-4
3.49c 4: 1.5e-5
0.02 # 1.0ee-4
3.58c 4: 5.0c-6
9.389c-4 1.197--4
0.024 2.71e-1
2.42% 3.43c-5
0.1
0.501
1.098--
0.38
1.74e-3
1.320i=-3 1.129c-4
0.022 =: 1.009e1
2.873 1 6.08e-6
RGAN
1
0.637
3: 6. 785c-
3.452t-3
0.336 1: 128c 3
3.2476-
1.004c-4 2.253e-3-
2.661e 3 1.87e-
0.024 :: 6. 464e
3.07 1.710-6
2.792 4 2.95c 5
100
0.638
709c-3
0.333
1.219e
1.991-33
2.17e-5
0.025 = 1, 88 11
29/9 2.20e
50
0.636
+ 6.85c
0.326 +: 7.19c-
2.613e-3 3 2.852e-4
0.024 : 3.006c
2,85% 4 1.5e-5
0.308
0.500
3.80Me--4000
0.183 2.33--4
1.114 4.1966
0.1
0.781 =: 6.35e
0.191 = 5.37e-3
8.220ic-3 5
0.024 1: -5
2,53% 1 1.881-6
dpGAN
1
0.786
= 5. Ie
0.187 = 5 818-1
2.409. -: 2.89
0.024 + 65
2.78- 1 + 5.787e-7
10
0.806
+ 7.34e
0.169 1: 6. 006c
2.380c-311 1.470e-5
0.023 1.113
2.16fc 2.749c-
0.161
2.86
2.208c
1888
1.013e
00
0.818 +: 4
0.36 + 1 306e -3
3.193
3: 2.67Te-5
0.018 : 1.60 1.60--4
3.166
+ 5 5.2086-4
Looking the grids, can see that GlucoSynth performs the best, with most of the values along the diagonal axis (Zone A and B) and less around the other zones (Zones C-E) as compared to the other models. This means that most of the predicted glucose values from the model trained on our synthetic data are in the Clinically Accurate and Acceptable ranges, with less in the erroneous zones. Moreover, by examining the table we see that GlucoSynth outperforms all other models across all privacy budgets as well.
23


Reviews:
[{"review_id": "HRR4tMP7PH", "reviewer": "Reviewer_Q4qV", "summary": "In this paper, the problem of generating differentially private synthetic glucose traces using a GAN framework, by conserving the relationships among glucose events or motifs within the traces, as well as the temporal dynamics.", "questions": "- More detailed privacy proofs needed.\n- In section 4.2, it is stated that $\\tau = 48$ is used, corresponding to 4 hours of time. Is this a common $\\tau$ value or does this just work for the experiments in this paper.", "limitations": "The authors discuss the limitations of the work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper presents GlucoSynth, an approach that generates private and high-utility synthetic data. The method combines some existing ideas in a clever way. The work is interesting and the results show an improvement to the previous literature on the topic. ", "weaknesses": "- The major issue is that more information is needed for proving the privacy guarantees offered by this method. In section 6, the addition of differential privacy is discussed, however, the explanation is brief and more rigorous proofs are needed.\n\n- Moreover, some more details are needed when explaining the motif causality block in section 4.2. For instance, when discussing $\\tau$ as a hyperparameter, then stating that \"we use $\\tau = 48$, corresponding to 4 hours of time\", is this related to the experiments or why is this used here?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and review. For our privacy integration, we use a straightforward application of prior differential privacy methods to our framework. Since we have not developed any new differential privacy mechanism, and are using established privacy techniques in a standard way, we do not believe a formal proof is required. \n\nRelated to the comments about needing more details for the motif causality block, we provide additional justification for the hyperparameter selection (including for $\\tau$) in the Appendix B. $\\tau = 48$ was chosen because it is a clinically significant threshold: the effect of any behaviors on glucose occur within 4 hours of the event (e.g., the effect from eating a meal – a rise in glucose – will occur within 4 hours after eating.) If there are other questions about the motif causality block, we are happy to answer them."}, {"review_id": "w4M20c4Qkv", "reviewer": "Reviewer_yHP4", "summary": "This manuscript presents GlucoSynth to generate synthetic glucose traces. The main idea is to conserve relationships among motifs within the time series and to preserve the privacy of the glucose traces. Specifically, the manuscript proposes a motif causality block to learn the causal relationships amongst the motifs and incorporates differential privacy into the framework. The experiments on the clinical dataset demonstrate that the proposed method outperforms the baselines in terms of fidelity, breadth and utility.", "questions": "1. Can the temporal dependencies learned by the model be visualized or aligned to medical knowledge? Are they approved by healthcare experts, e.g., doctors?\n2. As descirbed the Sec. 7.3, an LSTM is trained for glucose forecasting. How many times does the model run? Have you tested with other models? Are there any differences between LSTM and other models?\n3. What about the scalabilty of the proposed model? For instance, do you test the performance of GlucoSynth on a small dataset? How much samples are required at least for training GlucoSynth usable? Have you evaluate the models on other datasets, as mentioned in Limitations?", "limitations": "1. The authors can adequately recognize the limitations of this work. It is worth emphasizing that the motivation is not supported well by the statistics and the experiments. Besides, the scalabilty of the model is not explore.\n2. There is no significant potential negative societal impact of this work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The topic of this manuscript is important. Synthesizing data for sharing is meaningful for collaboration with healthcare institutions.\n2. The challenge of generating synthetic glucose traces identified in this manuscript is interesting, i.e., the manuscript claims that \" they are more event-driven than many other types of time series.\"\n3. The manuscript designs several evaluation metrics to compare GlucoSynth and previous studies.", "weaknesses": "1. The distribution depicted in Figure 2 is not able to support the motivation (glucose traces are event-driven) of this work. It would be better to present the distributions of glucose traces under a subtype of patients, if the distributions are very diverse within the whole population.\n2. The contributions of the method are two-folds, i.e., motif causality block and DP. It's not clear how each module contribute to the whole model through the experiments. A well-designed ablation study would be better.\n3. The organization of the manuscript should be improved. Several important results are in Appendix, while the method is too long.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and review. Responding to Weakness 1: The main point of Figure 2 was to show that glucose motifs are not temporally-dependent, and provide insight about why methods that only focus on conserving temporal relationships fail. The motivation of the work was to develop a method that produces high-quality synthetic glucose traces. As a way to explain perhaps why other methods fail to produce high-quality synthetic traces, we describe glucose characteristics and how they are driven by behaviors (a clinically-supported fact) and not temporally dependent in Section 3.2 using Figure 2.\n\nResponding to Weakness 2: We thank the reviewer for this suggestion. Due to space constraints we decided not to include an ablation study and instead focused on comparing and evaluating the overall framework performance. That being said, this would be a nice addition for the Appendix.\n\nResponding to Weakness 3: We thank the reviewer for this suggestion. Since we have an unusual architecture, we wanted to make sure enough details were provided about the methods, and due to space constraints, felt that putting some of the results in the appendix was a fair trade-off.\n\nResponding to Question 1: Yes. We plotted motif behaviors on graphs and worked with a group of diabetes clinicians during model development to ensure our behaviors were human physiologically possible and would not cause patient harm if used for medical tasks.  As a way to quantitatively test this, we plotted Clarke Error Grids in the Utility experiments (see Appendix E), which is a basis for evaluating the safety of diabetes-related medical devices.\n\nResponding to Question 2: Thanks for these interesting questions. We run the experiment 10 times and train the LSTM for 10,000 epochs. We have tested with other models including RNNs, attention-based models and other LSTM architectures (such as bidirectional LSTMs). We show the results for this particular glucose-focused LSTM architecture because it performs the best at the glucose forecasting task compared to the other architectures. That being said, even when using the other models we still find that the RMSE decreases as our privacy budget increases, and that GlucoSynth performs the best compared to previous methods. We will add these additional details to Appendix E.\n\nResponding to Question 3: Thanks for this important question. We have tested GlucoSynth on smaller datasets and, as is a common trade-off with differential privacy methods and generative models, find empirically that utility degrades as population sizes decrease. We are currently experimenting with population sizes vs. privacy trade-offs (e.g., what is the minimum number of traces needed to provide decent privacy guarantees). In addition, we are working on optimizing the model to make it more scalable."}, {"review_id": "R2zmiu7QTG", "reviewer": "Reviewer_VFZB", "summary": "The authors introduce a new differential privacy scheme where other, non-related signal recordings are used to anonymise the signal provided  that they have a motif that seems to predict the time signal forwards in time. For this a time dependent vector is used to contain multiple signals at the same time propagating forwards. Simple RN and LSTM are used. \n\nThe models are trained using differentially private stochastic gradient, and with an GAN discriminator component in the network\nit is ensured  that  resulting synthetic signal belongs to the same distribution as the actual signals.\n\nThe quality of the signal is measured with fidelity, breath, and utility estimates. According to these criteria the novel differential privacy preserving synthetic signals beat the current state of the art.", "questions": "Annex?\nI do not understand fully the role of the statistical tests in the evaluation. For fidelity, I would expect that it means that features that are needed to analyse the time trail are preserved, so that one can trust the diagnosis. As the system is generating the signal partly for other trails via the causal motifs, the generated signal may miss something essential, or it may have added components from the other trails. Checking with GAN that the signal is not easily distinguishable from the real traces is a good step and achievement, but at the same time one has to ensure that diagnoses from the original and anonymised trails remain the same for all cases individually - and report the error rate resulting from the anonymisation.\n", "limitations": "Could the method give different diagnostic results after anonymisation. This may be the no-go for the anonymisation. It should be clearly stated that the system is not designed to be used for the diagnostic purposes, but to publish a data set that can be used to train AI solutions - with an extra annotation made for large enough portion of the synthetic examples.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The manuscript is well written and concise explanation of the process. ", "weaknesses": "The he supporting material to me looks like a copy of manuscript. The paper refers to the Annexes? Please correct \n\nIn abstract, the statement below is a contradiction as indicates the novel privacy-preserving GAN framework does not exist, even the manuscript just presents it.  ( it is using GANs and it exists)\n\"Existing methods fo  time series data synthesis, such as those using Generative Adversarial Networks (GANs), are not able to capture the innate characteristics of glucose data and cannot provide any formal privacy guarantees without severely degrading the utility of the\nsynthetic data. In this paper we present GlucoSynth, a novel privacy-preserving GAN framework to generate synthetic glucose traces.\"\n\nI have a problem in understanding the evaluation of the performance. See the question I have in the question part. Please clarify.\n\n\n\n\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and review. Responding to the Weaknesses: Thanks for catching these corrections. We will correct the supplemental material submission and adjust the language in the abstract to read: \"Previous methods for time series data synthesis ... \" We have also responded to the question about the evaluation below.\n\nResponding to the Questions: We thank the reviewer for the opportunity to clarify this important part - the reviewer is correct in their statements, and we evaluated each of these aspects in the 3 parts of the evaluation. First, we used fidelity as a way to verify that on a population level, characteristics of the real data were conserved, i.e., did not change drastically between the real and the synthetic data. To do this, we tested if there was a statistically significant difference in clinically-relevant metrics between the synthetic and real data (if there was not, this is good). Secondly, we used the breadth evaluation to check about what parts of the traces the synthetic data missed, and what anomalous motifs it may have added to the signals (e.g., what components from other trails may have been added), as the reviewer described. Finally, we used the utility part to verify that synthetic traces could be used in place of the real ones (e.g., to ensure that diagnoses from the original and synthetic data were the same) by finding the error in a glucose forecasting task trained on the synthetic data and tested on the real data.\n\nResponding to the Limitations: This is a good point. It is not the authors intention that this synthetic data would be used for individual diagnostic purposes (and note that it would not need to be since there is not a privacy concern with an individual patient sharing their data to their doctor). "}, {"review_id": "DYVW4oDQf8", "reviewer": "Reviewer_2Sh2", "summary": "This paper proposes a DP generation model, i.e. GlucoSynth, particularly for glucose data. To address a few challenges that prior works fail in, GlucoSynth proposes a causality learning block, to learn the dependency between motifs, rather than merely learning the temporal dependencies. The generator is basically a GAN combined with autoencoder. By training GAN and autoencoder with DP-SGD, training causality block with PATE, GlucoSynth is claimed to achieve DP guarantees.", "questions": "1. What do different colors mean in Fig. 1 and 2? \n2. I don't get what Fig. 2 intends to mean. What is the relationship between motif 1 and 2? What should we expect to see from the radial graph?\n3. It is better to give full descriptions in the caption of figures, e.g. figure 3 and 4.\n4. Line 229, how is $M_{\\hat{x}}$ calculated from $M$ and $\\hat{x}$?\n5. What is the difference between MSE loss in Line 217 and 225?", "limitations": "This paper has discussed limitations in the end.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The problem is well-motivated, and the summary of challenges in prior works is clear.\n2. The idea of motif causality block is somehow novel, compared to prior works. (But I did not read too much related works in synthesizing glucose data, so this point is based on the summary in this work.)", "weaknesses": "1. Although Sec. 1 and 2 are clear, starting from Sec. 3, the writing is sometimes confusing. See my questions below for details.\n2. This method is not neat. Figure 3 clearly shows that you need to train a total of $m\\times r$ neural networks for the motif causality block, let alone the autoencoder and GAN in the next step. Given that the GAN and autoencoder are trained with DP-SGD, which introduces notorious time overhead due to the gradient perturbation, this method is both time- and space-inefficient.\n3. The main components in GlucoSynth lack proper explanation. For example, \n   + Why do you need to build motif networks for each motif? Can you build only one for all motifs? \n   + Why do you need an autoencoder? Do you have the ablation study on the autoencoder?\n4. In Line 223 and Figure 4, it seems that the generator needs real data as input to \"generate\" something. How can you use the generator in reality? \n5. The DP mechanisms applied in this work are not novel. Both DP-SGD and PATE are existing standard techniques to retain DP.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and review. Responding to Weakness 2: The reviewer is correct that our model is time and memory-intensive. Even so, since you only have to train the model once and then can use the produced synthetic data over and over, we believe this is an okay trade-off. We are also currently exploring methods to optimize the model to make it more efficient.\n\nResponding to Weakness 3(a) about motif networks: Thanks for the opportunity to clarify this important point. We build a motif network for each individual motif in order to learn an explicit mapping between all the other motifs and their predictiveness to a particular motif. If we trained all the motifs together using a single motif network, we would not be able to quantify the exact causal effects between each individual motif because we would not know which exact motifs helped with the prediction (only that there is some combination of unknown motifs that contribute to an accurate prediction of a particular motif.) By training each motif network separately, we are able to learn and quantify the exact effect each motif has on each other, without any confounding effects from other motifs.\n\nResponding to Weakness 3(b) about the autoencoder: We did do an ablation study for the autoencoder, and without the autoencoder the model fails to converge. We did not include these results due to space constraints, but we will add them to the appendix.\n\nResponding to Weakness 4: As is standard in GAN models, no real data is used to synthesize the synthetic data. As described on line 220, to generate synthetic data a random vector of noise $z$ is fed through the embedded generator and then the recovery network to produce the synthetic traces (see Figure 4 and the vertical path that goes from $z$ through Generator\\_e and Recovery networks to $\\hat{x}$).\n\nResponding to Weakness 5: Yes, this is true. We believe the novelty of our architecture comes from the overall GlucoSynth framework and the use of motif causality, with the integration of prior DP methods a plus that allows for training of our model in a differentially-private manner. \n\nResponding to Question 1: The different colors do not have any real meaning, they were just added for aesthetic purposes. Figure 1 plotted different traces in different colors and Figure 2 shows bars at different times of the day in different colors (e.g., morning has warmer colors and night has darker colors).\n\nResponding to Question 2: Thanks for the opportunity to clarify this. There is no relationship amongst the motifs; motif 1 and 2 are randomly selected glucose motifs (actual motifs shown in Figure 1) and temporal motif 1 and 2 are randomly selected motifs from a cardiology dataset. In each radial graph, there are 24 radial bars from 00:00 to 23:00 for each hour of the day, and the bar value is the percentage of total motif occurrences at that hour across the entire dataset (i.e., value of 10 would indicate that 10\\% of the time that motif occurs during that hour in the dataset). The point of this figure is to illustrate that the glucose motifs are not temporally dependent (they show up pretty evenly across the day) and the temporal motifs *are* (they show up only at specific times of the day). For example, glucose motif 2 shows up at every hour of the day whereas temporal motif 2 shows up most frequently in the morning (around 08:00) and evening (around 18:00).\n\nResponding to Question 3: Thanks for this constructive comment. We will expand the figure captions.\n\nResponding to Question 4: Using the weights from $M$, we compute the causality distribution for the sequence of motifs in each trace in $\\hat{x}$. \n\nResponding to Question 5: MSE loss on 217 is the error between the original data $x$ and the reconstructed data $\\tilde{x}$, the data that was fed through the embedder and then the recovery network. The point of this MSE is to give the autoencoder a way to determine how well it is doing (e.g., how close is the reconstructed trace to the original trace.) A perfect autoencoder perfectly reconstructs the input data. MSE loss on 225 is the error between the batch of real embedded data $x_{et}$ and the batch of synthetically generated data $\\hat{x}_{et}$ at the next timestep. The point of this loss is to give the generator a way to compare the temporal distributions it has generated with the real data. Basically, this is how the generator learns to conserve temporal relationships. "}, {"review_id": "TOm7M3Z9JA", "reviewer": "Reviewer_Eaai", "summary": "Instead of the existing DP GAN framework, which cannot capture the relationship in time-series data well when temporal length is not the only relationship but also different event types and do not perform well when more strict DP (differential privacy) condition is required, the paper proposes new architecture based on the glucose trace use-case which overcomes existing weakness by learning and generating (from the GAN generator) at the space of (Granger) causality matrix instead of the data level. From this, the GAN architecture must be adapted accordingly: autoencoder and discriminator are straightforward (autoencoder to encode time-series datapoint into a vector and discriminator looks at two points in this space), but the generator now can generate more realistic data with not just random noise but also the causality matrix which is learned separately.", "questions": "- Usually I think of generator's training in GAN as getting feedback from discriminator. In line 218-240, this is different: is generator trained directly with these information to warm start? Is this novel or similar to some previous work?\n- Similar to Section 5.2 -- is this framework to balance two losses new or similar to some existing work?\nFor the above: it should be more clear how your architecture is different from existing work, e.g. yours looks similar to [6]. In the related work it only lists how yours compare performance wise, but not architecture or novelty-wise.\n- Why do we only pick 4 hours (line 187)? It makes sense for meal-effect, but this would lose the day-rhythm effect such as sleep time, right?\n- Why is the fidelity not in increasing order as $\\epsilon$ increases in Figure 5, especially for dpGAN? This seems odd.\n- Why not compare to [6] as well? Table 2 suggests [6] uses extra features, though the paper [6] https://people.cs.vt.edu/~reddy/papers/IS22.pdf said it can work for unsupervised data as well -- could you explain more?\n- Any upperbound in Table 2 of length on your method (and go infinite and others can't?)?\n- How to reproduce others' methods? I think the hyperparameter and/or architecture details should be added to Appendix B.\n", "limitations": "- Authors noted about reproducibility issue of private data (line 350-356)\n- Have the owner (patients) of these traces given permission to use their data in this way? Given this is a sensitive data (healthcare), I think it's a good practice to add how this data is ethically appropriate to use (e.g. how we have the permission legally, etc.) I am not an expert and this is not to prevent this kind of work (as this is why we have private synthetic data -- for utilizing sensitive data), but I just want to make sure about this.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The application is practical (healthcare), but may possibly be useful for other similar applications with time-series data.\n- The modeling of time-series relationship is more general than previous works (not just time but other factors such as different type of events), which is appropriate for application such as healthcare\n- Seems already optimized with previous DP work -- with PATE and autoencoder for GAN.\n- Extensive evaluation of synthetic data quantitatively and then qualitatively in Appendix.", "weaknesses": "- The result and motivation of the work is from a specific motivation. While this means it's practical, we wouldn't know how well it generalizes (though I think it may).\n- I am not sure if differential privacy is really at the user-level (as is worded in line 268) or trace/motif/datapoint level (e.g. when one user gives many traces contributing to the dataset). If not a user-level, then the result is not DP with respect to user privacy. I wonder what would happen if dataset is weighted to account for this?\n- It is noted from the author that we cannot reproduce the result for private data. This is something the author acknowledged due to privacy of data; something we have to accept. You said \"Reproducibility: Yes; although we are unable to share the code and data, we have linked to similar public datasets and provide detailed descriptions of the architecture, its implementation and training regime. Therefore reproducing the results is straightforward.\", but shouldn't there then be an evaluation part using these public data then?\n\nSmall comments:\n- I wish more explanation in comparing the architecture (not pro and con) of this work compared to existing work is added for clearer visibility of what is novel vs adapted from previous work at the model design. It seems related work section only has the pro/con part. In Appendix A it has more information but not specific: e.g. line 507-509 what are these \"two methods\"? What's the main difference compared to each of [4] [11] and [14] in line 469? What's new compared to [6] (the architecture looks similar)?\n- Please make font size of plots in page 16 and after larger. Some are unreadable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and review. Responding to Weakness 2 about DP: This is a good point. The privacy is at the trace level, which we claimed was user-level because we assume that each patient has only one trace in the dataset. We will add a clarifying statement about this assumption in Section 6.\n\nResponding to Weakness 3 about reproducability: This is a good point and we thank the reviewer for this suggestion.\n\nResponding to Small Comments: Thanks for this constructive feedback. We will add more details about the network architectures in the Related Work and increase the plot sizes in the Appendix.\n\nResponding to Question 1: Yes, in addition to the extra information it receives from the other parts (stepwise, motif causality and distributional) the generator also receives the traditional discriminator feedback (see line 253). The point of these other losses is to allow the generator to learn to conserve temporal, motif causal and distributional characteristics of the data. Without them the generator does not converge and/or does not produce realistic glucose sequences. The stepwise and distributional losses are similar in ideology to other previous works, i.e., see TimeGAN [4] and RGAN [11] (though they do not use the same architecture / network configuration that we do), and the motif causality part is novel to our work.\n\nResponding to Question 2: Thanks for this constructive question. Although some other methods balance losses that are similar in ideology to ours, none of them use motif causality. This is an important contribution because without the motif causality part the other models are not able to generate realistic synthetic glucose sequences since they only focus on conserving temporal and distributional characteristics of the data, which do not adequately represent event-driven sequences.\n\nResponding to Question 3: We provide additional justification for the hyperparameter selection (including for $\\tau$) in the Appendix B. $\\tau = 48$ was chosen because the effect of most behaviors on glucose occur within 4 hours of the event (e.g., as the reviewer mentioned eating as well as things like stress or exercise). The reviewer is correct that day rhythm effects would not be captured in this shorter timeframe, and other sizes of $\\tau$ could be selected based on what types of phenomenon the users wish to replicate.\n\nResponding to Question 4: Is the reviewer talking about the Time-in-Range metric in Table 1? If so, Time-in-Range is the percentage of time a patient's glucose is in the proper range (between 70 and 180 mg/dL), and gives a measure of how well controlled an individual is. So a value of 61.8 means that 61.8\\% of the time, the patient had their glucose in the proper range. For individuals with T1D, it is really difficult to keep their glucose in the proper range 100\\% of the time, and actually most \"well-controlled\" individuals have time in ranges of 60-70\\%. As such, the synthetic data that has Time in ranges of 80-90\\% as in dpGAN for the lower privacy budgets are highly unrealistic. As the budgets increase and the time in range decreases, this is getting more and more realistic (closer to the 60-70\\% mark.)\n\nResponding to Question 5: Yes, this is true that RDP-CGAN [6] does work without labels (unsupervised) but it is for multivariate data and relies on learning the correlations amongst multiple features. Since our model is for univariate data with only one feature (glucose), we did not feel this would provide a fair comparison (as there would be no other features for RDP-CGAN to learn from).\n\nResponding to Question 6: Thanks for this important question. We are currently experimenting to find what the maximum length that GlucoSynth can handle may be. In the meantime, we will remove the \"+\" from Table 2 since we do not know the upperbound yet.\n\nResponding to Question 7: Thanks for this constructive comment. We trained all other methods for 10,000 epochs and using the suggested hyperparameters provided by their individual publications. We will add more explicit details about this in Appendix B.\n\nResponding to the last limitation: Thanks for bringing up this important point. Yes, we have received the proper ethical and legal consent from the individuals to use their data in this way (and for this purpose)."}]

Author rebuttal:

Meta-review: The paper presents a method for generation of private synthetic time course data using an innovative model that takes into account prior knowledge of the physical constraints underlying the data. All reviewers recommend acceptance as the authors were able to answer their questions, although most consider this only borderline accept.

I am also in favour of acceptance, as I like the radically new kind of approach for private synthetic data generation that may prove useful in other applications too.
Decision: Accept (poster)