Noise$^+$2Noise: Co-taught De-noising Autoencoders for Time-Series Data
Authors: Harry Rubin-Falcone
Joyce Lee
Jenna Wiens
Venue: Submitted to ICLR 2023 (2023)
Keywords: De-noising
Co-teaching
Noise recovery
Time-series
self-supervised
RNN

Abstract: We consider the task of learning to recover clean signals given only access to noisy data. Recent work in computer vision has addressed this problem in the context of images using denoising autoencoders (DAEs).  However, to date DAEs for learning from noisy data have not been explored in the context of time-series data.  DAEs for denoising images often rely on assumptions unlikely to hold in the context of time series, \textit{e.g.}, multiple noisy samples of the same example. Here, we adapt DAEs to cleaning time-series data with noisy samples only. To recover the clean target signal when only given access to noisy target data,  we leverage a noise-free auxiliary time-series signal that is related to the target signal. In addition to leveraging the relationship between the target signal and auxiliary signal, we iteratively filter and learn from clean samples using an approach based on co-teaching. Applied to the task of recovering carbohydrate values for blood glucose management, our approach reduces noise (MSE) in patient-reported carbohydrates from 72$g^2$ (95\% CI: 54,93) to 18$g^2$ (13,25), outperforming the best baseline (MSE = 33$g^2$ (27,43)). We demonstrate strong time-series denoising performance, extending the applicability of DAEs to a previously under-explored setting.

Under review as a conference paper at ICLR 2023
HOISE+2NOISE: CO-TAUGHT DENOISING AUTOEN- CODERS FOR TIME-SERIES DATA
Anonymous authors Paper under double-blind review
ABSTRACT
We consider the task of learning to recover clean signals given only access to noisy data. Recent work in computer vision has addressed this problem in the context of images using denoising autoencoders (DAEs). However, to date DAES for learning from noisy data have been explored in the context of time-series data. DAEs for denoising images often rely on assumptions unlikely to hold in the context of time-series, e.g. multiple noisy samples of the same example. Here, we adapt DAEs to cleaning time-series data with noisy samples only. To recover the clean target signal when only given access to noisy target data, we leverage a noise-free auxiliary time-series signal that is related to the target signal. In addi- tion to leveraging the relationship between the target signal and auxiliary signal, we iteratively filter and learn from clean samples using approach based on co- teaching. Applied to the task of recovering carbohydrate values for blood glucose management, our approach reduces noise (MSE) in ratient-reported carbohydrates from 72g² (95% CI: 54,93) to 18g² (13,25), outperforming the best baseline (MSE 33g² (27,43), We demonstrate strong time-series denoising performance, tending the applicability DAEs to a previously under-explored setting.
10
11
12
14
16
17
INTRODUCTION
Background. Denoising autoencoders (DAEs) (Vincent al. 2008 have been used to accurately
19 denoise various signals, including medical images (Gondara 2016). ECG signals (Xiong et al., 2016), and power system measurements (Lin et al. 2019). With respect time-series data, DAEs have been used for forecasting (Romeu et al.) 2015). classification (Zheng et al. 2022) and impu- tation (Zhang Yin, 2019). but generally require access to clean samples at training and not provide de-noised outputs. In many real-world settings, clean samples are unavailable at training. Work in computer vision has addressed this problem through extensions that either require paired samples (Lehtinen et al., (2018) or rely patch-based analysis (Krull et al. 2018 Laine al. 2019
20
21
22
23
24
26 Xie et al., 2020) Batson & Royer 2019). Similar approaches do not extend to time-series data, where paired samples rarely exist and patch- patch-based techniques do not apply. Beyond approaches that rely on paired samples or patch-based analyses, researchers have recently proposed techniques that utilize knowledge of the noise distribution recover the clean signal. These approaches either the properties of the distribution to recover the clean signal after training on noisy data (Kim & Yel Moran et al. 2019), or rely on the noise having low expectation and variance compared the signal, in which case a model trained on noisy data can approximate one trained on clean data (Xu al. 2020). While these approaches may be considered in time-series setting (and treated as baselines here), their applicability is limited as noise in time-series settings is rarely weak or known.
28
30
31
2
32
33
34
35
Our Contribution. In light of this gap, we adapt denoising autoencoders for time-series data. Our approach. 'Noise 2Noise', learns to map a noisy target signal to a clean signal given only noisy samples and an auxiliary clean signal. Inspired by work in image denoising (Lehtinen et al., 2018 Xu et al. (020). we add additional noise to the noisy target signal during training and attempt to recover the original noisy signal. Provided that the noise has low expectation and variance, network trained in this manner can learn to recover the true signal because the noise will minimally impact
38
37
38
39
41 the expected value of the output of the network (Xu et al. 2020). The auxiliary signal input along with the target signal into a denoising autoencoder. which allows network to leverage the relationship between the auxiliary and target signals. To address the fact that the signal to noise
43
ratio might not be weak, we adapt co-teaching approach train DABs (Jiang et al. 2018

---

Under review as a conference paper at ICLR 2023
Han et al. 2018). We use this approach to identify the cleaner samples; the most likely low-noise 45
46 samples are identified as the low-loss samples of the other model and used for backpropagation. This co-teaching approach has never been applied to de-noising in time-series any other setting.
48 It has also never been utilized in a continuous output setting. By adapting this co-teaching approach
49
to DAFs, we provide a solution to denoising in this novel (time-series) setting.
50
Real-world Inspiration. Disparate levels of noise across variables are common in data streams. Measurement reliability can vary across sensors, from essentially noiseless to highly corrupted.
51
52 Throughout this work we take inspiration from a real-world problem affecting millions in the US: blood glucose management Individuals with diabetes monitor several variables over time, including their blood glucose, insulin administrations, and carbohydrate intake. Blood glucose when measured
53
54
55 by a continuous glucose monitor (CGM) has little noise, while the number of grams of carbohydrates
56 in meal are patient-reported, and, as a result, are often inaccurate. Recognizing the variation in the level of noise across signals in a data stream, we propose an approach that utilizes the more reliable
57
58 variables (e.g., blood glucose) to update noisy variables like carbohydrates. Our approach aims to
59 retrospectively correct measurements of patient-reported signals, like carbohydrates in the context
60 of diabetes, which could lead to improved treatment decisions: patients can learn when they are
61 over- or undortne and adjust in the future, ultimately improving health outcomes.
RELATED WORK
62
63 In denoising autoencoder training, a model input is corrupted and a network is tasked with recovering the original input. In this way, the network cannot learn the identity, unlike in basic autoencoder training (Vincent et al. 2008). Recent work in machine learning has focused on using DAEs to recover clean signals from only noisy signals. The vast majority of this work lies in image analysis and builds off of "noise2noise" (Lehtinen et al. (2088). an approach that uses multiple noisy instances of the same image to train model to learn to denoise the image. Briefly, the approach relies on the fact that if the noise is mean zero, using a secondary noisy instance (besides the input image) as
65
66
67
68
69
70
a target will produce a network that learns the clean image, in expectation, when enough training data is available. When paired samples are unavailable, others have proposed approaches based
71
72
on exploiting patches sampled from the image (Krull et al., 2018 Laine et al.] 2019)) Xie et al.,
73
2020 Batson & Royer] 2019), but these are not applicable to time-series data. Other approaches eschew relying on inter-variabl
74
et al. 2019 Kim & Ye| 2021) relationships but rely heavily on a known noise function (Moran low expectation and variance noise function (Xu et al.) 2020).
75
76
Our approach builds off of Xu et al. (2020). learning to reconstruct a noisy signal from only noisy samples of that signal, but in contrast to Moran et al. (219) or Kim & Ye (202T), we do not make strong assumptions about the noise distribution. To get around these assumptions, we leverage an auxiliary signal and iteratively identify the cleaner samples within the dataset. Combined with the auxiliary signal these samples are used to train a DAE that treats these samples as ground truth.
78
80
Our approach denoising a noisy target signal by iteratively identifying clean samples is, in part, related to work in noisy-label learning. A common approach to learning from noisy labels involves identifying and reweighting samples with clean labels during training. Samples are filtered based on gradient values (Ren et al. 2020). Jacobian MiraannMMrzasolleiaan et al. 2020) or some latent state Lee et al. 2019 Wu et al. (020). Co-teaching (Han et al.) 2018), which builds off of mentor net (Jang et al. 2018), is performed by training two networks in parallel. Each network is back- propgated using only samples within the current mini-batch for which the loss of the other network 88 is lowest. Intuitively, samples with incorrect labels are likely to have higher loss and therefore be
81
82
83
84
85
86
87
89 removed. Using two networks in parallel provides robustness to outliers and initially misclassified samples, which single-network boosting-style approaches are sensitive to. These approaches have
91 been primarily explored in a supervised setting. In contrast, we consider an unsupervised setting in which labels are unavailable and, instead, the input signals themselves are comrupted. To the best of our knowledge, such a co-teaching approach has not been explored in the context of denoising.
83
PROBLEM SETUP
94
95 Problem Definition. Given a noisy target variable and a reliably measured auxiliary time-series, we
96 aim to recover the true values of the target variable. We assume relationship between the auxiliary
2

---

Under review as a conference paper at ICLR 2023
Approsiannte clean value (+2)
Algorithm Naise"
Co-taught Denoising cuto--nscderss Input DAE' and DAE² with randomly initinlized weights, Loss function noise distribution M, fixod opoch Es, epoch Bmuz
2Noise: Noise=NNoise:
Arsilitty Limas seciice
Istent nurea
cundien natio
nーN
b
(
dlea lunge [c:)
DAE
-{87)
for t= 1,2,, Emax
Petch batch (z0) and (bx)
Select sompllo with loo valuon below the B(t) percentile for training
Add naise to original somples
((b))
VApprocimate clean values
6 ({b.) im.)
-= Ue (1x,0)) PmolE
mandam
-M
Es Llu, 3 < Peco (2) FD)
|/elect samples
I [s: L(v,.)) .3) (R.) (8)
mー Masag) Pmg (m (0
value
mokey
{ 31. + 2z,)
Sample (m²) ~
Reconstract target samples selected by the OTHER DAR
[+]] + mc)
(/Doubly noney values
Backgropagate
V iE J
JEJ
DAE
[69)
((b,, (151) for E
((b,) (151) for J E
(Recomstrues noley signal
BACKPROP L(a,, 8)) for j J and L/y,, for S
ME JR. 4 JE JR.
DAR!
{50
L)
Update R(() = (
Output DAE and DAE
Figure 1: 'Noise 2Noise'. Sample selection is performed with each DAE's output when given the uncorrupted y signal, but backpropagation is performed on the model's output when given a corrupted signal. The loss values of DAE are used to select the sample for backpropagation for DAEa and vice versa. PR()) denotes the R(t)th percentile, where R(t) is a function of iteration t.
97 and target variables, and that some samples from the noise distribution associated with the target variable will be close to zero, although which samples is unknown in advance.
98
Formalization. Let x R denote sample of the target variable. Let n N, where E R denotes a random variable drawn from an unknown distribution AC Let R" denote an auxiliary dime-series that is related to æ, such that mapping f(b) - exists. Given sample y x + n and b. we aim to learn to recover x.
100
102
Assumptions. We assume that the distribution N is independent of both b and Y. We also assume that some values of n are near zero such that given training data with k samples {.r),= with noise values { and true signal values { there exists a subset S with sufficient size for training such that the mean and variance of n. E S are negligible compared to the mean and variance of
104
105
107
I vs E S. We assume that the distribution of low noise samples is such that they cover all regions of the input; i.e. that S does not exclude entire regions of Y. Finally, we assume that the relationship
108
109
between b and x can be accurately captured with a recurrent neural network (RNN).
METHODS
110
111
Overview, Our method, 'Noise" 2Noise' (N 2N), is summarized in Figure 1 We filter out noisy 2Noise
112
samples during training, refining the model parameters the least noise. These filtered samples are used to learn to denoise the signal. To identify the samples within a batch to use during training, y and b are passed to DAE that outputs ÿ. or a de-noised y value. Assuming the DAE accurately reconstructs x, the loss L(J, y) represents the expected error between y and x. Given a batch // we identify the subset of samples with values of L(ū,y) below the R(()th percentile (where R(t)) is an increasing function of iteration (). These 'low-noise" samples, Vj E J, are then augmented with additional noise my ~ M and, along with corresponding b vectors, are input to the DAE, which outputs reconstruction of yy: 9z- We then backpropagate using the squared сπтоr between Új and yj. In this way, the DAE is trained on the samples estimated to have the least noise while utilizing the noisy signal as input without the susceptibility to learning the identity function that comes with standard auto-encoder training. Co-teaching is utilized in that two DAES are used at each step (DAE" and DAE"), and during training, the samples identified as low-loss by DAE are augmented and passed to DAE for backpropagation, and vice-versa. This adds the benefit of ensembling and curtails sensitivity to outliers incorrectly selected samples. We note that the auxiliary signal b is input to the model during both the sample selection and backpropagation steps. This allows our approach to function even when the variable of interest y is too noisy or low-dimension be de-noised alone.
selected samples of y estimated to have
113
114
115
116
117
118
119
120
121
122
123
125
127
128
3

---

Under review as a conference paper at ICLR 2023
130
Denoising Autoencoder. In our setup, additional noise (m) is added to y to produce 2, a 'doubly' noisy measurement of z. 2 and b are imput to network (henceforth denoted DAE) that outputs 4 = DAE(z, b). and the network is trained to reconstruct y: loss is measured between y and ý. As shown by Xu et al. (2020), when the expectation and variance of the noise distribution N are negligible compared to those of the signal, the model parameters that minimize the loss between ğ and y are very close the optimal parameters of a model trained on clean data. Thus, provided the signal to noise level is high enough, we can pass y to DAE and expect a reduction in noise, with an output much closer to F, at inference time.
132
133
134
135
136
137
138
Co-teaching DAEs. We do not expect the noise to be weak in general, but we do assume that some of the samples will be lower noise than others. We identify and train using these samples via an adapted co-teaching approach (Han et al. 2018). We utilize two DAEs, and for each, we backpropagate using only the samples for which the denoised y values from the other DAE are near the original y values. If the denoised y values approach z, we are then selecting samples for which the estimated noise n is lowest.
139
140
141
142
143
Claim. When using co-teaching to train two DAEs (DAF) and DAE?) in parallel, g² DAF (y. b) and jj2 = DAF? (y. b) approach x.
144
145
146
Justification. Based on the main result ofXu et al. (2000), these values approximately equal x when a DAE is trained on a dataset where the expectation and variance of the signal are much greater than those of the noise. We have assumed that such sub-sample exists in our dataset, and we hypothesize that co-teaching will select such a sub-sample. Note that because the noise n is independent of both
147
149
150
y and b, any signal learned by DAE' must be of the form DAE' (y. b) g(x) + 9. for some function g. where is independent of all variables. The DAE might begin to learn a biased function of x; for example, if the noise distribution N is not mean-zero, a DAE might learn the function g' ('(x) = x N. We curtail this behavior by initializing the DAES to output the identity function (y). Because the model is near the identity function g(x) = y early in training. if it began to learn
151
152
153
154
155
a biased function such as g', it would begin to output values g' (xx ~ g'yy) = y + N. These values would be further from the identity than if the model learned non- non-biased function of ar which would in turn guide the model away from learning g'. By similar logic, the model would be encouraged to learn non-biased function of x, the simplest of which is g(x) = x. This is only a conjecture, but in practice, we have found that this approach works well even when there is fairly substantial bias in the noise. We note that although the two DAEs are initialized to output the identity, their weights are otherwise random they not likely to converge to the same minima. We further prevent convergence by utilizing the co-teaching+ variant (see below).
156
157
158
159
160
162
163
Sample selection. We select the samples with the lowest estimated noise to backpropagate with. If y' and d² approach each network's 's estimated value of x, then DAE 's estimate of n, the noise between x and y, is approximately y - 5¹ (and similar for DAE²). For loss function L, we use L v) and L(i²: y) to select samples. Given a batch {yr J at iteration I, we identify the subset of samples with values of L y:) below the R(t)th th percentile as J = { Lu,, 8)) <
164
165
186
187
PRoy (L(5 }. D)), and similarly define J2 for DAE As in Han et al. 2018). we begin by training on the full sample. Over the course of training, as the DAES are expected to become more accurate, we gradually reduce the sample. This prevents the memorization of noisy samples that can occur later in training. Hyper-parameter 7 E (0. 1) represents the maximum proportion of samples removed and Fg represents the iteration at which we stop increasing the proportion of samples removed. A linear decrease in sample size as a function of iteration t is implemented by using the lowest-loss R(f)= = (1 Mazimumm Fxx
168
189
170
171
172
173
- 7) 100% of samples for backpropagation.
175 Training. Each DAE is trained on the samples for which the other network estimates that the noise is lowest: samples selected by DAE (μjvī E J²) are augmented with additional noise m, ~ M to generate 2j values. Zy. along with corresponding b vectors, are input to DAE? which outputs a reconstruction of yj: 1/z. We then backpropagate using the squared error between Dj and yj- Similarly, we only use samples yj E J2, augmented with m, ~ M, to backpropagate D DAF By selecting samples based on L(j. y) rather than based on L 2), we are able to select sample independent of secondary noise value m. Selecting samples dependent on m would be confounding because samples might then be selected based on how low the value of m is at the current iteration, rather than the value of n, which is hidden. Back nack-propagation is performed on an input that does not
176
177
178
179
181
182
183
4

---

Under review as a conference paper at ICLR 2023
include clean y values, the model is not likely to learn the identity function. Sample selection is always performed by the other DAE, so compared to boosting or other one-network approaches, our method is less sensitive to error propagation from wrongly selected samples early in training.
185
187
Co-teaching+. We utilize co-teaching plus (Yu ct al. 2019)) where samples for which the models disagree selected for backpropagation. As a result, each model learns from the samples for which the other model's estimates were better. This also prevents the models from learning from the samples that they agree upon, which prevents convergence, maintaining unique strengths in each model. We remove the a% of samples for which the models' outputs are closest (herea is a hyperparameter). This step is performed prior the sample selection step: the 0% of samples for which the distance between U' and 8 are lowest are removed, and then the remaining samples for which (ii"",, y) is lowest are used for the backpropagation of DAF and vice versa.
189
190
181
192
193
194
5 REAL-WORLD PROBLEM SETUP: BLOOD GLUCOSE MANAGEMENT
196
To explore the benefit of our proposed approach, we consider a real-world problem setup based on blood glucose management that inspired the setting described in Section 3. Nearly two million people in the US have type I diabetes and require insulin to maintain healthy glucose levels due to a pancreatic deficiency in insulin production. Because this, they must deliver boluses of insulin through an injection or an insulin pump prior to eating to counteract the rise in blood sugar that re- sults from the ingestion of meals. Bolus amounts are calculated based on patient-reported estimates of carbohydrates. Carbohydrates and bolus insulin generally cause blood glucose values to increase or decrease after a delay of 30 minutes to an hour. In our setup, carbohydrates correspond to x values, glucose levels are b.
197
199
201
203
205
Blood glucose forecasting and control have been extensively studied in the past (Silvia Oviedo 2016 Fox al. 2020 Accurate models for blood glucose dynamics are critical to the devel- opment of algorithms for managing blood glucose in individuals with diabetes both in terms of
206
207
208
seteent-selected treatment options and automated solutions. Work in this field is popular largely be- cause the ubiquity of devices for measuring blood glucose and administering insulin make obtaining fairly clean measurements for those values straightforward. However, carbohydrates consumed are patient-reported and as result are often inaccurate. This in turn leads to inaccurate doses of insulin and poor blood glucose management. Besides misestimation, there are other sources of inconsis- tency between recorded carbohydrate values and their effects on blood glucose. Variability in meal types is generally poorly captured, which is problematic because the effect of carbs on blood glucose can be moderated by how quickly meal consumed, the amount of protein, fat and other nu- trients eaten. Additionally, the timing of meal may not recorded accurately. These factors alone make utilizing carbohydrate information difficult, even when carbs are accurately recorded. In unsupervised setting, denoising approaches could learn representations of carbohydrate values that incorporate these other sources of variability. These representations could be more relevant to blood glucose management than the exact number of grams consumed. This could improve performance of forecast and control algorithms.
209
210
212
214
216
218
219
220
221
222
6
EXPERIMENTAL SETUP
223
We implement our approach in the context of learning to correct noisy patient-reported carbohydrate measurements. We compare performance to several baselines across real and simulated datasets.
224
225
6.1
DATASETS
We utilize two TiD-based datasets. The simulated dataset provides access to ground truth which 226
227
we can directly compare our method's 's denoised outputs. The real dataset provides more challeng- ing setting for quantifying the efficacy of our approach, but corresponds to real-world scenarios.
228
Both datasets are publicly available and have been previously explored in the context of forecasting
230
and control (Man et al. 2014 Xie 2018 Marling & Bunescu] 2018 2020). Both datasets consist insulin, and carbohydrate values. All variables were scaled to be between zero and one. For both datasets, time-series trajectories for
of blood glucose, bolus (fast-acting) insulin, basal (slow-acting
232
233
each patient were split into windows of 2 hour length (T 24 5-minute time points). We ignore
5

---

Under review as a conference paper at ICLR 2023
windows where carbohydrate occurs in anywhere but the first position, using only windows with no carbohydrates or carbohydrates at the beginning of the window during training. This means we also ignore windows with more than one carbohydrate present. In real-world setting these values could be updated recursively, but we simplify our setting here.
235
237
238
Simulated. Our primary analyses are performed on data generated with commonly-used T1D simulator. We use the UVA-Padova simulator (Man et al. (014) via a publicly available implemen- tation [ For ten simulated individuals (the "adult" patients modeled in the simulator), we generated approximately 150 days worth of data each, in 30 day roll-outs of the simulator. Days where a patient either had more than 25 timepoints of glucose at the minimum value of 40, or more than 35 timepoints over 450 were thrown out for being non-realistic. The meal schedule used to generate simulated data was based on the Harrison-Benediac equation (Harris & Benedict 1919) as implemented in (Fox et al. 2020). In our simulation, for all datasets generated, we used the default basal-boll bolus controller from the existing implementation of the simulator to administer insulin, but we delayed five sixths (randomly selected) of the bolus administrations up to 3.5 hours, with delay time randomly sampled from a uniform distribution. The delay allows for disentanglement between carbohydrate and bolus effects. The carbohydrate values serve as values, while the CGM values output by the simulator serve as b values. Noise was added to carbs during data generation.
239
240
241
242
243
244
245
246
248
250
We use noise proportional to the true carb value, as studies on the accuracy of carbe counting port errors relative to the total carbs consumed (Brazeau et al., 2012). Also hased on Brazeau et al.] 2012), we use noise distribution with negative bias, as the carbs were found to be more-often under-reported than not. We therefore set y = (1 + [1+ (-.25, Jx. We then cap y below and above and 200 to keep values realistic. We consider additional noise distributions as sensitivity anal- yses. Bolus values were calculated based on the noisy carbohydrate values. Additionally, 20% of carbohydrates are not reported, to make the dataset more realistic, as missingness is commonplace.
252
253
254
255
256
257
258
This dataset includes both the OHIOTIDM 2018 and 2020 datasets, developed for the Knowledge Discovery in Healthcare Data Blood Glucose Level Predication Challenge (Marling & Bunescu 2018 2020). The data pertain to 12 individuals, each with approximately 10,000 5-minute samples for training and 2,500 for testing. 12% of glucose values are missing, but we do not include windows with missing glucose values. We do not include windows with more than one carbohydrate measurement in our analysis. We sum carbohydrates to the first timepoint they are less than 15 minutes apart to maximize the amount of usable data. We include only individuals with at least 100 training carbohydrate measurements, as fewer than this are not sufficient for learning a model.
Real.
259
260
261
263
285
286
6.2
BASELINES AND UPPER BOUND
For all non-coteaching methods, we train two DAEs in parallel and report results their averaged output for fair comparison. We also note that all models receive the same auxiliary variables (blood glucose/ insulin) as input in an identical fashion.
268
289
270
CAE: An upper performance bound. This model is an autoencoder trained with clean data, which would expect to perform better than any method without access to clean data.
271
272
NAC: Our first baseline. A DAE that treats the noisy data as clean which has been shown to perform well in low noise settings (Xu et al. 2020).
273
274
NR2N: Our second baseline is noisier2noise (Moran et al. 2019). which uses the known noise distribution recover the clean signal. NR2N trains similarly to NAC, but evaluation time transform is used to recover the clean values (briefly, if the distribution of N is known and we set NN the model should learn recover half of the noise so the value used at evaluation is 2ÿ-z). 2). SUP: Our motivating setting can be re- re-framed as a supervised learning problem: predict y (or x) values using b values as input. Depending on the noise distribution, it is possible that model trained
275
276
277 M-
278
279
280
on noisy y values could learn to predict the correct æ. using similar logic to that found in Lehtinen ct al. 2018). We therefore use this supervised setting as a naive baseline. We simply input to the same network used in the DAE setting and calculate loss as (j- y)² during training, but here the model has no information regarding y or 2. As in the DAE setting, at test time we evaluate (ū z SUPCT: We apply co-teaching to the supervised setting (SUP), to ensure that performance gains observed are due to the combination of DAEs and co-teaching, and not co-teaching alone. Here, the model is tuned and trained identically to N+2N, except the model does not receive y or values.
281
283
285
286

---

Under review as a conference paper at ICLR 2023
6.3
IMPLEMENTATION & TRAINING DETAILS
288
Each DAE is implemented as a 2-layer bidirectional LSTM with 100 hidden units. The final hidden state is passed FC layer with single outpur. The output of the model is added the input value corresponding to y, so that the network is tasked with learning the eтror term rather than a complete reconstruction. This initializes the network's output to be the identity (provided the output is initially low magnitude). which is useful for addressing biased data. Because we only aim to correct a single carbohydrate (y) value but use a time-based model, we set one dimension of the model input to be y for all timepoints. Blood glucose values b, bolus and basal insulin are input
289
290
291
292
293
294
295
to the model as time-series. We also carry over bolus insulin values (which occur sparsely) to the end of the input window, to increase their impact on gradient calculations. We threshold the output of each DAE at O, because carbohydrates (our r and y) values cannot be negative. For each co- teaching method and NR2N, we tune on a single subject's 's validation data (adult#001) and use the found parameters for all further analyses. Tuning is described in Appendix A. At evaluation we report the result of the average correction learned by both networks when y values are given as input (i.e., where ú = ( b), we report [(r, (7) + (2)/2))
297
289
300
301
302
We split each dataset into training, validation and test sets used for evaluation purposes. For the simulated dataset, we use 80 days for training, 20 for validation, and 50 for testing. For the real dataset, we split the training data into 80% train and 20% validation. The held-out test data were
303
304
305
used for evaluation only. We implement and train our models in Pytorch 1.9.1 with CUDA version 10.2, using Ubuntu 16.04.7. a GeForce RTX 2080, an Adam optimizer (Kingma & Ba, 2014), and a
308
307
batch size of 500. We use learning rate of 0.01 and a weight decay of TO We train for at least
308
500 iterations, and then until validation performance does not improve for 50 iterations, selecting
the model for which validation performance was best. For both datasets, we train and test a model on each individual and report across-individual averages. Such sadiiddal--seccffi models/evaluations are common in blood glucose control and forecasting (Silvia Oviedo, (0016), since dynamics vary greatly across individuals and individual -peciffc training data are typically available
310
312
313
We perform co-teaching on samples containing non-zero y values only. However, when training all models (including baselines) we also pass zero-valued samples (and their corresponding values) through both DAEs and take loss equal to ÿ² for these samples. We do this because there many more samples with vero-valued carbohydrates than there are with positive values, and this allows the models to learn from this larger corpus. We report results on only positive-valued ys, because denoising is only applied to such values.
314
315
316
317
318
For sample selection during co-teaching, we use mean squared percentage error (100%-((--))) to avoid eliminating all high-valued samples, as they are likely to have higher noise values. As a noise function during training, we use (1 + [N 5)) 5)y. i.e. we add random noise to half of the samples so that the model can learn to utilize noisy z information, and zero-out the other
319
320
321
322
323
half so that the model has to learn to distinguish zero from non-zero y values based on b alone.
324
6.4
EVALUATION
325
Simulated Data Metrics. When clean carbohydrate values are available, we MSE between the cleaned carbs and true carbohydrate values as our metric to report the remaining noise (mean((a ((x )) where = (z' + )2)) The lower this value, the more noise has been removed. Although we assume that these data are not available at training time, we use them for evaluation. Since it can be difficult interpret the meaning of difference in MSE, we also consider clinically motivated evaluation metric: time in range. Time in range is measure of blood glucose management and
326
327
328
329
330
331
varies with the accuracy of the carbohydrate measurements. The more accurate the carbohydrate estimates the more time an individual will spend 'in range.' Here, we run simulation of the subject of interest with the default basal-bolus controller using bolus values calculated from the updated carbohydrate values, and report the proportion of time in the simulation that each individual spent with blood glucose values between 70 and 180, or the euglycemic/healthy range. This metric serves to indicate the real-world impact each approach might have. For both metrics, 95% confidence intervals are calculated for each subject using 1,000 bootstrap re-samples, and the average 2.5 and
333
335
336
337
97.5 percentiles across subjects are reported.
339
Sensitivity analyses. To evaluate our model under different noise assumptions, we repeat our analy- sis with multiple noise generation methods (x y)) without altering hyperparameters or our y ->
340
7

---

Under review as a conference paper at ICLR 2023
noise function. We use various Gaussian and uniform distributions reported in Appendix B, which include zero and negative mean multiplicative and additive noise functions. We do not aim at a com- prehensive evaluation of all possible noise types, but rather we aim to include various distributions that are likely similar to those that might arise in our motivating domain.
342
344
Real Data analysis. Without access to clean carbohydrate values test time for the real dataset (nlike the simulated dataset), we evaluate the performance of our denoising approach based on proxy. We take advantage of the fact that poorly estimated carbohydrates result in inaccurate bolus calculations, which result in poor blood glucose management. We expect that inaccurate carbohydrates estimates (large [ y] values) result in the poor blood glucose management. For a model that has come close to estimating z correctly, we would observe a correlation between [y - y] values and blood glucose control in the time period following meal. We assess this with Correction- Wask-Correlation henceforth referred to as CRC, defined as the Spearman correlation between the squared carbohydrate correction value (( )) and the average Magni Risk (Magni et al. 2007) of blood glucose in the second hour following the carbohydrate. We use the Spearman corre correlation to
346
347
348
349
350
351
352
353
355
account for non-linearities in the risk and correction value distributions. Magni risk is a measure of how far from safe value blood glucose is; higher risk values correspond to blood glucose values that
357
are either dangerously high or dangerously low. We use the second hour following the carbohydrate
358
because the effects of the carbohydrate consumption and insulin bolus have not fully taken effect in
359
the first hour. We calculate this correlation across all carbohydrates observed in all individuals. For
380
validation purposes, we also calculate this metric for the simulated dataset.
7 RESULTS AND DISCUSSION
382
Through our experiments, we aim answer the following questions. 1) Does our approach mean- ingfully reduce error across a variety of simulated individuals, compared to existing approaches? 2) 384 Is our model robust different domain-approprrite noise distributions? 3) Does model show strong performance in real data, indicating accurate denoising?
363
385
Table 1: Our approach outperforms the baselines with respect to all of the evaluation metries and across both SIM and REAL datasets, falling only 2% short of the upperbound w.r.t. the clinical measurement of 'Time in Range.' 95% Cls are calculated from 1,000 bootstrap ramsaee.es.
Model
Remaining Carb MSE (g" SIM)
% Time Range (SIM)
SIM CRC (np)
REAL CRA (r.p)
N/A-clean carb
0.00 (0.00, 0.00)
73.18 (72.49 (72.49.73.88)
N/A
N/A
N/A-noisy carb
72.26 (54.16, 92.58)
65.43 (64.70.66.16)
N/A 0.32, 4c-31
N/A
CAE
6.96 (5.03, 9.18)
72.44 (71.76,73.11)
N/A
SUP SUPCT
58.37 (45.11, 73.31)
64.59 (63. 39,63,00)
0.04, 0.14
0.19, 3e-3
100.50 (84.14.11)44)
60.22 (3,694)))
3c-3, 0.91
0.03, 0.61
NAC
36.40 (27.02, 46.95)
68.22 (67.53.68.93)
0.11, 1c-4
0.13, 0.05
NR2N
33.44 (26.59, 43.23)
68.79 (68.09,69.51
0,11, le-4
0.13, 0.05
NN (Ours)
17.91 (12.61, 24.98)
71.24 (70.54,71.90)
0.19, 3e-11
0.22, 9e-4
366
Error Reduction for simulated data. Co-taught denoising autoencoders, N 2N, outperform base- lines in terms of noise reduction (MSE) (Table 2 reduces MSE 18g², but fall short of the value achieved by the clean-data DAE (7g2), as expected. CAE does not achieve perfect MSE, probably due to insufficient training data or small amount of noise in the CGM signal. Our approach's 's reduction in noise is meaningful since it leads to significantly better time in range. Most methods offer an improvement in % time in range when used in a basal bolus controller, with base- lines increasing over the noisy value from 65% to 69%, and N 2N further improving performance to 1%, recovering 6% time range out of a total of 8% lost when using noisy versus clean values. SUPCT performs worse than any other method including SUP, likely because without the noisy car- bohydrate measurement as input, co-teaching cannot learn the relationship between b and as easily, and therefore does not identify the less corrupted samples during training. This results in essentially random sub-sample selection, hampering performance as less training data becomes available. SUP does not suffer from this problem because it utilizes the entire dataset for all iterations.
387
388
389
370
371
373
375
376
377
379
Alternate noise assumptions. N 2N outperforms all baselines across the majority of noise dis- tributions (Figure 2. For zero-mean uniform multiplicative noise, NAC outperforms the proposed
380
8

---

Under review as a conference paper at ICLR 2023
Remanin Car NSE (10-Subeect Average, g
140
EAC
120
(curs)
NAC
100
NITN
00
20-
XNN 0
X.N
+N0
NN U.0
X-
U.0
U-
Moree Tunction
Figure 2: Performance on datasets with multiplicative (x) vs. additive (+), Normal (N) vs. Uniform 2N
(U), and zero (0) vs. negative mean noise functions. N+2N generally outperforms baselines.
381
approach. We hypothesize that NAC performs well in this setting because the expected value of the noise is zero and the variance is lower than in other settings (it is 33%, which is approximately 30g, compared to 75% in the multiplicative normal setting, or 40g and 60g in the additive noise settings, Appendix B), which is the setting for which NAC has been shown perform best. Of note, this analysis was carried out without additional tuning, demonstrating the resilience of our approach to varying noise assumptions. Across biased noised distributions, our proposed approach consistently outperforms all baselines. This resilience is likely due to our approach's 's selection and a lack of dependence on matching M and N noise distributions.
383
385
386
387
initialized sample
388
389
Experiments on Real Data. For the real dataset, NN outperforms all baselines with respect CRC. For simulated data, we see that, without exception, models with lower remaining MSE after denoising have higher or equal CRC. This indicates that our metric serves as a reasonable proxy for remaining error when true values are unavailable. Plots showing the components used to calculate CRC (magnitude of carbohydrate correction versus Magni risk an hour after the meal) can be found Appendix C. Interestingly, the SUP baseline performs fairly well for this task on the real dataset. We hypothesize that this may be because carbohydrate measurements
390
391
392
394
unreliable for this dataset that learning to predict them from scratch (without access to noisy values at test time) is sufficient. Four out of twelve individuals in the real dataset had too few carbohydrate measurements to be included in our analysis, indicating that more work would need to be done for our algorithm to be applicable broader population.
396
398
CONCLUSIONS
400
401
We propose a new approach to denoising, 2Noise', that does not assume access to clean samples and applies time-series data. Our approach leverages an auxiliary time-series that is re- lated to the target signal to help identify target samples with less noise. Our approach is the first to adapt co-teaching to de-noising. In the context of carbohydrate recovery for blood glucose manage- ment, compared to existing approaches, Noise++NNois leads to better signal reconstruction that is both statistically significant and clinically significant. While promising, our approach is not without limitations. Our primary analyses are on simulated data where ground truth labels are available, but in real datasets common evaluation metrics (e.g.. MSE) do not apply and we must rely on proxies. As presented, our approach is designed for retrospective carbohydrate correction; more work is neces- correction. While our approach shows clear
403
405
407
408
409
410
sary to investigate its applicability to closer-to-real-tine significance for the individuals available, evaluation on a much larger population would be necessary prior to widespread adoption. Finally, while we have empirically shown that co-teaching appears to select low-noise sample, we have not proven any statistical guarantees. Despite these limitations,
411
412
413
414
we have demonstrated that is is feasible to correct a noisy variable without access to clean samples, expanding the utility of ideas the image analysis and noisy label learning to time-series reconstruc-
416
tion. Applied to domains which datastreams are composed of both individual- repporeed data and data measured from relioable sensors (e.g., mHealth), datastreams composed of series with dis- parate noise levels (e.g., extermal vs internal temperature, multiple audio recordings), our approach
418
could aid in improving the reliability of uncertain time-series data.

---

Under review as a conference paper at ICLR 2023
REFERENCES
421
Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. ICML, 2019.
Anne-Sophie Brazeau, H Mircescu, Katherine Desjardins, C Leroux, I Strychar, J.M. Ekoé, and R Rabesa-Lhoret. Carbohydrate couting accuracy and blood glucose variability in adults with type 1 diabetes. Diabetes research and clinical practice, 99, 2012.
423
425
Fox, Joyce Lee, Rodia Pop-Busui, and Jenna Wiens. Deep reinforcement learning for closed- loop blood glucose control. Proceedings of Machine Learning Research, 2020.
Lovedeep Gondara. Medical image denoising using convolutional denoising autoencoders. IEEE 16th International Conference on Data Mining Workshops, 2016.
428
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. NeurlPS, 2018.
430
432 James Arthur Harris and Francis Gano Benedict. biometric study of basal metabolism in man. Carnegie institution of Washington, 1919.
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data- driven curriculum for very deep neural networks on corrupted labels. ICML, 2018.
435
436
K wanyounn Kim and Jong Chul Ye. Noise2score: Tweedie's approach to self-supervised image denoising without clean images. NeurlPS, 2021.
437
438
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optiiization International Conference for Learning Representations, 2014.
439
440
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void learning denoising from single noisy images. CVPR, 2018.
442
Samuli Laine, Tero Karras, Jaakko Lehtinen, and Timo Aila. High-quality self-supervised deep image denoising. NeurlPS, 2019.
Kimin Lee, Sukmin Yun, Kibok Lee, Honglak Lee, Bo Li, and Jinwoo Shin. Robust inference via generative classifiers for handling noisy labels. ICML, 2019.
445
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. ICML, 2018
447
448
You Lin, Juanhui Wang, and Mingjian Cui. Reconstruction of power system measurements based on enhanced denoising autoencoder. IEEE General Meeting Power Energy Society, 2019.
449
450
Lalo Magni, Davide M. Raimondo, Luca Bossi, Chiara Dalla Man, Giuseppe De Nicolao, Boris Kovatchev, and Claudio Cobelli. Model predictive control of type diabetes: An in silico trial. Journal of diabetes science and technology, 1, 2007.
452
453
Chiara Dalla Man, Francesco Micheletto, Dayu Lv, Marc Breton Boris Kovatchev, and Claudio Cobelli. The uva/padova type diabetes simulator. J Diabetes Sei Technol., 8, 2014.
454
455
Ciny Marling and Razvan C. Bunescu. The OhioT1DM dataset for blood glucose level prediction. International Workshop on Knowledge Discovery Healthcare Data-KHD@G IJCA, 2018.
456
457
Ciny Marling and Razvan C. Bunescu. The OhioTIDM dataset for blood glucose level prediction: Update 2020. International Workshop Knowledge Discovery in Healthcare Data- (ota-KDDGG IJCA, 2020.
459
480
Baharan Mirzasoleiman, Kaidi Cao, and Jure Leskovec, Coresets for robust training of deep neural networks against noisy labels. NeurIPS, 2020.
461
462
Nick Moran, Dan Schmidt, Yu Zhong, and Patrick Coady. Noisier2noise: Learning to denoise from unpaired noisy data. CVPR, 2019.
10

---

Under review as a conference paper at ICLR 2023
Zhongzheng Ren, Raymond Yeh, and Alexander Schwing. Not all unlabeled data are equal: Learn- ing to weight data in sumervupee learning. NeurlPS, 2020.
485
466
Pablo Romeu, Francisco Zamora-Martines Paloma Botella-Rocamora, and Juan Pardo. Stacked denoising auto-encoders for short-term time series forecasting. Artificial Neural Networks, 2015.
467
468
Remei Calm Joaquim Armengol Silvia Oviedo, Josep Vehí. A review of personalized blood glucose prediction strategies for tldm patients. Int J Numer Method Biomed Eng. 2016.
469
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. JCML, 2008.
471
472
Pengxiang Wu, Songzhu Zheng, Mayank Goswami, Dimitris Metaxas, and Cho Chen. A topological filter for learning with label noise. NeurIPS, 2020.
474
Jinyu Xie. Simglucose v0.2.1 [online]. avaible: https//ggihbbbooom//xx/////mooos Accessed on: Jan-20-2020, 2018.
475
Yaochen Xie, Zhengyang Wang, and Shuiwang Ji. Noise2same: Optimizing self-supervised bound for image denoising. NeurlPS, 2020.
477
478
Peng Xiong, Hongrui Wang, Ming Liu, Suiping Zhou, Zengguang Hou, and Xiuling Liu. Ecg signal enhancement based improved denoising auto-encoder. Engineering Applications of Artificial Intelligence, 52, 2016.
480
481
Jun Xu, Yuan Huang, Ming-Ming Cheng, Li Liu, Fan Zhu, Zhou Xu, and Ling Shao. Noisy-as-clean: Learning self- supervised denoising from the corrupted image. 77P, 2020.
482
485
Xingrui Yu, Han, Jiangchao Yao, Gang Niu, Ivor W. Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? ICML, 2019.
Jinnye Zhang and Peng Yin. Multivariate time series missing data imputation using recurrent de- noising autoencoder. IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2019.
486
488
Zhong Zheng, Zijun Zhang, Long Wang, and XiongLuo. Denoising temporal convolutional recurrent autoencoders for time series classification. Information Sciences, 2022.
489

---

Under review as a conference paper at ICLR 2023
9 APPENDICES (SUPPLEMENTAL)
481
A TUNING DETAILS
482
For each model, tuning performed on simulated adult#001 using validation performance. No additional tuning was performed for other individuals noise functions. For Nois- ier2Noise, we selected a, the parameter that controls the relative noise distributions, from [0. 1...0.77..991111122...1....
493
494
ultimately selecting 1. Because we do not assume access to the exact noise we would not expect this method to perform spectacularly, but note that often outperforms other basclines.
496
498
For co-teaching methods, we performed a simple grid search over the values of [x,0(200,000] (where 500 is the minimum number of training iterations), ==100333.0.5.06667] and (0.1.0.3.0.500.7) For N+2N, we selected Es = 250, 0.333, and a = 0.1. For the super- vised setting co-teaching (SUPCT), set selected Tę 500, 0.5 and 0.3.
500
502
B ALTERNATE NOISE FUNCTIONS
We consider noise functions that might arise in carbohydrate counting. None highly dissimilar from our main analysis noise function: we aim here at feasibility, rather than comprehensive survey on broad selection of loss functions, which our method would likely be unable to address without further tuning or modification. Here, (a,, b) denotes a uniform distribution with values between and b. Carbohydrate values range between 0 and 200. After adding noise, y values are capped above and below by and 200. Alternate noise functions include:
504
506
507
508
509
Zero-mean multiplicative Gaussian: y = (1 + (N .75)zz
510
2. Negative-mean multiplicative Gaussian (primary noise function): y (1
3. Zero-mean additive Gaussian:
(0, 40)
4. Negative-mean additive Gaussian: y == + (-(30, 50)
513
5. Zero-mean multiplicative Uniform: M(.5, 1.5)z
514
6. Negative-mean multiplicative Uniform: = (0, 1.6)
515
7. Zero-mean additive Uniform: y = x ((0600, 60)
516
8. Negative-man mean additive Uniform: y = x + 41-60, 40)
12

---

Under review as a conference paper at ICLR 2023
CAE
N + 2N
NAC
NR2N
SUP
SUPCT
80-
80
90-
80
Axtrig Risk Baloing
SIM:
30-
30
30
30
20-
200
430
600
200
400
500
1000
3500
500
1900
1500
500
1003
1500
40
40
falloiing
REAL:
100
200
100 0
200
400
560
1050
2003
Magnitude Carb correction (g²)
Figure 3: Risk following the carbohydrate vs, magnitude of carbohydrate correction learned for all models and both datasets. Besides the clean autoencoder, N 2N performs best.
C
CRC PLOTS
518 With N we see higher correlation between the magnitude of carbohydrate correction and risk
519 following the meal compared baselines for both real and simulated data (Figure
13


Reviews:
[{"review_id": "oKs97hWFC4", "reviewer": "Reviewer_SnFL", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Authors present a de-noising method using auto-encoders for time series data that does not need clean data to be trained. However, the method requires an auxiliary clean signal that is related to the target signal. The method also makes assumptions about the SNR level which has to be high. To alleviate this limitation, the authors adapt a co-teaching technique to choose the less noisy examples from the training data. The authors validate their approach using synthetic data and demonstrate that the performance of their algorithm is on par or superior to similar methods. Authors illustrate the applicability of their proposal by tackling a real-world problem blood-glucose management.    ", "summary_of_the_review": "This submission is interesting and it is well executed, however the novelty and applicability are very limited and therefore its impact is small. For these reasons, I think that this submission is a bit below the threshold of the conference.   ", "strength_and_weaknesses": "Strengths\n\nThe paper's approach is well validated (using synthetic data) and clearly compared to relevant related methods. The performance of the algorithm is shown to be on par or superior to that of comparable algorithms. \n\nWeaknesses\n\nThe method itself is not really novel. As the authors state in their paper, their work mostly shows that approaches which had already been shown to work with image data can also be applied to time series data. The real-world application of the paper, although compelling, is also very limited and not really feasible. As the authors themselves explain in the paper, their approach only works to de-noise data retrospectively and cannot work for real-time de-noising as presented in the paper. \nThe method itself also makes a number of strong assumptions that further limits its potential applicability. It assumes that there exists a clean signal that is related to the target signal, and that the co-teaching method can be sufficient to increase the SNR enough for the algorithm to recover a clean version of the target.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "d3ASdxu_lH", "reviewer": "Reviewer_reMC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose an unsupervised denoising method for situations in the high noise regime with side information. They frame the problem for time-series applications, though the proposed method appears general. Their method is a variant of the Noise2Noise method (Lehtinen et al 2018), which itself builds off of Denoising Auto-encoders (Vincent et al 2008). They propose using dual independently initialized auto encoders to find subsets of the training set with low error on one encoder but not the other, assigning the remaining , which mark \"easy\" or nearly correct examples. They then learn using only these samples.\n\nThey present computational results on a ", "summary_of_the_review": "A novel dual DAE scheme involving co-teaching, with strong empirical results, but strange choices with respect to side data, and one particular missing experiment.", "strength_and_weaknesses": "Strengths:\n* The overall co-teaching DAE idea is novel and interesting, in my opinion.  \n* Experimental results are fully fleshed out, in that the baselines and upper bounds show many of the experimental conditions/ablations that I desired to see, e.g. a supervised case, a clean data upper bound, etc..  \n\nWeaknesses:\n* My major theoretical concern are cases where the hardness of differing samples under (stationary) Gaussian noise is not uniform. If the data domain is biased toward easy denoising in specific regions and hard problems in others, then it seems that this dual DAE scheme will not reconstruct the relatively harder regions where both DAEs fail, excluding them from the training set. While it is reasonable to ascribe this high error to high noise, it is also reasonable to ascribe this to the x domain also; this confounding seems unavoidable with the presented co-teaching scheme.  \n* The entire framing of the timeseries data (in the title, section 1, etc.) seems superfluous, except for the fact that the timeseries are used as side channel data. Is the additional empirical performance due to the fact that the proposed method is the only noise2noise method that includes the side-data? I think applying the NAC baseline to an expected y output would be a good experiment to test this proposition. It's not clear to me either way, but the entire side-channel/time-series context makes the rest of the analysis more complicated in my opinion.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bDMJFsg09r", "reviewer": "Reviewer_Azxw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This manuscript addresses the problem of recovering a clean signal given only the noisy data using denoising autoencoders in the context of time series data. Taking advantage of time series data, the authors use an auxiliary noise-free time series signal that is related to the target signal and a co-teaching-based approach for training DAEs. The proposed model can leverage the relationship between the auxiliary and target signals and does not get affected by not having a weak SNR. They show the benefit of their approach by using a real-world problem setup, i.e. blood glucose forecasting and control.", "summary_of_the_review": "A nice paper that addresses a relevant task, builds upon existing ideas. More elaboration could be given to the details of the method itself.", "strength_and_weaknesses": "STRENGTHS:\n1. This works is inspired by a real-world problem and contributes a method that leads to very good results on that problem.\n2. The authors are evaluating the model robustness wrt. different noise distributions and are showing that their work outperforms all baselines for most data.\n\nWEAKNESSES:\n1. I would have appreciated if the authors would have elaborated more on the usage of the auxiliary noise-free time series in the structure of the network (in both steps).\n2. While useful, the proposed method lacks novelty.\n3. It is not clear to me what the reason are for choosing any specific threshold R(t) (for L).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "9ErbTlimDEO", "reviewer": "Reviewer_yPFM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to recover clean data from corrupted inputs by denoising audoencoders (DAEs) in an unsupervised setup. Specifically it assumes a situation where ground truth does not exist but related noisy targets and a clean auxiliary signal exist. Authors proposed a co-training approach for DAE models to leverage the noisy target that is related to the clean target. This approach was applied to the task of recovering carbohydrate values for blood glucose management, where the carbohydrate values are the noisy data and the blood glucose values are the clean auxiliary signal. ", "summary_of_the_review": "Although the proposed method seems novel and interesting for the particular problem setup, the authors' experimental setup is flawed and the potential applications are quite limited. ", "strength_and_weaknesses": "Strength:\n* The paper is clearly written and the method is carefully described\n* The problem setup and approach are novel\n\nWeaknesses: \n* The experimental setup is flawed, where the authors split train/valid/test based on time for each individual. This means that individuals presented in the training set are identical to the test set. This is unrealistic and cannot demonstrate generalization of the model.\n* Although collecting data for this task is challenging, only 10 individuals in the simulated dataset and 12 individuals in the real dataset are just too few.\n* Applications of the proposed method is very limited.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject