Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to recover clean data from corrupted inputs by denoising audoencoders (DAEs) in an unsupervised setup. Specifically it assumes a situation where ground truth does not exist but related noisy targets and a clean auxiliary signal exist. Authors proposed a co-training approach for DAE models to leverage the noisy target that is related to the clean target. This approach was applied to the task of recovering carbohydrate values for blood glucose management, where the carbohydrate values are the noisy data and the blood glucose values are the clean auxiliary signal. 
strength_and_weaknesses: Strength:
* The paper is clearly written and the method is carefully described
* The problem setup and approach are novel

Weaknesses: 
* The experimental setup is flawed, where the authors split train/valid/test based on time for each individual. This means that individuals presented in the training set are identical to the test set. This is unrealistic and cannot demonstrate generalization of the model.
* Although collecting data for this task is challenging, only 10 individuals in the simulated dataset and 12 individuals in the real dataset are just too few.
* Applications of the proposed method is very limited.
clarity,_quality,_novelty_and_reproducibility: * The paper is present clearly and easy to understand. 
* The DAE model for time-series data seems novel. But the co-training method has been proposed before.
* Reproducibility - not sure. 
summary_of_the_review: Although the proposed method seems novel and interesting for the particular problem setup, the authors' experimental setup is flawed and the potential applications are quite limited.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Authors present a de-noising method using auto-encoders for time series data that does not need clean data to be trained. However, the method requires an auxiliary clean signal that is related to the target signal. The method also makes assumptions about the SNR level which has to be high. To alleviate this limitation, the authors adapt a co-teaching technique to choose the less noisy examples from the training data. The authors validate their approach using synthetic data and demonstrate that the performance of their algorithm is on par or superior to similar methods. Authors illustrate the applicability of their proposal by tackling a real-world problem blood-glucose management.    
strength_and_weaknesses: Strengths

The paper's approach is well validated (using synthetic data) and clearly compared to relevant related methods. The performance of the algorithm is shown to be on par or superior to that of comparable algorithms. 

Weaknesses

The method itself is not really novel. As the authors state in their paper, their work mostly shows that approaches which had already been shown to work with image data can also be applied to time series data. The real-world application of the paper, although compelling, is also very limited and not really feasible. As the authors themselves explain in the paper, their approach only works to de-noise data retrospectively and cannot work for real-time de-noising as presented in the paper. 
The method itself also makes a number of strong assumptions that further limits its potential applicability. It assumes that there exists a clean signal that is related to the target signal, and that the co-teaching method can be sufficient to increase the SNR enough for the algorithm to recover a clean version of the target.  
clarity,_quality,_novelty_and_reproducibility: Clarity

The paper is sufficiently clear, although the complexity the algorithm makes it a bit difficult to easily understand parts of the methods section.

Quality

The paper's approach seems sound and it is well validated. The novelty and the applicability of the algorithm to real-world problems is limited as described in the weaknesses of the paper section above, which reduces the quality of this submission.

Novelty

Very limited, authors leverage already existing approaches and apply them to time series data. The novel result is that those already known approaches also work for time-series data.

Reproducibility

The authors provide enough information to enable others to reproduce their work. I have not personally attempted to reproduce the key results of the paper, but I believe this will be easily possible once the source code is released. 
summary_of_the_review: This submission is interesting and it is well executed, however the novelty and applicability are very limited and therefore its impact is small. For these reasons, I think that this submission is a bit below the threshold of the conference.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose an unsupervised denoising method for situations in the high noise regime with side information. They frame the problem for time-series applications, though the proposed method appears general. Their method is a variant of the Noise2Noise method (Lehtinen et al 2018), which itself builds off of Denoising Auto-encoders (Vincent et al 2008). They propose using dual independently initialized auto encoders to find subsets of the training set with low error on one encoder but not the other, assigning the remaining , which mark "easy" or nearly correct examples. They then learn using only these samples.

They present computational results on a 
strength_and_weaknesses: Strengths:
* The overall co-teaching DAE idea is novel and interesting, in my opinion.  
* Experimental results are fully fleshed out, in that the baselines and upper bounds show many of the experimental conditions/ablations that I desired to see, e.g. a supervised case, a clean data upper bound, etc..  

Weaknesses:
* My major theoretical concern are cases where the hardness of differing samples under (stationary) Gaussian noise is not uniform. If the data domain is biased toward easy denoising in specific regions and hard problems in others, then it seems that this dual DAE scheme will not reconstruct the relatively harder regions where both DAEs fail, excluding them from the training set. While it is reasonable to ascribe this high error to high noise, it is also reasonable to ascribe this to the x domain also; this confounding seems unavoidable with the presented co-teaching scheme.  
* The entire framing of the timeseries data (in the title, section 1, etc.) seems superfluous, except for the fact that the timeseries are used as side channel data. Is the additional empirical performance due to the fact that the proposed method is the only noise2noise method that includes the side-data? I think applying the NAC baseline to an expected y output would be a good experiment to test this proposition. It's not clear to me either way, but the entire side-channel/time-series context makes the rest of the analysis more complicated in my opinion.  
clarity,_quality,_novelty_and_reproducibility: In general Figure 1 is not clear. While it can be understood somewhat after reading the text, overall it is a missed opportunity to give the procedure the co-teaching scheme graphically. Instead, it mostly serves as a separate textbox. Second, the presentation of the same content of the text is dense. This is understandable given careful reading, but it is the most dense portion of the text where it should be the most lucid.

Overall, this seems like a novel application of co-teaching, and seems simple enough to reproduce with the explicit exception dependence on the time-series/side-channel data.
summary_of_the_review: A novel dual DAE scheme involving co-teaching, with strong empirical results, but strange choices with respect to side data, and one particular missing experiment.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This manuscript addresses the problem of recovering a clean signal given only the noisy data using denoising autoencoders in the context of time series data. Taking advantage of time series data, the authors use an auxiliary noise-free time series signal that is related to the target signal and a co-teaching-based approach for training DAEs. The proposed model can leverage the relationship between the auxiliary and target signals and does not get affected by not having a weak SNR. They show the benefit of their approach by using a real-world problem setup, i.e. blood glucose forecasting and control.
strength_and_weaknesses: STRENGTHS:
1. This works is inspired by a real-world problem and contributes a method that leads to very good results on that problem.
2. The authors are evaluating the model robustness wrt. different noise distributions and are showing that their work outperforms all baselines for most data.

WEAKNESSES:
1. I would have appreciated if the authors would have elaborated more on the usage of the auxiliary noise-free time series in the structure of the network (in both steps).
2. While useful, the proposed method lacks novelty.
3. It is not clear to me what the reason are for choosing any specific threshold R(t) (for L).

clarity,_quality,_novelty_and_reproducibility: The paper is written clearly, but I would wish the method was laid out more clearly.
summary_of_the_review: A nice paper that addresses a relevant task, builds upon existing ideas. More elaboration could be given to the details of the method itself.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes to recover clean data from corrupted inputs by denoising audoencoders (DAEs) in an unsupervised setup. Specifically it assumes a situation where ground truth does not exist but related noisy targets and a clean auxiliary signal exist. Authors proposed a co-training approach for DAE models to leverage the noisy target that is related to the clean target. This approach was applied to the task of recovering carbohydrate values for blood glucose management.

Strength:

The paper is clearly written and the method is carefully described. The problem setup and approach are interesting.

Weaknesses:

The experimental setup is quite different from that of typical machine learning applications, it is hard to tell if the methodology is generalizable to more typical/popular domains (audio, video). The method is not backed up by rigorous/theoretical arguments; the reasoning in Section 4 is not formal and appears hand-wavy. Due to the lack of technical development and evidence of wide applicability, I think the paper is more suitable for a specialized venue than ICLR.
justification_for_why_not_higher_score: I believe ICLR papers should have significant development in the understanding of methodology. The current paper is lacking in the "understanding" part and is not interesting to the general ICLR audience.
Decision: Reject