Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work studies a method to perform non linear ICA where the observed signals $x$ are a non-linear mixing of two independent variables, the sources $s$ and the observed condition $t$: $x = f(s, t)$. The goal of the problem is to recover $s$ and a way to generate new samples $x$ by providing only samples of $x$ and $t$.

To solve the problem, the authors consider an auto-encoder architecture, where the encoder should output the sources $s$ given $x$ as input, the decoder takes the estimated source of the encoder and the condition $t$ as input, and tries to reconstruct the input $x$. A discriminator is also used, to enforce independence between the condition and the estimated sources. Training is performed by solving a min-max optimization problem, where the discriminator tries to minimize independence between the output of the encoder and the condition, while the encoder and decoder are trained to maximize a weighted sum of independence and reconstruction.

The authors demonstrate that if the signals are perfectly recovered by the AE and if the sources $s'$ output by the encoder are independent from T, then $s$ and $s'$ are identical up to an entropy-preserving transform.

The authors then demonstrate the usefulness of the framework on a variety of tasks: synthetic examples, a computer vision disentanglement example, a voice cloning experiment, and a fetal ECG extraction problem.
strengths_and_weaknesses: Strengths
-------------
- This paper is extremely well written and pleasant to read. I have learned a lot by reading it.
- The framework proposed is sound and seems novel (even though I am far from an expert on non-linear ICA)
- Theory seems sound
- The experiments are very convincing and clearly demonstrate the wide applicability of the method. They do a great work to show that these ideas go way beyond what was first proposed in [45] for voice conversion.

Weaknesses
-----------------

- The authors could more clearly delineate their work, especially regarding [45], e.g. it should be made clearer that both the architecture in Fig1 and the training objective come from this paper (I find the sentence "We remark that while the algorithm we propose here has been presented before" quite vague)
- The ground truth doppler recording could be displayed in fig.5.


Misc: typo L 218 "lossto" 
questions: - In lemma 4.1, wouldn't it be clearer to write the conclusion as $h(S|S') = 0$? 

limitations: I cannot see potential negative societal impact with this work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper introduces a novel technique for the extraction of latent variable (or hidden source) form a mixture of independent components. The work is a special case of ICA, extending on the nonlinearity and also offering an identifiable solution. The authors highlight the capabilities on four different applications, including image analysis (synthesis), voice cloning and biomedical data analysis.
strengths_and_weaknesses: Strengths: (i) The proposed approach has been tested on several applications, ranging form image analysis, voice cloning, and biomedical data analysis.
(ii) The proposed approach is quite original (to my knowledge), and the use of a discriminator is quite interesting for the extraction of a “disentangled” representation.
Weaknesses: (i) The authors have not evaluated their approach quantitively, and compared their technique with other SOFA techniques. The use of already suggested quantitative metrics is possible for FECG application, and would have been useful.
(ii) The authors did not perform an ablation study, and have not highlighted the added-vale of the discriminator part for analysis purpose. It would have been great to assess a simple VAE for fECG extraction, using the same network architecture to highlight the higher performance introduced by the use of the discriminator.

questions: The proposed approach was only evaluated qualitatively and no quantitative evaluation were provided for any of the applications. Could the authors comment why they did not use any quantitative measure (image similarity measure, quality of fetal heart rate monitoring for the fetal ECG application)?
The authors claim that there is no gold-standard for the evaluation metrics for fECG extraction. But that was the purpose of the 2013 PhysioNet/Computing in Cardiology challenge, which proposed metrics for the quality of the fetal heart rate monitoring. The authors could have assessed how the extracted source (fECG) was reliable for estimating the fetal heart rate and if it was in accordance was the fHR estimated from the Doppler signal. 
I would also refer the authors to the review Behar, et al. (2016). A practical guide to non-invasive foetal electrocardiogram extraction and analysis. Physiological measurement, 37(5), R1. This review present methods and metrics to be used for the evaluation of fECG extraction techniques.
The interpretation of figures 3, 4, 5 is also really difficult, and readers might struggle to understand the takeaway messages of these figures. For example, the explanation of the qualitative estimation of the fECG problem is given in annex C.6.4, which makes it very hard to understand and interpret the figure 5.
Could the authors comment on how their techniques are coping with noisy measurements, as it seems most of the database used have relatively low level of noise.
Could the authors discuss the impact of the choice of the encoder decoder part of the proposed approach. They suggested the use of a ResNet for image analysis, and simple CNN for fECG analysis, how did they make these decisions.
The caption of the figures should be self-explanatory, (for example figure 1 remind the reader what the different variables stand for (x, t, s’…). 
Some acronyms have been introduced without being defined beforehand. For example, line 64, iVAE was introduced without explanations, whereas VAE was introduced and explained on line 82.

limitations: The potential negative impact of the voice cloning applications are discussed, but this discussion is relegated to the annex, which I believe not to be appropriate.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies a special case of non-linear ICA where one of the latent source is known and the goal is to recover the other. 
While identifiability results are already known in this setting [27], this paper formulates its guarantees thanks to an entropy-preserving map rather than statistical identifiability and comes with multiple experiments on synthetic and real data (from very different fields).
strengths_and_weaknesses: Strength:

- The paper is overall very well written. The high level description is very clear, the goals are well described and well motivated.

- The large number of experiments on data from different field shows that the problem is indeed of interest.

- The results are overall convincing

Weaknesses:

- The novelty of the identifiability result is questionable (see the question section) 

- l162 -> 170 are a bit unclear (see the question section)

- I tried to open the notebook notebooks/2D_demo.ipynb and it failed with the error 
Unreadable Notebook: notebooks/2D_demo.ipynb NotJSONError("Notebook does not appear to be JSON: ''...")

Then, I opened the ECG folder and run Main.py. The code seems to run but with worrying warnings: 
padasip/filters/rls.py:183: RuntimeWarning: overflow encountered in matmul
  R1 = self.R @ (x[:, None] * x[None, :]) @ self.R
padasip/filters/rls.py:183: RuntimeWarning: invalid value encountered in matmul
  R1 = self.R @ (x[:, None] * x[None, :]) @ self.R
<class 'ValueError'>
("Input contains NaN, infinity or a value too large for dtype('float64').",)
Input contains NaN, infinity or a value too large for dtype('float64').
In general, the code quality contrasts strongly with the quality of the paper. It does not follow most of the coding guidelines in Python and in particular it comes without any documentation or unit tests. The lack of a clear API prevents easy reproducibility.

In order for others to be able to re-use your work, it would be nice to correct at least the 2D-demo and make it a proper python package with tests, clear documentation, examples (the 2D-demo comes as a perfect example), continuous integration (although I understand this last one is difficult to include in a submission but should definitely be there after publication). It does not take that much time to do and would definitely help other researchers build upon your work.
 

questions: - Identifiability: How does the identifiability result compare with [27] ? It seems to me that statistical identifiability is a stronger result than entropy preserving maps. Could your results be derived from [27] by letting the sources by high dimensional ?


- Unclear paragraph: 
(l163): "Correl(ˆt, t))2 = 0 implies that S′ and T are approximately statistically independent". In what sense are they approximately statistically independent ? Under which measure of independence ?

(l164-170): Could you explain why training the discriminator in a contrastive fashion leads to minimization of the mutual information between S' and T ? I also believe there is a typo l166 where you wrote $x = (s, \tilde{t})$ I believe it should be $x = f(s, \tilde{t})$. I believe I just need more details to understand this part.

- Other concerns:
 In the legend of Figure 2. Using chi square test this way is wrong, it can only be used to reject independence, not to prove it. 
limitations: Ethical limitations are properly addressed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies the problem of nonlinear independent component analysis in a special case where there are only two components and one is observed. The authors proposed an autoencoder model with a GAN-style additional regularization term which encourages the latent component to be independent to the observed one. The authors proved that the proposed method can recover the latent component up to entropy-preserving transformation. The performance of the proposed method is demonstrated on real world data.

strengths_and_weaknesses: Strength:
- A new autoencoder-based formulation for discovering one hidden component with nonlinear mixing.
- A theoretical guarantee for the discovery of the hidden component
- The proposed method is evaluated on real world data

Weaknesses:
- The proposed method can only be applied to a restricted nonlinear ICA setting. It can only handle two components, in which one needs to be observed.
- The proof only covers the ideal scenario in which the autoencoder model has zero generalization error, which is impractical. 

questions: 1. The coefficient for the independence loss lambda takes a different value for each experiment. How is it tuned? Are the results sensitive to the values of lambda?
2. The performance of the proposed method is evaluated in terms of the quality of reconstruction of the same hidden component with a new observed component. This task could also be solved by content-style factorization approaches, although content-style factorization approaches do not force the hidden component to be independent. It would be nice to compare the proposed method with content-style factorization methods.

limitations: The proof only covers the ideal scenario in which the autoencoder model has zero generalization error, which is impractical.
review_title: null

Meta-review: metareview: Thanks to the authors for this submission, which tackles an interesting and widely appearing problem with a novel approach.  The reviewers agreed that the submission is very well written, motivating the problem and detailing their approach quite clearly. 

We also thank the authors for their thorough responses to reviewer questions and comments.  One concern described is the extent to which the fECG experiment realistically and meaningfully evaluates the author’s approach.  Reviewer gAAc expressed some concerns about the quantitative metric used by the authors in the fECG experiment.  Their back and forth revealed the depth of the author’s knowledge of this application area, but did leave some questions unaddressed — namely experiments using simulated ECGs and stress tests on rare observations like premature ventricular contractions, twin pregnancies, or contractions.  Additionally, reviewer gAAc asks a pointed question — how does the heart rate estimate from the extracted fECG and the Doppler signal directly compare?  If I’m not mistaken, this comparison is feasible with the dataset analyzed, and could even be prepared in a subsequent draft. 

Despite these open questions, this work does look at a breadth of applications in their experiments, strengthening the submission greatly.  While I recommend accept, I strongly urge the authors to address these last points brought up by reviewer gAAc.  
confidence: Certain
Decision: Accept