Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper use Neurochaos Learning architecture for the classification task of cause-effect for data generated from coupled chaotic maps. Results show that NL preserves causality under a chaotic transformation and can successfully classify cause and effect time-series. One more exciting finding is that even a general NL architecture is capable of some essential causal learning. Hence, promising development of more sophisticated causal ML algorithms is required for different tasks.

strengths_and_weaknesses: Strengths:
1. This paper introduces the Neurochaos Learning architecture (NL) into the causality domain, which can be seen with more applications.
2. The proposed method consistently outperforms a five-layer Deep Neural Network architecture, which is convincing.

Weakness:
1. Lack of novelty: Neurochaos Learning is not first proposed in this work, which may hurt the contribution of this paper.
2. For O3 in #53, transfer learning is not very important in this setting, and a more detailed analysis of the proposed method itself may be helpful.

questions: 1. Lack of novelty: Neurochaos Learning is not first proposed in this work, which may hurt the contribution of this paper.
2. For O3 in #53, transfer learning is not very important in this setting, and a more detailed analysis of the proposed method itself may be helpful. Or the reviewer may want to see why transfer learning needs to be addressed.
limitations: The authors claimed well in the limitations parts.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors have shown that the brain inspired learning algorithm- Neurochaos learning is capable of performing the classification of cause-effect in four different coupled systems namely, coupled autoregressive processes, coupled 1D chaotic skew tent maps, and coupled 1D chaotic logistic maps and predator-prey.  They generated simulated datasets and performed experiments using the Master-slave configuration on three simulated and one real dataset. In addition, the authors showed that Granger causality is preserved under chaotic transformation for the coupled systems. The classification performance of the coupled configuration is compared with 5-layered DL architecture. 
strengths_and_weaknesses: 1. Authors have indicated a classification method of cause and effect for any time series data using the NL algorithm. The method shows a fair performance on the simulated and real datasets when their F1-scores were compared. Clearly, F1-scores for ChaosNet were better than DL.
2. The manuscript also demonstrates the preservation of Granger Causality in the ChaosFEX feature space.
3. Usage of “Transfer learning” like approach by testing the algorithm on a dataset completely different from the one it was trained on. 

The aim in the paper is not to build a causal ML algorithm, but rather to explore if the existing NL architecture which does classification can also distinguish between cause and effect data. The study answers this in the affirmative and slightly outperforms DNN in this regard. When we ask an important question regarding features such as whether features extracted from the learning architecture (in the cause-effect classification task) in both DNN and NL preserve causality as measured by Granger Causality (GC) and Compression-Complexity Causality (CCC). This is important because it determines whether NL is doing a causality informed classification or not. For this the authors used GC and CCC to check for causal relationship between coupled AR processes and master-slave chaotic skew tent-map system respectively (Note: In the cause-effect classification task they are not using GC and CCC, but the plain NL architecture which was used for classification). In the experiments, they show unambiguously the failure of GC for DNN features (features extracted from the layer just before the output layer), owing to the failure of cause-effect preservation. Whereas in the case of ChaosNet, using GC and CCC, the cause-effect relationship in the input data was faithfully preserved. This is a verification of causality informed classification - i.e., the output from the input layer of NL preserves cause-effect relationship. This is a very important and novel finding for developing causal NL
algorithms in the future (and not been shown in any of the previously published work on NL).

questions: 1. Authors have only used F1-score for the performance comparison. I would like to understand from the authors why are the other metrics were not computed like class-wise accuracy or precision. 
2. As the training data derived from the different coupled systems are balanced (2000), the authors have experimented on a balanced data set. However it will be important to see how the model performs on unbalanced dataset. 
3. Adaptive learning rate – how will the model perform is  a setting where the continuity of tke loss landscape is leveraged instead of treating the Learning rate as a tunable parameter.
4. Authors have experimented on data points of predator (Didinium  nasutum) and prey (Paramecium aurelia) populations,  I suggest experimenting more on datasets like Predator prey:  Lynx and snowshoe hare, Wolves vs rabbits.
5.	The predator prey systems are natural test beds for neuro chaos learning because they are governed by coupled dynamical systems. In this context can the authors comment on the stability of the learning process?

limitations: NL is not a causal ML algorithm. If the input data has causal structure, NL preserves that structure.  If the problem is not framed as classification of cause effect , then NL requires causality testing methods like GC, CCC etc to identify which is cause and which is effect.
The following are the suggestions:
1. Cause-Effect preservation is an important property for the interpretability of the algorithm. This can be used to understand why the algorithm works. Interpretability of Neurochaos features + ML algorithms can be further investigated using this property. 
2. Application of NL in the classification of breast cancer curated dataset with heavy class imbalance may be investigated. Particularly I would like to see a discussion on how NL can be used to detect anomalies where the minority class instances are possibly anomalous, like in the breast cancer data, for specific survival year cut offs.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper discusses Neurochaos learning and evaluates how it performs on a classification task where it is asked to identify the cause and the effect.

In brief, Neurochaos learning is this: a 1D skew tent map is a chaotic process. Given an input $x$, we run the process until it comes within $\epsilon$ of $x$. Various statistics for the process are then computed and these are used as the feature representation of $x$. Consider a multi-class classification task. The training data is used to create average-representations, one per class. Classification on the test data is done by computing the cosine similarity with the per-class representations.

The authors consider the task of classifying whether a time series is a cause or effect of another time series. In this task, the authors show that NL representations are found to preserve causality -- meaning, for example, that in the NL feature space, Granger causality increases with more coupled systems, as it should. 

Throughout the paper, NL and Deep Learning (DL) are compared. In general, the authors find that for this task, NL and DL achieve similar accuracy, but NL is argued to be more desirable, because of the causality-preserving properties discussed above.
strengths_and_weaknesses: ## Strengths
- Reading about Neurochaos Learning was interesting
- The authors are honest about the limitations of Neurochaos learning and do not omit plots or experiments where DL is shown to perform better. I think this is commendable.

## Weaknessess
- In the introduction, Deep Learning is criticized for having poor interpretability and trustworthiness. But the paper does not present a convincing case that Neurochaos Learning is much better on those fronts. In particular, section 2 and section 4.3 describe the basics of NL prediction and parameter-tuning, but not much interpretation of these are given. It's hard to see how NL can be said to /learn/. From the description given in section 2, the whole process seems to have more in common with non-parametric clustering methods.
- For comparison with Deep Learning, a 5 layer dense feed forward net is used. But given that the task is time-series classification, I feel a more fair comparison would be an RNN, Transformer, or even 1D CNN
- I don't think the results show conclusively that NL captures causality better than DL. Figure 2a shows that Deep learning outperforms NL in as many cases as it underperforms
- The paper claims that the NL representations exhibit the same causal relationships that are present in the input data, but the results presented (figure 2c, figure 5a) don't seem to conclusively show that. Additionally, no baselines are given for a transformation that does /not/ preserve causality, so the quality of the results are hard to judge in figure 2c. 
-  My main concern is with significance. The task domain seems extremely tailored to the type of thing NL would be good at. I would be more convinced of the usefulness of NL for causal tasks if it were clear how these methods could be applied to more complex problems. As it is, it's hard to see how this approach would be able to handle, e.g., language modeling.
questions: - Table 1: Why aren't class-0 and class-1 the same number? Aren't the causes and effects generated in pairs?
- line 89: Is the discrimination threshold $b$ the same as the skewness parameter $b$ in equation (1)?
- Section 3.1: What is the train/test split for the AR processes? Or is Table 1 applicable for sections 3.1 and 3.2?
- Section 3: Can you explain briefly what the classification task is? I have two guesses: (1) the classifier is given 2000 steps of a time series and must decide if the time series is the cause or effect. (2) the classifier is given 1000 steps of one time-series and 1000 time-steps of another time-series. The steps are concatenated into a 2000 dimensional vector. The classifier must decide if the first 1000 steps are the cause or the effect. Are either of these correct?
- Section 4.1: How is the data fed as input to ChaosNet? The input is a 2000 dimensional vector. Is each element of the vector fed as stimulus to a single GLS neuron? Or is there a k-dimensional generalization of the 1D neuron presented in section 2?
- line 173: Why were these hyper-parameters chosen? 
- Figure 1b: What is the Granger causality computed between? I have two guesses: (1) It is computed between and input time series and the vector of neuron values in the fourth layer (2) It is computed between the average neuron value between the cause and effect time series, after being pushed through the DNN
- Figure 1: What is the difference between "M causes S" and "S causes M"? Is the training data swapped? It's not mentioned in line 173, and unless I'm missing something, it's the first time the terms have been used. My best guess is that "M causes S" is the GC computed between M and S in NL feature-space, where M is the cause and S is the effect. Is this correct?

## minor things
- Line 97: Delete "For"
limitations: Limitations are adequately discussed.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: In this paper, the authors utilized a recently proposed Neurochaos Learning method for the classification of cause-effect. To verify the brain-inspired learning method, multiple experiments have been conducted on synthetic and real-world datasets. Compared to a five-layer Neural Network, the utilized NL method achieves superior performance.   
strengths_and_weaknesses: Strengths:  
1. The idea of applying a brain-inspired learning method is interesting and has not been extensively explored yet, this could potentially bring novel insights complementary to the mainstream data-driven model design.

2. The experimental results seem plausible and reflect the feasibility of the NL method compared to a baseline neural network.


Weakness:

1 Causality: I think the main drawback of this manuscript is the discussion of causality. In line 25, the authors claim that causality has been mathematically defined by Wiener et.al.. it would be nice to explicitly give the definition here, as reviewers may not familiar with this definition. Importantly, the nuance of causality definition varies from literature [1]  to literature [2]. Without presenting the exact definition of causality quoted in this paper and discussing related definitions, it makes the readers hard to understand the main idea. In terms of 'classification of cause-effect', I am not sure if this terminology makes sense or not. What does it mean by classifying cause-effect (later causality detection is brought in line 38)? I believe the authors should discuss its connection to causal variable identification. This also relates to the fact the study is conducted on observational data.  

[1] Peters J, Janzing D, Schölkopf B. Elements of causal inference: foundations and learning algorithms[M]. The MIT Press, 2017.
[2] Hernán M A, Robins J M. Causal inference. 2010.

2 Unclear model design: The model architecture and learning details are fragmented or missing. The authors could either provide a plot of model illustration, pseudo-code table, or code repository. Considering that Neurochaos Learning is not a well-known method, it is important to demonstrate integrated details to facilitate reproductivity.

3 Experimental design: The experiments regarding Coupled autoregressive (AR) processes and Coupled 1D chaotic maps etc. don't seem to be well-motivated. Could the authors reason why particularly using such setting to investigate cause-effect of time series. Lastly, the comparison to a five-layer neural network seems to be less convincing, given the rapid developments of deep learning architectures.

questions: It would be great if the author could address the issues raised in the weakness section. 
limitations: Yes, the authors fairly discussed the limitations of the method. The potential negative impact may not be applicable to this study.
review_title: null

Meta-review: metareview: This paper has been thoroughly evaluated by four competent reviewers. One of them voted strongly for accepting it, while the three others bid borderline rejections. The work tackles an important problem, and it is well written-up. The most important aspect of controversy between the reviewers’ opinions is in whether the proposed method can or cannot preserve causality. Based on my understanding, it appears designed for just that.  The authors have provided extensive rebuttals and, in my opinion, have addressed most of the issues brought up in the initial review. In summary, even though this paper should be rejected based on the straight vote of the reviewers, I would like to encourage the program committee to consider accepting it, provided that there is enough room for it in the program.
confidence: Less certain
Decision: Accept