Paper: Automata Learning for Neural Event ODEs: An Interpretable Model of Piecewise Dynamics (ICLR-2024)

Reviewer comment (Experiments):
This paper is still in its preliminary form. It is shorter, compared with other submissions to a major conference. Authors only demonstrate the evaluations on the dataset of bouncing ball, which is a toy example in neural event ODE.

Author response:
Given the page limit, our focus was on the method and discussing the well-posedness of the problem guaranteed by the abstraction layer (Sections 2.1.1 and 2.1.2). We respectfully disagree with the reviewer's perspective. We are confident that our proposed theory is sound, complete, and mature enough to be recognized as a significant contribution.

Reviewer comment (Experiments):
A lot of previous methods are mentioned and cited, but they are not compared numerically compared with the proposed method, such as LatentODE and LatSegODE. Also, ablation studies are missing, and performance comparisons should be conducted for the proposed methods with different choices of important parameters.

Author response:
Our primary contribution is the inference of an explainable model for systems implementing piecewise dynamics, without the need for their ODEs (aka. black-box ODEs). This model is immediately interpretable by humans. In contrast, LatentODEs and LatSegODEs use latent space representations that require extensive post-processing to provide some level of insight and interpretability. 
We also assert that our training method for NEVs is more efficient. It's crucial to note that segments of Event ODEs are still ODEs. The training method, initially proposed by Legaard et al. (2023) for NODEs, has been generalized by us to the context of NEVs. This generalization was facilitated by our first contribution, which ensures the well-posedness of the problem and immediately reveals all ODE segments. While other methods could theoretically benefit from our proposed data slicing technique (as Chen et al. (2021) manually performed such slicing in their implementation), we believe our focus on eXplainable AI outweighs performance comparisons with these methods.

Reviewer comment (Experiments):
Authors claim that the proposed method has reduced computational cost. But convincing experimental evidence needs to be shown. Neural event ODE methods always have significant computational cost, including the proposed method, so its comparison with previous methods is needed.

Author response:
Our primary contribution is the inference of an explainable model for systems implementing piecewise dynamics, without the need for their ODEs (aka. black-box ODEs). This model is immediately interpretable by humans. In contrast, LatentODEs and LatSegODEs use latent space representations that require extensive post-processing to provide some level of insight and interpretability. 
We also assert that our training method for NEVs is more efficient. It's crucial to note that segments of Event ODEs are still ODEs. The training method, initially proposed by Legaard et al. (2023) for NODEs, has been generalized by us to the context of NEVs. This generalization was facilitated by our first contribution, which ensures the well-posedness of the problem and immediately reveals all ODE segments. While other methods could theoretically benefit from our proposed data slicing technique (as Chen et al. (2021) manually performed such slicing in their implementation), we believe our focus on eXplainable AI outweighs performance comparisons with these methods.

Reviewer comment (Novelty):
Authors propose to use L* for automaton learning. Why to choose this method? Is there any better choice? The motivation should be presented more.

Author response:
If the reviewer is interested in other automata learning algorithms such as Aalergia, we selected L* because it's a widely recognized active learning algorithm that offers PAC guarantees. If the question pertains to the specific use of L* in this context, we recommend referring to the revised introduction, which explains how the work of Bloem et al. (2020) influenced our choice.

Reviewer comment (Theory):
Since the automaton learning algorithms always need a lot of data to cover all the possible prefixes, can authors provide some complexity analysis on the minimum amount of necessary training data?

Author response:
This isn't entirely accurate, particularly when compared to NNs learning the same system. Studies have shown that L* requires up to three orders of magnitude fewer samples in some instances, and it can still enhance NN training (Meinke et al., 2011; Aichernig et al., 2019).
Furthermore, Automata Learning has been and continues to be the preferred approach for numerous academic and industrial applications, ranging from reverse engineering legacy systems to formally verifying the safety and security of systems. For more information, refer to Vaandrager (2017) and Fisman (2018).

[0] Meinke, K., et al. "Learning-Based Software Testing: A Tutorial." In: ISoLA 2011. Communications in Computer and Information Science. https://doi.org/10.1007/978-3-642-34781-8_16

Reviewer comment (Experiments):
Can the proposed method handle the situation of noisy data? Can the automaton still be learned from noisy data?

Author response:
Such scenarios could be considered SDEs. For SDEs, we can use stochastic automata learning. Since our method is predicated on a change detector, we would first recommend a robust predicate evaluation and only resort to complex automata formalisms if the issue persists. Lastly, it's important to remember that none of the other methods can handle SDEs as they represent a different class of systems, that is outside our scope.

Reviewer comment (Experiments):
Please include comparison with alternative methods to verify the improvements over state of art.

Author response:
Regrettably, the state-of-the-art is unable to learn a model that is comparably explainable or interpretable. Also, some studies provide artefacts that are tailored to their setup & do not easily extend to other examples.

Reviewer comment (Experiments):
The proposed approach seems to hinge on whether the human designer has a decent knowledge of the dynamics of the plant so that the designer can provide the predicate for the automata learner to capture the event change.

Author response:
**On predicates:**
Thanks for your comment. Our method doesn't strictly require user-provided predicates. The automata learning process only needs the ODE variables. We can write a linear predicate in one variable as $ax + b > 0$, where $a$ and $b$ are symbolic values. This allows us to enumerate symbolic predicates. We iteratively learn an abstract model of the ODE (i.e., $\mathcal{A}_f$), and if $\mathcal{A}_f$ detects an event, we update $\Theta$ and continue with the next symbolic predicate. Later, we learn the actual parameters of the symbolic predicates as described in Section 3.2.

Reviewer comment (Theory):
The author did not illustrate the specification construction approach in organized manner. All the procedures are described in the pattern of "what I did" rather than explaining the motivation first. There is no soundness and completeness analysis for the automata learning approach in the paper. Let alone the query complexity and efficiency. If the proposed approach can be reduced from the L* algorithm, author should at least illustrate and prove it.

Author response:
**On Specification construction:**
You're right. We aimed to use the L* algorithm in its original form. The L* learner differentiates the locations of $\mathcal{E}_f$ based on the immediate output of the function and the state update set returned by the look-ahead mechanism. Transitions that have an empty set of state updates do not specify jumps in the ODE’s dynamics. The motivation for constructing the specification $\mathcal{S}_f$ is to obtain a compressed automaton structure that resembles Hybrid Automata. This enhances the readability and interpretability of the automaton.
We revised this section in the paper.

**On soundness and completeness:**
The conflicting behavior of the system as discussed in (Bloem et al., 2020) suggests that if there is a conflicting history then the original L* algorithm is incapable of learning a transducer. In our context, this is not the case as the (arithmetic) functions would not produce any conflicting history so the soundness and completeness of the proposed approach depends on

1. The L* algorithm that requires the complexity analysis for the Membership and Equivalence queries
2. The correctness of the abstraction layer

Regarding the latter, in Section 2.2. we describe the correctness of the abstraction layer.
Concerning the former, the complexity of the Membership queries and coverage of the Equivalence queries depend on the ODE's characteristics and a variety of factors (e.g., the initial value, and boundary conditions). We are well-aware that we introduced a methodology whose complexity is parameterized by its SUL, though the unconventional nature of this SUL provides an immense opportunity for further studies and future investigations in this line of research to distinguish different complexity classes that might be attributed to distinct dynamic classes. For example, the MQ's complexity depends on the ODE's output vector length and the completeness of the method depends on the coverage on the ODE's state space achieved by the Equivalence Oracle. However, we believe providing such rudimentary speculations about the method's complexity, soundness and completeness will not provide any valuable insight given the current page limits of the submission; therefore, we incorporated this comment in the revision but we will thoroughly address the soundness and completeness concerns in an extended version.

Reviewer comment (Experiments):
There is no numerical experiments for the proposed approach. There is no qualitative nor quantitative evaluation for the proposed approach.

Author response:
**On experiments:**
We added two more examples to the paper. We believe the proposed event specification learning also complements other Neural ODE learning methods; therefore, all methods could theoretically benefit from the proposed data slicing. We defer this comment to future work as we think it is essential to focus on proposed theory, and demonstrate its applicability and well-posedness using a simple training. Lastly, some of the state-of-the-art methods do not provide a reproducible, reusable artefact.

Reviewer comment (Reproducibility):
The paper misses a description of the actual learning algorithm. It seems to be a variant of L*, but too many details are left to the reader to decipher.

Author response:
Firstly, we'd like to clarify that our work does not propose an extension of the L* algorithm for learning DFAs or Mealy machines. Our methodology involves using L* to learn an abstract Mealy machine, with an abstraction layer that is comprehensively described in the paper (please see 2.1.1. and 2.2.2.). Additionally, we incorporate a CacheTree in our Membership Oracle, which employs a simple look-ahead mechanism. This mechanism is designed to discover state updates of the ordinary differential equations (ODEs). The output to each membership query is the valuation of the predicate on the immediate output of the function, plus a set comprising outputs for all extensions of the current query. Such a look-ahead mechanism does not involve extending the L* algorithm niether for DFAs nor for Mealy machines.

Reviewer comment (Novelty):
How is L* being adapted for your setting? It is assumed that this is not a straightforward application of L*. A lot of the formalism provides a foundation for L*, but it is difficult to see how L* has a novel adaptation in this domain.

Author response:
1. In our research, we did not adapt the L* algorithm. Instead, we applied it in the context of abstract automata learning. The application of L* in our setting is straightforward and does not involve any modifications to the algorithm itself.
2. Our work does not aim to adapt L* in a novel way. Instead, we leverage the existing properties of L*. As Bloem et al. (2020) demonstrated, in the absence of conflicting history of computations, the L* algorithm converges to a transducer. If there is a conflicting history, the algorithm must split. However, in our case, since (arithmetic) functions produce one value for each input, there should not be any conflicting history in the function's history of computation. If there were, it would not be a function. Therefore, the only adaptation necessary for using L* in this field of research is devising the appropriate abstraction layer.

Reviewer comment (Writing):
Minor comment — The phrase "between the them" should be changed to "between them".

Author response:
We’ve addressed the minor comments and fixed the typos in the paper’s PDF.

Reviewer comment (Theory):
The paper might not thoroughly discuss the assumptions and limitations of the proposed method, which is important for understanding its scope and potential constraints.

Author response:
2. We have discussed the weaknesses, applications, limitations and rooms for improvements to the best of our knowledge.\
        2.1. We made minimal assumptions on the ODE and its events. Subsequently, we used minimal domain or expert information in designing and training the Neural Networks.\
        2.2. We made no assumptions defining the proposed abstraction layer for automata learning.