Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: In this work, authors suggest a gradient-based semi-supervised approach to infer the parameters of ODEs for producing a bifurcation diagram. A bifurcation measure is defined that uses the determinant of the state-space Jacobian as an indicator for bifurcating parameter regimes in the unsupervised term of the cost function. They demonstrate parameter inference with minimal models which explore the space of saddle-node and pitchfork diagrams and the genetic toggle switch from synthetic biology.
main_review: 1. The results of this work seem very interesting and important
2. The paper is well written and clear; but some math parts still need more clarification
3. I should point out some comments that need to be addressed (please see below):

- In line 79, authors have been mentioned that "a co-dimension
one bifurcation can be defined by a set of conditions on the determinant of the Jacobian |∂F_θ\∂u|". But, based on Sotomayor's Theorem (for codimension-1 local bifurcations), it is only one sufficient condition and to investigate such bifurcations we also need some conditions on "∂^2F_θ /∂u^2" and "∂F_θ\∂p" (although they have discussed such conditions briefly in Appendix). So, this sentence is misleading & confusing. 

- In Appendix B, it is mentioned "If we are not specifically looking for transcritical or pitchfork bifurcations, it is sufficient to consider a non-zero Hessian determinant and non-zero vector slope (Eq. (21))".  But, it is not correct; please see Sotomayor's Theorem for more clarity. Especially, since for saddle-node bifurcations the conditions in Eq.(21) are not enough and such conditions depend on **left eigenvectors** as well. 

- Although the authors have mentioned different kinds of bifurcations, from Eq.(2) the type of bifurcation is not clear and they have only discussed relationships between conditions (2) and the conditions for saddle-node bifurcation. Is it possible to add any other conditions from which the type of bifurcation is determined as well? Otherwise, the authors should clarify that (2) is related to only one bifurcation e.g. saddle-node bifurcation.

- In line 91, it is mentioned "Pitchfork bifurcations are special cases of the saddle-node where a single steady state splits into two stable and one unstable steady state (Figure 1B)." But, this is mathematically **incorrect** since pitchfork bifurcations are **not** special cases of the saddle-node bifurcations; please see Sotomayor's Theorem for more clarity. 
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors present an approach to inference of parameters of a limited scope of dynamical systems with codimension one bifurcations from user-defined number and location of bifurcation points. The approach solves the parameter inference problem by gradient-based optimization function consisting of two terms. A term matching the provided to the estimated bifurcation locations and a term forcing desired properties of the determinant of the Jacobian of the system. The approach is demonstrated on two simple examples, minimal models and a synthetic toggle switch.
main_review: The paper is well organized and written in a clear and understandable manner. The task that the authors aim to solve is, in general, relevant. Their solution somewhat novel. Previous work on solving this task exists and the authors acknowledge it but don't consider it for direct comparison. Previous work on biasing a goodness of fit with domain knowledge drive the optimization exists, especially coming from the domain of modeling dynamical systems for systems and synthetic biology, however it is not reported in this paper.

The definition of the task being solved as being semi-supervised might not be appropriate, as it doesn't fit the frame defined by the machine learning community, from where the term is borrowed. The inference of parameters of dynamical systems, given measurements, qualitative or quantitative descriptions of desired or expected behavior better fits a supervised learning task. The task definition and the theoretical analysis are sufficient and convincing.

The aimed application of the presented work is to inverse problems from biology or engineering. However, the presented applications are toy models and a small synthetic biology model. Inverse problems in biology in general are much more complex than the examples presented in this work and rarely fall in the scope of dynamical systems with codimension one bifurcation. Furthermore, inverse problems from the domain of biology have consistently been proven to be difficult to solve due to being underdetermined, lack of data or various sources of noise. Although the authors acknowledge the model selection problem, they address it superficially in the conclusion section. Additionally, although their minimal models are extensible in the number of parameters they do not analyse their impact on the computational cost. The application of the presented approach to such real world problems is therefore severely limited.

limitations_and_societal_impact: The presented work doesn't have negative societal impact.

The authors acknowledge the limitations of their work adequately. The main limitation that is not addressed however is the limited applicability of the approach. See main review.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper introduces a method for fitting parameterised differential equations to qualitative behaviour: the location and number of bifurcations. Several new techniques are introduced: a loss for matching bifurcations to target locations, a loss for producing the correct number of bifurcations, and finally an approach to differentiating these terms via implicit differentiation methods.
main_review: ### Summary

This paper is excellent.

The problem considered is difficult, interesting, and has a clear audience.

The technical contributions of the paper include several techniques, which to the best of my knowledge are new, technically sound, and frankly quite ingenious.

The clarity of the presentation is generally good. I was able to understand the paper without too much difficulty. I do have some suggestions for improving this below.

For these reasons I definitely recommend acceptance.

### Discussion

The experiments are undoubtedly the weakest part of the paper. Only two are considered (three including the complexity study), and these are on relatively simple synthetic problems. Extensions to less toy, practical, problems would substantially strengthen the paper. In other words, what are some impressive use cases that help sell this technique?

The differential equations considered are essentially straightforward traditional parameterised differential equations. Do the authors envisage applications to neural differential equations, no doubt on more complicated problems? Perhaps combined with a traditional supervised loss to fit the quantitative behaviour. For example, much of my own work involves fitting neural differential equations to complicated dynamical systems, for which no mechanistic description is available -- only broad qualitative descriptions. It would certainly pique my interest to see an example of this type.

Given that there is a little extra space at the end of the paper: I think some (brief, intuitive) discussion on the steady-state solver(s) would be the most helpful piece of additional background material to include.

### Minor points

Line 86, equation (5), and elsewhere: $\frac{d}{ds}|\frac{\partial F_\theta}{\partial u}|$ does not feature an $s$ to differentiate with respect to. Moreover the definition of the curve parameterised by $s$ (presumably a level set of $F_\theta$) is not made clear. Leaving these details implicit hurts readability.

Equation (2) is lacking a dividing line between the $p$ and the $\exists$.

The upper graphs in Figure 1 could do with a y-axis label $u$.

Section 3.1: it would be helpful to explicitly reference the model (from earlier) which is being optimised. The optimiser used is not stated.

I do not think I would describe the approach as semi-supervised, or that $(|P| - |D|)\log\Psi$ represents an unsupervised loss. Whilst these terms are often blurred and without precise meaning: I would take semi-supervised learning to mean that some of the training data could have had a label, but is missing. (For example, training to perform MNIST image classification, except only some 10% of the data has its label.) Likewise I would describe $(|P| - |D|)\log\Psi$  as a supervised loss: it pushes together $|P|$ and $|D|$.
limitations_and_societal_impact: No discussion is included, but I think this is fine for a work of this type.
review_title: null

Meta-review: All reviewers agree that the paper propose an interesting approach to the problem of estimating parameters in differential equations. Although some reviewers have some technical concerns at their first reviews, they satisfy the authors' responses to address those. It seems that, although there are some points that should be modified from the current form including the terms used in the paper (such as semi-supervised learning), I think we can expect the authors modify the paper in the camera-ready by reflecting the discussion. Based on these, I recommend acceptance (poster) for this paper.
Decision: Accept (Poster)