Predicting Time-Varying Flux and Balance in Metabolic Systems using Structured Neural ODE Processes

Authors: Santanu Rathod, Pietro Lio, Xiao Zhang
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Neural ODEs, Metabolic Networks

Abstract:
We develop a novel data-driven framework as an alternative to dynamic flux balance analysis, bypassing the demand for deep domain knowledge and manual efforts to formulate the optimization problem. The proposed framework is end-to-end, which trains a structured neural ODE process (SNODEP) model to estimate flux and balance samples using gene-expression time-series data. SNODEP is designed to circumvent the limitations of the standard neural ODE process model, including restricting the latent and decoder sampling distributions to be normal and lacking structure between context points for calculating the latent, thus more suitable for modeling the underlying dynamics of a metabolic system. Through comprehensive experiments ($156$ in total), we demonstrate that SNODEP not only predicts the unseen time points of real-world gene-expression data and the flux and balance estimates well but can even generalize to more challenging unseen knockout configurations and irregular data sampling scenarios, all essential for metabolic pathway analysis. We hope our work can serve as a catalyst for building more scalable and powerful models for genome-scale metabolic analysis.

Decision: reject
Reviewer scores: 5, 3, 3, 3

Reviews and discussion:
<thread>
creation date:
1730268827421

thread_type:
official_review

<thread_object>
creator: reviewer_uxun

<content>
summary:
In this submission, the author propose a data-driven solution to Dynamic Flux Balance Analysis (DFBA), a classical tool from Systems Biology to analyze genome-scale metabolic network. At each timestep, DFBA solves a time-varying constrained optimization problem where the goal is to determine the value of the fluxes that lead to maximal biomass production, under time-varying biological constraints, which typically can only be brought by a  deep understanding of the system at hand.

Leveraging gene expression data in both control and knockout experiments thanks to single-cell flux estimation analysis, the authors aim at predicting time-resolved unseen gene expression, flux, balance, in both control and knockout experiments. This is achieved by means of Neural ODE processes (NODEP). Particularly, the main contribution of the paper is to cast DFBA in the framework of NODEP, and to highlight the need for a modification of NODEP to better handle the 1) irregularly sampled data and 2) non-gaussianity of certain distributions involved. This leads to Structural NODEP (SNODEP).

Finally, the authors carry a number of experiments on different metabolic pathways, demonstrating the clear improvement provided by SNODEP, particularly for sparsely sampled data across time.

strengths:
- The paper is well-organized and well-motivated by a biological question.

- The empirical assessment clearly demonstrate the superiority of the proposed method.

weaknesses:
- There is little to no technical novelty. The paper feels like the application of Neural ODE processes to DFBA, with minor tweaks to make it work better. Although the results are convincing, and there is definitely value in framing the DFBA problem in terms of Neural ODE processes, I am not sure that ICLR is a relevant venue for this work.

questions:
None
</content>
</thread_object>

</thread>
<thread>
creation date:
1730580849763

thread_type:
official_review

<thread_object>
creator: reviewer_djzo

<content>
summary:
This papers proposes a new method (SNODEP) to predict temporal flux and balance in metabolic systems. The architecture relies on a Neural ODE approach where they model the joint temporal evolution of scRNAseq, flux, and balance. The model is then trained to predict the future of the trajectories based on a contextual window. The authors state that their machine learning contributions includes modifying the emission distribution, as well as a new structured encoder approach for encoding context. The experiment section shows comparison with neural processes and neural ode processes, and shows that SNODEP performs better on scRNAseq forecasting, as well as flux and balance prediction.

strengths:
Predicting flux and balance from metabolic systems in a temporal way is a very interesting topic and the Neural ODE formalism seems well adapted for this task. Therefore, the motivations of the paper are clear and the intended impact is evident.

The results suggest than SNODEP outperforms the considered baselines, highlighting its potential for metabolic predictions.

weaknesses:
A. The contributions of this paper are insufficient from a machine learning perspective.

1. changing the emission distribution of a neural ODE model is typically considered a design choice in all existing NODE architectures. Such that choosing a Poisson emission can hardly qualify as a contribution. 
2. I am afraid that the identified limitation of the lack of structure for the conditioning is artificial. The authors state that : "The encoded representation is calculated using context points without any particular structure in NODEP. [...] The order between the context points and their sequential dependence on each other is not efficiently utilized." This may be the case for the specific NODEP model the authors refer too but this statement is inaccurate for most NODE models such as RNN-ODE (Rubanova et al.) or GRU-ODE (De Brouwer et al.)

Given that these are the two contributions laid out by the authors, the machine learning impact is minimal. That said, that does not take away anything from their impact for biological modeling (metabolic).

B. Modeling problems.

1. The authors appear to consider that cells taken at different time points form a time series. However, they recognize that "The challenge, however, lies in that gene expression trajectories for individual cells cannot be tracked over time since cells die once their gene expression is read.". I could not find anything in the present paper that directly addresses that challenge. Instead, it seems different cells at different time points are directly taken as a time series, which seems inaccurate. At the very least, the authors should make this assumption clear and motivate it.

C. Results.

1. Most of the results are given in the form of training curves which make rapid evaluation of the performance difficult. The scales of some subplot is sometimes off, such that the different methods cannot be distinguished.
2. I am pretty sure that RNN-ODE and GRU-ODE could also be used for this problem. I would encourage the authors to expand their list of baselines. I would also include very simple baselines like RNNs - or even one-step-ahead linear regressors.

questions:
- Can you please address the concern regarding the destructive measurement process of scRNAseq ? It seems different cells at different time points are directly taken as a time series, which seems inaccurate.

- I didn't fully understand why the distribution of the latent is that important, given that you end up modulating it with a non-linear function for the final predictions. Could you please motivate this choice more rigorously ? Line 260 - Latent distributions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730590464398

thread_type:
official_review

<thread_object>
creator: reviewer_j9xr

<content>
summary:
The paper proposes a forecasting architecture based on Neural Ordinary Different Equation Processes (NODEP) which they call Strutrude Neural ODE Process (SNODEP) and applies it to the problem of forecasting gene expression and metabolic flux and balance values across time. More specifically, given a time series of single-cell gene expression data, the authors first generate metabolic flux and balance values from this data using the previously published [scFEA algorithm](https://genome.cshlp.org/content/31/10/1867.short). They do this both for the actual data as well as for synthetic genetic knockout data produced by overwriting some genes' expression value to zero and then applying the scFEA algorithm to this semi-synthetic dataset. Given a partially observed time course of this data, the task then is to predict the value at future time points.

In order to solve the task, the authors modify the original NODEP architecture. NODEP parametrizes distributions over ODEs through an encoder-decoder framework. In this framework, the authors change the latent distribution from Gaussian to Log-normal and the sampling distribution/loss function from Gaussian to Poisson to better match the nature of the gene expression data. Moreover, they replace the order-invariant encoder architecture of the original NODEP framework with time-structured networks such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU).

They apply this framework to one dataset of human pluripotent stem cell differentiation and observe improved MSE performance compared to Neural Process (NP) and NODEP baselines.

strengths:
The specific proposed changes to the NODEP architecture look novel to me.

weaknesses:
* **Limited overall contribution:** The paper makes two contributions, introducing the metabolic system forecasting problem as a test case of interest and developing a novel architecture to solve it. In the following, I will outline why I think it does not hit the mark in either of those areas.
* **Poorly motivated metabolic system forecasting problem**
    * While understanding metabolic flux and balances is useful, the authors don't make a strong point for why the forecasting problem is specifically impactful. They consider an 80%/20% data split, so if they could show near-perfect prediction of future values, this would correspond to a 20% saving in experimental budget. Is that impactful enough in this domain for people to adopt this? Moreover, they make no attempt in characterizing how good is "good enough". The only evaluation shown is mean squared error. Are there specific scientific questions that can be solved or cannot be solved at the attained accuracy as compared to the baselines?
    * ODE systems often come with the advantage of understanding inherent dynamics of the underlying system. It seems to me like the NODEP formulation foregoes this in favor of an abstract latent space. However, even latent spaces could lend themselves to interpretations.  Unfortunately, the authors do not show any interpretation of the learned systems.
* **Narrow scope of ML contributions**
    * While SNODEP definitely seems superior to NODEP in the presented experiments, the paper leaves me wondering which of the changes (different neural network architecture, different loss function, different latent distribution) is responsible for the performance boost. This could easily be explored in ablation studies.
    * To meet the bar of a significant machine learning contribution, I would have liked to see comparisons of SNODEP on problems in the original NODEP paper and related subsequent publications, not only one very bespoke biological problem setup.
* **Limited choice of baselines:** The considered baselines are verry narrow in scope, essentially copied over from the NODEP paper. If I understand the metabolic systems prediction task correctly as a time series prediction task, what about trying simpler methods (like LSTM or GRU directly applied to the data) or time series foundation models?
* **Poor presentation of problem & results:**
    * I could not fully understand the inputs and outputs to the method (see questions below).
    * I am confused as to why all the MSE results are shown as MSE curves over training epochs. This suggests no replicate experiments were performed and thus that no uncertainty quantification on the results can be performed. It would have been simpler to summarize the final MSE values as bar plots or tables with error bars/intervals.

questions:
* I did not fully understand the inputs and outputs to the method. For example, are the gene expression values and metabolic flux and balance values treated as separate time series or does the method process them jointly?
* Could the problem be set up as a regular prediction task, with a simple LSTM or GRU architecture to predict, or are there specific problem characteristics that force the use of ODE based architectures?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730685375786

thread_type:
official_review

<thread_object>
creator: reviewer_vvou

<content>
summary:
This paper addresses the challenging problem of predicting metabolic flux and metabolic balance from single-cell data. The authors formulate this prediction problem as learning the underlying dynamics of metabolic flux and metabolic balance and propose a novel method, structured neural ODE process (SNODEP), based on neural ordinary differential equations (Neural ODEs). Through extensive empirical experiments, the authors demonstrate that their novel method, SNODEP, yields improved performance on metabolic flux and metabolic balance prediction relative to baseline methods.

strengths:
The proposed method, SNODEP, presents several key contributions that help address the challenging problem of metabolic flux and balance modelling and prediction: 

- SNODEP is able to effectively and efficiently model the dynamics of metabolic flux and balance, normally a challenging endeavour, by formulating metabolic flux and balance prediction as a data-driven problem. 
- SNODEP is evaluated over an extensive set of different pathways and experiments to showcase its utility for predicting metabolic flux and balance.
- The proposed method can learn the dynamics of metabolic flux and metabolic balance in regularly and irregularly sampled time-series settings as well as under various gene-knockout conditions.

weaknesses:
Although this work presents a novel perspective for tackling the problem of predicting metabolic flux and metabolic balance under a data-driven framework, it contains several unaddressed items that limit the overall quality and contribution of this work. For example:

- At times, the presentation and exposition of text in the manuscript is difficult to follow. For example, the learning objective in the problem formulation section, which appears to be a key section of the paper, is not very clear. Moreover, equations throughout the text don't have labels -- it would be helpful to numerically label all equations and reference them in the text. See the questions below for further details.
- Evaluation metrics are limited, only the mean squared error (MSE) metrics is considered. This provides a narrow view and comparison of model performance relative to baselines. The task appears to be to predicting over distributions of cells, in which case it is worth considering a wider range of metrics that quantify prediction performance over distributions (see questions below). 
- It feels that the novelty and methodological contribution is limited. SNODEP incorporates a variety of data-driven learning tools, including Neural ODEs, variational inference, and recurrent architectures. Given this, the primary contributions and novelty of the proposed method appear to be the application to predicting metabolic flux and metabolic balance in a data-driven way. In contrast, there is limited advancement in the individual methodological components used in this work.

questions:
- Line 181: "the goal is to learn a model $F: t \rightarrow Y(\theta) ...$". Does the model only take time as input? My understanding is that $F$ is the decoder. Is using just time sufficient?
- What does the $Y$ variables represent? The distributions $\{G, M^f, M^b, \tilde{M}^f, \tilde{M}^b \}$? This is not entirely clear.
- If the objective is to predict entire distributions, why only consider the mean squared error (MSE) metric? Why not consider other metrics which can be specifically used to quantify the distance between distributions? For example, the Wasserstein distance and/or maximum mean discrepancy (MMD) distance. 
- What is the setup for predicting metabolic flux and metabolic balance with gene knockouts, i.e. is this for *test* gene knockouts that are seen during training, or gene-knockouts *unseen* by the model during training? This is not entirely clear.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732611578385

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
GENERAL RESPONSE TO ALL REVIEWERS

comment:
We thank all the reviewers for their insightful comments and detailed feedback. Below, we respond to the reviewers' common concerns about our work.  
The remaining reviewer's questions will be clarified in each of our individual responses.

---

### 1. Why and how is the studied problem important?

Metabolic pathway analysis plays an essential role in identifying potentially novel therapeutic targets in organisms and is thus extremely important from the point of view of developing new drugs and treatments. However, gathering data about the reactions in these pathways is not easy, and as a result, people have developed genome-scale metabolic models that are used via flux balance analysis (FBA) to generate insights. For example, gene knockouts are simulated by modifying the FBA constraints and solving the optimization problem again. However, devising an accurate FBA optimization problem itself takes a lot of domain expertise, limiting its scalability. This is why we approach it from a data-driven perspective, which is a novel way to tackle the problem. 

Moreover, we wanted to introduce the ML community to this very important problem. Despite the problem being motivated from a biomedical perspective, we believe it is also of significant interest from an ML perspective. For reasons (other than the ones already studied in our paper) such as:
- The presence of a graphical structure between gene-enzyme-reactions that's evolving with time.
- The potential to study gene-knockout via graph perturbations.
- Integrating both genomics and metabolomics data, leading to the generation of metabolomic data from genomic data since it's easier to procure.
- The destructive nature of single-cell sequencing reads, leading to its random process formulation with potential for better dynamical neural network models for random processes.

---

### 2. Why choose NP and NODEP as baselines?

The time-varying observations in the form of metabolic flux and balance are directly modeled as a random process, which is why we treat NP and Neural ODE processes as our baselines. 

The reason we model them as a random process is that we estimate the metabolic flux and balance via scRNA-seq data, which is comparatively accessible but suffers from a destructive measurement process. This means that we cannot track the property of any individual cell (e.g., the gene-expression, metabolite concentration, or reaction flux) over time. Instead, we have samples of these quantities coming from different cells across different sampling time steps. Therefore, when we consider a metabolic-flux trajectory, the elements belong to different cells, suggesting the trajectory is a noisy one. This, in turn, implies that future values cannot be precisely predicted, and we thus predict a distribution over them. 

While deep learning models aimed at extrapolating or interpolating trajectories and processing sequential data, like LSTM, GRU, or ODE-RNN, can certainly be used, getting a distribution from the predictions of a trajectory would require us to run the experiment several times, each resulting in a slightly different prediction (on account of randomness from the trajectory and the latent sampling). From these different predictions, we can estimate a mean and variance for our parametric family of choice. Neural Process family models like NP and NODEP, however, do this inherently.

---

### 3. Novelty and methodological contributions.

While we agree that our framework uses several methodological components that could already be found in earlier works on an individual basis in different contexts, we believe that the sum total of those components is still novel with respect to our problem statement and biological motivation of modeling the flux-balance estimates as a random process. 

To the best of our knowledge, we haven't encountered any work that takes the time order/structure into account when modeling random processes via deep learning. While there are works like ODE-RNN or GRU-ODE that do exploit the structured-ness in the encoder, they're done in the context of interpolation or extrapolation of time-series trajectories, not random processes. Thus, we wanted to highlight that structured consideration is lacking in existing neural process methodologies, while also providing a solution for it.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The reviewers and the authors engaged in a good discussion. Although the reviewers appreciate the problem tackled in the paper (neural ODEs for metabolic networks), the reviewers were not convinced that the paper is ready for ICLR.

The authors are encouraged to continue their work and submit the updated version of the paper elsewhere.