<figure><img src="image_1.png" /></figure>

HAL
open science

# Hierarchical MPC applied to bacterial resource allocation and metabolite synthesis

Agustin Gabriel Yabo, Jean-Baptiste Caillau, Jean-Luc Gouzé

► To cite this version:

Agustin Gabriel Yabo, Jean-Baptiste Caillau, Jean-Luc Gouzé. Hierarchical MPC applied to bacterial resource allocation and metabolite synthesis. CDC 2021 - 60th IEEE Conference on Decision and Control, Dec 2021, Austin, United States. pp.667-672, 10.1109/CDC45484.2021.9683025. hal-03189960v2

HAL Id: hal-03189960

https://hal.science/hal-03189960v2

Submitted on 9 Jan 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
# Hierarchical MPC applied to bacterial resource allocation and metabolite synthesis

Agustín Gabriel Yabo, Jean-Baptiste Caillau and Jean-Luc Gouzé

**Abstract**— Microorganisms have evolved submitted to a continuous optimisation process that has improved their capacity to proliferate in nature, developing highly optimized distribution mechanisms of their resources. Considering the microbial self-replication process as a resource allocation problem is a novel approach that has motivated numerous applications to the artificial production of metabolites of interest. Model-based optimal control studies are essential in understanding these naturally-evolved allocation strategies, but they are usually represented by open-loop control laws. In this context, we introduce a hierarchical shrinking-horizon non-linear MPC scheme that aims to maximise the production of a metabolite of interest. The control loop acts on an external signal that is able to disrupt the natural allocation process. The approach uses an optimal control-based input parametrisation that takes into account the structure of the open-loop natural allocation strategy of the cell, to emulate a closed-loop control law. We provide examples of the open-loop control strategies, and simulations of the hierarchical scheme.

## I. INTRODUCTION

Microorganisms have evolved over millions of years under natural selection, submitted to a continuous optimisation process that has improved their capacity to proliferate in nature. Thus, they have developed highly optimized distribution mechanisms of their internal resources to cellular functions enabling them to face changing environments. Unraveling these internal mechanisms has always been of great interest for the scientific community, not only from a pure biological point of view, but also for biotechnological purposes. In this context, being able to understand and control the growth process is key for several industrial applications, such as in combating antibiotics resistance, food preservation, and biofuel production [1].

Considering the microbial self-replication process as a resource allocation problem is a novel approach that has successfully answered some of the underlying question in the field [2]. The latter has also motivated numerous applications to the artificial production of metabolites of interest [3], [4], [5], [6]. These studies aim to find how to divert the cell internal resources into a heterologous pathway in order to efficiently synthesize a specific protein. This is done through an external control that is able to disrupt the cellular

allocation process of a growing culture by reengineering the transcriptional control of the expression of RNA polymerase [7]. In a dynamical systems framework, the problem can be posed as an Optimal Control Problem (OCP), which can be approached through the well-known Pontrjagin's Maximum Principle (PMP).

Model-based optimal control studies are essential in understanding the overall allocation process, as they are able to provide the gold-standard strategies, i.e. the best that can be achieved from a theoretical point of view. However, in most cases, it is impossible to obtain a closed-loop control strategy: the obtained optimal control often depends on the so-called adjoint state, which hinders its implementation (as it is the case in [3]). Additionally, such approaches depend on the accuracy of the model and the precision of its parameters, which often tend to be limited for most biochemical and biological processes. At the same time, the existing industrial applications that allow a closed-loop implementation are mainly based on general schemes such as non-linear Model Predictive Controllers (NMPC) which tend to disregard the structure of each particular problem [8].

Motivated by the lack of synergy between pure theoretical approaches and very general implementations, in this work we revisit the metabolite production problem. We summarize the open-loop optimal allocation strategies found in the literature, which are characterized by sharing the same simple structure and a common parametrisation. Then, we propose a hierarchical NMPC scheme designed on the basis of these open-loop optimal controllers. In particular, we resort to the shrinking horizon NMPC (sh-NMPC) [9], an approach targeted to control processes of known time duration, such as batch processes [10]. In contrast to typical receding horizon approaches, in the sh-NMPC, the final time of the process is fixed, and so the time window considered in the optimisation problem (i.e. the prediction horizon) shrinks at each step.

Our approach uses an optimal control-based input parametrisation that takes into account the structure of the open-loop natural allocation strategy of the cell. Thus, we first implement an MPC loop which creates a closed-loop natural allocation, followed by a second MPC that computes the external control maximising the production of a metabolite of interest. Similar approaches have been proposed to control batch and semi-batch processes through the manipulation of the feedrate [9], [11], which is a standard scheme in the bioreactor framework. The novelty of this work resides in a hierarchical control scheme that aims to affect the internal pathways of the cells in a bacterial growing culture by affecting the expression of RNA polymerase.

This work was partially supported by ANR project Maximic (ANR-17-CE40-0024-01), Inria IPL Cosy and Labex SIGNALIFE (ANR-11-LABX-0028-01). We also acknowledge the support of the FMJH Program PGMO and the support to this program from EDF-THALES-ORANGE.

A. G. Yabo and J. L. Gouzé are with Université Côte d'Azur, Inria, INRAE, CNRS, Sorbonne Université, Biocore Team, Sophia Antipolis, France. agustin.yabo@inria.fr, jean-luc.gouze@inria.fr

J. B. Caillau is with Université Côte d'Azur, CNRS, Inria, LJAD, France. jean-baptiste.caillau@univ-cotedazur.fr
<figure><img src="image_2.png" /><figcaption>Fig. 1: Coarse-grained self-replicator model. The external substrate S is consumed by bacteria and transformed into precursor metabolites P through the action of the metabolic machinery M. These precursors are used to produce macro-molecules of the gene expression machinery R, the metabolic machinery M, the housekeeping machinery Q, and metabolites X. The external control I is able to externally affect the natural allocation parameter α in order to channel resources into the production of metabolites of interest.</figcaption></figure>

We start the paper by defining the model, and the naturally-evolved resource allocation strategy used by the bacteria. Then, we propose a suboptimal parametrisation intended to emulate the open-loop strategy. In Section III, we introduce the open-loop metabolite maximisation problem, and the closed-loop hierarchical scheme. Finally, we provide a numerical simulation of the approach and a comparison with the optimal case, followed by a conclusion.

## II. MODEL DEFINITION

### A. Self-replicator system

Based on [2], we define the self-replicator system composed of the mass (in grams) of: precursor metabolites P, the gene expression machinery R, the metabolic machinery M, the housekeeping machinery Q, and a metabolite of interest X. As illustrated in Figure 1, substrate S is taken from the environment and transformed into P at rate $V_M$ through a reaction catalyzed by M. Then, the precursors P are transformed into M, Q, R and X at rates $r_{max}(1 - \alpha)V_R$, $(1 - r_{max})V_R$, $r_{max}\alpha V_R$, and $V_X$, respectively. While the reactions that produce M, Q and R are catalyzed by R, the reaction synthesizing X is catalyzed by M. In short, the ribosomal proteins R are responsible of producing new proteins, and the metabolic proteins M are responsible for the uptake of nutrients into the cell, and the production of metabolites X. The latter represents a classical trade off in synthetic biology, and is modeled through the parameter α, defined as a time function with bounds $\alpha(t) \in [0, 1]$. The

dynamical system is

$$ egin{cases} 
\dot{S} = V_{in} - V_M \\
\dot{P} = V_M - V_X - V_R, \\
\dot{R} = r_{max}\alpha V_R, \\
\dot{M} = r_{max}(1 - \alpha)V_R, \\
\dot{Q} = (1 - r_{max})V_R, \\
\dot{X} = V_X. 
\end{cases} 
$$

where the time variable t is measured in hours. The bacterial volume is defined as $V \doteq \beta(R+M+Q)$, and the growth rate given by $\mu \doteq \dot{V}/V$. We define the intracellular concentrations

$$ p = \frac{P}{V}, \quad r = \frac{R}{V}, \quad m = \frac{M}{V}, \quad q = \frac{Q}{V} $$

and the extracellular concentration of substrate s. Using the definition of bacterial volume, we obtain the relation $\beta(r + m + q) = 1$. Then, following [12], we assume the transcription of proteins Q to be internally autoregulated to a constant value, such that

$$ \beta(r + m) = r_{max}, \quad \beta q = q_{max} \doteq 1 - r_{max}. \quad (1) $$

We define the rates of mass flow per unit volume, which we assume to be functions of the concentrations s, m and r, as $v_M(s, m) \doteq V_M/V$ and $v_R(p, r) \doteq V_R/V$. In this new system, the growth rate becomes $\mu = \beta V_R/V = \beta v_R(p, r)$. Taking into account that a minimal concentration of ribosomes $r_{min}$ is required in order for bacteria to self replicate, we define the kinetics of the problem as $v_M(s, m) \doteq w_M(s)m$, $v_R(p, r) \doteq w_R(p)(r - r_{min})$ and $v_X(p, m) \doteq w_X(p)m$, with $w_R(p) \doteq k_R p/(K_R + p)$, $w_M(s) \doteq k_M s/(K_S + s)$ and $w_X(p) \doteq k_X p/(K_X + p)$. We will model a production process in which the substrate remains constant. This could be the result of an external control regulating through an inflow of fresh medium to the bioreactor, or due to high availability in the medium. Thus, we replace $w_M(s) = e_M$, with $e_M > 0$ constant. We define the non-dimensional timescale $\hat{t} = k_R t$, as well as the mass fractions of the total volume $V$: $\hat{p} \doteq \beta p$, $\hat{r} \doteq \beta r$, $\hat{r}_{min} \doteq \beta r_{min}$, $\hat{m} \doteq \beta m = r_{max} - r$, $\hat{q} \doteq \beta q = 1 - r_{max}$. Additionally, we define non-dimensional synthesis rates $\hat{w}_R(p) = w_R(p)/k_R$, $\hat{w}_X(p) = w_X(p)/k_R$, and parameter $E_M \doteq e_M/k_R$. Then, dropping all hats, the model becomes

$$ egin{cases} 
\dot{p} = E_M (r_{max} - r) - w_X(p)(r_{max} - r) \\ 
-(p + 1)w_R(p)(r - r_{min}), \\ 
\dot{r} = (r_{max}\alpha - r)w_R(p)(r - r_{min}), \\ 
\dot{X} = w_X(p)(r_{max} - r)V, \\ 
\dot{V} = w_R(p)(r - r_{min})V, 
\end{cases} 
\quad (2) $$

where q and m have been removed using equations (1). The parameter values of the kinetics and of bounds $r_{min}$ and $r_{max}$ are fixed based on previous studies [2], [3], [13].
## B. Naturally-evolved resource allocation strategy

A common assumption in biology is that microorganisms have evolved resource allocation strategies that maximise their growth rate, which allow them to outgrow competing organisms. Such assumption can be represented by an OCP, in which the objective is to maximise the synthesis of biomass in an interval of time T given by $\Delta V(T) = V(T) - V(0)$. This defines the cost function

$$J_N(\alpha) = \int_{0}^{T} w_R(p)(r - r_{\min})V dt.$$

Thus, as neither the states nor the cost function depend on variable X, we will define the OCP for the reduced state $(p, r, V)$ with dynamics

$$\left\{
\begin{array}{l}
\dot{p} = E_M (r_{\max} - r) - w_X(p)(r_{\max} - r) \\
\qquad - (p + 1)w_R(p)(r - r_{\min}), \\
\\
\dot{r} = (r_{\max}\alpha - r)w_R(p)(r - r_{\min}), \\
\\
\dot{V} = w_R(p)(r - r_{\min})V,
\end{array}
\right.
\qquad (S_N)$$

and initial conditions

$$p(0) = p_0, \quad r(0) = r_0 \quad V(0) = V_0. \qquad (\text{IC})$$

with $p_0 > 0$, $r_0 \in (r_{\min}, r_{\max})$ and $V_0 > 0$. The OCP is then defined as

$$\left\{
\begin{array}{ll}
\underset{\alpha}{\text{maximise}} & \text{biomass production } J_N(\alpha), \\
\text{subject to} & \text{dynamics } (S_N), \\
& \text{initial conditions } (\text{IC}), \\
& \alpha(\cdot) \in \mathcal{U}, \\
& t \in [0, T].
\end{array}
\right.
\qquad (\text{OCP}_N)$$

where $\mathcal{U}$ is the set of admissible controllers, which are Lebesgue measurable real-valued functions defined on the time interval $[0, T]$ and satisfying $\alpha(t) \in [0, 1]$. In [2], the particular case where $w_X(p) = 0$, $r_{\max} = 1$ and $r_{\min} = 0$ has been studied, and similar analyses have been carried out in [3], [4]. By application of the PMP, it is possible to show that the optimal control that solves OCP$_N$ has bang and singular arcs, where the singular arc corresponds to the solution of the static optimal control problem obtained through the addition of the constraint $(\dot{p}, \dot{r}, \dot{V}) = 0$. The solution is characterized by the presence of the Fuller phenomenon before and after the constant singular arc, which produces an infinite number of bangs (also known as *chattering*), a feature that, due to obvious physical limitations, is not possible to implement (nor expected to be found in nature). An example of this kind of structure is shown in Figure 2.

**Remark 1:** Problem OCP$_N$ can be further simplified by considering the cost function $\ln V(T)$ instead of $V(T)$. Then, neither the dynamics nor the cost function depend on $V$, so the problem can be rewritten in terms of the state $(p, r)$ [2].

## C. MPC parametrisation of the natural allocation

In order to incorporate the natural allocation strategy of the cell into the MPC loop, we propose a sub-optimal parametric

<figure><img src="image_3.png" /><figcaption>Fig. 2: Optimal control $\alpha$ obtained with Bocop [14]. Simulation in a rich medium with $e_M = k_R$, meaning the medium enables the maximum growth rate. Initial conditions are $p_0 = 0.024$, $r_0 = 0.2$, and $V_0 = 0.003$, and the simulation time is set to $T = 15$.</figcaption></figure>

form of $\alpha$ given by

$$\alpha_{so}(\theta, t) = \begin{cases} b_1 & \text{if } t < t_1, \\ \alpha^* & \text{if } t_1 \le t \le t_2, \\ b_2 & \text{if } t > t_2, \end{cases}$$

with the set of parameters $\theta = (b_1, b_2, t_1, t_2, \alpha^*)$ subject to

$$\begin{cases} b_1 \in \{0, 1\}, & b_2 \in \{0, 1\}, \\ t_f \ge t_2 \ge t_1 \ge 0, & 1 \ge \alpha^* \ge 0, \end{cases} \qquad (3)$$

where $b_1$ and $b_2$ are Boolean parameters. The suboptimal parametric allocation $\alpha_{so}$ deliberately neglects the chattering artifact from the optimal control $\alpha$, replacing it by pure bang controls during the intervals $[0, t_1)$ and $(t_2, T]$. In order to compare the performance of the proposed controller, we write an optimisation problem with the same biomass production objective $J_N$. At each time instant of the control loop, the algorithm finds the vector of parameters $\theta$ that maximises the final volume of biomass $V(T)$. The latter amount to solving four optimisation problems in terms of $(t_1, t_2, \alpha^*)$ given by all possible combinations of Boolean parameters $(b_1, b_2)$. At each iteration $k$, the control loop starts by measuring the system and getting an estimation $(\tilde{p}_k, \tilde{r}_k, \tilde{V}_k)$ of the system state. Thus, the optimisation problem at iteration $k$ is formulated with initial conditions

$$(p(k\tau), r(k\tau), V(k\tau)) = (\tilde{p}_k, \tilde{r}_k, \tilde{V}_k). \qquad (4)$$

This defines the optimisation problem

$$\left\{
\begin{array}{ll}
\underset{\theta}{\text{maximise}} & \text{biomass production } J_N(\theta) \\
\text{subject to} & \text{dynamics } (S_N), \\
& \text{initial conditions } (4) \\
& \alpha(\cdot) = \alpha_{so}(\theta, t), \\
& \text{input constraints } (3) \\
& t \in [k\tau, T]
\end{array}
\right.
\qquad (\text{OP}^k)$$

which is solved at each instant $k\tau$. The scheme proposes a closed-loop form of the open-loop optimal control found in Subsection II-B, with the purpose of implementing it in the
hierarchical MPC loop.

### D. Numerical example

Figure 3 shows a comparison of the optimal control $\alpha$ and the proposed suboptimal control $\alpha_{so}$. The initial and final Fuller arcs are approximated by pure bang arcs (which are $\alpha = 1$ for this particular case), and the parameter $\alpha^*$ of the suboptimal control takes exactly the same value of the static optimal control $\alpha_{opt}^*$. The difference between both control functions is minor, which translates into an imperceptible difference in the trajectories.

<figure><img src="image_4.png" /><figcaption>Fig. 3: Comparison of the optimal control $\alpha(t)$ solution of OCP_N and the MPC scheme parametrized with the suboptimal control $\alpha_{so}(\theta, t)$. Initial conditions are set to $p_0 = 0.024$, $r_0 = 0.2$, and $V_0 = 0.003$. The scheme is executed with time step $\tau = 0.3$. The quantity $\Delta X$ amounts to $X(T) - X(0)$.</figcaption></figure>

## III. ARTIFICIAL METABOLITE PRODUCTION

The artificial metabolite production problem is to maximise the synthesis of X over a fixed interval of time $[0, T]$, which is equal to $\Delta X(T) = X(T) - X(0)$, and can be expressed as

$$J_X(u) = \int_0^T w_X(p)(r_{\max} - r) \mathcal{V} dt.$$

As neither the states nor the cost function depend on variable X, the reduced system ($S_N$) can be used.

### A. Optimal Control Problem

In the original approach [3], the naturally-evolved resource allocation parameter $\alpha$ is overridden by the external control

$u$, and so the dynamical equation of $r$ becomes

$$\dot{r} = (r_{\max}u - r)w_R(p)(r - r_{\min}).$$

Then, the optimal control problem is defined as

$$\left\{
\begin{array}{ll}
\underset{u}{\text{maximise}} & \text{metabolite production } J_X(u) \\
\text{subject to} & \text{dynamics } (S_N) \\
& \text{initial conditions (IC)} \\
& \qquad u(\cdot) \in \mathcal{U}, \\
& \qquad t \in [0, T].
\end{array}
\right.
\qquad (\text{OCP}_X)$$

### B. On the solution of the OCP

Applying PMP, we see that the Hamiltonian is affine in the control, so it has the form $H = H_0 + uH_1$, meaning that the solution is bang-singular-bang, given by

$$u(t) = \begin{cases} 0 & \text{if } H_1 < 0, \\ 1 & \text{if } H_1 > 0, \\ u_{\text{sing}}(t) & \text{if } H_1 = 0. \end{cases}$$

Examples of optimal trajectories are shown in Figure 4 and Figure 5, where both problems OCP_N and OCP_X are compared for different environmental conditions representing rich and poor qualities of the nutrient in the medium. While the structures of the optimal control for both problems are similar, the optimal strategy maximizing the production of X is characterized by a non-constant singular arc, which is close to the solution $u_{opt}^*$ of the static OCP, but deviates from it towards the end. Additionally, the times at which the junctions between bang and singular arc are produced differ, as well as the values of the bangs. In particular, in both Figures, the final bang of the natural control is $\alpha = 1$, while that of the artificial control is $u = 0$. More detailed calculations of the PMP approach can be found in [3].

<figure><img src="image_5.png" /><figcaption>Fig. 4: Optimal control obtained with Bocop. Simulation in a rich medium with $e_M = k_R$, meaning that the substrate enables the maximum growth rate. Initial conditions are $p_0 = 0.024$, $r_0 = 0.1$ and the simulation time is set to $T = 30$.</figcaption></figure>

### C. Product maximisation including naturally-evolved allocation

The comparison between OCP_N and OCP_X proves useful to observe that the natural behavior of microbes does not necessarily match the artificial objective of producing a certain
<figure><img src="image_6.png" /><figcaption>Fig. 5: Optimal control obtained with Bocop. Simulation in a poor medium with $e_M = 0.5k_R$. Initial conditions are $p_0 = 0.024$, $r_0 = 0.3$ and the simulation time is set to $T = 30$.</figcaption></figure>

metabolite. However, the assumption made in $OCP_X$ is a pure theoretical one, as $\alpha$ cannot be completely substituted by the external control. We then propose an approach that takes into account two distinct processes: (i) the ability of bacteria to maximise their biomass through the optimal allocation described by the internal control problem $OCP_N$, (ii) the external action of an operator intending to maximise the production of the metabolite of interest X. In practice, the new pathway associated to the dynamics of X in (2) is obtained by optogenetic engineering of a strain of bacteria: a light-induced control I is able to externally modulate the natural allocation process. This is modeled by replacing the control $u$ of $OCP_X$ by $u = \alpha(p, r)I$, so that the external control affects in a multiplicative way the internal allocation strategy. The crucial difference with the previous formulation [3] is that the internal control $\alpha$ of the bacteria now appears in feedback form (thus depending on the two states of $OCP_N$; see Remark 1), and is modulated by the external light-induced control I. It is noteworthy that no competition occur between the two objectives (biomass vs. metabolite production maximisation). The approach considers the internal control in feedback form, which is mitigated by an external control in relation with a global process that includes the new pathway to produce the metabolite. In this context, the proposed hierarchical approach proves to be more relevant from a biological point of view than a multicriterion one. Additionally, we assume that the feedback $\alpha(p, r)$ is known and smooth. While the latter seems to be a strong assumption from the control point of view (as the solution of $OCP_N$ comprises bang and singular arcs), it is a reasonable assumption in our biological setting, where the kinetics of the involved biochemical reactions prescribe continuous behaviours (see [2] for biologically relevant approximations of the feedback). Thus, the dynamical equation of $r$ becomes

$$ \dot{r} = (r_{\max}\alpha(p, r)I - r)w_R(p)(r - r_{\min}), \quad (5) $$

where the new control $I(t)$ is subject to bound constraints $0 \le I(t) \le I_{\max}$, and the cost $J_X$ remains unchanged. This defines problem $\overline{OCP}_X$.

### D. Hierarchical MPC for metabolite production

In this approach, we approximate the allocation feedback $\alpha(p, r)$ through the sh-MPC loop described in II-C. At each iteration $k$, solving on $[k\tau, T]$ (where $T$ is, as before, the fixed horizon) yields an approximation of $\alpha(p_k, r_k)$ (based on the suboptimal parametric form $\alpha_{so}$), and of $\alpha$ evaluated at further steps. Then, this suboptimal feedback is injected in the dynamics (5) of $\overline{OCP}_X$ so as to find the optimal external control $I$ maximizing $X(T)$. Thus, a second MPC is used "above" the first one. There is a quite large literature on such approaches combining several MPC loops (see, e.g., [15], [16] and references therein). Other relevant matters such as using different time grids for each MPC loop or, more generally, synchronisation issues, are not discussed here (see also the recent paper [17] on convergence of MPC methods in finite horizon). Instead, we focus on the biological application. We note, in particular, that when the feedback $\alpha(p, r)$ is zero (which would be expected for the genuine—though biologically unrealistic—feedback of $OCP_N$ as zero bang arcs can occur), the external control $I$ is not active. In practice, when the allocation is close to zero, the MPC loop would compensate through $I$ for this discrepancy. As the external control $I$ is bounded, the latter can induce certain performance loss between the results of the ideal model $OCP_X$ and the more realistic problem $\overline{OCP}_X$. Such comparisons are provided in the next paragraph.

## IV. NUMERICAL RESULTS

Figure 6 shows a comparison between the optimal trajectory solution of $OCP_X$ and the hierarchical MPC proposed in this paper. Differences between both trajectories are marginal, mainly given by the approximation of the singular arc by a constant control $u$. In Figure 7 we see how, in order to match the optimal control $u$ solution of $OCP_X$, the external signal $I$ completely arrests the natural allocation $\alpha$ (around $t = 12$), which implies allocating all the cellular resources to the metabolic machinery M, thus catalyzing the synthesis of X. The latter produces the suboptimal control $\alpha_{so}$ to increasingly compensate until it reaches the value 1. This result shows the difference between the simulated closed-loop behavior of the natural allocation strategy $\alpha_{so}(p, r)$, and the open-loop one (which remains constant almost over the whole interval $[0, T]$).

## V. CONCLUSION

In this paper, we presented a hierarchical sh-NMPC approach to the problem of optimally producing a metabolite of interest in bacteria. The scheme is based on a parametric version of the naturally-evolved research allocation strategy proposed in [2], which represents a closed-loop alternative to these existing open-loop studies. A second sh-NMPC loop is applied in a hierarchical manner, in order to achieve the metabolite maximisation objective while taking into account the closed-loop natural control. Despite the approach being at an early stage, with no experimental results, it represents a step towards plausible biosynthetic real-time implementations. In future works, we are interested in comparing the
<figure><img src="image_7.png" /><figcaption>Fig. 6: Comparison of the optimal control $u(t)$ solution of OCP$_X$ and the hierarchical MPC scheme parametrized that considers the natural allocation as an inner MPC loop. Initial conditions are set to $p_0 = 0.024$, $r_0 = 0.3$, and $V_0 = 0.003$. Final time is set to $T = 30$, the scheme is executed with time step $\tau = 1$ and the environmental constant $e_M = 0.5k_R$.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 7: Final control $u$ and external signal $I$ obtained from the MPC loop simulated in Figure 6.</figcaption></figure>

natural MPC approximation with alternatives proposed in the literature, such as ppGpp regulation [2]. Other extensions include the possibility of estimating the real value of the

natural allocation through online identification techniques.

ACKNOWLEDGMENTS

The authors thank Hidde de Jong (Inria) as well as colleagues from the ANR Maximic project for many inputs on the biological problem.

REFERENCES

[1] L. Huo, J. J. Hug, C. Fu, X. Bian, Y. Zhang, and R. Müller, "Heterologous expression of bacterial natural product biosynthetic pathways," *Natural Product Reports*, vol. 36, no. 10, pp. 1412–1436, 2019.

[2] N. Giordano, F. Mairet, J.-L. Gouzé, J. Geiselmann, and H. De Jong, "Dynamical allocation of cellular resources as an optimal control problem: novel insights into microbial growth strategies," *PLoS computational biology*, vol. 12, no. 3, p. e1004802, 2016.

[3] I. Yegorov, F. Mairet, H. De Jong, and J.-L. Gouzé, "Optimal control of bacterial growth for the maximization of metabolite production," *Journal of mathematical biology*, vol. 78, no. 4, pp. 985–1032, 2019.

[4] A. G. Yabo, J.-B. Caillau, and J.-L. Gouzé, "Singular regimes for the maximization of metabolite production," in *2019 IEEE 58th Conference on Decision and Control (CDC)*, pp. 31–36, IEEE, 2019.

[5] A. G. Yabo and J.-L. Gouzé, "Optimizing bacterial resource allocation: metabolite production in continuous bioreactors," in *IFAC World Congress 2020*, 2020.

[6] A. G. Yabo, J.-B. Caillau, and J.-L. Gouzé, "Optimal bacterial resource allocation: metabolite production in continuous bioreactors," *Mathematical Biosciences and Engineering*, vol. 17, no. 6, pp. 7074–7100, 2020.

[7] J. Izard, C. D. Gomez Balderas, D. Ropers, S. Lacour, X. Song, Y. Yang, A. B. Lindner, J. Geiselmann, and H. de Jong, "A synthetic growth switch based on controlled expression of rna polymerase," *Molecular systems biology*, vol. 11, no. 11, p. 840, 2015.

[8] S. J. Qin and T. A. Badgwell, "A survey of industrial model predictive control technology," *Control engineering practice*, vol. 11, no. 7, pp. 733–764, 2003.

[9] E. Aydin, D. Bonvin, and K. Sundmacher, "Nmpc using pontryagin's minimum principle-application to a two-phase semi-batch hydroformylation reactor under uncertainty," *Computers & Chemical Engineering*, vol. 108, pp. 47–56, 2018.

[10] A. S. Soni and R. S. Parker, "Closed-loop control of fed-batch bioreactors: A shrinking-horizon approach," *Industrial & engineering chemistry research*, vol. 43, no. 13, pp. 3381–3393, 2004.

[11] E. Aydin, D. Bonvin, and K. Sundmacher, "Computationally efficient nmpc for batch and semi-batch processes using parsimonious input parameterization," *Journal of Process Control*, vol. 66, pp. 12–22, 2018.

[12] A. Y. Weiße, D. A. Oyarzún, V. Danos, and P. S. Swain, "Mechanistic links between cellular trade-offs, gene expression, and growth," *Proceedings of the National Academy of Sciences*, vol. 112, no. 9, pp. E1038–E1047, 2015.

[13] M. Scott, S. Klumpp, E. Mateescu, and T. Hwa, "Emergence of robust growth laws from optimal regulation of ribosome synthesis," *Molecular Systems Biology*, vol. 10, p. 747, 2014.

[14] I. S. Team Commands, "Bocop: an open source toolbox for optimal control." http://bocop.org, 2017.

[15] V. Raghuraman, V. Renganathan, T. H. Summers, and J. P. Koeln, "Hierarchical mpc with coordinating terminal costs*," in *2020 American Control Conference (ACC)*, pp. 4126–4133, 2020.

[16] R. Scattolini and P. Colaneri, "Hierarchical model predictive control," in *2007 46th IEEE Conference on Decision and Control*, pp. 4803–4808, 2007.

[17] A. L. Dontchev, I. V. Kolmanovsky, M. I. Krastanov, V. M. Veliov, and P. T. Vuong, "Approximating optimal finite horizon feedback by model predictive control," *Systems Control Lett.*, vol. 139, p. 104666, 2020.