# Optimal resource allocation in micro-organisms under periodic nutrient fluctuations

Javier Innerarity Imizcoz, Walid Djema, Francis Mairet, Jean-Luc Gouzé

► To cite this version:

Javier Innerarity Imizcoz, Walid Djema, Francis Mairet, Jean-Luc Gouzé. Optimal resource allocation in micro-organisms under periodic nutrient fluctuations. Journal of Theoretical Biology, In press, 10.1016/j.jtbi.2024.111953. hal-04726319

HAL Id: hal-04726319

https://inria.hal.science/hal-04726319v1

Submitted on 8 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Optimal resource allocation in micro-organisms under periodic nutrient fluctuations

J. Innerarity<sup>a</sup>, W. Djema<sup>c</sup>, F. Mairet<sup>b</sup>, J.-L. Gouzé<sup>a</sup>

## Abstract

Although microorganisms often live in dynamic environments, most studies, both experimental and theoretical, are carried out under static conditions. In this work, we investigate the issue of optimal resource allocation in bacteria growing in periodic environments. We consider a dynamic model describing the microbial metabolism under varying conditions, involving a control variable quantifying the protein precursors allocation. Our objective is to determine the optimal strategies maximizing the long-term growth of cells under a piecewise-constant periodic environment. Firstly, we perform a theoretical analysis of the resulting optimal control problem (OCP), based on the application the Pontryagin's Maximum Principle (PMP). We determine that the structure of the optimal control must be bang-bang, with possibly some singular arcs corresponding to optimal equilibria of the system. If the control presents singular arcs, then these can only be reached and left through chattering arcs. We also use a direct optimization method, implemented in the BOCOP software, to solve the studied OCP. Our study reveals that the optimal solution over a large time horizon is related to the one over a single period of the varying environment with periodic constraints. Moreover, we observe that the maximal average growth rate attainable under periodic conditions can be higher than the one under a constant environment. We further extend our analysis to conduct a qualitative comparison between the predictions from our model and some recent biological experiments on *E. coli*. This analysis particularly highlights the mechanisms of action of the ppGpp signaling molecule, thus providing relevant explanations of the experimental observations. In conclusion, our study corroborates previous research indicating that this molecule plays a crucial role in the regulation of resource allocation of protein precursors in *E. coli*.

## Keywords:

Microbial Growth, Optimal Control, Periodic Environment, Pontryagin's Maximum Principle.

## 1. Introduction

Bacteria and other microorganisms form the basis of every ecosystem. Also, they play a key role in many bio-industrial processes, such as food and drug production or waste-water treatment [1-3]. However, important gaps remain in our knowledge of their physiology due to the complexity of studying their internal mechanisms.

Mathematical models have proved to be greatly effective in simplifying the study of bacteria, while being able to reproduce and explain many experimental observations on their growth and metabolism.

Control theory provides a natural tool to approach these problems, which has been applied in a wide variety of biological contexts [4]. Using the appropriate objective function, it provides an ideal behavior, and a benchmark for the comparison of other metabolic regulation mechanisms. It is usual to suppose that bacteria are naturally selected to maximize their growth rate, an assumption in accordance with the behavior observed in [5, 6].

Earlier articles investigating the mathematical modelization of the metabolism of microorganisms studied their growth in steady-state conditions [5-9]. Later studies introduced dynamical modeling of the response of bacteria after a single shift in their environment [2, 10-15]. These have been significant steps in the study of the mechanism of the optimization of resource allocation in micro-organisms. However, the type of conditions that these models represents is only in accordance with what microorganisms can face in a laboratory, where it is feasible to maintain a constant environment [16].

In a natural setting, it is not reasonable to assume that the environment of a given individual remains constant. We must therefore consider time-dependent environments if we want to study the mechanisms regulating the allocation of its resources in a more realistic context [17].

For this reason, we modify the resource allocation model proposed in [10] by setting the substrate availability in the environment of a given cell to a periodic piecewise-constant function. This results in a more realistic environmental input, representing the cyclic changes in substrate concentration that are caused for example by day-night dynamics

<footnote><sup>a</sup>Université Côte d'Azur, Inria, Macbes Project-Team, France</footnote>

<footnote><sup>b</sup>Ifremer, PHYTOX, Laboratoire Physalg, France</footnote>

<footnote><sup>c</sup>Université Côte d'Azur, Inria, BIOCORE Project-Team, France</footnote>
or for host-associated microbes by periodic feeding of the host.

One of the main results of this work is showing that there exists a non-constant periodic environment which improves on the maximal average growth rate attainable in a constant medium. We also find that the mechanism of regulation induced by the ppGpp molecule explains well the behavior of bacteria in dynamic environments. This nucleotide has been found to play a crucial role in bacterial response to changes in amino acid concentration by downregulating rRNA synthesis [18, 19].

The paper is organized as follows. We introduce the model of interest in Section 2, where the environment is non-constant. Then, we state the optimal control problem (OCP) in Section 3. We address this OCP through the application of a variant of Pontryagin's Maximum Principle (PMP) in Section 4. Next, we numerically obtain the solution of the problem in Section 5. Section 6 is devoted to studying the relationship between the infinite-time OCP and the problem with periodic boundary conditions of the same period as the environment. In Section 7, we study how two key parameters of the environment, its period and amplitude, affect the solution of the OCP. Finally, we study the ppGpp-regulated suboptimal strategy, which we compare with the optimal control and with experimental observations on E. coli [20] in Section 8.

## 2. The Mathematical Model

We modify the self-replicator model proposed in [10], which represents the metabolism of a given micro-organism. The external medium ($E_M$) of this cell has some substrate that the cell can absorb and convert into precursor metabolites (P) through some reactions catalyzed by the metabolic machinery (M). These precursors are then converted by the gene expression machinery (R) into the macromolecules that form R and M. The resource allocation parameter $\alpha(t)$ is the proportion of the precursor's mass used to build R at a given time t. This process is schematized in Figure 1.

The model can be simplified by dividing all the variables by the total volume of the cell, which is assumed to be proportional to the total macromolecular mass, $V = \beta(M + R)$, with $\beta > 0$. We also suppose that reactions follow Michaelis-Menten dynamics. After normalization, the system reads as follows.

$$ egin{cases} \dot{p} = (1-r)E_M(t) - (1+p)\frac{p}{K+p}r \\ \dot{r} = (\alpha-r)\frac{p}{K+p}r \\ \dot{V} = \frac{pr}{K+p}V, \end{cases} \quad (1) $$

where $p$ and $r$ are time-dependent variables. The first one represents the cytoplasmic concentration of P, whereas $r$ is the concentration of R. The concentration of M does not need to appear in these equations as it can be defined by $m = 1 - r$. The resource allocation parameter $\alpha(t)$ will be the

<figure><img src="image_2.png" /><figcaption>Figure 1: Diagram of our model of bacterial metabolism. The control variable $\alpha$ represents the proportion of protein precursors allocated to producing R. The environment $E_M$ is a piecewise-constant period input.</figcaption></figure>

control exerted on the system. The parameter $K$ is intrinsic to the metabolism of the cell, and will therefore be treated as a constant. In [10] the parameter $E_M$, representing the richness of the external medium, is considered to be constant.

We propose to modify this system by setting $E_M$ to be a piecewise-constant periodic input of period $T > 0$, described by

$$ E_M(t) = \begin{cases} E_M^{\max}, & \text{if } t \in [0, T/2), \\ E_M^{\min}, & \text{if } t \in [T/2, T), \end{cases} \quad (2) $$

with $0 < E_M^{\min} < E_M^{\max}$. This definition of $E_M$ yields a more realistic model, which takes into account the always-changing conditions in which microorganisms usually grow. These conditions nevertheless tend to exhibit some kind of periodic behavior. For example, one can think of ecosystems in which the substrate availability is dependent on the amount of sunlight it receives [21].

## 3. Statement of the Dynamic Optimal Control Problem

We seek to determine the resource allocation strategy that will maximize the volume increase of the cells. We define the quantity

$$ \mu(t) = \frac{pr}{K+p}(t), $$

which is the growth rate of the cell. That is,

$$ V(t) = V(0) \exp \int_{0}^{t} \mu(\tau) \, d\tau \quad \forall t \ge 0. \quad (3) $$

Therefore, maximizing (3) is equivalent to maximizing

$$ \int_{0}^{t} \mu(\tau) \, d\tau. $$

Here, we seek the strategy that is able to optimize growth on an infinite time-interval, i.e. we are interested in the optimal long-term strategy. The objective of our optimal control problem (OCP) is thus to find

$$ \alpha_{opt} \text{ such that } \liminf_{t_f \to \infty} \left( J_{t_f}(\alpha_{opt}) - J_{t_f}(\alpha) \right) \ge 0 \quad \forall \alpha \in U_{\infty}, $$
where

$$
\begin{align*}
J_{tf}(\alpha) &= \int_{0}^{tf} \frac{pr}{K+p} dt, \\
U_{\infty} &= \{\alpha : [0, +\infty) \to [0, 1] \mid \alpha \text{ is measurable}\}
\end{align*}
$$

and the system $(p, r, V)(t)$ follows System (1). Note that, since $p$ and $r$ are independent from the cellular volume $V$, we can omit this last variable from System (1).

We will start our study of the OCP by carrying out its theo-
retical analysis in the following section.

4. Application of Pontryagin’s Maximum Principle

4.1. Optimality Conditions

For the theoretical analysis of the optimization problem, we
will use the Infinite Horizon Maximum Principle [22], an ex-
tension of Pontryagin’s Maximum Principle (PMP), which in-
cludes most of its results. We set the Hamiltonian

$$
\begin{align*}
H &= H(p, r, \lambda_p, \lambda_r, \lambda_0, \alpha, t) = \lambda_p \dot{p} + \lambda_r \dot{r} + \lambda_0 \left(\frac{pr}{K+p}\right) \\
&= \lambda_p \left[ (1-r)E_M(t) - \frac{(1+p)pr}{K+p} \right] + \lambda_r \frac{(\alpha-r)pr}{K+p} + \lambda_0 \frac{pr}{K+p} \\
&= H_0 + H_1 \alpha,
\end{align*}
$$

where

$$
H_0 = \lambda_p \left[ (1-r)E_M(t) - (1+p) \frac{pr}{K+p} \right] - \lambda_r \frac{pr^2}{K+p} + \lambda_0 \frac{pr}{K+p}
$$

and

$$
H_1 = \lambda_r \frac{pr}{K+p}.
$$

The adjoint variables fulfill $(\lambda, \lambda_0) = (\lambda_p, \lambda_r, \lambda_0) \neq 0$, $\lambda_0 \ge 0$ and the differential equations

$$
\left\{
\begin{aligned}
\dot{\lambda}_p &= \lambda_p \frac{r(2Kp+K+p^2)}{(K+p)^2} + \lambda_r (r-\alpha) \frac{Kr}{(K+p)^2} - \lambda_0 \frac{Kr}{(K+p)^2} \\
\dot{\lambda}_r &= \lambda_p \left[E_M(t) + \frac{(1+p)p}{K+p}\right] + \lambda_r \frac{p(2r-\alpha)}{K+p} - \lambda_0 \frac{p}{K+p}.
\end{aligned}
\right.
\qquad (4)
$$

We also have the maximization condition

$$
H(t, p, r, \lambda, \lambda_0, \alpha(t)) = \max_{v \in [0,1]} H(t, p, r, \lambda, \lambda_0, v) \text{ a.e. on } [0, +\infty).
$$

Finally, the Hamiltonian obeys the differential equation

$$
\frac{dH}{dt} = \frac{\partial H}{\partial t} = \lambda_p (1 - r) \frac{dE_M}{dt}(t), \quad (5)
$$

which implies that *H* remains constant when *E*<sub>*M*</sub> is also con-
stant.

4.2. Analysis of Optimal Controls

The above conditions allow to find the possible values of α<sub>opt</sub>,
which are given in Theorem 4.1.

**Theorem 4.1.** The optimal control α<sub>opt</sub> is given by

$$
\alpha_{opt}(t) = \begin{cases} 1 & \text{if } \lambda_r > 0 \\ 0 & \text{if } \lambda_r < 0 \\ \alpha_{opt}^*(E_M(t)) & \text{if } \lambda_r = 0, \end{cases}
$$

where

$$
\alpha_{opt}^{*}(E_{M}) = \frac{E_{M} + \sqrt{KE_{M}}}{E_{M} + 2\sqrt{KE_{M}} + 1}.
$$

If $\lambda_r = 0$ on a time interval $t \in [t_1, t_2]$, with $t_1 < t_2$, the system must lie at the equilibrium point $(p_{opt}^*, r_{opt}^*)(E_M(t))$, where

$$
p_{opt}^{*}(E_{M}) = \sqrt{KE_{M}} \quad \text{and} \quad r_{opt}^{*}(E_{M}) = \alpha_{opt}^{*}(E_{M}). \quad (6)
$$

Proof. Since the control is affine in α, and by the maximization
condition on *H*, we have that *α*<sub>opt</sub> takes its maximal (resp.
minimal) value when *H*<sub>1</sub> > 0 (resp. *H*<sub>1</sub> < 0). By the positivity
of *p* and *r*, the sign of *H*<sub>1</sub> is simply the same as *λ*<sub>*r*</sub>.

If the optimal control has a singular arc (a time interval $t \in [t_1, t_2]$, with $t_1 < t_2$, over which $\lambda_r = 0$), then $\lambda_0$ must be non-zero and we can normalize the adjoint variables by setting $\lambda_0 = 1$. Indeed, if it were $\lambda_0 = 0$, then on the singular arc

$$
0 = \dot{\lambda}_r = \lambda_p \left[ E_M(t) + \frac{(1+p)p}{K+p} \right] \implies \lambda_p = 0 \implies (\lambda, \lambda_0) = 0,
$$

which contradicts the optimality conditions.

Therefore, over the singular arc,

$$
\begin{cases}
H = \lambda_p \left[ (1-r)E_M(t) - \frac{(1+p)pr}{K+p} \right] + \lambda_0 \frac{pr}{K+p} \\
\dot{\lambda}_r = \lambda_p \left[ E_M + \frac{(1+p)p}{K+p} \right] - \lambda_0 \frac{p}{K+p} = 0,
\end{cases}
$$

which implies that, in a singular trajectory,

$H = \lambda_p E_M(t).$

By (5), as long as E_M is constant, so is H. By the above
equation, the adjoint variable λ_p must also be constant. By
combining the equations λ_r = 0 and λ_p = 0, we obtain that
p = popt(E_M), given by Equation (6).

In particular, *p* is constant and the equation *ṗ* = 0 yields
*r* = *r*<sub>opt</sub>(*E*<sub>*M*</sub>). Since *r*(*t*) is constant we must have that *α* = *r* =
*α*<sub>opt</sub>(*E*<sub>*M*</sub>). □

We have just proved that singular arcs in a piece-wise con-
stant environment correspond to intervals on which the sys-
tem is constant. In the following subsection, we will see that
this constant value is precisely the equilibrium point that opti-
mizes steady-state growth. Since these points are different be-
tween the two sub-periods, a singular arc cannot contain the
discontinuities of E<sub>M</sub>.
### 4.3. Relation to the Static OCP

We seek to understand why the singular arcs of the optimal control correspond to such concrete static values of the system. In order to do so, we focus on the static OCP of finding which of these equilibrium points yields the highest growth rate. That is, for a given constant value of $E_M$, we seek to determine the tuple

$$ (\alpha^*, p^*, r^*) \in [0, 1] \times \mathbb{R}^+ \times [0, 1] $$

such that

$$ \begin{cases} (\dot{p}, \dot{r})(\alpha^*, p^*, r^*) = 0 \\ \mu^*(\alpha^*, p^*, r^*) = \frac{p^*r^*}{K+p^*} \text{ is maximal.} \end{cases} $$

For each possible value of $E_M > 0$, this problem admits only one solution (see [10]), which is precisely the tuple $(\alpha_{opt}^*, p_{opt}^*, r_{opt}^*) (E_M)$ given by Theorem 4.1. Therefore, we interpret the theorem as follows. The optimal control must take its maximal or minimal value, except when the system lies at its optimal equilibrium point. In the following, we shall call the optimal steady-state growth rate $\mu_{opt}^* = \frac{p_{opt}r_{opt}}{K+p_{opt}^*}$.

### 4.4. The Kelley Condition and Chattering Phenomena

To check whether a singular arc of the system verifies the Kelley Condition [23], we compute the derivatives

$$ \frac{\partial}{\partial \alpha} \frac{d^k}{dt^k} H_1 $$

over a singular arc. They are null for all $k < 4$. For $q=2$, we find

$$ \frac{\partial}{\partial \alpha} \frac{d^2 q}{dt^2 q} \phi(t) = - \frac{p^2 r^2 [2pE_M(t) + (K + 2p)E'_M(t)]}{(K + p)^4} $$

Since singular arcs of our control are contained in intervals over which $E_M$ is constant, the necessary condition to have a singular trajectory

$$ (-1)^q \frac{\partial}{\partial \alpha} \frac{d^2 q}{dt^2 q} \phi(t) < 0, $$

is fulfilled over any singular arc. This arc is of even order $q = 2$, which means that it can only be attained and left through chattering [24]. That is, in order to attain the singular arc, an infinite amount of bang arcs are needed in a finite time.

We have found that the optimal control consists of a concatenation of bang arcs, with possibly some singular arcs corresponding to the solution of the related static optimization problem. If these singular arcs appear, they can only be reached and left through chattering. In the next section, we will solve the OCP numerically, which will shed more light on the structure of the optimal control.

<table><caption>Table 1: Conditions of the numerical resolution</caption><thead><tr><th>K</th><th>E<sup>min</sup><sub>M</sub></th><th>E<sup>max</sup><sub>M</sub></th><th>T</th><th>(p<sub>0</sub>, r<sub>0</sub>)</th></tr></thead><tbody><tr><td>0.003</td><td>0.2</td><td>1.0</td><td>10</td><td>(0.024, 0.18)</td></tr></tbody></table>

## 5. Numerical Computation of the Optimal Control

We now compute the optimal control by means of a direct numerical resolution. We set a finite time horizon of $t_f = 40 = 4T$. We will omit the last period of the result, as the control will be seen to tend to a limit cycle, before changing on this last period due to boundary effects and thus depleting the cell of a great part of its precursor resources $p$ a short time before the final time of the simulation. We will see in Section 6 that this limit cycle is the solution of the similar optimization problem of maximizing growth over one period with periodic boundary conditions.

The values of the constant and initial values of the OCP fed into the software BOCOP [25] are given in Table 1. The initial value of the system corresponds to $(p_0, r_0) = (p_{opt}^*, r_{opt}^*) (E_M^{\min})$. We have used $N = 4000$ time steps and a relative tolerance of $10^{-14}$.

### 5.1. Result of the Numerical Resolution

Figure 2a shows the optimal control $\alpha_{opt}$ found for this piece-wise constant periodic environment. We see from the plot that the optimal control seems to tend to a limit cycle. We are able to numerically verify this convergence. Because of this limit cycle, and since the last period is affected by the proximity of the final time of the resolution, we have decided to plot $\alpha_{opt}$ only until the time $t = 30 = 3T$.

### 5.2. Structure of the Optimal Control

The optimal control starts with a first transient phase on the interval $[0, T]$ and then becomes periodic, with the same period as $E_M$. During its first period, it presents a very brief constant singular arc included in the interval over which the environmental variable is maximal. This singular arc is longer during the phase where $\alpha_{opt}$ becomes periodic. In both cases, the singular arc is preceded and followed by bang-bang arcs, with chattering occurring in between. This is in accordance with the theoretical study carried out in Section 4.

We notice that the control does not reach the second theoretically possible singular arc, which would correspond to $E_M = E_M^{\min}$. However, if the period is long enough, the control does reach both possible singular arcs. It may also have no singular arcs (see Figure 5) if the period is short enough.

Fig. 2b shows the time plot of the adjoint variable $\lambda_r$, obtained through the resolution of System (4). This variable determines the value of $\alpha_{opt}$. From this plot, we confirm that
<figure><img src="image_3.png" /><figcaption>(a) The optimal control. The top dotted line corresponds to $\alpha_{opt}^*(E_M^{\max})$ and the bottom one to $\alpha_{opt}^*(E_M^{\min})$.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>(b) Time plot of $\lambda_r$. The sign of this adjoint variable determines the value of $\alpha_{opt}$. The intervals on which $\lambda_r$ is zero correspond to the singular arcs of $\alpha_{opt}$.</figcaption></figure>

<caption>Figure 2: Result of the numerical simulation of the OCP for a finite final time of $t_f = 40$. The plots over the last period have been omitted.</caption>

$\lambda_r$ is null on the computed singular arcs.

As happens to the control, the system goes through a transient first period to then become periodic, as shown in Figure 3. The point $(p_{opt}^*, r_{opt}^*)(E_M^{\max})$ is reached during this periodic phase, but not $(p_{opt}^*, r_{opt}^*)(E_M^{\min})$.

### 5.3. Feasibility of the Optimal Strategy

We also notice that the control anticipates the changes in the environment of the bacteria, by having a bang arc $\alpha = 1$ or $\alpha = 0$ which begins shortly before a discontinuity in $E_M$. This increases the concentration of gene expression machinery ($r$) to prepare for a higher concentration of nutrients (given by

<figure><img src="image_5.png" /><figcaption>Figure 3: Phase plot of the variables $p$ and $r$ from (1), on the time-interval $[0, 3T]$ and under $\alpha_{opt}$. In blue, over the first period and in orange over the second and third periods. The solutions of the static OCP corresponding to both possible values of $E_M$ are highlighted.</figcaption></figure>

a higher $E_M$), or decreases $r$ in preparation for the upcoming nutrient shortage. This property is consistent with what has been found in other biological OCP in the literature [21].

One may wonder about how this particular aspect of the optimal control can be implemented in practice, as it requires knowledge of the future evolution of the environment of the cell. We can make the supposition that bacteria are specially prepared to thrive in an environment with a certain period. One can think for example of how day/night [26] or seasonal dynamics all have a fixed period. We shall also see in Section 8 how a simple feedback control independent from $E_M$ can result in near-optimal growth.

## 6. Relation to the Periodic Problem

In [10], it is shown that the optimal control (in a static environment) ended in a constant arc corresponding to a stable equilibrium point that maximized steady-state growth. In the case of a periodic environment, the equivalent of an equilibrium point under a constant control is a periodic orbit corresponding to a periodic control. We would like to investigate which control maximizes growth on such an orbit of the same period as the environment, and whether the optimal control that we found in Section 5 tends to it.

That is, we are now interested in solving the similar problem of finding a control

$$ \alpha_{opt}^* \in \arg \max \int_{0}^{T} \frac{p}{K+p} r \, dt, $$

where the system $(p, r)(t)$ follows System (1) and fulfills the periodic boundary condition $(p, r)(T) = (p, r)(0)$. The environment is given, as stated above, by Equation (2).

The solution to this problem is shown in Figure 4. This control and the optimal control for our initial problem (on the limit cycle shown in Figure 2a) completely overlap. This seems to be a general phenomenon associated with our model, irrespective of the parameters used. We see that in a periodic environment the infinite-horizon optimal control eventually leads the system along a periodic trajectory which is optimal along the set of all periodic trajectories of the same period $T$.
<figure><img src="image_6.png" /><figcaption>Figure 4: The optimal control for the periodic problem. Its plot coincides visually with that of the optimal control for the original problem over $t \in [2T, 3T)$.</figcaption></figure>

## 7. Effect of the Period and Amplitude of the Environment

In this section, we will study how two of the parameters defining the environment given by (2) affect the resulting long-term volume increase of the solution of our OCP. We will consider a fixed average environmental capacity, defined as

$$ \bar{E}_M = \frac{1}{T} \int_0^T E_M(t) \, dt. $$

For the environmental input defined in Equation (2), we have that $\bar{E}_M$ is the arithmetic mean of $E_M^{\min}$ and $E_M^{\max}$.

The parameters whose effect we will study are the period $T$ and the amplitude of the environment $\Delta E_M = E_M^{\max} - E_M^{\min}$. Thus, we can define the environmental input $E_M$ by giving, instead of its two possible values, its average and its amplitude. The condition that $E_M^{\min}$ be positive is equivalent to $\Delta E_M \le 2\bar{E}_M$.

In order to measure the asymptotic volume increase rate, we will use the equivalent average growth rate over a period of the limit cycle. This rate can be defined over any time interval $[0, t]$ as

$$ \bar{\mu}(t) = \frac{\log(V(t)) - \log(V(0))}{t} = \frac{\int_0^t \mu(\tau) d\tau}{t}, $$

such that $V(t) = V(0)e^{\bar{\mu}(t)t}$. If, as we will do in this section, we focus on $\bar{\mu}(T)$, the average growth rate over a period of duration $T$ of the environmental input, we obtain that $V(t)$ behaves asymptotically as $e^{\bar{\mu}(T)t}$. We will take $V(t)$ to be the volume variable of the solution to the problem defined in Section 6.

It is especially interesting to investigate whether, for a fixed $\bar{E}_M$, there exist some $T$ and $\Delta E_M$ that maximize the average exponential increase in biomass that the solution of the optimal control problem with those parameters yields. If this is so, we would want to approximate those values and measure how much it increases the potential cell growth with respect to the one attainable in a constant environment.

We will use the BOCOP solver to find the value of the parameters $T$ and $\Delta E_M$ that maximize this average growth $\bar{\mu}(T)$ over a period, assuming the control $\alpha = \alpha_{opt}^*$. In order to be able to include $T$ into the differential equations system we normalize by dividing the time by this parameter so that the normalized system reads as follows.

$$ \left\{ \begin{aligned} \frac{dp}{dt}(\hat{t} = t/T) &= T \left[ (1-r)E_M(\hat{t}) - (1+p) \frac{p}{K+p} r \right] \\ \frac{dr}{dt}(\hat{t} = t/T) &= T \left[ (\alpha-r) \frac{p}{K+p} r \right], \end{aligned} \right. $$

where the environmental constant with normalized time is given by

$$ E_M(\hat{t}) = E_M(t) = \begin{cases} E_M^{\max}, & \text{if } t \in [0, T/2) \iff \hat{t} \in [0, 1/2) \\ E_M^{\min}, & \text{if } t \in [T/2, T) \iff \hat{t} \in [1/2, 1) \end{cases} $$

and the periodic conditions are now

$$ \left\{ \begin{aligned} p(\hat{t} = 1) &= p(\hat{t} = 0) \\ r(\hat{t} = 1) &= r(\hat{t} = 0) \end{aligned} \right. $$

The average growth rate $\bar{\mu}(T)$ to be maximized can be expressed as a function of the variables in normalized time as

$$ \bar{\mu}(T) = \int_{0}^{1} \frac{pr}{K+p}(\tau) d\hat{t}. $$

As before, we take the constants $K = 0.003$ and $\bar{E}_M = 0.6$ for the numerical resolution of this problem. We now seek to optimize over three variables: the resource allocation parameter $\alpha: [0, +\infty) \to [0, 1]$ and the real parameters $T \in (0, +\infty)$ and $\Delta E_M \in [0, 2\bar{E}_M]$.

When initialized with the right initial conditions, the BOCOP software [25] gives a solution of $(T, \Delta E_M) \approx (0.26, 1.2 = 2\bar{E}_M)$, which results in an average growth rate of $\bar{\mu}_{opt} \approx 0.3571$, slightly better than in a constant environment ($\bar{\mu}_{opt}^* \approx 0.3561$). Note that the optimal amplitude of the environment appears to be the maximal possible one. The optimal control for this environment, which is plotted in Figure 5, has a simple bang-bang structure, with just two switches per period.

Even though it only slightly enhances the maximal growth rate attainable in a constant environment, this result is in our opinion very interesting, as it shows that it is possible to have periodic conditions which are more advantageous than constant ones. This result has been proved in the literature for other biological process [27], but it is not generally true that periodic inputs improve on constant ones [28].

We shall see in the following section how our model explains some of the behaviors actually observed in real-life bacteria.
<figure><img src="image_7.png" /><figcaption>Figure 5: Plot of the optimal control $\alpha_{opt}$ for the environmental input defined by Equation (2) with the optimal parameters ($T, \Delta E_M) \approx (0.26, 1.2 = 2\bar{E}_M$).</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 6: Average growth rate on a period of the limit cycle, under the optimal $\alpha$ (red) and the on-off control (blue). The top dotted line corresponds to $\mu_{avg}^*$ and the bottom one to $\mu_J$.</figcaption></figure>

## 8. Study of a control strategy

In this section, we will study one control strategy that appears to be the one used by real-life bacteria. The ppGpp signaling molecule has been found to play an important role in the regulation of the transcription of growth-related genes [29–31]. Following our assumption that bacteria are selected by evolutionary pressure according to their ability to outgrow their opponents, we will see how the $\alpha$ determined by the mechanisms of ppGpp compares to the optimal control in terms of resulting growth rate in a periodically changing environment. This suboptimal control strategy was found in [10] to result in a near-optimal growth when the bacteria were exposed to a single upshift in the concentration of nutrients.

We will also compare the experimental observations realized on bacteria evolving in fluctuating environments [20] to the ones that can be drawn from our model, assuming the cell population implements this ppGpp-mediated strategy. These comparisons will yield insights on the importance of this molecule in regulating the response of bacteria to fluctuations in their environment.

### 8.1. Optimality of ppGpp

We now compare the growth rate attained using the optimal control and the ppGpp-regulated one. We consider the on-off control given in [10], which is found in that same article to be very similar to the actual functioning of the ppGpp molecule.

This on-off control is given by

$$ \alpha = \begin{cases} 1 & \text{if } r < g(p) \\ 0 & \text{if } r > g(p), \\ r & \text{if } r = g(p) \end{cases}, $$

where

$$ g(p) = \frac{p^2 + Kp}{p^2 + 2Kp + K}, $$

so that

$$ g(p_{opt}^*(E_M)) = r_{opt}^*(E_M) \quad \forall E_M \ge 0. $$

Thus, for a constant $E_M$, the system under this control has $(p_{opt}^*, r_{opt}^*)(E_M)$ as a globally asymptotically stable equilibrium

point. This implies that the rate $\bar{\mu}(T)$ of bacteria under this on-off control tends to $\mu_J$, the mean between $\mu_{opt}^*(E_M^{\max})$ and $\mu_{opt}^*(E_M^{\min})$, when $T$ grows to infinity.

In order to find the theoretically optimal behavior of bacteria accustomed to a periodic environment, we take the periodic conditions $(p, r)(0) = (p, r)(T)$ for the OCP, where $T$ represents the period of the environment. The solution $\alpha_{opt}^*$ to this problem corresponds to the limit cycle of the infinite-horizon optimal control, as shown in Section 6. For simulating the on-off strategy, we have numerically computed its limit cycle.

We have solved the resulting optimal control problems using BOCOP, with the same parameters as in Table 1, except for the period $T$ and the boundary conditions.

For a large range of values of $T$, we have calculated the average growth rate on a given period $\bar{\mu}(T)$ under the optimal and the on-off control. We have compared both average rates with each other and with $\mu_{avg}^*$, the optimal steady-state growth rate in a constant environment $E_M^{\text{avg}} = (E_M^{\max} + E_M^{\min})/2$. The results are shown in Figure 6. We can appreciate how the average growth rate under the optimal control tends to $\mu_J$, as it does under the on-off control. This is due to the fact that $\alpha_{opt}^*$ tends to spend most of each subperiod on the singular arc corresponding to $(p_{opt}^*, r_{opt}^*)(E_M)$. This property has also been observed in the optimal control after a single shift in the environment in [10], and is in accordance with the findings of [5, 6] on the behavior of bacteria in steady environments.

Neither the optimal strategy nor the on-off control show a decrease in the obtainable growth rate with respect to a constant medium for the most rapidly varying periodic environments. For longer periods $T$, we observe that the growth rate becomes significantly smaller, although this difference is more pronounced in the case of the on-off control.

Fig. 7 shows the volume loss per hour of a cell using the ppGpp-regulated control when compared to a cell that uses the optimal control. The former control strategy is close to optimal for very short or long periods $T$, although it shows a
significant loss in growth rate for periods ranging from a dozen
minutes to an hour. It can lose up to 15% of mass every hour
when compared to an optimal strategy for a period *T* of ap-
proximately 30 minutes. This may result from an evolutionary
adaptation to either fast or slow adaptations, or from an intrin-
sic difficulty in adapting to fluctuating environments of an in-
termediate period. Future experiments on bacterial evolution
in dynamical conditions may shed more light on this matter.

8.2. *Comparison with experimental data*

We now compare some of the theoretical results that can be
obtained through our model to the ones directly measured on
*E. coli* in [20]. In the series of experiments described in that
article, some of the microorganisms are kept in a constant
environment (corresponding to *E*<sub>M</sub> = *E*<sub>M</sub><sup>max</sup> or *E*<sub>M</sub> = *E*<sub>M</sub><sup>min</sup> in
our model) and others evolve in a fluctuating environment
which follows the same piecewise-constant periodic shape
as our *E*<sub>M</sub>. These last bacteria are divided into groups for
which the environment is of period *T* = 30 seconds, 5, 15
or 60 minutes. When a periodic regime is achieved, the
resulting growth rate is measured by single-cell microscopy
and compared to a single shift experiment.

We have also simulated the reaction (in terms of the
time-evolution of their growth rate $\mu$) to a single upshift or
downshift of bacteria previously evolving in a constant poor
or rich environment, respectively. The initial conditions for
these simulations are therefore ($p_0, r_0$) = ($p_{opt}^*, r_{opt}^*$)($E_M^{min}$)
or ($p_0, r_0$) = ($p_{opt}^*, r_{opt}^*$)($E_M^{max}$) for bacteria accustomed to a
(constant) low or rich environment, respectively. We have
then compared it with the response to the same scenarios
of bacteria acclimatized to periodic environments, that is
on the limit cycle produced by the on-off control and their
respective environment $E_M$. Figures 8 and 9 summarize the
results obtained. We have plotted the growth rate on a long, 2
hour time span for cells used to a constant environment, and
on one semi-period of time length $T/2$ (the interval $[0, T/2]$ in
the case of an upshift and $[T/2, T]$ in the case of a downshift)
for the other bacteria, as has been done in [20].

The on-off control produces a sharper increase in growth
rate immediately after the upshift in bacteria dwelling in peri-
odic environments in comparison to a single upshift, as shown

<figure><img src="image_9.png" /><figcaption>Figure 7: Ratio between the cellular volumes after one hour on the limit cycle, under the on-off control and under the optimal control.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Figure 8: Time plot of the growth rate $\mu(t)$ in a downshift scenario and under an on-off control $\alpha = \alpha(p, r)$. Dotted lines indicate experimental data.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 9: Time plot of the growth rate $\mu(t)$ in an upshift scenario and under an on-off control $\alpha = \alpha(p, r)$. Dotted lines indicate experimental data.</figcaption></figure>

in Figure 9. This behavior can also be seen in the experimental
data shown in the same plots. It has led the authors of [20] to
propose that the mechanisms regulating bacterial growth may
be different between constant and periodic environments. In
a downshift scenario, the on-off strategy exhibit a very similar
behavior in all conditions (Figure 8).

One may wonder why the same on-off control yields differ-
ent growth rates in bacteria previously dwelling in different
environments. The answer is that the differential equations
system has different limit cycles depending on the period of
*E*<sub>M</sub>, as can be seen in Figure 10. None of the cycles studied
in this article attain the equilibrium points (*p*<sub>opt</sub><sup>*</sup>, *r*<sub>opt</sub><sup>*</sup>) (*E*<sub>M</sub><sup>min</sup>)
and (*p*<sub>opt</sub><sup>*</sup>, *r*<sub>opt</sub><sup>*</sup>) (*E*<sub>M</sub><sup>max</sup>), the starting points of the simulations
for the bacteria previously in a constant environment. The
difference in the time-evolution of the growth rates is thus
explained by the difference in the starting points of the system.

In conclusion, the on-off feedback strategy, which we re-
mind the reader is similar to the actual dynamic of the ppGpp
signaling molecule, appears to explain well the differences ob-
<figure><img src="image_12.png" /><figcaption>Figure 10: Different limit cycles of the system under the on-off control corresponding to different values of the environmental period *T*. The dotted line corresponds to $r = g(p)$, the switch in $\alpha$. The equilibrium points corresponding to $E_M^{\max}$ and $E_M^{\min}$ are also marked.</figcaption></figure>

served in [20] between bacteria accustomed to a constant environment and those evolving in a periodic one. In addition, the growth rate attainable under this control is close to the maximal theoretically possible one. This corroborates previous findings [29–31] highlighting the crucial role the ppGpp signaling molecule plays in the regulation of the allocation of protein precursors in *E. coli*.

## 9. Conclusion

In this work, we studied the problem of the optimization of resource allocation in bacteria in a dynamic environment. Basing ourselves on the model proposed in [10], we modified the input representing the substrate availability to a periodical one, which is better able to capture the real-life conditions of most microorganisms.

We gave a precise definition of our optimal control problem, and carried out a theoretical study of its solution using a variant of Pontryagin's Maximum Principle.

We proved, using this theorem, that the optimal infinite-horizon control follows a bang-bang structure, with two possible distinct singular arcs. We saw that each of these arcs corresponds to a steady-state which is the solution of the related static OCP. In addition, we demonstrated that singular arcs can only be reached and left through chattering.

We then computed this optimal control numerically, using the BOCOP software. We saw how, for the set of parameters that we used, the control goes through a transient phase before repeating the solution of the related periodic problem at infinity. These numerical computations also showed how, in

periodic environments, the optimal control strategy requires microorganisms to anticipate the changes in the environment. Additionally, we found, among all environments of the same shape and given mean, the one that resulted in the highest optimal growth rate.

Finally, we considered a control strategy that is likely the one implemented by real-life bacteria. We compared the average growth-rate produced by the ppGpp-mediated control to the solution of the OCP. We also compared the time evolution of growth rate of bacteria that use this suboptimal control with the one measured experimentally. Through these simulations, we have found strong evidence that corroborate that the ppGpp signaling molecule regulates the protein resource allocation in *E. coli*.

## Acknowledgments

This work has been supported by the French government, through the EUR DS4H Investments in the Future Project managed by the National Research Agency (ANR) with the reference number ANR-17-EURE-0004.

## References

[1] James C Liao et al. "Fuelling the future: microbial engineering for the production of sustainable biofuels". In: *Nature Reviews Microbiology* 14.5 (2016), pp. 288-304.

[2] Ivan Yegorov et al. "Optimal control of bacterial growth for the maximization of metabolite production". In: *Journal of mathematical biology* 78 (2019), pp. 985-1032.

[3] Agustín Gabriel Yabo, Jean-Baptiste Caillau, and Jean-Luc Gouzé. "Optimal bacterial resource allocation strategies in batch processing". In: (2023).

[4] Jan Ewald, Martin Bartl, and Christoph Kaleta. "Deciphering the regulation of metabolism with dynamic optimization: an overview of recent advances". In: *Biochemical Society Transactions* 45.4 (2017), pp. 1035-1043.

[5] Jeremy S Edwards, Rafael U Ibarra, and Bernhard O Palsson. "In silico predictions of Escherichia coli metabolic capabilities are consistent with experimental data". In: *Nature biotechnology* 19.2 (2001), pp. 125-130.

[6] Rafael U Ibarra, Jeremy S Edwards, and Bernhard O Palsson. "Escherichia coli K-12 undergoes adaptive evolution to achieve in silico predicted optimal growth". In: *Nature* 420.6912 (2002), pp. 186-189.

[7] Nathan E Lewis et al. "Omic data from evolved *E. coli* are consistent with computed optimal growth from genome-scale models". In: *Molecular systems biology* 6.1 (2010), p. 390.

[8] Matthew Scott et al. "Emergence of robust growth laws from optimal regulation of ribosome synthesis". In: *Molecular systems biology* 10.8 (2014), p. 747.

[9] Arijit Maitra and Ken A Dill. "Bacterial growth laws reflect the evolutionary importance of energy efficiency". In: *Proceedings of the National Academy of Sciences* 112.2 (2015), pp. 406-411.

[10] Nils Giordano et al. "Dynamical allocation of cellular resources as an optimal control problem: novel insights into microbial growth strategies". In: *PLoS computational biology* 12.3 (2016), e1004802.

[11] Michael Y Pavlov and Mans Ehrenberg. "Optimal control of gene expression for fast proteome adaptation to environmental change". In: *Proceedings of the National Academy of Sciences* 110.51 (2013), pp. 20527-20532.
[12] Hugo A van den Berg et al. "Optimal allocation between nutrient uptake and growth in a microbial trichome". In: *Journal of Mathematical Biology* 37 (1998), pp. 28–48.

[13] Agustín G Yabo et al. "Dynamical analysis and optimization of a generalized resource allocation model of microbial growth". In: *SIAM Journal on Applied Dynamical Systems* 21.1 (2022), pp. 137–165.

[14] Mans Ehrenberg, Hans Bremer, and Patrick P Dennis. "Medium-dependent control of the bacterial growth rate". In: *Biochimie* 95.4 (2013), pp. 643–658.

[15] Steffen Waldherr, Diego A Oyarzún, and Alexander Bockmayr. "Dynamic optimization of metabolic networks coupled with gene expression". In: *Journal of theoretical biology* 365 (2015), pp. 469–485.

[16] Orawan Borirak, Martijn Bekker, and Klaas J Hellingwerf. "Molecular physiology of the dynamic regulation of carbon catabolite repression in Escherichia coli". In: *Microbiology* 160.6 (2014), pp. 1214–1223.

[17] Jen Nguyen, Juanita Lara-Gutiérrez, and Roman Stocker. "Environmen tal fluctuations and their effects on microbial communities, populations and individuals". In: *FEMS microbiology reviews* 45.4 (2021), fuaa068.

[18] Katarzyna Potrykus and Michael Cashel. "(p) ppGpp: still magical?" In: *Annu. Rev. Microbiol.* 62 (2008), pp. 35–51.

[19] Lisa U Magnusson, Anne Farewell, and Thomas Nyström. "ppGpp: a global regulator in Escherichia coli". In: *Trends in microbiology* 13.5 (2005), pp. 236–242.

[20] Jen Nguyen et al. "A distinct growth physiology enhances bacterial growth under rapid nutrient fluctuations". In: *Nature Communications* 12.1 (2021), p. 3662.

[21] Francis Mairet and Térence Bayen. "The promise of dawn: microalgae photoacclimation as an optimal control problem of resource alloca tion". In: *Journal of Theoretical Biology* 515 (2021), p. 110597.

[22] Dean A Carlson, Alain B Haurie, and Arie Leizarowitz. *Infinite horizon optimal control: deterministic and stochastic systems*. Springer Science & Business Media, 2012.

[23] Christian Marchal. "Chattering arcs and chattering controls". In: *Journal of Optimization Theory and Applications* 11 (1973), pp. 441–468.

[24] VF Borisov. "Fuller's phenomenon". In: *Journal of Mathematical Sciences* 100.4 (2000), pp. 2311–2354.

[25] Inria Saclay Team Commands. "BOCOP: an open source toolbox for optimal control". In: See http://bocop.org (2017).

[26] Daniel Muratore et al. "Complex marine microbial communities par tition metabolism of scarce resources over the diel cycle". In: *Nature Ecology & Evolution* 6.2 (2022), pp. 218–229.

[27] PL Silveston, H Budman, and E Jervis. "Forced modulation of biologi cal processes: A review". In: *Chemical Engineering Science* 63.20 (2008), pp. 5089–5105.

[28] M Ali Al-Radhawi, Michael Margaliot, and Eduardo D Sontag. "Maximizing average throughput in oscillatory biochemical synthesis sys tems: an optimal control approach". In: *Royal Society open science* 8.9 (2021), p. 210878.

[29] Katarzyna Potrykus et al. "ppGpp is the major source of growth rate control in E. coli". In: *Environmental microbiology* 13.3 (2011), pp. 563– 575.

[30] Anthony O Gaca, Cristina Colomer-Winter, and José A Lemos. "Many means to a common end: the intricacies of (p) ppGpp metabolism and its control of bacterial homeostasis". In: *Journal of bacteriology* 197.7 (2015), pp. 1146–1156.

[31] Kuanqing Liu, Alycia N Bittner, and Jue D Wang. "Diversity in (p) ppGpp metabolism and effectors". In: *Current opinion in microbiology* 24 (2015), pp. 72–79.