2007 International Conference in Honor of Claude Lobry

# Uncoupling Isaacs'equations in nonzero-sum two-player differential games

## The example of conflict over parental care

Frédéric Hamelin<sup>a</sup> — Pierre Bernhard<sup>b*</sup>

<sup>a</sup> UMR 1099 INRA-Agrocampus Ouest-Université de Rennes 1 Bio3P, 65 rue de Saint-Brieuc CS 84215 35042 Rennes cedex, France
FHamelin@agrocampus-ouest.fr

<sup>b</sup> I3S Université de Nice-Sophia Antipolis et CNRS BP 145 06903 Sophia Antipolis cedex, France

* Corresponding author
Pierre.Bernhard@polytech.unice.fr

**ABSTRACT.** We use a recently uncovered decoupling of Isaacs PDE's of some mixed closed loop Nash equilibria to give a rather complete analysis of the classical problem of conflict over parental care in behavioural ecology, for a more general set up than had been considered heretofore.

**RÉSUMÉ.** On utilise un découplage récemment mis en évidence des équations d'Isaacs d'un jeu différentiel pour des stratégies mixtes singulières particulières pour donner une analyse assez complète d'un problème classique en écologie comportementale concernant le conflit à propos des soins parentaux.

**KEYWORDS:** : Behavioural ecology, differential games, mixed strategies

**MOTS-CLÉS:** : Écologie comportementale, jeux différentiels, strategies mixtes
# 1. Introduction

A pair of parents breeding offspring encounters a classical conflict: while both parents, as gene reproducers, have a stake at brining the young to adulthood in order to spread their genes, each bears a cost in terms of fitness by caring for the young, be it in terms of lost opportunities to spread its genes elsewhere, or of lost time to gather energy for itself. Building on the initial work [13], this conflict has been the object of much investigations. See [5, 14]. Dynamical models have been considered, e.g. in [17, 9, 12].

As compared to these earlier works, we shall consider both a given level of welfare to reach for the offspring and a maximum time to reach it: the end of the breeding season. Also, we let that welfare decrease in the absence of parental care, down to death of the young if left careless. Finally, we allow male and female to be asymmetric on two counts: first in terms of “cost” of breeding their offspring. One of the two, for instance, might be more prone to predation, either because it is more visible, or less apt to defend its life. Second, we let them differ in their efficiency at gathering food or otherwise breeding the child. Finally, we shall investigate the incentive for each to defect, leaving the nest and abandoning the offspring.

“*Nothing in Biology makes sense except in the light of evolution*”<fnref n="1" />. In this respect, Behavioral Ecology interprets animal behavior from an evolutionary point of view, i.e., in terms of being a stable outcome of the Natural Selection process. As this is genetic information which is conserved through Selection, animal behavior is expected to be shaped in such a way that it is particularly efficient at transmitting individual’s genetic inheritance throughout generations [18]. This provides the standard paradygm of behavioral ecology: *i.e.* that analysis of animal —or for that matter plant— behavior in terms of decision theory and optimality may have some relevance and say something about their actual behavior, without assuming any *intelligent will* of the animals. This is in line with the famous “selfish gene” theory of R. Dawkins [6], and more specifically Maynard-Smith’s Evolutionarily Stable Strategies (ESS) [19, 18]

Our investigation in terms of the Nash equilibrium of a differential game, and thus a candidate ESS, leads to dynamic state feedback mixed strategies, a novel feature in that literature. The literature on differential games bears a striking difference with that on classical game theory in that, while the latter is mainly concerned with mixed strategies —up to the point that ordinary decisions have to be called *pure* strategies to recall that they are not mixed—, mixed strategies have had little impact on differential games research. On the one hand, differential games have been mainly concerned with state feedback or non anticipative strategies, and the concept of mixed state feedback, or, for that matter mixed nonanticipative strategy, is surely not simple. On the other hand, most of that literature has considered continous decision sets, as opposed to finite, thus allowing for enough convexity or concavity without relying on mixed strategies. Here we have a game with finite decision sets *at each instant of time*, although its dynamic nature makes the decision sets infinite, and of course the strategies sets even more so.

However, we recently showed ([12]) that in the case of a two-player (non-zero-sum) game where each player has only two possible controls —the framework of this article—, not only do mixed strategies come up as a natural concept, but moreover they lead to a

<footnote n="1">1. Theodosius Dobzhansky, geneticist, 1900–1975.</footnote>
concept of bi-singular trajectory fields which seems to have no counterpart in control theory. And moreover, the striking fact is that the pair of Isaacs partial differential equations of Nash equilibria uncouples, allowing for a simple solution in terms of characteristics.

Looking into older literature, this concept should have been uncovered in the late 60's or early 70's. We are surprised—and a bit suspicious— that we did not find any mention of it. The present example shows its effectiveness. At least one other instance is known by now, much more complicated however. See [1].

## 2. The parental care game

### 2.1. Game description

Two animals, 1 and 2, have jointly given birth to an offspring. Let $x \in \mathbb{R}$ be the weight increase of the young. At initial time, $x = 0$. The offspring is adult and viable when $x = 1$. But this must happen during the year it was born, say at or before time $T$. Let $u_i = 1$ if parent $i$ takes care full time of the young, $u_i = 0$ if it defects. In the “pure” dynamics $\dot{x}$ is given as follows:

<table><tr><td>$u_1 \setminus u_2$</td><td>0</td><td>1</td></tr><tr><td>0</td><td>$-\delta$</td><td>$\alpha_2$</td></tr><tr><td>1</td><td>$\alpha_1$</td><td>$\gamma$</td></tr></table>

The coefficients $\alpha_i$, $\gamma$ and $\delta$ are all assumed positive, with $\gamma > \alpha_1 > \alpha_2$. We let $\beta = \gamma - \alpha_1 - \alpha_2$ be the synergy coefficient.

Allowing for “mixed strategies” or partial efforts $u_i \in [0, 1]$ leads to

$$ \dot{x} = a_1 u_1 + a_2 u_2 + c u_1 u_2 - \delta \quad [1] $$

$$ a_i = \alpha_i + \delta, \quad c = \gamma - \alpha_1 - \alpha_2 - \delta. $$

We allow both parents to behave in closed loop, i.e. use controls of the form $u_i = \phi_i(t, x)$. We shall encounter only constant controls, so that existence of solutions to our dynamic equations is not an issue.

The game ends at $\tau = \min\{t \mid x(t) = 1, T\}$. The reward of the parents are $M(x(\tau)) = 1$ or 0 according to whether the young has achieved viability or not, —i.e. $M(1) = 1, M(x) = 0 \forall x < 1—$, decreased by the cost of caring, say

$$ J_i(u_1(\cdot), u_2(\cdot)) = M(x(\tau)) - \varepsilon_i \int_0^\tau u_i(t) \, dt. $$

### 2.2. Pure equilibria

#### 2.2.1. Constant controls

We notice the following simple facts:

**Lemma 1**
1) Any effort that does not lead to $x(\tau) = 1$ is dominated by 0.

2) A parent who cares alone should use the pure strategy $u_i = 1$.

3) The best response to $u_i = 1$ is never $u_j = 1$ unless $\gamma T = 1$.

**Proof**

1) If $M(x(\tau)) = 0$, the payoff to each parent is negative, or 0 for whichever has used $u_i = 0$.

2) If a parent cares alone, to reach $x(\tau) = 1$, it needs to achieve

$$ \int_{0}^{\tau} (a_i u(t) - \delta) \, dt = 1, \quad \implies \quad a_i \int_{0}^{\tau} u_i(t) \, dt = 1 + \delta \tau. $$

Hence its reward is $J_i = 1 - (\varepsilon_i / a_i)(1 + \delta\tau)$ which is decreasing with $\tau$. Hence it should strive to minimize $\tau$.

3) Against $u_j = 1$, a constant response $u_i$ yields $\tau = 1/[(\gamma - \alpha_j)u_i + \alpha_j]$ which is decreasing with $u_i$, as is $J_i = 1 - \varepsilon_i \tau u_i$. Hence if $\tau < T$, a $u_i < 1$ still leads to termination before $T$ and a higher reward. ■

This simple fact suffices to allow us to investigate pure Nash equilibria. Consider the game space in the $(t, x)$ plane. Draw the lines $x = 1 - \alpha_i(T - t)$, called $\mathcal{L}_i$, and $x = 1 - \gamma(T - t)$ called $\mathcal{L}_\gamma$, as in figure 1. (We carry the discussion below for $x(0) = 0$, and with respect to the position of 0 on the time axis. This could easily be extended to an arbitrary initial pair $(t_0, x_0)$.)

<figure><img src="image_1.png" /><figcaption>**Figure 1.** The pure Nash equilibria if the $\varepsilon_i$ are small.</figcaption></figure>

We claim the following

**Theorem 1** The following discussion provides all pure Nash equilibria with constant controls

**Discussion** To the right of line $\mathcal{L}_\gamma$, the child cannot be brought to adulthood within the remaining time. Therefore, the only Nash equilibrium is $(0, 0)$.

Assume $\alpha_1 > \alpha_2$. To the right of line $\mathcal{L}_1$, no parent can bring the child to adulthood alone. Therefore, if the other parent plays $u_j = 0$, the optimum is $u_i = 0$, and $(0, 0)$ is Nash. A joint effort may drive $x$ to 1 before time $T$, but, according to the lemma, except on the line $\mathcal{L}_\gamma$, $(1, 1)$ cannot be a Nash equilibrium. We shall see mixed equilibria in that region.
Between lines $L_1$ and $L_2$, the parent 1 can succeed alone. If its reward in so doing is positive, it is its best response against $u_2 = 0$. And of course $u_2 = 0$ is the best response to $u_1 = 1$ since it yields a reward of 1 to parent 2. Therefore, $(1, 0)$ is the only Nash equilibrium if $\varepsilon_1 < \alpha_1$. Otherwise, the same situation as to the right of $L_1$ prevails.

To the left of line $L_2$, both parents are able to succeed alone. Therefore, if both $\varepsilon_i < \alpha_i$, there are two asymmetric Nash equilibria, $(1, 0)$ and $(0, 1)$. If any of the $\varepsilon_i > \alpha_i$, that parent has no incentive to breed the child alone. Therefore, its best response to 0 is 0. Therefore if one only, say 1, is in that situation, the only Nash equilibrium is $(0, 1)$. If both are, again $(0, 0)$ is a Nash equilibrium, and also $(1, 1)$ provided that $\varepsilon_i < \gamma$. ■

### 2.2.2. Synchronous on-off equilibria

If $c > 0$, Nash equilibria appear, where both parents care or rest simultanesouly. The following sufficient condition has no claim of optimality. Note that we have used $\alpha_1 \ge \alpha_2$ to keep the most stringent of symmetric conditions.

**Theorem 2** Assume $c > 0$. Let $\mathcal{T}_0$ be a subset of $[0, T]$ with measure $\tau_0 \le 1/\varepsilon_i$, $i = 1, 2$. Assume that the controls $\bar{u}_1(t) = \bar{u}_2(t) = \mathbf{1}_{\mathcal{T}_0}(t)$ generate a trajectory $\bar{x}(t)$ ending at $\bar{x}(\tau) = 1$ before time $T$, and that $(1, \bar{u}_2)$ generate a trajectory ending at $\tau_1$. Assume further that over $[\tau_1, \tau]$, the trajectory $\bar{x}(t)$ lies below the line of slope $\gamma - \alpha_2$ passing through its end-point. Then the pair $(\bar{u}_1, \bar{u}_2)$ is a Nash equilibrium.

**Proof** Fix $u_2 = \bar{u}_2$, and pick an arbitrary $u_1(\cdot)$. If the pair $(u_1(\cdot), \bar{u}_2)$ does not lead to termination before time $T$, parent 1 incurs a negative reward, while the condition $\tau_0 \le 1/\varepsilon_1$ insures a positive reward for the pair $(\bar{u}_1, \bar{u}_2)$. Let therefore $\tau'$ be the termination time on this new trajectory. Note that, necessarily, $\tau' \ge \tau_1$. Two cases arise depending on whether $\tau'$ is less or more than $\tau$.

If $\tau' < \tau$, the support of $\bar{u}_2$ may have been curtailed by the early termination. Let $\mathcal{T}_2$ be that curtailed support, and $\tau_2$ its measure. Let $\mathcal{T}_1 = [0, \tau'] - \mathcal{T}_2$, and let $v_1$ and $w_1$ be the integrals of $u_1(\cdot)$ respectively over $\mathcal{T}_1$ and $\mathcal{T}_2$. We have

$$x(\tau') = 1 = a_1(v_1 + w_1) + a_2\tau_2 + cw_1 - \delta\tau' = \bar{x}(\tau) = (a_1 + a_2 + c)\tau_0 - \delta\tau.$$

This can be rearranged in

$$(a_1 + c)(v_1 + w_1 - \tau_0) = cv_1 + a_2(\tau_0 - \tau_2) - \delta(\tau - \tau'). \quad [2]$$

The hypothesis in the theorem can be written, using $\gamma - \alpha_2 = a_1 + c$,

$$\bar{x}(\tau') = (a_1 + a_2 + c)\tau_2 - \delta\tau' \le (a_1 + a_2 + c)\tau_0 - \delta\tau - (a_1 + c)(\tau - \tau'),$$

which can be rearranged into

$$a_2(\tau_0 - \tau_2) - \delta(\tau - \tau') \ge (a_1 + c)[\tau - \tau' - (\tau_0 - \tau_2)].$$

Combining this with (2), and noting that necessarily, $\tau_0 - \tau_2 \le \tau - \tau'$, we get

$$(a_1 + c)(v_1 + w_1 - \tau_0) \ge cv_1 \ge 0.$$

Since $J_1(u_1(\cdot), \bar{u}_2) - J_1(\bar{u}_1, \bar{u}_2) = -\varepsilon_1(v_1 + w_1 - \tau_0)$, we conclude that $J_1$ has creased in the change.

Otherwise, if $\tau' \ge \tau$, then $\tau_2 = \tau_0$, and (2) directly yields the desired result.
## 2.3. Time sharing equilibria

If $\beta < 0$, that is $\gamma < \alpha_1 + \alpha_2$, i.e. if no synergy exists between the parents, but to the contrary a law of diminishing return prevails, another family of Nash equilibria shows up, where the parents agree to take their turn in caring for the child. Assume that $\alpha_1 T > 1$. Pick a time $\tau < T$ such that $\alpha_2 \tau < 1 < \alpha_1 \tau$. Let

$$ \tau_1 = \frac{1 - \alpha_2 \tau}{\alpha_1 - \alpha_2} \quad \text{and} \quad \tau_2 = \frac{\alpha_1 \tau - 1}{\alpha_1 - \alpha_2}. $$

This way, $\tau_1 + \tau_2 = \tau < T$ and $\alpha_1 \tau_1 + \alpha_2 \tau_2 = 1$. Choose a partition of $[0, \tau]$ into two (measurable) sets $\mathcal{T}_1$ and $\mathcal{T}_2$ of respective Lebesgue measures $\tau_1$ and $\tau_2$. Choose $\bar{u}_i(t) = \mathbb{1}_{\mathcal{T}_i}(t)$, i.e. $1$ if $t \in \mathcal{T}_i$, $0$ elsewhere.

We claim

**Theorem 3** If $\beta < 0$, and if both $\varepsilon_i \tau_i < 1$, the pair $(\bar{u}_1, \bar{u}_2)$ is a Nash equilibrium.

**Proof** Fix $\bar{u}_2$, and choose an arbitrary $u_1(\cdot)$. Let $\tau'$ be the time when the game ends, $\mathcal{T}_2'$ of measure $\tau_2'$ $\le \tau_2$ the support of $\bar{u}_2$ in $[0, \tau']$ —it might be less than $\tau_2$ if the game ends earlier— and $\mathcal{T}_1'$ of measure $\tau_1'$ its complement. Let also $v_1$ and $w_1$ be the integrals of $u_1(\cdot)$ over $c\mathcal{T}_1'$ and $\mathcal{T}_2'$ respectively. Notice that $v_1 \le \tau_1'$.

If $(u_1(\cdot), \bar{u}_2)$ do not bring the state to 1 before time $T$, $J_1$ is negative. Otherwise, using $v_1 + w_1 = \int u_1 dt$,

$$ J_1(u_1(\cdot), \bar{u}_2) - J_1(\bar{u}_1, \bar{u}_2) = -\varepsilon_1(v_1 + w_1 - \tau_1). $$

Also, writing the dynamics in terms of the greek parameters, we have that

$$ x(\tau') = (\alpha_1 + \delta)v_1 + \alpha_2\tau_2' + (\gamma - \alpha_2)w_1 - \delta\tau_1' = 1 = \alpha_1\tau_1 + \alpha_2\tau_2. $$

Using the second and the fourth terms of this equality, we easily get that

$$ \alpha_1(v_1 + w_1 - \tau_1) = \delta(\tau_1' - v_1) - \beta w_1 + \alpha_2(\tau_2 - \tau_2'). $$

If $\beta < 0$, the right hand side is positive, hence the variation in $J_1$ is negative.

<figure><img src="image_2.png" /></figure>

Notice that, contrary to the mixed equilibrium of the next paragraph, this is a strict Nash equilibrium, as the right hand side above can be zero only if $u_1 = \bar{u}_1$.

## 2.4. Mixed equilibria

### 2.4.1. Time unconstrained trajectories

We now turn to mixed equilibria, using the theory of section 3 (see also [12]) whereby each player renders the opponent's hamiltonian singular. This is therefore an pair of "dynamically equalizing strategies". The Isaacs equation is as follows. We let $V_i(t, x)$ be the two Value functions of the players. We write $\lambda_i(t, x)$ for their derivative in $x$. If they are of class $C^1$, they satisfy

$$ \frac{\partial V_i(t, x)}{\partial t} + \mathcal{H}_i(\lambda_i, \phi_1^*, \phi_2^*) = 0, \quad V_i(\tau, x) = M(x), \quad [3] $$
with

$$ \mathcal{H}_i(\lambda_i, u_1, u_2) = \lambda_i(a_1 u_1 + a_2 u_2 + cu_1 u_2 - \delta) - \varepsilon_i u_i . $$

In these equations, $(\phi_1^*, \phi_2^*)$ stands for a Nash equilibrium of the 2×2 game whose payoffs are the $\mathcal{H}_i$.

It is useful to rewrite this as

$$ \mathcal{H}_i(\lambda_i, u_1, u_2) = (u_i \quad 1 - u_i)H_i \begin{pmatrix} u_j \\ 1 - u_j \end{pmatrix} $$

with

$$ H_i = \lambda_i \begin{pmatrix} \gamma - \varepsilon_i & \alpha_i - \varepsilon_i \\ \alpha_j & -\delta \end{pmatrix} . $$

As a result, the Nash point sought is that of the bi-matrix game

<table><thead><tr><th>u<sub>1</sub> \ u<sub>2</sub></th><th>1</th><th>0</th></tr></thead><tbody><tr><th>1</th><td>λ<sub>2</sub>γ - ε<sub>2</sub></td><td>λ<sub>2</sub>α<sub>1</sub></td></tr><tr><th>0</th><td>λ<sub>1</sub>γ - ε<sub>1</sub></td><td>λ<sub>1</sub>α<sub>1</sub> - ε<sub>1</sub></td></tr><tr><th>0</th><td>λ<sub>2</sub>α<sub>2</sub> - ε<sub>2</sub></td><td>-λ<sub>2</sub>δ</td></tr><tr><th>0</th><td>λ<sub>1</sub>α<sub>2</sub></td><td>-λ<sub>1</sub>δ</td></tr></tbody></table>

The Nash equilibria of the above bi-matrix game are singular controls in the sense of control theory. They are

$$ \phi_i^* = \frac{\varepsilon_j - \lambda_j a_j}{\lambda_j c} \qquad [4] $$

We investigate a field of trajectories reaching the boundary $x = 1$. On such trajectories, locally, the final time is unconstrained. As the rest of the formulation is time invariant, the associated Value is stationary, and $\partial V_i / \partial t = 0$. Placing this and (4) in (3) yields

$$ \phi_i^* = \frac{\delta}{a_i}, \qquad [5] $$

and therefore

$$ \dot{x} = \delta \frac{a_1 a_2 + c \delta}{a_1 a_2} = \delta \frac{\alpha_1 \alpha_2 + \gamma \delta}{(\alpha_1 + \delta)(\alpha_2 + \delta)}. \qquad [6] $$

This slope is necessarily positive and less than $\gamma$. However, depending on $c$, it may be more or less than $\alpha_i$.

**Theorem 4** If

$$ T > \frac{a_1 a_2}{\delta(a_1 a_2 + c \delta)}, \quad \text{and} \quad \varepsilon_i < \frac{a_1 a_2 + c \delta}{a_j}, $$

the mixed strategies (5) are a Nash equilibrium over feedback strategies.

**Proof** Using (6), the first condition in the theorem insures that $\tau < T$, hence $M(x(\tau)) = 1$, and using this, the second one insures that both parents get a positive reward. (Otherwise, $u_i = 0$ is better.) If so, the functions

$$ V_i(x) = 1 - \frac{\varepsilon_i a_j}{a_1 a_2 + c \delta} (1 - x) $$

satisfy equations (3) in the region of the game space covered by the trajectories (6), which includes the initial state of interest, $x(0) = 0$.
### 2.4.2. Time constrained trajectories

We investigate now trajectories that end up exactly at time $T$ with $x(T) = 1$, such that both parents get a positive reward. Let $u_i \in [0, 1]$ be such that

$$T[a_1u_1 + a_2u_2 + cu_1u_2 - \delta] = 1, \quad T\varepsilon_iu_i < 1. \quad [7]$$

**Theorem 5** Under conditions (7) the pair of constant controls $(u_1, u_2)$ is a Nash equilibrium over feedback strategies if and only if for $i = 1, 2$, either $u_i = 1$ and $1 - T\alpha_i \in [0, (\gamma - \alpha_i)/\varepsilon_j]$, or $u_j \ge \phi_j^*$ as given by (5).

**Proof** We compare the constant control $u_i$ to any $u_i + v_i(t)$, assuming that the other parent keeps its control $u_j$ constant. Let $\tau$ be the final time on the trajectory generated by these new controls. If $\tau = T$ and $x(T) < 1$, both parents have a negative payoff. Parent $i$ loses in so doing. Therefore, the new control can be better only if $\tau \le T$, which is impossible to achieve by player $i$ alone if $u_i = 1$.

Assume thus that $u_i = 1$. Since we also assume $x(T) = 1$, this implies $u_j = (\frac{1}{T} - \alpha_i)/(\gamma - \alpha_i)$. This must be nonnegative, and yield $1 - \varepsilon_j T u_j > 0$, which is our condition insures.

Assume now that $u_i < 1$, and let $w_i = \int_0^\tau v_i(t) dt$. We have

$$\tau[a_1u_1 + a_2u_2 + cu_1u_2 - \delta] + (a_i + cu_j)w_i = 1, \quad [8]$$

We assume that indeed $\tau \le T$, thus that $w_i > 0$. (Recall that $a_i + cu_j \ge 0$, even though $c$ may be negative.) We have also $J_i(u_i + v_i, u_j) = 1 - \varepsilon_i\tau u_i - \varepsilon_i w_i$, Using (7) and (8) we find that

$$J_i(u_i + v_i, u_j) - J_i(u_i, u_j) = -\varepsilon_i w_i T (a_j u_j - \delta).$$

Therefore, if $a_j u_j - \delta < 0$, the open loop control $u_i + v_i(\cdot)$ improves the reward of player $i$, and $(u_1, u_2)$ was not Nash. Conversely, if $a_j u_j - \delta \ge 0$, no open loop control can improve $J_i$, and then no feedback strategy can either. (Just apply the above calculation with $u_i + v_i(t)$ equal to the control of player $i$ generated by a test closed loop strategy and $u_j$.) Notice also that if $u_j = \phi_j^*$, the variation in $J_i$ is identically 0. This is the classical equalization property of mixed Nash equilibria. ■

The trajectories generated by these strategies are straight lines through the point $t = T$, $x = 1$. They fill the void between the last bi-singular trajectory and the curve $\mathcal{L}_\gamma$ of Figure 2, and cut into the bi-singular field if a $u_i = 1$ is Nash.

## 2.5. Biological implications

Let us mention a few biological considerations drawn from this analysis.

We stressed in the introduction that, contrary to earlier literature, we allow for asymmetrical parents. In that respect, intuitively, if $\gamma > \alpha_1 + \alpha_2$, we may consider this as a synergetic effect, since both parents acting together do better than the sum of their lone efforts. But if we consider that the efficiency of a parent is in replacing a decrease rate of $\delta$ by an increase of $\alpha_i$, i.e. $a_i = \alpha_i + \delta$, and similarly for the pair $\gamma + \delta$, then the measure of synergy is rather $c$. Both play a role in the above results.

We do not claim to have described all Nash equilibria. But they are clearly highly non unique. More analysis in terms of biological interpretations is needed to sort them out. We give here a few hints.
We notice that some regions of the game space have the mixed strategy as their natural outcome. It is particularly so if *T* is large and the εᵢ small enough, so that the pure Nash equilibria are (1, 0) and (0, 1). Then, the mixed equilibrium appears as the “fair” outcome. The link with an ESS in a population comprising both males and females remains to be investigated further.

The peculiarity of the mixed Nash is that each parent does exactly the effort which, if made alone, keeps ẋ = 0. The interpretation is that this is true on locally time unconstrained trajectories. Therefore the same reasoning as in [12] holds. The fact that the available time be, globally, constrained by T is reflected, on the one hand, through the possible overlap of the bi-singular field of trajectories with the field (0,0), and on the other hand, by the existence of a new field of mixed equilibria trajectories, filling the gap between the bi-singular field and the fastest trajectory to just-in-time completion of the breeding process.

A last point we want to raise is that of the incentive to defect. It follows from the threshold εᵢ < aᵢ + cδ/aⱼ that, if c > 0, increasing the efficiency of the partner j will eventually lead to a choice for i to desert. An apparent paradox. The explanation we propose is that c > 0 means a large synergetic effect. In that case, a less efficient mate, having a lower aⱼ, has a larger φⱼ* = δ/aⱼ. (The threshold is precisely aᵢ + cφⱼ*.) Thus, under the mixed strategy, it will be more often present in the nest, and through the synergetic effect, this will compensate and over for its lower efficiency.

Is this a plausible explanation for the paradox of the handicap [10, 22, 21, 15] in sexual selection whereby a morphological trait which is a clear handicap to the individual enhances its sex-appeal ? We doubt, since it has been noticed that, as a rule, across species, the male takes the less care of the young that the morphological difference is larger.

**3. Uncoupling Isaacs equation**

**3.1. Background**

We deal here with two player differential games with scalar controls, dynamics and payoff affine in each of the controls (and not necessarily jointly affine). A knowledgeable situation where this arises, which will provide our example, is a game situation where each player has only two pure strategies and mixes among them. Thus this is also a result about mixed strategy Nash equilibria in differential games.

The existing literature on mixed strategies in differential games is rather scarce, and even more so concerning nonzero-sum games. Most of it concerns open-loop strategies and existence theorems (see [8, 7]). [16] do consider state feedback mixed strategies, in the context of zero sum two person differential games, and extend their “alternative” theorem, which is the equivalent of the existence of the Value.

As a matter of fact, mixed strategies in dynamical problems are closely related to the concept of relaxed controls, which amount to convexifying the holograph domain (or “vectorgram” in Isaacs’ parlance). The reader is referred to [8] and the references therein.

Here, we use the Isaacs equation approach. It is known that it leads to a pair of coupled first order partial differential equations, for which no simple characteristics theory
is available in general nor any existence result. Yet, we exploit the linearity in the control variables to uncouple these two PDE's in a fashion very much reminiscent of the equalization theorem of static games.

## 3.2. Bilinear differential game

We consider the following two-player nonzero-sum differential game:

$$ \dot{x} = F(x, u_1, u_2), \quad x(0) = \xi, \qquad [9] $$

$$ T = \inf\{t: x(t) \in \mathcal{T}\} \quad \text{and} \quad \forall i \in \{1, 2\}, $$

$$ J_i(\xi, u(\cdot), v(\cdot)) = \int_0^T \ell_i(x(t), u_1(t), u_2(t))dt + K_i(x(T)). $$

where

$$ F(x, u_1, u_2) = A(x) + B_1(x)u_1 + B_2(x)u_2 + D(x)u_1u_2 $$

$$ \ell_i(x, u_1, u_2) = p_i(x) + q_i(x)u_i + r_i(x)u_j + s_i(x)u_iu_j. $$

(Here as in future uses, $u_j$ means $u_{3-i}$.)

The vector $x \in \mathbb{R}^n$ is the state of the system. The controls $u_i$, $i = 1, 2$ are scalars belonging to compact intervals $U_i \subset \mathbb{R}$, the control functions $u_i(\cdot) \in \mathcal{U}_i$ are measurable functions from $\mathbb{R}_+$ to $U_i$. The functions $F$ and $\ell_i$ satisfy standard regularity and growth properties that guarantee existence and uniqueness of the generated trajectories and of the payoffs. Following a standard practice, we do not explicitly include time as an argument. If needed, this can be done by augmenting the state space with a coordinate which coincides with time. And this is true of feedback strategies.

We are interested in a Nash equilibrium, in feedback strategies. To handle state feedback strategies in a dynamic game context, we extend the set up of [2, 3] to non zero sum games. We call $\phi_i(x)$, $i = 1, 2$ state feedback strategies for player one and two respectively, i.e. maps from $\mathbb{R}^n$ to $U_i$, and provided that (9) has a unique solution, we use such notations as $J_1(\xi, \phi_1, u_2(\cdot))$ to mean the payoff obtained by player one when it uses the feedback $u_1(t) = \phi_1(x(t))$ while player two uses $u_2(t)$.

Therefore, in keeping with the set up of [3] we shall exhibit a pair of feedback maps $(\phi_1^*, \phi_2^*)$ such that, on the one hand, the differential equation (9) with $u = \phi_1^*(x)$ and $v = \phi_2^*(x)$ has a unique solution (we say that $J_i(\xi, \phi_1^*, \phi_2^*)$ is well defined) for all $\xi$, as well as $J_1(\xi, u(\cdot), \phi_2^*)$ for all $u(\cdot) \in \mathcal{U}$ and $J_2(\xi, \phi_1^*, v(\cdot))$ for all $v(\cdot) \in \mathcal{V}$. On the other hand, they satisfy the following Nash inequalities: for all $\xi$,

$$ \left\{ \begin{array}{l} \forall u(\cdot) \in \mathcal{U}, \quad J_1(\xi, u(\cdot), \phi_2^*) \le J_1(\xi, \phi_1^*, \phi_2^*) =: V_1(\xi), \\ \forall v(\cdot) \in \mathcal{V}, \quad J_2(\xi, \phi_1^*, v(\cdot)) \le J_2(\xi, \phi_1^*, \phi_2^*) =: V_2(\xi). \end{array} \right. $$

We call $V_i$ the value function of player $i$.

Each player faces one of the following stationary Hamilton-Jacobi-Bellman equations:

$$ \left\{ \begin{array}{l} \forall x \in \mathcal{T}, \quad V_1(x) = K_1(x) \quad \text{and} \quad \forall x \notin \mathcal{T}, \\ 0 = \max_u \{\langle \nabla V_1(x), F(x, u, \phi_2^*(x)) \rangle + \ell_1(x, u, \phi_2^*(x))\}, \end{array} \right. $$

$$ \left\{ \begin{array}{l} \forall x \in \mathcal{T}, \quad V_2(x) = K_2(x) \quad \text{and} \quad \forall x \notin \mathcal{T}, \\ 0 = \max_v \{\langle \nabla V_2(x), F(x, \phi_1^*(x), v) \rangle + \ell_2(x, \phi_1^*(x), v)\}. \end{array} \right. $$
For $i = 1, 2$, let $\lambda_i = \nabla V_i$, $H_i = \langle \lambda_i, F \rangle + \ell_i$. Write the hamiltonians as $H_i(x, \lambda_i, u_i, u_j) = \rho_i(x, \lambda_i, u_j) + \sigma_i(x, \lambda_i, u_j)u_i$. Thus $u_j$ arises linearly in $\rho_i$ and $\sigma_i$.

We shall make use of the following conditions, for $i = 1, 2$, each of which may or
may not be satisfied in any particular problem:

$$
\left|
\begin{array}{l}
\text{C1.i: } \langle \lambda_i, A \rangle + p_i \neq 0 \\
\text{C3.i: } \langle \lambda_i, B_j \rangle + r_i \neq 0
\end{array}
\right|
\quad
\left|
\begin{array}{l}
\text{C2.i: } \langle \lambda_i, B_i \rangle + q_i \neq 0 \\
\text{C4.i: } \langle \lambda_i, D \rangle + s_i \neq 0
\end{array}
\right|
$$

3.3. A bi-singular solution

3.3.1. Isaacs'equations

We are looking for a solution where both controls are singular, or, in the interpretation
of mixed strategies, where both optimal strategies are mixed.

Therefore, the equalities $\rho_1 = \sigma_1 = \rho_2 = \sigma_2 = 0$ should hold. This translates into

$$
\rho_i = \langle \lambda_i, A \rangle + p_i + (\langle \lambda_i, B_j \rangle + r_1)u_j = 0,
$$

$$
\sigma_i = \langle \lambda_i, B_i \rangle + q_i + (\langle \lambda_i, D \rangle + s_i)u_j = 0.
$$

These equations can have a solution only if the following two determinants, for $i = 1, 2$, are null :

$$
\zeta_i(x, \lambda_i) = (\langle \lambda_i, A \rangle + p_i)(\langle \lambda_i, D \rangle + s_i) - (\langle \lambda_i, B_i \rangle + q_i)(\langle \lambda_i, B_j \rangle + r_i) = 0.
$$

Assume that both conditions C4.i, $i = 1, 2$, hold. The players may choose $\phi_i^*(x) = \hat{\phi}_i(x, \lambda_j(x))$ according to

$$
\hat{\phi}_j(x, \lambda_i) = - \frac{\langle \lambda_i, B_i \rangle + q_i}{\langle \lambda_i, D \rangle + s_i}.
$$

Assume to the contrary that, say, C4.1 is violated. Then the equation $\zeta_i = 0$ implies that either C2.i or C3.i is also. If it is C3.i is satisfied, we may choose

$$
\hat{\phi}_j(x, \lambda_i) = - \frac{\langle \lambda_i, A \rangle + p_i}{\langle \lambda_i, B_j \rangle + r_i}.
$$

which, if C4.i is also satisfied, coincides with the previous form. If neither C3.i nor C4.i
hold, there is no bi-singular solution.

Now, the Isaacs equations degenerate into two uncoupled partial differential equations,
for $i = 1, 2$:

$$
\begin{cases}
\forall x \in \mathcal{T}, & V_i(x) = K_i(x), \\
\forall x \notin \mathcal{T}, & \zeta_i(x, \nabla V_i(x)) = 0.
\end{cases}
$$

Notice that if $x$ is scalar then each candidate adjoint vector $\lambda_i$ is solution of an (at most) quadratic equation, which may suffice to compute it. (This is the case in our parental care example, but not in [1].) Otherwise, one may attempt to solve these PDE's via their characteristic curves (see below).

Once the candidate adjoint vectors are obtained, one has to verify whether conditions C3.i or C4.i hold, so that one can calculate a candidate equilibrium feedback strategy. If so, it remains to check whether these are admissible: lying in $U_i$, and regular enough to get the existence properties sought for (9). If yes, we have exhibited a Nash equilibrium in state feedback.
### 3.3.2. Characteristic curves

According to the classical theory of characteristic curves of a first order PDE [4], we may construct two fields of such curves, one for each PDE : $x_i(s)^2$ according to the equations

$$ \frac{dx_i}{ds} = \nabla \lambda_i \zeta_i \qquad \frac{d\lambda_i}{ds} = -\nabla_x \zeta_i . $$

Notice that the characteristic curves are generally not game trajectories.

Let us investigate how these last two differential equations should be initialized. We assume that the field sought ends transversely on an hypersurface $S = \{x | S(x) = 0\}$. This may be the target set on which $V_i(x) = K_i(x)$ is known, and in that case let $\lambda_i^+(x) = \nabla K_i(x)$, or an hypersurface from which an emerging field, constructed backward from the target set, is known, in which the adjoint vectors $\nabla V_i(x) = \lambda_i^+(x)$ have been computed. In both cases, let $\nu(x)$, $x \in S$, be a local normal to $S$. We know from differential geometry that necessarily, in the incoming field denoted with the superscript $-$,

$$ \lambda_i^-(x) = \lambda_i^+(x) + k_i(x)\nu(x). $$

Placing this back in the equation $\zeta_i^- = 0$, $i = 1, 2$, yields a second degree algebraic equation for each $k_i$. These two equations are still uncoupled. Now, one of these equations may have no (real) solution. In that case there is no bi-singular field of the form sought. If both have two real solutions, in our experience, only one pair yielded controls $\phi_j$, $j = 1, 2$, both belonging to their respective control set $U_j$.

### 3.4. Enforcing bilinearity via mixed strategies

Assume that the players' controls are $u_1, u_2 \in \{0, 1\}$. Let $F_{u_1u_2}(x) = F(x, u_1, u_2)$ and the dynamics can be written

$$ \dot{x} = (1 - u_1)(1 - u_2)F_{00}(x) + (1 - u_1)u_2F_{01}(x) + u_1(1 - u_2)F_{10}(x) + u_1u_2F_{11}(x). $$

In a similar fashion, let $\ell_{u_1u_2}^i(x) = \ell_i(x, u_1, u_2)$; $\ell$ can thus also be written

$$ \ell^i(x, u_1, u_2) = (1 - u_2)(1 - u_1)\ell_{00}^i(x) + (1 - u_1)u_2\ell_{01}^i + u_1(1 - u_2)\ell_{10}^i + u_1u_2\ell_{11}^i(x). $$

We follow [8] and, to some extent, [16], in identifying mixed strategies with relaxed controls in the tradition of [20]. In a dynamic setting, this can be seen as the limit of a very high frequency chatter between the two possible controls, provided, though, that it be specified that the two chatters cannot be synchronized. Then, we let $u_1$ and $u_2 \in [0, 1]$ be the mathematical expectations of $u_1$ and $u_2$ respectively. (Linearity allows us to take the mathematical expectations on the controls rather than on $F$ and the $\ell_i$'s.)

Connecting these notations to those of the previous section lets $A = F_{00}$, $B = F_{10} - F_{00}$, $C = F_{01} - F_{00}$ and $D = F_{11} - F_{10} - F_{01} + F_{00}$. Similarly, we have $p_i = \ell_{00}^i$, $q_i = \ell_{10}^i - \ell_{00}^i$, $r_i = \ell_{01}^i - \ell_{00}^i$ and $s_i = \ell_{11}^i - \ell_{10}^i - \ell_{01}^i + \ell_{00}^i$.

<footnote n="2">2. the subscript $i$ of $x$ denotes a field of characteristics, not a coordinate.</footnote>
**Frédéric Hamelin**, après une thèse préparée à I3S, UNSA-CNRS, est Maître de Conférences à Agrocampus Ouest, membre de Bio3P, unité mixte INRA-Agrocampus Ouest-Université de Rennes

1

Pierre Bernhard est Professeur à Polytech'Nice Sophia, membre d'I3S, unité mixte Université de Nice-Sophia Antipolis, CNRS

**4. References**

AKHMETZHANOV, A., BERNHARD, P., GROGNARD, F., AND MAILLERET, L.: "Reproduction dynamics in a differential game between foraging predators and hiding preys", 13th International Symposium on Dynamic games and Applications, Wroclaw, Poland, June 30-July 3rd, 2008.

BERNHARD, P.: "Singular surfaces in differential games, an introduction", in Hagedorn, Knobloch, and Olsder eds., *Differential games and applications. Lecture notes in information and control sciences*, vol. 3, Springer Verlag, Berlin, 1977.

BERNHARD, P.: "Contribution à l'étude des jeux différentiels à deux joueurs, somme nulle et information parfaite", thèse d'État, University Paris 6, Paris, 1979.

CARATHÉODORY C.: *Calculus of variations and partial differential equations of the first order*, Holden-Day, 1965.

CLUTTON-BROCK, T.H. *The Evolution of Parental Care*, University Press, Oxford, 1991.

DAWKINS, R. *The Selfish Gene*, Oxford University Press, Oxford, U.K., 1975.

ELLIOTT, J.R.: "The existence of value in stochastic differential games", *SIAM Jal on Control* vol. 14, pp 85-94, 1976.

ELLIOT, J.R. AND KALTON, N AND MARKUS, L.: "Saddle point for differential games", *SIAM jal on Control* vol. 11, pp 100-112, 1973

EWALD C-O., MCNAMARA J.M., AND HOUSTON A.: "Parental care as a differential game: a dynamic extension of the Houston-Davis game.", *Applied Mathematics and Computations*, vol. 190, pp 1450-1465, 2007.

FISHER R.A. *The Genetical Theory of Natural Selection*, University Press, Oxford, 1930.

HAMELIN F.: "Jeux dynamiques en écologie du comportement", thèse de doctorat, Université de Nice, 4 juillet 2007.

HAMELIN F. AND BERNHARD P.: "Uncoupling Isaacs'equations in two-player nonzero-sum differential games. Parental conflict over care as an example", *Automatica*, vol. 44, pp 882-885, 2008.

HOUSTON A.I. AND DAVIES N.B.: "The evolution of cooperation and life history in the dunnock Prunella modularis". In *Behavioural Ecology*, R.M Silby and R.H. Smith eds., pp 471-487, Blacwell Scientific Publications, 1985.

HOUSTON A.I., SZEKELY T., AND MCNAMARA J.M.: "Conflict between parents over care", *Trends in Ecology and Evolution* vol. 20, pp 33-38, 2005.

KOKKO, H., JENNIONS, M. D., AND BROOKS, R.: "Unifying and testing models of sexual selection" *Annual Reviews of Ecology, Evolution and Systematics* vol. 37, pp 43-66, 2006.

KRASOVSKII, N.N. AND SUBBOTIN, A.I. *Jeux Différentiels*, Mir, Moscow 1977, substantially revised English transl., *Game-theoretical control problems*, Springer Series in Soviet Math., Springer-Verlag, New York 1988.
MCNAMARA J.M. et al.: "A dynamic game-theoretic model of parental care". *Journal of Theoretical Biology* vol. 205, pp 605-623, 2000.

MAYNARD-SMITH, J.: *Evolution and the Theory of Games*, Cambridge University Press, Cambridge, U.K., 1982.

MAYNARD-SMITH, J., AND PRICE, G.R.: "The logic of animal conflict", *Nature* vol. 246, pp 15-18, 1973.

Warga, J.: "Functions of relaxed controls", *SIAM Jal on Control* vol. 5, pp 628-641, 1967.

WINQUIST T., AND LEMON R.E.: "Sexual selection and exaggerated male tail length in birds", *The American Naturalist*, vol. 143, pp 95-116, 1994.

ZAHAVI, A.: "Mate selection —a selection for the handicap", *Jal of Theoretical Biology* vol. 53, pp 205-214, 1975.