# Closing Aubry sets I

Alessio Figalli, Ludovic Rifford

► To cite this version:

Alessio Figalli, Ludovic Rifford. Closing Aubry sets I. Communications on Pure and Applied Mathematics, 2015, 68 (2). hal-00935965

HAL Id: hal-00935965

https://hal.science/hal-00935965

Submitted on 30 Jan 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Closing Aubry sets I

A. Figalli*

L. Rifford†

July 7, 2011

Abstract

Given a Tonelli Hamiltonian $H : T^*M \to \mathbb{R}$ of class $C^k$, with $k \ge 2$, we prove the following results: (1) Assume there exist a recurrent point of the projected Aubry set $\bar{x}$, and a critical viscosity subsolution $u$, such that $u$ is a $C^1$ critical solution in an open neighborhood of the positive orbit of $\bar{x}$. Suppose further that $u$ is “$C^2$ at $\bar{x}$”. Then there exists a $C^k$ potential $V : M \to \mathbb{R}$, small in $C^2$ topology, for which the Aubry set of the new Hamiltonian $H+V$ is either an equilibrium point or a periodic orbit. (2) If $M$ is two dimensional, (1) holds replacing “$C^1$ critical solution + $C^2$ at $\bar{x}$” by “$C^3$ critical subsolution”.

These results can be considered as a first step through the attempt of proving the Mañé's conjecture in $C^2$ topology. In a second paper [27], we will generalize (2) to arbitrary dimension. Moreover, such an extension will need the introduction of some new techniques, which will allow us to prove in [27] the Mañé's density Conjecture in $C^1$ topology. Our proofs are based on a combination of techniques coming from finite dimensional control theory and Hamilton-Jacobi theory, together with some of the ideas which were used to prove $C^1$-closing lemmas for dynamical systems.

Contents

<table><tr><td>1 Introduction</td><td>2</td></tr><tr><td>1.1 Aubry-Mather theory from the Lagrangian viewpoint</td><td>3</td></tr><tr><td>1.2 Aubry-Mather theory from the Hamiltonian viewpoint</td><td>5</td></tr><tr><td>1.3 The Mañé Conjecture</td><td>8</td></tr><tr><td>2 Statement of the results</td><td>9</td></tr><tr><td>3 Connecting Hamiltonian orbits by potentials</td><td>12</td></tr><tr><td>3.1 Statement of the result</td><td>12</td></tr><tr><td>3.2 Proof of Proposition 3.1</td><td>14</td></tr><tr><td>4 Controlling the action by potentials</td><td>23</td></tr><tr><td>4.1 Statement of the result</td><td>23</td></tr><tr><td>4.2 Proof of Proposition 4.1</td><td>24</td></tr><tr><td>4.3 Remarks</td><td>29</td></tr></table>

<footnote n="0">⁰Both authors are supported by the program “Project ANR-07-BLAN-0361, Hamilton-Jacobi et théorie KAM faible”. AF is also supported by NSF Grant DMS-0969962.</footnote>

<footnote>*Department of Mathematics, The University of Texas at Austin, 1 University Station C1200, Austin TX 78712, USA (figalli@math.utexas.edu)</footnote>

<footnote>†Université de Nice-Sophia Antipolis, Labo. J.-A. Dieudonné, UMR CNRS 6621, Parc Valrose, 06108 Nice Cedex 02, France (rifford@unice.fr)</footnote>
<table><tbody><tr><td>5</td><td>Proof of Theorem 2.1</td><td>30</td></tr><tr><td>5.1</td><td>Introduction</td><td>30</td></tr><tr><td>5.2</td><td>Preliminary steps</td><td>30</td></tr><tr><td>5.3</td><td>Closing the Aubry set</td><td>35</td></tr><tr><td>5.4</td><td>Control of the action</td><td>40</td></tr><tr><td>5.5</td><td>Construction of a critical subsolution</td><td>42</td></tr><tr><td>6</td><td>Proof of Theorem 2.4</td><td>47</td></tr><tr><td>6.1</td><td>Preliminary step</td><td>47</td></tr><tr><td>6.2</td><td>Modification of the potential and conclusion</td><td>48</td></tr><tr><td>6.3</td><td>Construction of the potential</td><td>49</td></tr><tr><td>7</td><td>Final comments</td><td>50</td></tr><tr><td>A</td><td>Conventions and standing notation</td><td>51</td></tr><tr><td>B</td><td>Controllability of nonlinear control systems</td><td>52</td></tr><tr><td>B.1</td><td>Preliminaries</td><td>52</td></tr><tr><td>B.2</td><td>Singular controls</td><td>53</td></tr><tr><td>B.3</td><td>Application to partial controllability I</td><td>55</td></tr><tr><td>B.4</td><td>Application to partial controllability II</td><td>56</td></tr><tr><td>B.5</td><td>Application to partial controllability III</td><td>57</td></tr><tr><td>C</td><td>Quantitative Inverse Function Theorem</td><td>58</td></tr><tr><td>D</td><td>The Mai Lemma</td><td>58</td></tr><tr><td>E</td><td>Proofs of Lemmas 3.3, 4.3 and 6.1</td><td>59</td></tr><tr><td>E.1</td><td>Proof of Lemma 3.3</td><td>59</td></tr><tr><td>E.2</td><td>Proof of Lemma 4.3</td><td>60</td></tr><tr><td>E.3</td><td>Proof of Lemma 6.1</td><td>62</td></tr><tr><td></td><td>References</td><td>63</td></tr></tbody></table>

# 1 Introduction

Let $(M, g)$ be a smooth compact Riemannian manifold without boundary of dimension $n \ge 2$. Given $H: T^*M \to \mathbb{R}$ a smooth Tonelli Hamiltonian, the Mañé conjecture in $C^k$ topology (with $k \ge 2$) asserts that, for generic potentials $V \in C^k(M)$, the projected Aubry set $\tilde{A}(H+V)$ associated to the Hamiltonian $H+V$ is either an equilibrium point or a periodic orbit.

This paper is the first of a series of articles where we plan to make progress toward a proof of the Mañé Conjecture in $C^2$ topology. The aim of this first paper is to show how to prove the density part of the Mañé Conjecture in $C^2$ topology under the following assumptions (Theorem 2.1): there exist a recurrent point of the projected Aubry set $\bar{x}$, and a critical viscosity subsolution $u$, such that $u$ is a $C^1$ critical solution in an open neighborhood of the positive orbit of $\bar{x}$, and $u$ is “$C^2$ at $\bar{x}$”. Then, in two dimensions we show how to replace the above assumption by replacing “$C^1$ critical solution + $C^2$ at the point” with “$C^3$ critical subsolution” (Theorem 2.4). In a second paper we will perform the extension of this last result to arbitrary dimension [27, Theorem 1.1]. Moreover, the proof of this last result will involve the introduction of some new ideas and techniques, which will allow us to prove the (density part of the) Mañé Conjecture in $C^1$ topology [27, Theorem 1.2].
Before describing our results in detail, we first introduce the Aubry-Mather theory from both the Lagrangian and the Hamiltonian points of view. Some conventions and standing notation are gathered in Appendix A.

## 1.1 Aubry-Mather theory from the Lagrangian viewpoint

Let $L: TM \to \mathbb{R}$ be a $C^k$ Tonelli Lagrangian, that is, a Lagrangian of class $C^k$ (with $k \ge 2$) satisfying the two following assumptions:

(L1) Superlinear growth: For every $K \ge 0$, there is a finite constant $C(K)$ such that

$$L(x, v) \geq K \|v\|_{x} + C(K) \quad \forall (x, v) \in TM.$$

(L2) Strict convexity: For every $(x, v) \in TM$, the second derivative along the fibers $\frac{\partial^2 L}{\partial v^2}(x, v)$ is positive definite.

The critical value of $L$ is defined as

$$c[L] := - \inf_{T>0} \left\{ \frac{1}{T} \mathbb{A}(\gamma; [0, T]) \mid \gamma \in C^1([0, T], M), \gamma(0) = \gamma(T) \right\}, \quad (1.1)$$

where $\mathbb{A}(\gamma; [0, T])$ denotes the action of the $C^1$ curve $\gamma: [0, T] \to M$ on the time interval $[0, T]$, that is,

$$\mathbb{A}(\gamma; [0, T]) := \int_{0}^{T} L(\gamma(t), \dot{\gamma}(t)) dt.$$

By the assumptions on $L$, the critical value $c[L]$ is necessarily finite, and satisfies

$$\inf_{(x,v) \in TM} L(x, v) \le -c[L] \le \inf_{x \in M} L(x, 0).$$

To each closed curve $\gamma \in C^1_{\text{per}}([0, T], M)$, we can associate a probability measure $\mu_{\gamma}$ on $TM$ by

$$\int_{TM} f d\mu_{\gamma} := \frac{1}{T} \int_{0}^{T} f(\gamma(t), \dot{\gamma}(t)) dt \quad \forall f \in C^{0}(TM, \mathbb{R}).$$

Following Mañé [35], we call holonomic probability measure any element in the set

$$\mathcal{H} := \overline{\{\mu_{\gamma} | T > 0, \gamma \in C^1_{\text{per}}([0, T], M)\}},$$

where the closure is taken with respect to the weak-* topology on the space of measures. Define the action functional

$$\mathbb{A}_L : \begin{array}{ccc} \mathcal{P}(TM) & \to & \mathbb{R} \cup \{+\infty\} \\ \mu & \mapsto & \mathbb{A}_L(\mu) := \int_{TM} L d\mu. \end{array}$$

By construction, we have

$$\inf \{ \mathbb{A}_L(\mu) | \mu \in \mathcal{H} \} = -c[L].$$

The set $\mathcal{H}$ is a (nonempty) closed convex subset of $\mathcal{P}(TM)$, which is not compact (with respect to the weak-* topology). However, thanks to (L1), the set $\mathcal{H}_0 := \mathcal{H} \cap \{\mathbb{A}_L \le -c(L) + 1\}$ is a compact convex subset of $\mathcal{H}$. This implies that $\mathbb{A}_L$ attains a minimum on $\mathcal{H}$, that is,

$$c[L] = - \min_{\mu \in \mathcal{H}} \{\mathbb{A}_L(\mu)\}.$$
The measures $\mu \in \mathcal{H}$ such that $A_L(\mu) = -c[L]$ are called *minimizing measures*. It can be shown that they are invariant under the Euler-Lagrange flow $\phi_t^L$ [35], and they minimize the functional $A_L$ among all Borel probability measures on $TM$ which are invariant under $\phi_t^L$.

The *Mather set* of $L$ is the nonempty compact subset of $TM$ defined as

$$ \tilde{\mathcal{M}}(L) := \bigcup_{A_L(\mu)=-c[L]} \text{Supp}(\mu), $$

and the projected Mather set $\mathcal{M}(L) \subset M$ is given by

$$ \mathcal{M}(L) := \pi(\tilde{\mathcal{M}}(L)). $$

In [37], Mather proved the following result:

**Mather's Graph Theorem I.** The set $\tilde{\mathcal{M}}(L) \subset TM$ is invariant under $\phi_t^L$. Moreover the map $\pi|_{\tilde{\mathcal{M}}(L)}: \tilde{\mathcal{M}}(L) \to M$ is injective, and

$$ (\pi|_{\tilde{\mathcal{M}}(L)})^{-1}: \mathcal{M}(L) \to \tilde{\mathcal{M}}(L) $$

is Lipschitz.

Following Mather [38], for every $T > 0$ we define the function $h_T: M \times M \to \mathbb{R}$ as

$$ h_T(x, y) := \inf \{ A(\gamma; [0, T]) \mid \gamma \in C^1([0, T], M), \gamma(0) = x, \gamma(T) = y \}. $$

The *Peierls barrier* associated with $L$ is the function $h: M \times M \to \mathbb{R}$ defined by

$$ h(x, y) := \liminf_{T \to +\infty} \{h_T(x, y) + c[L]T\}. $$

It is immediately seen that the following inequalities hold for all $T > 0$, for every $x, y, z \in M$:

$$ h(x, z) \le h(x, y) + h_T(y, z) + c[L]T, $$

$$ h(x, z) \le h_T(x, y) + c[L]T + h(y, z). $$

In particular, we deduce that the following “triangle inequality” holds:

$$ h(x, z) \le h(x, y) + h(y, z) \quad \forall x, y, z \in M. $$

By compactness of $M$ and (1.1), it is not difficult to prove that there is at least one point $x \in M$ such that $h(x, x) = 0$. Hence the above triangle inequality shows that $h$ is finite everywhere on $M \times M$. The projected Aubry set $\mathcal{A}(L)$ is then defined as the nonempty compact set given by

$$ \mathcal{A}(L) := \{x \in M \mid h(x, x) = 0\}. \qquad (1.2) $$

We observe that for every $x \in \mathcal{A}(L)$ there exist a sequence $\{T_k\}_{k \in \mathbb{N}}$ of real numbers tending to $+\infty$, and a sequence $\{\gamma_k\}_{k \in \mathbb{N}}$ of $C^1$ curves $\gamma_k: [0, T_k] \to M$, such that $\gamma_k(0) = \gamma_k(T_k)$ and

$$ \lim_{k \to \infty} A(\gamma_k; [0, T_k]) + c[L]T_k = 0. $$

Applying the Arzelà-Ascoli Theorem, it can be shown that the sequence $\{\tilde{\gamma}_k\}$ of curves $(\gamma_k, \tilde{\gamma}_k): [0, T_k] \to TM$ is relatively compact, so that for each integer $l > 0$ the sequence of curves

$$ t \in [-l, l] \mapsto \begin{cases} \tilde{\gamma}_k(t) & \text{if } t \ge 0 \\ \tilde{\gamma}_k(T_k + t) & \text{if } t < 0 \end{cases} $$
admits, up to a subsequence, a uniform limit. Then, one can show that such limit curve is uniquely determined [38], and deduce that to each $x \in \mathcal{A}(L)$ it can be associated in a unique way a $C^{k-1}$ curve $\gamma_x : \mathbb{R} \to M$, with $\gamma_x(0) = x$, which solves the Euler-Lagrange equation

$$ \frac{d}{dt} \left\{ \frac{\partial L}{\partial v}(\gamma_x(t), \dot{\gamma}_x(t)) \right\} = \frac{\partial L}{\partial x}(\gamma_x(t), \dot{\gamma}_x(t)) \quad \forall t \in \mathbb{R}. $$

Then, the *Aubry set* of $L$ is the compact subset of $TM$ defined by

$$ \tilde{\mathcal{A}}(L) := \left\{ (\gamma_x(t), \dot{\gamma}_x(t)) \mid x \in \mathcal{A}(L), t \in \mathbb{R} \right\}. $$

It can be proved that *Aubry set* $\tilde{\mathcal{A}}(L)$ contains the *Mather set* $\tilde{\mathcal{M}}(L)$. Moreover, in [38] Mather showed the following result:

**Mather's Graph Theorem II.** The set $\tilde{\mathcal{A}}(L) \subset TM$ is invariant under $\phi_t^L$. Moreover the map $\pi_{|\tilde{\mathcal{A}}(L)} : \tilde{\mathcal{A}}(L) \to M$ is injective, its image coincides with $\mathcal{A}(L)$, and

$$ (\pi_{|\tilde{\mathcal{A}}(L)})^{-1} : \mathcal{A}(L) \to \tilde{\mathcal{A}}(L) $$

is Lipschitz.

In other terms, Mather's Graph Theorems state that $\tilde{\mathcal{M}}(L) \subset \tilde{\mathcal{A}}(L)$ are contained in the graph of a Lipschitz section of $TM$.

## 1.2 Aubry-Mather theory from the Hamiltonian viewpoint

The *Tonelli Hamiltonian* $H: T^*M \to \mathbb{R}$ associated to $L$ by Legendre-Fenchel duality is defined as

$$ H(x, p) := \max_{v \in T_x M} \left\{ p(v) - L(x, v) \right\} \quad \forall (x, p) \in T_x^* M. $$

Thanks to our assumptions on $L$, it is well-known that $H$ is of class $C^k$ and satisfies both properties of superlinear growth and strict convexity in $T^*M$:

(H1) *Superlinear growth:* For every $K \ge 0$, there is a finite constant $C^*(K)$ such that

$$ H(x, p) \ge K \|p\|_x + C^*(K) \quad \forall (x, p) \in T^*M. $$

(H2) *Strict convexity:* For every $(x, p) \in T^*M$, the second derivative along the fibers $\frac{\partial^2 H}{\partial p^2}(x, p)$ is positive definite.

Under the above assumptions, the Hamiltonian flow $\phi_t^H$ of $H$ is of class $C^{k-1}$, and is conjugated with the Euler-Lagrange flow $\phi_t^L$ of $L$. The *critical value* or *Mañé critical value* of $H$ is defined as

$$ c[H] := c[L], \qquad (1.3) $$

while the *Aubry set* “seen in $T^*M$” is defined as

$$ \tilde{\mathcal{A}}(H) := \mathcal{L}(\tilde{\mathcal{A}}(L)), $$

where $\mathcal{L}: TM \to T^*M$ denotes the Legendre transform (see Appendix A). By construction $\tilde{\mathcal{A}}(H)$ is a nonempty compact subset of $T^*M$ which is invariant under $\phi_t^H$. In a series of papers [16, 17, 18], Fathi established a deep link between the concept of *Aubry sets* and the concept of viscosity solutions of the Hamilton-Jacobi associated with $H$, which we now describe.
A continuous function $u : M \to \mathbb{R}$ is called a viscosity subsolution of the Hamilton-Jacobi equation

$$H(x, du(x)) = c \quad \forall x \in M, \qquad (1.4)$$

if, for every $C^1$ function $\phi : M \to \mathbb{R}$ such that $\phi \le u$ and every $z \in M$, the following holds:

$$\phi(z) = u(z) \implies H(z, d\phi(z)) \le c.$$

This is equivalent to asking that

$$u(\gamma(b)) - u(\gamma(a)) \le \int_a^b L(\gamma(t), \dot{\gamma}(t)) dt + c(b-a) \qquad (1.5)$$

for every $C^1$ curve $\gamma : [a, b] \to M$.

A continuous function $u : M \to \mathbb{R}$ is called a viscosity solution of (1.4) if, for every $C^1$ function $\phi : M \to \mathbb{R}$ such that $\phi \le u$ and every $z \in M$, the following holds<fnref n="1" />:

$$\phi(z) = u(z) \implies H(z, d\phi(z)) = c.$$

As shown by Fathi, a continuous function $u : M \to \mathbb{R}$ is a viscosity solution of (1.4) if and only if it is a viscosity subsolution of (1.4) and, for each $x \in M$, there is a $C^{k-1}$ curve $\gamma_x : (-\infty, 0] \to M$ such that

$$u(x) - u(\gamma_x(-T)) = \int_{-T}^{0} L(\gamma_x(t), \dot{\gamma}_x(t)) dt + cT \quad \forall T \ge 0. \qquad (1.6)$$

In [16], Fathi proved the following result:

**Fathi's Weak KAM Theorem.** The critical Hamilton-Jacobi equation

$$H(x, du(x)) = c[H] \quad \forall x \in M \qquad (1.7)$$

admits at least one viscosity solution.

Let us recall that, by the compactness of $M$, $c[H]$ is the only value of $c$ for which the Hamilton-Jacobi equation (1.4) admits a viscosity solution. Indeed, if a continuous function $u : M \to \mathbb{R}$ is a viscosity subsolution of (1.4) for some $c \in \mathbb{R}$, then for every $C^1$ curve $\gamma : [0, T] \to M$ one has

$$-2\|u\|_{\infty} \le u(\gamma(T)) - u(\gamma(0)) \le \int_{0}^{T} L(\gamma(t), \dot{\gamma}(t)) dt + cT,$$

$$F(x, u(x), du(x)) = 0 \quad \forall x \in M,$$

if it satisfies the two following properties:

(i) (u is supersolution) For every $C^1$ function $\phi : M \to \mathbb{R}$ such that $\phi \le u$ and every $z \in M$, it holds

$$\phi(z) = u(z) \implies F(z, \phi(z), d\phi(z)) \ge c,$$

(ii) (u is subsolution) For every $C^1$ function $\phi : M \to \mathbb{R}$ such that $\phi \ge u$ and every $z \in M$, it holds

$$\phi(z) = u(z) \implies F(z, \phi(z), d\phi(z)) \le c,$$

Since $H$ is convex in the $p$ variable with bounded sublevel sets, the above definitions are equivalent to the one given in the paper.

<footnote n="1">¹We notice that the definitions of viscosity subsolution and viscosity solution given here are equivalent to the usual definitions: usually, a continuous function $u : M \to \mathbb{R}$ is called a viscosity solution of the first-order partial differential equation</footnote>
where $\|u\|_{\infty}$ denotes the supremum norm of $u$. Hence, letting $T \to +\infty$, (1.1) yields<fnref n="2" />

$$c \geq c[L] = c[H]. \qquad (1.8)$$

On the other hand, if $\gamma_x : (-\infty, 0] \to M$ is a $C^1$ curve such that (1.6) is satisfied and $\bar{u}$ is a viscosity solution of (1.7), then for every $T \ge 0$ we have

$$ \begin{aligned} \bar{u}(\gamma_x(0)) - \bar{u}(\gamma_x(-T)) &= \int_{-T}^{0} L(\gamma_x(t), \dot{\gamma}_x(t)) dt + c[H]T \\ &= u(\gamma_x(0)) - u(\gamma_x(-T)) + (c[H] - c)T. \end{aligned} $$

Hence, letting $T \to +\infty$ we get $c \le c[H]$, which together with (1.8) proves that $c = c[H]$, as desired. Incidentally, the above argument shows that $c[H]$ may also be viewed as the infimum of the values $c \in \mathbb{R}$ for which there exists a smooth function $u : M \to \mathbb{R}$ satisfying

$$H(x, du(x)) \le c \quad \forall x \in M$$

(see also [13]). In the sequel, we call *critical viscosity solution* (resp. *subsolution*) any continuous function $u : M \to \mathbb{R}$ which is a viscosity solution (resp. subsolution) of (1.7). If the solution (resp. subsolution) $u$ is indeed $C^1$, then we call it simply a *critical solution* (resp. *subsolution*). We mention that critical viscosity solutions are sometimes referred as *weak KAM solutions*.

As shown by Fathi and Siconolfi [25], every critical viscosity subsolution is differentiable on the projected Aubry set, and it can always be extended outside the projected Aubry set to a (strict) critical subsolution of class $C^1$:

**Fathi-Siconolfi's Theorem.** Let $u : M \to \mathbb{R}$ be a critical viscosity subsolution. Then $u$ is differentiable on the projected Aubry set and satisfies

$$(x, du(x)) \in \tilde{\mathcal{A}}(H) \quad \forall x \in \mathcal{A}(H).$$

Moreover, there is a critical subsolution $v : M \to \mathbb{R}$ of class $C^1$ which coincides with $u$ on $\mathcal{A}(H)$ and satisfies

$$H(x, dv(x))) < c[H] \quad \forall x \in M \setminus \mathcal{A}(H).$$

The above result combined with Mather's Theorem implies that the differential of any critical viscosity subsolution $u : M \to \mathbb{R}$ is Lipschitz on the projected Aubry set, does not depend on $u$, and satisfies $H(x, du(x)) = c[H]$ for every $x \in \mathcal{A}(H)$. In [7] Bernard improved the Fathi-Siconolfi's Theorem as follows (we refer the reader to [20, 46] for a survey on the Fathi-Siconolfi's and Bernard's Theorems):

**Bernard's Theorem.** If $u$ is a critical viscosity subsolution, then there exists a critical subsolution $v$ of class $C^{1,1}$ whose restriction to the projected Aubry set is equal to $u$.

The latter result is optimal: there are Hamiltonians which admit $C^{1,1}$ critical subsolutions but no $C^2$ critical subsolutions (see [20]).

$$c[L] := - \liminf_{T \to +\infty} \left\{ \frac{1}{T} A(\gamma; [0, T]) \mid \gamma \in C^1([0, T], M), \gamma(0) = \gamma(T) \right\}.$$

<footnote n="2">²We leave the reader to check that, by an easy concatenation procedure, $c[L]$ could also be defined as</footnote>
Another result on the regularity of viscosity (sub)solutions which will be used in the sequel is the following theorem of Fathi [19] (see also [44]):

**Fathi's C<sup>1,1</sup> Theorem.** Let $u$ be a critical viscosity subsolution, and assume that $u$ is a $C^1$ viscosity solution on some open set $V$. Then $u$ is (locally) $C^{1,1}$ inside $V$.

Several works have been devoted to the regularity of critical viscosity solutions [3, 6, 19, 44], to the structure of general Aubry sets [23, 39, 40, 48], or to the structure of generic Aubry sets [8, 9, 35, 36]. The purpose of the present paper is to take a first step toward a proof of the Mañé Conjecture in $C^2$ topology.

## 1.3 The Mañé Conjecture

Following Mañé [35], given a Tonelli Lagrangian $L: TM \to \mathbb{R}$ of class $C^k$ (with $k \ge 2$) and a potential $V: M \to \mathbb{R}$ of class $C^k$ (with $k \ge 2$), we define the Lagrangian $L_V: TM \to \mathbb{R}$ by

$$L_V(x, v) := L(x, v) - V(x) \quad \forall (x, v) \in TM.$$

Denote by $C^k(M)$ the set of $C^k$ potentials on $M$ equipped with the $C^k$ topology. The Mañé conjecture in $C^k$ topology (with $k \ge 2$) can be stated as follows:

**Mañé's Conjecture.** For every Tonelli Lagrangian $L: TM \to \mathbb{R}$ of class $C^k$ (with $k \ge 2$), there is a residual subset (i.e., a countable intersection of open and dense subsets) $\mathcal{G}$ of $C^k(M)$ such that, for every $V \in \mathcal{G}$, the Aubry set of the Lagrangian $L_V$ is either an equilibrium point or a periodic orbit.

Equivalently, if we denote by $H_V$ the Hamiltonian $H_V: T^*M \to \mathbb{R}$ associated with $L_V$, that is

$$H_V(x, p) = H(x, p) + V(x) \quad \forall (x, p) \in T^*M,$$

the Mañé Conjecture asserts that for generic potentials $V \in C^k(M)$ the set $\tilde{\mathcal{A}}(H_V)$ is either an equilibrium point or a periodic orbit.

The Mañé's Conjecture in smooth topology was solved positively by Massart [36] in the case of orientable closed surfaces. However, Massart made use of purely two-dimensional arguments which cannot be generalized to higher dimension.

A natural way to attack the Mañé Conjecture in any dimension would be to prove first a density result, then a stability result. Namely, given an Hamiltonian of class $C^k$ satisfying (H1) and (H2), first one could show that the set of potentials $V \in C^k(M)$ such that $\tilde{\mathcal{A}}(H_V)$ is either a hyperbolic equilibrium point or a hyperbolic periodic orbit is dense, and then prove that the latter property is open in $C^k$ topology. Since the stability part is contained in the results in [12] (see Section 7), we can consider that the Mañé Conjecture reduces to the density part:

**Mañé's density Conjecture.** For every Tonelli Lagrangian $L: TM \to \mathbb{R}$ of class $C^k$ (with $k \ge 2$) there exists a dense set $\mathcal{D}$ in $C^k(M)$ such that, for every $V \in \mathcal{D}$, the Aubry set of the Lagrangian $L_V$ is either an equilibrium point or a periodic orbit.

The aim of the present paper and [27] is to show that the approach, which was adopted (by Pugh [41, 42], Pugh and Robinson [43], and Mai [34]) to prove closing lemmas for dynamical systems and Hamiltonian vector fields, proves the Mañé density Conjecture in $C^1$ topology, and could be used to show the validity of the Mañé density Conjecture in $C^2$ topology. In the next section, we present our results.
## 2 Statement of the results

Our first goal is to show how to close an Aubry set in $C^2$ topology under the assumption that there exists a critical viscosity subsolution which is a $C^1$ (or equivalently $C^{1,1}$, by Fathi's Theorem) critical solution in an open neighborhood of a positive orbit of a recurrent point of the projected Aubry set, and which is $C^2$ at that point.

Let $x \in A(H)$, fix $u : M \to \mathbb{R}$ a critical viscosity subsolution, and denote by $O^+(x)$ the positive orbit of $x$ in the projected Aubry set, that is,

$$O^+(x) := \{ \pi^*(\phi_t^H(x, du(x))) | t \ge 0 \},$$

Note that, thanks to Mather's and Fathi-Siconolfi's Theorems, the positive orbit of any point of the projected Aubry set belongs to $A(H)$ and does not depend on $u$. Moreover, if a point $x \in A(H)$ does not belong to the projection of a periodic orbit of $\tilde{A}(H)$, it is well-known that its positive orbit $O^+(x)$ cannot be closed. A point $x \in A(H)$ is called recurrent if there exists a sequence of times $t_k \to +\infty$ such that

$$\lim_{k \to \infty} \pi^*(\phi_{t_k}^H(x, du(x))) = x,$$

where $u : M \to \mathbb{R}$ is again any critical viscosity subsolution. As before, the above definition does not depend on $u$.

We now formalize the concept of a $C^{1,1}$ function being $C^2$ at one point. Let $v : V \to \mathbb{R}$ be a function of class $C^{1,1}$ in an open set $V \subset M$. Thanks to Rademacher's Theorem, its differential $dv$ is differentiable almost everywhere in $M$. Let $\text{Dom}(\text{Hess}^g v) \subset V$ be the set of points where $dv$ is differentiable. Then, for every $x \in \text{Dom}(\text{Hess}^g v)$, the function $v$ is two times differentiable at $x$, and its Hessian with respect to the metric $g$ is the symmetric bilinear form on $T_x M$ defined as

$$\text{Hess}^g v(x)[\xi, \eta] := \langle (\nabla^g \xi dv)(x), \eta \rangle \quad \forall \xi, \eta \in T_x M,$$

where $\nabla^g$ denotes the covariant derivative with respect to $g$ (see [47]). We call generalized Hessian of $v$ at $x \in V$ the set of symmetric bilinear form on $T_x M$ defined by

$$\mathcal{H}ess^g v(x) := \text{conv} \left( \left\{ \lim_{k \to \infty} \text{Hess}^g v(x_k) \mid x_k \to x, x_k \in \text{Dom}(\text{Hess}^g v) \right\} \right),$$

where $\text{conv}$ denotes the convex envelope, and the limit is taken in the fiber bundle of symmetric bilinear forms on the fibers of $TM$. By construction, $\mathcal{H}ess^g v(x)$ is a nonempty compact convex set of symmetric bilinear forms on $T_x M$ for any $x \in M$. Then, the informal sentence “$v$ is $C^2$ at a point $x$” that we used before in the introduction, means that $\mathcal{H}ess^g v(x)$ is a singleton. (This definition is motivated by the fact that a $C^{1,1}$ function is $C^2$ on an open set $V$ if and only if its generalized Hessian is a singleton at every point of $V$.)

Recall that, by Fathi's $C^{1,1}$ Theorem (see Subsection 1.2), $C^1$ viscosity solutions are $C^{1,1}$. So it makes sense to talk about their generalized Hessian. Our first result is the following:

**Theorem 2.1.** Let $H : T^*M \to \mathbb{R}$ be a Tonelli Hamiltonian of class $C^k$ with $k \ge 2$, and fix $\epsilon > 0$. Assume that there are a recurrent point $\bar{x} \in A(H)$, a critical viscosity subsolution $u : M \to \mathbb{R}$, and an open neighborhood $V$ of $O^+(\bar{x})$ such that the following properties are satisfied:

(i) $u$ is of class $C^1$ in $V$;

(ii) $H(x, du(x)) = c[H]$ for every $x \in V$;
(iii) $Hess^g u(\bar{x})$ is a singleton.

Then there exists a potential $V: M \to \mathbb{R}$ of class $C^k$, with $\|V\|_{C^2} < \epsilon$, such that $c[H_V] = c[H]$ and the Aubry set of $H_V$ is either an equilibrium point or a periodic orbit.

In the above theorem, the generalized Hessian of $u$ at $\bar{x}$ depends upon the Riemannian metric $g$. However, it is worth noticing that assumption (iii) does not depend on the metric $g$. Such an assumption is motivated by some recent results of Arnaud [3, 4, 5]. Let us also point out that, since the graph of $du$ is invariant under the Hamiltonian flow in $V$, assumption (iii) implies that $Hess_x^g u$ is a singleton for any $x \in O^+(\bar{x})$.

We note that since the Mather set is a compact set invariant under the Lagrangian flow, it necessarily contains recurrent points. (Indeed, given any minimal invariant subset of $M(L)$, minimality implies that all orbits are dense in such a subset.) Thus, the following result is a straightforward corollary of Theorem 2.1:

**Corollary 2.2.** Let $H: T^*M \to \mathbb{R}$ be a Tonelli Hamiltonian of class $C^k$ with $k \ge 2$, and fix $\epsilon > 0$. Assume that there is a critical viscosity solution which is of class $C^2$ in a neighborhood of $M(L)$. Then there exists a potential $V: M \to \mathbb{R}$ of class $C^k$, with $\|V\|_{C^2} < \epsilon$, such that $c[H_V] = c[H]$ and the Aubry set of $H_V$ is either an equilibrium point or a periodic orbit.

This result applies to the case of Mañé Lagrangians: given $X$ a $C^k$-vector field on $M$ with $k \ge 2$, the Mañé Lagrangian $L_X: TM \to \mathbb{R}$ associated to $X$ is defined by

$$L_X(x, v) := \frac{1}{2} \|v - X(x)\|_x^2 \quad \forall (x, v) \in TM,$$

while the Mañé Hamiltonian $H_X: TM \to \mathbb{R}$ is given by

$$H_X(x, p) = \frac{1}{2} \|p\|_x^2 + \langle p, X(x) \rangle \quad \forall (x, p) \in T^*M.$$

Since $L_X \ge 0$ and $u \equiv 0$ is solution of the Hamilton-Jacobi equation

$$H_X(x, du(x)) = 0 \quad \forall x \in M,$$

by the discussion in Subsections 1.1 and 1.2 we deduce that $c[H_X] = 0$ and $u \equiv 0$ is a critical solution for $H_X$. Then Theorem 2.1 yields the following closing-type result:

**Corollary 2.3.** Let $X$ be a vector field on $M$ of class $C^k$ with $k \ge 2$. Then for every $\epsilon > 0$ there is a potential $V: M \to \mathbb{R}$ of class $C^k$, with $\|V\|_{C^2} < \epsilon$, such that the Aubry set of $H_X + V$ is either an equilibrium point or a periodic orbit.

In the present paper we prove the following variant of Theorem 2.1 in the case of surfaces, leaving to [27] the (nontrivial) extension to arbitrary dimension:

**Theorem 2.4.** Assume that $\dim M = 2$, let $H: T^*M \to \mathbb{R}$ be a Tonelli Hamiltonian of class $C^k$ with $k \ge 2$, and fix $\epsilon > 0$. Assume that there are a recurrent point $\bar{x} \in A(H)$, a critical viscosity subsolution $u: M \to \mathbb{R}$, and an open neighborhood $V$ of $O^+(\bar{x})$, such that $u$ is at least of class $C^{k+1}$ on $V$. Then there exists a potential $V: M \to \mathbb{R}$ of class $C^k$, with $\|V\|_{C^2} < \epsilon$, such that $c[H_V] = c[H]$ and the Aubry set of $H_V$ is either an equilibrium point or a periodic orbit.

In analogy with Theorem 2.1, one could check that the above result is still true when replacing $C^3$ with $C^{2,1} + C^3$ at the point”. To achieve this, some minor modifications in the proof would be needed. However, since we did not see any big improvement in stating the result in this sharper form, we have preferred to state it under this more “conventional” assumptions.

The extension of Theorem 2.4 to arbitrary dimension will be performed in [27, Theorem 1.1], where we will need some refined versions of the results presented here. Moreover, the
combinations of some of the techniques and ideas introduced here and in the proof of [27, Theorem 1.1] will allow us to show the validity of the Mañé's density conjecture in $C^1$ topology (i.e., for every $\epsilon > 0$ there exists a potential $V : M \to \mathbb{R}$ of class $C^2$ such that $\|V\|_{C^1} < \epsilon$, $c[H_V] = c[H]$, and the Aubry set of $H_V$ is either an equilibrium point or a periodic orbit, see [27, Theorem 1.2]).

The proofs of both Theorems 2.1 and 2.4 involve techniques from finite dimensional control theory, together with ideas coming from the proof of the classical closing lemma [41, 42, 43, 34].

Let us point out that the assumptions of Theorem 2.1 have no reason to be satisfied for general Hamiltonians. This motivated us to introduce Theorem 2.4 (and then to extend Theorem 2.4 to any dimension in [27, Theorem 1.1]). Indeed, even if, in general, critical subsolutions are at most $C^{1,1}$ (see the discussion after the statement of Bernard's Theorem), it may be possible to prove the generic existence of smooth critical viscosity subsolutions (at least in a neighborhood of a positive orbit). We plan to address this question in a future work.

As we will see, Theorem 2.4 is proved from Theorem 2.1 by “locally transforming” a critical subsolution into a critical solution for a different Hamiltonian (see Section 6). Although this may look a “cheap trick”, the proof is still very involved. Moreover, at this moment we do not see how to adapt the construction used in the proof of Theorem 2.1 to address directly the case of subsolution (without passing to the case of solutions).

Let us now briefly explain the difficulties behind the proof of Theorem 2.1, and the strategy to bypass them. Since $\bar{x}$ is recurrent, the curve $t \mapsto \pi^*(\phi_t^H(\bar{x}, du(\bar{x})))$ passes near $\bar{x}$ infinitely many times. Then, the rough idea would be to choose a time $T \gg 1$ such that $\bar{x}_T := \pi^*(\phi_T^H(\bar{x}, du(\bar{x})))$ is sufficiently close to $\bar{x}$, and then try to “close” the trajectory in one step. There are many points to address here:

1) It is not possible to close the trajectory in one step by adding a potential small in $C^2$-norm: indeed, if we add a potential $V$ small in $C^2$ topology, the Hamiltonian vector field associated to $H_V$ is close in $C^1$ topology to the Hamiltonian vector field of $H$. However, if one wants to close the orbit in only one step, then $\nabla V$ can be small only in $C^0$ topology, due to the fact that the potential $V$ has to be supported in a small neighborhood of the orbit in order not to intersect with the curve $t \mapsto \pi^*(\phi_t^H(\bar{x}, du(\bar{x})))$ for $t \in [0, T]^3$. Hence, to close the trajectory we will use Mai Lemma D.1: roughly speaking, fixed an error size $\epsilon > 0$ and a small radius $r$ which “ideally represents” the distance between $\bar{x}$ and $\bar{x}_T$, the idea is to close the trajectory in $1/\epsilon$ steps where at each step we “move” $\bar{x}_T$ in the direction of $\bar{x}$ by a size $\epsilon r$. (Actually the strategy is much more involved, as we have to take care that the modification we do at every “approaching step” does not influence the modifications done before, and moreover does not “destroy” the property of $\bar{x}$ of being recurrent, see Subsection 5.3.)

In order to perform the strategy described above, we need to be able to go from one point to another by adding a small potential. To this aim, using techniques and results from control theory, in Section 3 we prove a general result which allows to connect points by Hamiltonian trajectories.

2) Point 1 above deals with the “closing part of our statement”, i.e., finding a closed orbit for $H_V$. However, we still need this new orbit to belong to the Aubry set of the new Hamiltonian. In order to do this, we have first to control the action of the Lagrangian $L_V$ along this closed trajectory (see Subsection 5.4) and then to construct a suitable global critical subsolution which will allow us to deduce that the curve belongs to the projected Aubry set (Subsection 5.5). The first part will need again a general “control theory” result proved in Section 4.

<footnote n="3">³This is the analogous of the classical “closing lemma”: fixed $k \ge 0$, one asks whether, given a vector field $X$ with a recurrent point $\bar{x}$, one can find a vector field $Y$ close to $X$ in $C^k$ topology which has a periodic orbit. The “cheap strategy” of closing the trajectory in one step proves that the closing lemma is true when $k = 0$, while for $k = 1$ new deep ideas have been introduced to solve the problem [41, 42, 43, 34]. Let us recall that the problem for $k \ge 2$ is still open, though many results suggest it may be false when $k$ is sufficiently large (or that at least there is no possibility to prove such a result by means of “local techniques”, see [28, 31, 32]), unless some additional assumptions are made [29, 30, 33].</footnote>
The combination of Points 1 and 2 will conclude the proof of Theorem 2.1.

The paper is organized as follows: in Sections 3 and 4, using techniques from finite dimensional control theory, we prove connecting results for Hamiltonian trajectories by adding potentials, where we further control the Lagrangian action of the trajectories. It is important to point out that these results (which are essential for the proof of Theorem 2.1) are very general, and they may be useful for other applications. The proofs of our two theorems are given in Sections 5 and 6. In Section 7, we will make some final comments on our results and the Mañé conjecture.

Finally, there are five short appendices that contain either technical results or auxiliary results, like some tools of control theory, and the exact statement of Mai Lemma which plays a crucial role in our proofs.

*Acknowledgements:* The authors are grateful to Didier Auroux, Patrick Bernard, Bernard Bonnard, Jean-Michel Coron, Albert Fathi, Gilles Lebeau, Jean-Baptiste Pomet, Rafael Ruggiero, Emmanuel Trélat, and Constantin Vernicos for enlightening discussions.

# 3 Connecting Hamiltonian orbits by potentials

## 3.1 Statement of the result

Let $n \ge 2$ be fixed. We denote a point $x \in \mathbb{R}^n$ either as $x = (x_1, \dots, x_n)$ or in the form $x = (x_1, \hat{x})$, where $\hat{x} = (x_2, \dots, x_n) \in \mathbb{R}^{n-1}$. Let $\bar{H} : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ be a Hamiltonian$^4$ of class $C^k$, with $k \ge 2$, satisfying (H1), (H2) and the additional hypothesis

(H3) Uniform boundedness in the fibers: For every $R \ge 0$ we have

$$A^*(R) := \sup\{\bar{H}(x, p) \mid |p| \le R\} < +\infty.$$

Note that, under these assumptions, the Hamiltonian $\bar{H}$ generates a flow $\phi_t^{\bar{H}}$ which is of class $C^{k-1}$ and complete (see [24, corollary 2.2]). Let $\bar{\tau} \in (0, 1)$ be fixed. We suppose that there exists a solution

$$(\bar{x}(\cdot), \bar{p}(\cdot)) : [0, \bar{\tau}] \to \mathbb{R}^n \times \mathbb{R}^n$$

of the Hamiltonian system

$$\left\{ \begin{array}{l} \dot{\bar{x}}(t) = \nabla_p \bar{H}(\bar{x}(t), \bar{p}(t)) \\ \dot{\bar{p}}(t) = -\nabla_x \bar{H}(\bar{x}(t), \bar{p}(t)) \end{array} \right. \qquad (3.1)$$

on $[0, \bar{\tau}]$ satisfying the following conditions$^5$:

$$(A1) \quad \bar{x}^0 = (0, \hat{\bar{x}}^0) := \bar{x}(0) = 0_n \text{ and } \dot{\bar{x}}(0) = e_1;$$

$$(A2) \quad \bar{x}^{\bar{\tau}} = (\bar{\tau}, \hat{\bar{x}}^{\bar{\tau}}) := \bar{x}(\bar{\tau}) = (\bar{\tau}, 0_{n-1}) \text{ and } \dot{\bar{x}}(\bar{\tau}) = e_1;$$

$$(A3) \quad |\dot{\bar{x}}(t) - e_1| < 1/2 \text{ for any } t \in [0, \bar{\tau}].$$

<footnote>$^4$Note that we identify $T^*(\mathbb{R}^n)$ with $\mathbb{R}^n \times \mathbb{R}^n$. For that reason, throughout Section 3 the adjoint variable $p$ will always be seen as a vector in $\mathbb{R}^n$.</footnote>

<footnote>$^5$The purpose of this section is to prove connecting results which can be applied to connect Hamiltonian trajectories associated with Hamiltonians $H: T^*M \to \mathbb{R}$ of class at least $C^2$. Let us remark that any local Hamiltonian trajectory of a Hamiltonian $H: T^*M \to \mathbb{R}$ of class $C^2$ can be sent via a local diffeomorphism of class $C^\infty$ (from an open set of $M$ to an open subset of $\mathbb{R}^n$) to a Hamiltonian trajectory of the form $(\bar{x}(\cdot), \bar{p}(\cdot))$ in $\mathbb{R}^n \times \mathbb{R}^n$ satisfying (A1)-(A3) and associated with a Hamiltonian $\bar{H}: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ of class $C^2$. We note however that, whenever $H$ is merely $C^2$, we cannot assume that $(x(\cdot), p(\cdot))$ in $\mathbb{R}^n \times \mathbb{R}^n$ satisfies $x(t) = (t, 0_{n-1})$ $\forall t \in [0, \bar{\tau}]$ up to a smooth change of coordinates.</footnote>
For every $(x^0, p^0) \in \mathbb{R}^n \times \mathbb{R}^n$ satisfying $\bar{H}(x^0, p^0) = 0$, we denote by

$$ (X(\cdot; (x^0, p^0)), P(\cdot; (x^0, p^0))) : [0, +\infty) \to \mathbb{R}^n \times \mathbb{R}^n $$

the solution of the Hamiltonian system

$$ \left\{ \begin{aligned} \dot{x}(t) &= \nabla_p \bar{H}(x(t), p(t)) \\ \dot{p}(t) &= -\nabla_x \bar{H}(x(t), p(t)) \end{aligned} \right. \qquad (3.2) $$

satisfying

$$ x(0) = x^0 \quad \text{and} \quad p(0) = p^0. \qquad (3.3) $$

Since the curve $\bar{x}(\cdot)$ is transverse to the hyperplane $\Pi^\tau := \{x = (\bar{\tau}, \hat{x}) \in \mathbb{R}^n\}$ at time $\bar{\tau}$, there is a neighborhood $\mathcal{V}^0$ of $(\bar{x}^0, \bar{p}^0) := \bar{p}(0)$ in $\mathbb{R}^n \times \mathbb{R}^n$ such that the Poincaré mapping $\tau : \mathcal{V}^0 \to \mathbb{R}$ with respect to the section $\Pi^\tau$ is well-defined. That is, it is $C^{k-1}$ and satisfies

$$ \tau(\bar{x}^0, \bar{p}^0) = \bar{\tau} \quad \text{and} \quad X_1(\tau(x^0, p^0); (x^0, p^0)) = \bar{\tau} \quad \forall (x^0, p^0) \in \mathcal{V}^0. \qquad (3.4) $$

Our aim is to show that, given a point $(x^0 = (0, \hat{x}^0), p^0)$ such that $\bar{H}(x^0, p^0) = 0$ and sufficiently close to $(\bar{x}^0, \bar{p}^0)$, and chosen a point $(x^f = (\bar{\tau}, \hat{x}^f), p^f)$ satisfying $\bar{H}(x^f, p^f) = 0$ and sufficiently close to the final state

$$ (X(\tau(x^0, p^0); (x^0, p^0)), P(\tau(x^0, p^0); (x^0, p^0))) $$

there exists a time $T^f$ close to $\tau(x^0, p^0)$, together with a potential $V : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ whose support and $C^2$-norm<fnref n="6" /> are controlled, such that the solution $(x(\cdot), p(\cdot)) : [0, T^f] \to \mathbb{R}^n \times \mathbb{R}^n$ of the Hamiltonian system<fnref n="7" />

$$ \left\{ \begin{aligned} \dot{x}(t) &= \nabla_p \bar{H}_V(x(t), p(t)) = \nabla_p \bar{H}(x(t), p(t)) \\ \dot{p}(t) &= -\nabla_x \bar{H}_V(x(t), p(t)) = -\nabla_x \bar{H}(x(t), p(t)) - \nabla V(x(t)) \end{aligned} \right. \qquad (3.5) $$

starting at $(x(0), p(0)) = (x^0, p^0)$ satisfies $(x(T^f), p(T^f)) = (x^f, p^f)$. Since we also want to estimate the action of the new “connecting” Hamiltonian trajectory, we introduce some more notation.

We denote by $\bar{L}_V : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ the Lagrangian associated to $\bar{H}_V$ by Legendre-Fenchel duality, i.e.,

$$ \bar{L}_V(x, v) = \bar{L}(x, v) - V(x) \quad \forall (x, v) \in \mathbb{R}^n \times \mathbb{R}^n, $$

where $\bar{L}$ is the Lagrangian associated to $\bar{H}$. For every $(x^0, p^0) \in \mathbb{R}^n \times \mathbb{R}^n$, $T > 0$, and every $C^2$ potential $V : \mathbb{R}^n \to \mathbb{R}$, we denote by $A_V((x^0, p^0); T)$ the action of the curve $\gamma : [0, T] \to \mathbb{R}^n$ defined as the projection (onto the $x$ variable) of the Hamiltonian trajectory $t \mapsto \phi_t^{\bar{H}_V}(x^0, p^0)$, that is,

$$ A_V((x^0, p^0); T) := \int_0^T \bar{L}_V \left( \pi^* \left( \phi_t^{\bar{H}_V}(x^0, p^0) \right), \frac{d}{dt} \left( \pi^* \left( \phi_t^{\bar{H}_V}(x^0, p^0) \right) \right) \right) dt \quad (3.6) $$

$$ = \int_0^T \bar{L} \left( \pi^* \left( \phi_t^{\bar{H}_V}(x^0, p^0) \right), \frac{d}{dt} \left( \pi^* \left( \phi_t^{\bar{H}_V}(x^0, p^0) \right) \right) \right) \quad (3.7) $$

$$ -V \left( \pi^* \left( \phi_t^{\bar{H}_V}(x^0, p^0) \right) \right) dt. $$

$$ \|V\|_{C^2} := \|V\|_{\infty} + \|\nabla V\|_{\infty} + \|\text{Hess } V\|_{\infty}, $$

$$ \bar{H}_V(x, p) := \bar{H}(x, p) + V(x) \quad \forall (x, p) \in \mathbb{R}^n \times \mathbb{R}^n. $$

<footnote n="6">⁶Recall that the $C^2$-norm of a compactly supported $C^2$ function $V : \mathbb{R}^n \to \mathbb{R}$ is defined as</footnote>

<footnote>where $\| \cdot \|_{\infty}$ denotes the supremum norm. For $C^{1,1}$ function, the definition of the $C^{1,1}$-norm is the same just replacing the sup norm of the Hessian with the esssup (since the Hessian is only defined a.e.).</footnote>

<footnote n="7">⁷As in Section 1, we define $\bar{H}_V : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ by</footnote>
Note that, when $V = 0$, we have

$$A((x^0, p^0); T) := A_0((x^0, p^0); T) = \int_0^T \bar{L}(X(t; (x^0, p^0)), \dot{X}(t; (x^0, p^0))) dt.$$

We are now ready to state our result:

**Proposition 3.1.** Let $\bar{H} : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ be a Hamiltonian of class $C^k$ with $k \ge 2$ satisfying (H1)-(H3), and let $(\bar{x}(\cdot), \bar{p}(\cdot)) : [0, \bar{\tau}] \to \mathbb{R}^n \times \mathbb{R}^n$ be a solution of (3.1) with $\bar{H}(\bar{x}^0, \bar{p}^0) = 0$ and satisfying (A1)-(A3). Then there are $\bar{\delta}, \bar{r}, \bar{\epsilon} \in (0, 1)$ with $B^{2n}((\bar{x}^0, \bar{p}^0), \bar{\delta}) \subset \mathcal{V}^0$, and $K > 0$, such that the following property holds: For every $r \in (0, \bar{r})$, $\epsilon \in (0, \bar{\epsilon})$, and every $x^0 = (0, \hat{x}^0), p^0, x^f = (\bar{\tau}, \hat{x}^f), p^f \in \mathbb{R}^n$ satisfying

$$|\hat{x}^0|, |p^0 - \bar{p}^0| < \bar{\delta}, \qquad (3.8)$$

$$|(\bar{\tau}, \hat{x}^f) - X(\tau(x^0, p^0); (x^0, p^0))|, |p^f - P(\tau(x^0, p^0); (x^0, p^0))| < r\epsilon, \qquad (3.9)$$

$$\bar{H}(x^0, p^0) = \bar{H}(x^f, p^f) = 0, \qquad (3.10)$$

there exist a time $T^f > 0$ and a potential $V : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ such that:

$$(i) \text{Supp}(V) \subset \mathcal{C}((x^0, p^0); \tau(x^0, p^0); r);$$

$$(ii) \quad \|V\|_{C^2} < K\epsilon;$$

$$(iii) \quad |T^f - \tau(x^0, p^0)| < Kr\epsilon;$$

$$(iv) \quad \phi_{T^f}^{\bar{H}_V}(x^0, p^0) = (x^f, p^f);$$

$$(v) \quad |\bar{A}_V((x^0, p^0); T^f) - \bar{A}((x^0, p^0); \tau(x^0, p^0)) - \Delta((x^0, p^0); \tau(x^0, p^0); x^f)| < Kr^2\epsilon^2.$$

Here $\mathcal{C}((x^0, p^0); \tau(x^0, p^0); r)$ is the “cylinder” defined as

$$\mathcal{C}((x^0, p^0); \tau(x^0, p^0); r) := \{X(t; (x^0, p^0)) + (0, \hat{y}) \mid t \in [0, \tau(x^0, p^0)], |\hat{y}| < r\}, \qquad (3.11)$$

and

$$\Delta((x^0, p^0); \tau(x^0, p^0); x^f) := \langle P(\tau(x^0, p^0); (x^0, p^0)), x^f - X(\tau(x^0, p^0); (x^0, p^0)) \rangle. \qquad (3.12)$$

## 3.2 Proof of Proposition 3.1

Given $x^0 = (0, \hat{x}^0), p^0, x^f = (\bar{\tau}, \hat{x}^f), p^f$ such that (3.8)-(3.10) are satisfied, we are going to show the existence of a time $T^f > 0$ and a function $v : [0, T^f] \to \mathbb{R}^n$ of class $C^{k-1}$ such that the solution to the system

$$\left\{ \begin{array}{l} \dot{x}(t) = \nabla_p \bar{H}(x(t), p(t)) \\ \dot{p}(t) = -\nabla_x \bar{H}(x(t), p(t)) - v(t), \end{array} \right. \qquad (3.13)$$

starting at $(x^0, p^0)$, satisfies $(x(T^f), p(T^f)) = (x^f, p^f)$. In this way, if we can find a function $V : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ such that $\nabla V(x(t)) = v(t)$ for all $t \in [0, T^f]$, then the solution of the Hamiltonian system (3.5), starting at $(x^0, p^0)$, will satisfy (iv). By suitably estimating the $C^1$-norm of $v$ and by constructing $V$ carefully, we will also ensure that all the other properties
<figure><img src="image_2.png" /><figcaption>Figure 1: By adding a potential V, small in $C^2$ topology and supported inside the “cylinder” $C((x^0, p^0); \tau(x^0, p^0); r)$, we can connect a point $x^0$ to any point $x^f = (\bar{\tau}, \hat{x}^f)$ such that $|x^f - X(\tau(x^0, p^0); (x^0, p^0))| < r\epsilon$.</figcaption></figure>

are satisfied.

Since the Hamiltonian is preserved along the flow, we will work in the hypersurface

$$ \{(x, p) | \bar{H}(x, p) = 0\} \subset \mathbb{R}^n \times \mathbb{R}^n. $$

For every $p \in \mathbb{R}^n$, denote by $\hat{p}$ the $n-1$ last coordinates of $p$, that is the element $\hat{p} \in \mathbb{R}^{n-1}$ such that $p = (p_1, \hat{p})$. (We use the same convention as for $x, y \in \mathbb{R}^n$.) By (A3) and the Implicit Function Theorem, there is a bounded open neighborhood $\mathcal{W}$ of the set

$$ \{(\bar{x}(t), \bar{p}(t)) \mid t \in [0, \bar{\tau}]\} \subset \mathbb{R}^n \times \mathbb{R}^n, $$

a bounded open neighborhood $\hat{\mathcal{W}}$ of the set

$$ \{(\bar{x}(t), \hat{p}(t)) \mid t \in [0, \bar{\tau}]\} \subset \mathbb{R}^n \times \mathbb{R}^{n-1}, $$

and a function $\varphi : \hat{\mathcal{W}} \to \mathbb{R}$ of class $C^k$ such that

$$ \left\{ \begin{array}{l} \forall (x, p) \in \mathcal{W} : \bar{H}(x, p) = 0 \implies p_1 = \varphi(x, \hat{p}); \\ \forall (x, q) \in \hat{\mathcal{W}} : (x, (\varphi(x, q), q)) \in \mathcal{W} \text{ and } \bar{H}(x, (\varphi(x, q), q)) = 0. \end{array} \right. \qquad (3.14) $$

Define the $C^k$ function $\psi : \hat{\mathcal{W}} \to \mathbb{R}^n$ by

$$ \psi(x, q) := (\varphi(x, q), q). \qquad (3.15) $$
Then, any solution $(x(t), q(t)) \in \mathbb{R}^n \times \mathbb{R}^{n-1}$ of

$$ egin{cases} \dot{x}(t) = \nabla_p \bar{H}(x(t), \psi(x(t), q(t))) \\ \dot{q}(t) = -\nabla_{\dot{\psi}} \bar{H}(x(t), \psi(x(t), q(t))) - u(t). \end{cases} \qquad (3.16) $$

induces a unique solution $(x(t), \psi(x(t), q(t))) \in \mathbb{R}^n \times \mathbb{R}^n$ of (3.13), as proved in the following lemma:

**Lemma 3.2.** Let $(x(\cdot), q(\cdot)) : [0, T] \to \hat{W}$ be a solution of (3.16) on $[0, T]$ starting at $(x^0, q^0)$ and associated with a control $u : [0, T] \to \mathbb{R}^{n-1}$ of class $C^\infty$. Then, the extended trajectory $(x(\cdot), p(\cdot)) : [0, T] \to W$ defined by

$$ p(t) := \psi(x(t), q(t)) \quad \forall t \in [0, T] \qquad (3.17) $$

is the unique solution of the Hamiltonian system (3.13) starting at $(x^0, p^0 := (\varphi(x^0, q^0), q^0))$ and associated with the control $v = (v_1, u) : [0, T] \to \mathbb{R}^n$ of class $C^{k-1}$ defined by

$$ v_1(t) := - \left( \frac{\partial \bar{H}}{\partial p_1}(x(t), \psi(x(t), \hat{p}(t))) \right)^{-1} \langle u(t), \nabla_{\hat{p}} \bar{H}(x(t), \psi(x(t), \hat{p}(t))) \rangle. \qquad (3.18) $$

In particular, $\langle v(t), \dot{x}(t) \rangle = 0$ for all $t \in [0, T]$.

*Proof of Lemma 3.2.* It is sufficient to show that $\dot{p}_1(t)$ is given by

$$ \dot{p}_1(t) = - \frac{\partial \bar{H}}{\partial x_1}(x(t), \psi(x(t), \hat{p}(t))) - v_1(t), $$

with $v_1$ as in (3.18). Differentiating (3.17) with respect to $t$ we get

$$ \dot{p}_1(t) = \langle \nabla_x \varphi(x(t), q(t)), \dot{x}(t) \rangle + \langle \nabla_q \varphi(x(t), q(t)), \dot{q}(t) \rangle $$

for all $t \in [0, T]$. Moreover, differentiating the equality $\bar{H}(x, (\varphi(x, q), q)) = 0$ (given by (3.14)) with respect to both $x$ and $q$ gives

$$ \left\{ \begin{array}{l} \nabla_x \varphi(x, q) = - \left( \frac{\partial \bar{H}}{\partial p_1}(x, \psi(x, q)) \right)^{-1} \nabla_x \bar{H}(x, \psi(x, q)), \\ \nabla_q \varphi(x, q) = - \left( \frac{\partial \bar{H}}{\partial p_1}(x, \psi(x, q)) \right)^{-1} \nabla_{\hat{p}} \bar{H}(x, \psi(x, q)). \end{array} \right. $$

We conclude easily. □

Restricting $\mathcal{V}^0$ if necessary, we can assume that there is $\bar{\mu} > 0$ such that, for any starting point

$$ (x^0 = (0, \hat{x}^0), q^0) \in \hat{W}^0 := \{(x, q) | (x, \psi(x, q)) \in \mathcal{V}^0\}, \qquad (3.19) $$

any time $T \in (\bar{\tau} - \bar{\mu}, \bar{\tau} + \bar{\mu})$, and any control $u : [0, T] \to \mathbb{R}^{n-1}$ of class $C^\infty$ with $\|u\|_{C^1} < \bar{\mu}$, the solution

$$ (X_{(x^0, q^0)}^u(\cdot), Q_{(x^0, q^0)}^u(\cdot)) : [0, T] \to \mathbb{R}^n \times \mathbb{R}^{n-1} $$

of (3.16) starting at $(x^0, q^0)$ satisfies

$$ (X_{(x^0, q^0)}^u(t), Q_{(x^0, q^0)}^u(t)) \in \hat{W} \quad \forall t \in [0, T]. \qquad (3.20) $$

Define the mapping

$$ E^{(x^0, q^0), T} : \begin{array}{rcl} C^\infty([0, T]; \mathbb{R}^{n-1}) & \longrightarrow & \mathbb{R}^n \times \mathbb{R}^{n-1} \\ u & \longmapsto & (X_{(x^0, q^0)}^u(T), Q_{(x^0, q^0)}^u(T)). \end{array} \qquad (3.21) $$
Given $\delta, r, \epsilon > 0$ small enough (the smallness to be chosen later) and points $(x^0 = (0, \hat{x}^0), q^0), (x^f = (\bar{\tau}, \hat{x}^f), q^f)$ satisfying

$$|\hat{x}^0|, |q^0 - \hat{p}^0| < \delta, \qquad (3.22)$$

and

$$\left\{ \begin{array}{l} |(\bar{\tau}, \hat{x}^f) - X(\tau(x^0, \psi(x^0, q^0)); (x^0, \psi(x^0, q^0)))| < r\epsilon, \\ |q^f - \hat{P}(\tau(x^0, \psi(x^0, q^0)); (x^0, \psi(x^0, q^0)))| < C_\varphi r\epsilon, \end{array} \right. \qquad (3.23)$$

for some universal constant $C_\varphi$ depending only on $\varphi$, we want to find $T^f \in (\bar{\tau} - \bar{\mu}, \bar{\tau} + \bar{\mu})$ and a control $u: [0, T^f] \to \mathbb{R}^{n-1}$ of class $C^\infty$ such that

$$E^{(x^0, q^0), T^f}(u) = (x^f, q^f), \quad \text{with a bound on the } C^1\text{-norm of } u.$$

We will apply the controllability results which are given in Appendix B.

Consider the following nonlinear control system in $\mathbb{R}^n \times \mathbb{R}^{n-1}$:

$$\dot{\xi} = F_0(\xi) + \sum_{i=1}^{n-1} u_i F_i(\xi), \qquad (3.24)$$

where the $C^{k-1}$ vector fields $F_0, F_i: \mathbb{R}^n \times \mathbb{R}^{n-1} \to \mathbb{R}^n \times \mathbb{R}^{n-1}$ are defined by

$$F_0(\xi) := \begin{pmatrix} \nabla_p \bar{H}(x, \psi(x, q)) \\ -\nabla_{\hat{x}} \bar{H}(x, \psi(x, q)) \end{pmatrix}, \quad F_i(\xi) := \begin{pmatrix} 0_n \\ -e_i^{n-1} \end{pmatrix}, \qquad (3.25)$$

for every $i = 1, \dots, n-1$, $\xi = (x, q) \in \mathbb{R}^n \times \mathbb{R}^{n-1}$. (Recall that $e_1^k, \dots, e_k^k$ denotes the canonical basis of $\mathbb{R}^k$, see Appendix A.) Set $\bar{p}^\tau := P(\bar{\tau}; (\bar{x}^0, \bar{p}^0)) = \bar{p}(\bar{\tau})$, and define the map

$$\Phi: \mathbb{R} \times \mathbb{R}^{n-1} \times \mathbb{R}^{n-1} \longrightarrow \mathbb{R}^n \times \mathbb{R}^{n-1} \\ (t, \hat{x}, q) \longmapsto (X(t; ((\bar{\tau}, \hat{x}), \psi((\bar{\tau}, \hat{x}^f), q)), Q(t; ((\bar{\tau}, \hat{x}), \psi((\bar{\tau}, \hat{x}^f), q))), \qquad (3.26)$$

where $Q = \hat{P}$ denotes the last $n-1$ components of $P$. The function $\Phi$ is of class $C^1$, and its differential at $(0, \hat{x}^\tau, \hat{p}^\tau)$ is invertible. Denote by $\Psi = (\Psi_t, \tilde{\Psi}) \in \mathbb{R} \times (\mathbb{R}^{n-1} \times \mathbb{R}^{n-1})$ the local $C^1$ inverse of $\Phi$ in an open neighborhood $\hat{W}^\tau$ of $(\bar{x}^\tau, \hat{p}^\tau)$ ($\Psi$ is a map which transforms the Hamiltonian trajectories $(X, Q)$ into straight lines), and define the $C^1$ mapping

$$G: \hat{W}^\tau \subset \mathbb{R}^n \times \mathbb{R}^{n-1} \longrightarrow \mathbb{R}^{n-1} \times \mathbb{R}^{n-1} \\ (x, q) \longmapsto \tilde{\Psi}(x, q).$$

Set $\bar{q}^\tau := \hat{p}^\tau$. By construction $G$ is a submersion at $(\bar{x}^\tau, \bar{q}^\tau)$, and the fact that $\dot{x}(\bar{\tau}) = e_1^n$ gives that the kernel of $dG$ at $(\bar{x}^\tau, \bar{q}^\tau)$ is the one dimensional vector space $\mathbb{R}e_1^{2n-1}$. In order to apply Theorem B.5, let us compute the Lie brackets $[F_0, F_i]$ at $\bar{\xi}^\tau := (\bar{x}^\tau, \bar{q}^\tau)$ for every $i = 1, \dots, n-1$. The first $n$ components of $[F_0, F_i]$ at $\bar{\xi}^\tau$ are given by

$$\frac{\partial^2 \bar{H}}{\partial p^2}(\bar{x}^\tau, \bar{p}^\tau) \frac{\partial \psi}{\partial q_i}(\bar{\xi}^\tau).$$

Moreover, since $\psi(x, q) = (\varphi(x, q), q), \bar{H}(x, \psi(x, q)) = 0$ for any $x, q$, and $\dot{x}(\bar{\tau}) = \nabla_p \bar{H}(\bar{x}^\tau, \bar{p}^\tau) = e_1^n$, one has $\frac{\partial \psi}{\partial q_i}(\bar{\xi}^\tau) = e_{i+1}^n$. Therefore, the first $n$ components of the bracket $[F_0, F_i]$ at $\bar{\xi}^\tau$ correspond to the $(i+1)$-th column of the Hessian of $\bar{H}$ in the $p$ variable at $(\bar{x}^\tau, \bar{p}^\tau)$. Since the Hessian of $\bar{H}$ in the $p$ variable is positive definite,

$$\mathrm{Span}\{F_i(\bar{\xi}^\tau) | i = 1, \dots, n-1\} = \{0_n\} \times \mathbb{R}^{n-1}, \quad \text{and} \quad \mathrm{Ker}(dG(\bar{\xi}^\tau)) = \mathbb{R}e_1^{2n-1},$$
we easily deduce that assumption (B.14) is satisfied with $N = 2n - 1$.

Set $\bar{\xi}^0 := (\bar{x}^0, \bar{q}^0)$, recall that $\bar{\xi}^\tau = (\bar{x}^\tau, \bar{q}^\tau)$, and for every $\xi^0 = (x^0, q^0) \in \hat{W}^0$ (with $\hat{W}^0$ defined in (3.19)) and $T \in (\bar{\tau} - \bar{\mu}, \bar{\tau} + \bar{\mu})$, consider the End-Point mapping $E^{\xi^0, T}$ associated with $\xi^0$ in time $T$ (see (3.21)). From Theorem B.5, there are $\delta \in (0, \bar{\mu})$ with $B^{2n-1}(\xi^0, \delta) \subset \hat{W}^0$, constants $K_U, \Lambda, \nu > 0$, and $k := 2n - 2$ smooth controls $u^1, \dots, u^k: [0, +\infty) \to \mathbb{R}^{n-1}$ satisfying

$$ \mathrm{Supp}(u^i) \subset [\delta, \bar{\tau} - \delta] \quad \forall i = 1, \dots, k, \qquad (3.27) $$

such that the following holds: set $\bar{u} \equiv 0$. Then, for every $\xi^0 \in \mathbb{R}^n \times \mathbb{R}^{n-1}$ and $T > 0$ satisfying

$$ |\xi^0 - \bar{\xi}^0|, |T - \bar{\tau}| < \delta, \qquad (3.28) $$

there exists a $C^1$ function

$$ U^{\xi^0, T} = (U_1^{\xi^0, T}, \dots, U_k^{\xi^0, T}) : B^k \left( G(E^{\xi^0, T}(\bar{u})), \nu \right) \to B^k(0, \Lambda), $$

with Lipschitz constant bounded by $K_U$, such that $U^{\xi^0, T}(G(E^{\xi^0, T}(\bar{u}))) = 0$ and

$$ (G \circ E^{\xi^0, T})(\sum_{i=1}^{k} U_i^{\xi^0, T}(z)u^i) = z \quad \forall z \in B^k(G(E^{\xi^0, T}(\bar{u})), \nu). $$

Moreover, there exist $\bar{r}, \bar{\epsilon} \in (0, 1)$ such that, for any $r \in (0, \bar{r})$, $\epsilon \in (0, \bar{\epsilon})$, and any vectors

$$ \xi^0 = (x^0 = (0, \hat{x}^0), q^0), \quad \xi^f = (x^f = (\bar{\tau}, \hat{x}^f), q^f) \qquad (3.29) $$

satisfying (3.22) and (3.23), it holds

$$ |(\hat{x}^f, q^f) - G((E^{\xi^0, T}(\bar{u}))| < \nu, $$

where $\mathcal{T} := \tau(x^0, \psi(x^0, q^0))$ and

$$ \begin{aligned} G((E^{\xi^0, T}(\bar{u})) &= G(X(\mathcal{T}; (x^0, \psi(x^0, q^0))), Q(\mathcal{T}; (x^0, \psi(x^0, q^0)))) \\ &= G(\bar{\tau}, \hat{X}(\mathcal{T}; (x^0, \psi(x^0, q^0))), Q(\mathcal{T}; (x^0, \psi(x^0, q^0)))) \\ &= (\hat{X}(\mathcal{T}; (x^0, \psi(x^0, q^0))), Q(\mathcal{T}; (x^0, \psi(x^0, q^0)))) . \end{aligned} $$

Take $r \in (0, \bar{r})$ with $3r \le \delta$ (with $\delta$ as above, given by Theorem B.5), $\epsilon \in (0, \bar{\epsilon})$, and fix $\xi^0, \xi$ as in (3.29) and satisfying (3.22) and (3.23). From the above discussion, there exists a smooth control $u: [0, \mathcal{T}] \to \mathbb{R}^{n-1}$ given by

$$ u := \sum_{i=1}^{k} U_i^{\xi^0, T}(\hat{x}^f, q^f)u^i \qquad (3.30) $$

such that

$$ (G \circ E^{\xi^0, T})(u) = (\hat{x}^f, q^f). $$

By the definition of $G$, this gives

$$ E^{\xi^0, T^f}(u) = ((\bar{\tau}, \hat{x}^f), q^f), \qquad (3.31) $$
where $T^f$ is defined as

$$T^f := \mathcal{T} - \Psi_t(E^{\xi^0, \mathcal{T}}(u)). \qquad (3.32)$$

Since the function $U^{\xi^0, \mathcal{T}}$ is $K_U$-Lipschitz and $U^{\xi^0, \mathcal{T}}(G(E^{\xi^0, \mathcal{T}}(\bar{u}))) = 0$, we have

$$ \begin{aligned} \|u\|_{C^1} &\le K_U N_U |(\hat{x}^f, q^f) - (\hat{X}(\mathcal{T}; (x^0, \psi(x^0, q^0))), Q(\mathcal{T}; (x^0, \psi(x^0, q^0))))| \\ &\le K_U N_U \sqrt{1 + C_{\varphi}^2} r\epsilon, \end{aligned} \qquad (3.33) $$

where

$$N_U := \max\{\|u^i\|_{C^1} \mid i = 1, \dots, k\}. \qquad (3.34)$$

Note that, up to choosing $\nu$ smaller, we can assume that $K_U N_U \sqrt{1 + C_{\varphi}^2} \nu < \bar{\mu}$, so that any trajectory $(X_{\xi^0}^u(\cdot), Q_{\xi^0}^u(\cdot))$ associated with $\xi^0 = (x^0, q^0) \in B^{2n-1}(\bar{\xi}^0, \delta)$ and $u$ given by (3.30) is contained in $\hat{W}$ (thanks to (3.20)). Note also that

$$\Psi_t(E^{\xi^0, \mathcal{T}}(\bar{u})) = \Psi_t((\bar{\tau}, \hat{X}(\mathcal{T}; (x^0, \psi(x^0, q^0))), Q(\mathcal{T}; (x^0, \psi(x^0, q^0))))) = 0.$$

Hence, if we denote by $K_t$ the Lipschitz constant of the function $\Psi_t$ in $\hat{W}^{\bar{\tau}}$ and by $K_E$ a uniform (as $\xi^0$ and $\mathcal{T}$ vary) local Lipschitz constant for the functions $E^{\xi^0, \mathcal{T}}$, thanks to (3.33) we get

$$ \begin{aligned} |T^f - \mathcal{T}| &= |\Psi_t(E^{\xi^0, \mathcal{T}}(u)) - \Psi_t(E^{\xi^0, \mathcal{T}}(\bar{u}))| \\ &\le K_t |E^{\xi^0, \mathcal{T}}(u) - E^{\xi^0, \mathcal{T}}(\bar{u})| \qquad (3.35) \\ &\le K_t K_E \|u\|_{C^1} \le K_t K_E K_U N_U \sqrt{1 + C_{\varphi}^2} r\epsilon. \end{aligned} $$

Denote respectively by $(x(\cdot) = (x_1(\cdot), \hat{x}(\cdot)), p(\cdot) = (p_1(\cdot), \hat{p}(\cdot))) : [0, T^f] \to W$ and $v = (v_1, u) : [0, T^f] \to \mathbb{R}^n$ the trajectory and the control given by Lemma 3.2. Then by (A3), (3.18) and (3.33), we have (note that $x(\cdot), p(\cdot)$ and $\psi(\cdot)$ are of class $C^k$)

$$\|v\|_{C^1} \le \bar{K} K_U N_U \sqrt{1 + C_{\varphi}^2} r\epsilon, \qquad (3.36)$$

where $\bar{K}$ is a positive constant which depends on the $C^2$-norm of the restriction of $\bar{H}$ to $W$. Moreover, our construction gives also

$$\langle v(t), \dot{v}(t) \rangle = 0 \quad \forall t \in [0, T^f] \qquad (3.37)$$

(see Lemma 3.2). Now, starting from the control $v$, we construct the potential $V$ given in the statement of Proposition 3.1. We state a general lemma which will be useful again in the proof of Proposition 4.1, and whose proof is postponed to Appendix E.1. Let us point out that, for the purpose of this paper, in assertion (ii) of the lemma below it would suffice to write $\|\tilde{v}_1\|_{\infty}$ in place of $\|\tilde{V}_1\|_{\infty}$. However, this slightly stronger version will be useful in the proof of [27, Lemma 4.1] (see [27, Lemma A.1]).

**Lemma 3.3.** Let $\bar{\tau}, \delta, r \in (0, 2)$ with $3r \le \delta < \bar{\tau}$, and let $\tilde{v} = (\tilde{v}_1, \dots, \tilde{v}_n) : [0, \bar{\tau}] \to \mathbb{R}^n$ be a function of class $C^{k-1}$ with $k \ge 2$ satisfying

$$\tilde{v}(t) = 0_n \quad \forall t \in [0, \delta] \cup [\bar{\tau} - \delta, \bar{\tau}] \qquad (3.38)$$
and

$$ \int_0^{\bar{\tau}} \tilde{v}_1(t) dt = 0. \qquad (3.39) $$

Set $\tilde{V}_1(t) := \int_0^t \tilde{v}_1(s) ds$ for $t \in [0, \bar{\tau}]$. Then, there exist a universal constant $K$ depending only on the dimension, and a function $W: \mathbb{R}^n \to \mathbb{R}$ of class $C^k$, such that the following properties hold:

(i) $\text{Supp}(W) \subset [\delta/2, \bar{\tau} - \delta/2] \times B^{n-1}(0_{n-1}, 2r/3) \subset \mathbb{R} \times \mathbb{R}^{n-1}$;

(ii) $\|W\|_{C^2} \le K \left( \frac{1}{r^2} \|\tilde{V}_1\|_{\infty} + \frac{1}{r} \|\tilde{v}\|_{\infty} + \|\dot{\tilde{v}}\|_{\infty} \right)$;

(iii) $\nabla W(t, 0_{n-1}) = \tilde{v}(t)$ for every $t \in [0, \bar{\tau}]$.

Define the function $\Gamma: [0, \bar{\tau}] \times \mathbb{R}^{n-1} \to \mathbb{R}^n$ by

$$ \Gamma(t, \hat{z}) := x \left( \frac{tT^f}{\bar{\tau}} \right) + (0, \hat{z}) \quad \forall (t, \hat{z}) \in [0, \bar{\tau}] \times \mathbb{R}^{n-1}, \qquad (3.40) $$

where $x(\cdot)$ is the trajectory associated to the control $v$ constructed above. Since $\bar{H}$ is of class $C^k$ and $v$ of class $C^{k-1}$, the curve $t \mapsto x(t)$ is of class $C^k$, thus $\Gamma$ is of class $C^k$, too. Moreover, since $x_1(0) = 0$ and $x_1(T^f) = \bar{\tau}$, we can easily check that $\Gamma$ is a $C^k$ diffeomorphism from $[0, \bar{\tau}] \times \mathbb{R}^{n-1}$ into $[0, \bar{\tau}] \times \mathbb{R}^{n-1}$ which sends the cylinder $[0, \bar{\tau}] \times B_r^{n-1}$ into the “cylinder”

$$ C' := \{ x(t) + (0, \hat{y}) \mid t \in [0, T^f], |\hat{y}| < 2r/3 \} $$

and which satisfies

$$ \|\Gamma\|_{C^2}, \|\Gamma^{-1}\|_{C^2} \le \bar{K}', \qquad (3.41) $$

for some positive constant $\bar{K}'$ depending on the $C^2$-norm of the restriction of $\bar{H}$ to $W$ and on the $C^0$-norm of $v$ (since $\ddot{x}(t)$ can be written in terms of $x(t)$, $p(t)$, $\dot{x}(t)$ and $\dot{p}(t)$). Define the function $\tilde{v} = (\tilde{v}_1, \dots, \tilde{v}_n) : [0, \bar{\tau}] \to \mathbb{R}^n$ by

$$ \tilde{v}(t) := (d\Gamma(t, 0_{n-1}))^* \left( v \left( \frac{tT^f}{\bar{\tau}} \right) \right) \quad \forall t \in [0, \bar{\tau}]. \qquad (3.42) $$

The function $\tilde{v}$ is $C^{k-1}$; in addition, thanks to (3.37) and (3.40), for every $t \in [0, \bar{\tau}]$ we have

$$ \tilde{v}_1(t) = 0 \quad \text{and} \quad \tilde{v}_i(t) = v_i \left( \frac{tT^f}{\bar{\tau}} \right) \quad \forall i = 2, \dots, n. $$

Hence $\tilde{v}$ satisfies both (3.38) and (3.39), so that applying Lemma 3.3 yields a function $W: \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ satisfying assertions (i)-(iii) of Lemma 3.3, with

$$ \|W\|_{C^2} \le \frac{C}{r} \|\tilde{v}\|_{C^1} \qquad (3.43) $$

(as $\tilde{v}_1 = 0$). Define the potential $V: \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ by

$$ V(x) = \begin{cases} W(\Gamma^{-1}(x)) & \text{if } x \in C' \\ 0 & \text{otherwise.} \end{cases} $$

Thanks to Lemma 3.3(i) we have $\text{Supp}(V) \subset C'$. Furthermore, since the mapping

$$ \begin{array}{ccc} \hat{W}^0 \times C^1([0, \bar{\tau} + \bar{\mu}], \mathbb{R}^m) \times [0, \bar{\tau} + \bar{\mu}] & \to & \mathbb{R}^n \\ (\xi^0, u, t) & \to & X_{\xi^0}^u(t) \end{array} $$
is locally Lipschitz, there exists $\hat{K} > 0$ such that

$$|x(t) - X(t; \psi(\xi^0))| = |X_{\xi^0}^u(t) - X_{\xi^0}^{\bar{u}}(t)| \le \hat{K} \|u\|_{C^1} \quad \forall t \in [0, T^f]. \quad (3.44)$$

(By abuse of notation, we write $\psi(\xi^0) := (x^0, \psi(q^0))$ for $\xi^0 = (x^0, p^0)$. Thanks to (3.33), this implies that, for $\epsilon$ sufficiently small, $C'$ is contained in the “cylinder” defined in (3.11):

$$\mathrm{Supp}(V) \subset C' \subset \mathcal{C}(\psi(\xi^0); \tau(x^0, p^0); r). \quad (3.45)$$

Now, the gradient of $V$ at a point $x$ is given by

$$\nabla V(x) = (d\Gamma^{-1}(x))^* \nabla W(\Gamma^{-1}(x)),$$

so that by Lemma 3.3(iii) and (3.42) we get

$$\nabla V(x(t)) = v(t) \quad \forall t \in [0, T^f]. \quad (3.46)$$

Moreover,

$$\|\mathrm{Hess}\,V\|_{\infty} \le \|\nabla W\|_{\infty} \|d^2\Gamma^{-1}\|_{\infty} + \|\mathrm{Hess}\,W\|_{\infty} \|d\Gamma^{-1}\|_{\infty}^2. \quad (3.47)$$

Thanks to (3.31), (3.35), (3.36), (3.41), (3.42), (3.43), (3.45), (3.46), (3.47), we conclude easily that there are $\bar{\delta}, \bar{r}, \bar{\epsilon} \in (0, 1)$ small enough and $K > 0$ such that assertions (i)-(iv) of Proposition 3.1 hold.

It remains to show that, up to choosing $K$ larger, assertion (v) holds. Let us compute the differences of the actions between the two trajectories

$$x(\cdot) : [0, T^f] \to \mathbb{R}^n \quad \text{and} \quad X^0(\cdot) := X(\cdot; (x^0, p^0)) : [0, T] \to \mathbb{R}^n.$$

Set $P^0(t) := P(t; (x^0, p^0))$ for every $t \in [0, T]$. Observe that, by (3.37) and (3.46), $V = 0$ along the new trajectory $x(\cdot)$. Moreover, due to (3.36) and a simple Gronwall argument, we have

$$|x(t) - X^0(t)| + |\dot{x}(t) - \dot{X}^0(t)| \le \bar{K}r\epsilon \quad \forall t \in [0, T^f],$$

for some constant $\bar{K}$ depending only on $\bar{H}$. Hence, thanks to this estimate there exists a constant $\tilde{K}$ such that

$$\begin{aligned}
& \left| \mathbb{A}_V((x^0, p^0); T^f) - \mathbb{A}((x^0, p^0); T) - \Delta((x^0, p^0); \tau(x^0, p^0); x^f) \right| \\
& = \left| \int_0^{T^f} \bar{L}_V(x(t), \dot{x}(t)) dt - \int_0^T \bar{L}(X^0(t), \dot{X}^0(t)) dt - \langle P^0(T), x(T^f) - X^0(T) \rangle \right| \\
& = \left| \int_0^{T^f} \bar{L}(x(t), \dot{x}(t)) dt - \int_0^T \bar{L}(X^0(t), \dot{X}^0(t)) dt - \langle P^0(T), x(T^f) - X^0(T) \rangle \right| \\
& = \left| \int_0^T \left( \bar{L}(x(t), \dot{x}(t)) - \bar{L}(X^0(t), \dot{X}^0(t)) \right) dt \right. \\
& \qquad \left. + \int_T^{T^f} \bar{L}(x(t), \dot{x}(t)) dt - \langle P^0(T), x(T^f) - X^0(T) \rangle \right| \\
& \le \left| \langle \nabla_v \bar{L}(X^0(T), \dot{X}^0(T)), x(T) - X^0(T) \rangle \right. \\
& \qquad \left. + \int_T^{T^f} \bar{L}(x(t), \dot{x}(t)) dt - \langle P^0(T), x(T^f) - X^0(T) \rangle \right| + \tilde{K}r^2\epsilon^2,
\end{aligned}$$
where we use Taylor's formula at second order for $\bar{L}$, together with an integration by part and the fact that the Euler-Lagrange equations

$$ \frac{d}{dt} \{ \nabla_v \bar{L}(X^0(t), \dot{X}^0(t)) \} = \nabla_x \bar{L}(X^0(t), \dot{X}^0(t)) $$

are satisfied for any $t \in [0, T]$. Using now that

$$ P^0(\mathcal{T}) = \nabla_v \bar{L}(X^0(\mathcal{T}), \dot{X}^0(\mathcal{T})) \quad \text{and} \quad \bar{L}(x(t), \dot{x}(t)) = \langle p(t), \dot{x}(t) \rangle \quad \forall t \in [0, T^f] $$

(since $\bar{H}(x^0, p^0) = 0$, and $\langle \nabla V(x(t)), \dot{x}(t) \rangle = 0$ by (3.37) and (3.46)), we obtain

$$
\begin{align*}
& \left| \mathbb{A}_V((x^0, p^0); T^f) - \mathbb{A}((x^0, p^0); \mathcal{T}) - \Delta((x^0, p^0); \tau(x^0, p^0); x^f) \right| \\
& \le \left| \langle P^0(\mathcal{T}), x(\mathcal{T}) - X^0(\mathcal{T}) \rangle + \int_{\mathcal{T}}^{T^f} \langle p(t), \dot{x}(t) \rangle dt - \langle P^0(\mathcal{T}), x(T^f) - X^0(\mathcal{T}) \rangle \right| + \tilde{K}r^2\epsilon^2 \\
& \le \left| \langle P^0(\mathcal{T}), x(\mathcal{T}) - x(T^f) \rangle + \int_{\mathcal{T}}^{T^f} \langle P^0(\mathcal{T}), \dot{x}(t) \rangle dt \right| + \left| \int_{\mathcal{T}}^{T^f} \langle p(t) - P^0(\mathcal{T}), \dot{x}(t) \rangle dt \right| + \tilde{K}r^2\epsilon^2 \\
& = \left| \langle P^0(\mathcal{T}), x(\mathcal{T}) - x(T^f) \rangle + \langle P^0(\mathcal{T}), \int_{\mathcal{T}}^{T^f} \dot{x}(t) dt \rangle \right| + \left| \int_{\mathcal{T}}^{T^f} \langle p(t) - P^0(\mathcal{T}), \dot{x}(t) \rangle dt \right| + \tilde{K}r^2\epsilon^2 \\
& = \left| \int_{\mathcal{T}}^{T^f} \langle p(t) - P^0(\mathcal{T}), \dot{x}(t) \rangle dt \right| + \tilde{K}r^2\epsilon^2 \\
& \le \int_{\mathcal{T}}^{T^f} |p(t) - P^0(\mathcal{T})| |\dot{x}(t)| dt + \tilde{K}r^2\epsilon^2.
\end{align*}
$$

Now, note that by our assumptions on $(x^f, p^f := \psi(x^f, q^f))$ we have $|p(T^f) - P^0(\mathcal{T})| = |p^f - P^0(\mathcal{T})| < r\epsilon$. Moreover, (3.35) holds. Hence, since the function $t \mapsto p(t)$ is Lipschitz and $t \mapsto |\dot{x}(t)|$ is bounded (both with bounds depending only on $\bar{H}$), we can find $K > 0$ such that (v) holds.

*Remark 3.4.* Let us point out that the above bound on the action can be slightly refined: indeed (3.33) shows that

$$ \|u\|_{C^1} \le K_U N_U \left| (\hat{x}^f, q^f) - (\hat{X}(\mathcal{T}; (x^0, p^0)), Q(\mathcal{T}; (x^0, p^0))) \right|, $$

so it is easily seen that the above proof actually gives

$$
\begin{align*}
& \left| \mathbb{A}_V((x^0, p^0); T^f) - \mathbb{A}((x^0, p^0); \mathcal{T}) - \Delta((x^0, p^0); \tau(x^0, p^0); x^f) \right| \\
& \le K' \left| (\hat{x}^f, q^f) - (\hat{X}(\mathcal{T}; (x^0, p^0)), Q(\mathcal{T}; (x^0, p^0))) \right|^2
\end{align*}
$$

for some uniform constant $K'$, which of course implies (v). Moreover, the above estimates hold also with different final times: for any $\tau \in [0, \bar{\tau}]$, $t \in [0, \tau(x^1, p^1)]$ and $t_V \in [0, T^f]$ such that $x(t_V), X(t; (x^0, p^0)) \in \Pi^\tau := \{x = (\tau, \hat{x}) \in \mathbb{R}^n\}$, it holds:

$$ |t_V - t| \le K' \left| (\hat{x}(t_V), q(t_V)) - (\hat{X}(t; (x^0, p^0)), Q(t; (x^0, p^0))) \right|, \quad (3.48) $$

$$
\begin{align}
& \left| \mathbb{A}_V((x^0, p^0); t_V) - \mathbb{A}((x^0, p^0); t) - \Delta((x^0, p^0); t; x(t_V)) \right| \nonumber \\
& \le K' \left| (\hat{x}(t_V), q(t_V)) - (\hat{X}(t; (x^0, p^0)), Q(t; (x^0, p^0))) \right|^2 \tag{3.49}
\end{align}
$$

Although these two refined bounds will never be used in this paper, they will be crucial for future applications (see [27, Propositions 2.1 and 2.2]).
# 4 Controlling the action by potentials

## 4.1 Statement of the result

Fix $n \ge 2$, $\bar{\tau} \in (0,1)$, and consider a Hamiltonian $\bar{H} : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ of class $C^k$, with $k \ge 2$, satisfying (H1)-(H3). Let

$$ (\bar{x}(\cdot), \bar{p}(\cdot)) : [0, \bar{\tau}] \to \mathbb{R}^n \times \mathbb{R}^n $$

be a trajectory satisfying (A1)-(A3). We keep the same notation as the ones in Subsection 3.1. Recall that the Poincaré mapping $\tau = \tau(x, p)$, with respect to the section $\Pi^{\tau}$, is defined on an open neighborhood $\mathcal{V}^0$ of $(\bar{x}^0, \bar{p}^0)$. Our aim is to show that, given $(x^0 = (0, \hat{x}^0), p^0)$ with $\bar{H}(x^0, p^0) = 0$ sufficiently close to $(\bar{x}^0, \bar{p}^0)$, and $\sigma \in \mathbb{R}$ sufficiently small, there exist a time $T^f$ close to $\tau(x^0, p^0)$ and a potential $V : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ whose support and $C^2$-norm are controlled, such that the solution

$$ (X^V(\cdot), P^V(\cdot)) : [0, T] \to \mathbb{R}^n \times \mathbb{R}^n $$

of the Hamiltonian system

$$ \left\{ \begin{aligned} \dot{x}(t) &= \nabla_p \bar{H}_V(x(t), p(t)) = \nabla_p \bar{H}(x(t), p(t)) \\ \dot{p}(t) &= -\nabla_x \bar{H}_V(x(t), p(t)) = -\nabla_x \bar{H}(x(t), p(t)) - \nabla V(x(t)) \end{aligned} \right. \quad (4.1) $$

starting at $(X^V(0), P^V(0)) = (x^0, p^0)$ satisfies

$$ (X^V(T), P^V(T)) = \phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0) $$

and

$$ \mathbb{A}_V((x^0, p^0); T^f) = \mathbb{A}((x^0, p^0); \tau(x^0, p^0)) + \sigma, $$

where $\mathbb{A}_V$ is defined in (3.6) and $\mathbb{A} = \mathbb{A}_0$. We now state our result.

We recall that $\mathcal{C}((x^0, p^0); \tau(x^0, p^0); r)$ denotes the “cylinder” defined in (3.11), and we define the two matrices

$$ \frac{\partial^2 \bar{H}}{\partial \bar{p}^2}(\bar{x}^{\tau}, \bar{p}^{\tau}) := \left( \frac{\partial^2 \bar{H}}{\partial p_i \partial p_j}(\bar{x}^{\tau}, \bar{p}^{\tau}) \right)_{i,j=2,...,n} \in M_{n-1}(\mathbb{R}), $$

$$ \frac{\partial^2 \bar{H}}{\partial p^2}(\bar{x}^{\tau}, \bar{p}^{\tau}) := \left( \frac{\partial^2 \bar{H}}{\partial p_i \partial p_j}(\bar{x}^{\tau}, \bar{p}^{\tau}) \right)_{i,j=1,...,n} \in M_n(\mathbb{R}), $$

where $\bar{x}^{\tau} = \bar{x}(\bar{\tau})$ and $\bar{p}^{\tau} = \bar{p}(\bar{\tau})$.

**Proposition 4.1.** Let $\bar{H} : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ be a Hamiltonian of class $C^k$ with $k \ge 2$ satisfying (H1)-(H3), and let $(\bar{x}(\cdot), \bar{p}(\cdot)) : [0, \bar{\tau}] \to \mathbb{R}^n \times \mathbb{R}^n$ be a solution of (3.1) with $\bar{H}(\bar{x}^0, \bar{p}^0) = 0$ and satisfying (A1)-(A3). Set $\bar{p}_1^{\tau} := \bar{p}(\bar{\tau})_1$, and assume that the following property is satisfied:

$$ (A4) \det \left( \frac{\partial^2 \bar{H}}{\partial \bar{p}^2} (\bar{x}^{\tau}, \bar{p}^{\tau}) \right) + \bar{p}_1^{\tau} \det \left( \frac{\partial^2 \bar{H}}{\partial p^2} (\bar{x}^{\tau}, \bar{p}^{\tau}) \right) \neq 0. $$

Then there are $\bar{\delta}, \bar{r} \in (0,1)$ with $B^{2n}((\bar{x}^0, \bar{p}^0), \bar{\delta}) \subset \mathcal{V}^0$, and $K > 0$, such that the following property holds: For every $r \in (0, \bar{r})$, $\epsilon \in (0,1)$, and every $x^0 = (0, \hat{x}^0), p^0 \in \mathbb{R}^n, \sigma \in \mathbb{R}$ satisfying

$$ |\hat{x}^0|, |p^0 - \bar{p}^0| < \bar{\delta}, \quad (4.2) $$

$$ |\sigma| < 2r^2\epsilon, \quad (4.3) $$

$$ \bar{H}(x^0, p^0) = 0, \quad (4.4) $$

there exist a time $T^f > 0$ and a potential $V : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ such that:
$$ (i) \text{Supp}(V) \subset \mathcal{C}((x^0, p^0); \tau(x^0, p^0); r); $$

$$ (ii) \quad \|V\|_{C^2} < \frac{K|\sigma|}{r^2} \le 2K\epsilon; $$

$$ (iii) \quad |T^f - \tau(x^0, p^0)| < K|\sigma| \le 2Kr^2\epsilon; $$

$$ (iv) \quad \phi_{T^f}^{\bar{H}_V}(x^0, p^0) = \phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0); $$

$$ (v) \quad \mathbb{A}_V((x^0, p^0); T^f) = \mathbb{A}((x^0, p^0); \tau(x^0, p^0)) + \sigma. $$

## 4.2 Proof of Proposition 4.1

Given $(x^0 = (0, \hat{x}^0), p^0)$ and $\sigma$ such that (4.2)-(4.4) are satisfied, we are going to look for a function $v : [0, T^f] \to \mathbb{R}^n$ of class $C^{k-1}$ such that the solution to the system

$$ \begin{cases} \dot{x}(t) = \nabla_p \bar{H}(x(t), p(t)) \\ \dot{p}(t) = -\nabla_x \bar{H}(x(t), p(t)) - v(t), \end{cases} $$

starting at $(x^0, p^0)$ satisfies

$$ (x(T^f), p(T^f)) = \phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0) \quad \text{and} \quad \int_0^{T^f} \langle p(t), \dot{x}(t) \rangle dt = \mathbb{A}((x^0, p^0); \tau(x^0, p^0)) + \sigma. $$

In that way, if we find a function $V : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ such that $\nabla V(x(t)) = v(t)$ for all $t \in [0, T^f]$, then the solution $(X^V, P^V)$ of the Hamiltonian system (4.1) starting at $(\bar{x}^0, \bar{p}^0)$ satisfies $(X^V(T^f), P^V(T^f)) = \phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0)$. Moreover, since $\bar{H}^V$ is preserved along the trajectory $t \mapsto (X^V(t), P^V(t))$, we have $\bar{H}_V(X^V(t), P^V(t)) \equiv 0$ and we get

$$ \begin{aligned} \mathbb{A}_V((x^0, p^0); T^f) &= \int_0^{T^f} \bar{L}_V(X^V(t), \dot{X}^V(t)) dt \\ &= \int_0^{T^f} \langle P^V(t), \dot{X}^V(t) \rangle - \bar{H}_V(X^V(t), P^V(t)) dt \\ &= \int_0^{T^f} \langle P^V(t), \dot{X}^V(t) \rangle dt \\ &= \mathbb{A}((x^0, p^0); \tau(x^0, p^0)) + \sigma. \end{aligned} $$

Thus assertions (iv) and (v) will be satisfied. It will remain to control the support and the $C^2$-norm of $V$. In particular, since $v$ has to be the gradient of a function $V$ supported in $\mathcal{C}((x^0, p^0); \tau(x^0, p^0), r)$, it must satisfy

$$ \int_0^{T^f} \langle v(t), \dot{x}(t) \rangle dt = 0. $$

For that reason, we study the control system

$$ \begin{cases} \dot{x}(t) = \nabla_p \bar{H}(x(t), p(t)) \\ \dot{p}(t) = -\nabla_x \bar{H}(x(t), p(t)) - v(t) \\ \dot{\vartheta}(t) = \langle v(t), \nabla_p \bar{H}(x(t), p(t)) \rangle \\ \dot{\sigma}(t) = \langle p(t), \nabla_p \bar{H}(x(t), p(t)) \rangle. \end{cases} \quad (4.5) $$

For every $(x^0, p^0) \in \mathcal{V}^0$, set

$$ \bar{\sigma}(x^0, p^0) := -\mathbb{A}((x^0, p^0); \tau(x^0, p^0)). \quad (4.6) $$
For each $(x^0, p^0) \in \mathcal{V}^0$ and every smooth function $v : [0, +\infty) \to \mathbb{R}^n$, there is a unique trajectory

$$ (X_{(x^0, p^0)}^v(\cdot), P_{(x_0, p^0)}^v(\cdot), \Theta_{(x^0, p^0)}^v(\cdot), \Sigma_{(x^0, p^0)}^v(\cdot)) : [0, \tau(x^0, p^0)] \to \mathbb{R}^n \times \mathbb{R}^n \times \mathbb{R} \times \mathbb{R} $$

starting at $(x^0, p^0, 0, \bar{\sigma}(x^0, p^0))$ which satisfies (4.5) for every $t \in [0, +\infty)$. For every $T > 0$, define the mapping $E^{(x^0, p^0), T} : C^\infty([0, T]; \mathbb{R}^n) \to \mathbb{R}^n \times \mathbb{R}^n \times \mathbb{R} \times \mathbb{R}$ by

$$ E^{(x^0, p^0), T}(v) := (X_{(x^0, p^0)}^v(T), P_{(x_0, p^0)}^v(T), \Theta_{(x^0, p^0)}^v(T), \Sigma_{(x^0, p^0)}^v(T)). \quad (4.7) $$

Given $(x^0, p^0)$ and $\sigma \in \mathbb{R}$, our aim is to find $T^f > 0$, together with a function $v : [0, T^f] \to \mathbb{R}^n$ of class $C^\infty$, such that

$$ E^{(x^0, p^0), T^f}(v) = (\phi_{\tau(x^0, p^0)}^H(x^0, p^0), 0, \sigma), \quad \text{with a control on } \|v\|_{C^1}. $$

We observe that the control system (4.5) is over-determined: indeed, along any trajectory $(X_{(x^0, p^0)}^v, P_{(x_0, p^0)}^v, \Theta_{(x^0, p^0)}^v, \Sigma_{(x^0, p^0)}^v)$ of (4.5) there holds

$$ \bar{H} \left( X_{(x^0, p^0)}^v(t), P_{(x_0, p^0)}^v(t) \right) + \Theta_{(x^0, p^0)}^v(t) = \bar{H}(x^0, p^0) = 0 \quad \forall t \in [0, +\infty). $$

This means that we have at most $2n + 1$ degrees of freedom in the choice of the final state $(X_{(x^0, p^0)}^v(T^f), P_{(x_0, p^0)}^v(T^f), \Theta_{(x^0, p^0)}^v(T^f), \Sigma_{(x^0, p^0)}^v(T^f))$. By (A3) and the Implicit Function Theorem, there are a bounded open neighborhood $\mathcal{W}$ of the set

$$ \{ (\bar{x}(t), \bar{p}(t), 0) \mid t \in [0, \bar{\tau}] \} \subset \mathbb{R}^{2n+1}, $$

a bounded open neighborhood $\mathcal{W}$ of the set

$$ \{ (\bar{x}(t), \bar{p}(t), 0) \mid t \in [0, \bar{\tau}] \} \subset \mathbb{R}^{2n}, $$

and a function $\varphi : \mathcal{W} \to \mathbb{R}$ of class $C^k$ such that

$$ \left\{ \begin{array}{l} \forall (x, p, h) \in \mathcal{W} : \bar{H}(x, p) + \vartheta = 0 \implies p_1 = \varphi(x, \hat{p}, \vartheta); \\ \forall (x, q, \vartheta) \in \mathcal{W} : (x, (\varphi(x, q, z), q), \vartheta) \in \mathcal{W} \quad \text{and} \quad \bar{H}(x, (\varphi(x, q, \vartheta), q)) + \vartheta = 0. \end{array} \right. \quad (4.8) $$

Define the $C^k$ function $\psi : \mathcal{W} \to \mathbb{R}^n$ by

$$ \psi(x, q, \vartheta) := (\varphi(x, q, \vartheta), q). $$

Then, any solution of

$$ \left\{ \begin{aligned} \dot{x}(t) &= \nabla_p \bar{H}(x(t), \psi(x(t), q(t), \vartheta(t))) \\ \dot{q}(t) &= -\nabla_{\hat{\vartheta}} \bar{H}(x(t), \psi(x(t), q(t), \vartheta(t))) - \hat{v}(t) \\ \dot{\vartheta}(t) &= \langle v(t), \nabla_p \bar{H}(x(t), \psi(x(t), q(t), \vartheta(t))) \rangle \\ \dot{\sigma}(t) &= \langle \psi(x(t), q(t), \vartheta(t)), \nabla_p \bar{H}(x(t), \psi(x(t), q(t), \vartheta(t))) \rangle \end{aligned} \right. \quad (4.9) $$

generates a unique solution of (4.5), where $\hat{v}(t) = (v_2(t), \dots, v_n(t))$:

**Lemma 4.2.** Let $(x(\cdot), q(\cdot), \vartheta(\cdot), \sigma(\cdot)) : [0, T] \to \mathcal{W} \times \mathbb{R}$ be a solution of (4.9) starting at $(x^0, \hat{p}^0, 0, \sigma^0)$ and associated with a smooth control $v : [0, T] \to \mathbb{R}^n$. Then, the extended trajectory $(x(\cdot), p(\cdot), \vartheta(\cdot), \sigma(\cdot)) : [0, T] \to \mathcal{W} \times \mathbb{R}$ defined by

$$ p_1(t) = \varphi(x(t), q(t), \vartheta(t)) \quad \forall t \in [0, T], \quad (4.10) $$

$$ \hat{p}(t) = q(t) \quad \forall t \in [0, T], $$

is the solution of (4.5) starting at $(x^0, p^0, 0, \sigma^0)$ and associated with the control $v$.
Proof of Lemma 4.2. It is sufficient to show that, for every $t \in [0, T]$, $\dot{p}_1(t)$ is given by

$$ \dot{p}_1(t) = -\frac{\partial \bar{H}}{\partial x_1}(x(t), \psi(x(t), \hat{p}(t), \vartheta(t))) - v_1(t). $$

Differentiating (4.10) with respect to $t$ we get

$$ \dot{p}_1(t) = \langle \nabla_x \varphi(x(t), q(t), \vartheta(t)), \dot{x}(t) \rangle + \langle \nabla_q \varphi(x(t), q(t), \vartheta(t)), \dot{q}(t) \rangle + \langle \nabla_{\vartheta} \varphi(x(t), q(t), \vartheta(t)), \dot{\vartheta}(t) \rangle $$

for all $t \in [0, T]$. Moreover, differentiating the equality $\bar{H}(x, (\varphi(x, q, \vartheta), q)) + \vartheta = 0$ (given by (4.8)) with respect to $x, q$ and $\vartheta$ yields

$$ \left\{ \begin{array}{l} \nabla_x \varphi(x, q, \vartheta) = - \left( \frac{\partial \bar{H}}{\partial p_1} (x, \psi(x, q, \vartheta)) \right)^{-1} \nabla_x \bar{H} (x, \psi(x, q, \vartheta)), \\ \\ \nabla_q \varphi(x, q, \vartheta) = - \left( \frac{\partial \bar{H}}{\partial p_1} (x, \psi(x, q, \vartheta)) \right)^{-1} \nabla_{\hat{p}} \bar{H} (x, \psi(x, q, \vartheta)), \\ \\ \nabla_{\vartheta} \varphi(x, q, \vartheta) = - \left( \frac{\partial \bar{H}}{\partial p_1} (x, \psi(x, q, \vartheta)) \right)^{-1} \cdot \end{array} \right. \quad (4.11) $$

We conclude easily.

<figure><img src="image_3.png" /></figure>

As in the proof of Proposition 3.1, we apply the controllability results given in Appendix B.

Consider the following nonlinear control system in $\mathbb{R}^n \times \mathbb{R}^{n-1} \times \mathbb{R} \times \mathbb{R}$:

$$ \dot{\xi} = F_0(\xi) + \sum_{i=1}^{n} v_i F_i(\xi), \quad (4.12) $$

where the $C^{k-1}$ vector fields $F_0, F_i$ are defined by

$$ F_0(\xi) := \left( \begin{array}{c} \nabla_p \bar{H}(x, \psi(x, q, \vartheta)) \\ -\nabla_{\hat{x}} \bar{H}(x, \psi(x, q, \vartheta)) \\ \langle \psi(x, q, \vartheta), \nabla_p \bar{H}(x, \psi(x, q, \vartheta)) \rangle \end{array} \right), \quad F_i(\xi) := \left( \begin{array}{c} 0_n \\ -e_{i-1}^{n-1} \\ \frac{\partial \bar{H}}{\partial p_i}(x, \psi(x, q, \vartheta)) \\ 0 \end{array} \right), \quad (4.13) $$

for every $i = 1, \dots, n$, $\xi = (x, q, \vartheta, \sigma) \in \mathbb{R}^n \times \mathbb{R}^{n-1} \times \mathbb{R} \times \mathbb{R}$ (with the convention $e_0^{n-1} = 0$). Recall that $\phi_t^{\bar{H}}$ denotes the Hamiltonian flow associated with $\bar{H}$ on $\mathbb{R}^n \times \mathbb{R}^n$. Set

$$ \hat{\pi}(x, p) := (x, \hat{p}) \quad \forall (x, p) \in \mathbb{R}^n \times \mathbb{R}^n, $$

and define the $C^1$ map $\Phi: \mathbb{R} \times \mathbb{R}^{n-1} \times \mathbb{R}^{n-1} \times \mathbb{R} \times \mathbb{R} \to \mathbb{R}^n \times \mathbb{R}^{n-1} \times \mathbb{R} \times \mathbb{R}$

$$ \Phi(t, \hat{x}, q, \vartheta, \sigma) := \left( \hat{\pi} \left( \phi_t^{\bar{H}} \left( (\bar{\tau}, \hat{x}), \psi \left( (\bar{\tau}, \hat{x}), q, \vartheta \right) \right) \right), \vartheta, \sigma + \mathcal{A} \left( ((\bar{\tau}, \hat{x}), \psi((\bar{\tau}, \hat{x}), q, \vartheta)); t \right) \right). $$

(Observe that the first $2n-1$ components of the map $\Phi$ above coincides, up to the presence of a dependence on $\vartheta$, with the map $\Phi$ defined in the previous section, see (3.26).) The function $\Phi$ is of class $C^1$ and its differential at $(0, \hat{x}^{\bar{\tau}}, \hat{p}^{\bar{\tau}}, 0, 0)$ is invertible. Denote by $\Psi = (\Psi_t, \tilde{\Psi})$ the local $C^1$ inverse of $\Phi$ in an open neighborhood $\mathcal{W}^{\bar{\tau}}$ of $(\bar{x}^{\bar{\tau}}, \hat{p}^{\bar{\tau}}, 0, 0)$ (as in the previous section, the map $\Psi$ straightens the Hamiltonian trajectories), and define the $C^1$ mapping

$$ G: \begin{array}{rcl} \hat{W}^1 & \subset & \mathbb{R}^n \times \mathbb{R}^{n-1} \times \mathbb{R} \times \mathbb{R} \\ & \longrightarrow & \mathbb{R}^{n-1} \times \mathbb{R}^{n-1} \times \mathbb{R} \times \mathbb{R} \\ (x, q, \vartheta, \sigma) & \longmapsto & \tilde{\Psi}(x, q, \vartheta, \sigma). \end{array} $$

By construction $G$ is a submersion at $\bar{\xi}^{\bar{\tau}} := (\bar{x}^{\bar{\tau}}, \bar{q}^{\bar{\tau}} := \hat{p}^{\bar{\tau}}, 0, 0)$, and the fact that $\dot{x}(\bar{\tau}) = e_1^n$ gives that the kernel of $dG$ at $\bar{\xi}^{\bar{\tau}}$ is the vector line $\mathbb{R}e_1^{2n+1}$. As in the proof of Proposition 3.1, we check that the following result holds (the proof of Lemma 4.3 is postponed to Appendix E.2):
**Lemma 4.3.** *Assumption (B.14) is satisfied.*

Now, restricting $\mathcal{V}^0$ if necessary, we can assume that there is $\bar{\mu} > 0$ such that, for any starting point

$$ (x^0 = (0, \hat{x}^0), q^0) \in \hat{\mathcal{W}}^0 := \{ (x, q) | (x, \psi(x, q, 0)) \in \mathcal{V}^0 \}, $$

any time $T \in (\bar{\tau} - \bar{\mu}, \bar{\tau} + \bar{\mu})$, and any control $v: [0, T] \to \mathbb{R}^n$ of class $C^\infty$ with $\|v\|_{C^1} < \bar{\mu}$, we have

$$ (x(t), q(t)) \in \hat{\mathcal{W}} \quad \forall t \in [0, T] $$

for every solution $(x(t), q(t), \vartheta(t), \sigma(t))$ of (4.9) starting at $\xi^0 := (x^0, q^0, 0, \bar{\sigma}(x^0, p^0))$, where $p^0 := \psi(x^0, q^0, 0)$ and $\bar{\sigma}(x^0, p^0)$ was defined in (4.6).

Recall that $\bar{\xi}^\tau = (\bar{x}^\tau, \bar{q}^\tau, 0, 0)$, and for every

$$ (x^0 = (0, \hat{x}^0), q^0) \in \hat{\mathcal{W}}^0, \quad T \in (\bar{\tau} - \bar{\mu}, \bar{\tau} + \bar{\mu}), $$

denote by $E^{\xi^0, T} = E^{(x^0, p^0), T}$ the End-Point mapping associated with $\xi^0 = (x_0, q_0, 0, \bar{\sigma}(x^0, p^0))$ in time $T$ (see (4.7)). From Theorem B.5, there are $\delta \in (0, \bar{\mu})$ such that $B^{2n-1}(\bar{\xi}^0, \delta) \subset \mathcal{V}^0$, constants $K_U, \Lambda, \nu > 0$, and $k := 2n$ smooth controls $v^1, \dots, v^k: [0, +\infty) \to \mathbb{R}^{n-1}$ such that

$$ \text{Supp}(v^i) \subset [\delta, \bar{\tau} - \delta] \quad \forall i = 1, \dots, k, \qquad (4.14) $$

and the following property is satisfied: For every $\xi^0 \in \mathbb{R}^n \times \mathbb{R}^{n-1}$ and $T > 0$ satisfying

$$ |\xi^0 - \bar{\xi}^0|, |T - \bar{\tau}| < \delta \qquad (4.15) $$

there is a $C^1$ function of class

$$ U^{\xi^0, T} = (U_1^{\xi^0, T}, \dots, U_k^{\xi^0, T}) : B^k(G(E^{\xi^0, T}(\bar{v}))), \nu) \to B^k(0, \Lambda), $$

whose Lipschitz constant is bounded by $K_U$, such that $U^{\xi^0, T}(G(E^{\xi^0, T}(\bar{v}))) = 0$ (we are setting $\bar{v} \equiv 0$) and

$$ (G \circ E^{\xi^0, T})(\sum_{i=1}^k U_i^{\xi^0, T}(z)v^i) = z \quad \forall z \in B^k(G(E^{\xi^0, T}(\bar{v})), \nu). $$

Hence, there exists $\bar{r} \in (0, \delta/3)$ such that, for any $r \in (0, \bar{r})$ and any vectors

$$ \xi^0 = (x^0 = (0, \hat{x}^0), q^0, 0, \bar{\sigma}(x^0, p^0)), \quad \xi = (\hat{\pi}(\phi_{\tau(x^0, p^0)}^{\bar{H}}((0, \hat{x}^0), p^0)), 0, \sigma) \qquad (4.16) $$

satisfying $|\sigma| < 2r^2$, it holds

$$ |(\tilde{\pi}(\phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0)), 0, \sigma) - G((E^{\xi^0, T}(\bar{u})))| = |\sigma| < \nu, $$

where $\tilde{\pi}(x, p) := (\hat{x}, \hat{p})$, $T := \tau(x^0, p^0)$, and

$$ \begin{align*} G((E^{\xi^0, T}(\bar{u}))) &= G(X_{(x^0, p^0)}^{0, T}(\mathcal{T}), \hat{P}_{(x_0, p^0)}^{0, T}(\mathcal{T}), \Theta_{(x^0, p^0)}^{v, T}(\mathcal{T}), \Sigma_{(x^0, p^0)}^{v, T}(\mathcal{T})) \\ &= G(\bar{\tau}, \hat{X}_{(x^0, p^0)}^{0, T}(\mathcal{T}), \hat{P}_{(x_0, p^0)}^{0, T}(\mathcal{T}), 0, \bar{\sigma}(x^0, p^0)) \\ &= (\tilde{\pi}(\phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0)), 0, 0). \end{align*} $$
Let $r \in (0, \bar{r})$, $\epsilon \in (0, 1)$, and $\xi^0, \xi$ as in (4.16) with $|\sigma| < 2r^2\epsilon$. By the discussion above, there exists a smooth control $v: [0, T] \to \mathbb{R}^n$ given by

$$v := \sum_{i=1}^{k} U_{i}^{\xi^{0}, T}(\hat{x}, q)v^{i} \qquad (4.17)$$

such that

$$(G \circ E^{\xi^0, T})(v) = (\tilde{\pi}(\phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0)), 0, \sigma).$$

By the definition of $\Phi$, this gives

$$E^{\xi^0, T^f}(v) = (\hat{\pi}(\phi_{\tau(x^0, p^0)}^{\bar{H}}(x^0, p^0)), 0, \sigma), \qquad (4.18)$$

where $T^f$ is defined by

$$T^f := \mathcal{T} - \Psi_t(E^{\xi^0, T}(v)).$$

Since the function $U^{\xi^0, T}$ is $K_U$-Lipschitz and $U^{\xi^0, T}(G(E^{\xi^0, T}(\bar{v}))) = 0$, arguing as in (3.33) we have

$$\|v\|_{C^1} \leq K_U N_U |\sigma|, \qquad (4.19)$$

where

$$N_U := \max\{\|v^i\|_{C^1} \mid i = 1, \dots, k\}. \qquad (4.20)$$

Furthermore, since $\Psi_t(E^{\xi^0, T}(\bar{u})) = 0$, if $K_t$ denotes a Lipschitz constant for $\Psi_t$ in $\hat{W}^{\bar{\tau}}$, and $K_E$ is a uniform (as $\xi^0$ and $T$ vary) local Lipschitz constant for the functions $E^{\xi^0, T}$, as in (3.35) we get

$$|T^f - T| \leq K_t K_E K_U N_U |\sigma|. \qquad (4.21)$$

Denote respectively by

$$(x(\cdot), p(\cdot), \vartheta(\cdot), \sigma(\cdot)) : [0, T^f] \to \mathcal{W} \times \mathbb{R} \quad \text{and} \quad v : [0, T^f] \to \mathbb{R}^n$$

the trajectory and the control given by Lemma 4.2. Then, there exists positive constant $\bar{K}$, depending only on the $C^2$-norm of $\bar{H}$ in a neighborhood of $(\bar{x}(\cdot), \bar{p}(\cdot)) : [0, \bar{\tau}] \to \mathbb{R}^n \times \mathbb{R}^n$, such that

$$|\langle v(t), \dot{x}(t) \rangle| + \left| \frac{d}{dt} \{ \langle v(t), \dot{x}(t) \rangle \} \right| \leq \bar{K} \|v\|_{C^1} \leq \bar{K} K_U N_U |\sigma| \quad \forall t \in [0, T^f], \qquad (4.22)$$

and

$$\int_{0}^{T^f} \langle v(t), \dot{x}(t) \rangle dt = 0. \qquad (4.23)$$

Let us now show how to construct the potential $V$ given in the statement of Proposition 4.1 from $v$. We proceed as in the proof of Proposition 3.1.

Define the function $\Gamma : [0, \bar{\tau}] \times \mathbb{R}^{n-1} \to \mathbb{R}^n$ of class $C^k$ by

$$\Gamma(t, \hat{z}) := x\left(\frac{tT^f}{\bar{\tau}}\right) + (0, \hat{z}) \quad \forall (t, \hat{z}) \in [0, \bar{\tau}] \times \mathbb{R}^{n-1}$$
and the $C^{k-1}$ control $\tilde{v} : [0, \bar{\tau}] \to \mathbb{R}^n$, with coordinates $(\tilde{v}_1, \dots, \tilde{v}_n)$, by

$$ \tilde{v}(t) := (d\Gamma(t, 0_{n-1}))^* \left( v \left( \frac{tT^f}{\bar{\tau}} \right) \right) \quad \forall t \in [0, \bar{\tau}]. \quad (4.24) $$

By construction $\tilde{v}_1(\cdot)$ is given by

$$ \tilde{v}_1(t) = \frac{T^f}{\bar{\tau}} \langle v \left( \frac{tT^f}{\bar{\tau}} \right), \dot{x} \left( \frac{tT^f}{\bar{\tau}} \right) \rangle \quad \forall t \in [0, \bar{\tau}], $$

so that (4.14) and (4.23) allow to apply Lemma 3.3. Set $V := W \circ \Gamma^{-1}$, with $W$ given by Lemma 3.3. Arguing as in the proof of Proposition 3.1, thanks to (4.18), (4.19), (4.21), (4.22), (4.24), we conclude easily that there are $\delta, \bar{r}, \in (0, 1)$ small, and $K > 0$, such that assertions (i)-(v) of Proposition 4.1 hold.

## 4.3 Remarks

Let us observe that (A4), together with the assumption $\dot{x}(\bar{\tau}) = e_1$, is intrinsic. (Although we will never use this fact, we think it is interesting to point this out.) Indeed, let $\tilde{H} : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ be a Hamiltonian of class $C^k$ with $k \ge 2$ and let $(\bar{x}(\cdot), \bar{p}(\cdot)) : [0, \bar{\tau}] \to \mathbb{R}^n \times \mathbb{R}^n$ be a solution of (3.1) which satisfies

$$ \dot{x}(\bar{\tau}) = e_1 \quad \text{and} \quad \det \left( \frac{\partial^2 \tilde{H}}{\partial \hat{p}^2}(\bar{x}^{\bar{\tau}}, \bar{p}^{\bar{\tau}}) \right) + \bar{p}_1^{\bar{\tau}} \det \left( \frac{\partial^2 \tilde{H}}{\partial p^2}(\bar{x}^{\bar{\tau}}, \bar{p}^{\bar{\tau}}) \right) \neq 0. \quad (4.25) $$

Consider a smooth diffeomorphism $\Phi : \mathbb{R}^n \to \mathbb{R}^n$, and let $\tilde{H} : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ denote the Hamiltonian obtained from $\tilde{H}$ by $\Phi$:

$$ \tilde{H}(X, P) := \tilde{H}(\Phi^{-1}(X), (d_{\Phi^{-1}(X)}\Phi)^*(P)) \quad \forall (X, P) \in \mathbb{R}^n \times \mathbb{R}^n. \quad (4.26) $$

Any Hamiltonian trajectory of $\tilde{H}$ is sent via $\Phi$ onto a trajectory of $\tilde{H}$, and it can be easily checked that if

$$ (\tilde{X}(\cdot), \tilde{P}(\cdot)) := (\Phi(\bar{x}(\cdot)), (d_{\Phi(\bar{x}(\cdot))}\Phi)^{-1} * \bar{p}(\cdot)) : [0, \bar{\tau}] \to \mathbb{R}^n \times \mathbb{R}^n \quad (4.27) $$

satisfies $\dot{\tilde{X}}(\bar{\tau}) = e_1$, then

$$ \det \left( \frac{\partial^2 \tilde{H}}{\partial \hat{P}^2}(\tilde{X}(\bar{\tau}), \tilde{P}(\bar{\tau})) \right) + \bar{P}_1(\bar{\tau}) \det \left( \frac{\partial^2 \tilde{H}}{\partial P^2}(\tilde{X}(\bar{\tau}), \tilde{P}(\bar{\tau})) \right) \neq 0. \quad (4.28) $$

Indeed, the condition $\dot{\tilde{X}}(\bar{\tau}) = \dot{x}(\bar{\tau}) = e_1$ yields that the matrix $R := d_{\bar{x}^{\bar{\tau}}} \Phi \in M_n(\mathbb{R})$ has the form

$$ R = \begin{pmatrix} 1 & w^* \\ 0_{n-1} & R' \end{pmatrix} \quad \text{with } w \in \mathbb{R}^{n-1} \quad \text{and} \quad R' \in M_{n-1}(\mathbb{R}). $$

Therefore, $\bar{P}_1(\bar{\tau}) = \bar{p}_1^{\bar{\tau}}$ and

$$ \begin{aligned} \frac{\partial^2 \tilde{H}}{\partial P^2}(\bar{X}(\bar{\tau}), \bar{P}(\bar{\tau})) &= R \frac{\partial^2 \tilde{H}}{\partial p^2}(\bar{x}^{\bar{\tau}}, \bar{p}^{\bar{\tau}}) R^* \\ &= \begin{pmatrix} 1 & w^* \\ 0_{n-1} & R' \end{pmatrix} \begin{pmatrix} * & * \\ * & \frac{\partial^2 \tilde{H}}{\partial \hat{p}^2}(\bar{x}^{\bar{\tau}}, \bar{p}^{\bar{\tau}}) \end{pmatrix} \begin{pmatrix} 1 & 0 \\ w & R'^* \end{pmatrix} \\ &= \begin{pmatrix} * & * \\ * & R' \frac{\partial^2 \tilde{H}}{\partial \hat{p}^2}(\bar{x}^{\bar{\tau}}, \bar{p}^{\bar{\tau}}) R'^* \end{pmatrix}. \end{aligned} $$
This shows that

$$
\left\{
\begin{aligned}
\det\left(\frac{\partial^2 \tilde{H}}{\partial p^2}(\bar{X}(\bar{\tau}), \bar{p}(\bar{\tau}))\right) &= \det(R)^2 \det\left(\frac{\partial^2 \tilde{H}}{\partial \bar{p}^2}(\bar{x}^{\bar{\tau}}, \bar{p}^{\bar{\tau}})\right) \\
\det\left(\frac{\partial^2 \tilde{H}}{\partial p^2}(\bar{X}(\bar{\tau}), \bar{P}(\bar{\tau}))\right) &= \det(R')^2 \det\left(\frac{\partial^2 \tilde{H}}{\partial \bar{p}'^2}(\bar{x}^{\bar{\tau}}, \bar{p}^{\bar{\tau}})\right) \\
\det(R') &= \det(R) \\
\bar{P}_1(\bar{\tau}) &= \bar{p}_1^{\bar{\tau}},
\end{aligned}
\right.
$$

which together with (4.25) implies (4.28).

# 5 Proof of Theorem 2.1

## 5.1 Introduction

Let $H: T^*M \to \mathbb{R}$ be a Tonelli Hamiltonian of class $C^k$ with $k \ge 2$, and let $\epsilon \in (0, 1)$ be fixed. Without loss of generality, up to adding a constant to $H$ which does not change the dynamics, we can assume that $c[H] = 0$. Let $L$ denote the Lagrangian associated to $H$. Our goal is to find a potential $V: M \to \mathbb{R}$ of class $C^k$ with $\|V\|_{C^2} < \epsilon$, together with a $C^1$ function $v: M \to \mathbb{R}$ and a curve $\gamma: [0, T] \to M$ with $\gamma(0) = \gamma(T)$, such that the following properties are satisfied:

$$ (P1) \quad H_V(x, dv(x)) \le 0 \quad \forall x \in M. $$

$$ (P2) \quad \int_0^T L_V(\gamma(t), \dot{\gamma}(t)) dt = 0. $$

Indeed, if we are able to do this, then (P1) implies that $c[H_V] \le 0$ (see Subsection 1.2), while (P2) together with (1.1) yields $c[L_V] = c[H_V] \ge 0$. Therefore, by (1.2) the closed curve $\Gamma := \gamma([0, T])$ is contained in the projected Aubry set of $H_V$. Now, if $W: M \to \mathbb{R}$ is any smooth function such that $W = 0$ on $\Gamma$, $W > 0$ outside $\Gamma$, and $\|W\|_{C^2} < \epsilon - \|V\|_{C^2}$, then the function $v$ is a critical subsolution of $H_{V-W} = H+V-W$ which is strict outside $\Gamma$, and we have $\int_0^T L_{V-W}(\gamma(t), \dot{\gamma}(t)) dt = 0$. By the description of the projected Aubry set given in Subsection 1.2, this implies that $A(H_{V-W})$ coincides with the periodic curve $t \mapsto \gamma(t)$, which concludes the proof.

From now on, we assume that the Aubry set $\tilde{\mathcal{A}}(H)$ does not contain an equilibrium point or a periodic orbit (otherwise, by the discussion above, the proof is trivial), and we fix $\bar{x} \in \mathcal{A}(H)$ as in the statement of the theorem. By assumption, we know that there is a critical subsolution $u: M \to \mathbb{R}$ and an open neighborhood $V$ of $O^+(\bar{x})$ such that $u$ satisfies assertions (i)-(iii) in the statement of Theorem 2.1. We set $\bar{p} := du(\bar{x})$, and define the curve $\tilde{\gamma}: \mathbb{R} \to M$ by

$$ \tilde{\gamma}(t) := \pi^* \left( \phi_t^H (\bar{x}, \bar{p}) \right) \quad \forall t \in \mathbb{R}. $$

The idea is to find a time $\bar{t} > 0$ such that, up to a change of coordinates, all assumptions (A1)-(A4) hold at $\bar{y} := \tilde{\gamma}(\bar{t})$ (here, (A1)-(A3) are the assumptions introduced in Subsection 3.1, while (A4) was introduced in Proposition 4.1), so that we can apply Propositions 3.1 and 4.1 to connect Hamiltonian trajectories by controlling the action. As we will see in Subsection 5.3, in order to close the trajectory $\tilde{\gamma}(t)$ using a potential small in $C^2$ topology we will need to apply Mai Lemma D.1. Finally, in Subsection 5.5 we will show that this closed trajectory belongs to a projected Aubry set by adding another small potential and constructing a critical viscosity subsolution.

## 5.2 Preliminary steps

First of all, we claim that there is a time $\bar{t} > 0$ such that

$$ \frac{d}{dt} \left\{ u \left( \phi_t^H (\bar{x}, \bar{p}) \right) \right\}_{|t=\bar{t}} = \langle du(\tilde{\gamma}(\bar{t})), \dot{\tilde{\gamma}}(\bar{t}) \rangle \ge 0. \quad (5.1) $$
Indeed, argue by contradiction and assume that

$$ \frac{d}{dt} \{ u(\phi_t^H(\bar{x}, \bar{p})) \} = \langle du(\bar{\gamma}(t)), \dot{\bar{\gamma}}(t) \rangle < 0 \quad \forall t > 0, $$

so that

$$ u(\bar{\gamma}(T)) - u(\bar{x}) = u(\bar{\gamma}(T)) - u(\bar{\gamma}(0)) = \int_{0}^{T} \langle du(\bar{\gamma}(t)), \dot{\bar{\gamma}}(t) \rangle dt \le -c_0 < 0 \quad \forall T \ge 1 $$

for some positive constant $c_0$. As $\bar{x}$ is recurrent we have $\lim_{k \to \infty} \int_0^{t_k} \langle du(\bar{\gamma}(t)), \dot{\bar{\gamma}}(t) \rangle dt = 0$, a contradiction.

Set $\bar{y} := \bar{\gamma}(\bar{t})$, and fix $\bar{\tau} \in (0, 1)$ small. Then, there exist an open neighborhood $U_{\bar{y}} \subset \mathcal{V}$ of $\bar{y}$ in $M$ (where $\mathcal{V}$ is as in the statement of the theorem) with $\bar{x} \notin U_{\bar{y}}$, and a smooth diffeomorphism

$$ \theta_{\bar{y}} : U_{\bar{y}} \to B^n(0, 2), $$

such that

$$ \theta_{\bar{y}}(\bar{y}) = (\bar{\tau}, 0_{n-1}) \quad \text{and} \quad \langle d\theta_{\bar{y}}(\bar{y}), \dot{\bar{\gamma}}(\bar{t}) \rangle = e_1^n. $$

Denote by $\Pi^0$ the hyperplane passing through the origin which is orthogonal to the vector $e_1$ in $\mathbb{R}^n$, let $\Pi^\bar{\tau} := \bar{\tau}e_1 + \Pi^0$, $\Pi^{3\bar{\tau}} := 3\bar{\tau}e_1 + \Pi^0$, and set

$$ \Pi_r^0 := \Pi^0 \cap B^n(0, r), \quad \Pi_r^{\bar{\tau}} := \Pi^{\bar{\tau}} \cap B^n(\bar{\tau}e_1, r), \quad \Pi_r^{3\bar{\tau}} := \Pi^{3\bar{\tau}} \cap B^n(3\bar{\tau}e_1, r) \quad \forall r > 0. $$

The Hamiltonian $H: T^*M \to \mathbb{R}$ is sent, via the smooth diffeomorphism $\theta_{\bar{y}}$, onto a Hamiltonian $\bar{H}$ of class $C^k$ on $B^n(0, 2) \times \mathbb{R}^n$. Moreover, since $U_{\bar{y}} \subset \mathcal{V}$, the critical subsolution $u: M \to \mathbb{R}$ is sent via $\theta_{\bar{y}}$ onto the $C^{1,1}$ function $\bar{u} : B^n(0, 2) \to \mathbb{R}$,

$$ \bar{u}(z) := u(\theta_{\bar{y}}^{-1}(z)) \quad \forall z \in B^n(0, 2) $$

which solves the Hamilton-Jacobi equation<fnref n="8" />

$$ \bar{H}(z, \nabla \bar{u}(z)) = 0 \quad \forall z \in B^n(0, 2). \qquad (5.2) $$

Actually, the Hamiltonian $\bar{H}$ can be seen as the restriction of a Hamiltonian $\bar{H}$ defined on $\mathbb{R}^n \times \mathbb{R}^n$ satisfying (H1)-(H3). Moreover, assuming $\bar{\tau} > 0$ sufficiently small (so that $\langle d\theta_{\bar{y}}(\bar{\gamma}(t)), \dot{\bar{\gamma}}(t) \rangle$ is sufficiently close to $e_1^n$ for $t \in [\bar{t} - \bar{\tau}, \bar{t}]$), we can modify $\theta_{\bar{y}}$ in such a way that the integral trajectory of $\bar{H}$

$$ (\bar{x}(t), \bar{p}(t)) := (\theta_{\bar{y}}(\gamma(t - \bar{t} + \bar{\tau})), (d_{\theta_{\bar{y}}(\bar{\gamma}(t - \bar{t} + \bar{\tau}))} \theta_{\bar{y}}^{-1})^* du(\bar{\gamma}(t - \bar{t} + \bar{\tau}))) \quad (5.3) $$

satisfies (A1)-(A3) over the interval $[0, \bar{\tau}/2]$ (i.e., replacing $\bar{\tau}$ by $\bar{\tau}/2$, with obvious notation), and satisfies (A1)-(A4) on $[\bar{\tau}/2, \bar{\tau}]$ (i.e., replacing 0 by $\bar{\tau}/2$)<fnref n="9" />. Moreover, by choosing $\bar{\tau}$ even smaller, we can assume that the Hamiltonian trajectory $(\bar{x}(\cdot), \bar{p}(\cdot))$ is defined from $[0, 3\bar{\tau}]$ to $B^n(0, 2) \times \mathbb{R}^n$, satisfies $\bar{x}(3\bar{\tau}) = (3\bar{\tau}, 0_{n-1})$, and moreover the following hold<fnref n="10" />:

**Lemma 5.1.** The following properties are satisfied:

<footnote n="8">⁸As in Sections 3 and 4, we identify $T^*(\mathbb{R}^n)$ with $\mathbb{R}^n \times \mathbb{R}^n$.</footnote>

<footnote n="9">⁹Observe that, thanks to the uniform convexity of $\bar{H}$ in the $p$ variable, (5.1) implies that condition (A4) holds with a strict inequality at $(\bar{\tau}, 0_{n-1})$, and then by continuity it also holds in some uniform neighborhood.</footnote>

<footnote n="10">¹⁰Properties (i)-(iv) in Lemma 5.1 are immediate to check. (v) follows observing that, if $\bar{\tau}$ is small enough, then the Poincaré map $P_t$ from $\Pi_{1/2}^0$ to $P_t(\Pi_{1/2}^0) \subset (\Pi^0 + te_1)$ is bi-Lipschitz for any $t \in [0, 3\bar{\tau}]$, with bi-Lipschitz constant bounded by 2.</footnote>
(i) the Poincaré time mapping $\mathcal{T}_{\bar{\tau}} : \Pi_{1/2}^0 \to \mathbb{R}$ satisfying $\mathcal{T}_{\bar{\tau}}(\bar{x}(0)) = \bar{\tau}$ and

$$ \phi_{\mathcal{T}_{\bar{\tau}}(z^0)}^{\bar{H}}(z^0, \nabla \bar{u}(z^0)) \in \Pi_1^{\bar{\tau}} \quad \forall z^0 \in \Pi_{1/2}^0, $$

is well-defined, Lipschitz, and valued in $(\bar{\tau}/2, 3\bar{\tau}/2)$;

(ii) the Poincaré mapping $\mathcal{P}$ defined by

$$ \mathcal{P}: \Pi_{1/2}^0 \to \Pi_1^{\bar{\tau}} \\ z^0 \longmapsto \mathcal{P}(z^0) := \pi^* \left( \phi_{\mathcal{T}_{\bar{\tau}}(z^0)}^{\bar{H}}(z^0, \nabla \bar{u}(z^0)) \right) $$

is 2-Lipschitz;

(iii) The Poincaré time mapping $\mathcal{T}_{3\bar{\tau}} : \Pi_{1/2}^0 \to \mathbb{R}$ satisfying $\mathcal{T}_{3\bar{\tau}}(\bar{x}(3\bar{\tau})) = 3\bar{\tau}$ and

$$ \phi_{\mathcal{T}_{3\bar{\tau}}(z)}^{\bar{H}}(z, \nabla \bar{u}(z)) \in \Pi_1^{3\bar{\tau}} \quad \forall z \in \Pi_{1/2}^0, $$

is well-defined, Lipschitz, and valued in $(5\bar{\tau}/2, 7\bar{\tau}/2)$;

(iv) $\forall z^0 = (0, \hat{z}^0) \in \Pi_{1/4}^0, \forall r \in (0, 1/8)$, the inclusion

$$ \mathcal{C}((z^0, \nabla \bar{u}(z^0)); \mathcal{T}_{\bar{\tau}}(z^0); r) \subset [0, \bar{\tau}] \times B^{n-1}(0_{n-1}, 1/2) $$

holds (here the “cylinder” $\mathcal{C}((z^0, \nabla \bar{u}(z^0)); \mathcal{T}_{\bar{\tau}}(z^0); r)$ is defined analogously to (3.11));

(v) $\forall z^0 = (0, \hat{z}^0) \in \Pi_{1/4}^0, \forall z = (0, \hat{z}) \in \Pi_1^0, \forall t \in (0, \mathcal{T}_{3\bar{\tau}}(z)), \forall r \in (0, 1/8)$:

$$ \pi^* \left( \phi_t^{\bar{H}}(z, \nabla \bar{u}(z)) \right) \in \mathcal{C}((z^0, \nabla \bar{u}(z^0)); \mathcal{T}_{3\bar{\tau}}(z^0); r) \implies z \in B^n(z^0, 4r/3). $$

Denote by $K_{\bar{u}}$ the $C^{1,1}$-norm of $\bar{u}$ on $B^n(0, 2)$, and recall that $\bar{u}$ is solution of (5.2). Thanks to the above discussion and combining Propositions 3.1 and 4.1, we can easily show that the following holds:

**Proposition 5.2.** With the same notation as above, there are $\bar{\delta}, \bar{r}, \bar{\epsilon} \in (0, 1/4)$ and $K > 0$ such that the following property holds: For any $r \in (0, \bar{r}), \hat{\epsilon} \in (0, \bar{\epsilon})$, $z^0 \in \Pi_1^0, z^f \in \Pi_1^{\bar{\tau}},$ and $\sigma \in \mathbb{R}$ satisfying

$$ |z^0| < \bar{\delta} \qquad (5.4) $$

and

$$ |z^f - \mathcal{P}(z^0)| < r\hat{\epsilon}, \quad |\sigma| < r^2\hat{\epsilon}, \qquad (5.5) $$

there exist a time $T^f > 0$ and a potential $V : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ such that:

(i) $\text{Supp}(V) \subset \mathcal{C}((z^0, \nabla \bar{u}(z^0)); \mathcal{T}_{\bar{\tau}}(z^0); r);$

(ii) $\|V\|_{C^2} < (K\sqrt{1+K_{\bar{u}}^2})\hat{\epsilon};$

(iii) $|T^f - \tau(x^0, p^0)| < (K\sqrt{1+K_{\bar{u}}^2})r\hat{\epsilon};$

(iv) $\phi_{T^f}^{\bar{H}}(z^0, \nabla \bar{u}(z^0)) = (z^f, \nabla \bar{u}(z^f));$
<figure><img src="image_4.png" /><figcaption>Figure 2: By using first Proposition 3.1 on $[0, \bar{\tau}/2]$ we can add a first potential to connect the trajectories, and then, by Proposition 4.1 on $[\bar{\tau}/2, \bar{\tau}]$, we can add a second potential to fit the action without changing the starting and final point of the trajectory.</figcaption></figure>

$$ (v) \quad A_V((z^0, \nabla \bar{u}(z^0)); T^f) = A((z^0, \nabla \bar{u}(z^0)); \tau(z^0, \nabla \bar{u}(z^0))) + \langle \nabla \bar{u}(\mathcal{P}(z^0)), z^f - \mathcal{P}(z^0) \rangle + \sigma. $$

*Proof.* Let us observe that, given $z^f$ satisfying (5.5), the vectors $p^f := \nabla \bar{u}(z^f)$ and $p^0 := \nabla \bar{u}(\mathcal{P}(z^0))$ satisfy

$$ |p^f - p^0| < \bar{K}_{\bar{u}} r \hat{\epsilon}, $$

so that

$$ \left| (z^f, p^f) - (\mathcal{P}(z^0), p^0) \right| < \sqrt{1 + \bar{K}_{\bar{u}}^2 r \hat{\epsilon}}. \qquad (5.6) $$

Now, if $\Phi_{\bar{\tau}}^{\bar{\tau}/2} : \Pi_{1/2}^{\bar{\tau}/2} \to \Pi_1^{\bar{\tau}/2}$ denotes the Poincaré map going “backward” from $\Pi^{\bar{\tau}}$ to $\Pi^{\bar{\tau}/2}$, we can observe that if $\bar{\tau}$ is sufficiently small (the smallness being independent of $r$ and $\hat{\epsilon}$) then

$$ \left| \Phi_{\bar{\tau}}^{\bar{\tau}/2}(z^f, p^f) - \Phi_{\bar{\tau}}^{\bar{\tau}/2}(\mathcal{P}(z^0), p^0) \right| < 2\sqrt{1 + \bar{K}_{\bar{u}}^2 r \hat{\epsilon}}. $$

Hence, since the trajectory given by (5.3) satisfy (A1)-(A3) over the interval $[0, \bar{\tau}/2]$, we can apply Proposition 3.1 on $[0, \bar{\tau}/2]$ to connect $(z_0, \nabla \bar{u}(z_0))$ to $\Phi_{\bar{\tau}}^{\bar{\tau}/2}(z^f, p^f)$ in a time $T_1^f \sim \bar{\tau}/2$ with a “default” of action bounded by $Kr^2\hat{\epsilon}^2$. Then, assuming $\bar{\epsilon}$ sufficiently small, since the trajectory given by (5.3) satisfy (A1)-(A4) over the interval $[\bar{\tau}/2, \bar{\tau}]$ we apply Proposition 4.1 on $[\bar{\tau}/2, \bar{\tau}]$ to “compensate” the default of action so that (v) above holds. Moreover it is easily seen that also all the other properties are satisfied. We leave the details to the reader. $\square$

The above lemma will be the key tool to “close” a piece of trajectory of the Aubry set and to control the action so that (P2) is satisfied. But once this construction will be performed (see Subsections 5.3 and 5.4 below), we will still need to modify the potential and our critical solution in order to obtain (P1). This will be done in Subsection 5.5 below. However, in order to be able to perform the construction of this new subsolution, we need a few preliminary results on solutions to Hamilton-Jacobi equations which we discuss below.
First of all, let us observe that in the construction of $\theta_{\bar{y}}$, $\bar{\tau}$, $\bar{H}$ made above we assumed that the Hamiltonian trajectory $(\bar{x}(\cdot), \bar{p}(\cdot))$ is defined from $[0, 3\bar{\tau}]$ to $B^n(0, 2) \times \mathbb{R}^n$ and satisfies $\bar{x}(3\bar{\tau}) = (3\bar{\tau}, 0_{n-1})$. By taking $\bar{\tau} > 0$ sufficiently small, we can further assume that the following holds (see [19]):

**Lemma 5.3.** Set $\mathcal{H}_{[0,3\bar{\tau}]} := \{z = (z_1, \hat{z}) \in \mathbb{R}^n \mid z_1 \in [0, 3\bar{\tau}]\}$. For every potential $W : \mathbb{R}^n \to \mathbb{R}$ of class $C^2$ with $\|W\|_{C^2} < 1$ and $\text{Supp}(W) \subset B^n(0, 1)$, there exists a unique solution $w : B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]} \to \mathbb{R}$ of the Dirichlet problem

$$ \left\{ \begin{array}{ll} \bar{H}(z, \nabla w(z)) + W(z) = 0 & \text{in } B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}, \\ w = \bar{u} & \text{on } \Pi_2^0. \end{array} \right. \qquad (5.7) $$

Moreover this solution can be constructed by the method of characteristics, and is of class $C^{1,1}$ on $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$.

To be more precise, given $z^0 \in \Pi_2^0$, let $(z(\cdot), q(\cdot)) : [0, t_{z_0}) \to B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]} \times \mathbb{R}^n$ be the (maximal) solution of the Hamiltonian system

$$ \left\{ \begin{array}{ll} \dot{z}(s) = \nabla_q \bar{H}_W(z(s), q(s)) = \nabla_q \bar{H}_W(z(s), q(s)) \\ \dot{q}(s) = -\nabla_z \bar{H}_W(z(s), q(s)) = -\nabla_z \bar{H}(z(s), q(s)) - \nabla W(z(s)) \end{array} \right. \qquad (5.8) $$

starting at $(z^0, \nabla \bar{u}(z^0))$. (Here $t_{z_0} > 0$ is the first time such that $z(s)$ touches the boundary of $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$.) Then we assume that the solution $w : B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]} \to \mathbb{R}$ to the Dirichlet problem (5.7) is of class $C^{1,1}$ and is given by the method of characteristics, i.e., it satisfies

$$ w(z(t)) - \bar{u}(z^0) = \int_0^t \langle q(s), \dot{z}(s) \rangle ds = \int_0^t \bar{L}(z(s), \dot{z}(s)) - W(z(s)) ds, \quad \nabla w(z(t) = q(t). $$

(We refer the reader to [11, 19] for more details on the method of characteristics.) Let us recall that the linearized Hamiltonian system along the trajectory $(z(\cdot), q(\cdot))$ is given by

$$ \left\{ \begin{aligned} \dot{\delta z}(t) &= \frac{\partial^2 \bar{H}}{\partial z \partial q}(z(t), q(t)) \delta z(t) + \frac{\partial^2 \bar{H}}{\partial q^2}(z(t), q(t)) \delta q(t) \\ \dot{\delta q}(t) &= -\frac{\partial^2 \bar{H}}{\partial z^2}(z(t), q(t)) \delta z(t) - \frac{\partial^2 \bar{H}}{\partial q \partial z}(z(t), q(t)) \delta q(t) - \text{Hess } W(z(t)). \end{aligned} \right. \qquad (5.9) $$

Moreover, $\bar{u}$ is twice differentiable at a point $z^0 = z(0) \in \Pi_2^0$ if and only if it is twice differentiable at $z(t)$ for some $t > 0$. From this fact and the Lipschitz regularity of the flow, it is not difficult to deduce that $\bar{u}$ is twice differentiable a.e. (with respect to the $(n-1)$-dimensional Lebesgue measure) on $\Pi_2^0$.

For every $z^0 = z(0)$ such that $\bar{u}$ is two times differentiable at $z^0$ and any $t \ge 0$, let $R(t) := (R_1(t), R_2(t)) : \mathbb{R}^n \to \mathbb{R}^n \times \mathbb{R}^n$ denote the linear mapping such that $R(t)(\delta z(0))$ is the unique solution of (5.9) starting at $(\delta z(0), \text{Hess } \bar{u}(z(0)) \delta z(0))$. Then it can be easily checked that $w$ is two times differentiable along $z(t)$ and that its Hessian at $z(t)$ is given by

$$ \text{Hess } w(z(t)) = R_2(t) R_1(t)^{-1} \quad \forall t \in [0, t_{z(0)}). \qquad (5.10) $$

Since $\bar{H}$ is at least $C^2$ and $R_1(0) = I_n$, we can assume without loss of generality that the matrix $R_1(t)$ is invertible and satisfies $(\bar{u}$ is two times differentiable almost everywhere with an upper bound on its Hessians):

$$ \| R_1(t) - I_n \| \le \frac{1}{4} \quad \forall t \in [0, t_{z(0)}). \qquad (5.11) $$

As we observed above, this preliminary discussion will be useful in Subsection 5.5.
In the next subsections we are going to show that there is a continuous nondecreasing function $\bar{\omega} : [0, +\infty) \to [0, +\infty)$, with $\bar{\omega}(0) = 0$, such that the following property holds: For every $\epsilon > 0$ there exists a potential $\bar{V} : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$, with $\|\bar{V}\|_{C^2} < \bar{\omega}(\epsilon)$ and $\text{Supp}(\bar{V}) \subset B^n(0, 2)$, such that the $C^k$ potential $V : M \to \mathbb{R}$ defined by

$$V(x) = \begin{cases} 0 & \text{if } x \notin \mathcal{U}_{\bar{y}} \\ \bar{V}(\theta_{\bar{y}}(x)) & \text{if } x \in \mathcal{U}_{\bar{y}} \end{cases} \qquad (5.12)$$

satisfies $c[H_V] = 0$, and $\mathcal{A}(H_V)$ is a periodic orbit.

## 5.3 Closing the Aubry set

Define the function $\Psi : [0, +\infty) \times M \to M$ by

$$\Psi(t, z) := \pi^* (\phi_t^H(z, du(z))) \quad \forall z \in M.$$

By assumption, $\Psi$ is well-defined, Lipschitz on $V$ (here, $V$ is as in the statement of the theorem), and $C^1$ at the point $(t, \bar{x})$ for any $t \ge 0^{11}$.

Let $\mathcal{U}_{\bar{x}} \subset V$ be a small neighborhood of $\bar{x}$ such that $\mathcal{U}_{\bar{x}} \cap \mathcal{U}_{\bar{y}} = \emptyset$ (here $\mathcal{U}_{\bar{y}}$ is the neighborhood of $\bar{y} = \bar{\gamma}(\bar{t})$, $\bar{t} > 0$, defined in the previous subsection). We can suppose that there exists a smooth diffeomorphism

$$\theta_{\bar{x}} : \mathcal{U}_{\bar{x}} \to B^n(0, 1)$$

such that

$$\theta_{\bar{x}}(\bar{x}) = 0_n \quad \text{and} \quad d\theta_{\bar{x}}(\bar{x})(\dot{\gamma}(\bar{t})) = e_1^n.$$

Let $\bar{\delta} > 0$ be as in Proposition 5.2, and set

$$S_{\bar{x}} := \theta_{\bar{x}}^{-1} \left( \Pi_{\bar{\delta}/2}^{0} \right), \quad S_{\bar{y}} := \theta_{\bar{y}}^{-1} \left( \Pi_{\bar{\delta}/2}^{0} \right),$$

and let $\mathcal{T}$ be the countable discrete set defined by

$$\mathcal{T} := \{ \bar{t}_i \mid i \ge 1 \} = \{ t > 0 \mid \bar{\gamma}(t) = \Psi(t, \bar{x}) \in S_{\bar{y}} \}. \qquad (5.13)$$

(Observe that $\bar{y} = \Psi(\bar{t}, \bar{x})$ is recurrent, since so is $\bar{x}$.) For every integer $i \ge 1$, there are $\delta_i \in (0, \bar{\delta}/2)$ and a Lipschitz Poincaré time mapping $\mathcal{T}_{\bar{t}_i} : \Pi_{\delta_i}^0 \to (0, +\infty)$ such that $\mathcal{T}_{\bar{t}_i}(0_{n-1}) = \bar{t}_i$ and

$$\Psi(\mathcal{T}_{\bar{t}_i}(w), \theta_{\bar{x}}^{-1}(w)) \in S_{\bar{y}} \quad \forall w \in \Pi_{\delta_i}^0. \qquad (5.14)$$

We observe that, since $u$ is $C^2$ at any point of $\mathcal{O}^+(\bar{x})$ (as observed after the statement of Theorem 2.1), the maps $\mathcal{T}_{\bar{t}_i}$ are $C^1$ at the point $0_{n-1}$. Then the Poincaré mappings

$$\Phi_i : \Pi_{\delta_i}^0 \longrightarrow \Pi_{\bar{\delta}/2}^0 \\ w \longmapsto \theta_{\bar{y}}(\Psi(\mathcal{T}_{\bar{t}_i}(w), \theta_{\bar{x}}^{-1}(w))),$$

are well-defined, Lipschitz, and $C^1$ at the point $0_{n-1}$. Moreover, for every $i \ge 1$ the map $\Phi_i$ is

$$\mathcal{D}\Psi(t, x) := \text{conv} \left( \left\{ \lim_{k \to \infty} D\Psi(t_k, x_k) \mid (t_k, x_k) \to (t, x), (t_k, x_k) \in \text{Dom}(D\Psi) \right\} \right),$$

and we say that “$\Psi$ is $C^1$ at a point $(t, x)$” if $\mathcal{D}\Psi(t, x)$ is a singleton. We note that the assumption of $u$ being $C^2$ at $\bar{x}$ (and so at any point of $\mathcal{O}^+(\bar{x})$, as observed after the statement of Theorem 2.1) implies that $\Psi$ is $C^1$ at $(t, \bar{x})$ for any $t \ge 0$.

<footnote n="11">¹¹The definition of being “$C^1$ at one point” is analogous to the definition of “$C^2$ at one point” given right before Theorem 2.1. More precisely, let $\text{Dom}(D\Psi) \subset V$ be the set of points where $\Psi$ is differentiable (which is of full measure). Then its generalized differential at a point $(t, x) \in [0, +\infty) \times V$ is defined as</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 3: The point $\bar{y}_i$ corresponds to the ith-intersection of the curve $\bar{\gamma}(t) = \Psi(t, \bar{x})$, $t > 0$, with the hypersurface $S_{\bar{y}}$. Observe also that $\bar{y}$ does not belong to $S_{\bar{y}}$, since by definition $\theta_{\bar{y}}(\bar{y}) = (\bar{\tau}, 0_{n-1})$, while $\theta_{\bar{y}}(S_{\bar{y}}) = \Pi_{\delta/2}^0$.</figcaption></figure>

a bi-Lipschitz homeomorphism from $\Pi_{\delta_i}^0 \subset \mathbb{R}^{n-1}$ onto an open neighborhood (in $\mathbb{R}^{n-1}$) of

$$ \bar{w}_i := \theta_{\bar{y}}(\bar{y}_i), \quad \bar{y}_i := \Psi(\bar{T}_{\bar{t}_i}(0_{n-1}), \bar{x}) = \Psi(\bar{t}_i, \bar{x}). \qquad (5.15) $$

Set $P_i := \mathcal{D}\Phi_i(0_{n-1})$ the generalized differential of $\Phi_i$ at $w = 0_{n-1}$, and define<fnref n="12" />

$$ E_i := \{ w \in \mathbb{R}^{n-1} \mid |P_i(w)| \le \|P_i\| \}. $$

The following lemma is a simple consequence of the $C^1$ regularity of $\Phi_i$ at $0_{n-1}$:

**Lemma 5.4.** For every integer $i \ge 1$ there exists $r_i \in (0, \delta_i)$ such that, for any $w, w' \in B^{n-1}(0_{n-1}, r_i)$, we have

$$ \forall \mu > 0: \quad w' \in w + \mu E_i \implies \Phi_i(w') \in B^{n-1}(\Phi_i(w), 2\mu), $$

$$ \forall \nu > 0: \quad w' \notin w + 2\nu E_i \implies \Phi_i(w') \notin B^{n-1}(\Phi_i(w), \nu). $$

*Proof of Lemma 5.4.* Since $\Phi_i$ is $C^1$ at $0_{n-1}$, it is simple to check that then any element of $\mathcal{D}\Phi_i(w)$ has to converge to $P_i$ as $w \to 0_{n-1}$. In particular, we can find $r_i \in (0, \delta_i)$ such that

$$ \|L - P_i\| \le \frac{1}{\|P_i^{-1}\|} \quad \forall w \in B^{n-1}(0_{n-1}, r_i), L \in \mathcal{D}\Phi_i(w). $$

Fix $w, w'$ in $B^{n-1}(0_{n-1}, r_i)$ and $\mu > 0$ such that $w' \in w + \mu E_i$. By the Mean Value Inequality applied to the function $[0, 1] \ni s \mapsto \Phi_i(w + s(w' - w))$ we infer that

$$ \begin{aligned} |\Phi_i(w') - \Phi_i(w)| & \le \max_{v \in [w, w'], L \in \mathcal{D}\Phi_i(v)} \|L\| |w' - w| \\ & \le \|P_i\| |w' - w| + \frac{1}{\|P_i^{-1}\|} |P_i^{-1} \circ P_i(w' - w)| \\ & \le 2\mu. \end{aligned} $$

Taking $r_i$ smaller if necessary, we leave the reader to show that the second property is satisfied as well. □

<footnote n="12">¹²Note that, since $\bar{x} \in \mathcal{A}(H)$, the curve $\bar{\gamma}$ minimizes the action with fixed endpoints on any time interval. In particular there are no conjugate points along $O^+(\bar{x})$, and $P_i$ is always invertible.</footnote>
Now, given $\epsilon \in (0, 1)$ fixed, set

$$ \hat{N} := \left\lfloor \frac{32K \sqrt{1 + K_{\bar{u}}^2}}{\epsilon} \right\rfloor + 1, \qquad (5.16) $$

and let $\hat{\rho} \ge 3$ and $\eta > 0$ be the numbers provided by Mai Lemma D.1 applied to the family of ellipsoids $\{E_i\}$ defined above. Hence, thanks to Lemma 5.4 and the fact that the points $\bar{w}_1, \dots, \bar{w}_\eta \in \Pi_{\delta/2}^0$ are all distinct (since the curve $\bar{\gamma}$ is not periodic), we deduce that if $0 < \bar{r} < \min\{r_1, \dots, r_\eta\}/\hat{\rho}$ is sufficiently small, then the following properties hold (recall that $\mathcal{V}$ denotes the open neighborhood of $\mathcal{O}^+(\bar{x})$ where $u$ satisfies assertions (i) and (ii) in the statement of the theorem):

(p1) For any $w \in \Pi_{\hat{\rho}\bar{r}}^0$ and $t \in [0, T_{\bar{t}_\eta}(w)]$, $\Psi(t, \theta_{\bar{x}}^{-1}(w)) \in \mathcal{V}$.

(p2) For any $w, w' \in \Pi_{\hat{\rho}\bar{r}}^0$, any $i \in \{1, \dots, \eta\}$,

$$ \forall \mu > 0: \quad w' \in w + \mu E_i \implies \Phi_i(w') \in B^{n-1}(\Phi_i(w), 2\mu), $$

$$ \forall \nu > 0: \quad w' \notin w + 2\nu E_i \implies \Phi_i(w') \notin B^{n-1}(\Phi_i(w), \nu). $$

(p3) The sets $C_i$ defined by

$$ C_i := \bigcup_{z^0 \in \Phi_i(B^{n-1}(0_{n-1}, \hat{\rho}\bar{r}))} C\left((z^0, \nabla \bar{u}(z^0)); \mathcal{T}_{3\bar{r}}(z^0); \hat{\rho}\bar{r}\right) \qquad (5.17) $$

are disjoint for $i = 1, \dots, \eta - 1$.

(p4) For every $i \in \{1, \dots, \eta\}$, $\Phi_i(B^{n-1}(0, \hat{\rho}\bar{r})) \subset B^{n-1}(0, \bar{\delta})$.

Let $\bar{r} > 0$ small enough to be chosen later. Since $\bar{x}$ is recurrent and $d\theta_{\bar{x}}(\bar{x})(\dot{\bar{\gamma}}(\bar{t})) = e_1^n$, there exist a time $T_{\bar{r}} > 0$ such that

$$ \theta_{\bar{x}}(\Psi(T_{\bar{r}}, \bar{x})) \in \Pi_{\bar{r}}^0. $$

Let us consider the set of nonnegative times

$$ T' := \{t \in [0, T_{\bar{r}}] \mid \bar{\gamma}(t) \in S_{\bar{x}}\}, $$

that is,

$$ T' = \{0 = t'_1 < t'_2 < \dots < t'_J = T_{\bar{r}}\} $$

for some integer $J \ge 1$ (actually, for $\bar{r}$ small, $J \gg \eta$). Set

$$ W := \{w_0 := \theta_{\bar{x}}(\bar{x}), w_1 := \theta_{\bar{x}}(\bar{\gamma}(t'_1)), \dots, w_J := \theta_{\bar{x}}(\bar{\gamma}(t'_J))\} \subset \Pi^0 \simeq \mathbb{R}^{n-1}. \qquad (5.18) $$

Then, by Mai Lemma D.1 applied to the ordered set $W$, there exist $\eta$ points $\hat{w}_1, \dots, \hat{w}_\eta \in \Pi^0$, and radii $\hat{r}_1, \dots, \hat{r}_\eta > 0$, such that the following properties are satisfied:

(p5) There exist $j, l \in \{0, \dots, J\}$ with $j > l$ such that $\hat{w}_1 = w_j$ and $\hat{w}_\eta = w_l$.

(p6) $\forall i \in \{1, \dots, \eta - 1\}$, $E_i(\hat{w}_i, \hat{r}_i) \subset \Pi_{\hat{\rho}\bar{r}}^0$.

(p7) $\forall i \in \{1, \dots, \eta - 1\}$, $E_i(\hat{w}_i, \hat{r}_i) \cap (W \setminus \{w_j, w_l\}) = \emptyset$.

(p8) $\forall i \in \{1, \dots, \eta - 1\}$, $\hat{w}_{i+1} \in E_i(\hat{w}_i, \hat{r}_i/\hat{N})$.
<figure><img src="image_6.png" /><figcaption>Figure 4: An illustration of Mai Lemma: there exist two points $w_j, w_l$, which can be connected using a sequence of $\eta - 1$ small ellipsoids $E_i(\hat{w}_i, \hat{r}_i/\hat{N})$, so that none of the points $w_k$ ($k \neq j, l$) belongs to $E_i(\hat{w}_i, \hat{r}_i)$ for $i = 1, \dots, \eta - 1$ (in the figure above, we just drew two of the ellipsoids $E_i(\hat{w}_i, \hat{r}_i)$).</figcaption></figure>

Fix $i \in \{1, \dots, \eta - 1\}$, and observe that by (p6) $|\hat{w}_i|, |\hat{w}_{i+1}| < \hat{\rho}\bar{r}$. Thus, thanks to (p4) and recalling the definition of $\mathcal{P}$ in Lemma 5.1(ii), we can set

$$z_i^0 := \Phi_i(\hat{w}_i), \quad z_i := \mathcal{P}(z_i^0), \quad \tilde{z}_i^0 := \Phi_i(\hat{w}_{i+1}), \quad \tilde{z}_i := \mathcal{P}(\tilde{z}_i^0)$$

(see Figure 5 below). Moreover, we also set $z_\eta^0 := \Phi_\eta(\hat{w}_\eta)$. By Lemma 5.1(ii) and properties (p2), (p4) and (p8) above, we have $|z_i^0| < \bar{\delta}$ and

$$|z_i - \tilde{z}_i| \le 2|z_i^0 - \tilde{z}_i^0| < \frac{4\hat{r}_i}{\hat{N}} < \left(\frac{\hat{r}_i}{8}\right) \left(\frac{\epsilon}{K\sqrt{1+K_u^2}}\right). \quad (5.19)$$

Therefore, thanks to Proposition 5.2$^{13}$, for every $\sigma_i \in \mathbb{R}$ (to be chosen later) such that

$$|\sigma_i| < \left(\frac{\hat{r}_i^2}{64}\right) \left(\frac{\epsilon}{K\sqrt{1+K_u^2}}\right) \quad (5.20)$$

there exist a time $T_i^f > 0$, together with a potential $\bar{V}_i : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$, such that

$$(p9) \text{ Supp}(\bar{V}_i) \subset \mathcal{C}((z_i^0, \nabla \bar{u}(z_i^0)); \mathcal{T}_{\bar{\tau}}(z_i^0); \hat{r}_i/8).$$

$$(p10) \quad \|\bar{V}_i\|_{C^2} < \epsilon.$$

$$(p11) \quad |T_i^f - \mathcal{T}_{\bar{\tau}}(z_i^0)| < \hat{r}_i \epsilon / 8.$$

$$(p12) \quad \phi_{T_i^f}^{\bar{H}_{\bar{V}_i}}(z_i^0, \nabla \bar{u}(z_i^0)) = (\tilde{z}_i, \nabla \bar{u}(\tilde{z}_i)).$$

$$(p13) \quad \mathbb{A}_{\bar{V}_i}((z_i^0, \nabla \bar{u}(z_i^0)); T_i^f) = \mathbb{A}((z_i^0, \nabla \bar{u}(z_i^0)); \mathcal{T}_{\bar{\tau}}(z_i^0)) + \langle \nabla \bar{u}(z_i), \tilde{z}_i - z_i \rangle + \sigma_i.$$

<footnote>$^{13}$Without loss of generality, we can assume that $\frac{\epsilon}{K\sqrt{1+K_u^2}} < \hat{\epsilon}$, with $\hat{\epsilon}$ given by Proposition 5.2.</footnote>
<figure><img src="image_7.png" /><figcaption>Figure 5: The point $z_i^0$ (resp. $\tilde{z}_i^0$) is obtained by considering the $i$-th intersection of the curve $t \mapsto \Phi(t, \dot{w}_i)$ (resp. $t \mapsto \Phi(t, \dot{w}_{i+1})$) with the hypersurface $S_{\bar{y}}$. Then, we use Proposition 5.2 to connect $z_i^0$ to $\tilde{z}_i$, see also Figure 2 in Subsection 5.2.</figcaption></figure>

Let us now define the $C^k$ potential $\bar{V} : \mathbb{R}^n \to \mathbb{R}$ as follows: notice that, for every $i = 1, \dots, \eta - 1$, the open set $C((z_i^0, \nabla \bar{u}(z_i^0)); \mathcal{T}_{\bar{\tau}}(z_i^0); \hat{r}_i/8)$ is contained in the set $C_i$ defined in (5.17). Hence, thanks to (p3), all the supports $\text{Supp}(\bar{V}_i)$ are disjoints. Define $\bar{V} : \mathbb{R}^n \to \mathbb{R}$ by

$$ \bar{V}(z) := \begin{cases} \bar{V}_i(z) & \text{if } z \in \text{Supp}(\bar{V}_i) \text{ for some } i \in \{1, \dots, \eta - 1\}, \\ 0 & \text{otherwise,} \end{cases} $$

and define $V : M \to \mathbb{R}$ as in (5.12). Let $\Psi^V(t, y_j)$ denote the projection onto $M$ of the Hamiltonian trajectory of $H_V$ starting at

$$ (y_j, du(y_j)) := (\theta_{\bar{x}}^{-1}(w_j), du(\theta_{\bar{x}}^{-1}(w_j))) = (\bar{\gamma}(t'_j), du(\bar{\gamma}(t'_j))), $$

i.e.,

$$ \Psi^V(t, y_j) := \pi^* (\phi_t^{H_V}(y_j, du(y_j))). $$

By construction, there is a sequence of positive times

$$ 0 < \tilde{t}_1 < \tilde{t}_1 + T_1^f < \tilde{t}_2 < \tilde{t}_2 + T_2^f < \dots < \tilde{t}_{\eta-1} < \tilde{t}_{\eta-1} + T_{\eta-1}^f < \tilde{t}_{\eta} \quad (5.21) $$

such that the corresponding states

$$ (\theta_{\bar{y}}(\Psi^V(t, y_j)), \nabla \bar{u}(\theta_{\bar{y}}(\Psi^V(t, y_j)))) $$

in $B^n(0, 2) \times \mathbb{R}^n$ are respectively given by

$$ \begin{aligned} & (z_1^0, \nabla \bar{u}(z_1^0)), (\tilde{z}_1, \nabla \bar{u}(\tilde{z}_1)), (z_2^0, \nabla \bar{u}(z_2^0)), (\tilde{z}_2, \nabla \bar{u}(\tilde{z}_2)), \\ & \dots, (z_{\eta-1}^0, \nabla \bar{u}(z_{\eta-1}^0)), (\tilde{z}_{\eta-1}, \nabla \bar{u}(\tilde{z}_{\eta-1})), (z_{\eta}^0, \nabla \bar{u}(z_{\eta}^0)). \end{aligned} \quad (5.22) $$
Recalling the definition of the times $\bar{t}_i$, see (5.13), note that

$$z_{\eta}^{0} = \Phi_{\eta}(\hat{w}_{\eta}) = \Phi_{\eta}(w_{l}) = \Phi_{\eta}(\theta_{\bar{x}}(\bar{\gamma}(t'_{l}))) = \theta_{\bar{y}}(\bar{\gamma}(t'_{l} + \bar{t}_{\eta})).$$

**Claim 1:** $t'_l + \bar{t}_\eta \le t'_j$.

Indeed, if not, since $t'_l < t'_j$ (see (p5)) there would exist $h \in (0, \bar{t}_\eta)$ such that $t'_l + \bar{t}_\eta = t'_j + h$, so that

$$z_{\eta}^{0} = \theta_{\bar{y}}(\bar{\gamma}(t'_{j} + h)) = \theta_{\bar{y}}(\Psi(h, w_{j})).$$

Let us observe that $w_j = \bar{\gamma}(t'_j) \in \Pi_{\bar{r}}^0 \subset \Pi_{\delta_i}^0$ for all $i = 1, \dots, \eta$. Hence, since $h \in (0, \bar{t}_\eta)$, by the definition of the Poincaré time mappings $\mathcal{T}_{\bar{t}_i}$ (see (5.14)) there exists $i \in \{1, \dots, \eta\}$ such that $h = \mathcal{T}_{\bar{t}_i}(w_j)$. But since $z_\eta^0 = \Phi_\eta(w_l)$, this implies that $z_\eta^0$ belongs to the intersection $\Phi_i(\Pi_{\bar{\rho}\bar{r}}) \cap \Phi_\eta(\Pi_{\bar{\rho}\bar{r}})$, which contradicts (p3).

**Claim 2:** The curve

$$t \in [t'_{l} + \bar{t}_{\eta}, t'_{j}] \mapsto \bar{\gamma}(t)$$

never intersects the support of the potential $V$.

Indeed, if not, by (p9) there would exist $t \in [t'_l + \bar{t}_\eta, t'_j]$ and $i \in \{1, \dots, \eta - 1\}$ such that

$$\theta_{\bar{y}}(\bar{\gamma}(t)) \in \text{Supp}(V_i) \subset \mathcal{C}((z_i^0, \nabla \bar{u}(z_i^0)); \mathcal{T}_{\bar{r}}(z_i^0); \hat{r}_i/8) \subset \mathcal{C}((z_i^0, \nabla \bar{u}(z_i^0)); \mathcal{T}_{3\bar{r}}(z_i^0); \hat{r}_i/4)$$

By Lemma 5.1(v), this implies that there is $t' \in [t'_l + \bar{t}_\eta, t'_j]$ such that $\theta_{\bar{y}}(\bar{\gamma}(t'))$ belongs to $B^{n-1}(z_i^0, \hat{r}_i/2) = B^{n-1}(\Phi_i(\hat{w}_i), \hat{r}_i/2)$, which together with (p2) gives

$$w := \Phi_i^{-1}(\theta_{\bar{y}}(\bar{\gamma}(t'))) \in E_i(\hat{w}_i, \hat{r}_i) \cap W. \qquad (5.23)$$

On the other hand, by the definition of $W$ (see (5.18)) there exists $\bar{j} \in \{1, \dots, J\}$ such that $w = \theta_{\bar{x}}(\bar{\gamma}(t'_{\bar{j}}))$. This means that $t' = t'_{\bar{j}} + \bar{t}_i$. Now, since $\bar{t}_i < \bar{t}_\eta$, we deduce that $t'_l + \bar{t}_\eta \le t' = t'_{\bar{j}} + \bar{t}_i$, so that $\bar{j} \ne l$. On the other hand, since $\bar{t}_1 > 0$ we have $t'_{\bar{j}} < t' \le t'_j$, so that $\bar{j} \ne j$. Hence $w = \theta_{\bar{x}}(\bar{\gamma}(t'_{\bar{j}}))$ for some $\bar{j} \notin \{j, l\}$, which together with (5.23) contradicts (p7).

Thanks to Claims 1 and 2 above, we obtain that the Hamiltonian trajectory

$$[0, +\infty) \ni t \mapsto (x(t), p(t)) := \phi_t^{H_V}(y_j, du(y_j))$$

goes from $(y_j, du(y_j)) = (\bar{\gamma}(t'_j), du(\bar{\gamma}(t'_j)))$ to $(\bar{\gamma}(t'_l + t_\eta), du(\bar{\gamma}(t'_l + t_\eta)))$ on $[0, \bar{t}_\eta]$, and then it goes back to $(y_j, du(y_j))$ on $[\bar{t}_\eta, \bar{t}_\eta + t'_j - t'_l - \bar{t}_\eta]$. Hence it is closed. The aim of the next section is to show that we can add a small potential to $H_V$ so that this closed trajectory actually belongs to the projected Aubry set.

## 5.4 Control of the action

In the previous section, given $\bar{r} > 0$ small enough, we constructed a $C^k$ potential $V : M \to \mathbb{R}$ and a $C^k$ curve $\tilde{\gamma} : [0, t_f] \to M$, $t_f := \bar{t}_\eta + t'_j - t'_l - \bar{t}_\eta$, made of two curves

$$\gamma_1 : [0, \bar{t}_\eta] \to M \quad \text{and} \quad \gamma_2 : [\bar{t}_\eta, \bar{t}_\eta + t'_j - t'_l - \bar{t}_\eta] \to M$$

given by

$$\gamma_1(t) := \pi (\phi_t^{H_V} (y_j, du(y_j))) \quad \text{for } t \in [0, \bar{t}_\eta], \qquad \gamma_2(t) := \bar{\gamma}(t + t'_l + \bar{t}_\eta - \bar{t}_\eta) \quad \text{for } t \in [\bar{t}_\eta, t_f],$$

and satisfying

$$\gamma_1(0) = y_j, \quad \gamma_1(\bar{t}_\eta) = \gamma_2(\bar{t}_\eta) = \theta_{\bar{y}}^{-1}(z_\eta^0) = \bar{\gamma}(t'_l + \bar{t}_\eta), \quad \gamma_2(t_f) = y_j.$$
Our aim is to show that, if $\bar{r} > 0$ is small enough, then the real numbers $\sigma_i$ in (p13) can be chosen in such a way that (5.20) holds and

$$ A_V(\tilde{\gamma}; [0, t_f]) := \int_0^{\tilde{t}_\eta} L_V(\gamma_1(t), \dot{\gamma}_1(t)) dt + \int_{\tilde{t}_\eta}^{t_f} L_V(\gamma_2(t), \dot{\gamma}_2(t)) dt = 0. \quad (5.24) $$

Since $\gamma_2$ is contained in the projected Aubry set $\mathcal{A}(H)$ and does not intersect the support of $V$ (see Claim 2 above), we have

$$ A_V(\gamma_2; [\tilde{t}_\eta, t_f]) = A(\gamma_2; [\tilde{t}_\eta, t_f]) = u(\gamma_2(t_f)) - u(\gamma_2(\tilde{t}_\eta)) = u(y_j) - u(\theta_{\bar{y}}^{-1}(z_\eta^0)). \quad (5.25) $$

(Here we are using that $u$ is a critical viscosity subsolution.) Let us now evaluate the quantity

$$ \Delta := \int_0^{\tilde{t}_\eta} L_V(\gamma_1(t), \dot{\gamma}_1(t)) dt - (u(\theta_{\bar{y}}^{-1}(z_\eta^0)) - u(y_j)) = A_V(\tilde{\gamma}; [0, t_f]). \quad (5.26) $$

Recalling (5.21) and (5.22) we have (recall that $\bar{u} = u \circ \theta_{\bar{y}}^{-1}$)

$$ \Delta = \left[ \int_0^{\tilde{t}_1} L_V(\gamma_1(t), \dot{\gamma}_1(t)) dt - (u(\theta_{\bar{y}}^{-1}(z_1^0)) - u(y_j)) \right] + \sum_{i=1}^{\eta-1} \left[ \int_{\tilde{t}_i}^{\tilde{t}_i+T_i^f} L_V(\gamma_1(t), \dot{\gamma}_1(t)) dt - (\bar{u}(\tilde{z}_i) - \bar{u}(z_i^0)) \right] + \sum_{i=1}^{\eta-1} \left[ \int_{\tilde{t}_i+T_i}^{\tilde{t}_i+1} L_V(\gamma_1(t), \dot{\gamma}_1(t)) dt - (\bar{u}(z_{i+1}^0) - \bar{u}(\tilde{z}_i)) \right]. $$

By construction, the curve $[0, \tilde{t}_1] \ni t \mapsto \gamma_1(t) \in M$ does not intersect the support of $V$. This shows that the first term appearing in the right hand side of the above formula equals

$$ \Delta_0 := \int_0^{\tilde{t}_1} L(\gamma_1(t), \dot{\gamma}_1(t)) dt - (u(\gamma_1(\tilde{t}_1)) - u(\gamma_1(0))). $$

Since $t \in [0, t_1] \mapsto (\gamma_1(t), du(\gamma_1(t)))$ belongs to the Aubry set $\tilde{\mathcal{A}}(H)$ and $u$ is a critical subsolution, we deduce that $\Delta_0 = 0$. On the other hand, for each $i \in \{1, \dots, \eta-1\}$, the piece of curve $\gamma_1|_{[\tilde{t}_i+T_i^f, \tilde{t}_{i+1}]}$ does not intersect the support of $V$. This shows that the last terms of the above formula equal

$$ \Delta_i := \int_{\tilde{t}_i+T_i^f}^{\tilde{t}_{i+1}} L(\gamma_1(t), \dot{\gamma}_1(t)) dt - (\bar{u}(z_{i+1}^0) - \bar{u}(\tilde{z}_i)), $$

and since $u$ is a critical solution along $\gamma_1 \subset \mathcal{A}(H)$ we deduce that $\Delta_i = 0$ as well. Finally, thanks to (p13), for every $i = 1, \dots, \eta-1$ we have

$$ \begin{align*} \delta_i &:= \int_{\tilde{t}_i}^{\tilde{t}_i+T_i^f} L_V(\gamma_1(t), \dot{\gamma}_1(t)) dt - (\bar{u}(\tilde{z}_i) - \bar{u}(z_i^0)) \\ &= A_{\bar{V}_i}((z_i^0, \nabla \bar{u}(z_i^0)); T_i^f) - (\bar{u}(\tilde{z}_i) - \bar{u}(z_i^0)) \\ &= A((z_i^0, \nabla \bar{u}(z_i^0)); \bar{T}(z_i^0)) + \langle \nabla \bar{u}(z_i), \tilde{z}_i - z_i \rangle + \sigma_i - (\bar{u}(\tilde{z}_i) - \bar{u}(z_i^0)) \\ &= [A((z_i^0, \nabla \bar{u}(z_i^0)); \bar{T}(z_i^0)) - (\bar{u}(z_i) - \bar{u}(z_i^0))] + [\langle \nabla \bar{u}(z_i), \tilde{z}_i - z_i \rangle - (\bar{u}(\tilde{z}_i) - \bar{u}(z_i))] + \sigma_i \\ &= 0 + [\langle \nabla \bar{u}(z_i), \tilde{z}_i - z_i \rangle - (\bar{u}(\tilde{z}_i) - \bar{u}(z_i))] + \sigma_i, \end{align*} $$
where for the last equality we used (p1) and the fact that $\bar{u}$ is a critical solution on $B^n(0, 2)$.
By (5.19) we deduce that $\delta_i = \alpha_i + \sigma_i$, with

$$|\alpha_i| \le K_u |\tilde{z}_i - z_i|^2 \le K_{\bar{u}} \left(\frac{\hat{r}_i^2}{64}\right) \left(\frac{\epsilon}{K\sqrt{1+K_{\bar{u}}^2}}\right)^2. \quad (5.27)$$

(Recall that $K_{\bar{u}}$ denotes the $C^{1,1}$-norm of $\bar{u}$ on $B^n(0, 2)$.) Define the $\sigma_i$'s by

$$\sigma_i := -\alpha_i \quad \forall i = 1, \dots, \eta - 1. \quad (5.28)$$

(This is an admissible choice for $\epsilon$ sufficiently small, see (5.20).) Then

$$\delta_i = 0 \quad \forall i = 1, \dots, \eta - 1, \quad (5.29)$$

and we conclude that

$$\Delta = \Delta_0 + \sum_{i=1}^{\eta-1} (\delta_i + \Delta_i) = 0,$$

as desired.

## 5.5 Construction of a critical subsolution

The constructions performed in Subsections 5.3 and 5.4 show that, given $\epsilon \in (0, 1)$, for every $\bar{r} > 0$ sufficiently small, there exist a potential $\bar{V} : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ with

$$\|\bar{V}\|_{C^2} < \epsilon, \quad \text{Supp}(\bar{V}) \subset \cup_{i=1}^{\eta-1} C\left((z_i^0, \nabla \bar{u}(z_i^0)); \mathcal{T}_{\bar{r}}(z_i^0); \hat{r}_i/8\right), \quad \hat{r}_1, \dots, \hat{r}_{\eta-1} \le \bar{r},$$

and a periodic curve $\gamma : [0, t_f] \to M$, such that property (P2) is satisfied (see Subsection 5.1), where $V : M \to \mathbb{R}$ is the $C^k$ potential given by (5.12). Moreover, by (p6) and the Lipschitz regularity of the functions $\Phi_1, \dots, \Phi_{\eta-1}$, we have (recall that $\bar{w}_i = \Phi_i(0_{n-1})$, see (5.15))

$$|z_i^0 - \bar{w}_i| \le \bar{K} \bar{r} \quad \forall i = 1, \dots, \eta - 1, \quad (5.30)$$

for some constant $\bar{K} > 0$ independent of $\bar{r}$.

Now, it remains to construct a function $v : M \to \mathbb{R}$ for which (P1) is satisfied. In fact, we have still to slightly modify the potential $V$. Given $\epsilon \in (0, 1)$, we are going to show how to build $\bar{V} : B^n(0, 2) \to \mathbb{R}$ of class $C^k$ with $\|\bar{V}\|_{C^2}$ controlled by $\epsilon$ and $\text{Supp}(\bar{V}) \subset \cup_{i=1}^{\eta-1} C\left((z_i^0, \nabla \bar{u}(z_i^0)); \mathcal{T}_{\bar{r}}(z_i^0); \hat{r}_i/4\right)$, and a function $\tilde{u} : B^n(0, 2) \to \mathbb{R}$ of class $C^{1,1}$, so that the following properties are satisfied:

$$(P1') \quad H_{V'}(x, dv'(x)) \le 0 \quad \forall x \in M.$$

$$(P2') \quad \int_0^{t_f} L_{V'}(\gamma(t), \dot{\gamma}(t)) dt = 0.$$

Here $v', V' : M \to \mathbb{R}$ are the functions defined by

$$v'(x) := \begin{cases} u(x) & \text{if } x \notin \mathcal{U}_{\bar{y}} \\ \tilde{u}(\theta_{\bar{y}}(x)) & \text{if } x \in \mathcal{U}_{\bar{y}}, \end{cases}$$

and

$$V'(x) := \begin{cases} 0 & \text{if } x \notin \mathcal{U}_{\bar{y}} \\ V(x) + \tilde{V}(\theta_{\bar{y}}(x)) & \text{if } x \in \mathcal{U}_{\bar{y}}. \end{cases}$$

This will conclude the proof of Theorem 2.1.
In order to construct the function $\tilde{u}$, we will use the results described in Subsection 5.2: let $\epsilon \in (0,1)$ and $i \in \{1, \dots, \eta - 1\}$ be fixed. We denote by $\tilde{Z}_i^0(\cdot) : [0, T_{3\bar{\tau}}(\tilde{z}_i^0)] \to B^n(0, 2)$ the projection of the solution of the Hamiltonian system

$$ \left\{ \begin{aligned} \dot{z}(t) &= \nabla_q \bar{H}(z(t), q(t)) \\ \dot{q}(t) &= -\nabla_z \bar{H}(z(t), q(t)), \end{aligned} \right. \qquad (5.31) $$

associated with $\bar{H}$ and starting at $(z_i^0, \nabla \bar{u}(z_i^0))$. Let us recall that, by the proof of Claim 2 in Subsection 5.3, the cylinder

$$ C'_i := C((z_i^0, \nabla \bar{u}(z_i^0)); T_{3\bar{\tau}}(z_i^0); \hat{r}_i/4) = \{\tilde{Z}_i^0(t) + (0, \hat{z}) \mid t \in [0, T_{3\bar{\tau}}(\tilde{z}_i^0)], |\hat{z}| < \hat{r}_i/4\}, $$

never intersects the curve $[t'_i + t_\eta, t'_j] \ni t \mapsto \gamma(t)$.

Denote by $\bar{u}_i : C'_i \to \mathbb{R}$ the unique solution to the Dirichlet problem

$$ \left\{ \begin{array}{ll} \bar{H}_{\bar{V}_i}(z, \nabla \bar{u}_i(z)) = 0 & \text{in } C'_i, \\ \bar{u}_i = \bar{u} & \text{on } C'_i \cap \Pi^0, \end{array} \right. \qquad (5.32) $$

with $\bar{V}_i$ the potential constructed in Subsection 5.3 (see Lemma 5.3). The function $\bar{u}_i$ is of class $C^{1,1}$ on $C'_i$. In addition, since $\bar{u}$ is a $C^{1,1}$ critical solution of (5.2) and $\bar{V}_i$ vanishes outside $C((z_i^0, \nabla \bar{u}(z_i^0)); T_{3\bar{\tau}}(z_i^0), \hat{r}_i/8)$ by property (p9) in Subsection 5.3, using Lemma 5.1(v) it is easily seen that $\bar{u}_i$ coincides with $\bar{u}$ in the annulus

$$ A_i := C'_i \setminus C((z_i^0, \nabla \bar{u}(z_i^0)); T_{3\bar{\tau}}(z_i^0); \hat{r}_i/6). $$

By the discussion after Lemma 5.3, any solution $(z(\cdot), q(\cdot)) : [0, +\infty) \to \mathbb{R}^n \times \mathbb{R}^n$ of the Hamiltonian system

$$ \left\{ \begin{aligned} \dot{z}(t) &= \nabla_q \bar{H}_{\bar{V}_i}(z(t), q(t)) = \nabla_q \bar{H}(z(t), q(t)) \\ \dot{q}(t) &= -\nabla_z \bar{H}_{\bar{V}_i}(z(t), v(t)) = -\nabla_z \bar{H}(z(t), q(t)) - \nabla \bar{V}_i(z(t)), \end{aligned} \right. \qquad (5.33) $$

starting at $(z^0, \nabla \bar{u}(z^0))$ with $z^0 \in \Pi_1^0$, satisfies

$$ \bar{u}_i(z(t)) - \bar{u}(z^0) = \int_0^t \langle q(s), \dot{z}(s) \rangle ds = \int_0^t \bar{L}(z(s), \dot{z}(s)) - \bar{V}_i(z(s)) ds \qquad (5.34) $$

and

$$ \nabla \bar{u}_i(z(t)) = q(t) \qquad (5.35) $$

for all $t \ge 0$ such that $z(t) \in B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$. Now, denote by $Z_i^0(\cdot) : [0, T_i^e] \to C'_i$ the solution (of class $C^k$) of the Hamiltonian system (5.33) starting at $(z_i^0, \nabla \bar{u}(z_i^0))$, where $T_i^e \in (5\bar{\tau}/2, 7\bar{\tau}/2)$ is the “exit time” for $Z_i^0(\cdot)$ with respect to $C'_i$, i.e., $Z_i^0(T_i^e) \in \partial C'_i \cap \Pi^{3\bar{\tau}}$ (see Lemma 5.1(iii)). Note that, thanks to (5.34), (5.35), properties (p9) and (p12) in Subsection 5.3, and (5.29), the following hold:

$$ (\pi 1) \quad \bar{u}_i(z) = \bar{u}(z) \quad \text{for every } z \in A_i. $$

$$ (\pi 2) \quad Z_i^0(t) = \tilde{Z}_i^0 (T_{\bar{\tau}}(z_i^0) + (t - T_i^f)) \quad \text{for every } t \in [T_i^f, T_i^e]. $$

$$ (\pi 3) \quad \bar{u}_i(Z_i^0(t)) = \bar{u}(Z_i^0(t)) \quad \text{and} \quad \nabla \bar{u}_i(Z_i^0(t)) = \nabla \bar{u}(Z_i^0(t)) \quad \text{for every } t \in [T_i^f, T_i^e]. $$

Furthermore, given $\epsilon > 0$, we can choose $\bar{r}$ sufficiently small so that the following holds:
**Lemma 5.5.** There exists a continuous nondecreasing function $\omega_0: [0, +\infty) \to [0, +\infty)$, satisfying $\omega_0(0) = 0$ and independent of $i \in \{1, \dots, \eta - 1\}$ and $\epsilon > 0$, such that

$$ \|\bar{u}_i - \bar{u}\|_{C^{1,1}(C'_i)} \le \omega_0(\epsilon). \qquad (5.36) $$

*Proof of Lemma 5.5.* For any $z^0 \in \Pi_1^0 \cap B^{n-1}(z_i^0, \hat{r}_i/4)$, denote by $(\bar{z}(\cdot, z^0), \bar{q}(\cdot, z^0))$ (resp. $(\bar{z}_i(\cdot, z^0), \bar{q}_i(\cdot, z^0))$) the solution of (5.31) (resp. (5.33)) starting at $(z^0, \nabla \bar{u}(z^0))$. Since both $\bar{u}$ and $\bar{u}_i$ are given by characteristics inside $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$ and $|\nabla \bar{V}_i(z)| < \epsilon$ for every $z \in C'_i$, Gronwall's Lemma yields a uniform constant $K_1 > 0$ such that

$$ \begin{aligned} & |(\bar{z}(t, z^0), \nabla \bar{u}(\bar{z}(t, z^0))) - (\bar{z}_i(t, z^0), \nabla \bar{u}_i(\bar{z}_i(t, z^0)))| \\ & = |(\bar{z}(t, z^0), \bar{q}(t, z^0)) - (\bar{z}_i(t, z^0), \bar{q}_i(t, z^0))| \le K_1 \epsilon, \quad (5.37) \end{aligned} $$

for every $z^0 \in \Pi_1^0 \cap B^{n-1}(z_i^0, \hat{r}_i/4)$ and $t \ge 0$ such that $\bar{z}(t, z^0)$ and $\bar{z}_i(t, z^0)$ both belong to $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$. Recalling that $K_{\bar{u}}$ denotes the $C^{1,1}$-norm of $\bar{u}$, we deduce that

$$ \begin{aligned} & |\nabla \bar{u}(\bar{z}_i(t, z^0)) - \nabla \bar{u}_i(\bar{z}_i(t, z^0))| \\ & \le |\nabla \bar{u}(\bar{z}_i(t, z^0)) - \nabla \bar{u}(\bar{z}(t, z^0))| \\ & \quad + |\nabla \bar{u}(\bar{z}(t, z^0)) - \nabla \bar{u}_i(\bar{z}_i(t, z^0))| \\ & \le (K_{\bar{u}} + 1) K_1 \epsilon. \end{aligned} $$

Since every point $z \in C'_i$ can be written as $\bar{z}_i(t, z^0)$ for some $z^0 \in \Pi_1^0 \cap B^{n-1}(z_i^0, \hat{r}_i/4)$ and $t \ge 0$, the above bound on $\nabla(\bar{u} - \bar{u}_i)$ together with $(\pi 1)$ implies

$$ \|\bar{u} - \bar{u}_i\|_{C^1(C'_i)} \le K_2 \epsilon $$

for some uniform constant $K_2 > 0$. It remains to estimate the difference between Hess $\bar{u}$ and Hess $\bar{u}_i$ at any point $C'_i$ where they both exist. To this aim, we recall that the Hessians of $\bar{u}$ and $\bar{u}_i$ can be recovered from the linearized Hamiltonian systems associated with $\bar{H}$ and $\bar{H}_{\bar{V}_i}$ (see (5.10)).

Fix $z^0 \in \Pi_0^1 \cap B^{n-1}(z_i^0, \hat{r}_i/4)$ such that $\bar{u}$ is twice differentiable at $z^0$ (this is a set of full measure on $\Pi_0^1$, as observed after (5.9)). Given $h \in \mathbb{R}^n$ with $|h| = 1$, and let

$$ (\delta \bar{z}(\cdot, z^0, h), \delta \bar{q}(\cdot, z^0, h)) \quad (\text{resp.} \quad (\delta \bar{z}_i(\cdot, z^0, h), \delta \bar{q}_i(\cdot, z^0, h))) $$

denote a solution of the linearized system (5.9) along the trajectory $(\bar{z}(\cdot, z^0), \bar{q}(\cdot, z^0))$ (resp. $(\bar{z}_i(\cdot, z^0), \bar{q}_i(\cdot, z^0))$) with $W = 0$ (resp. with $W = \bar{V}_i$), and starting at $(h, \text{Hess } \bar{u}(z^0, h))$. Since $\|\bar{V}_i\|_{C^2} < \epsilon$ and $\bar{H}$ is of class at least $C^2$, the linearized systems associated with $W = 0$ and $W = \bar{V}_i$ are close to each other: by Gronwall's Lemma there exists a nondecreasing continuous function $\omega_1: [0, +\infty) \to [0, +\infty)$, with $\omega_1(0) = 0$ and independent of $i \in \{1, \dots, \eta - 1\}$ and $\epsilon > 0$, such that<fnref n="14" />

$$ |(\delta \bar{z}(t, z^0, h), \delta \bar{q}(t, z^0, h)) - (\delta \bar{z}_i(t, z^0, h), \delta \bar{q}_i(t, z^0, h))| \le \omega_1(\epsilon), $$

as long as both $\bar{z}(t, z^0)$ and $\bar{z}_i(t, z^0)$ belong to $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$.

Denoting by $(\bar{R}_1(\cdot, z^0), \bar{R}_2(\cdot, z^0))$ and $((\bar{R}_i)_1(\cdot, z^0), (\bar{R}_i)_2(\cdot, z^0))$ the matrices associated with the two linearized systems under consideration (see the discussion after Lemma 5.3) and recalling (5.11), we deduce that there is a nondecreasing continuous function $\omega_2: [0, +\infty) \to [0, +\infty)$, with $\omega_2(0) = 0$ and independent of $i \in \{1, \dots, \eta - 1\}$ and $\epsilon > 0$, such that

$$ \begin{aligned} & \|\text{Hess } \bar{u}(\bar{z}(t, z^0)) - \text{Hess } \bar{u}_i(\bar{z}_i(t, z^0))\| \\ & = \|\bar{R}_2(t, z^0)\bar{R}_1(t, z^0)^{-1} - (\bar{R}_i)_2(t, z^0)(\bar{R}_i)_1(t, z^0)^{-1}\| < \omega_2(\epsilon), \end{aligned} $$

<footnote n="14">¹⁴The function $\omega_1$ depends on $\epsilon$ and on a uniform modulus of continuity for $\frac{\partial^2 \bar{H}}{\partial z^2}$, $\frac{\partial^2 \bar{H}}{\partial z \partial v}$, and $\frac{\partial^2 \bar{H}}{\partial v^2}$ on $B^n(0, 2) \times \{\nabla \bar{u}(B^n(0, 2))\}$.</footnote>
as long as both $\bar{z}(t, z^0)$ and $\bar{z}_i(t, z^0)$ belong to $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$. We now recall that $u$ is $C^2$ along $\mathcal{O}^+(\bar{x})$, which implies that Hess $\bar{u}$ exists and is continuous along $t \mapsto \bar{z}(t, \bar{w}_i)$ (see (5.15)). Hence, if $K_2$ denotes a uniform Lipschitz constant for the flow $(t, z_0) \mapsto \bar{z}(t, z^0)$, by (5.30) we deduce that for any $z \in C'_i$ there exists a time $t_z$ such that

$$|z - \bar{z}(t_z, \bar{w}_i)| \le 2K_2 \bar{K} \bar{r} \qquad (5.38)$$

(recall that $\hat{r}_i \le \bar{r}$). In particular, since for any $z^0 \in \Pi_0^1 \cap B^{n-1}(z_i^0, \hat{r}_i/4)$ both curves $t \mapsto \bar{z}(t, z^0)$ and $t \mapsto \bar{z}_i(t, z^0)$ remain inside $C'_i$ (at least as long as they belong to $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$), by the triangle inequality we deduce that

$$\begin{aligned} \| \text{Hess } \bar{u}(\bar{z}_i(t, z^0)) - \text{Hess } \bar{u}(\bar{z}(t, z^0)) \| &\le 2\omega_3^i(2K_2\bar{K}\bar{r}) + \| \text{Hess } \bar{u}(\bar{z}(t_{\bar{z}_i(t,z^0)}, \bar{w}_i) - \text{Hess } \bar{u}(\bar{z}(t_{\bar{z}(t,z^0)}, \bar{w}_i)) \|, \quad (5.39) \end{aligned}$$

where $\omega_3^i$ is a nondecreasing modulus of continuity for Hess $\bar{u}$ along $t \mapsto \bar{z}(t, \bar{w}_i)$ (at least as long as the curve remain inside $B^n(0, 1) \cap \mathcal{H}_{[0,3\bar{\tau}]}$)$^{15}$, and $t_z$ is as in (5.38).

We now observe that, thanks to (5.37), there exists a uniform constant $K_3 > 0$ such that $|t_{\bar{z}_i(t,z^0)} - t_{\bar{z}(t,z^0)}| \le K_3\epsilon$. Moreover, the last term in the right hand side of (5.39) can be written in terms of the linearized system only. Hence, there exists a nondecreasing continuous function $\omega_4: [0, +\infty) \to [0, +\infty)$, with $\omega_4(0) = 0$ and independent of $i \in \{1, \dots, \eta - 1\}$ and $\epsilon > 0$, such that

$$\| \text{Hess } \bar{u}(\bar{z}(t_{\bar{z}_i(t,z^0)}, \bar{w}_i) - \text{Hess } \bar{u}(\bar{z}(t_{\bar{z}(t,z^0)}, \bar{w}_i)) \| \le \omega_4(K_3\epsilon).$$

Thus, by combining the above estimates together and choosing $\bar{r}$ sufficiently small (the smallness may depend on $\epsilon$), we get

$$\begin{aligned} \| \text{Hess } \bar{u}(\bar{z}_i(t, z^0)) - \text{Hess } \bar{u}_i(\bar{z}_i(t, z^0)) \| &\le \| \text{Hess } \bar{u}(\bar{z}_i(t, z^0)) - \text{Hess } \bar{u}(\bar{z}(t, z^0)) \| \\ &\quad + \| \text{Hess } \bar{u}(\bar{z}(t, z^0)) - \text{Hess } \bar{u}_i(\bar{z}_i(t, z^0)) \| \\ &\le \omega_3^i(4K_2\bar{K}\bar{r}) + \omega_4(K_3\epsilon) + \omega_2(\epsilon) \\ &\le 2[\omega_4(K_3\epsilon) + \omega_2(\epsilon)]. \end{aligned}$$

Since a.e. $z \in C'_i$ can be written as $\bar{z}_i(t, z^0)$ for some $t \ge 0$ and $z^0 \in \Pi_0^1 \cap B^{n-1}(z_i^0, \hat{r}_i/4)$ belonging to a set of full measure (which is independent of $t$), we conclude easily. $\square$

Thanks to $(\pi 1)-(\pi 3)$ and the lemma above, we will see that, by adding a suitable potential supported inside the cylinder $C'_i \cap \{(t, \hat{z}) | t \in [\bar{\tau}, 3\bar{\tau}]\}$, we can “glue” together $\bar{u}_i$ and $\bar{u}$ so that they coincide outside $C'_i$ and the new function is a critical subsolution. Moreover the potential that we add will vanish together with its gradient along $Z_i^0$, so that the curve $t \mapsto Z_i^0(t)$ will still be an extremal curve for the new Hamiltonian.

More precisely, we claim that there exist a continuous nondecreasing function $\omega: [0, +\infty) \to [0, +\infty)$, satisfying $\omega(0) = 0$ and independent of both $i \in \{1, \dots, \eta - 1\}$ and $\epsilon > 0$, a function $\tilde{u}_i: C'_i \to \mathbb{R}$ of class $C^{1,1}$, and a potential $\tilde{V}_i: C'_i \to \mathbb{R}$ of class $C^k$, such that the following properties are satisfied:

$$(\pi 4) \quad \bar{H}_{\tilde{V}_i}(z, \nabla \tilde{u}_i(z)) + \tilde{V}_i(z) \le 0 \quad \text{for every } z \in C'_i.$$

$$(\pi 5) \quad \text{Supp}(\tilde{V}_i) \subset C'_i \cap \{(t, \hat{z}) | t \in [\bar{\tau}, 3\bar{\tau}]\}.$$

$$(\pi 6) \quad \| \tilde{V}_i \|_{C^2(C'_i)} \le \omega(\epsilon).$$

$$(\pi 7) \quad \tilde{u}_i(Z_i^0(t)) = \bar{u}_i(Z_i^0(t)) = \bar{u}(Z_i^0(t)) \text{ and } \nabla \tilde{u}_i(Z_i^0(t)) = \nabla \bar{u}_i(Z_i^0(t)) = \nabla \bar{u}(Z_i^0(t)) \text{ for all } t \in [T_i^f, T_i^e].$$

<footnote>$^{15}$Observe that $\omega_3^i$ may depend on $\epsilon$, since the $C^2$ regularity of $u$ along the orbit $\mathcal{O}^+(\bar{x})$ is a priori not uniform. However this is not a problem since, once $\epsilon$ has been fixed, we can choose $\bar{r}$ as small as desired.</footnote>
$$ (\pi 8) \quad \tilde{V}_i(Z_i^0(t)) = \nabla \tilde{V}_i(Z_i^0(t)) = 0 \text{ for all } t \in [T_i^f, T_i^e]. $$

$$ (\pi 9) \quad \tilde{u}_i(z) = \bar{u}_i(z) = \bar{u}(z) \text{ for every } z \in A_i. $$

$$ (\pi 10) \quad \tilde{u}_i(z) = \bar{u}_i(z) \text{ for every } z = (z_1, \hat{z}) \in C_i' \text{ with } z_1 \in [0, 3\bar{\tau}/2]. $$

$$ (\pi 11) \quad \tilde{u}_i(z) = \bar{u}(z) \text{ for every } z = (z_i, \hat{z}) \in C_i' \text{ with } z_1 \in [5\bar{\tau}/2, 3\bar{\tau}]. $$

To construct such a potential, let us consider $\Theta : B^n(0, 2) \to [0, 1]$ a smooth function such that

$$ \left\{ \begin{array}{ll} \Theta(z) = \Theta(z_1) = 1 & \text{if } z_1 \in [0, 3\bar{\tau}/2], \\ \Theta(z) = \Theta(z_1) = 0 & \text{if } z_1 \in [5\bar{\tau}/2, 3\bar{\tau}], \end{array} \right. $$

and define $\tilde{u}_i : C_i' \to \mathbb{R}$ by

$$ \tilde{u}_i(z) := \Theta(z)\bar{u}_i(z) + (1 - \Theta(z))\bar{u}(z) \quad \forall z \in C_i'. $$

<figure><img src="image_8.png" /><figcaption>Figure 6: The function $\tilde{u}_i$ is obtained by interpolating (using a cut-off function) between $\bar{u}$ (the viscosity solution for $\bar{H}$) and $\bar{u}_i$ (the viscosity solution for $\bar{H}_{\bar{V}_i}$) inside the “cylinder” $C_i' := C((z_i^0, \nabla \bar{u}(z_i^0)) ; \mathcal{T}_{3\bar{\tau}}(z_i^0) ; \hat{r}_i/4)$. Then, by adding a new potential $\tilde{V}_i$, small in $C^2$ topology and supported inside $C_i' \cap \{z = (z_1, \hat{z}) | z_1 \in [\bar{\tau}, 3\bar{\tau}]\}$, we can ensure that $\bar{H}_{\bar{V}_i+\bar{V}_i}(z, \nabla \tilde{u}_i(z)) \le 0$ on the whole ball $B^n(0, 2)$. Since the cylinders $C_i'$ are disjoint, we can repeat this construction for $i = 1, \dots, \eta - 1$ to find $\bar{u} : B^n(0, 2) \to \mathbb{R}$ and $\bar{V} : B^n(0, 2) \to \mathbb{R}$ so that (P1') and (P2') hold.</figcaption></figure>

By construction, $\tilde{u}_i$ is of class $C^{1,1}$ on the cylinder $C_i'$. Moreover, for every $z \in C_i'$ we have

$$ \nabla \tilde{u}_i(z) = (\bar{u}_i(z) - \bar{u}(z)) \nabla \Theta(z) + \Theta(z) \nabla \bar{u}_i(z) + (1 - \Theta(z)) \nabla \bar{u}(z) \quad \forall z \in C_i'. $$

By (π1), (π3) and the definition of $\Theta$, assertions (π7) and (π9)-(π11) are satisfied. Moreover, since

$$ \text{Supp}(\bar{V}_i) \subset C_i' \cap \{z = (z_1, \hat{z}) | z_1 \in [0, \bar{\tau}]\}, $$

both $\bar{u}, \bar{u}_i$ are solutions of the Hamilton-Jacobi equation associated with $\bar{H}$ on the cylinder

$$ C_i'' := C_i' \cap \{z = (z_1, \hat{z}) | z_1 \in [\bar{\tau}, 3\bar{\tau}]\}, $$
so
$$ \bar{H}(z, \nabla \tilde{u}_i(z)) \le 0 \quad \text{on } C_i' \cap \{z = (z_1, \hat{z}) \mid z_1 \in [\bar{\tau}, 3\bar{\tau}/2] \cup [5\bar{\tau}/2, 3\bar{\tau}]\}. $$

Moreover, by the convexity of $\bar{H}$ in the $p$ variable,

$$ \bar{H}(z, \Theta(z)\nabla \tilde{u}_i(z) + (1-\Theta(z))\nabla \bar{u}(z)) \le 0 \quad \forall z \in C_i'', $$

which gives

$$ \bar{H}(z, \nabla \tilde{u}_i(z)) \le K' |\bar{u}_i(z) - \bar{u}(z)| \quad \text{on } C_i' \cap \{z = (z_1, \hat{z}) \mid z_1 \in [3\bar{\tau}/2, 5\bar{\tau}/2]\}, $$

for some uniform constant $K' > 0$ depending only on $\frac{\partial \bar{H}}{\partial p}$ and $\nabla \Phi$. Recalling (π3) and (5.36), we deduce the existence of a uniform constant $K'' > 0$ such that

$$ |\bar{u}_i(z) - \bar{u}(z)| \le K'' \omega_0(\epsilon) \operatorname{dist}(z, \Gamma_i)^2 \quad \text{on } C_i' \cap \{z = (z_1, \hat{z}) \mid z_1 \in [3\bar{\tau}/2, 5\bar{\tau}/2]\}, $$

where $\Gamma_i := \{Z_i^0(t) | t \in [T_i^f, T_i^e]\}$ and $\operatorname{dist}(\cdot, \Gamma_i)$ denotes the distance function to the curve $\Gamma_i$. Again by (5.36) and (π1), there is a uniform constant $K''' > 0$ such that

$$ |\bar{u}_i(z) - \bar{u}(z)| \le K''' \omega_0(\epsilon) \operatorname{dist}(z, \partial_{\operatorname{lat}} C_i'')^2, $$

where $\partial_{\operatorname{lat}} C_i''$ denotes the “lateral boundary” of $C_i''$, i.e.,

$$ \partial_{\operatorname{lat}} C_i'' := \{ \bar{z}_i(t) + (0, \hat{z}) \mid t \in [\bar{\tau}, 3\bar{\tau}], |\hat{z}| = \hat{r}_i/4 \}. $$

All in all, we have proved

$$ \bar{H}(z, \nabla \tilde{u}_i(z)) \le \begin{cases} 0 & \text{on } C_i' \cap \{z_1 \in [\bar{\tau}, 3\bar{\tau}/2] \cup [5\bar{\tau}/2, 3\bar{\tau}]\}, \\ \omega_0(\epsilon) \min\{K''\operatorname{dist}(z, \Gamma_i)^2, K'''\operatorname{dist}(z, \partial_{\operatorname{lat}} C_i'')^2\} & \text{on } C_i' \cap \{z_1 \in [3\bar{\tau}/2, 5\bar{\tau}/2]\}. \end{cases} $$

Thanks to this estimate and recalling that $Z_i^0$ is of class $C^k$, we easily deduce the existence of a nondecreasing function $\omega: [0, +\infty) \to [0, +\infty)$ with $\omega(0) = 0$, and a potential $\tilde{V}_i: C_i' \to \mathbb{R}$ of class $C^k$, satisfying $(\pi4)-(\pi6)$ and $(\pi8)$.

Repeating this construction for $i = 1, \dots, \eta - 1$, since the sets $C_i'$ are disjoint we obtain a function $\tilde{u} : \mathbb{R}^n \to \mathbb{R}$ of class $C^{1,1}$, together with a potential $\tilde{V} : \mathbb{R}^n \to \mathbb{R}$ of class $C^k$ with $\|\tilde{V}\|_{C^2} < \bar{\omega}(\epsilon)$ and $\operatorname{Supp}(\tilde{V}) \subset \cup_{i=1}^{\eta-1} C_i'$ (so that $\operatorname{Supp}(\tilde{V})$ never intersects $\gamma$; see Claim 2 in Subsection 5.3), such that both properties (P1') and (P2') are satisfied. This concludes the proof of Theorem 2.1.

# 6 Proof of Theorem 2.4

We use the same notation as in the proof of Theorem 2.1.

## 6.1 Preliminary step

Recall that $\dim M = 2$, $H: T^*M \to \mathbb{R}$ is a Tonelli Lagrangian of class $C^k$ with $k \ge 2$, $L: TM \to \mathbb{R}$ is its associated Lagrangian, and $\epsilon > 0$ is fixed. As is the proof of Theorem 2.1, we can assume that $c[H] = 0$ and that $\tilde{A}(H)$ does not contain an equilibrium point or a periodic orbit. Fix $\bar{x}$ as in the statement of the theorem. By assumption, there is a critical subsolution $u: M \to \mathbb{R}$ and an open neighborhood $V$ of $O^+(\bar{x})$ such that $u$ is at least $C^{k+1}$ on $V$. Define $V_0: V \to \mathbb{R}$ by

$$ V_0(x) := -H(x, du(x)) \quad \forall x \in V. $$
By the assumptions on $u$, the potential $V_0$ is of class $C^k$, nonnegative, and $u$ is a critical solution of

$$H(x, du(x)) + V_0(x) = 0 \quad \forall x \in \mathcal{V}. \qquad (6.1)$$

Hence, by the proof of Theorem 2.1 (applied to the Hamiltonian $H+V_0$ inside $\mathcal{V}$), given $\bar{r}, \epsilon > 0$ small enough, there exist an open set $\mathcal{U} := \mathcal{U}_{\bar{y}} \subset \mathcal{V}$ (here $\mathcal{U}_{\bar{y}}$ is as in Subsection 5.2), a potential $V_\epsilon : M \to \mathbb{R}$ of class $C^k$, a function $v : M \to \mathbb{R}$ of class $C^{1,1}$, and a closed curve $\gamma : [0, t_f] \to M$ such that the following properties are satisfied:

$$(\bar{\pi}1) \quad \|V_\epsilon\|_{C^2} < \epsilon/2.$$

$$(\bar{\pi}2) \quad \text{Supp}(V_\epsilon) \subset \mathcal{U}.$$

$$(\bar{\pi}3) \quad H(x, dv(x)) + V_0(x) = 0 \quad \text{for every } x \in M \setminus \mathcal{U}.$$

$$(\bar{\pi}4) \quad H(x, dv(x)) + V_0(x) + V_\epsilon(x) \le 0 \quad \text{for every } \mathcal{U}.$$

$$(\bar{\pi}5) \quad \int_0^{t_f} L(\gamma(t), \dot{\gamma}(t)) - V_0(\gamma(t)) - V_\epsilon(\gamma(t)) dt = 0.$$

Moreover, recalling the construction of the curve $\gamma$, it is easily seen that there is some constant $K > 0$ such that the closed curve $\gamma$ is made of two curves

$$\gamma_1 : [0, \tilde{t}_\eta] \to M \quad \text{and} \quad \gamma_2 : [\tilde{t}_\eta, t_f] \to M$$

(see Subsection 5.4) which satisfy<fnref n="16" />

$$(\bar{\pi}6) \quad \text{For every } t \in [\tilde{t}_\eta, t_f], \gamma(t) = \gamma_2(t) \in \mathcal{A}(H);$$

$$(\bar{\pi}7) \quad \text{dist}(\gamma_1(t), \bar{\Gamma}_1) \le K\bar{r} \quad \text{for all } t \in [0, \tilde{t}_\eta].$$

Here $\bar{\Gamma}_1 := \bar{\gamma}([0, \tilde{t}_\eta])$, where $\tilde{t}_\eta$ denotes the positive time such that $\bar{\gamma}(\tilde{t}_\eta) = \bar{y}_\eta$, see (5.15). Furthermore, we notice that the number $\bar{r} > 0$, appearing in assertion $(\bar{\pi}7)$ above, can be chosen as small as we wish.

## 6.2 Modification of the potential and conclusion

In the previous subsection we found a potential $W := V_0 + V_\epsilon$ of class $C^k$ associated with a closed curve $\gamma : [0, t_f] \to M$ which corresponds to the Aubry set for the Hamiltonian $H+W$ inside $\mathcal{V}$. Now, the strategy is to construct a new potential $V_1 : M \to \mathbb{R}$ of class $C^k$ such that the following properties are satisfied:

$$(\bar{\pi}8) \quad \|V_1\|_{C^2} < \epsilon/2.$$

$$(\bar{\pi}9) \quad V_1(x) \le V_0(x) \quad \text{for every } x \in \mathcal{V}.$$

$$(\bar{\pi}10) \quad V_1(x) = 0 \quad \text{for every } x \in M \setminus \mathcal{V}.$$

$$(\bar{\pi}11) \quad V_1(\gamma(t)) = V_0(\gamma(t)) \quad \text{for every } t \in [0, t_f].$$

<footnote n="16">¹⁶The existence of the constant $K > 0$ is a consequence of the following facts:</footnote>

<footnote>- the function $(t, x) \mapsto \Psi(t, x) = \pi^* (\phi_t^H(x, du(x)))$ is well-defined and of class $C^1$ in a neighborhood of $[0, +\infty) \times \{\bar{x}\}$;</footnote>

<footnote>- the curve $\gamma_1$ is contained in the image by $\Psi$ of a bounded interval (since, once $\epsilon > 0$ is fixed, the number $\eta$ is fixed and given by Mai Lemma) times a small ball (see (p6) in Subsection 5.3).</footnote>

<footnote>However, let us remark that, for our purposes, instead of (vii) it would suffice to know that $\text{dist}(\gamma_1(t), \bar{\Gamma}_1) \to 0$ as $r \to 0$, which is clearly true.</footnote>
Assuming that we are able to perform such a construction, we will define the potential $V : M \to \mathbb{R}$ by

$$V := V_1 + V_\epsilon.$$

Observe that $V$ is $C^k$, and by $(\tilde{\pi}1)$ and $(\tilde{\pi}8)$ it satisfies $\|V\|_{C^2} < \epsilon$. Moreover, by $(\tilde{\pi}2)-(\tilde{\pi}4)$ and $(\tilde{\pi}9)$ we have

$$H(x, dv(x)) + V(x) \le H(x, dv(x)) + V_0(x) + V_\epsilon(x) \le 0 \quad \forall x \in \mathcal{V},$$

while the nonnegativity of $V_0$ together with $(\tilde{\pi}2)$, $(\tilde{\pi}3)$ and $(\tilde{\pi}10)$ yields

$$H(x, dv(x)) + V(x) = H(x, dv(x)) = -V_0(x) \le 0 \quad \forall x \in M \setminus \mathcal{V}.$$

Finally, by $(\tilde{\pi}5)$ and $(\tilde{\pi}11)$,

$$\int_0^{t_f} L(\gamma(t), \dot{\gamma}(t)) - V(\gamma(t)) dt = \int_0^{t_f} L(\gamma(t), \dot{\gamma}(t)) - V_0(\gamma(t)) - V_\epsilon(\gamma(t)) dt = 0.$$

This shows that $\gamma : [0, t_f] \to M$ is contained in the Aubry set for the new Hamiltonian $H_V$, and we conclude the proof of Theorem 2.4 by adding a smooth potential $W$, small in $C^2$-topology, which vanishes on $\gamma$ and is strictly positive outside (see Subsection 5.1). Hence we are left with the construction of $V_1$, that we perform in the next subsection.

## 6.3 Construction of the potential

Let us recall that the function $V_0 : \mathcal{V} \to \mathbb{R}$ is of class $C^k$ with $k \ge 2$, is nonnegative, and vanishes on $\mathcal{A}(H)$. Hence we immediately deduce that

$$V_0 = dV_0 = 0 \quad \text{on } \mathcal{A}(H).$$

Since $\bar{x} = \tilde{\gamma}(0)$ is a recurrent point of $\mathcal{A}(H)$ and $M$ is two-dimensional, it is easy to show the existence of a continuous nondecreasing function $\omega : [0, +\infty) \to [0, +\infty)$, with $\omega(0) = 0$, such that

$$\|d^2V_0(x)\|_x < \omega(\text{dist}(x, \bar{\Gamma}_1)) \quad \forall x \in \mathcal{V} \qquad (6.2)$$

(see also Remark 6.2 below), where $\bar{\Gamma}_1 \subset \mathcal{A}(H)$ has been defined after $(\tilde{\pi}7)$. Then, the existence of a potential $V_1 : M \to \mathbb{R}$ of class $C^k$, satisfying properties $(\tilde{\pi}8)-(\tilde{\pi}11)$ above, follows from $(\tilde{\pi}7)$ and from the following general lemma (whose proof is postponed to Appendix E.3) applied to $N = M$, $C = \bar{\Gamma}_1$, $\mathcal{O} = \mathcal{V}$, $g = V_0$ and $A = \gamma_1|_{[0, \tilde{t}_n]}$.

**Lemma 6.1.** Let $N$ be a smooth compact Riemannian manifold without boundary of dimension $n \ge 2$, $\mathcal{O} \subset N$ be open, and $C \subset \mathcal{O}$ compact. Let $g : \mathcal{O} \to \mathbb{R}$ be a nonnegative function of class $C^k$ with $k \ge 2$ satisfying

$$g = dg = 0 \quad \text{on } C, \quad \|d^2g(x)\|_x < \omega(\text{dist}(x, C)) \quad \forall x \in \mathcal{O} \qquad (6.3)$$

for some continuous nondecreasing function $\omega : [0, +\infty) \to [0, +\infty)$ with $\omega(0) = 0$. Then, for every $\epsilon' > 0$ there is $r > 0$ such that the following holds: Let $A$ be a closed set satisfying

$$\text{dist}(x, C) \le r \quad \forall x \in A. \qquad (6.4)$$

Then there exists a function $h : N \to \mathbb{R}$ of class $C^k$ such that:

(a) $0 \le h(x) \le g(x)$ for every $x \in \mathcal{O}$.

(b) $\text{Supp}(h) \subset \mathcal{O}$.
(c) $\|h\|_{C^2} < \epsilon'$.

(d) $h(x) = g(x)$ for every $x \in A$.

**Remark 6.2.** Let us point out that the whole argument given above, together with Lemma 6.1, holds true in arbitrary dimension, with the exception of (6.2). Indeed, the fact that $\bar{x}$ is recurrent implies that, for every $t \in [0, \bar{t}_{\eta}]$, there are points of $\mathcal{A}(H)$ which are arbitrarily close to $\bar{\gamma}(t)$ and “transversal” to $\bar{\gamma}$. In two dimension this implies that $d^2V_0 = 0$ on $\bar{\Gamma}_1$, from which (6.2) follows by continuity. On the hand, in higher dimension we can only deduce that $d^2V_0$ is small in the “directions tangent to $\mathcal{A}(H)$”. This fact creates much more difficulties, since in order to establish the analogue of Lemma 6.1 we will need to know that the connecting trajectories can be chosen to belong to “the tangent space to $\mathcal{A}(H)$”. This delicate construction is performed in [27].

# 7 Final comments

In [12], Contreras and Iturriaga proved the following: let $H : T^*M \to \mathbb{R}$ be a Hamiltonian of class $C^k$, $k \ge 3$, whose Aubry set is an equilibrium point (resp. a periodic orbit). Then, there is a smooth potential $V : M \to \mathbb{R}$, with $\|V\|_{C^k}$ as small as desired, such that the Aubry set of $H_V$ is a hyperbolic equilibrium (resp. a hyperbolic periodic orbit). In view of our results we obtain:

**Theorem 7.1.** Let $H : T^*M \to \mathbb{R}$ be a Tonelli Hamiltonian of class $C^k$ with $k \ge 3$, and fix $\epsilon > 0$. Assume that there are a recurrent point $\bar{x} \in \mathcal{A}(H)$, a critical viscosity subsolution $u : M \to \mathbb{R}$, and an open neighborhood $V$ of $O^+(\bar{x})$ such that one of the following properties is satisfied:

(i) $u$ is of class at least $C^1$ on $V$, $Hess^g u(\bar{x})$ is a singleton, and $H(x, du(x)) = c[H]$ for all $x \in V$.

(ii) $\dim M = 2$ and $u$ is of class $C^{k+1}$ on $V$.

Then, there exists a potential $V : M \to \mathbb{R}$ of class $C^k$, with $\|V\|_{C^2} < \epsilon$, such that $c[H_V] = c[H]$ and the Aubry set of $H_V$ is either a hyperbolic equilibrium or a hyperbolic periodic orbit.

In [6] Bernard proved that if the Aubry set of a Tonelli Hamiltonian $H : T^*M \to \mathbb{R}$ of class $C^k$, with $k \ge 2$, is a finite union of hyperbolic periodic orbits or equilibria, then at least one critical viscosity solution is of class $C^k$ in a neighborhood of $\mathcal{A}(H)$. Furthermore, Contreras and Iturriaga showed in [12] that if $V$ is a potential of class $C^2$ such that $\tilde{\mathcal{A}}(H_V)$ is a hyperbolic equilibrium or a hyperbolic periodic orbit, then there exists $\epsilon > 0$ such that the same property holds for every $W : M \to \mathbb{R}$ with $\|W\|_{C^2} < \epsilon$. Thus, thanks to Theorem 2.1, we can more or less consider that the Mañé Conjecture in $C^2$ topology for Hamiltonians of class at least $C^3$ is equivalent to the:

**Mañé regularity Conjecture for viscosity solutions.** For every Tonelli Hamiltonian $H : T^*M \to \mathbb{R}$ of class $C^k$, with $k \ge 3$, there is a set $\mathcal{D} \subset C^3(M)$ which is dense in $C^2(M)$ (with respect to the $C^2$ topology) such that the following holds: For every $V \in \mathcal{D}$, there are a recurrent point $\bar{x} \in \mathcal{A}(H)$, a critical viscosity subsolution $u : M \to \mathbb{R}$, and an open neighborhood $V$ of $O^+(\bar{x})$ such that $u$ is of class $C^2$ on $V$ and satisfies $H(x, du(x)) = c[H]$ for all $x \in V$.

By the extension to arbitrary dimension of Theorem 7.1(ii) performed in [27], the Mañé Conjecture in $C^2$ topology is also equivalent to an analogous version of the “Mañé regularity Conjecture” above, replacing smooth critical solution by smooth critical subsolution (see [27, Section 1]).
Let us note that, by a recent result of Fathi [22], the existence of a critical viscosity subsolution of class $C^k$ in a neighborhood of the projected Aubry set is equivalent to the existence of a global critical subsolution of class $C^k$ on $M$. We stress that the main assumption in Theorem 2.4 is only concerned with the regularity of a critical viscosity subsolution in a neighborhood of a positive orbit (which is not a closed set), which is a much weaker hypothesis than the existence of a critical viscosity subsolution which is of class $C^k$ in a neighborhood of the projected Aubry set. For instance, on a 2-torus, such an assumption is not in contradiction with Denjoy-type obstructions for the existence of regular critical subsolutions [20, Theorem 8.1].

A Conventions and standing notation

• $M$ is a smooth compact manifold without boundary of dimension $n \ge 2$.

• We denote by $TM$ the tangent bundle and by $\pi:TM \to M$ the canonical projection. A point in $TM$ is denoted by $(x,v)$, with $x \in M$ and $v \in T_x M = \pi^{-1}(x)$. In the same way, a point of the cotangent bundle $T^*M$ is denoted by $(x,p)$, with $x \in M$ and $p \in T_x^*M$ a linear form on the vector space $T_x M$. The canonical projection on $T^*M$ is denoted by $\pi^*:T^*M \to M$. For every $p \in T_x^*M$, $\langle p, v \rangle$ denotes the evaluation of $p$ at $v \in T_x M$.

• We suppose that $g$ is a fixed smooth Riemannian metric on $M$. For $v \in T_x M$, the norm $\|v\|_x$ is $g_x(v, v)^{1/2}$. We also denote by $\|\cdot\|_x$ the dual norm on $T^*M$.

• For every integer $k \ge 1$, we denote by $\cdot$ or $\langle \cdot, \cdot \rangle$ the Euclidean scalar product, and by $|\cdot|$ the Euclidean norm on $\mathbb{R}^k$. We denote by $B^k$ the open unit ball and by $\bar{B}^k$ the closed unit ball in $\mathbb{R}^k$. For every $x \in \mathbb{R}^k$ and $r > 0$, we set $B^k(x, r) := \{x' \in \mathbb{R}^k \mid |x' - x| < r\}$ and $S^k(x, r) := \{x' \in \mathbb{R}^k \mid |x' - x| = r\}$. Sometimes, for sake of simplicity, we denote the ball $B^k(x, r)$ (resp. the sphere $S^k(x, r)$) by $B(x, r)$ (resp. $S(x, r)$), or simply $B_r$ (resp. $S_r$) when $x = 0$. Given a linear mapping $P: \mathbb{R}^k \to \mathbb{R}^k$, we denote by $\|P\|$ its norm with respect to $|\cdot|$, that is $\|P\| := \max\{|P(x)| \mid x \in \bar{B}^k\}$.

• For every $k, l \ge 1$, $M_{k,l}(\mathbb{R})$ denotes the vector space of real matrices with $k$ rows and $l$ columns. If $k = l$, we simply set $M_k(\mathbb{R}) = M_{k,l}(\mathbb{R})$. Furthermore, $0_{k,l}$ denotes the zero matrix in $M_{k,l}(\mathbb{R})$, $0_k$ the zero vector in $\mathbb{R}^k$, and $e_1^k, \dots, e_k^k$ the canonical basis in $\mathbb{R}^k$. If there is no possible confusion, we denote the latter by $e_1, \dots, e_k$. For every $M \in M_{k,l}(\mathbb{R})$, $M^*$ denotes the transpose matrix in $M_{l,k}(\mathbb{R})$.

• For every $k \ge 0$, we denote by $C^k(M)$ the space of functions of class $C^k$ from $M$ to $\mathbb{R}$. Given a function $F \in C^k(M)$, we denote by $d^i F$ its derivative of order $i$ for every $i = 1, \dots, k$, and we denote by $\|F\|_{C^k}$ its $C^k$-norm (computed with respect to the metric $g$).

• Most of the time we work in local charts. If $F: \Omega \to \mathbb{R}^l$ is of class $C^1$ on the open set $\Omega \subset \mathbb{R}^k$, $dF(y)$ or $\frac{\partial F}{\partial y}(y)$ denotes its Jacobian matrix (which belongs to $M_{l,k}(\mathbb{R}))$ at $y \in \Omega$. If $F$ is real valued (i.e., $l = 1$), we denote by $\nabla F(y) = dF(y) \in \mathbb{R}^k$ its gradient and by Hess $F(y) = d^2F(y)$ its Hessian at $y$. If a $C^1$ function $F$ depends on several variables $(y_1, \dots, y_m)$, $\frac{\partial F}{\partial y_i}(y_1, \dots, y_m)$ denotes the partial derivative of $F$ with respect to the $y_i$ variable evaluated at the point $(y_1, \dots, y_m)$.

• Given a Hamiltonian $H: T^*M \to \mathbb{R}$ of class $C^k$ (with $k \ge 2$) satisfying (H1) and (H2) (see Subsection 1.2), we denote by $\phi_t^H$ the Hamiltonian flow on $T^*M$. We recall that the Lagrangian $L: TM \to \mathbb{R}$ associated with $H$ is defined by

$$L(x, v) := \max_{p \in T_x^*M} \{\langle p, v \rangle - H(x, p)\}.$$
Therefore the Fenchel inequality is always satisfied $\langle p, v \rangle \le L(x, v) + H(x, p)$. Moreover, we have equality in the Fenchel inequality if and only if

$$ (x, p) = \mathcal{L}(x, v), $$

where $\mathcal{L}: TM \to T^*M$ denotes the Legendre transform defined as

$$ \mathcal{L}(x, v) := \left( x, \frac{\partial L}{\partial v}(x, v) \right) \quad \forall (x, v) \in TM. $$

Under our assumption $\mathcal{L}$ is a diffeomorphism of class at least $C^{k-1}$. We denote by $\phi_t^L$ the Euler-Lagrange flow of $L$ on $TM$, it is of class $C^{k-1}$ and conjugated with the Hamiltonian flow $\phi_t^H$.

* Given a topological space $X$, we denote by $C_c(X)$ the vector space of compactly supported continuous function on $X$. The set $\mathcal{P}(X)$ denotes the space of measures on $X$. It corresponds to the dual space $C_c(X)^*$. The weak-* topology over $\mathcal{P}(X)$ is the topology of simple convergence, that is

$$ \mu_k \to \mu \iff \int_X f d\mu_k \to \int_X f d\mu, \quad \forall f \in C^0(X). $$

We recall that the support of a measure $\mu$ is defined as the (closed) set of points $x \in X$ such that the $\mu$-measure of every neighborhood of $x$ is positive.

# B Controllability of nonlinear control systems

## B.1 Preliminaries

Given $N, m \ge 1$, let us consider a nonlinear control system in $\mathbb{R}^N$ of the form

$$ \dot{\xi} = F_0(\xi) + \sum_{i=1}^{m} u_i F_i(\xi) \quad \text{for a.e. } t, \qquad (B.1) $$

where the state $\xi(t)$ belongs to $\mathbb{R}^N$, $t \mapsto \xi(t)$ is an absolutely continuous curve, the control $u(t) = (u_1(t), \dots, u_m(t))$ belongs to $\mathbb{R}^m$, and the functions $F_0, F_1, \dots, F_m: \Omega \subset \mathbb{R}^N \to \mathbb{R}^N$ are $C^1$-vector fields defined on an open set $\Omega$. Given $\tilde{\xi} \in \Omega$ and $\bar{u} \in L^1([0, +\infty); \mathbb{R}^m)$, the Cauchy problem

$$ \left\{ \begin{array}{l} \dot{\xi}(t) = F_0(\xi(t)) + \sum_{i=1}^{m} \bar{u}_i(t) F_i(\xi(t)) \quad \text{for a.e. } t, \\ \xi(0) = \tilde{\xi}, \end{array} \right. \qquad (B.2) $$

possesses a unique maximal solution $\xi_{\tilde{\xi}, \bar{u}}(\cdot) \subset \Omega$ defined on a maximal interval of the form $[0, T_{\tilde{\xi}, \bar{u}})$, with $T_{\tilde{\xi}, \bar{u}} \in [0, +\infty]$. Given $\tilde{\xi} \in \Omega$ and $\bar{T} > 0$, we denote by $\mathcal{U}_{\tilde{\xi}, \bar{T}}$ the set of controls $u \in L^1([0, +\infty); \mathbb{R}^m)$ such that $\bar{T} < T_{\tilde{\xi}, u}$. The set $\mathcal{U}_{\tilde{\xi}, \bar{T}}$ is an open (possibly empty) subset of $L^1([0, +\infty); \mathbb{R}^m)$.

Fix $G: \Omega \to \mathbb{R}^k$ a function of class $C^1$, and $\bar{u}$ a smooth control in $\mathcal{U}_{\tilde{\xi}, \bar{T}}$. Our aim is to give sufficient conditions on $F_0, F_1, \dots, F_m$, and $G$, for partial controllability of the control system (B.1) with respect to $G$. Roughly speaking, this amounts to showing that, for any neighborhood $V \subset \mathcal{U}_{\tilde{\xi}, \bar{T}}$ of $\bar{u}$ in $L^1([0, \bar{T}]; \mathbb{R}^m)$, the set

$$ \{G(\xi_{\tilde{\xi}, u}(\bar{T})) \mid u \in V\} $$

is a neighborhood of $G(\xi_{\tilde{\xi}, \bar{u}}(\bar{T}))$. Most of the results presented below cannot be found in classical references of control theory. However, we encourage the reader to have a look at the book [14] (see also the forthcoming book [45]) for more details about the material discussed in the next subsections.
## B.2 Singular controls

Assume that the set is $\mathcal{U}_{\bar{\xi}, \bar{T}}$ is nonempty. The *End-Point mapping* associated with $\bar{\xi}$ in time $\bar{T}$ is defined as

$$ E^{\bar{\xi}, \bar{T}} : \begin{array}{rcl} \mathcal{U}_{\bar{\xi}, \bar{T}} & \to & \Omega \\ u & \mapsto & \xi_{\bar{\xi}, u}(\bar{T}). \end{array} $$

Since $F_0, F_1, \dots, F_m$ are of class $C^1$, the map $E^{\bar{\xi}, \bar{T}}$ is $C^1$ on its domain, and its differential at $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}}$ is given by the linear operator

$$ dE^{\bar{\xi}, \bar{T}}(\bar{u}) : \begin{array}{rcl} L^1([0, \bar{T}]; \mathbb{R}^m) & \to & \mathbb{R}^N \\ v & \mapsto & \zeta(\bar{T}), \end{array} $$

where $\zeta(\cdot)$ is the unique solution to the Cauchy problem

$$ \begin{cases} \dot{\zeta}(t) = A(t)\zeta(t) + B(t)v(t) & \text{for a.e. } t \in [0, \bar{T}], \\ \zeta(0) = 0, & \end{cases} \qquad (B.3) $$

and the matrices $A(t) \in M_N(\mathbb{R})$ and $B(t) \in M_{N,m}(\mathbb{R})$ are defined by

$$ A(t) := dF_0(\bar{\xi}(t)) + \sum_{i=1}^{m} \bar{u}_i(t) dF_i(\bar{\xi}(t)), \qquad (B.4) $$

$$ B(t) := (F_1(\bar{\xi}(t)), \dots, F_m(\bar{\xi}(t))), \qquad (B.5) $$

with $\bar{\xi}(t) := \xi_{\bar{\xi}, \bar{u}}(t)$. In other terms, the differential of $E^{\bar{\xi}, \bar{T}}$ at $\bar{u}$ corresponds to the End-Point mapping associated with the system obtained by linearizing (B.1) along $(\bar{\xi}, \bar{u})$ with initial condition 0 at time $t = 0$. We can also represent $dE^{\bar{\xi}, \bar{T}}(\bar{u})$ as

$$ \langle dE^{\bar{\xi}, \bar{T}}(\bar{u}), v \rangle := S(\bar{T}) \int_{0}^{\bar{T}} S(t)^{-1} B(t) v(t) dt \quad \forall v \in L^1([0, \bar{T}]; \mathbb{R}^m), \qquad (B.6) $$

where $S(\cdot)$ is the solution to the Cauchy problem

$$ \begin{cases} \dot{S}(t) = A(t)S(t), \\ S(0) = I_n. \end{cases} \qquad (B.7) $$

A control $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}}$ is said to be *singular* with respect to $E^{\bar{\xi}, \bar{T}}$ if $dE^{\bar{\xi}, \bar{T}}(\bar{u})$ is not surjective. Otherwise, $\bar{u}$ is said to be *nonsingular* or *regular* (with respect to $E^{\bar{\xi}, \bar{T}}$). The concept of singular control plays a crucial role for regularity issues (see for example [10]). Let us define the pre-Hamiltonian $H_0 : \mathbb{R}^N \times \mathbb{R}^N \times \mathbb{R}^m \to \mathbb{R}$ by

$$ H_0(\xi, p, u) := \langle p, F_0(\xi) \rangle + \sum_{i=1}^{m} u_i \langle p, F_i(\xi) \rangle. $$

Adopting Hamiltonian formalism, we have the following well-known characterization of singular controls:

**Proposition B.1.** A control $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}}$ is singular with respect to $E^{\bar{\xi}, \bar{T}}$ if and only if there exists an absolutely continuous arc $p : [0, \bar{T}] \to \mathbb{R}^N \setminus \{0\}$ such that

$$ \left\{ \begin{array}{rcl} \dot{\bar{\xi}}(t) & = & \nabla_p H_0(\bar{\xi}(t), p(t), \bar{u}(t)) \\ \dot{p}(t) & = & -\nabla_{\bar{\xi}} H_0(\bar{\xi}(t), p(t), \bar{u}(t)) \end{array} \right. \qquad (B.8) $$
for a.e. $t \in [0, \bar{T}]$, and

$$ \nabla_u H_0(\bar{\xi}(t), p(t), \bar{u}(t)) = 0 \quad \forall t \in [0, \bar{T}]. \qquad (B.9) $$

In fact, if $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}}$ is singular with respect to $E^{\bar{\xi}, \bar{T}}$, then for every $\bar{p} \in (\mathrm{Im}(dE^{\bar{\xi}, \bar{T}}))^{\perp} \setminus \{0\} \subset \mathbb{R}^n \setminus \{0\}$, there is an absolutely continuous arc $p : [0, \bar{T}] \to \mathbb{R}^N \setminus \{0\}$, with $p(\bar{T}) = \bar{p}$, which satisfies (B.8) and (B.9).

**Proof.** If $dE^{\bar{\xi}, \bar{T}}(\bar{u})$ is not surjective, then there exists $\bar{p} \in \mathbb{R}^N \setminus \{0\}$ such that, for any $v \in L^1([0, \bar{T}]; \mathbb{R}^m)$,

$$ \langle dE^{\bar{\xi}, \bar{T}}(\bar{u}), v \rangle \cdot \bar{p} = 0. $$

By (B.6), this can be written as

$$ \int_{0}^{\bar{T}} \bar{p}^{*} S(\bar{T}) S(t)^{-1} B(t) v(t) dt = 0 \quad \forall v \in L^{1}([0, \bar{T}]; \mathbb{R}^{m}). $$

Taking $v(t) := (\bar{p}^* S(\bar{T}) S(t)^{-1} B(t))^*$ ($v(t)$ is continuous on $[0, \bar{T}]$, so it belongs to $L^1([0, \bar{T}]; \mathbb{R}^m)$), we deduce that

$$ \int_{0}^{\bar{T}} \left| (\bar{p}^* S(\bar{T}) S(t)^{-1} B(t))^* \right|^2 dt = 0, $$

which implies

$$ \bar{p}^* S(\bar{T}) S(t)^{-1} B(t) = 0 \quad \forall t \in [0, \bar{T}]. \qquad (B.10) $$

Set, for each $t \in [0, \bar{T}]$,

$$ p(t) := (S(t)^{-1})^* S(\bar{T})^* \bar{p}. \qquad (B.11) $$

By construction the arc $p : [0, \bar{T}] \to \mathbb{R}^N$ is absolutely continuous, and by (B.10) it satisfies (B.9). Moreover, since $p \neq 0$ and $S(t)$ is invertible for all $t \in [0, \bar{T}]$, $p(t)$ never vanishes. Finally, noticing that $\frac{d}{dt}(S(s)^{-1})^* = -A(t)^* (S(t)^{-1})^*$ for a.e. $t \in [0, \bar{T}]$ (see (B.7)), recalling the definition of $A(t)$ we conclude that $p$ satisfies (B.8).

Conversely, let us assume that there exists some absolutely continuous arc $p : [0, \bar{T}] \to \mathbb{R}^N \setminus \{0\}$ which satisfies (B.8) and (B.9). By the discussion above this means

$$ \dot{p}(t) = -A(t)^* p(t) \quad \text{for a.e. } t \in [0, \bar{T}], $$

and

$$ p(t)^* B(t) = 0 \quad \forall t \in [0, \bar{T}]. $$

Setting $\bar{p} := p(\bar{T}) \neq 0$, for any $t \in [0, \bar{T}]$ we have

$$ p(t) = (S(t)^{-1})^* S(\bar{T})^* \bar{p}, $$

so that

$$ \bar{p}^* S(\bar{T}) S(t)^{-1} B(t) = 0. $$

This implies

$$ \langle dE^{\bar{\xi}, \bar{T}}(\bar{u}), v \rangle \cdot \bar{p} = 0 \quad \forall v \in L^1([0, \bar{T}]; \mathbb{R}^m) $$

and concludes the proof. $\square$

Let us remark that, given a control $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}}$ and the associated trajectory $\bar{\xi} = \xi_{\bar{\xi}, \bar{u}} : [0, \bar{T}] \to \mathbb{R}^N$, we have

$$ \left\{ \begin{array}{l} \nabla_{\bar{\xi}} H_0(\bar{\xi}(t), p(t), \bar{u}(t)) = A(t)^* p(t), \\ \nabla_p H_0(\bar{\xi}(t), p(t), \bar{u}(t)) = F_0(\bar{\xi}(t)) + B(t) \bar{u}(t), \\ \nabla_u H_0(\bar{\xi}(t), p(t), \bar{u}(t)) = B(t)^* p(t), \end{array} \right. $$

for any $t \in [0, \bar{T}]$ and any continuous curve $t \mapsto p(t) \in \mathbb{R}^N$. Consequently, a control $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}}$ is singular if and only if there exists an absolutely continuous arc $p : [0, \bar{T}] \to \mathbb{R}^N \setminus \{0\}$ such that
• (B.8) is satisfied a.e. on $[0, \bar{T}]$,

• $p(t)$ is orthogonal to each vector $F_1(\bar{\xi}(t)), \dots, F_m(\bar{\xi}(t))$ on $[0, \bar{T}]$.

## B.3 Application to partial controllability I

The characterization of singular controls given by Proposition B.1 allows to give sufficient conditions for partial controllability of nonlinear systems. First, given $G : \Omega \to \mathbb{R}^k$ a function of class $C^1$, we provide a result which gives a sufficient condition for the map $G \circ E^{\bar{\xi}, \bar{T}}$ to be a submersion at $\bar{u}$. Then, in the next section we explain how it implies partial controllability. Although all the following results hold for controls which are only $L^1$, in order to avoid technical issues which would come from the fact that some identities hold only almost everywhere, we will assume that the controls are continuous. This is enough for the applications we have in mind.

We recall that, given $X, Y$ two smooth vector fields on $\mathbb{R}^N$, their Lie bracket $[X, Y]$ at a point $\xi \in \mathbb{R}^N$ is defined as

$$[X, Y](\xi) := dY(\xi)(X(\xi)) - dX(\xi)(Y(\xi)).$$

Moreover, we recall that $S(t)$ is given by (B.7).

**Theorem B.2.** Let $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}} \cap C([0, T]; \mathbb{R}^m)$, assume that $G$ is a submersion at $\bar{\xi}(\bar{T}) = E^{\bar{\xi}, \bar{T}}(\bar{u})$, and that there exists $\bar{t} \in [0, \bar{T}]$ such that

$$\operatorname{Span}\{[F_0, F_i](\bar{\xi}(\bar{t})) + \left[\sum_{j=1}^{m} \bar{u}_i(\bar{t})F_j, F_i\right](\bar{\xi}(\bar{t})) \mid i = 1, \dots, m\} \\ + \operatorname{Span}\{F_i(\bar{\xi}(\bar{t})) \mid i = 1, \dots, m\} + S(\bar{t})S(\bar{T})^{-1}\operatorname{Ker}(dG(\bar{\xi}(\bar{T}))) = \mathbb{R}^N \quad (B.12)$$

Then the differential of the mapping $G \circ E^{\bar{\xi}, \bar{T}} : \mathcal{U}_{\bar{\xi}, \bar{T}} \to \mathbb{R}^k$ at $\bar{u}$ is onto.

*Proof of Theorem B.2.* Since by assumption $G$ is a submersion at $\bar{\xi}(\bar{T}) = E^{\bar{\xi}, \bar{T}}(\bar{u})$, it suffices to show that, if (B.12) is satisfied, then

$$\mathrm{Im}(dE^{\bar{\xi}, \bar{T}}(\bar{u})) + \mathrm{Ker}(dG(\bar{\xi}(\bar{T}))) = \mathbb{R}^N. \quad (B.13)$$

We argue by contradiction. If (B.13) does not hold, there exists a vector $\bar{p} \in \mathbb{R}^N \setminus \{0\}$ such that

$$\bar{p} \perp \{ \mathrm{Im}(dE^{\bar{\xi}, \bar{T}}(\bar{u})) + \mathrm{Ker}(dG(\bar{\xi}(\bar{T}))) \}.$$

Then, by Proposition B.1 there exists an absolutely continuous arc $p : [0, \bar{T}] \to \mathbb{R}^N \setminus \{0\}$ with $p(\bar{T}) = \bar{p}$ which satisfies (B.8) and (B.9). In particular, by (B.9) we know that

$$\langle p(t), F_i(\bar{\xi}(t)) \rangle = 0 \quad \forall t \in [0, \bar{T}], i = 1, \dots, m.$$
Fix $i \in \{1, \dots, m\}$. Differentiating the above equality and using (B.8) yields

$$
\begin{align*}
0 &= \frac{d}{dt} \langle p(t), F_i(\bar{\xi}(t)) \rangle = \langle \dot{p}(t), F_i(\bar{\xi}(t)) \rangle + \langle p(t), dF_i(\bar{\xi}(t))(\dot{\bar{\xi}}(t)) \rangle \\
&= -\langle dF_0(\bar{\xi}(t))^{*} p(t) + \sum_{j=1}^{m} \bar{u}_j(t) dF_j(\bar{\xi}(t))^{*} p(t), F_i(\bar{\xi}(t)) \rangle \\
&\quad + \langle p(t), dF_i(\bar{\xi}(t))(F_0(\bar{\xi}(t))) + \sum_{j=1}^{m} \bar{u}_j(t) dF_i(\bar{\xi}(t))(F_j(\bar{\xi}(t))) \rangle \\
&= -\langle p(t), dF_0(\bar{\xi}(t))(F_i(\bar{\xi}(s))) \rangle + \langle p(t), dF_i(\bar{\xi}(t))(F_0(\bar{\xi}(t))) \rangle \\
&\quad + \sum_{j=1}^{m} \bar{u}_j(t) (-\langle p(t), dF_j(\bar{\xi}(t))(F_i(\bar{\xi}(t))) \rangle + \langle p(t), dF_i(\bar{\xi}(t))(F_j(\bar{\xi}(t))) \rangle) \\
&= \langle p(t), [F_0, F_i](\bar{\xi}(t)) \rangle + \sum_{j=1}^{m} \bar{u}_j(t) \langle p(t), [F_j, F_i](\bar{\xi}(t)) \rangle \\
&= \langle p(t), [F_0, F_i](\bar{\xi}(t)) + \left[ \sum_{j=1}^{m} \bar{u}_j(t) F_j, F_i \right](\bar{\xi}(t)) \rangle \quad \forall t \in [0, \bar{T}].
\end{align*}
$$

Finally, since $\bar{p} \perp \text{Ker}(dG(\bar{\xi}(\bar{T})))$ and the arc $p: [0, \bar{T}] \to \mathbb{R}^N$ is given by

$$p(t) = (S(t)^{-1})^* S(\bar{T})^* \bar{p},$$

we obtain

$$p(t) \perp S(t)^{-1} S(\bar{T})^{-1} \text{Ker}(dG(\bar{\xi}(\bar{T}))) \quad \forall t \in [0, \bar{T}].$$

This contradicts (B.12) and concludes the proof. □

Notice that, assuming $\bar{u} \equiv 0$ and that (B.12) is satisfied at final time, yields:

**Corollary B.3.** If $\bar{u} \equiv 0 \in \mathcal{U}_{\bar{\xi}, \bar{T}}$, $G$ is a submersion at $\bar{\xi}(\bar{T}) = E^{\bar{\xi}, \bar{T}}(\bar{u})$, and

$$\text{Span}\{F_i(\bar{\xi}(\bar{T})), [F_0, F_i](\bar{\xi}(\bar{T})) \mid i = 1, \dots, m\} + \text{Ker}(dG(\bar{\xi}(\bar{T}))) = \mathbb{R}^N, \quad (B.14)$$

then the differential of the mapping $G \circ E^{\bar{\xi}, \bar{T}} : \mathcal{U}_{\bar{\xi}, \bar{T}} \to \mathbb{R}^k$ at $\bar{u} \equiv 0$ is onto.

## B.4 Application to partial controllability II

Let us now explain how a simple application of the Inverse Function Theorem yields partial controllability.

**Theorem B.4.** Let $\bar{u} \in \mathcal{U}_{\bar{\xi}, \bar{T}} \cap C([0, T]; \mathbb{R}^m)$, assume that $G$ is a submersion at $\bar{\xi}(\bar{T}) = E^{\bar{\xi}, \bar{T}}(\bar{u})$, and that there exists $\bar{t} \in [0, \bar{T}]$ such that (B.12) is satisfied. Then there are $\Lambda, \nu > 0$, $k$ controls $u^1, \dots, u^k$ in $L^1([0, \bar{T}]; \mathbb{R}^m)$, and a $C^1$ mapping

$$U = (U_1, \dots, U_k) : B^k(G(\bar{\xi}(\bar{T})), \nu) \to B^k(0, \Lambda)$$

such that

$$(G \circ E^{\bar{\xi}, \bar{T}}) \left( \bar{u} + \sum_{i=1}^{k} U_i(z) u^i \right) = z \quad \forall z \in B^k(G(\bar{\xi}(\bar{T})), \nu).$$
*Proof of Theorem B.4.* From Theorem B.2, we know that the mapping $\mathcal{G} := G \circ E^{\tilde{\xi}, \bar{T}} : \mathcal{U}_{\tilde{\xi}, \bar{T}} \to \mathbb{R}^k$ is a $C^1$ submersion at $\bar{u}$. Thus, there are $k$ controls $u^1, \dots, u^k$ in $L^1([0, \bar{T}]; \mathbb{R}^m)$ such that

$$ \mathrm{Span} \{ \langle d\mathcal{G}(\bar{u}), u^i \rangle \mid i = 1, \dots, k \} = \mathbb{R}^k. \qquad (\text{B.15}) $$

Let $\Lambda > 0$ be such that, for every $\lambda \in B^k(0, \Lambda)$, the control $\sum_{i=1}^k \lambda_i u^i$ belongs to $\mathcal{U}_{\tilde{\xi}, \bar{T}}$. Define $F : B^k(0, \Lambda) \to \mathbb{R}^k$ by

$$ F(\lambda) := \mathcal{G}\left(\bar{u} + \sum_{i=1}^{k} \lambda_i u^i\right) \quad \forall \lambda = (\lambda_1, \dots, \lambda_k) \in B^k(0, \Lambda). $$

The function $F$ is well-defined, of class $C^1$ on $B^k(0, \Lambda)$, and satisfies $F(0_k) = \mathcal{G}(\bar{u}) = G(\tilde{\xi}(\bar{T}))$. Its differential at $\lambda = 0_k$ is given by

$$ \langle dF(0_k), \lambda \rangle = \sum_{i=0}^{k} \lambda_i \langle d\mathcal{G}(\bar{u}), u^i \rangle \quad \forall \lambda \in \mathbb{R}^k, $$

hence it is invertible by (B.15). Set $\bar{z} := F(0_N) = \mathcal{G}(\bar{u}) = G(\tilde{\xi}(\bar{T})) \in \mathbb{R}^k$. We apply the Inverse Function Theorem (see Theorem C.1 below) to deduce that there are $\nu > 0$ and a function of class $C^1$

$$ U = (U_1, \dots, U_k) : B^k(\bar{z}, \nu) \to B^k(0, \Lambda) $$

such that

$$ \mathcal{G}\left(\bar{u} + \sum_{i=1}^{k} U_i(z) u^i\right) = z \quad \forall z \in B^k(\bar{z}, \nu). $$

This concludes the proof. $\square$

## B.5 Application to partial controllability III

The conclusion of Theorem B.4 holds as well for any initial state $\xi$ and time $T$ sufficiently close to $\tilde{\xi}, \bar{T}$. For sake of simplicity we only treat the case $\bar{u} \equiv 0$ (which, however, is enough for our purposes).

**Theorem B.5.** If $\bar{u} \equiv 0 \in \mathcal{U}_{\tilde{\xi}, \bar{T}}$, $G$ is a submersion at $\tilde{\xi}(\bar{T}) = E^{\tilde{\xi}, \bar{T}}(\bar{u})$ and (B.14) is satisfied, then there are $\delta \in (0, \bar{T}/2)$, $K_U, \Lambda, \nu > 0$, and $k$ controls $u^1, \dots, u^k : [0, +\infty) \to \mathbb{R}^m$ of class $C^\infty$, such that

$$ \mathrm{Supp}(u^i) \subset [\delta, \bar{T} - \delta] \quad \forall i = 1, \dots, k, \qquad (\text{B.16}) $$

and the following property holds: For any $\xi \in \mathbb{R}^N$ and $T > 0$ satisfying

$$ |\xi - \bar{\xi}|, |T - \bar{T}| < \delta, \qquad (\text{B.17}) $$

there exists a $C^1$ mapping

$$ U^{\xi, T} = (U_1^{\xi, T}, \dots, U_k^{\xi, T}) : B^k \left( G(E^{\xi, T}(\bar{u})), \nu \right) \to B^k(0, \Lambda) $$

whose Lipschitz constant is bounded by $K_U$, such that $U^{\xi, T}(G(E^{\xi, T}(\bar{u}))) = 0_k$ and

$$ (G \circ E^{\xi, T})(\sum_{i=1}^{k} U_i^{\xi, T}(z) u^i) = z \quad \forall z \in B^k(G(E^{\xi, T}(\bar{u})), \nu). $$
Proof of Theorem B.5. Since the set of controls $u$ in $L^1([0, \bar{T}]; \mathbb{R}^m)$ which are smooth and strictly supported in $[0, \bar{T}]$ is dense in $L^1([0, \bar{T}]; \mathbb{R}^m)$, thanks to Corollary B.3 and the argument used in the proof of Theorem B.4, there are $\delta > 0$ and $k$ smooth controls $u^1, \dots, u^k$ in $L^1([0, \bar{T}]; \mathbb{R}^m)$ satisfying (B.16) such that

$$ \mathrm{Span} \left\{ \langle d(G \circ E^{\bar{\xi}, \bar{T}})(\bar{u}), u^i \rangle \mid i = 1, \dots, k \right\} = \mathbb{R}^k. $$

Extend the controls $u^1, \dots, u^k$ on $[0, +\infty)$ by setting $u^i(t) = 0$ for any $t \in [\bar{T}, +\infty)$. By continuity of the mapping $(\xi, T) \mapsto d(G \circ E^{\xi, T})$, up to choosing $\delta > 0$ smaller, we can assume that

$$ \mathrm{Span} \left\{ \langle d(G \circ E^{\xi, T})(\bar{u}), u^i \rangle \mid i = 1, \dots, k \right\} = \mathbb{R}^k, $$

for every pair $\xi, T$ satisfying (B.17). Let $\Lambda > 0$ be a constant to be fixed later, and for any $\xi, T$ satisfying (B.17) define the $C^1$ function $F^{\xi, T}: B^k(0, \Lambda) \to \mathbb{R}^k$ by

$$ F^{\xi, T}(\lambda) := (G \circ E^{\xi, T}) \left( \sum_{i=1}^{k} \lambda_i u^i \right) \quad \forall \lambda = (\lambda_1, \dots, \lambda_k) \in B^k(0, \Lambda). $$

Since $dF^{\bar{\xi}, \bar{T}}(0_n)$ is invertible and the function $(\xi, T, \lambda) \mapsto dF^{\xi, T}(\lambda)$ is continuous in a neighborhood of $(\bar{\xi}, \bar{T}, 0_n) \in \mathbb{R}^N \times \mathbb{R} \times \mathbb{R}^n$, we can still restrict $\delta$ and take $\Lambda > 0$ small enough so that assumptions (i) and (ii) of Theorem C.1 below are satisfied for any $F = F^{\xi, T}$ with $\xi, T$ satisfying (B.17). Then Theorem C.1 concludes the proof. $\square$

## C Quantitative Inverse Function Theorem

For sake of completeness, we state below the quantitative version of the Inverse Function Theorem that we used in Appendix B. We refer the reader to [1, 45] for a proof.

**Theorem C.1.** Let $\Lambda > 0$ and $F: B^n(0, \Lambda) \to \mathbb{R}^n$ be a function of class $C^1$ which satisfies the following properties:

(i) $dF(\lambda)$ is nonsingular for any $\lambda \in B^n(0, \Lambda)$;

(ii) $\|DF(\lambda') - DF(\lambda)\| \le (2\|dF(0)^{-1}\|)^{-1}$ for any $\lambda, \lambda' \in B^n(0, \Lambda)$.

Then there is a $C^1$ function

$$ F^{-1} : B^n(F(0), 5\Lambda \|dF(0)^{-1}\|^{-1}) \to B^n(0, \Lambda) $$

such that $F \circ F^{-1} = \mathrm{Id}$ on $B^n(F(0), 5\Lambda \|dF(0)^{-1}\|^{-1})$ and $F^{-1} \circ F = \mathrm{Id}$ on $B^n(0, \Lambda)$. Moreover, $F^{-1}$ is $(2\|dF(0)^{-1}\|)$-Lipschitz.

## D The Mai Lemma

The Mai Lemma, which was introduced in [34] to give a new and simpler proof of the closing lemma in $C^1$ topology, is one of the main tools in the proof of our results. Let us state it.

Let $\{E_i\}_{i \in \mathbb{N}}$ be a countable family of ellipsoids in $\mathbb{R}^k$, that is, a countable family of compact sets in $\mathbb{R}^k$ associated with a countable family of invertible linear mappings $P_i: \mathbb{R}^k \to \mathbb{R}^k$ such that

$$ E_i = \{v \in \mathbb{R}^k \mid |P_i(v)| \le \|P_i\|\}. $$
For every $y \in \mathbb{R}^k$, $r > 0$ and $i \in \mathbb{N}$, we call $E_i$-ellipsoid centered at $y$ with radius $r$ the set defined by

$$E_i(y, r) := \{y + rv \mid v \in E_i\} = \{y' \mid |P_i(y' - y)| \le r \|P_i\|\}.$$

We note that such an ellipsoid contains the open ball $B(y, r)$. The Mai Lemma can be stated as follows:

**Lemma D.1 (Mai Lemma).** Let $\hat{N} \ge 2$ be an integer. There exist a real number $\hat{\rho} \ge 3$ and an integer $\eta > 0$, which depend on the family $\{E_i\}$ and on $\hat{N}$ only, such that the following holds: For every $r > 0$ and every finite set $Y = \{y_1, \dots, y_J\} \subset \mathbb{R}^k$ such that $Y \cap B_r$ contains at least two points, there exist $\eta$ points $\hat{y}_1, \dots, \hat{y}_\eta$ in $\mathbb{R}^k$ and $\eta$ positive real numbers $\hat{r}_1, \dots, \hat{r}_\eta$ satisfying:

(i) there exist $j, l \in \{1, \dots, J\}$, with $j > l$, such that $\hat{y}_1 = y_j$ and $\hat{y}_\eta = y_l$;

(ii) $\forall i \in \{1, \dots, \eta - 1\}, E_i(\hat{y}_i, \hat{r}_i) \subset B_{\hat{\rho}r}$;

(iii) $\forall i \in \{1, \dots, \eta - 1\}, E_i(\hat{y}_i, \hat{r}_i) \cap (Y \setminus \{y_j, y_l\}) = \emptyset$;

(iv) $\forall i \in \{1, \dots, \eta - 1\}, \hat{y}_{i+1} \in E_i(\hat{y}_i, \hat{r}_i/\hat{N})$.

We refer the reader to [34] or the monograph [2] for a proof of the above result.

# E Proofs of Lemmas 3.3, 4.3 and 6.1

## E.1 Proof of Lemma 3.3

Let $\phi: [0, +\infty) \to [0, 1]$ be a function of class $C^\infty$ satisfying the following properties:

(a) $\phi(s) = 1$ for $s \in [0, 1/3]$;

(b) $\phi(s) = 0$ for $s \ge 2/3$;

(c) $|\phi'(s)|, |\phi''(s)| \le 20$ for any $s \in [0, +\infty)$.

Extend the function $\tilde{v}$ on $\mathbb{R}$ by $\tilde{v}(t) := 0$ for $t \le 0$ and $t \ge \bar{\tau}$, and define the function $W: [0, \bar{\tau}] \times \mathbb{R}^{n-1} \to \mathbb{R}$ by

$$W(t, \hat{z}) = \phi\left(\frac{|\hat{z}|}{r}\right) \left[ \int_{0}^{t} \tilde{v}_{1}(s) ds + \sum_{i=1}^{n-1} \int_{0}^{\hat{z}_{i}} \tilde{v}_{i+1}(t+s) ds \right] \quad \forall (t, \hat{z}) \in [0, \bar{\tau}] \times \mathbb{R}^{n-1}.$$

Since $\tilde{v}$ is $C^{k-1}$ and $\phi$ is smooth, it is easy to check that $W$ is of class $C^k$. (Actually, this is obvious in view of the formulas (E.1) and (E.2) below.) Using (b), (3.38), (3.39), and the fact that $r \le \delta/3$, we check easily that assertion (i) holds. Moreover, thanks to (b) again and recalling the definition of $\tilde{V}_1$, we have

$$\|W\|_{\infty} \le \|\phi\|_{\infty} \left[ \|\tilde{V}_1\|_{\infty} + r \sum_{i=1}^{n-1} \|\tilde{v}_{i+1}\|_{\infty} \right].$$

We now observe that the first partial derivatives of $W$ at $(t, \hat{z})$ are given by

$$\left\{ \begin{array}{l} \frac{\partial W}{\partial t}(t, \hat{z}) = \phi\left(\frac{|\hat{z}|}{r}\right) \left[ \tilde{v}_1(t) + \sum_{i=1}^{n-1} \int_0^{\hat{z}_i} \dot{\tilde{v}}_{i+1}(t+s) ds \right], \\ \frac{\partial W}{\partial \hat{z}_i}(t, \hat{z}) = \frac{\hat{z}_i}{r|\hat{z}|} \phi'\left(\frac{|\hat{z}|}{r}\right) \left[ \int_0^t \tilde{v}_1(s) ds + \sum_{i=1}^{n-1} \int_0^{\hat{z}_i} \tilde{v}_{i+1}(t+s) ds \right] \\ \qquad + \phi\left(\frac{|\hat{z}|}{r}\right) \tilde{v}_{i+1}(t+\hat{z}_i), \end{array} \right. \quad (E.1)$$
which combined with (a) yields (iii). Observe that $\frac{\partial W}{\partial t}(t, \hat{z})$ can also be written as

$$ \frac{\partial W}{\partial t}(t, \hat{z}) = \phi\left(\frac{|\hat{z}|}{r}\right) \left[ \tilde{v}_1(t) + \sum_{i=1}^{n-1} \left( \tilde{v}_{i+1}(t + \hat{z}_i) - \tilde{v}_{i+1}(t) \right) \right], \quad (E.2) $$

and moreover

$$ \left| \int_{0}^{\hat{z}_{i}} \tilde{v}_{i+1}(t+s) ds \right| \leq |\hat{z}_{i}| \|\tilde{v}_{i+1}\|_{\infty} \leq r \|\tilde{v}_{i+1}\|_{\infty} \quad \text{for } |\hat{z}_{i}| \leq r. $$

These estimates, together with (E.1), (b), and (c), imply

$$ \|\nabla W\|_{\infty} \leq K \left[ \frac{1}{r} \|\tilde{v}_1\|_{\infty} + \|\tilde{v}\|_{\infty} \right] $$

where K is a universal constant depending on the dimension n only. Let us now compute the second derivatives of W. For every $(t, \hat{z})$ we have

$$ \left\{ \begin{aligned} \frac{\partial^2 W}{\partial t^2}(t, \hat{z}) &= \phi\left(\frac{|\hat{z}|}{r}\right) \left[ \dot{\tilde{v}}_1(t) + \sum_{i=1}^{n-1} \left( \dot{\tilde{v}}_{i+1}(t + \hat{z}_i) - \dot{\tilde{v}}_{i+1}(t) \right) \right], \\ \frac{\partial^2 W}{\partial \hat{z}_i \partial t}(t, \hat{z}) &= \frac{\hat{z}_i}{r|\hat{z}_i|} \phi'\left(\frac{|\hat{z}|}{r}\right) \left[ \tilde{v}_1(t) + \sum_{i=1}^{n-1} \left( \tilde{v}_{i+1}(t + \hat{z}_i) - \tilde{v}_{i+1}(t) \right) \right] \\ &\quad + \phi\left(\frac{|\hat{z}|}{r}\right) \left[ \sum_{i=1}^{n-1} \dot{\tilde{v}}_{i+1}(t + \hat{z}_i) \right], \\ \frac{\partial^2 W}{\partial \hat{z}_i \partial \hat{z}_j}(t, \hat{z}) &= \frac{\hat{z}_i \hat{z}_j}{r^2|\hat{z}|^2} \phi''\left(\frac{|\hat{z}|}{r}\right) \left[ \int_0^t \tilde{v}_1(s) ds + \sum_{i=1}^{n-1} \int_0^{\hat{z}_i} \tilde{v}_{i+1}(t+s) ds \right] \\ &\quad + \left( \delta_{ij} \frac{1}{r|\hat{z}_i|} - \frac{\hat{z}_i \hat{z}_j}{r|\hat{z}_i|^3} \right) \phi'\left(\frac{|\hat{z}|}{r}\right) \left[ \int_0^t \tilde{v}_1(s) ds + \sum_{i=1}^{n-1} \int_0^{\hat{z}_i} \tilde{v}_{i+1}(t+s) ds \right] \\ &\quad + \frac{1}{r|\hat{z}|} \phi'\left(\frac{|\hat{z}|}{r}\right) \left[ \hat{z}_i \tilde{v}_{j+1}(t + \hat{z}_j) + \hat{z}_j \tilde{v}_{i+1}(t + \hat{z}_i) \right] + \delta_{ij} \phi\left(\frac{|\hat{z}|}{r}\right) \dot{\tilde{v}}_{i+1}(t + \hat{z}_i), \end{aligned} \right. $$

where $\delta_{ij} = 1$ if $i = j$, $\delta_{ij} = 0$ if $i \neq j$. Since by (a) $\phi'\left(\frac{|\hat{z}|}{r}\right) = 0$ if $|\hat{z}| \le r/3$, and by (b) $\phi\left(\frac{|\hat{z}|}{r}\right) = \phi'\left(\frac{|\hat{z}|}{r}\right) = \phi''\left(\frac{|\hat{z}|}{r}\right) = 0$ if $|\hat{z}| \ge 2r/3$, the validity of (ii) follows easily.

## E.2 Proof of Lemma 4.3

Let us compute the Lie brackets $[F_0, F_t]$ at $\bar{\xi}^\tau = (\bar{x}^\tau, \bar{q}^\tau := \hat{p}^\tau, 0, 0)$ for every $i = 1, \dots, n$. Recalling that $\frac{\partial \bar{H}}{\partial p_1}(\bar{x}^\tau, \bar{p}^\tau) = 1$ and that $\frac{\partial \varphi}{\partial h}(\bar{x}^\tau, \bar{q}^\tau, 0) \frac{\partial \bar{H}}{\partial p_1}(\bar{x}^\tau, \bar{p}^\tau) = -1$, we observe that the first n components of $[F_0, F_1]$ at $\bar{\xi}^\tau$ are given by

$$ -\frac{\partial^2 \bar{H}}{\partial p^2}(\bar{x}^\tau, \bar{p}^\tau) \frac{\partial \psi}{\partial h}((\bar{x}^\tau, \bar{q}^\tau, 0)) = \frac{\partial}{\partial p_1} \nabla_p \bar{H}(\bar{x}^\tau, \bar{p}^\tau), $$

while its last component at $\bar{\xi}^\tau$ is given by

$$ \begin{aligned} & -\frac{\partial}{\partial h} \left\langle \psi(\bar{x}^\tau, \bar{q}^\tau, h), \nabla_p \bar{H}(\bar{x}^\tau, \psi(\bar{x}^\tau, \bar{q}^\tau, h)) \right\rangle \Big|_{h=0} \\ &= -\frac{\partial}{\partial h} \left\langle \varphi(\bar{x}^\tau, \bar{q}^\tau, h) \frac{\partial \bar{H}}{\partial p_1}(\bar{x}^\tau, \psi(\bar{x}^\tau, \bar{q}^\tau, h)) \right\rangle \Big|_{h=0} - \frac{\partial}{\partial h} \left\langle \langle \bar{q}^\tau, \nabla_p \bar{H}(\bar{x}^\tau, \psi(\bar{x}^\tau, \bar{q}^\tau, h)) \rangle \right\rangle \Big|_{h=0} \\ &= -\frac{\partial \varphi}{\partial h}(\bar{x}^\tau, \bar{q}^\tau, 0) \frac{\partial \bar{H}}{\partial p_1}(\bar{x}^\tau, \bar{p}^\tau) - \varphi(\bar{x}^\tau, \bar{q}^\tau, 0) \frac{\partial^2 \bar{H}}{\partial p_1^2}(\bar{x}^\tau, \bar{p}^\tau) \frac{\partial \varphi}{\partial h}(\bar{x}^\tau, \bar{q}^\tau, 0) \\ &\quad - \left\langle \bar{q}^\tau, \frac{\partial}{\partial p_1} \nabla_p \bar{H}(\bar{x}^\tau, \bar{p}^\tau) \right\rangle \frac{\partial \varphi}{\partial h}(\bar{x}^\tau, \bar{q}^\tau, 0) = 1 + \left\langle \bar{p}^\tau, \frac{\partial}{\partial p_1} \nabla_p \bar{H}(\bar{x}^\tau, \bar{p}^\tau) \right\rangle. \end{aligned} $$