Moreover, since $\frac{\partial \bar{H}}{\partial p_i}(\bar{x}^{\tau}, \bar{p}^{\tau}) = 0$ for $i = 2, \dots, n$, as in the proof of Proposition 3.1 the first $n$ components of $[F_0, F_i]$ at $\bar{\xi}^{\tau}$ for $i = 2, \dots, n$ are given by

$$ \frac{\partial^2 \bar{H}}{\partial p^2}(\bar{x}^{\tau}, \bar{p}^{\tau}) \frac{\partial \psi}{\partial q_{i-1}}((\bar{x}^{\tau}, \bar{q}^{\tau}, 0)) = \frac{\partial}{\partial p_i} \nabla_p \bar{H}(\bar{x}^{\tau}, \bar{p}^{\tau}), $$

where for the last equality we used that $\frac{\partial \psi}{\partial q_{i-1}}(\bar{x}^{\tau}, \bar{q}^{\tau}) = e_i^n$ (see (4.11)). Therefore the first $n$ components of the bracket $[F_0, F_i]$ at $\bar{\xi}^{\tau}$, for $i = 2, \dots, n$, correspond to the $i$-th column of the Hessian of $\bar{H}$ in the $p$ variable at $(\bar{x}^{\tau}, \bar{p}^{\tau})$. Finally, using again that $\frac{\partial \bar{H}}{\partial p_i}(\bar{x}^{\tau}, \bar{p}^{\tau}) = 0$ for $i = 2, \dots, n$, the last component of $[F_0, F_i]$ at $\bar{\xi}^{\tau}$ is given by

$$ \langle \bar{p}^{\tau}, \frac{\partial}{\partial p_i} \nabla_p \bar{H}(\bar{x}^{\tau}, \bar{p}^{\tau}) \rangle \quad \forall i = 2, \dots, n. $$

All in all, the $(2n+1) \times (2n)$ matrix $(F_1(\bar{\xi}^{\tau}), \dots, F_n(\bar{\xi}^{\tau}), [F_0, F_1](\bar{\xi}^{\tau}), \dots, [F_0, F_n](\bar{\xi}^{\tau}))$ equals

$$ \begin{pmatrix}
0 & \cdots & \cdots & 0 & \frac{\partial^2 \bar{H}}{\partial p_1^2} & \frac{\partial^2 \bar{H}}{\partial p_1 \partial p_2} & \cdots & \cdots & \frac{\partial^2 \bar{H}}{\partial p_1 \partial p_n} \\
0 & \cdots & \cdots & 0 & \frac{\partial^2 \bar{H}}{\partial p_2 \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_2^2} & \cdots & \cdots & \frac{\partial^2 \bar{H}}{\partial p_2 \partial p_n} \\
\vdots & & & \vdots & & & \ddots & & & \vdots \\
0 & \cdots & \cdots & 0 & \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_2} & \cdots & \cdots & \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_n} \\
0 & \cdots & \cdots & 0 & \frac{\partial^2 \bar{H}}{\partial p_n \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_n \partial p_2} & \cdots & \cdots & \frac{\partial^2 \bar{H}}{\partial p_n^2} \\
0 & -1 & 0 & \cdots & 0 & \frac{*}{*} & \frac{*}{*} & \cdots & \frac{*}{*} & \frac{*}{*} \\
0 & 0 & \cdots & 0 & \frac{*}{*} & \frac{*}{*} & \cdots & \frac{*}{*} & \frac{*}{*} & \frac{*}{*} \\
\vdots & & & \vdots & \frac{*}{*} & \frac{*}{*} & \cdots & \frac{*}{*} & \frac{*}{*} & \frac{*}{*} \\
0 & 0 & \cdots & 0 & -1 & \frac{*}{*} & \frac{*}{*} & \cdots & \frac{*}{*} & \frac{*}{*} \\
1 & 0 & \cdots & 0 & 0 & \frac{*}{*} & \frac{*}{*} & \cdots & \frac{*}{*} & \frac{*}{*} \\
0 & \cdots & \cdots & 0 & 1 + \langle \bar{p}^{\tau}, \frac{\partial}{\partial p_1} \nabla_p \bar{H} \rangle & \langle \bar{p}^{\tau}, \frac{\partial}{\partial p_2} \nabla_p \bar{H} \rangle & \cdots & \cdots & \langle \bar{p}^{\tau}, \frac{\partial}{\partial p_n} \nabla_p \bar{H} \rangle
\end{pmatrix}, $$

where all the partial derivatives of $\bar{H}$ are evaluated at $(\bar{x}^{\tau}, \bar{p}^{\tau})$. Since $\text{Ker}(dG(\bar{\xi}^{\tau})) = \mathbb{R}e_1^{2n+1}$, we deduce that the assumption (B.14) is satisfied if and only if the matrix

$$ N := \begin{pmatrix}
1 + \langle \bar{p}^{\tau}, \frac{\partial}{\partial p_1} \nabla_p \bar{H} \rangle & \langle \bar{p}^{\tau}, \frac{\partial}{\partial p_2} \nabla_p \bar{H} \rangle & \cdots & \langle \bar{p}^{\tau}, \frac{\partial}{\partial p_n} \nabla_p \bar{H} \rangle \\
\frac{\partial^2 \bar{H}}{\partial p_2 \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_2^2} & \cdots & \frac{\partial^2 \bar{H}}{\partial p_2 \partial p_n} \\
\frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_2} & \cdots & \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_n} \\
\frac{\partial^2 \bar{H}}{\partial p_n \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_n \partial p_2} & \cdots & \frac{\partial^2 \bar{H}}{\partial p_n^2}
\end{pmatrix} $$

is invertible. But we observe that

$$ \det(N) = \det \begin{pmatrix} 1 & 0 & \cdots & 0 \\ \frac{\partial^2 \bar{H}}{\partial p_2 \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_2^2} & \cdots & \frac{\partial^2 \bar{H}}{\partial p_2 \partial p_n} \\ \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_2} & \cdots & \frac{\partial^2 \bar{H}}{\partial p_{n-1} \partial p_n} \\ \frac{\partial^2 \bar{H}}{\partial p_n \partial p_1} & \frac{\partial^2 \bar{H}}{\partial p_n \partial p_2} & \cdots & \frac{\partial^2 \bar{H}}{\partial p_n^2} \end{pmatrix} + \bar{p}_1^{\tau} \det \left( \frac{\partial^2 \bar{H}}{\partial p^2} \right) \\ = \det \left( \frac{\partial^2 \bar{H}}{\partial \bar{p}^2} \right) + \bar{p}_1^{\tau} \det \left( \frac{\partial^2 \bar{H}}{\partial p^2} \right), $$

which shows that (B.14) is satisfied if and only if assumption (A4) holds.
### E.3 Proof of Lemma 6.1

Since the construction is local, up to using a partition of unity we can assume for simplicity that $N = \mathbb{R}^n$.

Fix $r > 0$ such that $\{x | \text{dist}(x, C) \ge 21r\}$ is compactly supported inside $\mathcal{O}$. We claim that there exist a universal constant $K_0$, depending only on the dimension $n$, and a function $\psi: \mathbb{R}^n \to [0, 1]$ of class $C^\infty$ such that

1. $\psi = 1$ on $\{x | \text{dist}(x, C) \le r\}$;

2. $\psi = 0$ on $\{x | \text{dist}(x, C) \ge 21r\}$;

3. $\|\nabla\psi\|_{\infty} \le \frac{K_0}{r}$, $\|\text{Hess } \psi\|_{\infty} \le \frac{K_0}{r^2}$.

Assume that the claim is proved, and set $h := \psi g$. Obviously $h$ satisfies (a), (b), and (d). Moreover, thanks to (6.3) a Taylor expansion gives

$$0 \le g(x) \le 50r^2\omega(10r), \quad \|\nabla g(x)\| \le 10r\omega(10r) \quad \text{on } \{x | \text{dist}(x, C) \ge 10r\}.$$

Hence

$$0 \le h \le 50r^2\omega(10r),$$

$$\|\nabla h\|_{\infty} \le \frac{C_0}{r} 50r^2\omega(10r) + 10r\omega(10r) \le (50C_0 + 10)r\omega(10r),$$

$$\|\text{Hess } h\|_{\infty} \le \frac{C_0}{r^2} 50r^2\omega(10r) + 2\frac{C_0}{r} 10r\omega(10r) + \omega(10r) \le (70C_0 + 1)\omega(10r),$$

and (c) follows by choosing $r$ sufficiently small.

We are left with proving the claim. For every $x \in \mathcal{O}$, let us consider the family of balls $\{B(x, r)\}_{x \in \mathcal{O}}$. By Vitaly's Covering Theorem [15, Subsection 1.5.1] there exists a disjoint subfamily $\{B(x_j, r)\}_{j \in \mathbb{N}}$ such that

$$\mathcal{O} \subset \bigcup_{j \in \mathbb{N}} B(x_j, 5r). \qquad (E.3)$$

We claim that $\{B(x_j, 10r)\}_{j \in \mathbb{N}}$ has the finite overlapping property, i.e., there exists a constant $N(n)$, depending only on the dimension, such that any point $y \in \mathbb{R}^n$ belongs to at most $N(n)$ balls. Indeed, assume that $y \in B(x_j, 10r)$. Then $B(x_j, r) \in B(y, 11r)$. But since the balls $\{B(x_j, r)\}_{j \in \mathbb{N}}$ are disjoint we have

$$\sum_{\{j: y \in B(x_j, 10r)\}} |B(x_j, r)| \le |B(y, 11r)|,$$

i.e.,

$$\#\{j: y \in B(x_j, 10r)\} \le 11^n.$$

Hence the finite overlapping property holds with $N(n) := 11^n$.

Let now $\mu: \mathbb{R} \to [0, 1]$ be a function of class $C^\infty$ with $\mu(u) = 1$ for $u \le 1$ and $\mu(u) = 0$ for $u \ge 2$. For every $j \in \mathbb{N}$, set

$$u_j(x) := \mu\left(\frac{|x - x_j|}{5r}\right).$$

Observe that:

(i) $u_j = 1$ inside $B(x_j, 5r)$;

(ii) $u_j = 0$ outside $B(x_j, 10r)$.
Define

$$ \sigma_r := \sum_{\{d(x_j, C) \le 11r\}} u_j, \quad \sigma := \sum_{j \in \mathbb{N}} u_j. $$

By (ii) we have $\sigma = \sigma_r$ inside $\{x | \text{dist}(x, C) \le r\}$ and $\text{Supp}(\sigma_r) \subset \{x | \text{dist}(x, C) \le 21r\}$. Moreover (i) and (E.3) ensure that $\sigma \ge 1$ inside $\mathcal{O}$. Finally the finite overlapping property implies that $0 \le \sigma_r \le \sigma \le N(n)$, $\|\nabla \sigma_r\| + \|\nabla \sigma\| \le N(n) \frac{K}{r}$, $\|\text{Hess } \sigma_r\| + \|\text{Hess } \sigma\| \le N(n) \frac{K}{r^2}$, where K is a constant depending only on $\mu$.

Thanks to these properties, the claim is proved by setting $\psi := \sigma_r/\sigma$.

## References

[1] R. Abraham, J. E. Marsden and T. Ratiu. Manifolds, tensor analysis, and applications. Applied Mathematical Sciences, vol. 75, Springer-Verlag, New-York, 1988.

[2] M.-C. Arnaud. Le “closing lemma” en topologie $C^1$. Mém. Soc. Math. Fr., 74, 1998.

[3] M.-C. Arnaud. Fibrés de Green et régularité des graphes $C^0$-lagrangiens invariants par un flot de Tonelli. Ann. Henri Poincaré, 9(5): 881-926, 2008.

[4] M.-C. Arnaud. The link between the shape of the Aubry-Mather sets and their Lyapunov exponents. Ann. of Math., to appear.

[5] M.-C. Arnaud. Green bundles, Lyapunov exponents and regularity along the supports of minimizing measures. Preprint, 2010.

[6] P. Bernard. Smooth critical sub-solutions of the Hamilton-Jacobi equation. Math. Res. Lett., 14(3):503-511, 2007.

[7] P. Bernard. Existence of $C^{1,1}$ critical sub-solutions of the Hamilton-Jacobi equation on compact manifolds. Ann. Sci. École Norm. Sup., 40(3):445-452, 2007.

[8] P. Bernard. On the number of Mather measures of Lagrangian systems. Arch. Ration. Mech. Anal., to appear.

[9] P. Bernard and G. Contreras. A generic property of families of Lagrangian systems. Ann. of Math., 167(3):1099-1108, 2008.

[10] P. Cannarsa and L. Rifford. Semiconcavity results for optimal control problems admitting no singular minimizing controls. Ann. Inst. H. Poincaré Non Linéaire, 25(4): 773-802, 2008.

[11] P. Cannarsa and C. Sinestrari. Semiconcave functions, Hamilton-Jacobi equations, and optimal control. Progress in Nonlinear Differential Equations and their Applications, 58. Birkhäuser Boston Inc., Boston, MA, 2004.

[12] G. Contreras and R. Iturriaga. Convex Hamiltonians without conjugate points. Ergodic Theory Dynam. Systems, 19(4):901-952, 1999.

[13] G. Contreras, R. Iturriaga, G.P. Paternain and M. Paternain. Lagrangian graphs, minimizing measures and Mañé's critical values. GAFA, 8(5), 788-809, 1998.

[14] J.-M. Coron. Control and nonlinearity. Mathematical Surveys and Monographs, 136. American Mathematical Society, Providence, RI, 2007.

[15] L. C. Evans and R. F. Gariepy. Measure Theorem and Fine Properties of Functions. Studies in Advanced Mathematics. CRC Press, Boca Raton, FL, 1992.
[16] A. Fathi. Théorème KAM faible et théorie de Mather sur les systèmes lagrangiens. *C. R. Acad. Sci. Paris Sér. I Math.*, 324(9):1043–1046, 1997.

[17] A. Fathi. Solutions KAM faible conjuguées et barrières de Peierls. *C. R. Acad. Sci. Paris Sér. I Math.*, 325(6):649–652, 1997.

[18] A. Fathi. Sur la convergence du semi-groupe de Lax-Oleinik. *C. R. Acad. Sci. Paris Sér. I Math.*, 327(3):267–270, 1998.

[19] A. Fathi. Regularity of $C^1$ solutions of the Hamilton-Jacobi equation. *Ann. Fac. Sci. Toulouse*, 12(4):479–516, 2003.

[20] A. Fathi. *Denjoy-Schwartz and Hamilton-Jacobi. J. Math. Kyoto Univ.*, to appear.

[21] A. Fathi. *Weak KAM Theorem and Lagrangian Dynamics*. Cambridge University Press, to appear.

[22] A. Fathi. On the existence of smooth subsolution of the Hamilton-Jacobi equation. Personal communication, 2010.

[23] A. Fathi, A. Figalli and L. Rifford. On the Hausdorff dimension of the Mather quotient. *Comm. Pure Appl. Math.*, 62(4):445–500, 2009.

[24] A. Fathi and E. Maderna. Weak KAM theorem on non compact manifolds. *NoDEA Nonlinear Differential Equations Appl.*, 14(1-2):1–27, 2007.

[25] A. Fathi and A. Siconolfi. Existence of $C^1$ critical subsolutions of the Hamilton-Jacobi equation. *Invent. math.*, 1155:363–388, 2004.

[26] H. Federer. *Geometric measure theory*. Die Grundlehren der mathematischen Wissenschaften, Band 153. Springer-Verlag New York Inc., New York 1969.

[27] A. Figalli and L. Rifford. Closing Aubry sets II. Preprint, 2010.

[28] C. Gutierrez. A counter-example to a $C^2$ closing lemma. *Ergodic Theory Dynam. Systems*, 7(4):509–530, 1987.

[29] C. Gutierrez. On $C^r$-closing for flows on 2-manifolds. *Nonlinearity*, 13(6):1883–1888, 2000.

[30] C. Gutierrez. On the $C^r$-closing lemma. The geometry of differential equations and dynamical systems. *Comput. Appl. Math.*, 20(1-2):179–186, 2001.

[31] M.-R. Herman. Exemples de flots hamiltoniens dont aucune perturbation en topologie $C^\infty$ n'a d'orbites périodiques sur un ouvert de surfaces d'énergies. (French) [Examples of Hamiltonian flows such that no $C^\infty$ perturbation has a periodic orbit on an open set of energy surfaces] *C. R. Acad. Sci. Paris Sér. I Math.*, 312(13):989–994, 1991.

[32] M.-R. Herman. Différentiabilité optimale et contre-exemples à la fermeture en topologie $C^\infty$ des orbites récurrentes de flots hamiltoniens. (French. English summary) [Optimal differentiability and counterexamples to the $C^\infty$ closing lemma for Hamiltonian vector fields] *C. R. Acad. Sci. Paris Sér. I Math.*, 313(1):49–51, 1991.

[33] S. Lloyd. On the closing lemma problem for the torus. *Discrete Contin. Dyn. Syst.*, 25(3):951–962, 2009.

[34] J. Mai. A simpler proof of $C^1$ closing lemma. *Sci. Sinica Ser. A*, 29(10):1020–1031, 1986.

[35] R. Mañé. Generic properties and problems of minimizing measures of Lagrangian systems, *Nonlinearity*, 9(2):273–310, 1996.
[36] D. Massart. On Aubry sets and Mather's action functional. *Israel J. Math.*, 134:157-171, 2003.

[37] J. N. Mather. Action minimizing invariant measures for positive definite Lagrangian systems. *Math. Z.*, 207:169-207, 1991.

[38] J. N. Mather. Variational construction of connecting orbits. *Ann. Inst. Fourier*, 43:1349-1386, 1993.

[39] J. N. Mather. Total disconnectedness of the quotient Aubry set in low dimension. *Comm. Pure App. Math.*, 56:1178-1183, 2003.

[40] J. N. Mather. Examples of Aubry sets. *Ergod. Th. Dynam. Sys.*, 24:1667-1723, 2004.

[41] C. C. Pugh. The closing lemma. *Amer. J. Math.*, 89:956-1009, 1967.

[42] C. C. Pugh. An improved closing lemma and a general density theorem. *Amer. J. Math.*, 89:1010-1021, 1967.

[43] C. C. Pugh and C. Robinson. The $C^1$ closing lemma, including Hamiltonians. *Ergodic Theory Dynam. Systems*, 3(2):261-313, 1983.

[44] L. Rifford. On viscosity solutions of certain Hamilton-Jacobi equations: Regularity results and generalized Sard's Theorems. *Comm. Partial Differential Equations*, 33(3):517-559, 2008.

[45] L. Rifford. *Nonholonomic Variations: An introduction to sub-Riemannian geometry*. In progress.

[46] J.-M. Roquejoffre. Propriétés qualitatives des solutions des équations de Hamilton-Jacobi (d'après A. Fathi, A. Siconolfi, P. Bernard). In Séminaire Bourbaki Vol. 2006/2007 *Astérisque*, 317:269-293, 2008.

[47] T. Sakai. *Riemannian geometry*. Translations of Mathematical Monographs, Vol. 149. American Mathematical Society, Providence, RI, 1996. Translated from the 1992 Japanese original by the author.

[48] A. Sorrentino. On the total disconnectedness of the quotient Aubry set. *Ergodic Theory Dynam. Systems*, 28(1):267-290, 2008.