<figure><img src="image_1.png" /></figure>

HAL
open science

# Divergences de Bregman pour des mesures d'écart physiquement informées pour l'apprentissage et le calcul en thermomécanique

Stéphane Andrieux

► To cite this version:

Stéphane Andrieux. Divergences de Bregman pour des mesures d'écart physiquement informées pour l'apprentissage et le calcul en thermomécanique. Comptes Rendus. Mécanique, 2023, 351, pp. 59-81. 10.5802/crmeca.164. hal-03777968v2

HAL Id: hal-03777968

https://hal.science/hal-03777968v2

Submitted on 2 Feb 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Bregman divergences for physically informed discrepancy
measures for learning and computation in thermomechanics

Stéphane Andrieux

ONERA, Université Paris-Saclay, Chemin de la Hunière, 91123 Palaiseau, France

**ABSTRACT:** With view on the context of convex thermomechanics, we propose tools based on the concept of Bregman divergence, a notion introduced in the 1960s and used in learned and optimization as well. This study is motivated by the need of "discrepancy measures" between physically constrained fields that are used both in traditional algorithms, analysis methods and data driven modelling or applications as well.

We give also characterization of symmetrical Bregman divergences through their generating function which can
only be a quadratic function. Some properties of the Bregman divergence and the introduced concept of Bregman
Gap between couples of dual quantities are given, and some existing errors in thermomechanics are recovered.
Exploiting the framework of Standard Generalized Materials, we give a set of generating functions for a large
range of applications, including coupled multi-physics. Finally some results useful for computational geometry
are detailed.

**RESUME :** Dans le contexte de la thermomécanique convexe, nous proposons des outils basés sur le concept de divergence de Bregman, une notion introduite dans les années 1960 et utilisée dans d'autres domaines aussi bien en apprentissage qu'en optimisation. Cette étude est motivée par le besoin de "mesures de dissimilarité" entre des champs physiquement contraints, champs qui sont utilisés à la fois dans les algorithmes traditionnels, les méthodes d'analyse et les applications ou les modélisations basées sur les données.

Nous donnons également une caractérisation des divergences de Bregman symétriques à travers leur fonction génératrice qui ne peut être qu'une fonction quadratique. Certaines propriétés de la divergence de Bregman et du concept introduit d'écart de Bregman entre les couples de quantités duales sont donnés, et certaines erreurs existantes en thermomécanique sont retrouvées. En exploitant le cadre des matériaux standard généralisés, nous donnons un ensemble de fonctions génératrices pour une large gamme d'applications, y compris la multiphysique couplée. Enfin, certains résultats utiles pour la géométrie computationnelle sont détaillés.

**Keywords :** Themomechanics, convexity, Bregman divergences, discrepancy measures, data processing

**Mots-clés :** Thermomécanique, convexité, divergence de Bregman, mesures de dissimilarité, traitement de données
## 1 Introduction

In thermomechanics as for many multi-physics applications, the need for metrics, “discrepancy measures” or various concepts derived from a distance in order to compare fields of quantities of interest is growing, not only but mainly with the development of large amount of digitalized data, coming the experiments, computations or in-operation measurements. Indeed, a very large majority of algorithms or data processing methods use, at one time or another, the comparison of fields. We can cite mechanical analysis, inverse problems, model reduction as well as classification or knowledge extraction from massive data.

In order to introduce a correct processing of data, or more generally of fields with different physical dimensions, in order to try to introduce the *a priori* knowledge which we have, thanks to the modeling works and the descriptions of the physics, it is essential to go beyond the usual norms of $IR^n$ or even of functional spaces, used until now mainly to deal with a single physics. The more and more frequent irruption of high dimension spaces or probabilistic tools also raises questions about the usual norms and even about the notions of neighborhood ([1],[2]). In the case of thermomechanics, for example, "solution fields" such as displacement, temperature, strain and stress tensors, are structured both locally and spatially through the conservation laws, the thermodynamics requirements, the constitutive equations and the symmetries, and some invariant quantities or global properties are also well established. Even if in practice, data issued from experiments, measurements or even computations are corrupted with some noise, it seems that trying to incorporate as such as possible this *a priori* knowledge can lead to a guarantee of objectivity and performance, especially when mixing the different types or origins of the data that are manipulated.

Coming back to thermomechanics, the general concept of energy has been used for decades and some authors already proposed derived concepts which revealed fruitful in this direction in various applications. To mention only a few: the Kohn-Vogelius functional [3] for distributed parameters identification, the constitutive relation error ([4],[5],[6]) for finite element error estimation and model updating or identification of material parameters, the Reciprocity Gap [7] for crack identification, the Equilibrium Gap [8] used together with image correlation techniques, ..

Let's take a simple illustration. Suppose we want to identify the rigidity of springs distributed along a bar in the traction-compression regime. The bar is clamped at one end and submitted to a prescribed force $C$ at the other. The identification is performed provided the corresponding displacement is measured all along the bar and known. Two situations are considered, in the first one $a$), the springs are linear ones, whereas in the second one $b$), a supplementary cubic nonlinearity is added. The equations of the two systems are respectively:

$$ a) \quad -u'' + \alpha^2 u = 0 \quad , \quad u(0) = 0, u'(1) = C \qquad b) \quad -u'' + \alpha^2 u + \varepsilon\beta u^3 = 0 \quad , \quad u(0) = 0, u'(1) = C $$

In the case $a$) the rigidity $\alpha^2$ is sought whereas in the case $b$) it is known and the rigidity $\varepsilon\beta$ is sought. A classical approach is to minimize a squared measure of discrepancy between the actual displacement (for the true value of the rigidity, that is $\alpha_0$ for case $a$) and $\varepsilon\beta_0$ for case $b$) and the computed one for a given value of the rigidity. The figure 1 displays for the cases $a$) and $b$), the usual least-square function, the energy of the difference of the two fields, and what will be called below the symmetrized Bregman divergence between the two fields (which in case $a$) turns about to be exactly the energy of the difference) that is:

$$ a) G_1(\alpha^2) = \frac{\alpha_0^2}{2} \int_0^1 (u(\alpha^2) - u(\alpha_0^2))^2 dx \\ G_2(\alpha^2) = \frac{1}{2} \int_0^1 \left[ (u'(\alpha^2) - u'(\alpha_0^2))^2 + \alpha^2 (u(\alpha^2) - u(\alpha_0^2))^2 \right] dx $$
$$b) H_1(\varepsilon\beta) = \frac{\alpha^2}{2} \int_0^1 (u(\varepsilon\beta) - u(\varepsilon\beta_0))^2 dx,$$

$$H_2(\varepsilon\beta) = \frac{1}{2} \int_0^1 \left[ (u'(\varepsilon\beta) - u'(\varepsilon\beta_0))^2 + \alpha^2 (u(\varepsilon\beta) - u(\varepsilon\beta_0))^2 + \varepsilon\beta (u(\varepsilon\beta) - u(\varepsilon\beta_0))^4 \right] dx$$

$$H_3(\varepsilon\beta) = \frac{1}{2} \int_0^1 \left[ (u'(\varepsilon\beta) - u'(\varepsilon\beta_0))^2 + \alpha^2 (u(\varepsilon\beta) - u(\varepsilon\beta_0))^2 + \varepsilon\beta (u(\varepsilon\beta) - u(\varepsilon\beta_0))(u^3(\varepsilon\beta) - u^3(\varepsilon\beta_0)) \right] dx$$

In both cases, closed form solutions are used, and for case b) it is obtained by the first two terms of an asymptotic expansion with $\varepsilon$ as the small parameter.

<figure><img src="image_3.png" /><figcaption>Fig.1: a) Error functions for the identification of $\alpha$, b) Error functions for the identification of $\beta$
$\alpha_0 = 8$ , $\beta_0 = 25$ , $\varepsilon = 0.1$ , $C = 50$</figcaption></figure>

It can be observed that, even in this simplistic case, the functions $G_2$ and $H_3$ are more suited for the minimization procedure. Indeed the ratios of the Hessians are respectively 1.2 and 5.9 at points $\alpha_0$ and $\beta_0$. The difference being much more pronounced in the nonlinear case.

This paper is devoted to the introduction of various concepts, involving or derived from, the Bregman divergence and leading to various forms of discrepancy measures, in a special sense, for thermomechanics quantities. These concepts encompass some of the concepts cited previously and have been already used for solving nonlinear Cauchy problems in various situations in thermomechanics ([9]-[11]). The paper is organized as follows. First the Bregman divergence is defined and various properties are described together with the definition of derived concept such as the Symmetrized Bregman Divergence and the Bregman Gap. Then generating functions for the Bregman divergence and derived concepts are given for a large class of constitutive relations or situations in thermomechanics. Finally, some properties useful for elementary computational geometry are given.

# 2 The Bregman divergence: main properties and generalization

## 2.1 The Bregman divergence

The Bregman divergence has been introduced by Bregman [12] in convex programming context [13], although the name Bregman divergence has been coined later [14]. It has also been studied and used for some years in the field of statistical inference and data processing problems [15], such as estimation, detection, classification, compression, and recognition, in particular for classification or clustering applications [16], and even, more recently, in deep learning applications. In learning applications, the Bregman divergence is used as a discrepancy measure and in some optimization algorithms [17] it is recognized as superior to the usual Euclidian distance.

We give here a slight generalization of the Bregman divergence, which includes the case of non-differentiable generating functions, because they are to be encountered in thermomechanics.

**Definition:** Let $J$ be a proper convex function, $J(e): \mathbb{R}^n \to \mathbb{R}$, the Bregman divergence, generated by $J$, between two points $(e_1, e_2)$ belonging to $\text{dom}(J)^2$ is the non-negative scalar :
$$ D_J(e_1, e_2) = J(e_1) - J(e_2) - \langle p_{21}, e_1 - e_2 \rangle \\ p_{21} = \frac{1}{2} \left[ \underset{p \in \partial J(e_1)}{\arg \max} \langle p, e_1 - e_2 \rangle + \underset{p \in \partial J(e_2)}{\arg \min} \langle p, e_2 - e_1 \rangle \right] \quad (1) $$

where $\partial J(e)$ is the subdifferential of $J$ at $e$: $\partial J(e) = \{p, J(d) \ge J(e) + \langle p, d - e \rangle \ \forall d \in \text{dom}(J)\}$

If the generating function $J$ is differentiable at point $e_2$, then the Bregman divergence reduces to:

$$ D_J(e_1, e_2) = J(e_1) - J(e_2) - \langle \nabla J(e_2), e_1 - e_2 \rangle \quad (2) $$

<figure><img src="image_4.png" /><figcaption>Fig. 2: Geometrical interpretation of the Bregman divergence in $IR^d$, $d=1$ and $2$ for differentiable generating functions</figcaption></figure>

Roughly speaking, the Bregman divergence is the tail of the Taylor expansion of $J$. An extension to infinite dimensional spaces of the Bregman divergence has been provided [18]. The positivity of the Bregman divergence results simply from the genuine definition of a convex function which is “above its tangent planes”. The Bregman divergence can be used as a discrepancy measure between two vectors as it takes strictly positive values if the two vectors are different (when the generating function is strictly convex), and is zero on any pair $(e, e)$.

The definition (1) for non-differentiable generating functions is chosen as to obtain a “medium” discrepancy measure, and leads also to a non-zero divergence between zero and other points for generating functions that are positively homogeneous with degree one. The geometrical interpretation is slightly different than in the case of differentiable generating functions, and the value $p_{12}$, which is an element of $\partial J(e)$, by convexity of the subdifferential, turns out to be also the mean value of the directional derivatives in the directions $(e_1 - e_2)$ and $(e_2 - e_1)$.

<figure><img src="image_5.png" /><figcaption>Fig. 3: Geometrical interpretation of the Bregman divergence in $IR$ for non-differentiable generating function at $e=0$</figcaption></figure>

The choice retained within the subdifferential of $J$ in $e_2$ does not however guarantees the directional continuity of the Bregman divergence around the point of non-differentiability $e_2$. Consider for example

$$ J(e) = e^2 + \alpha e^+ - \beta e^-, \quad \alpha, \beta \ge 0, \text{ then } \partial J(0) = [-\beta, \alpha] \text{ and } D_J(e, 0) = J(e) - \frac{1}{2}(\alpha - \beta)e. $$
There are situations in the field of learning, for example in image classification, which do not require finer concepts such as true metrics that by the way do not really make sense, for example for the triangle inequality [19]. Nevertheless, even if $D_J(e_1, e_2)$ vanishes when $e_1=e_2$ (and reciprocally if the generating function $J$ is strictly convex), Bregman divergences are not distances as they are neither symmetric nor satisfying the triangle inequality in general. In table 1, some Bregman divergence are displayed, the two first ones are genuine distances whereas the last ones, which are not, are used in learning applications and statistics.

172

<table><thead><tr><th>Domain</th><th>Generating function J(x)</th><th>Bregman divergence D<sub>J</sub>(x, y)</th><th>Name</th></tr></thead><tbody><tr><td>IR<sup>n</sup></td><td>||x||<sup>2</sup></td><td>||x - y||<sup>2</sup></td><td>Euclidian Distance</td></tr><tr><td>IR<sup>n</sup></td><td>J(x) = x<sup>T</sup>Ax A symmetric positive</td><td>(x - y)<sup>T</sup>A(x - y)</td><td>Mahalanobis distance</td></tr><tr><td>IR<sup>*n</sup></td><td>∑ x<sub>i</sub> log x<sub>i</sub> - x<sub>i</sub></td><td>∑ x<sub>i</sub> log <sup>x<sub>i</sub></sup>&frasl;<sub>y<sub>i</sub></sub> - x<sub>i</sub> + y<sub>i</sub></td><td>Kullback-Leibler divergence or Relative Entropy</td></tr><tr><td>IR<sup>*n</sup></td><td>∑ -log x<sub>i</sub></td><td>∑ <sup>x<sub>i</sub></sup>&frasl;<sub>y<sub>i</sub></sub> - log <sup>x<sub>i</sub></sup>&frasl;<sub>y<sub>i</sub></sub> - 1</td><td>Itakura-Saito discrete distance</td></tr><tr><td>[0,1]</td><td>x log x + (1 - x) log(1 - x)</td><td>x log <sup>x<sub>i</sub></sup>&frasl;<sub>y<sub>i</sub></sub> + (1 - x) log <sup>1 - x</sup>&frasl;<sub>1 - y</sub></td><td>Logistic loss</td></tr></tbody></table>

173

174

175

Nevertheless, the following interesting properties [16] which are useful in optimization as well as in computational geometry have been established and have been extended to the infinite dimensional context [18].

179

Convexity. $D_J(e_1, e_2)$ is always convex in the first argument (but not necessarily in the second argument, take for example $J(e)=e^3$ in $IR^+$, $D_{e^3}(e_1, e_2)$ is not convex in $e_2$ when $e_2 < e_1$)

182

183

184

Linearity. The Bregman divergence is a linear operator with respect to generating functions

185

$$D_{\lambda, J+\mu F}(e_1, e_2) = \lambda D_J(e_1, e_2) + \mu D_F(e_1, e_2) \quad (J, F \text{ convex } \lambda, \mu \text{ positive})$$

186

Projection onto convex sets. There exists a Bregman projection onto convex sets $C$ with uniqueness if $J$ is differentiable at $e$.

188

189

$$\forall C \ \exists! \tilde{e} \text{ such as } \tilde{e} = \underset{g \in C}{\operatorname{ArgMin}} D_J(g, e)$$

191

Note that the order of argument in the definition of the projection is crucial as the Bregman divergence generally is not symmetric and not always convex with respect to its second argument.

194

Hyperplane separation. The set of points equidistant, in terms of Bregman divergence, to a pair of distinct points is an hyperplane.

197

Weak form of Pythagoras theorem.

199

$$D_J(e_1, e_3) = D_J(e_1, e_2) + D_J(e_2, e_3) - \langle p_3 - p_2, e_1 - e_2 \rangle$$

200

then

201

when $C$ is a convex set:

202

$$\forall e_2 \in IR^n \text{ et } \forall e_1 \in C, D_J(e_1, e_2) \geq D_J(e_1, \tilde{e}_2) + D_J(\tilde{e}_2, e_2)$$

203

when $C$ is an affine subspace:

204

$$\forall e_2 \in IR^n \text{ et } \forall e_1 \in C, D_J(e_1, e_2) = D_J(e_1, \tilde{e}_2) + D_J(\tilde{e}_2, e_2)$$

205

206

As the generating function $J$ is defined up to an affine function because of the linearity property:
$$D_{(a,e)+b}(e_1, e_2) \equiv 0, \quad D_{J+F}(e_1, e_2) = D_J(e_1, e_2) + D_F(e_1, e_2)$$

we can suppose, or prescribe, that $J(0)$ and $\nabla J(0)$ are zero: $D_J \equiv D_{J-J(0)-\nabla J(0),e}$. In this case, it can easily be seen that the following properties hold:

$$D_J(e, 0) \equiv J(e) \leq \langle \nabla J(e), e \rangle \quad (3)$$

As the Bregman divergence can nevertheless be symmetric, even it is generally not the case; it is interesting to derive a characterization of the symmetric Bregman divergences, because the preceding properties are clearly improved.

**Proposition: Characterization of symmetric Bregman divergences**

The Bregman divergence generated by the convex function $J$ is symmetric iff $J$ is a (positive) quadratic function.

The proof is provided in annex A. This condition is stronger than the condition of positive homogeneity of degree two. For example in $IR$, the Bregman divergence generated by the function positively homogeneous of degree two $H(x) = a(\langle x \rangle^+) + b(\langle x \rangle^-)^2$, with $a \neq b$, cannot be symmetric. Indeed for any $x > 0$:

$$D_H(x, -x) = ax^2 + 3bx^2 \neq bx^2 + 3ax^2 = D_H(-x, x)$$

This result, perhaps disappointing by the restriction it imposes on the generating functions $J$, has nevertheless as a corollary that symmetric Bregman divergences are actually distances (namely Mahalanobis distances in finite dimension).

**Corollary:** Symmetric Bregman divergences satisfy the triangle inequality and are therefore distances.

## 2.2 Symmetrized Bregman divergences, Bregman gaps and $\epsilon$-Bregman gaps

For application in thermomechanics, it can be desirable to have concepts of dissimilarity that first give the same role to both components of the pair $(e_1, e_2)$ to be analyzed, and second involve primal and dual variables as well, because their dual product has a strong energetic sense. That is the reason why we introduce first the notion of symmetrized Bregman divergence and second the notion of Bregman gap. The symmetrized Bregman divergence is simply taken as the sum of the Bregman divergences (the first idea seems to come back to 1946, [20]):

$$D_J^s(e_1, e_2) = D_J(e_1, e_2) + D_J(e_2, e_1) \quad (4)$$

Another definitions have been given : the Jensen-Bregman divergence [21]:

$$JB_J(e_1, e_2) = D_J(e_1, \frac{e_1 + e_2}{2}) + D_J(e_2, \frac{e_1 + e_2}{2}) \quad (5)$$

and the symmetrized divergence of Chen et al. [22] (a positive quantity because $J$ is convex):

$$m_J(e_1, e_2) = \frac{1}{2}[J(e_1) + J(e_2)] - J(\frac{e_1 + e_2}{2}) \quad (6)$$

grounded on the property : $m_J(e_1, e_2) = \min_z \frac{1}{2}[D_J(e_1, z) + D_J(e_2, z)]$. The properties of the symmetrized divergence (6), mainly for the scalar case, rely strongly on the regularity of the generating function, at
least a $C^2$. Nevertheless, in the context of thermomechanics, potentials useful for defining generating functions can be not differentiable (see part 3).

These definitions are not equivalent even for symmetric Bregman divergences and the first definition (4) is chosen here.

<figure><img src="image_6.png" /><figcaption>Fig. 3: Geometric interpretation of the Symmetrized Bregman divergence in IR for differentiable generating functions</figcaption></figure>

If the generating function of the Bregman divergence is differentiable, then the symmetrized Bregman divergence takes a very simple form, which can be used to calculate it:

$$D_j^s(e_1, e_2) = \langle \nabla J(e_1) - \nabla J(e_2), e_1 - e_2 \rangle \quad (7)$$

This expression provides another justification for the positivity the symmetrized Bregman divergence by the property of monotonicity of the gradient of a convex function. It also provides a convenient procedure for calculation of the symmetrized Bregman divergence, the following example is obtained in this way:

$$J(e) = \frac{1}{2} \left[ e^4 + \frac{e^2}{2} - e^2 \ln e \right] \Rightarrow D_j^s(e_1, e_2) = 2(e_1^3 - e_2^3 + e \ln e_1 - e_2 \ln e_2)(e_1 - e_2)$$

In the framework of linear elasticity in small perturbations, then if $e$ is the linearized strain tensor $\epsilon$, and $J$ is taken as $\phi$ the elastic energy density, the symmetric Bregman divergence is the so-called error in (elastic) constitutive equation, involving also the stress tensor $\sigma$, or twice the energy of the difference, or even up to a multiplication coefficient, the Khon-Vogelius functional integrand:

$$D_{\phi}^{s}(\epsilon_{1}, \epsilon_{2}) = \langle \nabla \phi(\epsilon_{1}) - \nabla \phi(\epsilon_{2}), \epsilon_{1} - \epsilon_{2} \rangle = (\sigma_{1} - \sigma_{2})(\epsilon_{1} - \epsilon_{2}) = 2\phi(\epsilon_{1} - \epsilon_{2}) \quad (8)$$

Moreover when the generating function $J$ of the Bregman divergence is twice differentiable, it can be given a quadratic expression of the symmetric divergence using the Hessian value at some point $e$ of the segment $[e_1, e_2]$, thanks to the intermediate value theorem, but obviously the point $e$ depends on the pair $(e_1, e_2)$:

$$D_j^s(e_1, e_2) = {}^t(e_1 - e_2) \nabla^2 J(e)(e_1 - e_2) \quad e \in [e_1, e_2]$$

The symmetric Bregman divergence, although symmetric by construction, still does not possess generically the triangle property; nevertheless it can be shown that the violation of triangle inequality is limited by a quantity related to the Hessian bounds [23]:

$$\exists m, M > 0 \quad m^t \xi \cdot \xi \le {}^t \xi \cdot \nabla^2 J \cdot \xi \le M^t \xi \cdot \xi \quad \Rightarrow \\ \sqrt{D_j^s(e_1, e_3)} \le \sqrt{D_j^s(e_1, e_2)} + \sqrt{D_j^s(e_2, e_3)} + (\sqrt{M} - \sqrt{m}) [D_j^s(e_1, e_2) D_j^s(e_2, e_3)]^{1/4}$$

It is worth noting that the symmetrized Bregman divergence, as a function of $(e_1, e_2)$, is generally not separately convex. This can be illustrated by the simple counter example of the generating function $J(e) = e - \sin e$ on $[0, \pi]$. The symmetrized divergence with the origin $D_j(0, e) = \nabla J(e) \cdot e = e(1 - \cos e)$
300, is not convex over the entire segment $[0, \pi]$. The same counter-example applies to the Jensen-Bregman divergence (5) and the Chen et al. symmetrized divergence (6) as well.

301
302 With a view to application in thermomechanics, it is of interest to introduce a concept that enjoys symmetry while simultaneously involving dual and primal variables. More precisely, it is useful in applications concerning data structured by an underlying physics (experimental data or data from numerical simulations involving partial differential equations), to have divergences or discrepancy measures between couples of dual variables (e.g. stress and strain tensors, temperature gradient and heat flux, ...).

308
309 **Definition:** Bregman Gap
310
Let $J$ be a convex, not necessarily differentiable function, the Bregman gap $BG_J$ generated by $J$ between $e_1$ and the couple of dual quantities $(e_2, p_2)$, $p_2 \in \partial J(e_2)$, is the non-negative quantity:

311
312

$$
313 \quad BG_J(e_1, [e_2, p_2]) = J(e_1) - J(e_2) - \langle p_2, e_1 - e_2 \rangle \quad (9)
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
$$

Here in contrast with the definition (1) of Bregman divergences for non-differentiable functions, no choice is made for the element in the subdifferential of $J$ at $e_2$ because it is itself an argument of the Bregman Gap. In turn, this notion allows to define a symmetric gap between any couples of dual variables $[e_2, p_1]$, and $[e_2, p_2]$.

320 **Definition:** Symmetrized Bregman Gap
321
322
323

Generated by the convex function $J$, the Symmetrized Bregman Gap $BG_J^s$ between the two couples of dual quantities $(e_1, p_1)$ and $(e_2, p_2)$, $p_1 \in \partial J(e_1)$, $p_2 \in \partial J(e_2)$, is the nonnegative scalar :

$$
324 \quad BG_J^s([e_1, p_1], [e_2, p_2]) = BG_J(e_1, [e_2, p_2]) + BG_J(e_2, [e_1, p_1]) \quad (10)
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
$$

A straightforward calculation gives the following expression for the Symmetrized Bregman Gap:

$$
328 \quad BG_J^s([e_1, p_1], [e_2, p_2]) = \langle p_1 - p_2, e_1 - e_2 \rangle \ge 0 \quad (10)
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
$$

The generating function does not appear anymore in this expression, but its positivity relies strongly on the existence of a convex function $J$ such that $p_i \in \partial J(e_i)$, because this expression is nothing but the monotony of the differential of convex functions. In applications, the determination of $J$ is then not necessary as soon as the $p_i$ are available, although its existence is mandatory. Going back to the example of linear elasticity where the elastic energy density $\varphi$ is quadratic (provided the reference state is stress-free), the Symmetrized Bregman Gap between two states $(\sigma_1, \varepsilon_1)$ and $(\sigma_2, \varepsilon_2)$, or their discrepancy measure, is simply twice the elastic energy of the difference of the strain tensors $2\varphi(\varepsilon_1 - \varepsilon_2)$, or twice the elastic complementary energy of the difference of the stress tensors $2\varphi^*(\sigma_1 - \sigma_2)$, provided that $\sigma_i \in \partial_e \varphi(\varepsilon_i)$, $\varepsilon_i \in \partial_\sigma \varphi^*(\sigma_i)$. Again, an expression already existing is the literature for application with elastic materials or structures with quadratic potentials is recovered.

340
341 More generally, the convex framework ([24],[25]) used here leads to two properties involving the convex conjugate $J^*$ of the generating function $J$.
342
343
344
345
346
347
348
349
350
$$

The Bregman Gap generated by the convex function $J$ is nothing but the Legendre-Fenchel residual

346 If $J^*$ is the Legendre-Fenchel conjugate function of $J$ then: $D_J(e_1, e_2) = D_{J^*}(\underline{p}_2, \underline{p}_1)$ or if the function

$$
347 \quad J \text{ is differentiable: } D_J(e_1, e_2) = D_{J^*}(\nabla J(e_2), \nabla J(e_1))
348
349
350
$$

These results come from the property of the Legendre-Fenchel transform:
351

$$
J(e) + J^*(p) = \langle p, e \rangle \Leftrightarrow e \in \partial J^*(p), p \in \partial J(e)
$$

352

So that:

353

$$
\begin{align*}
D_J(e_1, e_2) &= J(e_1) - \langle \underline{p}_2, e_1 \rangle - (J(e_2) - \langle \underline{p}_2, e_2 \rangle) \\
&= J(e_1) - \langle \underline{p}_2, e_1 \rangle + J^*(\underline{p}_2) \\
&= \langle p_1, e_1 \rangle - J^*(p_1) - \langle \underline{p}_2, e_1 \rangle + J^*(\underline{p}_2) \quad \forall p_1 \in \partial J(e_1) \\
&= J^*(\underline{p}_2) - J^*(\underline{p}_1) - \langle \underline{p}_2 - \underline{p}_1, e_1 \rangle \\
&= D_{J^*}(\underline{p}_2, \underline{p}_1)
\end{align*}
$$

354

355

356

357

358

359

360

361

362

363

364

365

366

367

This expression makes particularly sense in a thermodynamic framework were the thermodynamic potentials (cf. part 3) are used as generating functions: coherently, dual potentials are in correspondence in the Bregman divergence through the dual variables. Because the biconjugate $J^{**}$ of any function $J$ is convex (like its conjugate $J^{*}$), one can use it in order to derive a Bregman divergence for a nonconvex function. And then we have again (because $J^{***}=J^{*}$) :

$$
D_{J^{**}}(e_1, e_2) = D_{J^{*}}(\underline{p}_2, \underline{p}_1)
$$

368

As an example, the following double-well free energy potential, used in phase transformation models, is not convex, but its conjugate is convex, although not strictly convex, and can be calculated by taking advantage from the fact that the biconjugate of $J$ is also the largest closed convex function smaller than $J$.

369

370

$$
J(e) = \frac{1}{2}(e^2 - 1)^2, \quad J^{**}(e) = \begin{cases} \frac{1}{2}(e^2 - 1)^2 & \text{if } |e| < 1 \\ 0 & \text{if } |e| < 1 \end{cases}
$$

<figure><img src="image_7.png" /></figure>

371

372

373

374

375

376

However, the separation property of the generated Bregman divergence fails in the interphase segment [-1,1].

378

$$
D_{J^{*}}(e_{1}, e_{2}) = \frac{1}{2}(e_{1}^{2} - 1)^{2} I_{[-1,1]}(e_{1}) - \frac{1}{2}(e_{2}^{2} - 1)^{2} I_{[-1,1]}(e_{2}) - 2e_{2}(e_{2}^{2} - 1)(e_{1} - e_{2}) I_{[-1,1]}(e_{2})
$$

379

380

381

382

383

384

385

386

<caption>**Fig. 4:** Original function $J$ (dotted line) and its biconjugate (solid line) that can be used as generating function for a Bregman divergence and associated concepts</caption>

387

388

389

In applications, the dual quantities $p$, a gradient or a sub-gradient, are often prone to errors, coming either from computations or measurements. If an estimation of the approximation level, say $\epsilon$ is known, we can use the notion of $\epsilon$-subdifferential, coming from the optimization community ([26],[27]) and proved to be the right approximation notion for subdifferential, and incorporate it in a definition of $\epsilon$-Bregman Gaps.

390

$$
J(v) - J(e) - \langle p, v - e \rangle \geq -\epsilon \quad \forall v \text{ or equivalently } J(e) + J^*(p) - \langle p, e \rangle \leq \epsilon
$$
391 The set of all ε-subgradients at e is called the ε-sub differential of J at e, and denoted by $\partial_{\varepsilon}J(e)$.

392 If J is a quadratic function, $J(x) = \frac{1}{2}x'Qx$, then the ε-subdifferential is simply the set

393 $\partial_{\varepsilon}J(x) = \{Qx + \tau : \tau'Q^{-1}\tau \le 2\varepsilon\}$. In the context of linear elasticity, the ε-stress tensors associated to a

394 strain tensor e, are then the sum of the actual stress tensor σ associated by the constitutive equation to e

395 and any stress tensor with elastic energy lesser than ε. As an example in one dimension, the ε-

396 subdifferential of the function $J(e) = |e|$ is illustrated on figure 5

397

<figure><img src="image_8.png" /></figure>

398

<caption>Fig. 5: ε-sub-differential of $J(e) = |e|$ as a function of e along the real axis for ε=0.4. For ε=0, the subdifferential (heavy grey line) is the segment [-1,1] at e=0, and reduces to {1} and {-1} for e>0 and e<0 respectively.</caption>

402

403 **Definition: ε-Bregman Gap,**

404 Let J be a convex, not necessarily differentiable function, and ε a non-negative scalar:

405

406 i) the ε-Bregman gap $\varepsilon BG_J$ generated by J between $e_1$ and the couple of approximate dual quantities $(e_2, p_2)$, $p_2 \in \partial_{\varepsilon}J(e_2)$, is the non-negative quantity:

408

$$ \varepsilon BG_J(e_1, [e_2, p_2]) = J(e_1) - J(e_2) - \langle p_2, e_1 - e_2 \rangle + \varepsilon, \quad p_2 \in \partial_{\varepsilon}J(e_2) $$

409

410

411 ii) the ε-symmetrized Bregman gap $\varepsilon BG_J^s$ generated by J between the couple of approximate dual quantities $(e_1, p_1)$ and $(e_2, p_2)$, $p_1 \in \partial_{\varepsilon}J(e_1)$, $p_2 \in \partial_{\varepsilon}J(e_2)$ is the non-negative quantity:

412

413

$$ \varepsilon BG_J^s([e_1, p_1], [e_2, p_2]) = \varepsilon BG_J(e_1, [e_2, p_2]) + \varepsilon BG_J(e_2, [e_1, p_1]) = \langle p_1 - p_2, e_1 - e_2 \rangle + 2\varepsilon \quad p_1 \in \partial_{\varepsilon}J(e_1), p_2 \in \partial_{\varepsilon}J(e_2) $$

414

415

416 The interest of this definition is that, it leads to a positive discrepancy measure without knowing the generating function but only provided it exists with an approximation ε for the relation $p \in \partial J(e)$.

417 Unfortunately, any property of separated convexity does not make sense for $BG^s$ nor for $\varepsilon-BG^s$ because their domains are not convex unless the generating function is quadratic. Indeed, the convexity would necessitate the convexity of the domains, and therefore prescribe the following implication, which is not ensured except for quadratic functions:

422

$$ p_1 \in \partial J(e_1), p_2 \in \partial J(e_2) \Rightarrow (\lambda p_1 + (1-\lambda)p_2) \in \partial J(\lambda e_1 + (1-\lambda)e_2) $$

423

424

### 3 Thermodynamic potentials as Bregman generating functions in thermomechanics

425 The objective of section is to build suitable Bregman Divergences and Bregman Gaps for the processing of data that originate either from physical experiments or from simulation computations of these same physical phenomena. As mentioned in the introduction, it is believed that taking into account the specific structure of these data that is brought by the physical constraints, conservations laws or constitutive
equations can lead to better performance of the algorithms used for processing this kind of data. Indeed metrics and scalar products are at the heart of processing methods such as Proper Order Decomposition used for the sparse representation of the data, identification of structures buried in it or model reductions for the PDE governing it, or such as classification, clustering and regression algorithms in learning, often by the way of minimizing discrepancy functions between data. The applications to nonlinear cases, emanating notably from the nonlinearity of the constitutive equation (for small deformation), are appealing for the definition of non-quadratic pseudo (squared) metrics.

While the merits of various metrics or pseudo-metrics, scalar or pseudo scalar products, can be considered at the very least to be more or less the same when applied to data corresponding to a single physics, the situation is very different in the case of multi-physics data relating to coupled phenomena. This is due on the one hand to the question of the physical units of the data, and on the other hand to the need to make the best use of a priori information contained in the coupled equations satisfied by the data. Even with uncertainties, it is desirable to construct more elaborate metrics than the usual Euclidean metrics of $IR^n$, and the concepts associated with the Bregman divergence seem fruitful in that respect. The main reason for this is the flexibility they allow in the choice of generating functions

In mechanics, in thermics or more generally as soon as thermodynamic phenomena are involved, it is natural, when choosing generating functions for Bregman divergence, to turn to thermodynamic potentials or dissipation functions and potentials. Indeed, in a large majority of cases, these are convex functions of their arguments. Furthermore, these potentials were fundamentally conceived to introduce an energetic notion but also to define the relations between dual variables, and it is precisely this last particularity that can be exploited with Bregman divergences. It appears then the gap functions defined by the symmetric Bregman gaps coincide with some forms of errors in constitutive equations. In multi-physics situations, advantage can be taken of the additivity property of the Bregman divergence (cf. part 2.1):

$$D_{\lambda J+\mu F}(e_1, e_2) = \lambda D_J(e_1, e_2) + \mu D_F(e_1, e_2) \quad (J, F \text{ convex } \lambda, \mu \text{ positive scalars})$$

for combining various potentials either coupled or involving different physical quantities, in order to build pseudo-metrics covering all the involved data and taking into account all the couplings between them. In this section, advantage is taken from the convex framework of generalized standard materials to build ad hoc Bregman gaps, and extension is made for the generalization of this framework with implicit standard materials. Nevertheless, the classical framework of linear elliptic problems first is analyzed, where the (linear) constitutive equations are incorporated directly in the conservation laws.

### 3.1 Potentials for data from physical fields

When the data to be processed come from vector fields, issued from experiments or from numerical simulations, more or less accurately obeying a system of PDEs, energy quantities such as the potentials from which these equations derive can be used for the building of generating functions for Bregman divergences. An important first field of application is provided by the framework of linear symmetric PDEs. If, in the quasi-static assumption, the data field $u(x)$ on a spatial domain $\Omega$ is supposed to obey such a system of equations, the variational form of these equations reads:

$$u \in V, \quad a(u, v) = l(v) \quad \forall v \in V_0$$

Where $a$ is a symmetric bilinear form defined on the product of functions spaces $U(\Omega)xU(\Omega)$, $l$ is a linear form on $U$, $V \subset U$ is the space of admissible fields (with respect to boundary conditions on parts of $\partial\Omega$) and $V_0$ its tangent space. If, $a$ is continuous and coercive and $l$ is continuous, the solutions of the system are characterized by the optimization property of a potential $P$:

$$u = \arg \min_{V} P(v) = \frac{1}{2} a(v, v) - l(v)$$

The convexity and positivity of $a$ lead to consider the global Bregman divergence, i.e. having as argument the field $u$ over $\Omega$, and generated by the quadratic form associated with $a$.
$$J(u) = a(u, u)$$

because $J$ is positive, convex and, in addition, $J(u)=0$ implies $u$ equals zero as well. $J$ being a quadratic function, the Bregman divergence generated by $J$ is symmetric and takes the simple form:

$$D_{J}(u, v) = a(u, u) - a(v, v) - 2a(v, u - v) = a(u - v, u - v)$$

The local Bregman divergence is generated by the integrand $A$ of $a$:

$$a(u, v) = \int_{\Omega} A(u(x), v(x))dx, \quad D_{A}(u(x), v(x)) = A(u(x) - v(x), u(x) - v(x))$$

The previous expression is an abuse of notation because in general $A$ is also a function of the derivatives of $u$ and $v$: $A([u, \partial u / \partial x_1, \dots], [v, \partial v / \partial x_1, \dots])$. $A(u, u)$ can be often interpreted as the density of a global energy or a dissipation $V(u) = 1/2 a(u, u)$ as illustrated in the table below.

In some situations where $A$ is only a function of the derivatives of $u$, and not of the values of $u$, attention must be paid to the space $V$, which is usually a quotient space, in order to recover the strict convexity by eliminating the “rigid modes” with no energy. The following table displays some examples.

<table><caption>**Table 2:** quadratic energies or dissipations associated with various linear physics</caption><thead><tr><th>Physics</th><th>U space</th><th>Energy or dissipation density A</th></tr></thead><tbody><tr><td>Scalar conduction or diffusion (thermal, electrical, Darcean flow, incompressible Stokes flow)</td><td>H<sup>1</sup>(Ω)</td><td><sup>1</sup>&frasl;<sub>2</sub> k &#x2219; &nabla;u &nabla;u</td></tr><tr><td>Elasticity</td><td>H<sup>1</sup>(Ω)<sup>3</sup></td><td><sup>1</sup>&frasl;<sub>2</sub> C &#x2219; &#x2219; &epsilon;(u) &#x2219; &epsilon;(u)</td></tr><tr><td>Plane elasticity on elastic support</td><td>H<sup>1</sup>(Ω)<sup>2</sup></td><td><sup>1</sup>&frasl;<sub>2</sub> C &#x2219; &#x2219; &epsilon;(u) &#x2219; &epsilon;(u) + <sup>1</sup>&frasl;<sub>2</sub> ku<sup>2</sup></td></tr></tbody></table>

The discrepancy measure generated by the Bregman divergence is of particular interest in relation with usual Euclidean metrics in situation where the medium is anisotropic and/or heterogenous [28]. Another important field of application arises in fluid mechanics. For Navier-Stokes equations where the variable of interest is the velocity field $v$, two quadratic functions can be used for generating Bregman divergences: the kinetic energy: $\rho\|v\|^2$ and the enstrophy $\rho\|rot v\|^2$. The enstrophy is the square of the vorticity for incompressible flows. Then, using the associated Bregman divergence allows to compare two velocity fields (or more precisely their fluctuations) in terms of the vorticity contents, this can be of interest for example in POD in order to identify coherent structures like vortexes in a flow.

## 3.2 Generating functions in thermomechanics for standard generalized materials

The theory of Generalized Standard Materials (SGM) [29] is a coherent thermodynamic framework for modelling constitutive equations using only convex potentials. In addition to the usual description of the thermomechanical state constituted by the state variables $(\epsilon, \sigma, T)$ that is (mechanical deformation, stress tensor, temperature), some internal (hidden) variables collectively denoted by $\alpha$ are introduced.

Then complementing the free energy (Gibbs energy), $\varphi(\epsilon, T, \alpha)$, a pseudo potential of dissipation appears as a function of the rates of the internal variables $D(\dot{\alpha})$, and the constitutive equations are split into two groups:

$$\sigma = \frac{\partial \varphi}{\partial \epsilon}, \quad S = -\frac{\partial \varphi}{\partial T}, \quad A = -\frac{\partial \varphi}{\partial \alpha} \quad \text{and} \quad A = \frac{\partial D}{\partial \dot{\alpha}}$$

The first group expresses the relation between dual variables ($S$ is the entropy, $A$ the thermodynamic force), the second group gives the evolution equation of the internal variable $\alpha$, and is extended for non-differentiable pseudo-potentials thanks to the notion of subdifferential: $A \in \partial D(\dot{\alpha})$. This extension is necessary when modeling constitutive equation involving yield functions, which are associated with positively homogeneous dissipation pseudo-potentials with degree one.
For the simplicity of the presentation we now limit ourselves to isothermal evolutions. The Euler implicit time incremental form of these constitutive equations reads:

$$ \sigma + \Delta\sigma = \frac{\partial\varphi}{\partial\varepsilon}[\varepsilon + \Delta\varepsilon, \alpha + \Delta\alpha], \quad A + \Delta A = -\frac{\partial\varphi}{\partial\alpha}[\varepsilon + \Delta\varepsilon, \alpha + \Delta\alpha], \quad A + \Delta A = \frac{\partial\mathcal{D}}{\partial\dot{\alpha}}\left(\frac{\Delta\alpha}{\Delta t}\right) \quad (11) $$

showing that the (thermodynamically) conjugate couples for describing the thermodynamic state are $[(\sigma + \Delta\sigma, \varepsilon + \Delta\varepsilon), (A + \Delta A, \alpha + \Delta\alpha)]$. So, denoting:

$$ \hat{\varphi}(\Delta\varepsilon, \Delta\alpha) = \varphi(\varepsilon + \Delta\varepsilon, \alpha + \Delta\alpha), \quad \hat{\mathcal{D}}(\Delta\alpha) = \mathcal{D}\left(\frac{\Delta\alpha}{\Delta t}\right) \quad (12) $$

a Bregman divergence on the increments of the primal state variables $\Delta e = (\Delta\varepsilon, \Delta\alpha)$ can be built using $J_{\chi} = (1-\chi)\hat{\varphi} + \chi\Delta t \hat{\mathcal{D}}$ as the generating function, where $\chi$ is a non-dimensional weighting factor between the free energy potential and the dissipation pseudo-potential ($\chi \in [0,1]$ and $\chi < 1$ if $\mathcal{D}$ is homogeneous of degree one, for preserving the strict convexity). This leads to the following Symmetrized Bregman Gap on incremental couples of dual state variables $[\Delta e, \Delta p]$, with $\Delta e = (\Delta\varepsilon, \Delta\alpha)$ and $\Delta p = (\Delta\sigma, \Delta A)$:

$$ BG_{J_{\chi}}^{s}([\Delta e_{1}, \Delta p_{1}], [\Delta e_{2}, \Delta p_{2}]) = (1-\chi)(\Delta\sigma_{1} - \Delta\sigma_{2}):(\Delta\varepsilon_{1} - \Delta\varepsilon_{2}) + (1-2\chi)(\Delta A_{1} - \Delta A_{2}, \Delta\alpha_{1} - \Delta\alpha_{2}) \quad (13) $$

This reduces to the so called Drücker mechanical error ([30], [31]) when $\chi$ takes the value $1/2$, that is when the free energy and dissipation are equally balanced in building the discrepancy measure.

$$ BG_{J_{1/2}}^{s}([\Delta e_{1}, \Delta p_{1}], [\Delta e_{2}, \Delta p_{2}]) = \frac{1}{2}(\Delta\sigma_{1} - \Delta\sigma_{2}):(\Delta\varepsilon_{1} - \Delta\varepsilon_{2}) \quad (14) $$

For perfect isotropic elasto-plasticity for example, the primal state is described by the couple $(\varepsilon, \varepsilon^p)$ where $\varepsilon^p$ is the plastic strain, the free energy is simply $\varphi(\varepsilon, \varepsilon^p) = 1/2 C:(\varepsilon - \varepsilon^p):(\varepsilon - \varepsilon^p)$ and the pseudo-potential of dissipation is the non-differentiable positively homogeneous function of degree one $\mathcal{D}(\dot{\varepsilon}^p) = \sigma_0 \|\dot{\varepsilon}^p\|_{\text{dev}}$. The Symmetrized Bregman Gap reads then (for $\chi=0$, the elastic energy gap is of course retrieved):

$$ BG_{J}^{s}([\Delta\varepsilon, \Delta\varepsilon_{1}^{p}),(\Delta\sigma_{1}, -\Delta\sigma_{1}], [\Delta\varepsilon_{2}, \Delta\varepsilon_{2}^{p}),(\Delta\sigma_{2}, -\Delta\sigma_{2}]) = (\Delta\sigma_{1} - \Delta\sigma_{2}):(\Delta\varepsilon_{1} - \Delta\varepsilon_{2}) : [(1-\chi)(\Delta\varepsilon_{1} - \Delta\varepsilon_{2}) - (1-2\chi):(\Delta\varepsilon_{1}^{p} - \Delta\varepsilon_{2}^{p})] $$

By a reverse approach to the situations in section 3.1, Global Symmetrized Bregman Gaps are simply built by integrating over the geometrical domain of interest $\Omega$. It is worth noting, that a direct approach on the domain $\Omega$ is also possible following [32]. Because the Bregman Gap takes exactly the value of the Fenchel residual [25] of its generating function $J$, involving its conjugate $J^*$:

$$ r_{F}(e, p) = J^{*}(p) + J(e) - \langle p, e \rangle $$

the Symmetrized Bregman gaps are closely related to the errors in constitutive equations largely used in thermomechanics for used. Furthermore, it is possible to define a dual Bregman Gap with $J^*$ as generating function with the property:

$$ BG_{J}(e_{1}, [e_{2}, p_{2}]) = BG_{J^{*}}(p_{2}, [p_{1}, e_{1}]) $$

This result just reflects that conjugate thermodynamic potentials can equivalently be used for building the Bregman Gaps within the framework of Standard Generalized Materials. Nonetheless, it has to be kept in mind that even if the generating function $J$ does not appear anymore in the final expression (7) or (8) of the Bregman gaps, existence of a convex generating function is the essential condition for these expressions being positive. In finite elasticity for example, the Coleman-Noll error which involves the gradient of the deformation $F$ and which is exactly the expression of the Bregman Gap, is not a positive quantity. Fortunately here, it is possible to turn to a weaker notion of convexity, namely the poly-
579 convexity [33], and to use then as primal variables (F, Cof F, det F) in a correctly build, and
580 consequently positive, Bregman Gap.

581

### 3.3 Bregman gaps generated by bi-potentials

582 Unfortunately, while the SGM framework is very wide, there are still many constitutive laws that do not
583 fit into it, such as Armstrong-Frederick hardening plasticity, Drücker-Prager elastoplasticity, Clam-Clay
584 models, some damage models or even Coulomb friction. An extension of the framework of Standard
585 Generalized Materials has been proposed [34] in order to take into account these behaviors that do not
586 obey the normality law in the complementary equation. In this formulation, the pseudo-dissipation
587 potential becomes a function of both the primal variable e and its dual variable p, justifying the name of
588 bi-potential, or implicit potential since the relationship between e and p is expressed by a subdifferential
589 implicit equation:

590

591

$$e \in \partial_p b(e, p) \text{ or } p \in \partial_e b(e, p)$$

592

### Definition: Bi-potentials [34]

593 Let e and p be two vectors of variables in duality. A function b of $IR^n \times IR^n$ with value in $IR \cup \{+\infty\}$ is a
595 bi-potential for (e,p) if and only if:

596

597 i) b is separately convex and lower semi-continuous for e and p

598 ii) b satisfies the generalized Fenchel inequality: $b(e, p) \geq \langle e, p \rangle \quad \forall e, p$

599 iii) $b(e, p) = \langle e, p \rangle \Leftrightarrow e \in \partial_p b(e, p) \Leftrightarrow p \in \partial_e b(e, p)$

600

601 The material models defined using a bi-potential for the dissipation pseudo-potential are called Implicit
602 Generalized Standard Materials. Indeed, in the particular case where b is a separate variables function,
603 $b(e, p) = f(e) + g(p)$, then necessarily g and f are mutually conjugate. Therefore, Generalized Standard
604 Materials are a special cases of Implicit Generalized Standard Materials for which the bi-potential takes
605 the special form: $b(e, p) = f(e) + f^*(p)$. General bi-potentials can be used as generating functions for
606 a Bregman Gap provided that the definition of the latter is adapted as follows.

607

608 Definition: Bregman Gap generated by a bi-potential

609 The Bregman gap generated by the bi-potential $b(e,p)$ is defined by the positive quantity:

610

$$BG_b(e_1, [e_2, p_2]) = b(e_1, p_2) - b(e_2, p_2) - \langle p_2, e_1 - e_2 \rangle \quad \text{with } p_2 \in \partial_p b(e_2, p_2) \quad (15)$$

611

$$613 \qquad \textit{or equivalently:} \qquad BG_b(e_1, [e_2, p_2]) = b(e_1, p_2) - \langle p_2, e_1 \rangle \qquad (16)$$

614

615 This definition generalizes the initial definition of Bregman gaps: if the bi-potential is a separate
616 variables function, then the genuine definition is retrieved. Similarly, Symmetrized Bregman gaps can
617 be derived.

618

619 Definition: Symmetrized Bregman Gap generated by a bi-potential

620 Let b be a bi-potential, the Symmetrized Bregman Gap generated by b is:

621

$$623 \qquad \begin{aligned} BG_b^s([e_1, p_1], [e_2, p_2]) &= BG_b(e_1, [e_2, p_2]) + BG_b(e_2, [e_1, p_1]) \\ &= b(e_1, p_2) + b(e_2, p_1) - \langle p_2, e_1 \rangle - \langle p_1, e_2 \rangle \end{aligned} \qquad (17)$$

624

625 As soon as the bi-potential b is (separately) strictly convex, nullity of the Symmetric Bregman Gap
626 generated by b implies the equality between primal state variables $e_1$ and $e_2$ and the associated dual
627 variables $p_1$ and $p_2$. However, the normality law, characteristic of Generalized Standard Materials, being
628 deduced from the inequality $\langle p_1 - p_2, e_1 - e_2 \rangle \geq 0$, it can be then expected that, since it is violated for
629 non-standard materials, the reduction to (10) of the Symmetrized Bregman Gap to this expression is no
630 longer verified for Symmetrized Bregman Gaps generated by a bi-potential. Indeed,

631
632

$$BG_b^s([e_1, p_1], [e_2, p_2]) = \langle p_1 - p_2, e_1 - e_2 \rangle + R$$

633

$$R = b(e_1, p_2) + b(e_2, p_1) - \langle p_1, e_1 \rangle - \langle p_2, e_2 \rangle \neq 0$$

634

However, the following calculation ensures the positivity of $BG_b^s$:

635

$$R = b(e_1, p_2) + b(e_2, p_1) - b(e_1, p_1) - b(e_2, p_2) \geq \langle p_1, e_2 - e_1 \rangle - \langle p_2, e_1 - e_2 \rangle = -\langle p_1 - p_2, e_1 - e_2 \rangle$$

636

637

This results shows that for constitutive relations not entering the Standard Generalized Material framework but the implicit potential framework, convex generating functions for Bregman gaps can be built with bi-potentials. This extends considerably the application field of the Symmetrized Bregman Gaps.

642

643

## 3.4 Expression of Symmetrized Bregman Gaps for various modelling in thermomechanics

644

645

The following table gathers some Symmetrized Bregman Gaps associated with phenomena (or models) encountered in thermo-mechanics. Global symmetrized Bregman Gaps are obtained by spatial integration over the domain occupied by the solid or over the interface between two contacting solids.

646

647

648

<table><thead><tr><th>Phenomenon</th><th>Generating function</th><th>Symmetrized Bregman Gap</th></tr></thead><tbody><tr><td>Scalar conduction or diffusion</td><td>Dissipation function</td><td>K∇(u<sub>1</sub> - u<sub>2</sub>)·∇(u<sub>1</sub> - u<sub>2</sub>)</td></tr><tr><td>Linear elasticity</td><td>Elastic energy</td><td>C : ε(u<sub>1</sub> - u<sub>2</sub>) : ε(u<sub>1</sub> - u<sub>2</sub>)</td></tr><tr><td>Non-linear elasticity</td><td>Elastic energy (if convex)</td><td>(Δσ<sub>1</sub> - Δσ<sub>2</sub>) : (Δε(u<sub>1</sub>) - Δε(u<sub>2</sub>))</td></tr><tr><td>Hyper-elasticity (Finite strain)</td><td>Polyconvex elastic energy (variables : cofactors of F)</td><td>(T<sub>1</sub> - T<sub>2</sub>) : (M<sub>1</sub> - M<sub>2</sub>) + (cT<sub>1</sub> - cT<sub>2</sub>) : (N<sub>1</sub> - N<sub>2</sub>) + (p<sub>1</sub> - p<sub>2</sub>)(d<sub>1</sub> - d<sub>2</sub>)</td></tr><tr><td>Standard Elastoplasticity</td><td>Free energy Dissipation Pseudo-potential</td><td>(1 - χ)(Δσ<sub>1</sub> - Δσ<sub>2</sub>) : (Δε(u<sub>1</sub>) - Δε(u<sub>2</sub>)) + χ(ΔA<sub>1</sub> - ΔA<sub>2</sub>) : (Δα<sub>1</sub> - Δα<sub>2</sub>)</td></tr><tr><td>Contact Friction (contact zone Γ<sub>c</sub>)</td><td>Dissipation bi-potential</td><td>(σ<sup>1</sup><sub>nt</sub> - σ<sup>2</sup><sub>nt</sub>)(u<sub>t</sub><sup>1</sup> - u<sub>t</sub><sup>2</sup>) - (ρσ<sup>1</sup><sub>nt</sub> - ρσ<sup>2</sup><sub>nt</sub>)(|u<sub>t</sub><sup>1</sup>| - |u<sub>t</sub><sup>2</sup>|)</td></tr><tr><td>Non-standard elastoplasticity</td><td>Elastic energy Dissipation bi-potential</td><td>(1 - χ)(Δσ<sub>1</sub> - Δσ<sub>2</sub>) : (Δε(u<sub>1</sub>) - Δε(u<sub>2</sub>)) + χ(ΔA<sub>1</sub> - ΔA<sub>2</sub>) : (Δα<sub>1</sub> - Δα<sub>2</sub>)</td></tr><tr><td>Thermo-elasticity</td><td>Elastic energy Thermal Dissipation</td><td>(1 - χ)C : [ε(u<sub>1</sub> - u<sub>2</sub>) - α(T<sub>1</sub> - T<sub>2</sub>)Id] : [ε(u<sub>1</sub> - u<sub>2</sub>) - α(T<sub>1</sub> - T<sub>2</sub>)Id] + χDK∇(u<sub>1</sub> - u<sub>2</sub>)·∇(u<sub>1</sub> - u<sub>2</sub>)</td></tr></tbody></table>

649

650

651

652

653

The expression of Global Symmetrized Bregman Gaps can sometimes be drastically simplified by using in supplement the equilibrium equation satisfied by the stress field and the compatibility of the strain tensor field. This is the case for the Drücker error (14):

654

655

656

$$\int_{\Omega} (\Delta\sigma_{1} - \Delta\sigma_{2}) : (\Delta\varepsilon_{1} - \Delta\varepsilon_{2}) d\Omega \equiv \int_{\partial\Omega} (\Delta\sigma_{1} \cdot n - \Delta\sigma_{2} \cdot n) : (\Delta u_{1} - \Delta u_{2}) dS \quad (18)$$

658

659

This leads to significant reduction of the burden computations

660
## 4 Computational geometry with Bregman concepts

Various learning techniques like SVM, clustering or partitioning, manifold learning, topological data analysis, classification etc. rely on basic problems of computational geometry, where a distance or a metric usually named loss function in learning application, plays a central role. In order to use Bregman divergences or Bregman gaps as loss functions, hopefully better adapted to the processing of data structured by physics, we give below some essential geometric properties. Methods for dimension reduction like the Proper Orthogonal Decomposition methods or generalizations of them can also benefit from more physically informed scalar products or, even more importantly, inner-products on product-spaces of different physical quantities [35]. Computational geometry with Bregman divergences can be challenging because of the non-symmetry of the original Bregman divergence but also and over all because of the violation of the triangle inequality, except for quadratic Bregman divergences (Mahalanobis distance). Nevertheless, the convex basis of the Bregman divergence leads to properties that allow almost all the operations of computational geometry, sometimes at the cost of slightly altered concepts. In this section are given some geometric properties and indications on the computation of interesting geometric quantities regarding applications.

676

### 4.1 Centroid properties and Bregman balls

The first and essential geometric property of the Bregman divergence, especially for applications in clustering is the following. It expresses that the mean of a finite set $S$ of $N$ points minimizes the sum of Bregman divergences or Bregman Gaps between this point and any other points of the set $S$, whatever the choice of the Bregman divergence or gap.

682

**Property:** Centroid property for Bregman divergences [16] and for Symmetrized Bregman Gaps

For any set $S$ of $N$ points in $\mathbb{R}^n$, and any Bregman divergence $D_J$, one has:

$$ \mu = \frac{1}{N} \sum_{i=1,N} x_i = \arg\min_{s \in \mathbb{R}^n} C_J(s) \equiv \frac{1}{N} \sum_{i=1,N} D_J(x_i, s) \quad (19) $$

For any set $S$ of $N$ couples $[e_i, p_i]$, dual points with respect to any convex function $J$ in $\mathbb{R}^{2n}$, and for the symmetrized Bregman gap $BG_J^s$ generated by $J$, one has:

$$ \begin{bmatrix} \mu_e \\ \mu_p \end{bmatrix} = \frac{1}{N} \sum_{i=1,N} \begin{bmatrix} e_i \\ p_i \end{bmatrix} = \arg\min_{\begin{bmatrix} x \\ y \end{bmatrix} \in \mathbb{R}^n} C_J([x,y]) = \frac{1}{N} \sum_{i=1,N} BG_J^s([e_i, p_i], [x,y]) \quad (20) $$

*Proof:* Using the expression of $BG_J^s$, we can calculate the derivatives of $C_J$ with respect to $x$ and $y$:

$$ \begin{align*} C_J([x, y]) &= \frac{1}{N} \sum_{i=1,N} \langle e_i - x, p_i - y \rangle = \frac{1}{N} \sum_{i=1,N} \langle e_i, p_i \rangle - \left\langle \frac{1}{N} \sum_{i=1,N} e_i, y \right\rangle - \left\langle x, \frac{1}{N} \sum_{i=1,N} p_i \right\rangle + \langle x, y \rangle \\ \frac{\partial}{\partial x} C_J([e, p]) &= p - \frac{1}{N} \sum_{i=1,N} p_i, \quad \frac{\partial}{\partial y} C_J([e, p]) = e - \frac{1}{N} \sum_{i=1,N} e_i \end{align*} $$

Equating to zero the two derivatives leads to the announced result because the symmetrized Bregman gap, hence $C_J$, is separately convex. This also shows that the minimum is unique.

The property extends to the probabilistic context with the expectation replacing the mean,

$$ E_v[f(x)] = \frac{1}{N} \sum_{i=1,N} v_i f(x_i) $$

and an exhaustiveness property [36] states that the centroid property is in fact a characterization of Bregman divergences when the minimization is extended over every probability distributions. If the centroid property holds for a positive function $F$, such that $F(x, x) = 0$, then $F$ is necessarily a Bregman divergence. By the same arguments, this can be established also for the Symmetrized Bregman Gap.

705

The generalization of the ball and sphere concepts, usually defined by means of a distance, is straightforward for both Bregman divergences and gaps. Nevertheless, due to the non-symmetry of $D_J$, two distinct kinds of balls or spheres appear.
709

710 **Definition:** Bregman balls ([37], [38]) and Bregman Gap balls

711 The Bregman ball $BB_J$, respectively the second type Bregman Ball $BB'_J$, associated to the generating function $J$, with center $c$ and pseudo-radius $\rho$ is the set:

713

$$ BB_J(c, \rho) = \{e, D_J(e, c) \le \rho^2\} \\ BB'_J(c, \rho) = \{e, D_J(c, e) \le \rho^2\} \qquad (21) $$

714

715

716 the associated Bregman spheres are respectively $\partial BB_J(c, \rho)$ and $\partial BB'_J(c, \rho)$.

717
718

The Bregman Gap ball, associated to the generating function $J$, with center $\mu$ and radius $\rho$ is the set:

719

$$ BBG_J^s(\mu, \rho) = \{e, BG_J^s(e, \mu) \le \rho^2\} = \{e, BG_J^s(\mu, e) \le \rho^2\} \qquad (22) $$

720 the associated Bregman sphere is, $\partial BBG_J^s(c, \rho)$.

721

722 When the generating function is the Euclidean norm, the usual notions of balls and spheres are

723 recovered. As the Bregman divergences and symmetrized Bregman gaps are convex with respect to its

724 first variable, Bregman balls and Bregman Gap balls are convex sets (whereas the second type Bregman

725 balls $BB'_J$ are generally not). To get some insights into the concepts of Bregman balls, it is interesting

726 to look at origin centered balls, although these balls are not invariant by translation of their center.

727 Recalling that the generating function can always be chosen with $J(0)=\nabla J(0)=0$, the simple expressions are obtained:

728

$$ BB_J(0, \rho) = \{J(e) \le \rho\}, \quad BB'_J(0, \rho) = \{\nabla J(e) \cdot e - J(e) \le \rho\} \\ BBG_J^s(0, \rho) = \{\nabla J(e) \cdot e \le \rho\} \qquad (23) $$

730

731

732 Illustrations of these different balls are displayed on figure 4 for various generating functions.

733

<figure><img src="image_9.png" /><figcaption>Figure 6: Bregman Balls
a) $BB$ and $BB'$ for Itakura-Saito, and Kullback-Leibler Bregman divergences, from [NN1].
b) Symmetrized Bregman Gap origin-centered balls generated by the elastic potential for an isotropic elastic material with different moduli in traction and compression</figcaption></figure>

734

735

736

737

738

739

740 If the generating function $J$ is positively homogeneous with degree $\alpha$ (necessarily $\alpha >1$), these

741 expressions can be further simplified and involve only the value of the function $J$. This illustrates the

742 strong relation between the Bregman balls and the level-lines of their generating functions:

743

$$ BB_J(0, \rho) = \{J(e) \le \rho\}, \quad BB'_J(0, \rho) = \{(\alpha - 1)J(e) \le \rho\} \\ BBG_J^s(0, \rho) = \{\alpha J(e) \le \rho\} $$

744

745

746 These examples show how the physically information one can a priori have on the data can be taken into account and have potentially a strong qualitative impact on manipulating and comparing these data.

748 The concept of (convex) ball seems to be the right way of dealing with questions like nearest neighbor

749 determination or minimization of dissimilarity measures between physical fields. If the Bregman

750 divergence is strictly convex then the Smallest Enclosing Bregman Ball, in the sense of the smallest

751 radius $\rho$, is unique and an efficient algorithm has been given for computing it [39].

752
## 4.2 Computational geometry with Bregman concepts

This section is devoted to some basic tools in computational geometry that are commonly used in situations where a true distance or a true scalar product exists. Much work has been done with Bregman divergence in the learning community, especially for classification algorithms or building Voronoi diagrams ([38]-[42]). It was necessary for adapting methods that rely on the triangle inequality, a property that is not satisfied by most Bregman concepts, except of course for the Mahalanobis divergences which are norms (cf. section 2.1). Some results can be extended to Bregman gaps for use in applications in the field of thermomechanics and more generally in processing physically based data. The first useful concept is the bisector concept. Interestingly, bisectors are hyperplanes independently of the generating function, even strongly nonlinear.

763

764 **Definition:** Bregman bi-sectors [38] and Bregman gaps bisectors

765 The Bregman bisector of a pair $(e_1, e_2)$ is the set of equidistant points $x$ with respect to a Bregman divergence:

$$767 \qquad BBs_j(e_1, e_2) = \{x, D_j(x, e_1) = D_j(x, e_2)\}$$

768

769 The Bregman gap bisector of a couple of dual quantities $([e_1, p_1], ([e_2, p_2])$ is the set of equidistant points $x$ with respect to symmetrized Bregman gap:

770

$$772 \qquad BGBs_j([e_1, p_1], [e_2, p_2]) = \{[e, p], BG_j^s([e, p]) = BG_j^s([e, p], [e_2, p_2])\}$$

773

774 Bisectors are hyperplanes, the two points or couples lie on the different sides of this hyperplane.

775

776 The Bregman divergence being not symmetric, there is another notion of Bregman bisector, namely of the second type [38] which is generally not an hyperplane.

777

$$779 \qquad BBs'_j(e_1, e_2) = \{x, D_j(e_1, x) = D_j(e_2, x)\}$$

780

781 Regarding the Bregman balls, the classical approach for determining an orthogonal projection on a distance-defined ball cannot be followed because it relies strongly on the use of the triangle inequality. 782 Let $e$ be a point in $\mathbb{R}^n$ and a Bregman ball of center $c$ and radius $\rho$, with $e$ not belonging to the closure 783 of $BB_\lambda(c, \rho)$. The projection $\tilde{e}$ on $BB_\lambda(c, \rho)$ is:

784

785

$$786 \qquad \tilde{e} = \arg\min_{x \in BB_j(c, \rho)} D_j(x, e)$$

787

788 Because the ball is strictly convex, the projection lies on the boundary of $BB_\lambda(c, \rho)$ or hypersphere 789 $\partial BB_j(c, \rho)$, the projection $\tilde{e}$ can then be determined by the stationary point $(\tilde{e}, \lambda)$ of the following 790 Lagrangian:

791

$$792 \qquad L(x, \mu) = D_j(x, c) + \mu(BB_j(x, c) - \rho) \\ = (1 + \mu)J(x) - J(e) - \mu J(c) - \langle \nabla J(e) + \mu \nabla J(c), x \rangle + \langle \nabla J(e), e \rangle + \mu \langle \nabla J(c), c \rangle$$

793 So that:

$$794 \qquad D_x L(\tilde{e}, \lambda) \delta x = (1+\lambda) \langle \nabla J(\tilde{e}), \delta x \rangle - \langle \nabla J(e) + \lambda \nabla J(c), \delta x \rangle \quad \forall \delta x \\ = (\langle 1+\lambda \nabla J(\tilde{e}) - \nabla J(e) - \lambda \nabla J(c), \delta x \rangle) \quad \forall \delta x \\ = 0 \quad \forall \delta x$$

795

796 and, setting $\eta = \lambda / (1 + \lambda) \in ]0, 1[$, we obtain the characterization [Ca2]:

797

$$798 \qquad \nabla J(\tilde{e}) = (1 - \eta) \nabla J(e) + \eta \nabla J(c)$$

799

800 In the very special case where $J$ is quadratic, that is $D_j$ is a Mahalanobis distance, we get that the projection $\tilde{e}$ of $e$ lies is the intersection of the hypersphere and the line segment joining $e$ and the center $c$ of the ball. This is because the gradient at $x$ is simply $Ax$ with $A$ a symmetric definite positive matrix. 801 802 But in all other cases, the characterization of $\tilde{e}$ tells us that the alignment only takes place in the gradient 803 space. Because the symmetric Bregman divergences are in fact true (squared) distances, and in view to 804
805 derive efficient approximation procedures, the following definition of a subclass, relating a Bregman divergence to a symmetric one is of interest:

807

808 **Definition: $\mu$-similar Bregman divergences [44]**

809 A Bregman divergence $D_J$ defined on a domain $K$ of a vector space $V$ is $\mu$-similar for some $0 < \mu < 1$ if there exists a symmetric Bregman divergence generated by $Q_\square$ such that:

811 $\mu D_{Q_\square}(e_1, e_2) \le D_J(e_1, e_2) \le D_{Q_\square}(e_1, e_2) \quad \forall (e_1, e_2) \in K^2$

812

813 $Q_\square$ is necessarily a quadratic function, as shown previously, so that the Bregman divergence generated by $Q_\square$ is a Mahalanobis distance. $\mu$-similar Bregman divergences enjoy the following properties: they are approximately symmetric and satisfy a variant of the double triangle inequality [44]:

815

816 $D_J \ \mu\text{-similar} \Rightarrow D_J(e_1, e_2) \le \frac{1}{\mu} D_J(e_2, e_1), D_J(e_1, e_2) \le \frac{2}{\mu} [D_J(e_1, e_3) + D_J(e_2, e_3)] \quad \forall (e_1, e_2, e_3)$

818 This concept can then be used for deriving approximated methods involving the Bregman divergence or to obtain quickly a first result to be further refined, because for example the associated Bregman balls can be bordered by ellipsoids. As quoted by [AB], most Bregman divergences that are used in practice for clustering are $\mu$-similar when restricted to a domain $X$ that avoids the possible singularities of $J$. In the thermomechanics context however, this is not the case. As an example, the Bregman divergence generated by the function positively homogeneous of degree two $J(x) = a(\langle x \rangle^+)^2 + b(\langle x \rangle^-)^2$, with $a < b$, is $\mu$-similar with $Q_\square(x) = bx^2$ and $m=a/b$. But the Bregman divergence generated by the function

825 $J(x) = a(\langle x \rangle^+)^2 + b(\langle x \rangle^-)^2 + \alpha |x|, \quad \alpha > 0$ is not. (just examine the function $f(x) = D_J(x, -x)$).

826

## 5 Conclusion

828

829 In this paper, we try to extend the concept of Bregman divergence in thermomechanics, and we show how the large domain covered by the convex framework allows to systematically build discrepancy measures between fields of thermodynamic variables including couples of dual ones. Most of the already existing “errors” or “gaps” can then be encompass. This gives rise to discrepancy measures between variables with different dimensions, encountered especially in multi-physic applications, allowing to consider truly coupled approaches. We also give some useful properties related to these concepts for geometrical computation that is at the heart of the algorithms used or already imagined for the processing of massive data produced by experiments, computations or in-service measurements.

837

838
839 **Annex A :** Proof of the characterization of symmetric Bregman divergences

840
841 The condition for symmetry of D<sub>J</sub> reads, for the generating convex function J:

842
$$
2(J(e_1) - J(e_2)) = \langle \nabla J(e_1) + \nabla J(e_2), e_1 - e_2 \rangle \quad \forall (e_1, e_2) \in \mathbb{R}^n \times \mathbb{R}^n
$$

844
845 Step 1: Let's suppose first that $e_2=0$, then: $2J(e) = \langle \nabla J(0) + \nabla J(e), e \rangle$ $\forall e \in \mathbb{R}^n$, leading to the necessary condition: $J(e) = 1/2 \langle \nabla J(e), e \rangle$. Introduce now the real function $f_e$ parameterized with $e$:

846
847
$$
f_e(\lambda) = J(\lambda e) , \quad \lambda \in \mathbb{R}^+
$$

849 The derivative of $f_e$ with respect to $\lambda$ being $\frac{d}{d\lambda} f_e(\lambda) = \nabla J(\lambda e) \cdot e$, we have: $f_e(\lambda) = \frac{\lambda}{2} \frac{d}{d\lambda} f_e(\lambda) \cdot f_e$ is a positive function because $J$ is convex, $J(0)$ and $\nabla J(0)$ are null, so:

851
852
853
$$
f_e(\lambda) = J(\lambda e) - J(0) \geq \nabla J(0) \cdot \lambda e = 0 \quad \Rightarrow \quad f_e(\lambda) \geq 0
$$

854 Integrating, we get the result $f_e(\lambda) = K_e \lambda^2 \quad \forall \lambda \in \mathbb{R}^+$, leading to the following expression for $J$:

855
$$
J(e) = \frac{\|e\|^2}{2} k \left( \frac{e}{\|e\|} \right) \quad \forall e \in \mathbb{R}^n
$$

857
858 Note that the condition on $J$ for the symmetry of $D_J$ ensures recursively that $f_e$ is infinitely directionally derivable in each direction $e$. This result also shows that $J$ is necessarily positively homogeneous of degree 2. In addition, returning to the equation for characterizing the symmetry of $D_J$, and using
860
861 $e_1 = e = -e_2$, one establishes that $k$ enjoys the symmetry: $k(e)=k(-e)$ because $J(e)=J(-e)$:

862
863 Then the strictly convex generating functions of symmetric Bregman divergences are parabolic
864 functions along each line crossing the origin.

865
866 Step 2: Going back to the condition for symmetry of J, we have for any (e₁, e₂):

867
$$
\begin{gathered}
2(J(e_1) - J(e_2)) = \langle \nabla J(e_1), e_1 \rangle - \langle \nabla J(e_2), e_2 \rangle = \langle \nabla J(e_1) + J(e_2), e_1 - e_2 \rangle \\
\Downarrow \\
\langle \nabla J(e_2), e_1 \rangle = \langle \nabla J(e_1), e_2 \rangle
\end{gathered}
$$

869
870 Consider an orthonormal basis $(E_j)_{j=1,N}$, then for any $N$-uplets $(\alpha_i)$ in $\mathbb{R}^N$ and any index $1 \le j \le N$:

871
$$
\begin{align*}
\langle \nabla J \left( \sum_{i=1}^{N} \alpha_i E_i \right), E_j \rangle &= \sum_{i=1}^{N} \alpha_i \langle \nabla J(E_j), E_i \rangle \\
&= \sum_{i=1}^{N} \alpha_i \langle \nabla J(E_i), E_j \rangle \\
&= \left\langle \sum_{i=1}^{N} \alpha_i \nabla J(E_i), E_j \right\rangle
\end{align*}
$$

873 showing that is $\nabla J(e)$ linear, $\nabla J(e) = A_{ij} \langle e, E_j \rangle E_i$ with $A$ necessarily a symmetric matrix. The function $J$ is then necessarily a quadratic function: $J(e) = \frac{1}{2} \langle Ae, e \rangle$, $A = A^t$

875
876 **Annex B :** Proof of separate convexity of the Bregman Gap

877
878 The following inequality has to be established.

879
$$880 \quad \forall ([e_1, p_1], [e_2, p_2], [e_0, p_0]) \\ 881 \quad BG_j^s (\lambda[e_1, p_1] + (1-\lambda)[e_2, p_2], [e_0, p_0]) \le \lambda BG_j^s ([e_1, p_1], [e_0, p_0]) + (1-\lambda)BG_j^s ([e_2, p_2], [e_0, p_0])$$

Consider the two functions of $\lambda$:

$$882 \quad BG_j^s (\lambda[e_1, p_1] + (1-\lambda)[e_2, p_2], [e_0, p_0]) \le \lambda BG_j^s ([e_1, p_1], [e_0, p_0]) + (1-\lambda)BG_j^s ([e_2, p_2], [e_0, p_0]) \\ 883 \quad F(\lambda) = \langle \lambda e_1 + (1-\lambda)e_2 - e_0, \lambda p_1 + (1-\lambda)p_2 - p_0 \rangle \\ G(\lambda) = \lambda \langle e_1 - e_0, p_1 - p_0 \rangle + (1-\lambda) \langle e_2 - e_0, p_2 - p_0 \rangle$$

We are now led to show that the function $f(\lambda)=F(\lambda)-G(\lambda)$ is negative along the segment $[0,1]$. But $f(0)=0$, so that we can consider the derivative of $f$ which turns out to be expressed by :

$$884 \quad f'(\lambda) = (2\lambda - 1) \langle e_1 - e_2, p_1 - p_2 \rangle = C(2\lambda - 1) \quad C \ge 0 \\ 885 \quad G(\lambda) = \lambda \langle e_1 - e_0, p_1 - p_0 \rangle + (1-\lambda) \langle e_2 - e_0, p_2 - p_0 \rangle$$

The function $f$ can be calculated as $f(\lambda) = C\lambda(\lambda - 1)$ and is strictly negative for $\lambda$ in $]0, 1[$ as soon as

$$891 \quad C = BG_j^s ([e_1, p_1], [e_2, p_2]) = \langle e_1 - e_2, p_1 - p_2 \rangle > 0, \text{ and null if } C=0.$$
<table>
  <caption>References</caption>
  <tbody>
    <tr>
      <td>894</td>
      <td></td>
    </tr>
    <tr>
      <td>895</td>
      <td></td>
    </tr>
    <tr>
      <td>896</td>
      <td>[1] C. C. Aggarwal, A. Hinneburg, and D. A. Keim, "On the Surprising Behavior of Distance Metrics in High Dimensional Space," Lecture Notes in Computer Science, vol. 1973, (2001), pp. 420-434.</td>
    </tr>
    <tr>
      <td>897</td>
      <td></td>
    </tr>
    <tr>
      <td>898</td>
      <td></td>
    </tr>
    <tr>
      <td>899</td>
      <td>[2] Beyer K., Goldstein J., Ramakrishnan R. Schaft U., When is "nearest neighbor" meaningful? In: Beeri C., Buneman P. (eds) Database Theory — ICDT'99. ICDT 1999. Lecture Notes in Computer Science, vol 1540. Springer, Berlin, Heidelberg, (1999). https://doi.org/10.1007/3-540-49257-7_15</td>
    </tr>
    <tr>
      <td>900</td>
      <td></td>
    </tr>
    <tr>
      <td>901</td>
      <td></td>
    </tr>
    <tr>
      <td>902</td>
      <td></td>
    </tr>
    <tr>
      <td>903</td>
      <td>[3] Kohn R.V., Vogelius M; Relaxation of a variational method for impedance computed tomography. Comm. Pure Appl. Math., 40 (1987) 745-777</td>
    </tr>
    <tr>
      <td>904</td>
      <td></td>
    </tr>
    <tr>
      <td>905</td>
      <td></td>
    </tr>
    <tr>
      <td>906</td>
      <td>[4] Ladeveze P., Leguillon D., Error estimate procedure in the finite element method and applications, SIAM J. numerical Analysis, vol. 20, (1983), pp 485-509.</td>
    </tr>
    <tr>
      <td>907</td>
      <td></td>
    </tr>
    <tr>
      <td>908</td>
      <td></td>
    </tr>
    <tr>
      <td>909</td>
      <td>[5] Moës N., Ladeveze P., Douchin B., Constitutive relation error estimators for (visco)plastic finite element analysis with softening, CMAME, vol, 176, (1999), pp 247-264.</td>
    </tr>
    <tr>
      <td>910</td>
      <td></td>
    </tr>
    <tr>
      <td>911</td>
      <td></td>
    </tr>
    <tr>
      <td>912</td>
      <td>[6] Feissel P., Allix O., Modified constitutive relation error identification strategy for transient dynamics with corrupted data: the elastic case. Appl Mech Eng 196, (2007) 1968-1983.</td>
    </tr>
    <tr>
      <td>913</td>
      <td></td>
    </tr>
    <tr>
      <td>914</td>
      <td>https://doi.org/10.1016/j.cma.2006.10.005</td>
    </tr>
    <tr>
      <td>915</td>
      <td></td>
    </tr>
    <tr>
      <td>916</td>
      <td>[7] Andrieux S., Ben Adba A., Bui H.D., Sur identification de fissures planes via le concept d'écart à la réciprocité, C.R. Acad. Sci Paris, Série II, (1997), vol.324, pp431-438.</td>
    </tr>
    <tr>
      <td>917</td>
      <td></td>
    </tr>
    <tr>
      <td>918</td>
      <td></td>
    </tr>
    <tr>
      <td>919</td>
      <td>[8] Claire D., Hild F., Roux S., Identification of damage fields using kinematics measurements, Comptes Rendus Mécanique, vol. 330, (2002), pp 729-734,</td>
    </tr>
    <tr>
      <td>920</td>
      <td></td>
    </tr>
    <tr>
      <td>921</td>
      <td></td>
    </tr>
    <tr>
      <td>922</td>
      <td>[9] S. Andrieux S., Baranger T.N., Solution of nonlinear Cauchy problem for hyperelastic solids. Inverse Problems, 31(11), (2015), 115003-115022,</td>
    </tr>
    <tr>
      <td>923</td>
      <td></td>
    </tr>
    <tr>
      <td>924</td>
      <td></td>
    </tr>
    <tr>
      <td>925</td>
      <td>[10] Andrieux S., Baranger T.N., On the determination of missing boundary data for solids with nonlinear material behaviors, using displacement fields measured on a part of their boundaries. J. Mech. Phys. Solids (2016) http://dx.doi.org/10.1016/j.jmps.2016.02.008i</td>
    </tr>
    <tr>
      <td>926</td>
      <td></td>
    </tr>
    <tr>
      <td>927</td>
      <td></td>
    </tr>
    <tr>
      <td>928</td>
      <td></td>
    </tr>
    <tr>
      <td>929</td>
      <td>[11] Andrieux S., Baranger T.N, Nonlinear Cauchy problem and identification in contact mechanics : a solving method based on Bregman-gap, Inverse Problems, 36,11, (2020).</td>
    </tr>
    <tr>
      <td>930</td>
      <td></td>
    </tr>
    <tr>
      <td>931</td>
      <td></td>
    </tr>
    <tr>
      <td>932</td>
      <td>[12] Bregman L., The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics 7 (3) (1967) 200-217</td>
    </tr>
    <tr>
      <td>933</td>
      <td></td>
    </tr>
    <tr>
      <td>934</td>
      <td></td>
    </tr>
    <tr>
      <td>935</td>
      <td></td>
    </tr>
    <tr>
      <td>936</td>
      <td>[13] Kiwiel K. C., Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optim. 35 (4), (1997), 1142-1168</td>
    </tr>
    <tr>
      <td>937</td>
      <td></td>
    </tr>
    <tr>
      <td>938</td>
      <td></td>
    </tr>
    <tr>
      <td>939</td>
      <td>[14] Y. Censor, A. Lent, An Iterative Row-Action Method for Interval Convex Programming, J. Optimization theory and applications, Vol. 34 No3 (1981) 321-353</td>
    </tr>
    <tr>
      <td>940</td>
      <td></td>
    </tr>
    <tr>
      <td>941</td>
      <td></td>
    </tr>
    <tr>
      <td>942</td>
      <td>[15] Basserville C., Divergence measures for statistical data processing—An annotated bibliography, Signal Processing 93 (2013) 621-633</td>
    </tr>
    <tr>
      <td>943</td>
      <td></td>
    </tr>
    <tr>
      <td>944</td>
      <td></td>
    </tr>
    <tr>
      <td>945</td>
      <td>[16] A. Banerjee, S. Merugu, I. S. Dhillon, J. Ghosh, Clustering with Bregman Divergences, Journal of Machine Learning Research, 6, (2005) 1705-1749</td>
    </tr>
    <tr>
      <td>946</td>
      <td></td>
    </tr>
    <tr>
      <td>947</td>
      <td></td>
    </tr>
    <tr>
      <td>948</td>
      <td>[17] Dragomir R.-A., Bregman Gradient Methods for Relatively-Smooth Optimization, PhD Thesis, UT1 Capitole, France, 2021.</td>
    </tr>
    <tr>
      <td>949</td>
      <td></td>
    </tr>
    <tr>
      <td>950</td>
      <td></td>
    </tr>
    <tr>
      <td>951</td>
      <td>[18] Frigyik B. A., Srivastava S., Gupta M. R., Functional Bregman Divergence and Bayesian Estimation of Distributions, IEEE Transactions on Information Theory, 54, 11, (2008), 1681-1685</td>
    </tr>
    <tr>
      <td>952</td>
      <td></td>
    </tr>
    <tr>
      <td>953</td>
      <td></td>
    </tr>
  </tbody>
</table>
[19] Zhang Z., Ooi B.C., Parthasarathy S., Tung A. K.H., Similarity Search on Bregman Divergence: Towards Non-Metric Indexing, 35th International Conference on Very Large Data Bases, VLBD09, Lyon, France, 2009
[20] Jeffreys H., An invariant form of the prior probability in estimation problems, Proc. Royal Soc. of London, 186, (1946), 463-461
[21] Nielsen F., Nock R., Skew Jensen-Bregman Voronoi diagrams, Trans. on Comput. Sci. XIV, LCNS 6970, (2011), pp 102-128,
[22] Chen, P., Chen, Y., Rao, M., 2008. Metrics defined by Bregman divergences. Communications in Mathematical Sciences 6, 915-926
[23] Wu L., Choi C.H., Jin R., Zhu J., Yu N., Learning Bregman distance for semi supervised clustering, IEEE transactions on knowledge and data engineering, 24, 3, (2012), pp 478-491.
[24] Rockafellar R. T., Convex Analysis, Princeton University Press, (1970)
[25] Ekeland I., Témam R., Convex analysis and variational problems, Siam, Philadelphia (1999)
[26] Hiriart-Urruty, J. B., Lemarechal, C. Convex Analysis and Minimization Algorithms II, Advanced Theory and Bundle Methods, Grundlehren der mathematischen Wissenschaften 306, Springer, 2013
[27] Brøndsted, A, Rockafellar, R.T. On the subdifferentiability of convex functions. Proc. Amer. Math. Soc. 16 (1965), 605-611.
[28] S. Andrieux S., Baranger T.N., Ben Abda A., Solving Cauchy problem by minimizing an energy-like functional, Inverse Problems 22,(2006), pp 115-133.
[29] Halphen B., Nguyen Q.S., Sur les matériaux standards généralisés. J. Mécanique, 14, (1975), 39-63.
[30] Drucker D.C.; A more fundamental approach to plastic stress-strain relations, In Proc. First US Congress of Applied Mechanics, ed. ASME (New York), (1951), 487-491,
[31] Marigo, J.J., From Clausius-Duhem and Drücker-Ilyushin inequalities to standard materials, In Continuum Thermomechanics: The Art and Science of Modelling Material behaviour, Paul Germain's Anniversary Volume, A. Maugin, R. Douot & F. Sidoroff eds, Kluwer Academic Publishers, Volume 76, Solid Mechanics and Its Applications, (2000)
[32] Germain P. Nguyen Q.S., Suquet P., Continuum Thermodynamics, Transaction of the ASME, Vol 50,december, pp. 1010-1020, (1983)
[33] Ball J.M., Convexity conditions and existence theorems in nonlinear elasticity, Arch. Rat. Mech. Anal., 65, (1977), 193-201,
[34] de Saxcé G., Feng Z.Q., New inequation and functional for contact with friction: the implicit standard material approach, Mech. Struct. and Mach. 19(3), (1991), 301-325.
[35] Andrieux S., Revisiting the POD with Bregman divergence, submitted 2022
[36] Banerjee A., Guo X., Wang H., On the optimality condition expectations as a Bregman predictor, IEEE transactions on Information Theory, 51, (2005), pp 2664-2669.
[37] Nock R., Nielsen F., Fitting the Smallest Enclosing Bregman Ball. In: Gama J., Camacho R., Brazdil P.B., Jorge A.M., Torgo L. (eds) Machine Learning: ECML 2005. Lecture Notes in Computer Science, vol 3720. Springer, 2005
[38] Boissonnat J.D., Nielsen F., Nock R., Bregman Voronoi diagrams, Discrete and computational geometry, 44, (2010), pp 281-307.
1013 [39] Nielsen F. Nock R., On the smallest enclosing information disk, Information Processing Letter, 105, 2005,
1014 pp 93-97,

1015
1016 [40] Fischer A., Quantization and clustering with Bregman divergences, *Journal of Multivariate Analysis*, Volume
1017 101, Issue 9, (2010), pp 2207-2221.

1018
1019 [41] Luci M., Bachem O., Krause A., String coresets for hard and soft Bregman Clustering with application to
1020 exponential family mixture, *Proc. 19th Int. conf. on artificial intelligence and Statistics*, (AISTATS), Cadiz,
1021 Spain, 2016

1022
1023 [42] Cayton L. Efficient Bregman Range Search. *Advances in Neural Information Processing Systems 22* -
1024 Proceedings of the 2009 Conference. 243-251, 2009.

1025
1026 [43] Cayton L. Fast nearest Neighbor Retrieval for Bregman Divergences, *Proceedings of the 25th international
1027 Conference on Machine Learning*, Helsinki, Finland, 2008.

1028
1029 [44] Ackermann, M. R., Blömer J. Coresets and approximate clustering for Bregman divergences. Int Symposium
1030 on Discrete Algorithms (SODA), SIAM, (2009), pp.1088–1097.