<figure><img src="image_1.png" /></figure>

HAL
open science

# Derivation of the maximum entropy principle from the virtual work principle

Qiuping Alexandre Wang, Qiong Ye

► To cite this version:

Qiuping Alexandre Wang, Qiong Ye. Derivation of the maximum entropy principle from the virtual work principle. The European Physical Journal Plus, 2025, 140, pp.1234. 10.1140/epjp/s13360-025-07174-3. hal-00140657v6

HAL Id: hal-00140657

https://hal.science/hal-00140657v6

Submitted on 20 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Qiuping A. Wang<fnref n="1" />*, Qiong Ye<fnref n="2" />

**Abstract**

We know the principle of maximum entropy (maxent) as a mathematical inferential method based on the subjectivity of probability and of the information which reflect our ignorance about the probabilistic uncertainty. However, questions may arise about its applicability to thermodynamic systems where the second law entropy and the probability distributions are experimentally observable objective quantities which are independent of our knowledge and ignorance. In this work, we show that the fundamental principle of virtual work of mechanics, when applied to the random dynamics of thermodynamic systems, naturally yields equilibrium state maximizing thermodynamic entropy in whatsoever form of function of probability distribution. This result advocates the consideration of maxent as a law of physics governing the behavior of entropy beyond the second law, meaning that, at equilibrium, a system must arrange itself to reach the microstate having maximum entropy among all the possible equilibrium microstates the system may reach after evolution governed by the second law.

PACS numbers: 05.20.-y, 05.70.-a, 02.30.Xx,

<footnote n="1">¹Laboratoire SCIQ, ESIEA, 9 Rue Vésale, 75005 Paris, France</footnote>

<footnote n="2">²ISTeP, CY Cergy Paris Université, 95000 Neuville-sur-Oise, France</footnote>

<footnote>*Corresponding author, alexandre.wang@esiea.fr;</footnote>
# 1) Introduction

The principle of maximum entropy<fnref n="1" /> (maxent) is widely used in statistical sciences and engineering as a powerful tool for deriving probability distributions. Maxent was proposed by Jaynes [1][2][3] as an inferential method which, inspired by the works of Boltzmann, Gibbs and Shannon using the Shannon type formula (BS entropy, see below) [4][5][6], has been successfully applied to the derivation of probability distributions in a simple and straightforward manner. We have seen numerous applications of maxent in statistical mechanics, see for example the recent works using maxent to justify the molecular chaos or Stosszahlansatz, a crucial element for the proof of the *H*-theorem by Boltzmann [7][8], and the study of the probability distributions of non-equilibrium thermodynamics [9]. Nevertheless, in spite of its popularity and success in physics, maxent has always been at the center of scientific and philosophical debates due to its subjective nature based on the subjective view of the probability and associated information and entropy [3][10][11]. One of the central questions about maxent is why and how an evolving thermodynamic system arranges its microstates such that the thermodynamic entropy, obviously an objective physical quantity, must reach its maximum value at equilibrium.

As a basic assumption, maxent is not directly or indirectly justified by observation and undoubted facts. In the literature, maxent is postulated as a first principle for inference, and justified either a priori by the second laws with additional hypothesis that the entropy is given by BS formula [11], or a posteriori by the correctness of the probability distributions derived from it [3]. In statistical inference theory, it was often justified by intuitive arguments based on the subjectivity of probability [3] or by relating it to other principles such as the consistency requirement or the principle of insufficient reason of Laplace, which is itself the object of considerable criticisms [10]. Anyway, these kinds of plausible arguments are hardly sufficient for maxent to be applied to thermodynamic entropy, which is an objective physical quantity, and to the derivation of the probability distributions of molecules which are experimentally observable quantities and independent of our knowledge or ignorance.

In the present work, we advocate that maxent is a physical law on its own, stemming directly from a most fundamental principle of physics, the virtual work principle (VWP) [12][13]. As is well-known, VWP is one of the most basic and straightforward principles of classical mechanics. All the basic laws of statics and of dynamics of mechanical systems, including the

<footnote n="1">¹ https://en.wikipedia.org/wiki/Principle_of_maximun_entropy</footnote>
fundamental principle of least action, can be derived from VWP. We hope that this work illustrates an important link between two physical concepts (virtual work and entropy) and provides a solid socle for maxent as a law of physics (instead of merely a subjective inferential method), a law we can apply to the study of physical systems and their properties with confidence. In this regard, it is worth mentioning an interesting effort [16] relating maxent to the first law of thermodynamics in the special case of reversible constant-volume process. The present work has the same objective as [16] to give a physical background to maxent from a more fundamental principle of physics and in a more general way. We will come back to this point below.

## 2) Principle of virtual work

The principle of virtual work states that the total work done by all the forces acting on a system in static equilibrium is zero on all possible virtual displacements<fnref n="2" /> which are consistent with the constraints of the system. Let us suppose a simple case of a system of N points of mass in equilibrium under the action of the force $\vec{F}_i$ ($i=1,2,...,N$) on the point i at the position $\vec{r}_i$, and imagine virtual displacement $\delta\vec{r}_i$ of the point i, the virtual work on the i-th point reads $\delta W_i = \vec{F}_i \cdot \delta\vec{r}_i$. VWP states that the total virtual work $\delta W$ of all the forces $\vec{F}_i$ on all $\delta\vec{r}_i$ vanishes for the static equilibrium of all the points, i.e.

$$ \delta W = \sum_{i=1}^{N} \vec{F}_i \cdot \delta\vec{r}_i = 0 \qquad (1) $$

This principle for statics was extended to dynamical equilibrium of moving points by d'Alembert [13] who added the inertial force $-m_i \vec{a}_i$ on each point i:

$$ \delta W = \sum_{i=1}^{N} (\vec{F}_i - m_i \vec{a}_i) \cdot \delta\vec{r}_i = 0 \qquad (2) $$

where $m_i$ is the mass of the point i and $\vec{a}_i = \ddot{\vec{r}}_i$ its acceleration. From this principle, we can derive the Newtonian equation of motion and other fundamental principles such as least action principle.

<footnote n="2">² In mechanics, the virtual displacement of a system is a kind of imaginary infinitesimal displacement with no time passage and no influence on the forces. It should be perpendicular to the constraint forces.</footnote>
On the other hand, we have $m\ddot{\vec{r}}_i \cdot \delta\vec{r}_i = m\delta\dot{\vec{r}}_i \cdot \dot{\vec{r}}_i = \delta(\frac{1}{2}m\dot{\vec{r}}_i^2) = \delta e_{ki}$ which is the virtual variation of the kinetic energy $e_{ki}$ of the point $i$. Now we suppose that there is no dissipative force in the system, meaning the energy of the system does not change if the system is closed and isolated. In this case, the interaction between a pair of particles (or between a particle and its environment) should be conservative. This supposes the existence of a potential energy $e_{pi}$ for each point $i$ subject to the force $\vec{F}_i$ at the time of the virtual displacement, with $\vec{F}_i = -\nabla e_{pi}$ and

$$ \sum_{i=1}^{N} \vec{F}_i \cdot \delta \vec{r}_i = \sum_{i=1}^{N} -\nabla_i e_i \cdot \delta \vec{r}_i = -\sum_{i=1}^{N} \delta e_{pi} \quad (3) $$

where $\delta e_{pi}$ is the virtual variation of the potential energy of the point $i$ due to its virtual displacement $\delta \vec{r}_i$. It follows that

$$ \delta W = -\sum_{i=1}^{N} (\delta e_{pi} + \delta e_{ki}) = -\sum_{i=1}^{N} \delta e_i = -\delta E \quad (4) $$

where $\delta e_i$ is the virtual variation of the total energy $e_i = e_{pi} + e_{ki}$ of point $i$ and $E = \sum_{i=1}^{N} e_i$ is the total energy of the whole system.

### 3) Virtual work principle for random dynamics

At this stage, no statistics have been done, each point in the system is supposed to be undergoing regular motion without randomness. Now considering a thermodynamic system in which all points (or particles) are undergoing random motion we must describe using probability. Without mentioning the centuries-old debate about the objectivity/subjectivity of the randomness and of the probabilistic uncertainty, we just admit the fact that a particle in random motion can evolve from a given state to different likely states, such that at any moment a probability distribution of the particle over the different states must be used for the description of the motion, as has been done so far in statistical physics. In what follows, we introduce such a probabilistic formalism of VWP.

We look at an ensemble of $N$ particles of a thermodynamic system with different microstates $j$, each having the probability $p_j$ ($j=1,2, \dots, w$). The virtual displacements considered above now take place at different microstates. In this case, the virtual work given by Eq.(4) must be written for each microstate $j$, i.e., $\delta W_j = \sum_{i=1}^{N} (\delta e_i)_j = \delta E_j$ where $(\delta e_i)_j$ is the energy of the particle $i$ when the system is at the microstate $j$ with the total energy $\delta E_j$. Obviously, $\delta W_j$ is not to be
used for VWP in writing $\delta W_j = 0$ because it is just one of numerous likely virtual works each occurring with a probability $p_j$, meaning it may or may not happen at any given moment. In such a situation, VWP must be applied to the whole thermodynamic system for any moment of time, so the only useful virtual work is the ensemble average of $\delta W_i$ over all the possible microstates, i.e.,

$$ \delta W = \sum_j P_j \delta W_j = \sum_j P_j \delta E_j = \overline{\delta E} \qquad (5) $$

where $\overline{\delta E} = \sum_j P_j \delta E_j$ is the average of the virtual variation $\delta E_j$ of the energy $E_j$ when the system is at the microstate $j$.

Now let us consider the internal energy of the whole system given by the ensemble average $\overline{E} = \sum_j P_j E_j$. Its virtual variation due to the virtual displacements on the particles is given by $\delta \overline{E} = \delta \sum_j P_j E_j = \sum_j P_j \delta E_j + \sum_j E_j \delta P_j = \overline{\delta E} + \delta Q = \delta W + \delta Q$ where $\delta Q = \sum_j E_j \delta P_j$ is the virtual transfer of heat to the system. When the system is in thermodynamic equilibrium at absolute temperature $T$, $\delta Q = T \delta S$ where $\delta S$ is the virtual variation of thermodynamic entropy $S$ due to the heat transfer $\delta Q$ [14]. Finally, we get $\delta \overline{E} = T \delta S + \delta W$. VWP now is applied to this virtual work with $\delta W = 0$, yielding the expression $\delta \overline{E} - \beta \delta S = 0$ or

$$ \delta(S - \beta \overline{E}) = 0 \qquad (6) $$

which is the usual variational form of maxent using the Lagrange multiplier $\beta = \frac{1}{T}$ associated with the constraint of internal energy $\overline{E}$. We would like to stress here that the relation $\delta Q = T \delta S$ is borrowed from the reversible thermodynamic process but $\delta$ is just a virtual variation instead of a real reversible physical process.

The maximality of this calculus is not self-evident because $\delta W = 0$ is in general an extremum (maximum, minimum or stationary). The maximum nature of Eq.(6) can be justified thanks to the concave nature of entropy $S$ as a function of probability. As entropy is a measure of the randomness of the dynamics, it should depend on the probability distribution $P_j$ of the system, and the part of entropy depending on $P_j$ is actually positive for any value of $P_j$ between 0 and 1 and must be zero for $P_j = 0$ or $P_j = 1$. Hence in the interval $0 \le P_j \le 1$, $S$ is concave. This property was actually used as an axiom in the derivation of BS entropy formula for discrete probability distributions by Shannon [5] and Khinchin [15].
A typical example of application of Eq.(6) is the variational calculus for deriving the discrete probability distribution $P_j$ of canonical ensemble whose internal energy is given by $\bar{E} = \sum_j P_j E_j$. Suppose that the functional form of entropy can be expressed by $S = \sum_j P_j S(P_j)$. Then the calculus of Eq.(6), with a virtual variation of the probability distribution $\delta P_j$, implies $\sum_j [s(P_j) + P_j s'(P_j) + \beta E_j] \delta P_j = 0$. As all $\delta P_j$ are arbitrary with the restriction $\sum_j \delta P_j = 0$ due to normalization, we obtain $s(P_j) + P_j s'(P_j) + \beta E_j = K$ (see for example [16]). If now $s(P_j) = \ln P_j$ as in the case of BS entropy, we obtain the Boltzmann distribution $P_j \propto \exp(-\beta E_j)$ as expected, where the constant of proportion can be calculated by using the normalization condition.

Eq.(6) can be straightforwardly extended to grand-canonical ensemble when the number of particles $N$ becomes a variable for open systems. Consider a microstate $j$ of a system composed of $N$ particles which is a certain distribution of the $N$ particles over the one particle states $k$ of energy $\epsilon_k$ with $k=1, 2, 3 ... g$ (the number of one particle states $g$ can be very large). We imagine $N_j$ identical particles distributed over the $g$ states at a microstate $j$ which is a combination of $g$ numbers $n_k$ of particles over the $g$ states, i.e., $j=\{n_1, n_2, ... n_g\}$. We naturally have $N_j = \sum_{k=1}^g (n_k)_j$ and $E_j = \sum_{k=1}^g (n_k)_j \epsilon_k$. During the virtual work, only the energy of the one particle states changes, let $\delta \epsilon_k$ be the virtual change of the energy of the one particle state $k$. For a given microstate $j$, the virtual work given in Eq.(4) now reads

$$ \delta W_j = - \sum_k (n_k)_j \delta \epsilon_k = - \delta \sum_k (n_k)_j \epsilon_k + \sum_k (\delta n_k)_j \epsilon_k = - \delta E_j + \sum_k (\delta n_k)_j \epsilon_k. \quad (7) $$

The first term of the right-hand side is the total energy variation due to the one particle energy variation $\delta \epsilon_k$ caused by the virtual work as well as to the variation in particle number $\delta N_j$ of the system. The second term is just the energy variation caused by the particle number variation $\delta N_j = \sum_k (\delta n_k)_j$. Hence Eq.(5) becomes

$$ \delta W = - \sum_j p_j \delta E_j + \sum_j p_j \sum_k \epsilon_k (\delta n_k)_j = - \overline{\delta E} + \sum_k \epsilon_k \overline{\delta n_k} = - \overline{\delta E} + \overline{\mu \delta N}. \quad (8) $$
where we used an expression for the chemical potential $\mu = \sum_k \epsilon_k \overline{\delta n_k} / \overline{\delta N}$ with $\overline{\delta N} = \sum_j p_j \delta N_j = \sum_j p_j \sum_k (\delta n_k)_j = \sum_k \overline{\delta n_k}$ and $\overline{\delta E} = \sum_j p_j \delta E_j$. Since $\overline{\delta E} = \delta \overline{E} - \sum_j E_j \delta p_j$ and $\overline{\delta N} = \delta \overline{N} - \sum_j N_j \delta p_j$ with $\overline{E} = \sum_j p_j E_j$ and $\overline{N} = \sum_j p_j N_j$, we get

$$ \delta W = -\delta \overline{E} + \sum_j E_j \delta p_j + \mu \delta \overline{N} - \mu \sum_j N_j \delta p_j = -\delta \overline{E} + \mu \delta \overline{N} \qquad (9) \\ + \sum_j (E_j - \mu N_j) \delta p_j $$

Comparing this to the first law $\delta \overline{E} = \delta Q - \delta W + \mu \delta \overline{N}$ for Grand-canonical ensemble, we identify the heat transfer $\delta Q = \sum_j (E_j - \mu N_j) \delta p_j$. For a reversible virtual process, we can write $\delta S = \beta \delta Q = \beta \sum_j (E_j - \mu N_j) \delta p_j$ and get

$$ \delta W = -\delta \overline{E} + \mu \delta \overline{N} + \frac{\delta S}{\beta}. \qquad (10) $$

where S is the thermodynamic entropy of the second law.

According to VWP with $\delta W = 0$, we have

$$ \delta(S - \beta \overline{E} + \beta \mu \overline{N}) = 0 \qquad (11) $$

which is the usual calculus of maxent for grand-canonical ensemble.

Eq.(6) was considered in the work [16] to establish an equivalence between maxent and the first law of thermodynamics. Indeed, Eq.(6) can be written as $\delta S = \beta \delta \overline{E}$ or $\delta \overline{E} = T \delta S$ which is just the first law $d\overline{E} = T dS + \Delta W$ with the work term $\Delta W$ removed, which was interpreted as the first law restricted to isochoric (constant volume) process with only intensive variable change (see the second paper in [16]). The contribution of the present work is to obtain the same maxent from a fundamental principle of physics and in a general way suitable to any thermodynamic process, in accordance with maxent as a universal principle.

## 4) Concluding remarks

To summarize, the variational calculus of maxent has been shown to be a natural consequence of the virtual work principle applied to thermodynamic systems. This approach going from a basic principle of physics to maxent allows us to consider maxent as a law of physics ruling the behavior of entropy, in the same sense as all other physical laws derived from this fundamental
principle. In other words, at equilibrium state where virtual work must vanish, a thermodynamic
system must arrange itself into the microstate having maximum entropy among all the possible
equilibrium microstates. This choice of state by an equilibrium system cannot be seen from the
second law of thermodynamics which states only that entropy increases during the state
evolution of the system and stops changing at equilibrium. The present result tells us in addition
that the system must take the equilibrium state of maximum entropy among all possible states
at which the system may end after an evolution.

It is also worth noticing that this maximum entropy, or maxent calculus Eq.(6) or Eq.(11),
is not subject to any given functional form of entropy and is valid for whatsoever functional
form of entropy. This degree of freedom opens a door to the application of maxent to other
different entropy or information formula to derive corresponding probability distributions, or
to the derivation of entropy formulas from known probability distributions, see for example
[17][18] and references there-in where a general measure of probabilistic uncertainty or
maximizable information measure (referred to as varentropy [18]) has been defined using the
first law of thermodynamics and applied to both discrete [17] and continuous [18] probability to
derive their informational formula. The word “maximizable” means that the maximization of
the obtained informational formula following VWP can reproduce the corresponding
probability distributions.

In our opinion, this ground provides a more comfortable position for physicists to apply
maxent to the study of objective probability distributions using objective physical quantities
such as entropy, energy, number of particles, pressure, temperature, so on and so forth. Indeed,
within this work, the maxent calculus and the associated constraints (average energy for
instance) naturally come up from the vanishing virtual work, which is much more natural than
introducing these constraints with seemingly plausible even ambiguous arguments such as the
averaged value of an observable represents a factual information to be taken into account in the
maximization of information in order to derive the least biased probability distributions [10].
However, considering maxent as a physical law does not prevent us from using it as inferential
method, especially when the used entropy is not the thermodynamic one. On the contrary, going
from physical law to inference is more confident and reliable than the other way around.

Finally, we would like to emphasize a philosophical aspect of this work. As is well-known,
a century-old debate on thermodynamics is about the anthropocentric nature
(subjectivity/objectivity) of entropy and thermodynamics theory [19]. This work resonates with
the objectivity view and provides a solid argument for the objective nature of entropy, because
its behavior is required by a most fundamental principle of physics, not imposed by a least biased inference based on the subjectivity of entropy and probability. It is worth noticing that this conclusion is also questioning the widely accepted equivalence between entropy and information. Information is in general a subjective quantity perceptible and intelligible for human beings. This (speculative) equivalence has been inspired by the founders of information theory [1][2][3][5][15] who used certain properties of thermodynamic entropy as axioms to derive BS formula of information theory. In passing, we would like to mention the possibility to derive BS formula without considering the thermodynamic entropy [20]. It is also possible to extend the approach of the present work to nonequilibrium systems and to go from fundamental principles to the second law of thermodynamics, see for example [21] where a consequence of VWP (not VWP itself), an extension of the principle of least action to thermodynamics, is used to adapt the Liouville's theorem (established for deterministic dynamics) to random dynamics in order to derive the Boltzmann H-theorem as well as the second law of thermodynamics. An essential step is to calculate the virtual work in nonequilibrium process involving more forces than in equilibrium process. This work is in progress.

**Data Availability Statement:** No Data associated in the manuscript

**References**

[1] E.T. Jaynes, Gibbs vs Boltzmann Entropies. American Journal of Physics. 33 (1965) 391

[2] E.T. Jaynes, Information theory and statistical mechanics, Brandeis University Summer Institute Lectures in Theoretical Physics. 3 (1963)181

[3] E.T. Jaynes, The evolution of Carnot's principle, The opening talk at the EMBO Workshop on Maximum Entropy Methods in x-ray crystallographic and biological macromolecule structure determination, Orsay, France, April 24-28, 1984; Where do we go from here? in *Maximum entropy and Bayesian methods in inverse problems*, pp.21-58, edited by C. Ray Smith and W.T. Grandy Jr., D. Reidel, Publishing Company (1985)

[4] L. Boltzmann, Weitere Studien über das Wärmegleichgewicht unter Gasmolekülen. Sitzungsberichte Akademie der Wissenschaften 66 (1872) 275. English translation: L. Boltzmann, Further Studies on the Thermal Equilibrium of Gas Molecules. The Kinetic Theory of Gases. History of Modern Physical Sciences. 1. pp. 262–349.

[5] C.E. Shannon, A Mathematical Theory of Communication. Bell System Technical Journal, 27 (1948) 379.

[6] J.W. Gibbs, *Elementary Principles in Statistical Mechanics*. New York: Charles Scribner's Sons (1902)

[7] Gregor Chliamovitch, Orestis Malaspinas and Bastien Chopard, A Truncation Scheme for the BBGKY2 Equation, Entropy 17 (2015) 7522
[8] Gregor Chliamovitch, Orestis Malaspinas and Bastien Chopard, Kinetic Theory beyond the Stosszahlansatz, Entropy **19** (2017) 381

[9] G.E. Crooks, Beyond Boltzmann-Gibbs statistics: Maximum entropy hyperensembles out of equilibrium, Phys. Rev. E **75** (2007) 041119

[10] Jos Uffink, Can the maximum entropy principle be explained as a consistency requirement, *Studies in History and Philosophy of Modern Physics*, **26B** (1995) 223

[11] L.M. Martyushev and V.D. Seleznev, Maximum entropy production principle in physics, chemistry and biology, *Physics Reports*, **426** (2006) 1

[12] J.L. Lagrange, Mécanique analytique, Blanchard, reprint, Paris (1965) (Also: Oeuvres, Vol. 11.)

[13] J. D'Alembert, Traité de dynamique, Editions Jacques Cabay, Sceaux (1990)

[14] Y.P. Terletskii, Statistical physics, North-Holland Publishing Company, Amsterdam, 1971

[15] A. I. Khinchin, Mathematical Foundations of Information Theory, Dover, New York, 1957

[16] A. Plastino and E.F.M. Curado, *Phys. Rev. E*, **72**(2005)047103; E.F.M. Curado and A. Plastino, Physica A **386** (2007) 155

[17] Q.A. Wang, Probability distributions and entropy as a measure of uncertainty, J. Phys. A: Math. Theor. **41** (2008) 065004

[18] A. El Kaabouchi, F.X. Machu, R. Wang, Y.Y. Zhu, Q.A. Wang, Study of a measure of efficiency as a tool for applying the principle of least effort to the derivation of the Zipf and Pareto laws, Advances in Complex Systems **24** (2021) 2150013; R. Wang, F.X. Machu, J. Cocks, A. El Kaabouchi and Q.A. Wang, A non-negative measure of informational entropy for continuous probability distributions, to be submitted (2025), hal-03841203

[19] Katie Robertson and Carina Prunkl, Is Thermodynamics Subjective? Philosophy of Science, **90** (2023) 1320

[20] A. El Kaabouchi and Q.A. Wang, Axioms and uniqueness theorem of information entropies, Entropy: Thermodynamics – Energy – Environment – Economy, 3-4 (2025) 15

[21] Q.A. Wang, A. El Kaabouchi, From random motion of Hamiltonian systems to Boltzmann *H* theorem and second law of thermodynamics -- a pathway by path probability, Entropy, **16** (2014) 885