<figure><img src="image_1.png" /></figure>

HAL
open science

Logical entropy – special issue

Giovanni Manfredi

► To cite this version:

Giovanni Manfredi. Logical entropy – special issue. 4open, 2022, 5 (15), pp.E1. 10.1051/fopen/2022005. hal-03839217

HAL Id: hal-03839217

https://hal.science/hal-03839217

Submitted on 19 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Logical entropy – special issue

Giovanni Manfredi*

Université de Strasbourg, CNRS, Institut de Physique et Chimie des Matériaux de Strasbourg, UMR 7504, 67000 Strasbourg, France

Received 10 February 2022, Accepted 10 February 2022

Entropy is a fundamental quantity in many areas of knowledge, from physics to information science to biology. Originally put forward in the nineteenth century for very practical purposes (to quantify the reversibility of thermodynamic cycles, hence of thermal engines), entropy was the key concept that allowed Ludwig Boltzmann to bridge the gap between the (time irreversible) macroscopic thermodynamics and the (reversible) microscopic Newtonian physics. As defined by Boltzmann, the entropy $S_B$ represents the number of microscopic states that are compatible with a given macroscopic realization:

$$S_B = k_B \ln \Omega, \qquad (1)$$

where $k_B$ is Boltzmann's constant and $\Omega$ is the relevant phase space volume, which is a measure of the number of microscopic states. Note that the logarithm in the above definition is required so that Boltzmann's statistical entropy possesses the same additive properties as the thermodynamic entropy.

Later, Claude Shannon discovered that a similar formula to Boltzmann's (albeit with the opposite sign) can be used to quantify the information content of a signal. Following Shannon's work, it is customary to identify entropy with the (lack of) information or “disorder” of a system. As information is a concept that permeates many natural sciences, the concept of entropy quickly spread to other fields, such as biology and genetics.

It was John von Neumann who generalized the Boltzmann entropy to quantum physics. This is actually more than a mere generalization. Indeed, equation (1) is somewhat problematic, as $\Omega$ has the dimensions of a phase space volume, while the argument of the logarithm should be nondimensional – not to mention that $S_B$ can become negative. But considering that quantum mechanics introduces a minimal action given by Planck's constant $h$, Boltzmann's formula can be rewritten as: $S_B = k \ln(\Omega/h^d)$ (where $d$ is the number of dimensions of the system), which is always nonnegative as long as $\Omega \ge h^d$ and vanishes only when the equality sign holds. In terms of discrete quantum

states with occupation probabilities $p_i$, the von Neumann entropy reads as

$$S_{VN} = - \sum_{i=1}^{n} p_i \ln p_i, \qquad (2)$$

which is always positive and only vanishes when one state is occupied with probability equal to one.

While it is generally admitted that the Boltzmann/von Neumann/Shannon entropy is a measure of the lack of information of a system, it is also clear that such a measure is not unique. Given a set of events endowed with probabilities $p_i$, one can construct many other formulae that quantify our uncertainty about the actual observed outcome. And indeed, many other definitions of entropy have been proposed in the past [1], so many that it is virtually impossible to do justice to all of them. In physics, Tsallis' entropies are a popular example [2], as are Rényi's entropies [3]. In biology too, several measures of biological diversity have been put forward, see for instance the recent review [4].

How is one to navigate among this plethora of entropy definitions? An answer was provided recently by Ellerman [5], who emphasized the importance of making distinctions between elements of a given set $U$. If such a set is partitioned into a number $n$ of subsets $B_i$ (such that $\cup_{i=1}^n B_i = U$), each endowed with a probability $p_i$ of finding an element of $U$ in that subset, then the probability that in two independent draws one will obtain elements in distinct subsets $B_i$ and $B_{j \neq i}$ is: $p_i(1 - p_i)$. This is precisely the concept of distinction, i.e., the ability to establish that two independent draws are different from one another. Summing over all $n$ subsets, we obtain the total number of distinctions, which is the definition of what Ellerman termed logical entropy.

$$S_L = \sum_{i=1}^{n} p_i(1 - p_i) = 1 - \sum_{i=1}^{n} p_i^2. \qquad (3)$$

The subsets $B_i$ may contain one single element, in which case $S_L$ represents the probability that two consecutive draws yield different elements of $U$. $S_L$ varies in the interval [0, 1], and the lower bound is reached when one element has probability $p_i = 1$, while for all others $p_{j \neq i} = 0$. For equal

<footnote>*Corresponding author: giovanni.manfredi@ipcms.unistra.fr</footnote>
probabilities ($p_i = 1/n, \forall i$), one gets: $S_L = 1 - 1/n \to 1$, when $n \to \infty$.

Although the formula (3) is not itself original (for instance, it is a special case of the Tsallis entropy), its interpretation in terms of partitions of a set is new and illuminating. Indeed, the logical entropy enjoys a number of intriguing features (e.g., it has the properties of a measure, in the precise mathematical sense) which single it out among the many other definitions proposed in the literature.

This Special Issue contains a collection of four papers devoted to the logical entropy and its applications to physics, particularly quantum mechanics.

The paper by David Ellerman [6] contains an exhaustive introduction, both conceptual and historical, to the logical entropy. In particular, it discusses how the latter relates to the more usual Shannon entropy in quantifying the informational content of a set.

The paper by Boaz Tamir et al. [7] discusses the possible extension of the logical entropy to the quantum domain. Indeed, the logical entropy can be expressed in terms of the density matrix $\rho$ as: $S_L = 1 - \text{Tr}\rho^2$, and quantifies the purity of a quantum state. The authors prove several properties of this entropy for generic density matrices that are relevant to various areas of quantum mechanics and quantum information.

The paper by Denis Sunko [8] addresses the logical entropy in the context of many-body quantum mechanics. It shows how the logical entropy can be used to distinguish many-body fermion states by their information content, although they are pure states whose usual quantum entropies are equal to zero.

Finally, the paper by Giovanni Manfredi [9] points out that the definition of logical entropy (3) lends itself quite naturally to a generalization of probabilities to negative values, an idea that goes back to Feynman and Wigner. By combining negative probabilities with the definition of

logical entropy, one can recover many intriguing properties that are typical of quantum systems.

We hope this Special Issue will foster research into a topic that has a long and distinguished past history, but still much potential for future developments.

## References

1. Wehrl A (1978), General properties of entropy. Rev Mod Phys 50, 221. https://doi.org/10.1103/RevModPhys.50.221.

2. Tsallis C (1988), Possible generalization of Boltzmann-Gibbs statistics. J Stat Phys 52, 479.

3. Rényi A (1961), On Measures of Entropy and Information, in: J. Neyman (Ed.), Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, University of California Press, pp. 547–561.

4. Crupi V (2019), Measures of biological diversity: Overview and unified framework. in: E. Casetta, J. Marques da Silva, D. Vecchi (Eds.), From Assessing to Conserving Biodiversity: Conceptual and Practical Challenges, Springer, Cham, pp. 123–136. https://doi.org/10.1007/978-3-030-10991-2_6.

5. Ellerman D (2018), Logical entropy: introduction to classical and quantum logical information theory. Entropy 20, 679. https://www.mdpi.com/1099-4300/20/9/679.

6. Ellerman D (2022), Introduction to logical entropy and its relationship to Shannon entropy. 4open 5, 1. https://doi.org/10.1051/fopen/2021004.

7. Tamir B, De Paiva IL, Schwartzman-Nowik Z, Cohen E (2022), Quantum logical entropy: fundamentals and general properties. 4open 5, 2. https://doi.org/10.1051/fopen/2021005.

8. Sunko DK (2022), Entropy of pure states: not all wave functions are born equal. 4open 5, 3. https://doi.org/10.1051/fopen/2021006.

9. Manfredi G (2022), Logical entropy and negative probabilities in quantum mechanics. 4open 5, 8. https://doi.org/10.1051/fopen/2022004.