# The Odyssey of Entropy: Cryptography
## 2 Entropy Measures and Related Concepts
### Related Concepts  
In addition to entropy measures, there are some related concepts that have played significant roles in research on cryptography. As an example, we can mention relative entropy, conditional entropy and mutual information; the latter is obtained using conditional entropy. For discrete probability distributions \(P\) and \(Q\) defined on the probability space \(\mathcal{S}\), the _relative entropy_ (i.e., the _Kullback-Leibler (KL) divergence_) from \(Q\) to \(P\) is defined as [29]  
\[D_{\text{KL}}(P\parallel Q)=\sum_{x\in\mathcal{S}}P(x)\log\biggl{(}\frac{P(x) }{Q(x)}\biggr{)}. \tag{10}\]  
Consider two random variables \(X\) and \(Y\) with outcomes \(x_{i}\) and \(y_{i}\). The _conditional entropy_\(H(Y|X)\) is defined as  
\[H(Y|X)\] \[= -\sum_{x_{i}}\sum_{y_{i}}(P(x_{i},y_{i})\log_{2}P(y_{i}|x_{i})) \tag{11}\] \[= -\sum_{x_{i}}P_{X}(x_{i})\sum_{y_{i}}(P(y_{i}|x_{i})\log_{2}P(y_{ i}|x_{i})). \tag{12}\]  
\(H(Y|X)\) represents the amount of uncertainty in \(Y\) given the value of \(X\). _Mutual information_ between \(X\) and \(Y\) is calculated as  
\[I(X;Y)=H(X)-H(X|Y)=H(Y)-H(Y|X). \tag{13}\]  
\(I(X;Y)\) quantifies the amount of information (in bits) obtained about \(X\) via observing \(Y\) or vice versa. Moreover, the _joint entropy_\(H(X,Y)\) is calculated as  
\[H(X,Y)=I(X;Y)+H(X|Y)+H(Y|X). \tag{14}\]  
Moreover, for a distribution \(P_{XY}\) and \(0<\varepsilon<1\), the _inf-spectral entropy_\(H_{\text{s}}^{\varepsilon}(P)\) is defined as  
\[H_{\text{s}}^{\varepsilon}(P)=\sup\{r\mid P_{XY}\{-\log P_{XY}(x,y)\leq r\} \leq\varepsilon\}. \tag{15}\]A sequence of jointly distributed random variables \((X_{1},X_{2},\ldots,X_{m})\) has _next-block pseudoentropy_ of at least \(e\) if and only if there exist random variables \((Y_{1},Y_{2},\ldots,Y_{m})\) jointly distributed with \((X_{1},X_{2},\ldots,X_{m})\) such that, for every \(i\in\{1,2,\ldots,m\}\), \((X_{1},\ldots,X_{i-1},X_{i})\) is computationally indistinguishable from \((X_{1},\ldots,X_{i-1},Y_{i})\) and  
\[\sum_{i=1}^{m}H(Y_{i}\mid X_{1},X_{2},\ldots,X_{i-1})\geq e.\]  
Further, the _conditional Renyi entropy_ of order \(\alpha\) is calculated as  
\[H_{\alpha}^{(R)}(X|Y)=\frac{1}{1-\alpha}\max_{y_{i}}\left(\log_{2}\sum_{x_{i}} (P(x_{i}|y_{i}))^{\alpha}\right). \tag{16}\]  
Mutual information has played significant roles in several research works in the area of cryptology. For example, [PERSON] [30] used it in quantum cryptography in order to measure the amount of information gained by an eavesdropper in each individual attack session. As another example, one can mention the research reported by [PERSON] et al. [31], in which mutual information was used for modeling the information leaked from an embedded device containing a secret key during a side channel attack. Conditional Renyi entropy was used by [PERSON] and [PERSON] [32] in developing generalizations for [PERSON]'s secure encryption theorem.  
In addition to statistical distributions, entropy can be calculated over different structures, such as graphs. For example, for an undirected graph \(G(V,E)\), the _graph entropy_ is defined as [33]  
\[H(G)=\min_{X,Y}I(X;Y), \tag{17}\]  
where \(X\) is uniformly distributed over \(V\), and \(Y\) ranges over \(I_{S}(G)\) defined as  
\[I_{S}(G)=\{S|S\subset G,X\in S,\forall s_{1},s_{2}\in S:(s_{1},s_{1})\
otin E\}. \tag{18}\]  
Figure 2 shows the entropy measures and related concepts that have been used by researchers in the area of cryptography.