# Nonlinear Random Forest Classification, a Copula-Based Approach
## 2 Preliminaries and Related Works
For more details concerning copulas, their families and association measures, we recommend [PERSON] (2006) ([PERSON], 2006) and [PERSON] and [PERSON] (2016) ([PERSON] and [PERSON], 2016). Merits of copulas and dependence measures in dimension reduction have been discussed in the literature. See, for instance, [PERSON] et al. (2020) ([PERSON] et al., 2020) and [PERSON] et al. (2016) ([PERSON] et al., 2016) for copula-based feature selection; [PERSON] et al. (2017) ([PERSON] et al., 2020) and [PERSON] et al. (2010) ([PERSON] et al., 2020) for classification algorithms using copulas; [PERSON] et al. (2017) ([PERSON] et al., 2020) and [PERSON] et al. (2018) ([PERSON] et al., 2018) for copula-based clustering approaches; [PERSON] et al. (2016) ([PERSON] et al., 2016) and [PERSON] and [PERSON] (2009) ([PERSON] and [PERSON], 2010) for copula functions used in dimension reduction. A well-known measure of uncertainty in a probability distribution is its average Hartley information measure called (Shannon) entropy. For a discrete random variable \(X\) with values \(x_{1},x_{2},\ldots,x_{n}\) and mass density function \(p(. )\), its entropy is defined as:  
\[H(X)=-\sum_{i=1}^{n}p(x_{i})\log p(x_{i}), \tag{2}\]  
and for a continuous random variable \(X\), its (differential) entropy is given by:  
\[H(X)=-\int_{\mathcal{X}}\ p(x)\log p(x)dx, \tag{3}\]  
where \(\mathcal{X}\) is the support of \(X\). Similarly, for a (continuous) multivariate random vector \(\mathbf{X}\) of dimension \(k\) with the multivariate density \(p(\mathbf{X})\), the entropy is defined as:  
\[H(\mathbf{X})=-\oint_{\mathcal{X}}\ p(\mathbf{X})\log p(\mathbf{X})d\mathbf{X}, \tag{4}\]  
where \(\oint\) is an \(k\)-integral on \(\mathbf{\mathcal{X}}\). For two random variables \(X\) and \(Y\) with joint distribution \(p(x,y)\), the conventional information gain (IG) or mutual information (MI) is defined as:  
\[I(X,Y)=\int_{\mathcal{X}}\ \int_{\mathcal{Y}}\ p(x,y)\log\frac{p(x,y)}{p(x)p(y )}dxdy, \tag{5}\]  
which is used to measure the amount of information shared by \(X\) and \(Y\) together, with convention \(\frac{0}{0}=1\). Moreover, one may generalize this concept to a continuous random vector \(\mathbf{X}=(X_{1},X_{1},\ldots,X_{k})\) as:  
\[I(\mathbf{X})=\oint_{\mathcal{X}}\ p(\mathbf{X})\log\frac{p(\mathbf{X})}{\prod_{i=1}^{k}p( x_{i})}d\mathbf{X}, \tag{6}\]  
[PERSON] and [PERSON] (2011) [42] defined the concept of \"copula entropy\". Based on their definition, for a multivariate random vector \(\mathbf{X}\), which is associated with copula density \(c(\mathbf{u})\), its copula entropy is:  
\[h_{c}(\mathbf{X})=-\oint_{u}\ c(\mathbf{u})\log c(\mathbf{X})du.\]  
Additionally, they have pointed out that the mutual information is a copula entropy. Indeed we have the following lemmas  
**Lemma 1**. : _Ref. [32] For a multivariate random vector \(\mathbf{X}\) with the multivariate density \(p(\mathbf{X})\) and copula density \(c_{X}(\mathbf{u})\),_  
\[I(\mathbf{X})=-h_{c}(\mathbf{X}).\]  
Finally, the conditional mutual information is useful to express the mutual information of two random vectors conditioned by a third random vector.