## 3 Mixed-Attribute Fusion-Based Naive Bayesian Classifier
The gradient descent method is used to determine the updating rule of \(\beta_{lr}\). The partial derivative of \(\text{L}(\overline{\mathcal{X}},\alpha,\beta)\) with respect to \(\beta_{lr}\) is calculated as follows:  
\[\Delta\beta_{lr}=\frac{\partial\text{L}(\overline{\mathcal{X}},\alpha,\beta) }{\partial\beta_{lr}}=\lambda\frac{\partial\text{L}_{1}(\overline{\mathcal{X }},\alpha,\beta)}{\partial\beta_{lr}} \tag{20}\]  
and then the updating rule of \(\beta_{lr}\) is given as follows  
\[\beta_{lr}\leftarrow\beta_{lr}-\xi\times\Delta\beta_{lr}, \tag{21}\]  
where \(\xi>0\) is the learning rate. For the input-layer weight \(\alpha_{rl}\), the updating rule cannot be derived by using the gradient descent method because of the existence of mutual information terms. Here, a new updating strategy based on the Monte Carlo method [25] is designed as follows:  
\[\alpha_{rl}\leftarrow\alpha_{rl}-\zeta\times\Delta\alpha_{rl}, \tag{22}\]  
where \(\zeta>0\) is the learning rate and  
\[\Delta\alpha_{rl}=\frac{1}{\mathcal{N}}\sum_{m=1}^{\mathcal{M}}\sum_{n=1}^{ \mathcal{N}_{m}}\text{L}\left(\overline{\Xi}_{n}^{(m)},\alpha,\beta\right) \tag{23}\]  
is the approximation of the gradient \(\frac{\partial\text{L}(\overline{\mathcal{X}},\alpha,\beta)}{\partial\alpha_{rl}}\). Based on the IEA data set, mixed-attribute fusion-based NBC (MAF-NBC) determines the class label for a given new sample \(\text{x}=\left(a_{1},\cdots,a_{\mathcal{D}_{1}},b_{1},\cdots,b_{\mathcal{D}_ {2}}\right)\) as follows. First, the one-hot encoded form of x is expressed as follows:  
\[\overline{\text{x}}=\left(e_{1}^{(1)},\cdots,e_{\mathcal{K}_{1}}^{(1)},\cdots,e_{1}^{(\mathcal{D}_{1})},\cdots,e_{\mathcal{K}_{\mathcal{D}_{1}}}^{(\mathcal{ D}_{1})},b_{1},\cdots,b_{\mathcal{D}_{2}}\right), \tag{24}\]  
where the following. \[e_{k}^{(i)}=\left\{ \begin{array}{ll}1,&\text{if }a_{i}=A_{k}^{(i)}\\ 0,&\text{if }a_{i}\
eq A_{k}^{(i)}\end{array} \right.,k=1,2,\cdots,\mathcal{K}_{i},i =1,2,\cdots,\mathcal{D}_{1}. \tag{25}\]Second, \(\overline{x}\) is transformed into the following independent encoded expression:  
\[\overline{\overline{x}}=\left(\overline{\overline{x}}_{1},\overline{\overline{x}} _{2},\cdots,\overline{\overline{x}}_{\mathcal{L}}\right)=\left(\text{sigmoid}( \overline{x}\alpha_{1}),\text{sigmoid}(\overline{x}\alpha_{2}),\cdots,\text{ sigmoid}(\overline{x}\alpha_{\mathcal{L}})\right) \tag{26}\]  
based on the trained ANN with input weight matrix \(\alpha=\left(\alpha_{rl}\right)_{\mathcal{R}\times\mathcal{L}}\). Third, the class label of \(x\) is determined according to Equation (1), where the conditional probability is calculated as follows:  
\[ \begin{split}\text{P}(x|w_{m})&=\text{P}(\overline {x}|w_{m})=\text{P}(\overline{\overline{x}}|w_{m})\\ &=\prod_{l=1}^{\mathcal{L}}\text{P}(\overline{x}_{l}|w_{m})=\prod _{l=1}^{\mathcal{L}}\int_{-\infty}^{\overline{\pi}_{l}}\text{P}(s|w_{m})^{ \prime}\end{split}  \tag{27}\]  
where \(\text{P}(s|w_{m})\) is approximated with a normal _p.d.f._  
\[ \begin{split}& f_{l}^{(m)}(s)=\frac{1}{\sqrt{2\pi}\sigma_{l}^{(m)}} \exp\left[-\frac{1}{2}\left(\frac{s-\mu_{l}^{(m)}}{\sigma_{l}^{(m)}}\right)^{ 2}\right],\\ & s\in(-\infty,+\infty),m=1,2,\cdots,\mathcal{M},l=1,2,\cdots, \mathcal{L}\end{split}  \tag{28}\]  
with the mean value  
\[\mu_{l}^{(m)}=\frac{1}{\mathcal{N}_{m}}\sum_{n=1}^{\mathcal{N}_{m}}\overline {x}_{nl}^{(m)} \tag{29}\]  
and standard deviation (std). \[\sigma_{l}^{(m)}=\sqrt{\frac{1}{\mathcal{N}_{m}-1}\sum_{n=1}^{\mathcal{N}_{m} }\left[\overline{x}_{nl}^{(m)}-\mu_{l}^{(m)}\right]^{2}}. \tag{30}\]  
Here, an in-depth discussion regarding the normal _p.d.f._\(f_{l}^{(m)}(\bullet)\) is given. The sigmoid activation function is used in the designed ANN; thus, the outputs corresponding to each hidden layer's nodes obey a quasi-normal probability distribution. A visual comparison of the sigmoid activation function and normal probability distribution functions with standard deviations 0.1, 0.5, 1.0, and 2.0, as shown in Figure 1, clearly demonstrates this empirical conclusion.