# SCMs: Systematic Conglomerated Models for Audio Cough Signal Classification
## 3 Proposed Technique 2: Stacked CAE Network for Audio Cough Classification
### Stacked CAE Network  
To form a stacked CAE network with hidden layers, \(L\) CAEs and one softmax classifier are stacked together [40]. The encoder network of CAE1 comprises the input and the firsthidden layer. The encoder network of CAE2 comprises the first and second hidden layers. To the output layers of the stacked CAE network, the addition of a softmax classifier is performed so that the classification can be conducted efficiently. Pre-training and fine-tuning are the two important training processes of the stacked CAE network. Each CAE is pre-trained individually by the pre-training process, and to learn the intra-non-linear transformation relationship of every single CAE, a softmax classifier is used. The relationship between the target labels and the learned high-level features can be easily explored by this process. A better set of weight initialization is obtained by this process rather than utilizing random initialization. In between the layers, the inter-relationships are learned by the fine-tuning process, and the best suitable transformation relationships between the labels and the learned high-level intrinsic features are established well, and so, classification is enabled. In between the target and the predicted class labels, the error must be described, and the cost function of the softmax classifier \(J(\theta)\) is defined as follows:  
\[J(\theta)=\frac{1}{N}\left[\sum_{S=1}^{N}\sum_{q=1}^{k}1\{b^{s}=q\}\log\frac{ e^{\theta_{d}^{T}a^{(s)}}}{\sum_{s=1}^{k}e^{\theta_{s}^{T}a^{(s)}}}\right]+\frac{ \gamma}{2}\|\theta\|_{2}^{2} \tag{38}\]  
To prevent overfitting, the second term (weight penalty term) is used. To control the significance of the penalty term, the penalty term parameter used is \(\gamma\). To procure the optimal parameter matrix \(\theta\), the training of the softmax classifier is also conducted like the training of CAE so that by means of minimizing the cost function with the BP algorithm a suitable transfer function is established between the input and the target labels. Every element of \(\theta\) is updated after each iteration as follows:  
\[\theta_{ji}=\theta_{ji}-\epsilon_{2}\frac{\partial J(\theta)}{\partial\theta_ {ji}} \tag{39}\]  
where \(\epsilon_{2}\) expresses the learning rate and \(\theta_{ji}\) indicates the elements of \(\theta\) in the \(j^{th}\) row and \(i^{th}\) column. To train CAE1, the BP optimization algorithm is used so that the optimal parameters are obtained by means of minimizing the cost function [37]. Once the training of CAE1 is performed effectively, the computation of the first hidden representations \(h_{1}^{d}\) is conducted as the input vector of CAE2. The definition of \(h_{1}^{d}\) is expressed as follows:  
\[h_{1}^{d}=f\Big{(}W_{1}^{(1)}p^{d}+b_{1}^{(1)}\Big{)} \tag{40}\]  
Using the BP algorithm, CAE2 is also effectively trained, and the computation of the second hidden representation \(h_{2}^{d}\) is conducted as the input vector of CAE3. Until the effective training of the \(L^{th}\) CAE is carried out, this process is repeated, and the \(L^{th}\) hidden representation \(h_{L}^{d}\) is computed as the input vector of the softmax classifier and is expressed as follows:  
\[h_{L}^{d}=f\Big{(}W_{1}^{(1)}h_{L-1}^{d}+b_{1}^{(1)}\Big{)} \tag{41}\]  
The calculation of both the hypothesis \(Hyp_{\theta}\Big{(}h_{L}^{(d)}\Big{)}\) and the cost function \(J(\theta)\) stacked is conducted as follows:  
\[Hyp_{\theta}\Big{(}h_{L}^{(d)}\Big{)}= \begin{bmatrix}p|q^{(d)}=&1|h_{L}^{(d)} ;\theta\\ p|q^{(d)}=&2|h_{L}^{(d)};\theta\\ \vdots\\ p|q^{(d)}=&k|h_{L}^{(d)};\theta\end{bmatrix} =\frac{1}{\sum_{v=1}^{k}e^{\theta_ {v}^{T}h_{L}^{(d)}}} \begin{bmatrix}e^{\theta_{v}^{T}h_{L}^{(d)}}\\ e^{\theta_{v}^{T}h_{L}^{(d)}}\\ \vdots\\ e^{\theta_{v}^{T}h_{L}^{(d)}}\end{bmatrix}