# Paper Tissue Softness Rating by Acoustic Emission Analysis
## 3 Experimental Setup
### Performance Metrics  
Since our dataset is imbalanced (26% \"poor\", 17% \"medium\" and 57% \"good\" standard), evaluation with the accuracy rate and _F-score_ only would produce biased results [22]. Therefore we use the posterior balanced accuracy rate [23,24] and, additionally to the _F-score_, the Matthews Correlation Coefficient (_MCC_) [25] as more appropriate performance indicators.  
The accuracy rate for each class \(i\) (the recall) is defined as [26]:  
\[A_{i}=\frac{C_{i}}{N_{i}},\quad i\in\{0,1\}, \tag{1}\]  
where \(N_{i}=|\mathcal{D}_{i}|\) is the total number of samples and \(C_{i}\) is the number of correctly classified samples for class \(c_{i}\). According to [23], the posterior distribution of the class accuracy rate \(A_{i}\) can be expressed as a Beta distribution with parameters \(\alpha_{i}\) and \(\beta_{i}\):  
\[(A_{i}|C_{i})\sim\text{Beta}(\alpha_{i},\beta_{i}),\quad i\in\{0,1\}, \tag{2}\]  
where \(\alpha_{i}=1+C_{i}\) and \(\beta_{i}=1+N_{i}-C_{i}\), and the probability density \(x\) of correctly classifying unseen samples is:  
\[P_{A_{i}}(x;\alpha_{i},\beta_{i})=\frac{\Gamma(\alpha_{i}+\beta_{i})}{\Gamma( \alpha_{i})\Gamma(\beta_{i})}x^{\alpha_{i}-1}(1-x)^{\beta_{i}-1},\;\;i\in\{0,1\}. \tag{3}\]  
The balanced accuracy rate is estimated as the average of individual recalls \(\frac{1}{2}(A_{0}+A_{1})\), whose probability density can be estimated by convolution:  
\[P_{B}(x;\alpha_{0},\beta_{0},\alpha_{1},\beta_{1})=\int_{0}^{1}P_{A_{0}}\big{(} 2(x-y);\alpha_{0},\beta_{0}\big{)}\cdot P_{A_{1}}\big{(}2y;\alpha_{1},\beta_{1 }\big{)}\;dy. \tag{4}\]  
\[BAR=E|X|=\int_{0}^{1}x\cdot P_{B}(x)\;dx \tag{5}\]  
Hence, balanced accuracy is the expected value (5) of the probability density (3). Additionally, we compute the 95% Clopper-Pearson confidence interval (CI) according to [27].  
The macro average (\(\mu\)) and standard deviation (\(\sigma\)) of the _F-scores_ were calculated over the cross-validation folds and the repetitions. The _F-score_ is defined as the harmonic mean of precision and recall:  
\[\text{F-score}=\frac{2 tp}{2 tp+fp+fn} \tag{6}\]  
where \(tp\) are true positives, \(fp\) are false positives and \(fn\) false negatives. An _F-score_ of 1.0 means an ideal classifier with perfect precision and recall. For the multi-class task, the one-vs-rest approach calculates the per-class _F-score_ and its average value.  
The Matthews Correlation Coefficient metric was employed for all classification tasks, and it is calculated as follows:  
\[MCC=\frac{tp\times tn-fp\times fn}{\sqrt{(tp+fp)(tp+fn)(tn+fp)(tn+fn)}} \tag{7}\]  
for the binary case it provides a value between \(-1.0\) (random prediction) and \(+1.0\) (perfect prediction). For the multi-class task, \(MCC\) is generalized by K-class correlation coefficient \(R_{K}\)[28].  
To illustrate the binary classification ability, we also calculated the Receiver Operating Characteristic-Area Under the Curve (ROC-AUC), providing values between \(0\) (inverse) and \(1\) (ideal classification) where the random classifier has a score of \(0.5\).  
Since we use the \(BAR\) as a metric, we calculated the baseline for the random rate classifier as the inverse of the number of classes, e.g., 50% for binary, 0.33% for three, 0.25% for four classes, and so on.