## 4 Program Text Classification
### Multi-Class Classification Metrics  
The most commonly used classification metrics are _Accuracy_, _Precision_, _Recall_, and \(F_{1}\)_score_[46]. _Precision_ for binary classification is computed according to:  
\[Precision=\frac{TP}{TP+FP}, \tag{5}\]where \(TP\) denotes true positive elements--such elements that have been labeled as positive by a classification model and they are actually positive; \(FP\) denotes false positive elements--such elements that have been labeled as positive, but they are negative.  
_Recall_ for binary classification is computed according to:  
\[Recall=\frac{TP}{TP+FN}, \tag{6}\]  
where \(TP\) denotes true positive elements; \(FN\) denotes false negative elements--such elements that have been labeled as negative by a classifier, but they are actually positive.  
\(F_{1}\)_score_ is the harmonic mean of _Precision_ (5) and _Recall_ (6) scores, given by:  
\[F_{1}\;score=2\cdot\bigg{(}\frac{Precision\;\cdot\;Recall}{Precision\;+\;Recall} \bigg{)}. \tag{7}\]  
For a problem of data classification into \(K\) classes, accuracy, _Precision_, _Recall_, and \(F_{1}\)_score_ metrics are computed for every \(k\)-th class label separately, assuming elements with the \(k\)-th class label as positive and elements with any other class label as negative. Next, either a weighted or unweighted average is computed for scores for every class label.  
Unweighted averages of _Precision_ (5), _Recall_ (6), \(F_{1}\)_score_ (7) are also known as macro averages [46]. _Macro Precision_ (5) for multi-class classification is given by:  
\[MacroPrecision=\frac{1}{K}\sum\
olimits_{k=1}^{K}\bigg{(}\frac{TP_{k}}{TP_{k}+ FP_{k}}\bigg{)}, \tag{8}\]  
where \(k\) denotes class label index; \(TP_{k}\) denotes true positive elements for the \(k\)-th class label; \(FP_{k}\) denotes false positive elements for the \(k\)-th class label, \(K\) denotes total count of classes in a multi-class classification problem.  
_Macro Recall_ (6) for multi-class classification is given by:  
\[MacroRecall=\frac{1}{K}\sum\
olimits_{k=1}^{K}\bigg{(}\frac{TP_{k}}{TP_{k}+FN _{k}}\bigg{)}, \tag{9}\]  
where \(k\) denotes class label index; \(TP_{k}\) denotes true positive elements for the \(k\)-th class label; \(FN_{k}\) denotes false negative elements for the \(k\)-th class label, \(K\) denotes total count of classes in a multi-class classification problem.  
Finally, the _Macro \(F_{1}\)_score_ that is the harmonic mean for (8) and (9) is given by:  
\[Macro\;F_{1}\;score=2\cdot\bigg{(}\frac{MacroPrecision\;\cdot\;MacroRecall} {MacroPrecision\;+\;MacroRecall}\bigg{)}. \tag{10}\]  
Such metrics as (8), (9), and (10) are useful when searching for a classifier without bias towards classes that occur in a dataset most frequently.  
Finally, the _Accuracy_ score for multi-class classification is the sum of true positive and true negative elements of each class \(k\) divided by class count \(K\). The formula for the _Accuracy_ score is as follows:  
\[Accuracy=\frac{1}{K}\sum\
olimits_{k=1}^{K}\bigg{(}\frac{TP_{k}+TN_{k}}{TP_{ k}+TN_{k}+FP_{k}+FN_{k}}\bigg{)}, \tag{11}\]  
where \(k\) denotes class label index, \(TP_{k}\) denotes true positive elements for the \(k\)-th class label; \(TN_{k}\) denotes true negative elements for the \(k\)-th class label-such elements that have been correctly labeled as negative; \(FP_{k}\) denotes false positive elements; \(FN_{k}\) denotes false negative elements for the \(k\)-th class label, \(K\) denotes total count of classes in a multi-class classification problem.