TiedMeasure 0.2.1
Tools to compute and visualize group-weighted tie-aware specificity and sensitivity, as well as their combination score
Requires Python: >=3.7

This is a package to evaluate models' performance, in terms of group-weighted tie-aware measures, including TSpecificity, TSensitivity, and their combination - TScore. The package can produce the naïve specificity, sensitivity, and AUC as well. Here, tied data are particularly referred to a class of data composed of instances (i.e., each presented by a row of the dataframe) that are tied (i.e., they are collected from the same sample). 


# Group-weighted Tie-aware Measures

## Tie-aware Sensitivity is given by

<img src="https://latex.codecogs.com/png.latex?\text{TSensitivity} = \left(1-\frac{1}{1+e^{0.5-\alpha}}\right)\frac{1}{|K^+_\star|}\sum_{\forall k\in K^+_\star}A^+_k + \left(\frac{1}{1+e^{0.5-\alpha}}\right)\frac{1}{|K|-|K^+_\star|}\sum_{\forall k \in K/K^+_\star}A^-_k" />

## and Tie-aware Specificity by
<img src="https://latex.codecogs.com/png.latex?\text{TSpecificity} = \frac{\alpha}{|K^-_\star|}\sum_{\forall k\in K^\star}A^-_k + \frac{(1-\alpha)}{|K|-|K^-_\star|}\sum_{\forall k \in K/K^-_\star}A^-_k" /> 

* <b>Minimum = 0</b> when all samples are incorrectly identified 
* <b>Maximal = 1</b> when all samples are correctly identified
* <b>Medium = 0.5</b> when 50% of samples are correctly predicted if they are identically treated

The tie-aware TPR and TNR wrt a group can be the accuracy on the entropy basis here:

<img src="https://latex.codecogs.com/png.latex?A^\ast_k = \frac{\sum_{\forall i\in C^+_k}(-p^\ast_i\log p^\ast_i)\cdot {Acc}_i}{\sum_{\forall i\in C^+_k}(-p^\ast_i\log p^\ast_i)},\ \ \forall\ast\in\{+,-\}" />

* <img src="https://latex.codecogs.com/png.latex?\alpha\in[0,1]" /> is an user-defined group's weight (<img src="https://latex.codecogs.com/png.latex?K^\star" /> here!), value of 0.5 by default
* <img src="https://latex.codecogs.com/png.latex?Acc_i" /> is the accuracy wrt individual <img src="https://latex.codecogs.com/png.latex?i" />
* <img src="https://latex.codecogs.com/png.latex?\alpha\in[0,1]" /> is the accuracy wrt individual <img src="https://latex.codecogs.com/png.latex?i" />'s samples only

## Tie-aware Score

<img src="https://latex.codecogs.com/png.latex?\text{TScore}_\beta = \sqrt{\frac{(1+\beta^2)\cdot\text{TSensitivity}\cdot\text{TSpecificity}}{\beta^2\cdot\text{TSensitivity}+\text{TSpecificity}}}" />

<img src="https://latex.codecogs.com/png.latex?\beta=1" /> by default




## Example

```
# The result file 'results.csv' looks as below

ID, tied_ID, Group, true_label, predicted_label, predicted_probability
1, A, G1, 1, 1, 0.8
2, A, G1, 1, 0, 0.2
3, B, G1, 0, 0, 0.3
4, C, G2, 1, 1, 0.9
5, C, G2, 1, 1, 0.7
6, C, G2, 1, 1, 0.6
7, D, G2, 0, 1, 0.8
8, D, G2, 0, 0, 0.3
9, E, G3, 1, 1, 0.6

# sig: the weighted group of samples, .e.g., 'G1' and 'G2', which are the two values of the Group columns

# alpha is the user-defined weight for sig, alpha=0.5 by default

# Codes: 


from TiedMeasure.metrics import GWTscore
import numpy as np

df = np.read_csv('results.csv')

M = GWTscore(df, 
             ID = 'ID', 
             tied_ID= 'tied_ID', 
             Group='Group', 
             true_label='true_label', 
             predicted_label= 'predicted_label', 
             predicted_probability= 'predicted_probability',
             sig = ['G1', 'G2'],
             alpha = 0.7)

# compute naïve AUC
# pred_prob: predicted probabilities
# cut_prob: probability cutoff, e.g., 0.85 

cut_prob = 0.5
pred_prob = df['predicted_probability']

M.get_auc(pred_prob,cut_prob) 

# compute the tie-aware sensitivity and specificity, as well as TScore
M.score()

# visualize predicted probability (grouped by samples' true label)
M.draw_prob()

# statistical predicted probability
M.stat_prob()

# visualize the statistics about tied samples 
# only plot those samples repeated twice here, num_occur=2 by default
M.report_group_score('true_label', num_occur=2)

# visualize dataframe
display(M.data)
```