# The Role of Data Analytics in the Assessment of Pathological Speech--A Critical Appraisal
## 3 Review of Selected Works
### Paper Selection Criteria  
Due to the large number of publications produced during the past years, only specific works describing the most relevant contributions are selected following the criteria mentioned before, as older work is already well known to the research community. The paper selection was conducted on well-known databases, such as Scopus and PubMed, using the keywords laryngeal pathology, voice pathology, phonation pathology, and speech pathology, combined with acoustic analysis, and assessment, excluding functional and neurological etiology. Only papers published in indexed journals were taken into consideration. An impact index being estimated as the average number of citations in the years since the publication was used, selecting those papers with a minimum index value of two. The selected works were evaluated to meet the following characteristics:  
* **Exploratory or prospective advanced statistics (EPAS)**. Detection and classification methodologies must rely on robust statistical foundations. Separability and clustering properties rely strongly on feature distributions, connected with syndromic descriptions because differential diagnosis as practiced by specialists depends strongly on the reciprocal association of features linked to symptoms bi-directionally and in one-to-many and many-to-one relationships. In a further effort to ensure interpretability, features should be functionally validated in the context of their relationship to meaningful symptoms. Clinical studies very often have a clear exploratory or prospective character involving small datasets, not of enough size to claim statistical significance but which might gain insight from their assessment against normative databases, either using classical hypothesis tests from descriptive parametric or non-parametric methods, or advanced comparisons based on information theory, higher order statistics [58], or non-linear dynamical analysis [59]. EPAS methods help in offering the first view on pathological etiology, disease understanding, functional evaluation, and monitoring, aiming at an accurate diagnosis, management, treatment, and rehabilitation, to be further verified and generalized on extended-size databases. * **Statistical validation of performance (SVP)**. Many good studies decay in interest if performance results are given as simple scores, lacking statistical support as \(p\)-values or significance intervals. Supporting results on more specific performance data (such as ROC or DET curves, in the case of binary detection, on normalized multi-class performance measurements), may help in giving a clearer idea of the reliability of results, and their acceptability, supporting the confidence of not being produced by chance. Using pure ML data analytics without the support of appropriate statistical validation, no matter how well-performing ML methods may be, could lead to difficult and unrealistic clinical applicability. This is a common problem found in many studies analyzed in the preparation of this critical appraisal not selected for failing to meet this condition. Working on second-hand databases, on a problem modeled only from a shallow point of view, or using transfer-learning methods capitalizing on others' results obtained on a very different background by transfer-learning-based ML methods, might lead to apparently impressive results, which would not reproduce well when exposed to a real clinical scenario. The ease of second-source data access and simple re-training of off-the-shelf pre-trained ML products to generate higher scores lacking sufficient statistical support [60] might not contribute a great deal to the real significance and advancement of voice pathological study and comprehension. Research reported in [27,61] may exemplify good practice regarding EPAS and SVP as quality criteria.