How to pool ROC curves in R to better understand a model's performance (CC135)
Channel: Riffomonas Project
Date: 2021-08-09

In this Code Club, Pat shows how he would pool ROC curves so that you can directly assess a model's sensitivity for specificity. The area under the receiver operator characteristic (ROC) curve (AUC) is a useful metric of performance, but it isn't always the best way to assess performance since it looks over all possible specificities. The challenge is that with the mikropml framework we get one ROC curve per 80/20 training-testing split and we need to pool the curves to get a composite ROC curve. Even if you don't care about ROC curves, this episode is sure to have a lot of value for you including a little known R tip towards the end of the episode!

Pat uses functions from the #mikropml R package and the #ggplot2 and #caret packages in #RStudio. The accompanying blog post can be found at https://www.riffomonas.org/code_club/2021-08-09-roc-curves.


If you're interested in taking an upcoming 3 day R workshop, email me at riffomonas@gmail.com!

R: https://r-project.org
RStudio: https://rstudio.com
Raw data: https://github.com/riffomonas/raw_data/releases/latest
Workshops: https://www.mothur.org/wiki/workshops

You can also find complete tutorials for learning R with the tidyverse using...
Microbial ecology data: https://www.riffomonas.org/minimalR/
General data: https://www.riffomonas.org/generalR/

0:00 Introduction
3:19 Calculating sensitivity and specificity for a continuous variable
11:47 Interpolating between specificity values
16:44 Generating ROC curve data for many splits
21:23 Plotting pooled ROC curves
26:11 Recap