text:   Chest X-rays",9,"['How do deep learning models perform in the presence of diseases not labeled for or present during training?\n\nOur new paper investigates this key AI+Medicine deployment consideration\n<LINK>\n\n@siyumd @IshaanMalhi \n@AndrewYNg  @StanfordAILab \n\n1/9 <LINK>', ""Context: Datasets used to train models typically only provide labels for a limited number of common diseases.\n\nIt's unknown whether DL models can maintain performance in presence of diseases not seen during training or whether they can detect the presence of such diseases.\n\n2/9"", 'Question 1: Can we detect diseases not seen during training?\n\nWe first design a controlled experiment to evaluate whether deep learning models trained on a subset of diseases (seen diseases) can detect the presence of any one of a larger set of diseases.\n\n3/9', 'Finding 1: We find that models tend to falsely classify unseen diseases as “no disease”.\n\nWe also show that DL models may succeed in identifying “no disease” vs “any disease” when an unseen disease co-occurs with a seen disease, but not when an unseen disease appears alone.\n\n4/9 https://t.co/ew02Mtbmv5', 'Question 2: Is there a performance drop on labeled diseases?\n\nWe evaluate whether models trained on seen diseases can detect seen diseases when co-occurring with diseases outside the subset (unseen diseases).\n\n5/9', 'Finding 2: We find that models are still able to detect seen diseases even when co-occurring with unseen diseases.\n\nMoreover, a model trained with both seen and unseen diseases, but without labels for the unseen diseases, performs better on seen diseases!\n\n6/9 https://t.co/KTomws6J1i', 'Question 3: Can unseen diseases be detected without explicit training? \n\nWe evaluate whether feature representations learned by models may be used to detect the presence of unseen diseases given a small labeled set of unseen diseases.\n\n7/9', 'Finding 3: We find that the penultimate layer of the deep neural network provides useful features for unseen disease detection.\n\nOverall, our results can inform the safe clinical deployment of deep learning models trained on a non-exhaustive set of disease classes.\n\n8/9 https://t.co/7sBa3LFWjB', 'It was really fun working on this project with a talented team of first authors @siyumd and @IshaanMalhi, and Kevin Tran.\n\nRead more details in our paper here:\nhttps://t.co/wjOsxtf1Qk\n\n9/9']",21,03,2228