Immunohistochemistry Improves the Diagnosis of Small Cell Lung Cancer; Statistical Issue on Reproducibility Analysis
Authors: Siamak Sabour
Journal: Journal of Thoracic Oncology (2017)

I was interested to read the article by Thunnissen et al. published in the February 2017 issue of the Journal of Thoracic Oncology.1Thunnissen E. Borczuk A.C. Flieder D.B. et al.The use of immunohistochemistry improves the diagnosis of small cell lung cancer and its differential diagnosis. An international reproducibility study in a demanding set of cases.J Thorac Oncol. 2017; 12: 334-346Abstract Full Text Full Text PDF PubMed Scopus (82) Google Scholar The purpose of the authors was to examine whether judicious use of immunohistochemistry (IHC) improves diagnostic reproducibility for SCLC.1Thunnissen E. Borczuk A.C. Flieder D.B. et al.The use of immunohistochemistry improves the diagnosis of small cell lung cancer and its differential diagnosis. An international reproducibility study in a demanding set of cases.J Thorac Oncol. 2017; 12: 334-346Abstract Full Text Full Text PDF PubMed Scopus (82) Google Scholar Nineteen lung pathologists studied interactive digital images of 79 tumors. Images of resection and biopsy specimens were used to make diagnoses solely on the basis of morphologic features (level 1), on the basis of morphologic features along with requested IHC staining results (level 2), and on the basis of all available IHC staining results (level 3).1Thunnissen E. Borczuk A.C. Flieder D.B. et al.The use of immunohistochemistry improves the diagnosis of small cell lung cancer and its differential diagnosis. An international reproducibility study in a demanding set of cases.J Thorac Oncol. 2017; 12: 334-346Abstract Full Text Full Text PDF PubMed Scopus (82) Google Scholar According to their results, the rate of agreement for level 1 was 64.7%, and it increased to 73.2% and 77.5% in levels 2 and 3, respectively. With IHC, κ scores for four tumor categories (SCLC, large cell neuroendocrine carcinoma, carcinoid tumors, and other) increased in resection samples from 0.43 to 0.60 and in biopsy specimens from 0.43 to 0.64.1Thunnissen E. Borczuk A.C. Flieder D.B. et al.The use of immunohistochemistry improves the diagnosis of small cell lung cancer and its differential diagnosis. An international reproducibility study in a demanding set of cases.J Thorac Oncol. 2017; 12: 334-346Abstract Full Text Full Text PDF PubMed Scopus (82) Google Scholar It is crucial to know that using the κ value to assess agreement is a common mistake in reproducibility analysis. There are two important weaknesses of using the κ value to assess agreement of a qualitative variable: it depends on the prevalence in each category, which means that it can be possible to have different κ values having the same percentage for both concordant and discordant cells, and the value of κ also depends on the number of categories.2Sabour S. Reproducibility of circulating endothelial cell enumeration and activation: a methodological issue.Biomark Med. 2016; 10: 1215Crossref PubMed Scopus (8) Google Scholar, 3Sabour S. Reliability assurance of EML4-ALK rearrangement detection in non-small cell lung cancer: a methodological and statistical issue.J Thorac Oncol. 2016; 11: e92-e93Abstract Full Text Full Text PDF PubMed Scopus (42) Google Scholar, 4Sabour S. Li Z.Y. Reproducibility of image-based computational models of intracranial aneurysm; methodological issue.Biomed Eng Online. 2016; 15: 109Crossref PubMed Scopus (7) Google Scholar, 5Sabour S. Adherence to guidelines strongly improves reproducibility of brachial artery flow-mediated dilation. Common mistakes and methodological issue.Atherosclerosis. 2016; 251: 490-491Abstract Full Text Full Text PDF PubMed Scopus (35) Google Scholar In such a situation, weighted κ is a preferable test, giving us an unbiased result. Moreover, for reliability analysis, an individually based approach should be applied instead of a global average, which is usually applied for assessing validity (accuracy) of a test.2Sabour S. Reproducibility of circulating endothelial cell enumeration and activation: a methodological issue.Biomark Med. 2016; 10: 1215Crossref PubMed Scopus (8) Google Scholar, 3Sabour S. Reliability assurance of EML4-ALK rearrangement detection in non-small cell lung cancer: a methodological and statistical issue.J Thorac Oncol. 2016; 11: e92-e93Abstract Full Text Full Text PDF PubMed Scopus (42) Google Scholar, 4Sabour S. Li Z.Y. Reproducibility of image-based computational models of intracranial aneurysm; methodological issue.Biomed Eng Online. 2016; 15: 109Crossref PubMed Scopus (7) Google Scholar, 5Sabour S. Adherence to guidelines strongly improves reproducibility of brachial artery flow-mediated dilation. Common mistakes and methodological issue.Atherosclerosis. 2016; 251: 490-491Abstract Full Text Full Text PDF PubMed Scopus (35) Google Scholar Thunnissen et al. concluded that the diagnosis using hematoxylin and eosin staining alone showed moderate agreement among pathologists in tumors with neuroendocrine morphologic features but agreement improved to good in most cases with the judicious use of IHC, especially in the diagnosis of SCLC. Such a conclusion may be a misleading message on account of inappropriate use of a statistical test. Briefly, for reliability analysis, appropriate tests should be applied. The Use of Immunohistochemistry Improves the Diagnosis of Small Cell Lung Cancer and Its Differential Diagnosis. An International Reproducibility Study in a Demanding Set of CasesJournal of Thoracic OncologyVol. 12Issue 2PreviewThe current WHO classification of lung cancer states that a diagnosis of SCLC can be reliably made on routine histological and cytological grounds but immunohistochemistry (IHC) may be required, particularly (1) in cases in which histologic features are equivocal and (2) in cases in which the pathologist wants to increase confidence in diagnosis. However, reproducibility studies based on hematoxylin and eosin–stained slides alone for SCLC versus large cell neuroendocrine carcinoma (LCNEC) have shown pairwise κ scores ranging from 0.35 to 0.81. Full-Text PDF Open ArchiveReply to Letter “The Use of Immunohistochemistry Improves the Diagnosis of Small Cell Lung Cancer and Its Differential Diagnosis. An International Reproducibility Study in a Demanding Set of Cases.”Journal of Thoracic OncologyVol. 12Issue 6PreviewThe use of (weighted) κ is currently the standard method of reporting, and as such there should not be any controversy.1 The weighted κ is preferred when the data are ordinally categorized, which is not the case in our study, allowing use of the Fleiss κ, which suitable for our data. The limitations of κ statistics in relation to prevalence and number of categories are well known.2 However, we take issue with his statement “They concluded that the diagnosis using hematoxylin and eosin staining alone showed moderate agreement among pathologists in tumors with neuroendocrine morphologic features, but agreement improved to good in most cases with the judicious use of [immunohistochemistry], especially in the diagnosis of SCLC. Full-Text PDF Open Archive