Using an R Program to Monitor Pathology Reports for Omissions in Reporting Ancillary Tests and Errors in Test Names

To the Editor.—A key aspect of standardizing pathology reporting is to include all the necessary elements in the reports, including reporting the results of all the billable ancillary tests. Despite care exerted by pathologists, occasional omissions or wrong test names in reports do occur. We have been using a computer program written in the R programming language (https://www.r-project.org, accessed December 25, 2019) to continually monitor the newly finalized cases to detect reports with such omissions or errors.The pathology information system was PowerPath 10.0.0.19 (Sunquest Information Systems, Tucson, Arizona), with Microsoft SQL server as the database management system. A computer program (see supplemental digital content at www.archivesofpathology.org in the August 2020 table of contents) written in R (version 3.5.1) was hosted on a virtual Microsoft Windows server and ran once every 5 minutes to retrieve and analyze data on cases finalized during the preceding 5 minutes. The process by which the program obtains data from the database was described previously.1To detect the possible omission of billable ancillary tests (special stains, immunostains, and others), a list of tests that had been ordered and billable to the patient was retrieved from a table in the database for each case. The final report text for the same case was parsed to see if every test was mentioned in the report text. The program used a conversion text file to link the single way the test was designated in the data table to multiple ways that pathologists would refer to a test in the reports. Taking cyclin D1 as an example, it was designated as “CYCLIN D1” in the data table, but 10 variations such as “cyclin d1,” “cyclin D1,” or “Cyclin D-1” were seen in the report texts. Additional acceptable variations were added to the conversion file periodically to reduce the number of false alarms over time. If the immunostain for cyclin D1 was performed but the corresponding report text did not contain any of the above variations, the program considered the interpretation of this test not included in the report. If one or more tests for a given case were not detected in the report text by the program, such an omission would be brought to the attention of the pathologist via an email.For a 23-month period from August 2017 through June 2019, 547 emails were sent to the pathologists. In 42 cases, the pathologists intended to finalize the reports first and add the interpretations of ancillary tests as addenda subsequently. They were excluded from the analysis. The remaining 505 reports were classified into 3 categories: false alarms (149), test name errors (47), and omissions (309).The false alarms belonged to 2 subcategories. First, there was no error or omission. The reason an email was sent was because the test name conversion file had not contained the acceptable variation of the test name used in that particular report, so that the R program did not know the test was already mentioned in the report. The second category contained either a slightly vague description of the test or the use of unconventional uncapitalization of the letters or unconventional spacing between the characters for the test names. The examples include reporting Ki-67 as proliferation rate, CK5 as high molecular weight keratins, CDX-2 as CDx-2, and CD1a as CD1-A.Because the test name errors and omissions constituted the real deficiencies, the specificity of the alerting emails was 70% ([47 + 309]/505).During the same period, there were 13 890 cases where billable ancillary tests were performed. In 97.4% (13 534 of 13 890) of the cases, the tests were reported without error or omission when initially finalized. The computer program detected 2.6% (356 of 13 890) of reports with either omissions or test name errors that required remedial actions (averaging 15.5 reports/mo). Of these cases, remedial actions were taken in 298 reports (84%; 298 of 356). Computational monitoring made a difference in 2.1% (298 of 13 890) of the cases, increasing the percentage of reports with no error or omission from 97.4% (13 534 of 13 890) to 99.6% (13 832 of 13 890).The computer program was designed to detect inadvertent omissions in reporting ancillary tests; its ability to identify reports with test name errors was an unexpected benefit. These included nonsensical (Table 1) and potentially misleading test name errors (Table 2), together constituting 13.2% (47 of 356) of the deficiencies. In the latter category, some of the tests performed and the tests in the initial reports had very different diagnostic implications. Examples include CK5 versus CD5, CD30 versus CD38, CD117 versus CD34, BCL-2 versus BCL-6, and p63 versus p16.Not all the effective tools are practical if they are burdensome to use, specifically if the users have to remember to use the tools or a nontrivial effort is required to use the tools. The advantage of computational monitoring is that it does not require pathologists to remember to use the tool. The program keeps a constant vigilance in the background and notifies the pathologists in a timely fashion when possible omission is noted.R is not only strong in statistical computing and graphics, it is also powerful in dealing with texts/natural language. In surgical pathology, R has been reported to be used in statistical analysis,2 information extraction from the report texts,1,3 pathology report defect detection,4 and deep learning using report diagnosis texts.5The specific use example of R program in improving the reporting of ancillary tests demonstrates that computational approach can be an effective way to improve standardization in pathology reporting in general. R is a practical language that pathologists can learn and use to produce immediate positive impact in daily practice.