POS0262 IDENTIFYING EROSIVE DISEASE FROM RADIOLOGY REPORTS OF VETERANS WITH INFLAMMATORY ARTHRITIS USING NATURAL LANGUAGE PROCESSING
Authors: G. Penmetsa, S. Pei, B. Sauer, J. A. Walsh, B. Feng, J. Walker, K. Douglas, J. Clewell
Journal: Annals of the Rheumatic Diseases (2021)

Background: The presence of erosive disease influences diagnosis, management, and prognosis in inflammatory arthritis (IA).Research of IA in large datasets is limited by a lack of methods for identifying erosions. Objectives: To develop methods for identifying articular erosions in radiology reports from veterans with IA. Methods: Included veterans had ≥2 ICD codes for ankylosing spondylitis (AS), psoriatic arthritis (PsA), or rheumatoid arthritis (RA) between 2005- 2019, in Veterans Affairs Corporate Data Warehouse. Chart review &amp; annotation of radiology notes produced the reference standard, &amp; identified erosion terms that informed classification rule development. A rule-based natural language processing (NLP) model was created &amp; revised in training snippets. The NLP method was validated in an independent reference sample of IA patients at the snippet &amp; patient levels Step Description Number &amp; example 1 Radiology notes a.Select note titles potentially relevant to IA a. 35,141 notes titles b.Extract notes with titles potentially related to IA b. 2,926,113 radiology notes 2 Possible meaningful terms a.Compile list of root terms that may indicate erosion a. 11 root terms (i.e. ero*, pencil*cup, irreg*) b.Query radiology notes for root term variations b. 1178 variations (i.e. erosion, erotic, erode) c.Select possible meaningful terms c. 179 possible terms (i.e. erosion, erode) 3 Annotation a.Extract snippets^ containing possible meaningful terms a.5000 snippets from radiology notes b.Classify snippets according to: 1) Meaningful term, 2) Relevance to joint, 3) Attribution to IA, 4) Affirmation b.4068 classifications with 1017 snippets (in rounds of 50-417 snippets for NLP training &amp; testing) 4 Rule development a.Identify meaningful terms representing erosion a. 6 terms (pencil * cup, erosion, erosive, etc.) b.Exclude erosive processes irrelevant to joint(s) b. 28 irrelevant processes (i.e. gastric erosion) c. Exclude articular erosive processes not attributed to IA c. 5 non-IA processes IA (i.e. infection) d. Classify as affirmed/negated (erosion present/absent) d. 83 affirmation/negation rules 5 NLP training Design &amp; revise NLP model until accuracy ≥90% 6 rounds, 817 snippets (AS 417, RA 200, PsA 200) 6 NLP testing Test NLP model 200 snippets (AS 100, RA 50, PsA 50) 7 Pt classification a. Develop rules for classifying pts with discordant snippets a. 5 rules developed in 368 pts b. Build reference sample (pts classified as erosive or non-erosive via chart review) b. 30 IA pts (10 AS, 10 RA, 10 PsA) 8 NLP validation Validate NLP model in reference sample at snippet level 149 snippets (29 AS, 76 RA, 44 PsA) 9 Method validation Validate methods (NLP+pt classification) at pt level 30 IA pts (reference sample) pt= patient. ^Snippets include text containing 30 words before &amp; after meaningful terms Results: In 168,667 veterans with IA, the mean age was 63.1 &amp; 90.3% were male. Method development involved radiology note &amp; erosion term selection, rule development, NLP model building, &amp; method validation. The NLP model accuracy was 94.6% at the snippet level &amp; 90.0% at the patient level, for all IA patients. Accuracy of methods. Conclusion: The methods accurately identify erosions from radiology reports of veterans with IA. They may facilitate a broad range of research involving cohort identification &amp; disease severity stratification References: [1]Walsh JA, et al. J Rheumatol. 2020;47(1):42-49 Disclosure of Interests: Gopi Penmetsa: None declared, Shaobo Pei: None declared, Brian Sauer Grant/research support from: I have been an investigator on research contracts supported by Abbvie., Jessica A. Walsh Consultant of: AbbVie, Amgen, Janssen, Lilly, Novartis, Pfizer, UCB, Grant/research support from: AbbVie, Merck, Pfizer, Bingjian Feng Grant/research support from: Bing-Jian Feng reports funding and sponsorship to his institution on his behalf from Pfizer Inc., Regeneron Genetics Center LLC, and Astra Zeneca (UK). The PERCH software, for which Bing-Jian Feng is the inventor, has been non-exclusively licensed to Ambry Genetics for clinical genetic testing services and research., Jodi Walker Shareholder of: Abbvie and mutual funds containing various pharmaceutical companies, Employee of: Abbvie, Kevin Douglas Shareholder of: employed by Abbvie, Employee of: employed by Abbvie, Jerry Clewell Shareholder of: Own Abbvie Shares and mutual funds that hold pharmaceutical and other health care stocks, Employee of: I am current Abbvie Inc employee and past employee of Eli Lilly co