POS0918 VALIDATION OF A MODIFIED NEW YORK CRITERIA RETIC E-TOOL FOR INCREASING SCORING PROFICIENCY OF RADIOGRAPHS OF THE SACROILIAC JOINTS IN AXIAL SPONDYLOARTHRITIS
Authors: A. E. F. Hadsbjerg, M. Østergaard, J. Paschke, R. Micheroli, S. J. Pedersen, A. Ciurea, M. J. Nissen, K. Bubova, S. Wichuk, M. De Hooge, S. Krabbe, A. J. Mathew, M. Gregová, M. Wetterslev, K. Gorican, K. Pintaric, Z. Snoj, B. Moeller, A. Bernatschek, M. Donzallaz, R. G. Lambert, W. P. Maksymowych
Journal: Annals of the Rheumatic Diseases (2023)

<h3>Background</h3> Radiographic assessment of the sacroiliac joints (SIJ) according to the modified New York (mNY) criteria is essential in the classification of ankylosing spondylitis, but low to moderate inter-reader agreement and limited improvement by calibration have been reported [1]. A web-based real-time iterative calibration (RETIC) module for scoring SIJ radiographs according to the mNY criteria has been developed by experienced readers to allow remote standardized calibration. <h3>Objectives</h3> To test the performance of this online RETIC calibration module in enhancing scoring proficiency with an expert reader as gold standard reference. <h3>Methods</h3> This RETIC module consisted of 50 cases with radiographs of the SIJ at one timepoint with an integrated scoring interface with real time feedback to the reader's grading compared to the expert readers' grading. The mNY grade (0-4) was assessed for each joint. Reliability for fulfilling versus not fulfilling the mNY criteria was assessed in real-time by kappa data being provided every 10 cases, until proficiency targets (kappa &gt; 0.5) were achieved. In this study, 19 readers from the EuroSpA Imaging project were randomized to one of two reader calibration strategies (groups A and B) that comprised of 3 calibration steps, each followed by a reading exercise (Exercises 1-3; 25 patients per exercise). Calibration steps were as follows: Group A; 1. Review of original manuscripts describing the mNY method. 2. Review of PowerPoint summary of mNY method and video with grading examples PLUS completion of mNY<sub><i>RETIC</i></sub> module. 3. Re-review of PowerPoint summary and video. Group B; same 3-step strategy as group A except that the mNY<sub><i>RETIC</i></sub> module was completed at step 3, i.e. immediately before Exercise 3. The reliability of scoring was compared to an expert radiologist (RL) using kappa statistics for positivity of mNY criteria and intraclass correlation coefficients (ICCs) for the grades of the mNY criteria. <h3>Results</h3> The mean agreement for all readers on fulfillment of the mNY criteria with kappa was 0.69 in exercise 1, 0.72 in exercise 2 and 0.83 in exercise 3. Corresponding ICCs for mNY scoring 0-4 were 0.77, 0.77 and 0.82 in exercises 1, 2 and 3, respectively, see Table 1 and Figure 1. For Reader Group A, kappas for exercises 1-3 were 0.65, 0.76 and 0.83, respectively and ICCs 0.71, 0.82 and 0.81 (exercise 3). Interestingly, the main increase in agreement occurred between Exercises 1 and 2, i.e. after the RETIC module was performed. For Reader group B, corresponding kappas were 0.71, 0.77 and 0.83 and ICCs 0.80, 0.72 and 0.80, for exercises 1-3, respectively. Similarly, agreements predominantly improved directly after the RETIC module was performed. <h3>Conclusion</h3> Agreement on scoring radiographs of the SIJ according to the mNY criteria was noticeably improved when using a systematic online calibration module and in general readers achieved very good agreement with experts implying that systematic calibration of readers before scoring of images can enhance scoring proficiency. <h3>Reference</h3> [1]van Tubergen A, Heuft-Dorenbosch L, Schulpen G, Landewé R, Wijers R, van der Heijde D, et al. Radiographic assessment of sacroiliitis by radiologists and rheumatologists: does calibration improve quality? Ann Rheum Dis. 62(6):519–25. <h3>Acknowledgements:</h3> NIL. <h3>Disclosure of Interests</h3> Anna E F Hadsbjerg Grant/research support from: Novartis., Mikkel Østergaard Speakers bureau: AbbVie/Abbott, Eli Lilly, Gilead, Novartis, Janssen, Pfizer, UCB, Grant/research support from: AbbVie/Abbott, Eli Lilly, Novartis, Janssen, UCB, Amgen, BMS, Celgene, Joel Paschke: None declared, Raphael Micheroli: None declared, Susanne Juhl Pedersen Speakers bureau: AbbVie/Abbott, Merck/MSD, Novartis, Pfizer, UCB, Grant/research support from: Merck/MSD, Novartis, Adrian Ciurea Speakers bureau: AbbVie, Merck/MSD, Novartis, Michael J Nissen Speakers bureau: Janssen, Amgen, Novartis, Consultant of: AbbVie/Abbott, Pfizer, Kristyna Bubova: None declared, Stephanie Wichuk: None declared, Manouk de Hooge: None declared, Simon Krabbe Grant/research support from: Novartis, Ashish Jacob Mathew: None declared, Monika Gregová: None declared, Marie Wetterslev: None declared, Karel Gorican: None declared, Karlo Pintaric: None declared, Ziga Snoj: None declared, Burkhard Moeller: None declared, Alexander Bernatschek: None declared, Maurice Donzallaz: None declared, Robert G Lambert Consultant of: Calyx, CARE Arthritis, Image Analysis Group, Walter P Maksymowych Speakers bureau: Novartis, Consultant of: AbbVie, Boehringer-Ingelheim, Celgene, Eli Lilly, Galapagos, Janssen, Novartis, Pfizer, UCB, Grant/research support from: AbbVie, Janssen, Pfizer, Employee of: CARE Arthritis Limited