Results Although highlighting a general lack of high- quality real-life effectiveness observational research on these clinical ly important topics , the analysis provided insights into how identified observational studies might inform asthma guidelines developers and clinicians .
Overall , RELEVANT appeared reliable and easy to use by expert review ers .
Conclusions Using such quality appraisal tools is m and atory to assess whether specific observational real-life effectiveness studies can be used to inform guideline development and /or decision-making in clinical practice
Abstract Introduction A Task Force was commissioned jointly by the European Academy of Allergy and Clinical Immunology ( EAACI ) and the Respiratory Effectiveness Group ( REG ) to develop a  quality assessment tool for real-life observational research to identify high- quality real-life asthma studies that could be considered within future guideline development .
The REal Life EVidence AssessmeNt Tool (RELEVANT): development of a novel quality assurance asset to rate observational comparative effectiveness research studies
Abstract Background Evidence from observational comparative effectiveness research ( CER ) is ranked below that from r and omized controlled trials in traditional evidence hierarchies . However , asthma observational CER studies represent an important complementary evidence source answering different research questions and are particularly valuable in guiding clinical decision making in real-life patient and practice setting s. Tools are required to assist in quality appraisal of observational CER to enable identification of and confidence in high- quality CER evidence to inform guideline development . Methods The REal Life EVidence AssessmeNt Tool ( RELEVANT ) was developed through a step-wise approach . We conducted an iterative refinement of the tool based on Task Force member expertise and feedback from pilot testing the tool until reaching adequate inter-rater agreement percentages . Two distinct pilots were conducted — the first involving six members of the Respiratory Effectiveness Group ( REG ) and European Academy of Allergy and Clinical Immunology ( EAACI ) joint Task Force for quality appraisal of observational asthma CER ;  the second involving 22 members of REG and EAACI membership . The final tool consists of 21 quality sub-items distributed across seven methodology domains : Background , Design , Measures , Analysis , Results , Discussion /Interpretation , and Conflict of Interest . Eleven of these sub-items are considered critical and named “ primary sub-items ” . Results Following the second pilot , RELEVANT showed inter-rater agreement ≥ 70 % for 94 % of all primary and 93 % for all secondary sub-items tested across three rater groups . For observational CER to be classified as sufficiently high quality for future guideline consideration , all RELEVANT primary sub-items must be fulfilled . The ten secondary sub-items further qualify the relative strengths and weaknesses of the published CER evidence . RELEVANT could also be applicable to general quality appraisal of observational CER across other medical specialties . Conclusions RELEVANT is the first quality checklist to assist in the appraisal of published observational CER developed through iterative feedback derived from pilot implementation and inter-rater agreement evaluation . Developed for a REG-EAACI Task Force quality appraisal of recent asthma CER , RELEVANT also has wider utility to support appraisal of CER literature in general ( including pre-publication ) . It may also assist in manuscript development and in educating relevant stakeholders about key quality markers in observational CER
Results Although highlighting a general lack of high- quality real-life effectiveness observational research on these clinical ly important topics , the analysis provided insights into how identified observational studies might inform asthma guidelines developers and clinicians .
Overall , RELEVANT appeared reliable and easy to use by expert review ers .
Conclusions Using such quality appraisal tools is m and atory to assess whether specific observational real-life effectiveness studies can be used to inform guideline development and /or decision-making in clinical practice
Abstract Introduction A Task Force was commissioned jointly by the European Academy of Allergy and Clinical Immunology ( EAACI ) and the Respiratory Effectiveness Group ( REG ) to develop a  quality assessment tool for real-life observational research to identify high- quality real-life asthma studies that could be considered within future guideline development .