Practice evidence to put evidence into practice
Authors: Barbara Caracciolo, Astrid van Rijn, Sergio Bonini
Journal: Allergy (2004)

In the Allergy issue of March 2004 the article Allergic Rhinitis and its Impact on Asthma (ARIA) in the Pharmacy has been published. This consensus document extends the concepts and recommendations included in the original WHO guidelines on ARIA to pharmacists. The fight against allergic diseases calls for a collaboration of all health professionals. Therefore, we consider the involvement of pharmacists in a balanced management of diseases with high prevalence, chronicity and significant socio-economic impact of extreme relevance and novelty. The ARIA in the Pharmacy guidelines as well as the original ARIA document and other guidelines stress the concept of practice being based on evidence. Evidence-based medicine (EBM) has introduced a series of new terms and concepts [such as meta-analysis, systematic reviews (SRs), grading of evidence and recommendations, randomized clinical trials – RCTs, etc.] whose relevance is widely accepted. However, not all research workers and practicing doctors are fully familiar with the complex background of these terms and concepts. This editorial aims at addressing some aspects of ‘evidence’ that should be considered when writing or applying guidelines. It also provides a series of references to practice evidence, in order to better put evidence into practice. Evidence-based medicine is a new attitude of evaluating data and experiences, measuring them and extracting conclusions to be included in recommendations and applied in clinical practice. An insightful definition of EBM states: ‘The conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research’ (1). It is worth noticing that this definition of EBM does not cancel the value of individual clinical expertise (the so-called ‘clinical eye’ emphasized by many of our mentors), but requests that this is supported by ‘the best available external clinical evidence’ which comes from ‘systematic research’. However, to keep up-to-date on clinical evidence from research is far from easy, even when focusing upon highly specific topics. In the last 20 years, the volume of medical literature has steeply increased. Considering only randomized controlled trials (RCTs), about 20 000 articles are submitted every year to the NLM for inclusion in MEDLINE (2). Time constraints, together with uneasiness of research findings appraisal, often limit health care professionals’ updating and contribute to delay translation of evidence into practice. Clinical practice guidelines, such as ARIA, are meant to reduce this gap, by providing and disseminating practical recommendations based on systematic referencing. The production, dissemination, implementation and monitoring of guidelines is a highly complex stepwise process (see Table 1), as recently highlighted by several regulatory bodies including the Scottish Intercollegiate Guidelines Network (SIGN; 3) and the Agency for Healthcare Research and Quality (AHRQ; 4). Indeed, it is not sufficient to link recommendations to specific evidence background. Guidelines should also offer a critical appraisal of evidences used to support recommendations. Therefore, guidelines committees are requested to perform a grading of both evidences and recommendations strength. To fulfil this duty, guidelines committees should rely upon explicit, pre-established criteria. Referring to a specific grading system is of paramount importance in giving transparency to the process of guidelines creation and it is likely to promote the exercise of critical thinking among guidelines users. Currently, a number of grading systems are available worldwide. All of these classification tools share the same general structure, but they may differ for the importance given to a specific class of evidence or for their specificity. In a recent study, the AHRQ individuated 40 systems for grading the strength of evidence (5). Among them, the Agency for Health Care Policy and Research (AHCPR) grading system (6), the same used for the ARIA recommendations, has been a reference for many years since its publication in 1993. With respect to the ‘classic’ AHCPR system, recent systems such as the SIGN system (7) and the Oxford Centre for Evidence-based Medicine (CEBM) system (8) are more analytic. For example, level A recommendations were ‘classically’ supported by at least one RCT, as stated by the AHCPR grading system. By contrast, both SIGN and CEBM systems base level A recommendation upon evidences coming from SRs. Specifying that SR must be homogeneous, in the case of CEBM, or that they must be of high quality, in the case of SIGN. The latter also considers high quality meta-analysis or RCT with extremely low risk of bias and directly applicable to the target population, for level A recommendations. Apart from differences in the grading systems of evidence and recommendations, there is actually a general agreement on the superiority of evidence coming from SRs. Indeed, SR are much more than a simplistic review and to provide evidence it is not sufficient reviewing the literature. Review articles, as they usually appear in biomedical journals, still represent an individual biased appraisal of articles selected, certainly appreciable for the individual critical contribution to extract messages from available data. However, common review articles cannot replace the approach needed to provide evidence, which has to follow specific rules to be systematic. In order to make a proper SR, one must examine all evidence about a selected time frame and topic and must follow pre-established criteria to ascertain evidence quality. Moreover, at the end of each SR, evidence should be summarized in a quantitative way whenever possible. The greatest impulse to SRs production came from Archie Cochrane, a British medical researcher who in 1979 wrote: ‘It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials’. Cochrane's inspired vision lead to the foundation of the Cochrane Collaboration (1992), which is ‘an international not-for-profit organisation, providing up-to-date information about the effects of health care’, as stated on Cochrane Collaboration's home page (9). The Cochrane Collaboration is organized in different study groups, corresponding to different areas of interest in medical research. Currently there are 50 Cochrane Collaboration groups aimed at producing and disseminating SRs on a voluntary base. As we stated above, one of the differences between reviews and SRs is the exhaustivity of the search in the literature. Indeed, systematic reviewers must examine all available evidence, trying not to discriminate between different publication languages or between positive and negative results. This is fundamental in order to limit most common biases, such as publication bias and reference bias. The first relates to the tendency of investigators or editors to fund their decisions regarding submission or acceptance of manuscripts for publication depending on positivity of study results (10). The second relates to the tendency of reviewers to report studies in agreement with own opinions. An exhaustive, SR prevents, or at least reduces, such biases offering a more comprehensive grasp of the reality. After the literature search, collected evidence must be evaluated. As seen for the guidelines creation process, relying upon pre-established criteria minimizes the effect of subjective preferences. The AHRQ found more than 100 sources of information on systems to rate individual studies (5). Among them, the CONSORT statement (11) is particularly popular. It applies to the evaluation of published RCTs and it is structured in a stepwise manner, which follows the different steps of an RCT (enrollment, intervention allocation, follow-up, statistical analysis, etc.). With respect to the contents, just to make some examples, the CONSORT states that randomization methodology has to be detailed in the paper and that blinding (of participants, of those administering the intervention and of those assessing the outcomes) has not only to be performed but also to be evaluated, otherwise RCTs quality would be compromised. At the end of the SR process, evidence must be summarized. Narrative text is usually completed with evidence tables, where each line reports a single source of evidence with its effect size and sample size. Whenever possible, data findings must be pooled into a meta-analysis (12). This technique applies to studies using the same effect and whose data are sufficiently homogeneous, as measured by means of statistic heterogeneity tests. If these conditions do occur, effects from different single studies are weighted and pooled together to form a cumulative effect. This measure of effect is much more informative than the original one. In conclusion, however, it is worth saying that, even if SRs and meta-analysis represent the best evidence, their methods do not apply to every circumstance or to every kind of clinical question. In fact, there are issues that could not be scrutinized empirically – as highlighted by a recent paper on the untestability of parachute effectiveness in preventing death and major trauma (13). Moreover, even when available, information coming from experimental research does not automatically translate into practice. As summarized in Table 1, other aspects must be taken into account when formulating clinical practice guidelines. Among them, patients preferences and expectations (which are of paramount importance), social circumstances, logistic and economic resources. All these issues should be considered by the panelists when compiling or updating guidelines. Unfortunately, all the above considerations are not so clear to all those who write and/or apply guidelines. This is possibly the major cause why the substantial progress in knowledge is ‘lost in translation’ (14) to medical practice.