Reviewers may choose to use the risk of bias table format in the MSAC Therapeutic Guidelines, although it should be noted that not all elements of risk of bias are covered by this format. Alternatives could be the Cochrane risk of bias tool, GRADE approach for assessing the risk of bias, or other tools suitable for RCTs, AMSTAR or PRISMA for systematic reviews, and other validated checklists for non-randomised or observational studies. Whatever the choice, the method of assessing risk of bias should be transparent and justifiable. Please note – if an indirect comparison is presented then the risk of bias assessment of the individual trials/studies would need to be supplemented with an assessment of the exchangeability of the study populations being compared – that is, the results for the common comparator arms should suggest that the populations are similar. For some topics, it may make sense to combine this section with the following section, to reduce duplication. If following methodology, some elements of risk of bias will apply to the study as a whole (e.g. ‘Allocation concealment’), while others must be assessed at an outcome level or cluster of outcomes (e.g. ‘Blinding’). An overall risk of bias should be made for each outcome, across studies. See Appendix C for details on the individual studies included in the evidence base. A summary is provided in Table 11. Keep this section brief, and provide more detail in the study profiles, in Appendix C. Provide any information about the study/participant characteristics that is not reported elsewhere in B.3-B.5, but which is key to interpreting the implications of the evidence. Comment on the applicability (directness) of the included studies. Table 11 Key features of the included evidence comparing intervention with comparator Trial/Study N Design/ duration Risk of bias Patient population Key outcome(s) Result used in economic model Jones 2010 225 R, DB 6 mths Low Mortality Not used 2012 310 R, OL 3 mths High Response rate Not used 2005 75 CS, OL 8 mths Low QoL Not used etc etc Meta-analysis 410 k= - - <Fixed effect/random effect> model; <overall pooled> <and> <subgroup analyses> presented; heterogeneity analysis - Survival gain Trial
Summary – Research question If multiple indications are being assessed, consider changing the heading to “Is it safe for XXXX” , or provide sub-headings for the different populations. Concentrate on comparative safety, as measured by the patient-relevant outcomes specified in the PICO Confirmation, if these data are available. The emphasis should be on whether there are clinically relevant differences in the reported harms, irrespective of whether the results are statistically significant. The type of information needed per pre-specified outcome (from the PICO) is in the table below. The table can be copied for additional pre-specified outcomes. Additional graphical representations might be helpful. The meta-analysis is optional – it will depend on the available evidence base as to whether a meta-analysis can be conducted. If a meta-analysis is conducted, forest plots should be presented. For each important outcome, the GRADE approach specifies that the overall quality of the evidence (the confidence in estimates of effect) should be assessed. One domain, the risk of bias, has already been outlined. However, additional domains to be assessed per outcome include inconsistency, indirectness of the population, intervention, comparator, outcome or setting relative to the proposed use of the intervention in the Australian setting, imprecision and risk of publication bias. The quality of evidence for each important outcome may be determined following the GRADE methodology manually, or produced automatically through recording the quality assessment domains using the ‘Guideline Development Tool’. Statements regarding each critical and important safety outcome could therefore be given a grade. The full evidence profile table (with explanatory footnotes), incorporating all critical and important outcomes, should be included in Appendix D. The process also requires the assumed control risk to be determined, as a measure of the typical burden of the disease in the target population (i.e. the Australian population likely to make use of the proposed item number). If there are studies from a directly relevant population in the evidence base, the baseline control risk could be determined from this study or mean baseline risk from a group of studies. Otherwise, assess the baseline risk from relevant observational studies.
Assumptions incorporated into the model structure: Justify the economic evaluation characteristics summarised in Table 17 • Present, as a minimum, the following information for each variable used in the economic evaluation: – name (and definition, as necessary) – quantity in natural units (as appropriate; for example, this is not applicable for unit costs) – source. • Identify and list the direct health care resource items for which there would be a change in use associated with substituting the proposed medical service for the main comparator and define each in terms of natural units. • Estimate the present value of direct health care resource costs and health outcomes. • Discuss the implications for the economic evaluation of any important deficiencies in the available evidence base. • Summarise this information in a table for each type of variable and provide further details of calculations, as necessary. • Present the cost per patient per course if the proposed medical service is for acute or therapy, or the cost per patient per year if the proposed medical service is for chronic or continuing therapy. • Present the remaining results of the economic evaluation first in a disaggregated form, then in increasingly aggregated forms. Use discounting as appropriate. • Present the appropriately aggregated and discounted results separately for outcomes and and separately for the proposed medical service and its main comparator.
Profiles of studies on XXXX included in the systematic literature review Authors ID Publication Year Study design/ duration Level of evidencea and risk of bias assessmentb Location Setting Length of follow-up Study population characteristics Eg N, age, gender, disease description and severity, baseline function Description of Intervention <including duration of treatment> Description of Comparator <including duration of treatment> Relevant outcomes assessed (ie related to outcomes specified in PICO) Measurement of outcomes and methods of analysis <study design characteristics such as CC=case control; Coh=cohort; CS=case series; DB=double blind; MC=multi-centre; NR=non-randomised; OL=open label (unblinded); R=randomised; SB=single blind; X=cross-sectional etc> a source: see NHMRC hierarchy of evidence ; b risk of bias as it relates to primary outcomes of the systematic review
In order to follow GRADE methodology, the full evidence profile tables should be included (with footnotes), per comparison, including all the critical and important outcomes. If no evidence for critical outcomes were identified, a row in the table could be included with a comment that no data were found. Two examples of how an evidence profile table may be formatted are shown below in Table 23 and Table 24 (choose the most appropriate and amend as required). These data are condensed into a ‘Summary of findings’ table, to be included in the main body of the report. Table Evidence profile table example 1 for intervention compared to comparator for population Outcome (units, No. of studies and study design Risk of bias Inconsistency Indirectness Imprecision Other considerations (e.g. publication bias) No. of patients in intervention arm No. of patients in comparator arm Relative effect (95%CI) Absolute effect (95%CI) Quality Importance e.g. Outcome 1 (6 months) Outcome 1 (12 months) Outcome 2 Table Evidence profile table example 2 for intervention compared to comparator for population Outcome (units, No. of participants, No. of studies Risk of bias Inconsistency Indirectness Imprecision Other considerations (e.g. Quality Study events rates (%) events rates (%) Risk with intervention Relative effect (95%CI) Anticipated absolute risk with comparator Anticipated absolute risk with intervention