Quality of clinical practice guidelines in dermatological oncology
Authors: De Haas ERM , De Vijlder HC , Van Reesema W Siewertz , Van Everdingen JJE , Neumann HAM 
Published: June 2007

Background  Clinical practice guidelines are increasingly used. To determine the quality of guidelines the Appraisal of Guidelines and Research and Evaluation (AGREE) instrument was developed and introduced in 2001. The AGREE instrument consists of 23 criteria, grouped in six domains.

Objective  Assessment of quality of evidence‐based guidelines in dermatological oncological care according to the AGREE instrument.

Methods  We searched MEDLINE, PubMed, EMBASE and Cochrane literature and relevant websites of guidelines development programmes and the national dermatological society to identify evidence‐based dermatological guidelines especially in the treatment of to basal cell carcinoma, squamous cell carcinoma and melanoma. Twenty guidelines, published between 1990 and 2005, were appraised according to the AGREE instrument by three authors. Standardized domain scores were calculated as advised by AGREE. We compared guidelines published before 2002 with those published later.

Results  Domain scores in the domains Scope & Purpose and Clarity were scored best. Applicability and Editorial Independence were scored worst (see Table 1). In time a weak trend towards better guidelines was seen. This trend can be attributed to better scores in the domains Search Strategy and Level of Evidence which are closely related to evidence‐based medicine. The increase in score is due to more explicitly mentioning the search strategy, possible conflict of interest and involvement of different specialties in development of the guideline. Using the Mann–Whitney test to compare guidelines published before the AGREE and afterwards only a statistically significant better score was found for the domain Clarity (P < 0.05; Table 2).

Standardized domain scores

Author
Country
Year of publication
Scope & Purpose
Stakeholders
Rigour
Clarity
Applicability
Editorial Independence

Drake
USA
1992
 28
 21
 5
13
 0
 0

Drake
USA
1993
 72
 25
14
25
 0
 0

Drake
USA
1995
 67
 25
 5
25
 0
 0

De Ruiter
Netherlands
1997
 33
 21
29
66
11
 3

Cox
UK
1999
 67
 33
24
46
 6

Telfer
UK
1999
 89
 29
43
58
28
 0

Reeve
Australia
1999
100
100
88
71
44
 0

Negrier
France
2000
 72
 46
48
66
 0
50

Dummer
Switzerland
2001
 83
 63
36
63
 0
 8

Cook
USA
2001
 83
 25
67
58
11
50

Sober
USA
2001
100
 38
67
67
 0
33

Sober
USA
2001
100
 33
55
42
 0
 0

Motley
UK
2002
 83
 25
45
58
 0
33

Motley
UK
2002
 89
 38
40
67
 0
 0

Roberts
UK
2002
 78
 33
45
58
 0
 0

Roberts
UK
2002
100
 50
29
71
61
 0

Marks
Australia
2002
100
100
57
71
50
 0

Beljaards
Netherlands
2003
 78
 54
88
67
44
 0

Doherty
Scotland
2004
 89
 88
69
88
39
75

Rademaker
New Zealand
2004
 11
 25
17
50
 0
 0

Houghton
USA
2004
 56
 33
52
67
44
 0

Quirt
Canada
2004
100
 75
93
71
56
58

Miller
USA
2004
 33
  4
21
38
 0
 0

De Ruiter
Netherlands
2005
 62
 54
95
79
33
33

Statistics

(A) Median standardized domain score of guidelines published before the AGREE instrument (< 2002)

Guidelines before AGREE, n = 12
Median
Percentile <
>

Scope & Purpose
77.5
< 67.0
97.3 >

Stakeholders
31.0
< 25.0
44 >

Rigour
39.5
< 16.5
64.0 >

Clarity
58
< 29.3
66.0 >

Applicability
 0
< 0
11 >

Editorial Independence
 4
< 0
33 >

(B) Median standardized domain score of guidelines published after the AGREE instrument

Guidelines after AGREE, n = 11
Median
Percentile <
>

Scope & Purpose
83
< 56.0
100.0 >

Stakeholders
38
< 25.0
75.0 >

Rigour
45
< 29.0
69.0 >

Clarity
67
< 58.0
71.0 >

Applicability
39
< 0
50.0 >

Editorial Independence
 0
< 0
0 >

(C) Guidelines published before AGREE and guidelines published after AGREE. Using non‐parametric Mann–Whitney t‐test

Domain

P <

Scope & Purpose
0.74

Stakeholders
0.32

Rigour
0.35

Clarity
0.05
9.3–14.9

Applicability
0.21

Editorial Independence
0.38

Conclusions  Guidelines in dermatological oncological care are of reasonable quality according to the AGREE instrument. The domains in the AGREE instrument concern especially the methods of development of guidelines. The score according to AGREE can be improved by explicitly mentioning the different items. As clinical guidelines are regarded to be an important instrument to improve quality of care, improvements are needed.