Quality of clinical practice guidelines in dermatological oncology
Authors: De Haas ERM , De Vijlder HC , Van Reesema W Siewertz , Van Everdingen JJE , Neumann HAM 
Published: June 2007

AbstractBackground  Clinical practice guidelines are increasingly used. To determine the quality of guidelines the Appraisal of Guidelines and Research and Evaluation (AGREE) instrument was developed and introduced in 2001. The AGREE instrument consists of 23 criteria, grouped in six domains.Objective  Assessment of quality of evidence‐based guidelines in dermatological oncological care according to the AGREE instrument.Methods  We searched MEDLINE, PubMed, EMBASE and Cochrane literature and relevant websites of guidelines development programmes and the national dermatological society to identify evidence‐based dermatological guidelines especially in the treatment of to basal cell carcinoma, squamous cell carcinoma and melanoma. Twenty guidelines, published between 1990 and 2005, were appraised according to the AGREE instrument by three authors. Standardized domain scores were calculated as advised by AGREE. We compared guidelines published before 2002 with those published later.Results  Domain scores in the domains Scope & Purpose and Clarity were scored best. Applicability and Editorial Independence were scored worst (see Table 1). In time a weak trend towards better guidelines was seen. This trend can be attributed to better scores in the domains Search Strategy and Level of Evidence which are closely related to evidence‐based medicine. The increase in score is due to more explicitly mentioning the search strategy, possible conflict of interest and involvement of different specialties in development of the guideline. Using the Mann–Whitney test to compare guidelines published before the AGREE and afterwards only a statistically significant better score was found for the domain Clarity (P < 0.05; Table 2).

Standardized domain scores

Author
Country
Year of publication
Scope & Purpose
Stakeholders
Rigour
Clarity
Applicability
Editorial Independence

Drake
USA
1992
28
21
5
13
0
0

Drake
USA
1993
72
25
14
25
0
0

Drake
USA
1995
67
25
5
25
0
0

De Ruiter
Netherlands
1997
33
21
29
66
11
3

Cox
UK
1999
67
33
24
46
6

Telfer
UK
1999
89
29
43
58
28
0

Reeve
Australia
1999
100
100
88
71
44
0

Negrier
France
2000
72
46
48
66
0
50

Dummer
Switzerland
2001
83
63
36
63
0
8

Cook
USA
2001
83
25
67
58
11
50

Sober
USA
2001
100
38
67
67
0
33

Sober
USA
2001
100
33
55
42
0
0

Motley
UK
2002
83
25
45
58
0
33

Motley
UK
2002
89
38
40
67
0
0

Roberts
UK
2002
78
33
45
58
0
0

Roberts
UK
2002
100
50
29
71
61
0

Marks
Australia
2002
100
100
57
71
50
0

Beljaards
Netherlands
2003
78
54
88
67
44
0

Doherty
Scotland
2004
89
88
69
88
39
75

Rademaker
New Zealand
2004
11
25
17
50
0
0

Houghton
USA
2004
56
33
52
67
44
0

Quirt
Canada
2004
100
75
93
71
56
58

Miller
USA
2004
33
4
21
38
0
0

De Ruiter
Netherlands
2005
62
54
95
79
33
33

Statistics

(A) Median standardized domain score of guidelines published before the AGREE instrument (< 2002)

Guidelines before AGREE, n = 12
Median
Percentile <
>

Scope & Purpose
77.5
< 67.0
97.3 >

Stakeholders
31.0
< 25.0
44 >

Rigour
39.5
< 16.5
64.0 >

Clarity
58
< 29.3
66.0 >

Applicability
0
< 0
11 >

Editorial Independence
4
< 0
33 >

(B) Median standardized domain score of guidelines published after the AGREE instrument

Guidelines after AGREE, n = 11
Median
Percentile <
>

Scope & Purpose
83
< 56.0
100.0 >

Stakeholders
38
< 25.0
75.0 >

Rigour
45
< 29.0
69.0 >

Clarity
67
< 58.0
71.0 >

Applicability
39
< 0
50.0 >

Editorial Independence
0
< 0
0 >

(C) Guidelines published before AGREE and guidelines published after AGREE. Using non‐parametric Mann–Whitney t‐test

Domain

P <

Scope & Purpose
0.74

Stakeholders
0.32

Rigour
0.35

Clarity
0.05
9.3–14.9

Applicability
0.21

Editorial Independence
0.38

Conclusions  Guidelines in dermatological oncological care are of reasonable quality according to the AGREE instrument. The domains in the AGREE instrument concern especially the methods of development of guidelines. The score according to AGREE can be improved by explicitly mentioning the different items. As clinical guidelines are regarded to be an important instrument to improve quality of care, improvements are needed.