Example1. Performance of malaria
RDT for low parasite density

Evaluation of test performances:
ROC analyses, etc.
•To develop new method to detect diseases, what you need are:Calculating sensitivity and specificity
Data: (if originally category data) Positive/Negative by that test, Truly
disease/healthy by the gold standard

Several RDTs (Rapid DiagnosticTests) for malaria, originallydeveloped to distinguish malaria
patients from other fever patients•Sensitivity = Positive in Disease / All of Disease
Specificity = Negative in Healthy / All of Healthy
Data: (if originally continuous data) Values by that test, Truly
disease/healthy by the gold standard
ROC analysis: by changing threshold value of positive/negative, seeking
the best threshold as closest point to the upper left point where“sensitivity=1” and “1-specificity=0”.

Patients with fever must havemalaria parasites with high density
in their blood→ High specificity and moderatesensitivity•

Is it also useful in active casedetection study in low parasite density(less than 100 parasites / L) ?

•Pan-R malaria’s results for P.vivax in
Solomon Is. shown below[Statistical analysis][Accuracy ofdiagnostic test][Accuracy of qualitativetest]

Compare several methods by ROC analysis
The method to achieve highest area under the curve (AUC) is judged as theone with best performance.
Actual determination of method may also consider cost, feasibility, etc.

Example 2 (cont’d) ROC analysis of
the depression score

Example 2. Determination of numericalcriterion for diagnosis
By the depression score based on the questionnaire, screen
major depression.

• Enter the table from [File][New dataset] as shown in the right screen cap.

Requirement: Both patients who were clinically diagnosed as
depression and not depression. The depression scores forthem. (2nd line show the score, 3rd line is clinical diagnosis)• [Statistical analysis][Accuracy ofdiagnostic test][ROC …] and specify
options like bottom-left screen cap.

12

3

4

5

6

7

8

9

10

20

13

19

21

22

28

1125

16

19

dep

norm

norm

norm

dep

dep

norm

norm

norm

norm

1.0

0.8
19.000 (1.000, 0.714)

If we set criterion as “more than 18 is depression”, the crosstable of diagnosis below. Sensitivity is 1, specificity is 3/7
Depression NormalPositive
34
Negative
03

Sensitivity

0.6

0.4

The criterion “morethan 19 is depression” 0.2
gives the best pairs ofsensitivity and specificity
Area under the curve: 0.8571
0.0
95% CI: 0.6044-1 (DeLong)
1.0

By changing criteria, we can get the highest set of sens/spec•

PID

12

3

4

5

6

78

9

10

1112131415

Pathology

1111

111

10000000

Marker1

2.2

1.8

2.4

2.2

1.7

2.5

2.9

2.31.81.1

1.3

1.4

2.3

1.0

0.8

Marker2

3.5

2.8

3.9

3.4

1.8

3.0

3.1

2.0

2.1

0.9

2.7

0.9

2.0

0.5

0.4

•

Note: The name of dataset must not be ROC1 nor ROC2. If you doso, those are overwritten during calculation to cause error.

•[Statistical analysis][Accuracy of diagnostic test][Compare two ROC curves]•Z = -0.0981, p-value = 0.9218
AUC of roc1 AUC of roc2
0.8928571 0.9017857

How to draw BA-plot in R•

By manipulating 2 variables, it's
possible to calculate the 2 newvariables D (as X-Y) and M (as(X+Y)/2). Draw scattergram ofM as x-axis and D as y-axis.

•Using MethComp package,BA.plot()•library(MethComp)data(ox)BA.plot(ox)•ox is the results of blood
oxygen saturation of 61
children using gas (CO)measurements and pulse
measurements

0.4

Agreements of 2 quantitative
measurements

The results of 2 different tests to evaluate the same thing may differ.
We can compare them by AUC as the result of ROC analysis.

Get this data as http://minato.sip21c.org/ROC1.txt

0.6
Specificity

Example 3. Compare several
methods by ROC analysis
•

0.8

•

Reliability of newly developed cheapor rapid measurement method has tobe confirmed. For that purpose,
agreement of the results (X) by new
method obtained for the same
subjects with the results obtained by
the gold standard (Y)• Method of checking• Paired t-test: cannot detect theinteraction with absolute quantity• Correlation analysis and scattergram to check the match with theline of x=y.
• BA plot (Bland-Altman plot) isvery famous. Since 1985, this isde facto standard.

Basically, makescattergram with(X-Y) as y-axis,(X+Y)/2 as xaxis.

0.2

0.0