Statistical analysis of method comparison studies
Pre-conference course at the NBBC13, Sunday 9th June 2013

Bendix Carstensen
Senior Statistician, Steno Diabetes CenterNiels Steensens Vej 2, DK-2820 Gentofte, Denmark& Department of Biostatistics, University of Copenhagenbxc@steno.dk

http://BendixCarstensen.com/MethComp/Courses/Stockholm.2013

Attendance
You should attend this course if you ever have come across a clinician that asked you which of
two methods are the better for measuring blood-pressure (say). Or a researcher trying toconvince you (and the rest of the world) that a correlation of 0.93 is compelling evidence that
two methods are in agreement.
You will learn how to provide them with what they need rather than what they want.

What it’s about
The comparison of two methods of measurement using the so-called “Bland-Altman”
procedure of plotting the difference against the mean for each pair of observations has become
the de facto standard for analysis of method comparison studies without replicates [1, 2]. But
still some papers appear that rely on correlations in some form or another.
The Bland-Altman procedure produces limits of agreement, that is prediction limits for thedifference between a measurement by one method and a measurement by the other. TheBland-Altman approach has been expanded with practical methods for the analysis ofsituations with replicate measurements for multiple methods [5].
This half-day course will give an introduction to the standard procedures, and put these in aproper modeling framework. More elaborate designs of method comparison studies, inparticular studies with replicate measurements by each method will be covered. Studies withmore than two methods and with non-constant, and even non-linear relationships between
methods will be covered too. As a bonus you will also learn why you should only compute thecoefficient of variation from log-transformed data.
I will illustrate the methods by examples from the literature, and demonstrate the use of theMethComp package for R. There will be opportunities to try out the MethComp package on
your own computer, either using the examples or your own data.

Details of the subject
If the the traditional Bland-Altman plot is rotated 45 degrees we get a plot of measurements by
one method versus those of the other, with a line that allows conversion from measurement by
one method to what would have been the measurement by the other, with a prediction interval.
And this plot can be used both ways [4, 3].
The core of the approach I will present is a proper statistical model for the observations,which makes it easier to adequately address the parameters of interest (and identify those notof interest), as well as to assess the assumptions underlying the model.

1

If the comparison experiment has been properly conducted with replicate measurements, thestandard approach by Bland and Altman is a special case of standard random effects models.
The model to use depends on the structure of the replication scheme, i.e. whether the replicate
measurements are exchangeable or linked.
The general model for measurements ymir by method m on individual i, replication r is one
where we assume a “true” value µi for the ith individual and a method-specific bias αm andthree components of variance ω, τm and σm :ymir =αm + µi + air + cmi + emirs.d.(air ) = ω, s.d.(cmi ) = τm ,s.d.(emir ) = σmI will show how this model can be analyzed and reported in the form of a plot converting fromone method to another. Basically, the parameters of interest are the variance components τ andσ, whereas the variance component ω and the individual means µi are nuisance parameters,and depending on the design of the experiment, alien to the comparison of the methods.
If basic assumptions such as constant difference between methods or homoschedastic errors
are not met, transformation of measurements may remedy this. However, only thelog-transform provides results which are interpretable on the transformed scale.
However for any transformation of measurements it is possible to do the correct analysis in
the transformed scale, and subsequently back-transform resulting limits of agreement etc. tothe original measurement scale, providing a direct conversion between methods enabling
conversion with relevant prediction limits both ways.
Even if transformation can provide a scale with constant variance it may still be so that thedifference between the measurement methods is not constant:ymir = αm + βm (µi + air + cmi ) + emir
It is possible to estimate in this model allowing for linear relationships between methods, but it
is no longer a standard mixed-effects model, and requires either an extra layer of iteration orbrute force such as MCMC estimation.

References[1] JM Bland and DG Altman. Statistical methods for assessing agreement between two
methods of clinical measurement. Lancet, i:307–310, 1986.
[2] JM Bland and DG Altman. Measuring agreement in method comparison studies.
Statistical Methods in Medical Research, 8:136–160, 1999.
[3] B. Carstensen. Comparing Clinical Measurement Methods: A practical guide. Wiley,2010.
[4] B. Carstensen. Comparing methods of measurement: Extending the LoA by regression.
Stat Med, 29:401–410, Feb 2010.
[5] B Carstensen, J Simpson, and LC Gurrin. Statistical models for assessing agreement inmethod comparison studies with replicate measurements. International Journal of
Biostatistics, 4(1):Article 16, 2008.

2