amc technical brief
Editor M Thompson

Analytical Methods Committee

AMCTB No 16

Revised April 2007

Proficiency testing: assessing z-scores in the longer term
While a single z-score provides a valuable indication of theperformance of a laboratory, a set or sequence of z-scoresprovides a deeper insight. Both graphical and numericalmethods may be appropriate for assessing a sequence of zscores. However, due caution is required with numericalmethods to avoid incorrect conclusions. The use of asummary score derived from z-scores relating to a numberof different analytes is not recommended: it has a very
limited range of valid applications and tends to conceal
sporadic or persistent problems with individual analytes.
Moreover it is prone to misuse by non-scientists.
Summary scores
The following two types of summary score are statisticallysoundly based [1] and may be useful for individual participants
to assess a sequence of z-scores zi , i = 1,K, n derived from asingle combination of analyte, test material and method.

instances where z ≥ 3. The data illustrated immediately show
some noteworthy features. Results from round 11 are mostly too
low, demonstrating a procedure that was faulty in some generalfeature, while analyte 7 gives high results too frequently,
demonstrating a persistent problem with that specific analyte.
The remaining results are roughly consistent with fitness forpurpose, which on average would result in about 5% of z-scores
represented by a small symbol.
J-charts
A J-chart (otherwise known as a ‘zone chart’) [3] is even more
informative, because it combines the capabilities of theShewhart and the cusum charts. It does this by cumulating
special J-scores attributed to successive results on either side of
the zero line. This enables persistent minor biases to be detected
as well as abrupt large changes in the analytical system. Typicalrules for converting z-scores to J and cumulating them are asfollows.

The rescaled sum of the z-scores,

RSZ = ∑i z in,can be interpreted on the same basis as a single z-score, i.e., it isexpected to be zero-centred with unit variance if the z-scoresare. This statistic has the useful property of demonstrating apersistent bias or trend, so that the sequence of results [1.5, 1.5,1.5, 1.5] provides a statistically significant RSZ of 3.0, eventhough any single one result is not significant at the 95%
confidence level. However, RSZ could conceal two large zscores of opposite sign that roughly cancel.
The sum of the squared z-scores,

SSZ = ∑iz i2 ,could be interpreted as a χ n2 distribution for zero-centred zscores with unit variance. This statistic has the advantage of
avoiding the cancellation of large z-scores of opposite sign, but
is less sensitive to small biases. SSZ is especially sensitive tooutliers.

J = 8.
• Ifz ≥ 3,If,
J = 4.
2≤ z <3If,
J = 2.
1≤ z < 2If,
J = 0.
z <1• J-scores from successive rounds are cumulated until J ≥ 8 ,which defines an excursion beyond the action limits andtriggers investigative procedures.
• The cumulator is reset to zero after any such excursion,
before resuming cumulation.
• The cumulator is reset to zero, before resuming cumulation,
when the new value of z is of opposite sign to the previousvalue or reported as exactly zero.
ExamplesSuccessive values of z go from left to right.

Both of these summary statistics need to be protected (forexample, by robustification or filtering) against past outlyingscores, which would otherwise have a long-term persistencezJ
Cumulator

1.5
2
2

1.5
2
4-0.9
00-1.522

Control charts
Graphical methods of summarising a set of z-scores can be justas informative as summary scores and are probably less prone tomisinterpretation. Multiple univariate charts [2], such as thoseshown overleaf, give a clear overview and are especially usefulwhen scores from a group of analytes determined by a common
method are considered. Hand-drawn charts are quick to update
and serve just as well as those produced by computer.

zJCumulator

1.5
2
2

1.5
2
4

-1.5
2
2-1.5
2
4

The control chart (Figure 1 overleaf) shows upward-pointing
symbols to indicate z-scores greater than zero and downwardpointing symbols for those less than zero. Small symbols
represent instances where 2 ≤ z < 3 , and large symbols
Zassess v2.doc 13/04/2007

Page 1

It is an optional aid to add a minus sign to J-scores resulting
from negative z-scores thus:zJCumulator

1.5
22

1.5
24-1.5-2-2-1.5-2-4© Royal Society of Chemistry 2007

Several examples of the cumulative effect of bias are visible
in Figure 2 (which illustrates the same results as Figure 1 forcomparison). For example, Analyte 3 in Rounds 1-4 receives
z-scores of 1.5, 1.2, 1.5, and 1.1 respectively, translating into
J-values of 2, 2, 2, and 2, which cumulate to 8 by Round 4and trigger investigative procedures.

References
1. M Thompson, S L R Ellison and R Wood,
Pure Appl Chem, 2006, 78, 145-196.
2. M Thompson, K M Malik and R J Howarth,Anal Comm, 1998, 35, 205-208.
3. AMC Technical Briefs, No 12 (2003).

_______________________________________________________________________

Figure 1. A multiple control chart for z-scores.

Figure 2. A multiple J-chart for z-scores (same data as Figure 1).
Note that negatively-signed J-scores are used in this chart.
(Examples of standard (univariate) J-charts are shown in Reference 3.)

Other AMC Technical Briefs can be found on: www.rsc.org/amc
Zassess v2.doc 13/04/2007

Page 2© Royal Society of Chemistry 2007