amc background paper
Analytical Methods Committee Editor: Michael Thompson AMCBP No 2January 2005

What is proficiency testing?
Guide for end-users of chemical data
These comments are based on misunderstandings
reported by users and producers of analytical data.
Interpretation of analytical proficiency test results
should be, if possible, conducted in collaboration withan analytical chemist.

What is proficiency testing?
Proficiency testing comprises an interlaboratory
system for the regular testing of the accuracy that theparticipant laboratories can achieve. In its usual form,
the organisers of the scheme distribute portions of ahomogeneous material to each the participants, whoanalyse the material under typical conditions andreport the result to the organisers. The organisers
compile the results and inform the participants of theoutcome, usually in the form of a score relating to theaccuracy of the result.

What is the difference between proficiency testing
and accreditation?
Accreditation agencies require analytical laboratories
to participate in an appropriate PT scheme where one
is available, and demonstrate a system for handlingthe outcome. This is only one of many requirementsof accreditation.

What kinds of materials are distributed?
The materials distributed are as close as possible tothe materials being regularly analysed, so that theresults of the scheme represent the capability of thelaboratories working under routine conditions.

What is proficiency testing for?
The primary purpose of proficiency testing is to help
laboratories detect and cure any unacceptably large
inaccuracy in their reported results. In other words, it
is designed as a self-help system to tell theparticipants whether they need to modify theirprocedures. Proficiency tests are not ideally designed
for any other purpose, although their results can beused, with due regard to their limitations, and

AMC\AMCTB\STATSCC\PT ENDUSERS\BPV3

20/01/2005

combined with other information, for certain otherpurposes.

Why are there inaccuracies in analytical results?
All measurement gives rise to inaccuracies,
technically known as ‘errors’ in the measurementcommunity. (The word ‘error’ here does not imply
that a mistake has been made, merely that theoutcome of the measurement process varies.) Errors
arise because of unavoidable variation in the physicalor chemical procedure employed to make themeasurement. The measurement of chemical
concentration requires far more complicatedprocedures than typical physical measurements suchas length or time. It is straightforward to measure alength to an accuracy of one part in a million, butchemical measurements can seldom be made with anaccuracy of better one part in a hundred. Mostly theaccuracy is not as good as that, especially if
concentrations are very low, for instance as when
pesticide residues are being determined in foodstuffs.

AMC Background Papers are designed toprovide an interface between analyticalscientists and professionals from otherdisciplines who are involved in theprocurement and interpretation of analytical
data. Technical issues of current interest arepresented.

Is the available accuracy good enough?
That depends on the application. Some analyses haveto be extremely accurate. For example, in determining
the commercial value of a consignment of scrap gold,
the gold content has to be determined with thegreatest possible accuracy, better than one part in a
thousand – a small error could equate to many
thousands of pounds Sterling. In other applications,for example determining the concentration of copperin soil, an accuracy of one part in ten probably
suffices – it doesn’t matter whether the true value is
20 or 22 ppm if the only decision to be made is© The Royal Society of Chemistry 2005

whether the level is above or below 200 ppm. Cost
comes into consideration as well. As a rule of thumb,to improve the accuracy of a measurement by a factorof two decreases the chance of an incorrect (i.e.,expensive) decision, but increases the cost of analysis
by a factor of four. These considerations are known as‘fitness for purpose’.

How do proficiency testing schemes evaluate theaccuracy of individual laboratories?
Most schemes convert the participant’s result into a‘z-score’. This score reflects two separate features, (a)the actual accuracy achieved (i.e., the difference
between the participant’s result and the accepted truevalue), and (b) the scheme organiser’s judgement ofwhat degree of accuracy is fit for purpose.

How should z-scores be interpreted?
z-Scores must be interpreted on a statistical(probabilistic) basis and this requires expert
knowledge. Here is an outline interpretation.
• A score of zero implies a perfect result. This will
happen quite rarely even in perfectly competentlaboratories.
• Laboratories complying with the PT scheme’sfitness for purpose criterion will commonly produce
scores falling between - 2 and 2. They might expect
to produce a value somewhat outside this range
occasionally by chance, roughly about one time intwenty, so an isolated event of this kind is not ofgreat moment. The sign (i.e., + or -) of the scoreindicates a negative or positive error respectively.
• A score outside the range from –3 to 3 would bevery unusual for a laboratory operating under thegiven fitness for purpose criterion, and is taken to
indicate that the cause of the event should beinvestigated and remedied.

What mistakes are commonly made in using zscores?
It is important not to over-interpret z-scores. This
could happen in a number of ways.
• Comparing z-scores between rounds or between
laboratories has to be done with great caution. A
single laboratory operating consistently in line withthe fitness for purpose criterion would typically
produce z-scores in successive rounds covering therange –2 to +2: the following set [0.6, -0.8, 0.3, 1.7,0.7, -0.1] would be typical. The small ups anddowns between the scores do not indicate a change
in performance – they arise by chance. So 1.7 is not‘worse’ than 0.3: it does not indicate deteriorationin performance.
• Because of this ‘natural variation’ it is notsensible to make a ‘league table’ of laboratories
based on their z-scores in a round. It is not valid toclaim that a laboratory scoring 0.3 in a round isbetter than another scoring 1.7.
• Judgements based on average z-scores again
require caution. Averages of z-scores obtained on anumber of different analytes should not be used:
they may well hide the fact that one of the analytes
consistently gives a poor z-score. Averages ofscores from the same analyte over several rounds
may be more useful, but still need expert
interpretation on a statistical basis.

What are the limitations of proficiency testing?
• Proficiency testing has to be carried out within
the context of a complete system for appropriatequality in each laboratory. It cannot be used a
substitute for routine internal quality control. It is
not a means of training individual analysts, nor (forthe participant laboratory) a way of validating
analytical methods.
• Proficiency testing provides a participant
laboratory only with an indication of problems ifthey are present. It does not provide any diagnostics
to help solve the problem.
• Success in a proficiency test for one analyte does
not indicate that a laboratory is equally competentin determining an unrelated analyte.
This Background Paper was produced for theAnalytical Methods Committee by the Statistical
Subcommittee (Chairman M Thompson),which is
supported by the Food Standards Agency. The paper
has been endorsed by the Eurachem UK Working
Group on Proficiency Testing.
AMC Background Papers, Technical Briefs andRecommendations may be freely reproduced anddistributed in exactly the same form as publishedhere, in print or electronic media, without formal
permission from the Royal Society of Chemistry.
Copies must not be offered for sale and the copyrightnotice must not be removed or obscured in any way.
Any other reuse of this document, in whole or in part,
requires permission in advance from the Royal
Society of Chemistry. Correspondence should be
addressed to: The Secretary, The Analytical Methods
Committee, The Royal Society of Chemistry,
Burlington House, Piccadilly, London W1J 0BA.

Other AMC products can be found on:
www.rsc.org/lap/rsccom/amc/amc_index.htm
AMC\AMCTB\STATSCC\PT ENDUSERS\BPV3

20/01/2005© The Royal Society of Chemistry 2005