amc recommendation
Editor: M Thompson. No 1 Jul 2004

 Royal Society of Chemistry 2004

Test for ‘sufficient homogeneity’ in a reference material
ScopeThis recommendation covers the chemical composition ofreference materials that are already divided into packaged
units for distribution to different users. ‘Sufficienthomogeneity’ means that the distribution units differ fromeach other in composition only to an extent that does notaffect the interpretation of the results when the material isin its intended use.
Recommendation
1. The test shall be conducted on at least ten distribution
units selected at random from all of the packaged units
prepared for distribution.a
2. Each of the m selected units shall be homogenised, andduplicate test portions taken from it analysed, in arandom order, by using a method of appropriate
precision under repeatability conditions.b,c
3. The results of the analyses shall be scrutinised to ensurethat anomalous features indicating problems with theanalytical results (non-random features, lack ofresolution) are absent. Datasets affected by suchproblems shall be discarded and the test repeated abinitio.d, e
4. Acceptable datasets shall be treated by one-way
22
analysis of variance to provide estimates s sam, s analofthe between-unit variance and between-result variancerespectively.
5. A criterion of compliance shall be related to an
uncertainty u f that is regarded as fit for purpose

Notesa) ‘Selected at random’ means that a formal randomisation
procedure shall be used rather than an informal procedure suchas shuffling.
b) ‘Appropriate precision’ means that the repeatability standard
deviation of results obtained by the method shall be smaller
than 0.5 times the standard uncertainty u f that describes
fitness for purpose in the relevant application.
c) The balanced design recommended is probably optimallyrugged. (In other words, there may be designs, requiring thesame analytical outlay, that are slightly more powerful underideal conditions of execution, but may be more prone to givemisleading results under slight deviations of procedure.)d) Non-random effects in the results can be mistaken forheterogeneity, or can mask it. Such effects result from a failure(i) to randomise the order of analysis or (ii) to use precisionconditions other than repeatability (e.g., when the analysis issplit into several runs). If the analysis cannot be executed
except in two or more runs, the analyst must make every effort
to keep the conditions stable between runs.
e) Lack of digit resolution in the data may result in the variation
between results or units being represented inadequately or not
at all. Analysts conducting the tests may need to be advised torecord more significant figures than customary.
f) A full account of the statistical rationale of the recommended
method can be found in Reference 1.
g) Figure 1 below may assist in the recognition of the different
types of outliers. Outlying differences are most easily detectedby the use of Cochran’s test at the 95% level of significance,
although other statistically sound tests will suffice. A fuller
discussion of outliers can be found below.

defined as an allowable variance σ al2 = 0.09u 2f . The
material is deemed to be sufficiently homogeneous only
if2s sam≤χ m2 −1σ al2m −1+(Fm −1, m2− 1)s an2

where χ m2 −1 is the value exceeded with a probability of
0.05 by a chi-squared random variable with m-1 degrees
of freedom, and Fm −1, m is the value exceeded with aprobability of 0.05 by a random variable with the Fdistribution with m-1 and m degrees of freedom.f
6. When, among the differences between corresponding
duplicate results, there is a single outlying difference,
the respective pair of results may, after due
consideration, be deleted from the dataset before theanalysis of variance. No other data shall be deleted. If
two or more such outlying differences are present, thewhole dataset shall be discarded and the test repeated
ab initio. gFig. 1 Results from a test for sufficient homogeneity, showing
two types of outlier.
Background to the Recommendation
Tests for homogeneity have traditionally been based on
randomised repeated experiments followed by the use of2analysis of variance to test the hypothesis H : σ sam= 0.
Unfortunately the outcome of such an experiment depends asmuch on the quality of the measurement as on the quality of thematerial. If the analytical precision in the experiment is low, thetest may fail to detect heterogeneity. In contrast, a highprecision method may find as significant a degree ofheterogeneity that is entirely inconsequential. Moreover, most,perhaps all, candidate reference materials are actually orpotentially heterogeneous and are demonstrably so, given a
sufficiently precise analytical method. A naive test forheterogeneity per se is therefore pointless.
What is required is a test to demonstrate that the inevitablevariation in composition between distribution units of thematerial is inconsequential in relation to it intended use. Thisrequirement naturally calls for a comparison of the between-unit
variance with an independent criterion based on fitness forpurpose. An early test of this type [2] accepted the material as
sufficiently homogeneous if s sam < 0.3u f , where, as above,s sam is the estimated between-unit standard deviation and u f
is the standard uncertainty defining fitness for purpose. This test
has the defect that the standard error of the estimated betweenunit variance is highly dependent on the analytical precision.
This factor could lead to an undue tendency of the test to rejectmaterial that was in fact satisfactory, a circumstance that adds
considerably to the average cost of preparing a material.
The new test recommended here was designed to overcome that
defect by taking account of the analytical precision. It tests the222
hypothesis H : σ sam ≤ σ al , where σ al is allowable variancederived from the uncertainty regarded as fit for purpose. In

many instances, σ al = 0.09u f is a suitable criterion.
22Limitations of homogeneity tests
Tests for sufficient homogeneity can never be entirely
satisfactory, however. The main problem is that any conceivable
test will have a low statistical power (probability of rejecting thematerial when it is in fact unsuitable) in marginal cases unless alarge (m >> 10) number of units are analysed, at inordinate
expense. It is therefore better to keep the cost of the test withinreasonable bounds by minimising the false rejection ofsatisfactory material and regarding the test as a filter to prevent
gross mistakes.
Homogeneity of individual units
This Recommendation does not advocate experimental designs
that test the contents of individual units for homogeneity.
Although such a procedure is currently recommended by ISO[3], it doubles the cost of analysis (for balanced designs) and theoutcome is of no practical consequence. A unit that is‘homogeneous’ at the time of testing cannot be guaranteed to beso after transportation, handling and storage. It is therefore auniversally recognised responsibility of the individual analyst toensure that any laboratory sample, including the special case of
the reference material, is sufficiently homogeneous before thetest portion is taken.
Types of outliers
There are potentially two types of discrepant result stemmingfrom the test for sufficient homogeneity, namely outlying units
and outlying results, as shown in Figure 1. An outlying unit isindicated by two concordant results from the unit, the mean ofwhich is discordant with the means of the remaining pairs ofresults. A pair of outlying results from a single unit isindicative of heterogeneity and the results must not be
discarded before analysis of variance is applied. Outlying
results occur when the two results stemming from a single unit
differ from each other by an amount that is implausible inrelation to all of the other differences between pairs of results.
Contrary to previous recommendations [2], it is usually
appropriate to delete a pair of results showing an outlying
difference and then proceed with the analysis of variance.

Rationale for deleting an outlying result
The justification for deleting an outlying pair of results is asfollows. Analytical outliers resulting from uncontrolled variationin the analytical procedure are well recognised and occur at a
rate of perhaps a few percent. A dataset from a homogeneity test
contains at least 20 results, so datasets containing a single outlier(but otherwise reliable) are encountered quite often. Analyticaloutliers have the effect (perhaps unexpected prima facie) ofmaking the material more likely to pass the test for sufficienthomogeneity, so cannot be tolerated. However, rejecting thewhole dataset because of a single outlier would be unduly
costly.
The situation can be saved by the following consideration.
Analytical good practice requires that the test material of each
unit is homogenised to an appropriate degree before the test
portions are taken. Therefore discrepant duplicate results from aproperly homogenised single unit must result from analyticaloperations performed after the test portion is taken and do notreflect on the condition of the material under test.
It is therefore usually sensible to discard from the dataset one
pair of such results shown to be significantly different (forexample by Cochran’s test) before applying analysis of variance.
The occurrence of two such pairs of results in a dataset,however, calls into question the reliability of the wholeanalytical operation: the dataset should be discarded and thewhole test repeated ab initio.
Due consideration must be given to the nature of the material
before an outlying result is discarded. There are rare but wellknown instances (e.g., ores of precious metals) where theanalyte is present at low trace levels overall in the bulk material
but in the form of just a few particles containing the analyte at aconcentration approaching 100 %. In such a case, the presenceof an outlying result might result from this extreme type ofheterogeneity rather than an analytical outlier. More elaborate
investigations than the current Recommendation are required forthis type of material.
References
1. T Fearn and M Thompson, Analyst, 2001, 126, 1414-1417.
2. M Thompson and R Wood, Pure Appl Chem, 1993, 65, 21232144.
3. ISO Standard on preparation of reference materials, PartNo???
This recommendation was prepared for the AMC by theStatistical Subcommittee.
AMC Recommendations (Editor M Thompson) provide advice
on good practice in evolving areas of analytical chemistry where
there is insufficient help in reference documents or where thehelp in such documents has been superseded by improved
techniques. They are produced under the same procedure as that
established for AMC Technical Briefs. Correspondence should
be addressed to: The Secretary, The Analytical Methods
Committee, The Royal Society of Chemistry, Burlington House,
Piccadilly, London W1J 0BA. AMC Technical Briefs may befreely reproduced and distributed in exactly the same form as
published here, in print or electronic media, without formal
permission from the Royal Society of Chemistry. Copies mustnot be offered for sale and the copyright notice must not beremoved or obscured in any way. Any other reuse of thisdocument, in whole or in part, requires permission in advancefrom the Royal Society of Chemistry.

Other AMC documents and products can be found at www.rsc.org/lap/rsccom/amc/amc_index.htm