113

PfVDV uUrJL

3t remove NATIONAL BUREAU OF STANDARDS REPORT

1626

HOW TO DECIDE OBJECTIVELY WHETHER

AN OUTLYING OBSERVATION SHOULD BE REJECTED

by

Prank Proschan

U. S. DEPARTMENT OF COMMERCE

NATIONAL BUREAU OF STANDARDS
U. S. DEPARTMENT OF COMMERCE

Charles Sawyer, Secretary

NATIONAL BUREAU OF STANDARDS

A. V. Astin, Acting Director

THE NATIONAL BUREAU OF STANDARDS

The scope ©f activities of the National Bureau of Standards is suggested in the following listing of the divisions and sections engaged in technical work. la general, each section is en- gaged in specialised research, development, and engineering in the field indicated by its

title. A brief description of the activities, and of the resultant reports and publications,

appears on the inside of the back cover of this report.

|« ELECTRICITY. Resistance Measurements. Inductance and Capacitance. Electrical Instruments. Magnetic Measurements. Electrochemistry.

2. OPTICS AMD METROLOGY. Photometry and Colorimetry. Optical Instruments. Photographic

Technology. Length. Gage.

3. HEAT AND POWER. Temperature Measurestents . Thermodynamics. Cryogenics. Engines and Lnbricatiom. Engine Fuels.

*4. ATOMIC AMD RADIATION PHYSICS. Spectroscopy. Radiometry. Mass Spectrometry. Physical Electronics. Electron Physics. Atomic Physics. Neutron Measurements. Nuclear Physics. Radioactivity. X-Rays. Betatron. Nucleonic Instrumentation . Radiological Equipment. Atomic Energy Commission Instruments Branch.

5. CHEMISTRY. Organic Coatings. Surface Chemistry. Organic Chemistry. Analytical Chemistry. Inorganic Chemistry. Electrodepoaition. Gas Chemistry. Physical Chemistry. Thermochemistry. Spectrochemistry, pure Substances.

6. MECHANICS. Sound. Mechanical Instruments. Aerodynamics. Engineering Mechanics. Hydraulics. Mass. 'Capacity, Density, and Fluid Meters. 7. ORGANIC AND FIBROUS MATER I AL3. R»hber. Textile*. Paper. Leather. Testing and

Specifications. Organic Plastics. Dental Research.

8. METALLURGY. Thermal Metallurgy. Chemical Metallurgy. Mechanical Metallurgy. Corrosion.

9. MIMERAL PRODUCTS. Porcelain and Pottery. Glass. Refractories. Enameled Metals. Building Stone. Concreting Materials. Constitution and Microstructure. Chemistry ’ of Mineral

Products.

SO. BUILDING TECHNOLOGY. Structural Engineering. Fire Protection. Heating and Air Con- ditioning. Exterior and Interior Coverings,. Codes «®d Specifications.

11. APPLIED MATHEMATICS. Numerical Analysis. Computation. Statistical Engineering. Ma- chine Development.

12. ELECTRONICS. Engineering Electronics. Electron Ttobes. Electronic Computers. Electronic lastrrasesfcstiaa. 13. ORDHAHCE DEVELOPMENT. Mechanical Research and Development. Electromechanical

Fuses. Technical Service*. Missile Fusing Research. Missile Fuzing Development. Pro- jectile Fuses, Ordnance CosB>©nenfcs. Ordnance Teats. Ordnance Research.

W. RADIO PROPAGATION. Upper Atmosphere Research. Ionospheric Research. Regular Propagation Services. Frequency Utilisation Research. Tropospheric Propagation Research. High

Frequency Standards. Microwave Standards.

15. MISSILE 0EYE10PMEMT. Missile Engineering. Missile Dynamics. Missile Intelligence.

Mi ffisile Instrumentation. Technical Services. Ceatastioa.
NBS PROJECT NBS REPORT

3011-60 0002

25 April \9$£

1626

HOW TO DECIDE OBJECTIVELY WHETHER

AH OUTLYING OBSERVATION SHOULD BE REJECTED

by

Frank

This report is issued for in any form, either in who

from the Office of the Dir

)ved for public release by the

or of the National Institute of

dards and Technology (NIST)

on October 9, 2015

reprinting, or reproduction

ssion in writing is obtained

ington 25, D. C.
1
FOREWORD

A perennial problem vexing the experimenter is that of

rejection of suspected dataD *or on© hundred years attempts

at the solution of this problem have been advanced B most ©f

them t© be themselves rejected as suspeeto Fortunately xnod«=

ern statistical theory has proposed useful reliable methods

for objectively rejecting deviant values 0 However the solution

Is far from complete at prasento

This report describee for the experimenter two of the

modern statistical tests available for possible rejection

of outlying observations 0 Th@s© two methods hav© been

selected because they apply in a majority of the actually

occurring situations^ and because they are so easy to &pplyG

Tii© report liras originally motivated by Mr« Prosehan 9 ® consult®

ativ© work in the Ordnance Development Division^ national

Bureau ©f Standards*, However it is applicable more generally

to the scientific and engineering work at th© National Bureau

of Standard a

o

It should bo borne in mind that this report^ although

Intended .for practical every-day use in our laboratories*, Is

not the final word ©n th© subject of rejection of suspected

observations

o

Jo Ho Curtiss

Chiefs National Applied

Mathematics Laboratories

Ao Vo As tin : Acting Director

National Bureau of Standards
i it

-

'
HOW TO DECIDE OBJECTIVELY WHETHER

AN OUTLYING OBSERVAW5JT SHOULD BE REJECTED

Frank Proschan

Id PROBLEM ; Here is a very common problem facing ex®

perlmentbra at the National Bureau of Standards 0 The typical

scientist* call him Perry Men ter 0 makes a number !^&ay five) •

of repeated measurements of -some unknown quantity© On© of

the values (say the largest) is s© far removed from the other

four that he suspects that it may b© in error

o

However* Perry

has no specific knowledge that a mistake actually did occur 0 Let us assume * too* that h© has no previous data from which

to estimate the precision of measurement© How can h© decide*

from , the v alue a thems elv© 3 » whether the suspected value is

in error or not?

The answer seems clears He should consider the suspected

value as in error when it seems t@o far from th© other four

valitoBo But how can h© judge when it is 51 too far from th©

other four values75 ?

1®2 LOGICAL APPROACH s Hera is a simple* logical* objective

criterion © Suppose Perry could somehow make millions of sets

of five observations each© Suppose* too* that he could guarantee

that none of th©s® observations had any mistakes© Call a

typical set Xy Xy Xy Xy where the ar© arranged In

order of sis©* so that x

]L < xp < < x^ < x^© Now a logical
} ' , ,
measure of the distane© between th© largest v&lu© and th©

other four values Is

. *5’*k

1 k5"3!i

Cl)

io.^oe th© proportion of th© total range, that the distane©

between th© suspected value and its adjacent value* is 0 Now Perry records with what frequency*, among his millions

of sets of five values each*, different values of r„ ©ecuPo

He finds that a value of r^ larger than yjBQ occurs on© time in one

hundred o H© then reasons this ways

n

! have found that among sets of five observations each

( containing no mistakes ) a value of r^ larger than 0 ?80 is

quit© unlikely (occurs only one© in on© hundred) 0 If now g in

ay future experiments I get a set of five observations for

which Pj is larger than o?80 I will conclude that my largest

observation is in erroro^

!°3 CONFIDENCE IN THE 'lEST s This seems reasonable 0 But

what confidence can Perry have in such a procedure? How often

will h© consider as mistaken a perfectly good observation?

How often will he consider acceptable an incorrect observation?

dearly from th© way h© derived th© test 9 he will classify

a perfectly good largest observation as mistaken one© among

©n© hundred sets of fiv© each*, ©n the average o But there is

n© general answer to th© question of how often he will let

pass a mistaken observation o This depends on how 33 mi stakes1*

the mistaken observation is® If a very large error w©r© made^

his test would tend fc© reject the observation almost certainly©

If a very small error were made^ his test would tend to reject

the observation with a small probability©

The following chart [1] gives some idea of the perform

mane® of in detecting mistaken observations© It is based

on a sampling experiment in which samples of five from,

a normal population with mean^u standard deviation o wei’e'' v contaminated with values drawn from a normal population with

mssn yu *>£ o and standard deviation a Q The ordinate a hows the

percent discovery of- cpntaminatora {the proportion of tho time

the c on tamimating population provides an extreme value and

the tost discovers this value) while the abscissa shows X s the

magnitude of the shift (error) of the contaminator in stands

ard deviations©

o 1 1 3 !i M ? 3 ^

(Ft©m Dixon 0 ® article [XJ)
.

- Sdo flO& '

.•

.1

m ©°mw ‘Toi«io‘ lima

. 1® J30&1 • ©itxoa sevlg [!’; snJtwoIJCc', •

.

jm.hm3a.Oii edi to (*ioTrt©} tftirfa

4a&£> - 1® •aB«rsrtts>*t
W© said above that once in every 100 sets of Tallies

(on the average ) Perry would consider as mistaken a perfectly

good observation© If h© were to reject this observation©

and then compute the mean and standard deviation of the remaining values, these would be biased estimates© Also when a good

observation is rejected© any further statistical tests of

significance will become less reliable® Shis la the price

that he must pay for improving the data in the cases where

a mistaken observation is removed 0

l ok HA THEM(El CAL DERIVATI ON % Of course© o780© the value

of r..:

, exceeded by chance 1 percent of the time f

- called the 1

percent level of significance of x», ) 8 is not determined by

actually making millions of sets of five observations eacho

•+ . * Bather it imy be calculated mathematically [2j with greater

accuracy than if millions of sets of five observations had

been used© Ih© basis assumption is that repeated measurement®

would follow the normal dis trlbution0

Smallest Observation Suspected ® What if Perry suspects

the smallest observation in a set (of five© say)?

In this case© he computes

x <=> x r « 2 1

8 x

5

* x

i (2)

H© compaxes i t with 0 jQQe as before® Exactly th© asm© reason®

ing applie s a
, 0000 3 '"^3 ft¥*c#or4 53 . . . .
co ^

e?

lo5> Table X© Let us now define r as either r*. or r

, Is

depending on whether the largest or smallest of a set is being

tested© Table 1 at the ©nd of the paper gives R* the value

of r at various significance levels* ag for sample sis© n from

3 t© 30©

Thus for example® for n s 8 and a * ©05* the table gives

R ® ©4^9 © This moans that in 100 sots of 8 observations each

fro© of mistakes* five values of r^ will b© larger than ©468

and 5 values of r will b® larger than ©468 (on the average)©

Why &r© various significance levels given? The reason

is that no one significance level is appropriate to all problems© For example consider these two cases?

fa) Additional observations are not possible©

(b) Additional observations are possible©

In case (a) for many problems it might be appropriate

to compute r and test it at the 1 percent level of signifies me®©

If the observed value ©f r is larger than the tabl© value for

a 53 1 percent^ It might then b© a good idea to e xclude the

responsible observation©

In ©as© (b)® for many situations a reasonable procedure

might b@ to test r at the 5 percent level© If the sample

value of r Is significant' at the 5 percent level* on© or more

additional observations would be taken© If the observation

originally suspected remained outlying* it would b© tested

again* using the combined set of observations© This time*

however* the r test would b© performed at the 1 percent level

6

of significance® If the outlier were significantly deviant

at the X percent level 0 It would be re jee t@d 0 It should b©

noted that among many sets tested this way0 the proportion

of perfectly good largest values rejected this way will b®

less than 1 pereento This is because the observation has a

^second chance'®8 before it is finally rejected©

le.6 SUMMARY? A set of n obsirvations is made 0 Ho pro®

vioua data are available from which to estimate the variability

of a measurement 0 What is a rational procedure for tasting

whether the largest Cor smallest) of the set is too deviant

to b© explained by the ordinary errors of measurement?

Consider the n observations in order of sis©*

*! i x

2 i *•* i x

n

Compute

X - X _ r ® ,,n aT1 (if x suspected)

3

n

- X

1

©r r ® ^ «i (if tj is suspected)

*n - X

1

Tab!© I may use t© determine how likely it ia to get

as large a' va’ue of r as actually obtained^ simply by chance®

A procedure that might he appropriate for many problems in

as follow?

fa) Ho additional observations possible s In this case.,

cozsparr the computed r with tho Table I value at the 1 p©r~

cent -oyelo If the computed value of r is larger than the
; m «XBW 'toiaetf aJes y,mm |po«s Sarf* •

are si sno.sia-wiad© a to i >:i A - t:

'

aijibooaitv J&mk ' 4o*i s *1

: si Jess «ii* to (iaoXL'sM ip) ' '' " !il

.or. la 1c ai jk' •»£ '

,> ooo •

. g*

»* x. ' s^4-r! .- 1 , -.5- * w JE

f

x « ..;x

, f feed$acrfetf& ,al’ * V ? r ” «*'

s* Ji ^-.CoSlX.wcr 3S-Jfsa"S3 < o* **•« ' ®

I .-raq fm -sot ostjefwwwN* S<JailB t£:r ; ;
Table 1 value* exclude the deviant observation© Otherwise*

do noto

(b) Additional observation® possible s In this case®

compare the computed r with the table value of r at the

5 percent level© If th© computed value of r is larger than

* th© table value * tak© on© or more additional observations

(depending on convenience 9 cost of observations* etc 0 )o

Otherwise accept th© suspected value without taking additional

observations 9 In in th© enlarged set (containing all the original and

'the additional observation (s) ) 0 th© previously suspected value

remains outlying* compute r for the enlarged set o This time

compare It with the Table I- va3.ua at th© 1 percent level©

If th© computed value exceeds th© table value* exclude th©

outlier? otherwise do no to

L? EXAMPLES i 1® Anna Lis to* chemists determines five

values for the iron content ©f an unknown solution by a new

process: ? olj2 9 7 olid ? o 39 0 7a6l 0 7©44 (in percent)© She

suspect® ?o61 as being mistaken* since it is so much larger

than th© other value s« However she cannot determine additional

values because no more samples are available© How can she

decide whether to exclude 7e>oX or notf

Sine© no previous data ar© available from which to compute

th© precision of measurement* th© r test is appropriate© The

first step is to arrange th© five values in order of sis.es
I
8

7 <, 39 * 7 - 42* 7 . 44. 7 - 48* 7-61

I Then

r es x

5

" x

4 .

*5 * X

1

591

Sins© this Is lees than ©?809 the ©01 point of r fox* n ® 5®

Anna List retains th© suspected valu@ 0 7 ©6i

o

2© H© Go Neer fires projectiles froya a gun under constant

s©nd Iti ©ns o Here are the distances in feet (in order of

6801 7683

. Til2k 7720

7502 7799

7$kh

He suspects 6801 as being inconsistent with the other values © What shall h© do?

He compute®

7424 - 6801 S a&2hr

This value lies between “the ©01 and th© ©05 points of r for

n 53 ? © Hence I e G© leer fires an additional round and gets

a new value of 7603©

Sine© 6801 remains outlying in th© enlarged set ©f ©lghte he computes r for this set of eight-, Obviously r remain®

0624 ® "This time it is larger than the ©01 point ©f r for n g 8 a Hence No Go Keer rejects 6801 and uses only th® remaining seven

values©

9

CD 05

iL Estimate of Measurement Variability Available © In a great many laboratory situations^ past data are available

for estimating the undertain ty of a measurement^ Tims if a

laboratory makes routine analysis in triplicate of incoming

Bpeeimsnts d there will b© sets of triplicate values from

which to estimate th® precision ©f measurement© It Is clear

that 'inhere such information is available 8 it should b© used,

in deciding whether an outlier is mistaken or not, This will

make th® decision more reliable than if only th© on© set containing the suspected value is used©

2 e l u Taste The test ratio used now la

U « n (it x is th© suspected value)

(k)

or

where

x * z

i (if is th® suspected value) ©

s ® standard deviation of an individual measurement,

^ based on d degrees of fVoedom©

2©2 Calculating s^o To determine s from a single set

of measurements we would first calculate th© sum of th© squares

of the deviations of th© observations from their mean© Then

tre would divide by one less than the number of observations©

This would give us 8,% Thus d

a

p 2(*i=S3

a,"* 1=1

d • —3^— ( 5 )
, ,:av a" •••

’

' - • y V...VI •,!.'* : W f-JTi-i ' V. .

••

... V :

' r •- \ . .— '

'

. . .. ;q c-.,- s rm

:

: . •,

: ':.o ©d LUJtw m®d$: .. .trn

• ?v; &X 'i©!/., s erfci w'w ,

;•

. \ ,

•.

Ext© 11 .ra&ffd

- siXdBlXa^ ©•wait wo i . -

. sjt ©X^tS'; :' ,:)tL . C!n 8lie v /

. - /

' C X :: .C :,U.:k : / , X ; - $

vBMa^gBMCB •? aaSr- • ;

•••-

-.•l-i",, • - ir.**

;. !>'

..cdr.ov-xrsd o X • , .

'
10 CD

On the other hand s suppose a number of sets of observations

were available

s

at Now we could calculate from

SETS OBSERVATIONS MEAN ^OStseSSSSSSj 1 - . X

xi*' ^12* °°° e X

ln%

X

1

2 X21* *22* x

&s,

z

tmZ

2

k • XkV X

k2° 0 0 ol X, „ is.,k

*.

*—*

( 6 ) '~**OTTS^i) 4. (ng«l) coo*?’ (njr^l) . _

& 0 the member of degrees of freedom for estimating th© uncertain*--

ity of measurements is (n^«l) (n^l) (n^l) 0

2q3 EXAMPLE S An 'example will make th© whole procedure

clean

Dr?i Kern Mist has just run a set of three determinations

on the yield of an ore*»refining process & Hie values are?

39o35* 39o30# 39oOOs with mean 8 3c « 39o22o

H© wished to test whether the M 39oOOH is unusually deviant'o * He

has past data to estimate the precision of measurements
/

/

\ 01 ‘-‘ ,

o B .moqVjM ' * ’'•

&

.% " -• O <$£• «.A #

(! !» £-

<k>

v-V- *° *&S* «XS*

A

e ... T 6 <j ... ».-»«» o o <5 C 3 5‘ <>

..aai2 " !s '! Ss£

s: *J5t*

jso«xl

; ’V eJs Itfoi.su- •>/•••• •••

. <K, * U*t“*

*ol mofee'll to aoe-isoE) *:

j) *, ;0 ' (X^ aJt :

,?v rsi'-I c.8s?.e?o1<| mi v.o ->'• -

0 &S, or ?-® * a as©ts i&iw ;a 0O#?£ ^--,.0;, «

*

1

.IlBjjajjau .si wtfaifcf **•*

xtfasem to nsoi»l'oo*Eq,’Orfd taJssiltfee ol ' ; .l
11

Analysis

Set 1MB* 2 1

. 1. 36 o 5i

' 36 .57 36=70

2 30® 27 30=35

• 30 ol9

3 35 o ©0 . 35=53

‘ 35=36

4

' 43 . 51

' 43.65 43.65

5 5io06 Slol? 51=00

6 48«03 38=19 iffi <,31

T 39 ® ^7 39=51 39.36

8 33.46 33=21 33=28

Although he could calculate 2

' from ( 6 ) convenient (especially with' a computing machine' callable)

t© tisa (7)

:

\1* x. . 2 cM s

<

a

i *, g L^. xki II l /C

'« vi«i 1*1 ®- 7 It can easily bs shown that

1 1 ji (%“1 ) * (n2

o1 ) + » , . # (Hj,TM!

)

to lb,-,

2 r.. 2

k f CT}

C?) is algebraically equivalent

enostituting

»' “sluos into (?) ^ v w vi~ »*-„ C/ ^«Oa "'i £»

* 3©o2?a

36®53> 36®S?a

36o?©a ~

( &S±l^^|±3_6®l©) a

30 o 35a

3©®19^ <• ^S§1Z^|^I_5

^

306

- .I 9

1

a

33o28s « ^I3.^M±33 ® 21^33 <>28 3 ^

1 fr ,

* •••' ^ 33®462

+ 33„21'

H(3-X }*s> 0 „ 0 * (3°lJj^o3Q82 *.c

. .••• "•'

. : ; J -rar'"*

1*-
W£i •; :

«? QloQt

' .<JCoD£

‘ ?&&W

' V-rt ".' X5ot-..? QG TIoJ-V;

i£ ,84;

TSU«P£

’. BSoCC S&cZZ

‘4XI«s*fe^©s ©1 *(&) *’;rw ^.o A- ’ - : ' ,...^:..'C

C efi&foan &

1 a s

\

dS'-Sj

s .i; 4t •

.' H ’ ;/

; r' ;

\ -'-?' •

! \

..tf* &

•

•*

,5: B.C:i *

- '?• :-

. '&* j» <r. o

, 7

'

.
=> 12 "

Kane© a , m ©139

G

The u^ratio defined fey (2jJ gives?

u _

X * Si as 39 o22 - 39,00 , 1„S8

®d ol39

H© now uses Table It which give a the $ percent and 1 p©r<=>

eenfc levels of u for various values of n and do q Is the

sis© of th© sample which contains the suspected value ^ While

d is the number of degrees of freedom on which s is based e a

In th© present casi> n a 3 ©nd d “ I60

The observed, value of u® X©$8 » is considerably loss then

the table value of u at th© 5 percent lev©!® Xo 9©o Henc© h©

concludes that the suspected value 39 ® GO is not significantly

outlying© In other words the deviation of 39® 00 from the moan

©f the set of thro® -measurements is easily explainable in terms

of fch© precision of the measurement process© Henc© 9 39oQ0

is accepted into the fold of good measurements©

2oh TABLE Z o When past data are available * the u^r&fci®

may b@ computed and Table 2 used just th© r^ratio and Table

I were used for th© case where no past data were available©

Th© procedure outlined in paragraph 1©5 for fch© two cases (a)

and (b) may be .followed just as before (using u and Table 2

instead of r and Table I)«

III© Cautions and. Comments © a© Obviously® if the ©xperi®

metor knows that a mistake has occurred he should reject th®

observation© Th© tests of this, paper ar© used only if h®

doesn 8 t know that a mistake has occurred©
QlZi . V VG/V'X.

n

hi) h £v;:LXyvX -i,}, , X :

:

.

At, f m

*K>. &

r. . •'? . &

|tU ffc*

- xox ,i. Xt v.;vv, r X &.</u s vx.Lv XxMv Xl $„XXs,: v- r, vsr

. vdd u 1 x >& rrxv r Xc veyXs? xrvl v 1:.o ; /. tr .r>,

' X-XVX

«XMw * »*?£* * *«*.«- •**;.? yg vx-t vRXtvXxxx do.tdi- ei>,rW ..a-- Xr< r.-Xv

o

f -vvX v : y, Xv X.~X: "vs :;v.,XvovX; Xv xs,:-:v .:. 'V;

,.dX « h dm C

*® £2 s&.ss vvX ?u

v'l •xlQ'fi'V^X’Xsnoo si a 8.<el its. 1o iwlai' &®vn&mo vXX

! t.i, -.

;, :*9,. X i vv?l X V: ->j X; v.XX X- ;:s :

- v-

•' t tfotf .

. • < ',

# ^ r vvsXXq^x xXXtv e; v&vvX Xv (t<vv v

< XX :'. *e?-; x aforsw'ijrHtf vk .I* t\o, o.'v »

®l;ivx-vj oriX cl-xcL: X.iirm evx xXb.X &mq mtl: ,S Guv : X X,:'. reW^Wtr.'Sv.^wc^v'. f-v. ••• .

'

...s""' XXv^v v - : r/:b. i -.tv, .V/. vxvXxrs .Mr; .

C&> waaq ot# a^xvl: £<>X al Bettlltfao •

X >X<J&T :tf r,obos} 3-ic/.: ax sb tfeat iXr^:uIy-> ex •# ,;X; Xv.-X

Wfc' XXfc'onXs ®nr ftaYtraoo asrf ifl&tfalist .3 tfarii -kdiX . ..:v., ** 11 t Cao X- .y *vxa req&q. alife? 1© v, .

. ., X: ., :v .',

. om*xM®oo vni teXvfviitv^ xvxt; .• x
bo If the experimenter- uses this technique for a certain

routing type of measurement® h© should apply It® implicitly

or expliclty® every time he makes that type of measuremento

After several explicit applications of this technique® h©

will probably b© able to perform the r(or u) test in all but

th® most doubtful cases without actually explicitly doing the

arithmetic® since he will have the critical value of r(or u)

in mlndo H© should not® however® reject outlier-® by th© v

test in some cases® and accept others just as badly deviating^

simply because h© did not apply th© test in these latter

q as© So

Co Both the r and u tests are based on th© assumption

that repeated measurements of th© same unknown follow -th©

normal frequency dlstrlbutlono If® in actual practice® the

distribution of repeated measurements is markedly different

from the normal curve® than the us© of these tests will lead

to different risks than originally Intended©

d 0 The use of th© 0 Q1 and points is arbitrary,© Th©

individual experimenter should use whatever levels of. significance are most appropriate

o

Qc, Suppose th© type of measurement is such that th®

suspected value is practically always the smallest in th©

set (for example® . a chemical analysis where some of tha

material may b© washed out)® or practically always th® large a to

Then as stated above® 1 percent of th© time a perfectly good
p.l

> & X ri - loasj ' ... c , erf; :

' " • e?< . U rl i re s< j i © .

, ...

afq i - jr&

! - ©d • Jt

it * : • -

*X©H' IQ £5X(X3V. Xaold-I^o 9Mri il.iw

•

’ ^Iv3« Jrf.vV .:-

rf*- "$* 'S

:

' i.aj$ a^itt© ,?qt>$2r ;?m - i-*: xattal aserftf 3 a erf Mb erf •• ^.r,

ndt^qHOEmeA ©rf^ tx<s baaad e*s« .ef&oit u hue *s oj0 rfd©g

rf -rf- • x/Vrf . - •. \L ',

;

, ...-; .... - i / *

•' i -. > 5Sf'" r '

: f o -' a;.; . rfr!« rforfd

1

. avx'v c obecma#nb •?./ nerf^ erfelrf $a£rfe'rfi

*

'

-

• rf;

;

- I.

'••'

.

• v

; ..

:l

. v v,?

; .u? . ; .

' ;rs • ^Crf •• ar. !

• •••;;.•

... -;

' •, ai; era"

-

• ;r-.v eg:-/

rforfe el d

,,rfe«s94urea@ife *5© ©q^rf ..

' ®0 to tmoe. &«i&rtw Uolmorio a . ^alqisse; .-.

®ffo s^bwXs lit ae*xq *r© 9 (^&ro berfas^

C'-" y^.;/£i&'.,*t®q & astirf oiiS \o ;

j

, a®©*xeq 1 * dvoJb betyerfa • > ix-- .rf

1
observation will b© rejected in case (a) of paragraph lc*> Q

Suppose however the type of measurement is such that th© sus~

pae ted value may be either the largest or the smallest*, In

this ease about 2 percent of the time a perfectly good observa®

tion will be rejected* The appropriate tabular point should

b© selected with this in mind*

fo Other tests for rejection of suspected values are

available [ljc However ^ the r and u tests have been selected

because of their ease of application.,

go The whole question of rejection of suspected values

is a difficult one*, and has given rise to quit© a bit of

discussion* [See references* 3 If the scientist or engineer

at the National Bureau of Standards is In doubt about any

assumption* procedure * or conclusion involved in a specific

practical problem* we suggest that he consult the Statistical

Engineering Laboratory (11*33

o
i

: | a

'

. ,(«.) .. i , t f , £: £, ....

.

. ..

; ^ •: dr' e ,

' • -; v'-.’C"C . -ff.-CJ.: I

. J . .-.' v

% 0&’> .'v^OTT^oR

.

Xo*j

v

/•
REFERENCES

rij Dixon® W. Jog Analysis of extreme values® Annals of M&tho

State Volo 21® Noo 4* WOo

[2] Dixon® Wo Jog Ration involving extreme values® Annals of

K&tho Stato Volo 22® No® 1® 1951°

13 J Grubbs® Po E©® Sample criteria for testing outlying

observations® Annals of Hath© Stat© Volo 21® No© 1®

March 195^

©

ikl Nair® K© R©® Th© distribution of the extreme deviate

from the sample mean end its studentized form® Bieiactrika®

Volo 3$d Parts 1 and II® May 1948

©

[53 Pearson® E© S© and Sekar® C© Th© efficiency of

statistical tools and a criterion for the rejection

of outlying observations® Blometrika, Vol© XXVIII 0 Parts III and IV® Dee© 1936©

[6] Rider® P© R©® Criteria for rejection of observations®

Washington University Sfcudi©s®*N©%? Series® Science and

Technology » No© 8® Oct© 1933®
.V::.3OTT

. © ».*G , c'.;. ; v

f ,, ole'.' ,, o, L? •.

• "SOI •

• •; > Ji

$

o » ir£

o

.

: M 1c? X- fffii'!, 0 k rxc 1. < &-? iyss s. rf c

atfilvob oxse*x{fs© ©r& lo ao&jJd&tfeJ£& ««{X '

o&.X$X r: c;lj ort- " irf-r.jx .'gt o I vv

iarvc-.x'o . a«xo^BV«iG f.-;do

B>3oXerMo©!]

I
TABLE I

PpCp > R) ® S

T

o Oovn 0 O 0ai 0 ,05 ,10 ©20 o30 0I4.O

3 [ o99U o988 •976 •941 0S66 © ?8l 0684 o59l

4 S ,926 .889 o8i;6 .765 .679 ©560 .471 -394

p 1 o821 ©?80 =729 «642 .55? .451 o373 o308

6 ; cTli-O „698 ©644 .560 482 *>386 .316 ©261

7 088O .637 cS86 .507 =434 ©344 o28l o230

8 06 31). .590 =543 <468 ©399 =314 .255 ©208

9 o598 o555 o5lo .437 .370 o290 =234 oi9i

10 „568 =527 =483 .412 .349 o2?3 o2X^ ol?8

11 ©542 o502 oi|60 o392 =332 =259 ©208 0X88

1-2 ©522 <482 ollljl =376 ©3X8 =247 ol97 0X60

13 .503 olj.65 ©425 =361 .305 o23? = 188 =153

1.4 oi|,88 ol&Q 0i4.ll .349 .294 o228 0X8X 0X4* /- 15' «ws 0^.36 =399 =338 0285 O 220 .175 eXi|l

16 4)63 oi|26 ©388 .329 o277 o213 oX69 oX36

17 =452 0 1|X6 =379 ,o320 .269 o20? 0X65 oX32

18 ©44^ =407 o3?0 =313 o263 o202 o!6Q oX28

19 =433 =398 =363 c306 c-256 ol97 =157 =125

20 ©4.25 0 39X • 356 o300 .252 ol93 .153 oX22

21 oi|l8 =384 .350 .295 .247 .169 ol50 oX19

22 0 4II =378 ©344 o290 & 2k2 oX85 =147 oil?

23' 0 4Olt o3?2 .338 o285 .238 ol82 0,114 =115

2k o399 =367 '=333 ,281 o234 .179 .,142 oll3

'25 •393 o362 ©329 o277 O 230 ol?6 =139 an

26 = 388 =357 ©324 o2?3 o22? •173 0137 ao9

• o384 =353 f,320 =269 o224 el?l =135 a08

.380 =349 ,316 ©286 o220 0 l68 «X33 oio6

.29 .376 .345 a 312 =263 0 2l8 ©166 ol3X 0X05

30 o372 o3Al o309 ,260 .215 =164 oI30 ©103

©50

1

0

Ch|

0 ol0 o 80 ©90 =95

©500 J4-09 ©3x6 o219 0 iXt4* =359

=324 =257 0X93 d30 o 065 o 033

.250 ©196 oXi+6 o097 © 64.6 cD23

©210 0X8,1* oX21 «0?9 c.038 o018

olSIi- ©1.43 ol 05 0O68 0O32 o0x6

0X66 ©las oG9k 0O6Q ©029 oOik

0X52 ©1X8 0O86 .055 ©026 o 013

.142 olio 0Q8O o 05l o025 o012

C x 33 ©103 oQJk O.04.8 ©023 C01X

=126 o097 ©070 o04.5 o022 oOXX

ol£0 o092 -o067 <.043 o021 cOXO

.115 oO§8 0O64. oOiiX ,020 oOio

oXU =085 ©062 ©040 c019 0 OXQ

0X07 o082 0O6O =039 iOl? c 009

olOJj. 0O8O o056 0038 oOia o 009

oXOX o0?8 .056 0038 © 0x8 - p009

o098- ,o?6 =055 .036 oGX7 o008

C.096 =c?4 .053 ©035 , 0X7 OO08

.094 o 0?2 0 052 =034 ©Gib o008

o092 o 0?X o05x 0033 0OI6 o008

o090 .069 o050 0033 0OX6 0 008

o 089 „068 =049 0032 7016 o008

o088 o067 oOliS 0O32 .015 O.008

0 086 ©066 =047 0O3X oOX5 o007

©085 ©065 .047 003X 0015 o007

o©84 c064 0 04.6 ©030 ,015 ©007

0O83 o063 c046 0030 .014 oOO?

o082 o062 =045 ©029 ©oik oOO?
•

'

; r : XIP- ''

;

'

P'PJ- L
)

TABLE II

Upper per cent points of the studentised extreme deviate

(xn-x)/sd or (x<-x1 )/ad

•

V ! a ® ,05 .. a « o01

\

3 4 5 6 ? 8 9 3 4 5 6 7 8 9

1 C

11

12

13

14

-l 5

16

17

18

19

20

t

,

^4

30

40

6 ©

2Q

ob

2 a 02 2o29 2c49 2 0 6j 2 ,75 2

1.99 2 0 26 2*44 2,58 2of0 2

lo'97 2o22 2Jj.O 2*54 2«65 2

lo95 2„2Q 2 o-38. 2o5i 2o62 2

3=o 93 2ol8 2«35 2 a 48 2.59 2

=85 2 o 93

=79 2o8?

,75 2o83

=71 2c79

=68 2 o ?6

2o?6 3^05 ,3*25' 3.39 '3.50 3.59 3o6?

2o?l 3o00 3ol9 3.33 3. 44 3*53 3.6i

2o6? £ o95 3.14 3.28 3.39 3o48 3o55

2,63; .2*91 3ol0 3,24 3 •34 3.43 3 .51

2160, .2 o 87 3o06 3 o 20 3,30 3.39 3®4?

1-92 2«16 2 Q 33 2o46 2 0 .t? 6 2 o65 2©73 2e5if 2,84 3 o02 3ol6 3 .27 3®.35 3c43 logo 2,14 2,31 2*44 2o|4 2o63 2o?0 2,55 2 0 8l 3,00 3*13 3„24 3*32 3,39'

loQg 2 © X 3 2 o30 2o42 2«>52 2 c 6l 2.68 2,52 2,79 2,9? 3,10 3,21 3,,2Q 3,36 I088 2ol2 So 28 2.41 2,51 2o59 2,66 2o50 2 C ?7 2,95 3,08 3,18 3,2? 3,34 1-87 2oXl 2o2? 2o39 2^49 2,58 2,65 2.49 2*75 2,92 3«06 3.16 3o2ii 3*31

log? 2ol0 2o26 2o38 2J18 2,,56 2,63 2*47 2o?3 2,91 3o04 3dk 3 C 22 3,29

le>o4 2 o 0? 2c23 2o35 2 0 44 2«52 2,59 2o43 2,68 2,85 2 C 9? 3®©7 3d5 3<,2

2

I082 ^o04 2c 20 2,31 2c40 2o48 2,55 2o38 2 e 62 2 0 ?9 2,91 3,01 3*08 3d5

I08O 2,02 2ol7 2*28 2,37 2 c-44 2,51 2*3^2*57 2.73 2.85 2,94 3o02 3 o 08

fo?8 lo99 2.14 2.25 2o33 2o4l 2,4? 2,30 2o92 2,68 2,79 2 0 88 2,95 3 C 01

}°27 ?

ol l 2o2i 2o 3® 2,43 2,25 2,43 2 062 2o73 2 082 B089 2o95 lo?4 1.94 2o08 2 ©l8 2o2? 2 0 33 2.39 2,22 2,43 2.5? 2,68 2*76 2,83 2 0 88
(j
THE NATIONAL BUREAU OF STANDARDS

Functions and Activities

The National Bureau of Standards is the principal agency of

the Federal Government for fundamental and applied research in physics, mathematics, chemistry, and engineering. Its activities rang©

from the determination of physical constants and properties of ma- terials, the development and maintenance of the national standards

of measurement in the physical sciences, and the development of

methods and instruments of measurement, to the development of special

devices for the military and civilian agencies of the Government.

The work includes basic and applied research, development, engineering, instrumentation, testing, evaluation, calibration services,

and various scientific and technical advisory services.. A major

portion of the NBS work is performed for other government agencies,

particularly the Department of Defense and the Atomic Energy Commission. The functions of the National Bureau of Standards are set

forth in the Act of Congress, March 3, 1901, as amended by Congress

in Public Law 619, 1950. The scope of activities is suggested in

the listing of divisions and sections on the inside of the front

cover.

Reports and Publications

The results of the Bureau’s work take the form of either actual

equipment and devices or published papers and reports. Reports are issued to the sponsoring agency of a particular project or program.

Published papers appear either in the Bureau’s own series of publications or in the journals of professional and scientific societies.

The Bureau itself publishes three monthly periodicals, available

from the Government Printing Office : the Journal of Research, which

presents complete papers reporting technical investigations; the

Technical News Bulletin, which presents summary and preliminary re- ports on work Tn progress; and Basic Radio Propagation Predictions,

which provides data for determining the best frequencies to use for

radio communications throughout the world. There are also five

series of monperiodieal publications: the Applied Mathematics Series, Circulars, Handbooks, Building Materials and Structures Reports, and Miscellaneous Publications.

Information on the Bureau’s publications can be found in

NBS Circular 460, Publications of the National Bureau of Standards

($1.00). Information on calibration services and fees can be found

in NBS Circular 483, Testing by the National Bureau of Standards

(25 cents). Both are available from the Government Printing Office

.

Inquiries regarding the Bureau’s reports and publications should be

addressed to the Office of Scientific Publications, National Bureau

of Standards, Washington 25, D. C.
(( I