Calculations from EPA 1664 guidance online at:

http://www.epa.gov/waterscience/methods/method/oil/1664guide.pdf

performed by Tyler Martin, OSS

RMSD Analysis

THIS SET DOES NOT INCLUDE THE 'PHASE 2A' DATA, WHICH WAS PERFORMED AFTER THE COMPLETION OF THE SINGLE LAB STUDY! SEE WORKSHEET "RMSD Analysis - extended"

EPA NOTE: all of s 

1664 Result Concentration

Sample Number Date Tested (mg/L) (mg/L)

1 a 4.6.2009 17.2 st. dev. 22.6 st. dev. sample #

POTW 1 b 4.7.2009 16.3 3.9% 19.5 10.0% 1

1 c 4.8.2009 17.6 23.7 2

3

2 a 4.6.2009 29.8 25.6 4

POTW 2 b 4.7.2009 29.7 4.8% 29.9 7.9% 5

2 c 4.8.2009 27.3 28.6 6

7

3 a 4.6.2009 20.3 21.7 8

POTW 3 b 4.7.2009 21.2 2.5% 14.8 19.9% 9

3 c 4.8.2009 21.3 21.2 10

11

4 a 4.6.2009 23.2 23.2 12

POTW 4 b 4.7.2009 22.6 1.5% 20.8 9.3% 13

4 c 4.8.2009 22.7 25.1 14

5 a 4.6.2009 30.9 19.6

POTW 5 b 4.7.2009 27.4 7.5% 19.5 16.9%

5 c 4.8.2009 31.6 25.9

6 a 4.7.2009 28.1 28.8

POTW 6 b 4.8.2009 30.1 5.2% 25.7 12.1%

6 c 4.9.2009 31.2 22.6

7 a 4.7.2009 0.9 1.1

POTW 7 b 4.8.2009 1.5 1.3

7 c 4.9.2009 1.4 0.9

8I 36.1 31.3

8B 30.2 29.2

ASTM WK 23240 Results

1. compile data
8C 29.7 24.7

POTW 8D 31.3 8.2% 30.7 10.1%

8E 35.2 34.1

8F 29.9 29.7

8G 32.8 27.4

9A 77.4 76.1

9B 80.9 73.3

Gunsmith 9C 78.6 75.3

9D 79.3 1.6% 78.3 9.3%

9E 78.1 74.5

9F 80.0 91.2

9G 78.1 89.4

10A 22.4 21.0

Auto Garage 10B 21.0 5.7% 19.9 6.7%

10C 20.0 18.4

11A 108.1 89.5

Dairy 11B 110.8 1.6% 91.4 8.2%

11C 107.5 103.8

12A 89.1 74.0

Machine Shop 12B 88.7 0.9% 69.9 19.1%

12C 90.3 98.4

13A 7.2 5.8

Auto Salvage yar 13B 5.9 10.0% 4.8 10.8%

13C 6.5 5.8

14A 55.0 50.5

Fish Processor 14B 53.0 1.9% 54.1 3.8%

14C 54.0 50.9

Average RSD (% st. dev.)= 4.3% 11.1%

[We would also like to note that this is an unusually small average RSD for 1664, wherein a single operator RSD might be more likely expected to be near 10%. 

This precision of 1664 results affects the method comparability statistics, such that a more normal RSD in 1664, such as 10%, would have made it less likely the results of the two methods would not be comparable.]
Second Round - check for outliers

THIS SET DOES NOT INCLUDE THE 'PHASE 2A' DATA, WHICH WAS PERFORMED AFTER THE COMPLETION OF THE SINGLE LAB STUDY! SEE WORKSHEET "RMSD Analysis - extended set"

EPA NOTE: all of s 

1664 Result Concentration

Sample Number Date Tested (mg/L) (mg/L)

1 a 4.6.2009 17.2 22.6 sample #

POTW 1 b 4.7.2009 16.3 19.5 1

1 c 4.8.2009 17.6 23.7 2

3

2 a 4.6.2009 29.8 25.6 4

POTW 2 b 4.7.2009 29.7 29.9 5

2 c 4.8.2009 27.3 28.6 6

7

3 a 4.6.2009 20.3 21.7 8

POTW 3 b 4.7.2009 21.2 14.8 9

3 c 4.8.2009 21.3 21.2 10

11

4 a 4.6.2009 23.2 23.2 12

POTW 4 b 4.7.2009 22.6 20.8 13

4 c 4.8.2009 22.7 25.1 14

5 a 4.6.2009 30.9 19.6

POTW 5 b 4.7.2009 27.4 19.5

5 c 4.8.2009 31.6 25.9

6 a 4.7.2009 28.1 28.8

POTW 6 b 4.8.2009 30.1 25.7

6 c 4.9.2009 31.2 22.6

7 a 4.7.2009 0.9 1.1

POTW 7 b 4.8.2009 1.5 1.3

7 c 4.9.2009 1.4 0.9

8I 36.1 31.3

8B 30.2 29.2

8C 29.7 24.7

POTW 8D 31.3 30.7

8E 35.2 34.1

ASTM WK 23240 Results

1. compile data
8F 29.9 29.7

8G 32.8 27.4

9A 77.4 76.1

9B 80.9 73.3

Gunsmith 9C 78.6 75.3

9D 79.3 78.3

9E 78.1 74.5

9F 80.0 91.2

9G 78.1 89.4

10A 22.4 21.0

Auto Garage 10B 21.0 19.9

10C 20.0 18.4

11A 108.1 89.5

Dairy 11B 110.8 91.4

11C 107.5 103.8

12A 89.1 74.0

Machine Shop 12B 88.7 69.9

12C 90.3 98.4

13A 7.2 5.8

Auto Salvage yar 13B 5.9 4.8

13C 6.5 5.8

14A 55.0 50.5

Fish Processor 14B 53.0 54.1

14C 54.0 50.9

Third Round - remove outlier, check for more
THIS SET DOES NOT INCLUDE THE 'PHASE 2A' DATA, WHICH WAS PERFORMED AFTER THE COMPLETION OF THE SINGLE LAB STUDY! SEE WORKSHEET "RMSD Analysis - extended"

EPA NOTE: all of s 

1664 Result Concentration

Sample Number Date Tested (mg/L) (mg/L)

1 a 4.6.2009 17.2 22.6 sample #

POTW 1 b 4.7.2009 16.3 19.5 1

1 c 4.8.2009 17.6 23.7 2

3

2 a 4.6.2009 29.8 25.6 4

POTW 2 b 4.7.2009 29.7 29.9 5

2 c 4.8.2009 27.3 28.6 6

7

3 a 4.6.2009 20.3 21.7 8

POTW 3 b 4.7.2009 21.2 14.8 9

3 c 4.8.2009 21.3 21.2 10

11

4 a 4.6.2009 23.2 23.2 12

POTW 4 b 4.7.2009 22.6 20.8 13

4 c 4.8.2009 22.7 25.1 14

5 a 4.6.2009 30.9 19.6

POTW 5 b 4.7.2009 27.4 19.5

5 c 4.8.2009 31.6 25.9

6 a 4.7.2009 28.1 28.8

POTW 6 b 4.8.2009 30.1 25.7

6 c 4.9.2009 31.2 22.6

7 a 4.7.2009 0.9 1.1

POTW 7 b 4.8.2009 1.5 1.3

7 c 4.9.2009 1.4 0.9

8I 36.1 31.3

8B 30.2 29.2

8C 29.7 24.7

POTW 8D 31.3 30.7

8E 35.2 34.1

8F 29.9 29.7

8G 32.8 27.4

9A 77.4 76.1

9B 80.9 73.3

ASTM WK 23240 Results

1. compile data
Gunsmith 9C 78.6 75.3

9D 79.3 78.3

9E 78.1 74.5

9F 80.0 91.2

9G 78.1 89.4

10A 22.4 21.0

Auto Garage 10B 21.0 19.9

10C 20.0 18.4

11A 108.1 89.5

Dairy 11B 110.8 91.4

11C 107.5 103.8

12A 89.1 74.0

Machine Shop 12B 88.7 69.9

12C 90.3 98.4

13A 7.2 5.8

Auto Salvage yar 13B 5.9 4.8

13C 6.5 5.8

14A 55.0 50.5

Fish Processor 14B 53.0 54.1

14C 54.0 50.9

Fourth Round - check comparability after removing outliers

THIS SET DOES NOT INCLUDE THE 'PHASE 2A' DATA, WHICH WAS PERFORMED AFTER THE COMPLETION OF THE SINGLE LAB STUDY! SEE WORKSHEET "RMSD Analysis - extended"

EPA NOTE: all of s 

1664 Result Concentration

Sample Number Date Tested (mg/L) (mg/L)

1 a 4.6.2009 17.2 st. dev. 22.6 st. dev. sample #

POTW 1 b 4.7.2009 16.3 3.9% 19.5 10.0% 1

1 c 4.8.2009 17.6 23.7 2

3

2 a 4.6.2009 29.8 25.6 4

POTW 2 b 4.7.2009 29.7 4.8% 29.9 7.9% 5

2 c 4.8.2009 27.3 28.6 6

ASTM WK 23240 Results

1. compile data
7

3a 4.6.2009 20.3 21.78

POTW3b 4.7.2009 21.2 2.5% 14.8 19.9%9

3c 4.8.2009 21.3 21.2 10

11

4a 4.6.2009 23.2 23.2 12

POTW4b 4.7.2009 22.6 1.5% 20.8 9.3% 13

4c 4.8.2009 22.7 25.1 14

5a 4.6.2009 30.9 19.6

POTW5b 4.7.2009 27.4 7.5% 19.5 16.9%

5c 4.8.2009 31.6 25.9

6a 4.7.2009 28.1 28.8

POTW6b 4.8.2009 30.1 5.2% 25.7 12.1%

6c 4.9.2009 31.2 22.6

7a 4.7.2009 0.9 1.1

POTW7b 4.8.2009 1.5 1.3

7c 4.9.2009 1.4 0.9

8I 36.1 31.3

8B 30.2 29.2

8C 29.7 24.7

POTW 8D 31.3 8.2% 30.7 10.1%

8E 35.2 34.1

8F 29.9 29.7

8G 32.8 27.4

9A 77.4 76.1

9B 80.9 73.3

Gunsmith 9C 78.6 75.3

9D 79.3 1.6% 78.3 9.3%

9E 78.1 74.5

9F 80.0 91.2

9G 78.1 89.4

10A 22.4 21.0

Auto Garage 10B 21.0 5.7% 19.9 6.7%

10C 20.0 18.4

11A 108.1 89.5

 p
Dairy 11B 110.8 1.6% 91.4 8.2%

11C 107.5 103.8

12A 89.1 74.0

Machine Shop 12B 88.7 0.9% 69.9 19.1%

12C 90.3 98.4

13A 7.2 5.8

Auto Salvage yar 13B 5.9 10.0% 4.8 10.8%

13C 6.5 5.8

14A 55.0 50.5

Fish Processor 14B 53.0 1.9% 54.1 3.8%

14C 54.0 50.9
 set 7 and replicate 2 of D7575 for sample #13 were below the ML, and so are not included in comparability calculations per the EPA guidance

Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3

17.18 16.27 17.57 22.55 19.48 23.70 2.844 2.789 2.866

29.81 29.7 27.33 25.58 29.92 28.59 3.395 3.391 3.308

20.3 21.19 21.25 21.69 14.84 21.25 3.011 3.054 3.056

23.24 22.62 22.72 23.22 20.79 25.05 3.146 3.119 3.123

30.92 27.43 31.63 19.63 19.51 25.93 3.431 3.312 3.454

28.14 30.1 31.19 28.80 25.74 22.60 3.337 3.405 3.440

36.12 30.19 29.71 31.26 35.2 29.85 32.84 31.25544 29.1876 24.67855 30.68104 34.09872 29.66148 27.40696 3.587 3.408 3.391 3.442 3.561 3.396

77.35 80.89 78.63 79.34 78.07 80.00 78.1 76.11613 73.27284 75.31197 78.32757 74.45037 91.2085 89.44222 4.348 4.393 4.365 4.374 4.358 4.382

22.4 21 20 21.03111 19.86795 18.37451 3.109 3.045 2.996

108.1 110.78 107.52 89.52838 91.39518 103.831 4.683 4.708 4.678

89.11 88.7 90.3 74.0052 69.92696 98.44595 4.490 4.485 4.503

7.21 5.9 6.54 5.838212 5.823852 1.975 1.775 1.878

55 53 54 50.46914 54.10222 50.94302 4.007 3.970 3.989

EPA 1664 Result ASTM WK 23240 EPA 1664 Result

Log Transforme

2. log transform

 set 7 and replicate 2 of D7575 for sample #13 were below the ML, and so are not included in comparability calculations per the EPA guidance

Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3

17.18 16.27 17.57 22.55 19.48 23.70 2.844 2.789 2.866

29.81 29.7 27.33 25.58 29.92 28.59 3.395 3.391 3.308

20.3 21.19 21.25 21.69 14.84 21.25 3.011 3.054 3.056

23.24 22.62 22.72 23.22 20.79 25.05 3.146 3.119 3.123

30.92 27.43 31.63 19.63 19.51 25.93 3.431 3.312 3.454

28.14 30.1 31.19 28.80 25.74 22.60 3.337 3.405 3.440

36.12 30.19 29.71 31.26 35.2 29.85 32.84 31.25544 29.1876 24.67855 30.68104 34.09872 29.66148 27.40696 3.587 3.408 3.391 3.442 3.561 3.396

77.35 80.89 78.63 79.34 78.07 80.00 78.1 76.11613 73.27284 75.31197 78.32757 74.45037 91.2085 89.44222 4.348 4.393 4.365 4.374 4.358 4.382

22.4 21 20 21.03111 19.86795 18.37451 3.109 3.045 2.996

108.1 110.78 107.52 89.52838 91.39518 103.831 4.683 4.708 4.678

89.11 88.7 90.3 74.0052 69.92696 98.44595 4.490 4.485 4.503

7.21 5.9 6.54 5.838212 5.823852 1.975 1.775 1.878

55 53 54 50.46914 54.10222 50.94302 4.007 3.970 3.989

Log Transforme

EPA 1664 Result ASTM WK 23240 EPA 1664 Result

2. log transform

 set 7 and replicate 2 of D7575 for sample #13 were below the ML, and so are not included in comparability calculations per the EPA guidance

Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3

17.18 16.27 17.57 22.55 19.48 23.70 2.844 2.789 2.866

29.81 29.7 27.33 25.58 29.92 28.59 3.395 3.391 3.308

20.3 21.19 21.25 21.69 14.84 21.25 3.011 3.054 3.056

23.24 22.62 22.72 23.22 20.79 25.05 3.146 3.119 3.123

30.92 27.43 31.63 19.63 19.51 25.93 3.431 3.312 3.454

28.14 30.1 31.19 28.80 25.74 22.60 3.337 3.405 3.440

36.12 30.19 29.71 31.26 35.2 29.85 32.84 31.25544 29.1876 24.67855 30.68104 34.09872 29.66148 27.40696 3.587 3.408 3.391 3.442 3.561 3.396

77.35 80.89 78.63 79.34 78.07 80.00 78.1 76.11613 73.27284 75.31197 78.32757 74.45037 91.2085 89.44222 4.348 4.393 4.365 4.374 4.358 4.382

22.4 21 20 21.03111 19.86795 18.37451 3.109 3.045 2.996

108.1 110.78 107.52 89.52838 91.39518 103.831 4.683 4.708 4.678

89.11 88.7 90.3 74.0052 69.92696 98.44595 4.490 4.485 4.503

7.21 5.9 6.54 5.838212 5.823852 1.975 1.775 1.878

55 53 54 50.46914 54.10222 50.94302 4.007 3.970 3.989

Log Transforme

EPA 1664 Result ASTM WK 23240 EPA 1664 Result

2. log transform
 set 7 and replicate 2 of D7575 for sample #13 were below the ML, and so are not included in comparability calculations per the EPA guidance

Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3 Rep. 1 Rep. 2 Rep. 3

17.18 16.27 17.57 22.55 19.48 23.70 2.844 2.789 2.866

29.81 29.7 27.33 25.58 29.92 28.59 3.395 3.391 3.308

20.3 21.19 21.25 21.69 14.84 21.25 3.011 3.054 3.056

23.24 22.62 22.72 23.22 20.79 25.05 3.146 3.119 3.123

30.92 27.43 31.63 19.63 19.51 25.93 3.431 3.312 3.454

28.14 30.1 31.19 28.80 25.74 22.60 3.337 3.405 3.440

EPA 1664 Result ASTM WK 23240 EPA 1664 Result

Log Transforme

2. log transform
36.12 30.19 29.71 31.26 35.2 29.85 32.84 31.25544 29.1876 24.67855 30.68104 34.09872 29.66148 27.40696 3.587 3.408 3.391 3.442 3.561 3.396

77.35 80.89 78.63 79.34 78.07 80.00 78.1 76.11613 73.27284 75.31197 78.32757 74.45037 91.2085 89.44222 4.348 4.393 4.365 4.374 4.358 4.382

22.4 21 20 21.03111 19.86795 18.37451 3.109 3.045 2.996

108.1 110.78 107.52 89.52838 91.39518 103.831 4.683 4.708 4.678

89.11 88.7 90.3 74.0052 69.92696 98.44595 4.490 4.485 4.503

7.21 5.9 6.54 5.838212 5.823852 1.975 1.775 1.878

55 53 54 50.46914 54.10222 50.94302 4.007 3.970 3.989

Mean of Sample deviation

Rep. 1 Rep. 2 Rep. 3

3.116 2.969 3.166 2.833 3.084 2.958 0.251

3.242 3.399 3.353 3.365 3.331 3.348 -0.034

3.077 2.698 3.056 3.040 2.944 2.992 -0.097

3.145 3.034 3.221 3.129 3.133 3.131 0.004

2.977 2.971 3.255 3.399 3.068 3.233 -0.331

3.360 3.248 3.118 3.394 3.242 3.318 -0.152

3.492 3.442 3.374 3.206 3.424 3.529 3.390 3.311 3.468 3.382 3.425 -0.086

4.358 4.332 4.294 4.322 4.361 4.310 4.513 4.494 4.368 4.375 4.372 0.007

3.046 2.989 2.911 3.050 2.982 3.016 -0.068

4.495 4.515 4.643 4.689 4.551 4.620 -0.139

4.304 4.247 4.590 4.493 4.380 4.437 -0.112

1.764 1.762 1.876 1.763 1.820 -0.113

3.921 3.991 3.931 3.989 3.948 3.968 -0.041

Mm1= 3.469 Mm2= 3.399

Mo= 3.434

RMSD=

ASTM WK 23240 EPA 1664 Result ASTM WK 23240

ed Triplicate Means of Log-Transformed

3. triplicate means

4. deviation

3. triplicate means

-0.0700

0.1293

OUTLIER DETERMINATION. If the valu 

the value in the Grubbs table for the n 

2.480

2.021

mean deviation

Rep. 1 Rep. 2 Rep. 3

3.116 2.969 3.166 2.833 3.084 2.958 0.251

3.242 3.399 3.353 3.365 3.331 3.348 -0.034

3.077 2.698 3.056 3.040 2.944 2.992 -0.097

3.145 3.034 3.221 3.129 3.133 3.131 0.004

2.977 2.971 3.255 3.399 3.068 3.233 -0.331

3.360 3.248 3.118 3.394 3.242 3.318 -0.152

3.492 3.442 3.374 3.206 3.424 3.529 3.390 3.311 3.468 3.382 3.425 -0.086

4.358 4.332 4.294 4.322 4.361 4.310 4.513 4.494 4.368 4.375 4.372 0.007

3.046 2.989 2.911 3.050 2.982 3.016 -0.068

4.495 4.515 4.643 4.689 4.551 4.620 -0.139

4.304 4.247 4.590 4.493 4.380 4.437 -0.112

1.764 1.762 1.876 1.763 1.820 -0.113

3.921 3.991 3.931 3.989 3.948 3.968 -0.041

ed Triplicate Means of Log-Transformed

ASTM WK 23240 EPA 1664 Result ASTM WK 23240

3. triplicate means

4. perform Grubbs calculations on the difference 

between the log-transformed triplicate means
-0.0968

0.0901

OUTLIER DETERMINATION. If the valu 

the value in the Grubbs table for the n 

1.151
2.604

mean deviation

Rep. 1 Rep. 2 Rep. 3

3.116 2.969 3.166 2.833 3.084 2.958

3.242 3.399 3.353 3.365 3.331 3.348 -0.034

3.077 2.698 3.056 3.040 2.944 2.992 -0.097

3.145 3.034 3.221 3.129 3.133 3.131 0.004

2.977 2.971 3.255 3.399 3.068 3.233 -0.331

3.360 3.248 3.118 3.394 3.242 3.318 -0.152

3.492 3.442 3.374 3.206 3.424 3.529 3.390 3.311 3.468 3.382 3.425 -0.086

4.358 4.332 4.294 4.322 4.361 4.310 4.513 4.494 4.368 4.375 4.372 0.007

3.046 2.989 2.911 3.050 2.982 3.016 -0.068

4.495 4.515 4.643 4.689 4.551 4.620 -0.139

4.304 4.247 4.590 4.493 4.380 4.437 -0.112

1.764 1.762 1.876 1.763 1.820 -0.113

3.921 3.991 3.931 3.989 3.948 3.968 -0.041

ed Triplicate Means of Log-Transformed

ASTM WK 23240 EPA 1664 Result ASTM WK 23240

3. triplicate means

4. perform Grubbs calculations on the difference 

between the log-transformed triplicate means

Data points removed as an outlier Data points remove
Mean of Sample deviation

Rep. 1 Rep. 2 Rep. 3

3.116 2.969 3.166

3.242 3.399 3.353 3.365 3.331 3.348 -0.034

3.077 2.698 3.056 3.040 2.944 2.992 -0.097

3.145 3.034 3.221 3.129 3.133 3.131 0.004

2.977 2.971 3.255

3.360 3.248 3.118 3.394 3.242 3.318 -0.152

ASTM WK 23240 EPA 1664 Result ASTM WK 23240

ed Triplicate Means of Log-Transformed

3. triplicate means

4. deviation

3. triplicate means

Data points rem

Data points rem

Data points removed as an outlier

Data points removed as an outlier
3.492 3.442 3.374 3.206 3.424 3.529 3.390 3.311 3.468 3.382 3.425 -0.086

4.358 4.332 4.294 4.322 4.361 4.310 4.513 4.494 4.368 4.375 4.372 0.007

3.046 2.989 2.911 3.050 2.982 3.016 -0.068

4.495 4.515 4.643 4.689 4.551 4.620 -0.139

4.304 4.247 4.590 4.493 4.380 4.437 -0.112

1.764 1.762 1.876 1.763 1.820 -0.113

3.921 3.991 3.931 3.989 3.948 3.968 -0.041

Mm1= 3.533 Mm2= 3.457

Mo= 3.495

RMSD=

deviation sqared

(M1j - M2j)^2

0.063 0.040 0.102 0.001562

0.001 0.049 0.081 0.002412

0.009 0.026 0.213 0.000657

0.000 0.015 0.094 0.000210

0.110 0.077 0.163 0.005859

0.023 0.052 0.121 0.002731

0.007 0.080 0.103 0.006445

0.000 0.016 0.090 0.000244

0.005 0.057 0.068 0.003232

0.019 0.016 0.080 0.000253

0.013 0.009 0.183 0.000086

0.013 0.100 0.002 0.010054

0.002 0.019 0.038 0.000343

SDm1= 0.043 SDm2= 0.103 Mean Square Error (MSE)= 0.00812

RMSE 0.090

SDo= 0.073

0.143 determine the F critical value

using the parameters below to determin 

and the Excel function FINV to get F crit

for Fcrit:

i= 2 # test meth

1st degree freedom = df1 = j= 13 # samples

k= 3 # times eac 

2nd degree freedom = df2 = i*j*(k-1) 52

alpha = 5% (confidence 

EPA 1664 Result ASTM WK 23240 EPA 1664 Result 

Triplicate Standard Deviations of Log-Transformed Triplicate Standard Deviations Squ

6. 

5. 

7. 

8. 

Determining 'k' is the biggest question to 

me: I believe it must be an interger 

value, so I used 'three' for the number of 

times each sample tested
F,alpha,df1,df2= 1.913

RMSDmax= 0.102 formula from the referenced documents

RMSD/RMSDmax = 1.4011 the RMSD is greater than the maximum RMSD allowed,

indicating the methods do not give the same results

9. 

10. 
average

standard deviation

ue listed below for either max or min Grubbs is higher than 

number of samples, the max (or min) value below is an outlier

max Grubbs

min Grubbs

maximum

minimum

From "Grubbs Table" sheet

95% confidence, 13 obs: G,crit=2.331

Since the max G value is greater than this, the corresponding data 

point is an outlier.

(In fact, the point would be an outlier even at 97.5% confidence:

97.5% confidence, 13 obs: G,crit = 2.462)

The next step is to remove this value and re-calculate. 5.
average

standard deviation

ue listed below for either max or min Grubbs is higher than 

number of samples, the max (or min) value below is an outlier

max Grubbs
min Grubbs

5.

From "Grubbs Table" sheet

95% confidence, 12 obs: G,crit=2.285

Since the 'min Grubbs' value is greater than this, the corresponding 

data point is an outlier.

(In fact, the point would be an outlier even at 99% confidence:

99% confidence, 12 obs: G,crit = 2.550)

The next step is to now remove this value and re-calculate.

ed as an outlier
deviation sqared

(M1j - M2j)^2

0.001 0.049 0.081 0.002412

0.009 0.026 0.213 0.000657

0.000 0.015 0.094 0.000210

0.023 0.052 0.121 0.002731

EPA 1664 Result ASTM WK 23240 EPA 1664 Result 

Triplicate Standard Deviations of Log-Transformed Triplicate Standard Deviations Squ

6. 

7 

moved as an outlier

moved as an outlier

Data points removed as an outlier

Data points removed as an outlier

Data points removed as an outlier

Data points removed as an outlier
0.007 0.080 0.103 0.006445

0.000 0.016 0.090 0.000244

0.005 0.057 0.068 0.003232

0.019 0.016 0.080 0.000253

0.013 0.009 0.183 0.000086

0.013 0.100 0.002 0.010054

0.002 0.019 0.038 0.000343

SDm1= 0.040 SDm2= 0.098 Mean Square Error (MSE)= 0.00759

RMSE 0.087

SDo= 0.069

0.091 determine the F critical value

using the parameters below to determin 

and the Excel function FINV to get F crit

for Fcrit:

i= 2 # test meth

1st degree freedom = df1 = j= 11 # samples

k= 3 # times eac 

2nd degree freedom = df2 = i*j*(k-1) 44

alpha = 0.05 (confidence 

F,alpha,df1,df2= 2.014

RMSDmax= 0.101 formula from the referenced documents

RMSD/RMSDmax = 0.905 Conclusion:

The RMSD is less than the maximum allowed RMSD,

indicating the results of the two methods are indistinguish

Two outliers have been removed from 13 data points, which wo 

5. 

7. 

8. 

Note that the number of samples has 

changed, so df1 and df2 change and 

F,crit changes

9. 

10. 

MSINT = 1/2 sum over I and j of [ (Mij - Mmi - Msj +Mo)^2]

0.010398 0.02570 0.02570

0.006484 0.00033 0.00033

0.045505 0.00018 0.00018

0.008802 0.00137 0.00137

0.026418 0.01707 0.01707

0.014735 0.00167 0.00167

0.010540 0.00006 0.00006

0.008127 0.00148 0.00148

0.004596 0.00000 0.00000

0.006444 0.00117 0.00117

0.033609 0.00045 0.00045

0.000003 0.00046 0.00046

0.001423 0.00021 0.00021

MSINT= 0.050159

Fcalc = 6.175636

ne the degrees of freedom Since Fcalc > 2.45 there is significant interaction between method and sample

i.e. there is not a consistent bias between methods. Instead, the bias varies from sample to sample.

Thus no conversion factor can be calculated.

hods

ch sample tested However, suspected outliers are present.

The next step is to check for outliers and repeat the process

e = 1-alpha = 95%)

ASTM WK 23240 EPA 1664 Result ASTM WK 23240

uared MSINT Calculation




0.006484

0.045505

0.008802

0.014735

ASTM WK 23240

uared
0.010540

0.008127

0.004596

0.006444

0.033609

0.000003

0.001423

ne the degrees of freedom

hods

ch sample tested

e = 1-alpha = 95%)

hable.

ould seem to be a reasonable number of outliers.