Review of Survey 

Data to Support 

Revisions to DOE'S 

Dishwasher Test 

Procedure 

Addendum 

U.S. Department of Energy 

Energy Efficiency and 

Renewable Energy 

Office of Building Technology, 

State and Community Programs 

1000 Independence Avenue, SW 

Washington, DC 20585 

Date: March 5, 2002 

Arthur D. Little, Inc. 

Acorn Park 

Cambridge, MA 02140 

 EE-RM-TP-99-500 COMMENT 26

Review of Survey 

Data to Support 

Revisions to DOE's 

Dishwasher Test 

Procedure 

Addendum 

U.S. Department of Energy 

Energy Efficiency and 

Renewable Energy 

Office of Building Technology, 

State and Community Programs 

1000 Independence Avenue, SVV 

Washington, DC 20585 

Date: March 5, 2002 

Arthur D. Little, Inc. 

Acorn Park 

Cambridge, MA 02140 
Addendum 

The prime objective of this addendum is to expand on and readdress two areas of the report, "Review 

of Suwey Data to Support Revisions to DOE'S Dishwasher Test Procedure", prepared for the 

Department of Energy's Office of Building Technology, State and Community Programs and issued 

December 18,200 1. These areas are: 

1. To provide more detail on the analysis in the section, Quantitative Amount of Soil in a 

Dishwasher Load, 

a) Demonstrating the robustness of the prime source of data for the analysis - survey C, 

b) Introducing and analyzing additional data from survey C, and 

c) Refocusing on survey C as the primary basis for determining the portion of the 

AHAM DW-1 soil load that could be used to represent light, medium, and heavy soil 

levels; and 

2. To clarify the rationale for the recommended 200 to 233 average-use cycles per year for a 

dishwasher. 

Background on the Quantitative Amount of Soil in a Dishwasher Load 

This section of the report drew from three surveys identified as Survey C, Survey D, and Survey F. 

Manufacturers and energy interest groups provided these surveys. As in the report, a key requirement 

of this addendum is to maintain the confidentiality of the industry survey sources. To this end no 

source is identified and only general information on the data source is provided. Table 1 gives a 

comparative overview of the scope and size of these three surveys. 

Table I - Overview of scope and size of surveys 

Surveys C, D, and F, cumulatively representing over 2,000 U.S. households, provide text andfor 

graphic descriptions to link their categories of the level of soil to the three soil levels - Light, 

Medium, Heavy- and to quantitative amounts of soil for each level. Referencing ANSIIAHAM 

Standard DW-1 for the types of soil and, most importantly, the amounts of soil is essential in relating 

each of the three soil levels to an amount of soil. 

The relationship to the amounts of soil representative of the soil levels is derived differently for each 

survey, but for each survey the relationship is made in terms of the number of place settings in the 

DOE test load that are soiled according to ANSIIAHAM Standard DW-1. The results of the analysis 

to determine these relationships were presented in the report issued on December 18, 2001, and 

reflect an aggregation of the data from surveys C, D, and F. 

 EE-RM-TP-99-500 COMMENT 26

Addendum 

The prime objective of this addendum is to expand on and readdress two areas of the report, "Review 

of Survey Data to Support Revisions to DOE's Dishwasher Test Procedure", prepared for the 

Department of Energy's Office of Building Technology, State and Community Programs and issued 

December 18, 2001. These areas are: 

1. To provide more detail on the analysis in the section, Quantitative Amount of Soil in a 

Dishwasher Load, 

a) Demonstrating the robustness of the prime source of data for the analysis - survey C, 

b) Introducing and analyzing additional data from survey C, and 

c) Refocusing on survey C as the primary basis for determining the portion of the 

AHAM DW-I soil load that could be used to represent light, medium, and heavy soil 

levels; and 

2. To clarify the rationale for the recommended 200 to 233 average-use cycles per year for a 

dishwasher. 

Background on the Quantitative Amount of Soil in a Dishwasher Load 

This section of the report drew from three surveys identified as Survey C, Survey D, and Survey F. 

Manufacturers and energy interest groups provided tJRese surveys. As in the report, a key requirement 

of this addendum is to maintain the confidentiality of the industry survey sources. To this end no 

source is identified and only general information on the data source is provided. Table I gives a 

comparative overview of the scope and size of these three surveys. 

Table 1 - Overview of scope and size of surveys 

Nationally Representative 

Regional Limited Moderate 

Regional <i250 Moderate Moderate 

Surveys C, D, and F, cumulatively representing over 2,000 U.S. households, provide text and/or 

graphic descriptions to link their categories of the level of soil to the three soil levels - Light, 

Medium, Heavy- and to quantitative amounts of soil for each level. Referencing ANSI! AHAM 

Standard DW-I for the types of soil and, most importantly, the amounts of soil is essential in relating 

each of the three soil levels to an amount of soil. 

The relationship to the amounts of soil representative of the soil levels is derived differently for each 

survey, but for each survey the relationship is made in terms of the number of place settings in the 

DOE test load that are soiled according to ANSI! AHA.M Standard DW -1. The results of the analysis 

to determine these relationships were presented in the report issued on December 18, 2001, and 

reflect an aggregation of the data from surveys C, D, and F. 

2 
Detail on the Analysis of Quantitative Amount of Soil in a Dishwasher Load 

The following sections describe in detail how the relationships between representative amounts of 

soil in a dishwasher load and the number of place settings in the DOE test load that are soiled 

according to ANSIIAHAM DW-1 were developed for surveys C, D, and F. 

Survey C 

The authors of survey C determined representative amounts of soil in a dishwasher load using an 

extensive set of photographs taken by consumers in their households of the soiled dishware in their 

dishwasher. The data set of survey C includes photos of both the lower and upper rack of dishware 

for 5,849 cycles distributed across nearly 1,000 U.S. households nationwide. Because of this 

photographic data and other substantial data sets Survey C is considered the most robust and credible 

source of information to determine the amount of soil on the dishware in the dishwashers of U.S. 

households. 

In preparation of the report of December 18,200 1, the authors of survey C provided the results of 

their soil level analysis and a brief overview of their survey approach, photographic data, and soil 

level analysis. To prepare the following discussion on their analysis of quantitative amounts of soil 

in a dishwasher load, the authors of survey C provided additional detail information. Given this 

additional information, a later section of this addendum refocuses the analysis and results presented 

in the report of December 18,200 1 to consider only survey C. 

The authors of survey C assessed their photographic data by first establishing a Likert scale1 to 

compare and rank each photo. Their Likert scale has five levels for the lower rack and five levels for 

the upper rack. Images of their Likert scale are shown in the appendix. 

This assessment resulted in the distribution shown in Figure 1. The x-axis in Figure 1 runs from a 

value of 2 to a value of 10 and represents the combined value of the Likert scale rankings for the 

lower and upper racks for each dishwasher load. For example, if the lower rack of a dishwasher load 

ranked a 3 and the upper rack a 1, it has a combined value of 4. The authors clustered the combined 

values into three levels - Light, Medium, and Heavy. The combined values of 2, 3, and 4 were 

categorized as light. The combined values of 5, 6, and 7 were categorized as medium. The combined 

values of 8,9, and 10 were categorized as heavy. 

Likert scale - A response scale, developed by Rensis Likert, used for assessing opinions and usually consisting of 5 or 

more categories. As a semi-variable approach it provides more information than the simple attribute (yes-no) question. 

3 

 EE-RM-TP-99-500 COMMENT 26

Detail on the Analysis of Quantitative Amount of Soil in a Dishwasher Load 

The following sections describe in detail how the relationships between representative amounts of 

soil in a dishwasher load and the number of place settings in the DOE test load that are soiled 

according to ANSIIAHAM DW-l were developed for surveys C, D, and F. 

Survey C 

The authors of survey C determined representative amounts of soil in a dishwasher load using an 

extensive set of photographs taken by consumers in their households of the soiled dishware in their 

dishwasher. The data set of survey C includes photos of both the lower and upper rack of dishware 

for 5,849 cycles distributed across nearly 1,000 U.S. households nationwide. Because of this 

photographic data and other substantial data sets Survey C is considered the most robust and credible 

source of information to determine the amount of soil on the dishware in the dishwashers of U.S. 

households. 

In preparation of the report of December 18, 2001, the authors of survey C provided the results of 

their soil level analysis and a brief overview of their survey approach, photographic data, and soil 

level analysis. To prepare the following discussion on their analysis of quantitative amounts of soil 

in a dishwasher load, the authors of survey C provided additional detail information. Given this 

additional information, a later section of this addendum refocuses the analysis and results presented 

in the report of December 18, 2001 to consider only survey C. 

The authors of survey C assessed their photographic data by first establishing a Likert scale 1 to 

compare and rank each photo. Their Likert scale has five levels for the lower rack and five levels for 

the upper rack. Images of their Likert scale are shown in the appendix. 

This assessment resulted in the distribution shown in Figure 1. The x-axis in Figure 1 runs from a 

value of 2 to a value of 10 and represents the combined value of the Likert scale rankings for the 

lower and upper racks for each dishwasher load. For example, if the lower rack of a dishwasher load 

ranked a 3 and the upper rack aI, it has a combined value of 4. The authors clustered the combined 

values into three levels - Light, Medium, and Heavy. The combined values of2, 3, and 4 were 

categorized as light. The combined values of 5, 6, and 7 were categorized as medium. The combined 

values of 8,9, and 10 were categorized as heavy. 

1 Likert scale - A response scale, developed by Rensis Likert, used for assessing opinions and usually consisting of 5 or 

more categories. As a semi-variable approach it provides more information than the simple attribute (yes-no) question. 

3 
Figure 1 - Distribution of Survey C results along its combined Likert scale 

2 3 4 5 6 7 8 9 10 

Combined Lower & Upper Rack Likert Scale Values 

A single combined value was selected from each soil level to facilitate more detailed evaluation and 

quantification of the photographic data to determine the amounts of soil representative of each soil 

level. For the light level the selected combined value was 3. For the medium level the selected 

combined value was 6. For the heavy level the selected combined value was 10. On review of 

Figure 2, it is clear that the combined values selected to represent the three soil levels - Light, 

Medium, and Heavy - were conservative choices. The selection of the combined value of 10 to 

represent the heavy level was particularly conservative. Out of the 5,849 dishwasher cycles 

evaluated, only 18 cycles were found to rank a combined value of 10. 

Approximately 12 sets of photos for each of the selected combined values were analyzed by a 

professional home economist. The home economist estimated the types and amounts of the food 

soils shown in the photos. The home economist then expressed these estimates in terms of amounts 

of food soils listed in ANSIIAHAM Standard DW-1. With the amount of soil expressed in terms of 

DW-1 food soils, a corresponding number of place settings that could be soiled per DW- 1 with that 

amount of food soil was easily determined. The results of this analysis are shown in Table 2. 

Table 2 - Results of survey C 

* Of the 19 dishwasher cycles that ranked a 10 on the combined Likertscale, only 2 

approached the quantity of food soil of 5 DW-1 soiled place settings. 

 EE-RM-TP-99-500 COMMENT 26

Figure 1 - Distribution of Survey C results along its combined Likert scale 

35.0% 

30.0% u 

>-

u 

'" v 

00 

";25.0% 

'0 

:; 

.2-

:\l20.0% 

U 

>-

() 

.... 

., 

.s:: 

~ 15.0% 

~ 

.s:: 

III 

C 

'010.0% 

c: ., 

u 

:;; 

c.. 5.0% 

0.0% 

2 

Arrows indicate the combined likert scale values selected to 

-----represent of the amounts of soil for each of the soil levels. 

3 4 5 6 7 8 

Combined Lower & Upper Rack Likert Scale Values 

9 10 

A single combined value was selected from each soil level to facilitate more detailed evaluation and 

quantification of the photographic data to determine the amounts of soil representative of each soil 

level. For the light level the selected combined value was 3. For the medium level the selected 

combined value was 6. For the heavy level the selected combined value was 10. On review of 

Figure 2, it is clear that the combined values selected to represent the three soil levels - Light, 

Medium, and Heavy - were conservative choices. The selection of the combined value of 10 to 

represent the heavy level was particularly conservative. Out of the 5,849 dishwasher cycles 

evaluated, only 18 cycles were found to rank a combined value of 10. 

Approximately 12 sets of photos for each of the selected combined values were analyzed by a 

professional home economist. The home economist estimated the types and amounts of the food 

soils shown in the photos. The home economist then expressed these estimates in terms of amounts 

of food soils listed in ANSI! AHAM Standard DW -1. With the amount of soil expressed in terms of 

DW-I food soils, a corresponding number of place settings that could be soiled per DW-I with that 

amount of food soil was easily determined. The results of this analysis are shown in Table 2. 

Table 2 - Results of survey C 

Light Medium Heavy 

3 6 10 

* ::; 1/2 -2 <5 

* Of the 19 dishwasher cycles that ranked a 10 on the combined Likert scale, only 2 

approached the quantity of food soil of 5 DW-1 soiled place settings. r 

4 
Survey D 

For survey D the respondents directly select an amount of soil in terms of DW-1 soiled place settings. 

At a central testing location the authors of survey D set up 6 images of a DW-1 place setting. Table 3 

describes the 6 images of a place setting, increasingly soiled according to DW-1, shown to survey 

respondents. 

Table 3 - Description of the 6 images of a DW-1 soiled place setting shown to respondents in survey D 

Also shown in Table 3 is how Arthur D. Little grouped the six images to the three soil levels - Light, 

Medium, and Heavy, and associated the images to a corresponding number of DW-1 soiled place 

settings in the DOE test load. Arthur D. Little's grouping of the six images was conducted in light of 

all of the available survey information and represents Arthur D. Little's best judgement. 

Grouping image 1 (Clean) with the light soil level, image 4 (50% DW-1) with the medium soil level, 

and image 6 (100% DW-1) with the heavy soil level was straightforward. Judgement entered in 

grouping the 3 remaining images. Image 2 (25% DW-1 rinsed) showed very little soil and clearly 

belonged in the light soil level. Image 3 (25% DW-1) appeared substantially less soiled than image 4 

and was grouped with the light soil level. Similarly, image 5 (75% DW-1) appeared only somewhat 

heavier soiled than image 4 and substantially less soiled than image 6, and was grouped with the 

medium soil level. 

Associating the images to a corresponding number of DW- 1 soiled place settings in the DOE test 

load was very direct. Given that there are 8 place settings in the DOE test load, the corresponding 

number of DW-1 soiled place settings is the product of 8 place settings times the percentage of DW-1 

soil. In the case of image 2 (25% DW-1 rinsed), the corresponding number of DW-1 soiled place 

settings in the DOE test load was arrived at by splitting the difference between that for image 1 

(Clean) and image 3 (25% DW-1). 

 EE-RM-TP-99-500 COMMENT 26

Survey D 

For survey D the respondents directly select an amount of soil in terms ofDW-l soiled place settings. 

At a central testing location the authors of survey D set up 6 images of a DW -1 place setting. Table 3 

. describes the 6 images of a place setting, increasingly soiled according to DW -1, shown to survey 

respondents. 

Table 3 - Description of the 6 images of a DW-1 soiled place setting shown to respondents in survey D 

1 Clean o 

2 25% of DW-1 food soils rinsed with water Light 1 

3 25% of DW-1 food soils 2 

4 50% of DW-1 food soils 4 

5 75% of DW-1 food soils 

Medium 

6 

6 100% of DW-1 food soils Heavy 8 

Also shown in Table 3 is how Arthur D. Little grouped the six images to the three soil levels - Light, 

Medium, and Heavy, and associated the images to a corresponding number of DW-l soiled place 

settings in the DOE test load. Arthur D. Little's grouping of the six images was conducted in light of 

all of the available survey information and represents Arthur D. Little's best judgement. 

Grouping image I (Clean) with the light soil level, image 4 (50% DW-l) with the medium soil level, 

and image 6 (100% DW-I) with the heavy soil level was straightforward. Judgement entered in 

grouping the 3 remaining images. Image 2 (25% DW-I rinsed) showed very little soil and clearly 

belonged in the light soil level. Image 3 (25% DW-I) appeared substantially less soiled than image 4 

and was grouped with the light soil level. Similarly, image 5 (75% DW-l) appeared only somewhat 

heavier soiled than image 4 and substantially less soiled than image 6, and was grouped with the 

medium soil level. 

Associating the images to a corresponding number ofDW-l soiled place settings in the DOE test 

load was very direct. Given that there are 8 place settings in the DOE test load, the corresponding 

number of DW -1 soiled place settings is the product of 8 place settings times the percentage of DW-I 

soil. In the case of image 2 (25% DW -1 rinsed), the corresponding number of DW -1 soiled place 

settings in the DOE test load was arrived at by splitting the difference between that for image 1 

(Clean) and image 3 (25% DW-l). 

5 
Examples of the images used in survey D are shown in Figure 2. 

Figure 2 - Examples of the images of a DW-1 soiled place setting used in survey D 

 EE-RM-TP-99-500 COMMENT 26

Examples of the images used in survey D are shown in Figure 2. 

Figure 2 - Examples of the images of a DW-1 soiled place setting used in survey 0 

6 
The authors interviewed respondents and asked them to select the image that most closely represents 

the level of soil on the dishware in their dishwasher. The results of survey D distributed as shown in 

Table 4 

Table 4 - Results of survey D 

As discussed in the report, the authors of survey D noted that some number of respondents were not 

properly interviewed and instructed prior to their selection of one of the soiled place setting images. 

The authors suspected that because of this lack of instruction respondents may have selected an 

image based on that image representing the amount of food soil in their dishwasher, rather than, as 

desired, selecting the image that most closely represents the level of soil on the dishware in their 

dishwasher. 

Survey F 

For survey F, Arthur D. Little linked the survey's description of its soil categories to an amount of 

soil in terms of DW- 1 place settings. The authors of survey F generated 4 categories of soil level. 

Brief descriptions of those four categories were presented to the survey respondents. Table 5 

describes the four categories of soil level used in survey F. and Arthur D. Little's assessment of the 

relation to the three soil levels - Light, Medium, and Heavy, and a corresponding number of DW-I 

soiled place settings in the DOE test load. 

Table 5 - Description of the categories of soil level used in survey F 

Table 5 also shows how Arthur D. Little grouped the four categories of survey F to the three soil 

levels - Light, Medium, and Heavy, and associated the category descriptions to a corresponding 

number of DW-1 soiled place settings in the DOE test load. Arthur D. Little's grouping of the four 

Somewhat 

Clean 

Somewhat 

Dirty 

Very Dirty 

Small particles of 

food left 

Only the largest 

chunks of food 

gone 

No or almost no 

food removed 

Medium 

Heavy 

4 

8 

 EE-RM-TP-99-500 COMMENT 26

The authors interviewed respondents and asked them to select the image that most closely represents 

the level of soil on the dishware in their dishwasher. The results of survey D distributed as shown in 

Table 4 

Table 4 - Results of survey D 

As discussed in the report, the authors of survey D noted that some number of respondents were not 

properly interviewed and instructed prior to their selection of one of the soiled place setting images. 

The authors suspected that because of this lack of instruction respondents may have selected an 

image based on that image representing the amount of food soil in their dishwasher, rather than, as 

desired, selecting the image that most closely represents the level of soil on the dishware in their 

dishwasher. 

SurveyF 

For survey F, Arthur D. Little linked the survey's description of its soil categories to an amount of 

soil in terms of DW -1 place settings. The authors of survey F generated 4 categories of soil level. 

Brief descriptions of those four categories were presented to the survey respondents. Table 5 

describes the four categories of soil level used in survey F. and Arthur D. Little's assessment of the 

relation to the three soil levels - Light, Medium, and Heavy, and a corresponding number ofDW-l 

soiled place settings in the DOE test load. 

Table 5 - Description of the categories of soil level used in survey F 

Very Clean 

Somewhat 

Clean 

Somewhat 

Dirty 

Very Dirty 

All or almost all 

food gone 

Small particles of 

food left 

Only the largest 

chunks of food 

No or almost no 

food removed 

Light 

Medium 

Heavy 

1 

4 

8 

Table 5 also shows how Arthur D. Little grouped the four categories of survey F to the three soil 

levels - Light, Medium, and Heavy, and associated the category descriptions to a corresponding 

number of DW-I soiled place settings in the DOE test load. Arthur D. Little's grouping of the four 

7 
categories was conducted in light of all of the available survey information and represents Arthur D. 

Little's best judgement. 

Grouping the Very Clean category with the light soil level and the Very Dirty category with the 

heavy soil level was straightforward. Here, judgement entered in grouping the 2 remaining 

categories. The Somewhat Clean category implied very little soil and clearly belonged in the light 

soil level. The Somewhat Dirty category implied much more food soil than the Somewhat Clean 

category, but substantially less soiled than the Very Dirty category, and was grouped in the medium 

soil level. 

Associating the categories to a corresponding number of DW-I soiled place settings in the DOE test 

load was less direct. Instead the soil levels were assigned a corresponding number of DW- I soiled 

place settings. Given that there are 8 place settings in the DOE test load, the light soil level was 

assigned 1 DW-1 soiled place setting, the medium level - 4 DW-1 soiled place settings, and the 

heavy level - 8 DW-1 soiled place settings. 

The results of suivey F are shown in Table 6. 

Table 6 - Results of survey F 

 EE-RM-TP-99-500 COMMENT 26

categories was conducted in light of all of the available survey information and represents Arthur D. 

Little's best judgement. 

Grouping the Very Clean category with the light soil level and the V ~ry Dirty category with the 

heavy soil level was straightforward. Here, judgement entered in grouping the 2 remaining 

categories. The Somewhat Clean category implied very little soil and clearly belonged in the light 

soil leveL The Somewhat Dirty category implied much more food soil than the Somewhat Clean 

category, but substantially less soiled than the Very Dirty category, and was grouped in the medium 

soil leveL 

Associating the categories to a corresponding number of DW-1 soiled place settings in the DOE test 

load was less direct. Instead the soil levels were assigned a corresponding number ofDW-1 soiled 

place settings. Given that there are 8 place settings in the DOE test load, the light soil level was 

assigned 1 DW -1 soiled place setting, the medium level - 4 DW -1 soiled place settings, and the 

heavy level- 8 DW-1 soiled place settings. 

The results of survey F are shown in Table 6. 

Table 6 - Results of survey F 

8 
Summary of Analysis on Quantitative Amount of Soil in a Dishwasher Load 

The following tables and figure result directly from the review and analysis of surveys C, D, and F 

outlined above. These tables and figure were discussed and presented in the report of December 18, 

200 1. They are presented here again for clarity in demonstrating the progression of the detailed 

analysis to the results in the report of December 18, 200 1. In the following section the analysis and 

results will be reconsidered in light of the additional information provided by the authors of survey C 

for this addendum. 

Table 7 is the consolidation of the data found in Tables 2,4, and 6. Figure 3 is the distribution of that 

data weighted by the number of survey respondents. Table 8 shows the weighted averages of the 

distribution shown in Figure 3 for each of the three soil levels - Light, Medium, and Heavy. 

Table 7 - Range of Survey Data on the Representative Amounts of Soil in the Three Soil Levels 

Figure 3 - Distribution of Survey Respondents on Representative Amounts of Soil in Each of the Three 

Soil Levels 

4 Heavy Soil Level 

112 1 2 2 4 6 5 8 

Number of Place Settings in the DOE Test Load Soiled According to DW-1 

 EE-RM-TP-99-500 COMMENT 26

Summary of Analysis on Quantitative Amount of Soil in a Dishwasher Load 

The following tables and figure result directly from the review and analysis of surveys C, D, and F 

outlined above. These tables and figure were discussed and presented in the report of December 18, 

2001. They are presented here again for clarity in demonstrating the progression of the detailed 

analysis to the results in the report of December 18, 2001. In the following section the analysis and 

results will be reconsidered in light of the additional information provided by the authors of survey C 

for this addendum. 

Table 7 is the consolidation of the data found in Tables 2, 4, and 6. Figure 3 is the distribution of that 

data weighted by the number of survey respondents. Table 8 shows the weighted averages of the 

distribution shown in Figure 3 for each of the three soil levels - Light, Medium, and Heavy. 

Table 7 - Range of Survey Data on the Representative Amounts of Soil in the Three Soil Levels 

Figure 3 - Distribution of Survey Respondents on Representative Amounts of Soil in Each of the Three 

Soil Levels 

100% 

90% 

Cii 

> 

CI> 

..J 80% 

·0 

rJ) 

.s: 70% Light Soil Level u 

.. - 1 Soiled Place w 

.:: 

!l 60% 

!: 

CI> 

" !: 50% 0 

cUI 

CI> 

0:: 40% >-

CI> 

~ ::s 

rJ) 30% 

'0 

.... 

!: 

CI> 20% u 

... 

CI> 

0.. 

10% 

0% 

1/2 2 2 4 6 5 8 

Number of Place Settings in the DOE Test Load Soiled According to DW-1 

9 
Table 8 - Representative Amounts of Soil in Each of the Three Soil Levels Found in Dishwashers of 

U.S. households 

 EE-RM-TP-99-500 COMMENT 26

Table 8 - Representative Amounts of Soil in Each of the Three Soil Levels Found in Dishwashers of 

U.S. households 

10 
Additional Data from Survey C and Refocus of Analysis on Survey C 

In the preparation of the report issued December 18,2001 the authors of survey C provided only the 

results of their analysis - the number of place settings in the DOE test load soiled per DW-1 as shown 

in Table 2, and only a brief description of their approach to the analysis. In preparation of this 

addendum the authors of survey C provided a substantial amount of additional information to fill in 

the detail of their analysis. This additional information includes their Likert scale (Appendix), the 

ranking of their data against that Likert scale (Figure I), and a professional home economist's 

evaluation of the amount of soil shown in samples of the photographic data taken from the combined 

Likert scale rankings of 3,6, and 10. This section presents the evaluation of the professional home 

economist of survey C and reconsiders the results of that evaluation. 

Recall from the discussion above on survey C that the authors of survey C grouped their combined 

Likert scale rankings into three soil levels - Light, Medium, and Heavy. The combined Likert scale 

rankings of 2, 3, and 4 were grouped into the light soil level. The combined Likert scale rankings of 

5, 6, and 7 were grouped into the medium soil level. The combined Likert scale rankings of 8, 9, and 

10 were grouped into the heavy soil level. The combined Likert scale rankings of 3,6, and 10 were 

selected to represent the light, medium and heavy soil level, respectively. 

A professional home economist analyzed a number of sets of photos for each of the combined Likert 

scale rankings of 3, 6, and 10. The home economist estimated the types and amounts of food soils 

shown in the photos. Table 9 lists the averages and medians of the mass of food soils estimated fi-om 

the samples of photo sets for the possible Likert scale combinations corresponding to the combined 

Likert scale rankings of 3, 6, and 10. 

Table 9 -Averages and medians of the mass of food soils estimated by a home economist 

Translating these masses of food soils to corresponding numbers of place settings soiled according to 

ANSIIAHAM DW-1 is readily done given that DW-1 specifies approximately 3 1.3 grams of food 

soils per place setting as shown in Table 10. 

Light 

Medium 

Heavy 

3 

6 

10 

211 

313 

412 

51 1 

515 

10 

9 

10 

10 

18 

12.7 

27.0 

52.3 

72.3 

109.5 

10.85 

24.9 

50.25 

62.5 

115.8 

 EE-RM-TP-99-500 COMMENT 26

Additional Data from Survey C and Refocus of Analysis on Survey C 

In the preparation of the report issued December 18, 2001 the authors of survey C provided only the 

results of their analysis - the number of place settings in the DOE test load soiled per DW -1 as shown 

in Table 2, and only a brief description of their approach to the analysis. In preparation of this 

addendum the authors of survey C provided a substantial amount of additional information to fill in 

the detail of their analysis. This additional information includes their Likert scale (Appendix), the 

ranking of their data against that Likert scale (Figure 1), and a professional home economist's 

evaluation of the amount of soil shown in samples of the photographic data taken from the combined 

Likert scale rankings of3, 6, and 10. This section presents the evaluation of the professional home 

economist of survey C and reconsiders the results of that evaluation. 

Recall from the discussion above on survey C that the authors of survey C grouped their combined 

Likert scale ranlcings into three soil levels - Light, Medium, and Heavy. The combined Likert scale 

ranlcings of 2, 3, and 4 were grouped into the light soil level. The combined Likert scale rankings of 

5, 6, and 7 were grouped into the medium soil level. The combined Likert scale rankings of 8,9, and 

10 were grouped into the heavy soil level. The combined Likert scale rankings of 3, 6, and 10 were 

selected to represent the light, medium and heavy soil level, respectively. 

A professional home economist analyzed a number of sets of photos for each of the combined Likert 

scale rankings of 3, 6, and 10. The home economist estimated the types and amounts of food soils 

shown in the photos. Table 9 lists the averages and medians of the mass of food soils estimated from 

the samples of photo sets for the possible Likert scale combinations corresponding to the combined 

Likert scale rankings of 3, 6, and 10. 

Table 9 - Averages and medians of the mass of food soils estimated by a home economist 

Light 3 211 10 12.7 10.85 

3/3 9 27.0 24.9 

Medium 6 4/2 10 52.3 50.25 

5/1 10 72.3 62.5 

Heavy 10 5/5 18 109.5 115.8 

Translating these masses of food soils to corresponding numbers of place settings soiled according to 

ANSIIAHAM DW-l is readily done given that DW-l specifies approximately 31.3 grams of food 

soils per place setting as shown in Table 10. 

11 
Table 10 - Averages and medians of the number of DW-1 soiled place settings based on the estimated 

masses of food soils in Table 9 

Comparing Table 10 to the results initially provided by the authors of survey C in Table 2 further 

indicates the conservatism of the authors in interpreting the results of their analysis. Discussions with 

the authors of survey C indicated that the average number of DW-1 soiled place settings in Table 10 

were rounded up to yield the results that they had reported in Figure 2. However, their reported value 

of <5 DW-I soiled place setting for the heavy soil level seems inconsistent with simple rounding 

upward of the averages for the number of DW-1 soiled place settings. It appears more consistent 

with a value that represents an extreme bound. This inconsistency is addressed in the following 

recommendations. 

Light 

Medium 

Heavy 

3 

6 

10 

211 

313 

412 

511 

515 

10 

9 

10 

10 

18 

0.41 

0.86 

1.67 

2.31 

3.50 

0.35 

0.80 

1.61 

2.00 

3.70 

 EE-RM-TP-99-500 COMMENT 26

Table 10 - Averages and medians of the number of DW-1 soiled place settings based on the estimated 

masses of food soils in Table 9 

Light 3 2/1 10 0.41 0.35 

3/3 9 0.86 0.80 

Medium 6 4/2 10 1.67 1.61 

5/1 10 2.31 2.00 

Heavy 10 515 18 3.50 3.70 

Comparing Table 10 to the results initially provided by the authors of survey C in Table 2 further 

indicates the conservatism of the authors in interpreting the results of their analysis. Discussions with 

the authors of survey C indicated that the average number of DW -1 soiled place settings in Table 10 

were rounded up to yield the results that they had reported in Figure 2. However, their reported value 

of <5 DW-I soiled place setting for the heavy soil level seems inconsistent with simple rounding 

upward of the averages for the number ofDW-1 soiled place settings. It appears more consistent 

with a value that represents an extreme bound. This inconsistency is addressed in the following 

recommendations. 

12 
Clarification on the Recommended Range of Average-use Cycles per Year 

The report of December 18,2001 presented three approaches to establish a number for the averageuse cycles per year of a dishwasher. The first approach referenced the Energy Information 

Administration's 1997 Residential Energy Consumption Survey (1997 RECS) as a large, welldocumented, and publicly available study on the average-use of a dishwasher. Although a large and 

well-documented survey, 1997 RECS use of only three response categories, two of which were 

unbounded, made determining an average number difficult. The 1997 RECS data on average-use of a 

dishwasher did clearly indicate that 50 percent of its respondents use a dishwasher 4 or less times per 

week, or approximately 208 times per year. The 1997 RECS set a target area for the number of 

average-use cycles per year. 

The second approach considered five nationally representative surveys (surveys A, B, G, H, and 1997 

RECS) using consistent definitions for the broad or unbounded categories of the surveys'. A low 

value, an average value, and a high value were defined for each category found in the surveys. These 

values were used to calculate three averages for each survey - one average using the low values, a 

second average using the 'average' values, and a third average using the high values. As presented in 

the report of December 18,2001 the averages of theses averages were 180 for the low values, 206 for 

the average values, and 233 for the high values. This approach produced averages centered near the 

target area set by 1997 RECS and established a range, although broad, for the number of average-use 

cycles per year. 

The third approach narrowed the range for the number of average-use cycles per year by building on 

the previous approaches. The first approach using the 1997 RECS data set a target area for the 

number of average-use cycles per year. The second approach produced averages centered on this 

target area and suggested that all of the available data pointed to this target area. The third approach 

used a weighted average to test whether the relative sizes of the five surveys would skew the averages 

from the target area. Using the average values for the categories of the five surveys fi-om the second 

approach, an average weighted by the number of respondents in each survey was calculated. This 

calculation yielded a weighted average of 200 average-use cycles per year. As this weighted average 

of the five nationally representative surveys, was near the target area and near the middle of the 

previously calculated range of averages, it indicated that the size of no one survey dominated over the 

others. This weighted average became the new minimum of the range for the number of average-use 

cycles per year. Therefore, the lower end of the broad range established in the previous approach 

moved up from 180 to 200 average-use cycles per year. 

The above analyses of available survey data on the frequency of households' usage of a dishwasher 

set a range for the number of average-use cycles per year. The lower end of this range, 200, was set 

by the third approach and the upper end of this range, 233, remained at the high average set by the 

second approach. To refine this recommended range further additional data for the representative 

average-use cycles per year was sought from the dishwasher detergent industry. This additional data 

was anticipated for late December 2001 or early January 2002, however no additional data has been 

made available to augment the recommendation. 

 EE-RM-TP-99-500 COMMENT 26

Clarification on the Recommended Range of Average-use Cycles per Year 

The report of December 18, 2001 presented three approaches to establish a number for the averageuse cycles per year of a dishwasher. The first approach referenced the Energy Information 

Administration's 1997 Residential Energy Consumption Survey (1997 RECS) as a large, welldocumented, and publicly available study on the average-use of a dishwasher. Although a large and 

well-documented survey, 1997 RECS use of only three response categories, two of which were 

unbounded, made determining an average number difficult. The 1997 RECS data on average-use of a 

dishwasher did clearly indicate that 50 percent of its respondents use a dishwasher 4 or less times per 

week, or approximately 208 times per year. The 1997 RECS set a target area for the number of 

average-use cycles per year. 

The second approach considered five nationally representative surveys (surveys A, B, G, H, and 1997 

RECS) using consistent definitions for the broad or unbounded categories of the surveys'. A low 

value, an average value, and a high value were defined for each category found in the surveys. These 

values were used to calculate three averages for each survey - one average using the low values, a 

second average using the 'average' values, and a third average using the high values. As presented in 

the report of December 18,2001 the averages of theses averages were 180 for the low values, 206 for 

the average values, and 233 for the high values. This approach produced averages centered near the 

target area set by 1997 RECS and established a range, although broad, for the number of average-use 

cycles per year. 

The third approach narrowed the range for the number of average-use cycles per year by building on 

the previous approaches. The first approach using the 1997 RECS data set a target area for the 

number of average-use cycles per year. The second approach produced averages centered on this 

target area and suggested that all of the available data pointed to this target area. The third approach 

used a weighted average to test whether the relative sizes of the five surveys would skew the averages 

from the target area. Using the average values for the categories of the five surveys from the second 

approach, an average weighted by the number of respondents in each survey was calculated. This 

calculation yielded a weighted average of 200 average-use cycles per year. As this weighted average 

of the five nationally representative surveys, was near the target area and near the middle of the 

previously calculated range of averages, it indicated that the size of no one survey dominated over the 

others. This weighted average became the new minimum of the range for the number of average-use 

cycles per year. Therefore, the lower end of the broad range established in the previous approach 

moved up from 180 to 200 average-use cycles per year. 

The above analyses of available survey data on the frequency of households' usage of a dishwasher 

set a range for the number of average-use cycles per year. The lower end of this range, 200, was set 

by the third approach and the upper end of this range, 233, remained at the high average set by the 

second approach. To refine this recommended range further additional data for the representative 

average-use cycles per year was sought from the dishwasher detergent industry. This additional data 

was anticipated for late December 2001 or early January 2002, however no additional data has been 

made available to augment the recommendation. 

13 
Recommendations 

Given the depth of the additional information provided by the authors of survey C and the detail of 

the analysis of survey C presented in this addendum, it is Arthur D. Little's recommendation that the 

data of survey C stand as the basis for determining the amount of soil on dishware in the dishwashers 

of U.S. households. However, Arthur D. Little also recommends that the survey C results reported in 

Table 2 should be modified to more closely reflect the approach of rounding upward the average 

number of DW-1 soiled place settings in Table 10. To this end the number of DW-1 soiled place 

settings corresponding to the heavy soil level should be set equal to 4. Table 11 displays the number 

of DW-1 soiled place settings reported by the authors of survey C and highlights the modified 

number of place settings recommended by Arthur D. Little on review of greater detail from survey C. 

Table 11 - Revised recommendation on the number of DW-1 soiled place settings corresponding to the 

soil levels 

To be consistent with the recommendation to let survey C stand as the basis for determining the 

amount of soil on dishware in the dishwashers of U.S. households, it is important to revisit the 

distribution of U.S. households by the level of soil in dishwasher loads (December 18,2001 report, 

Figure 2). In the report of December 18,2001 this distribution was based on an aggregation of 

surveys A and C. As discussed in the report both surveys A and C are large, nationally representative 

surveys and both present very similar data on the qualitative level of soil in dishwasher loads. Talung 

survey C alone would shift this distribution somewhat from the medium soil level to the light soil 

level. However, ignoring survey A would be discounting a quality survey. Therefore, Arthur D. 

Little further recommends that the distribution of U.S. households by the level of soil in dishwasher 

loads should stand as it was presented in the report of December 18,2001. 

 EE-RM-TP-99-500 COMMENT 26

Recommendations 

Oi ven the depth of the additional infonnation provided by the authors of survey C and the detail of 

the analysis of survey C presented in this addendum, it is Arthur D. Little's recommendation that the 

data of survey C stand as the basis for detennining the amount of soil on dishware in the dishwashers 

of U.S. households. However, Arthur D. Little also recommends that the survey C results reported in 

Table 2 should be modified to more closely reflect the approach of rounding upward the average 

number ofDW-1 soiled place settings in Table 10. To this end the number of DW-l soiled place 

settings corresponding to the heavy soil level should be set equal to 4. Table II displays the number 

of D W -I soiled place settings reported by the authors of survey C and highlights the modified 

number of place settings recommended by Arthur D. Little on review of greater detail from survey C. 

Table 11 - Revised recommendation on the number of DW-1 soiled place settings corresponding to the 

soil levels 

To be consistent with the recommendation to let survey C stand as the basis for detennining the 

amount of soil on dishware in the dishwashers of U.S. households, it is important to revisit the 

distribution of U.S. households by the level of soil in dishwasher loads (December 18, 200 I report, 

Figure 2). In the report of December 18,2001 this distribution was based on an aggregation of 

surveys A and C. As discussed in the report both surveys A and C are large, nationally representative 

surveys and both present very similar data on the qualitative level of soil in dishwasher loads. Taking 

survey C alone would shift this distribution somewhat from the medium soil level to the light soil 

level. However, ignoring survey A would be discounting a quality survey. Therefore, Arthur D. 

Little further recommends that the distribution of U.S. households by the level of soil in dishwasher 

loads should stand as it was presented in the report of December 18, 2001. 

14 
Appendix - Survey C Likert scale 

 EE-RM-TP-99-500 COMMENT 26

Appendix - Survey C Likert scale 

A1 
Appendix - Survey C Likert scale 

 EE-RM-TP-99-500 COMMENT 26

Appendix - Survey C Likert scale 

2 

2 A2 
Appendix - Survey C Likert scale 

 EE-RM-TP-99-500 COMMENT 26

Appendix - Survey C Likert scale 

3 A3 
Appendix - Survey C Likert scale 

 EE-RM-TP-99-500 COMMENT 26

Appendix - Survey C Likert scale 

4 A4 
dix - Survey C Likert scale 

 EE-RM-TP-99-500 COMMENT 26

Appendix - Survey C Likert scale 

5 A5