NISTIR 6865 

BEES 2.0 

Building for Environmental and Economic Sustainability 

Peer Review Report 

Mary Ann Curran 

Jonathan G. Overly 

Patrick Hofstetter 

Rick Muller 

Barbara C. Lippiatt 
 

NISTIR 6865 

BEES 2.0 

Building for Environmental and Economic Sustainability 

Peer Review Report 

Peer Review Chair: 

Mary Ann Curran 

National Risk Management Research Laboratory 

U.S. Environmental Protection Agency 

Peer Review Panel: 

Jonathan G. Overly 

Center for Clean Products and Clean Technologies 

University of Tennessee 

Patrick Hofstetter 

ORISE Research Fellow 

U.S. Environmental Protection Agency 

Rick Muller 

California Integrated Waste Management Board 

Review Requested by: 

Barbara C. Lippiatt 

Building and Fire Research Laboratory 

National Institute of Standards and Technology 

April 2002 

U.S. Department of Commerce 

Donald L. Evans, Secretary 

Technology Administration 

Phillip J. Bond, Under Secretary for 

Technology

National Institute of Standards and Technology

Arden L. Bement, Jr., Director
ii 

Abstract 

This report documents the results of a technical review of the BEES® (Building for Environmental 

and Economic Sustainability) 2.0b software and user manual. BEES measures the life-cycle 

environmental and economic performance of building products using methods based on consensus 

standards. Comments by four external experts are compiled in this report, along with responses from 

the BEES developers. While the life-cycle environmental performance measurement field has 

advanced dramatically over the last decade, valid disagreements and limitations remain. The report 

sheds light on the key issues to help BEES users place BEES results in their proper context, and to 

help environmental researchers and tool developers struggling with the same concerns. 

Key words: Building products, economic performance, environmental performance, green 

buildings, life cycle assessment, life-cycle costing, multiattribute decision analysis, sustainable 

development 

Disclaimer

The United States Department of Commerce and NIST do not endorse any particular brand,

product, or service. BEES 2.0 compares generic, U.S. industry-average product classes only and

no representations are made as to the quality or fitness of any specific manufacturer's product.

Users shall not in any way say or imply that the information obtained from BEES is an

endorsement of any particular product, service, or brand. 

The BEES tool bears no warranty, neither express nor implied. NIST does not assume legal

liability nor responsibility for a User's utilization of BEES. NO WARRANTIES AS TO ANY

MATTER WHATSOEVER ARE MADE BY NIST, INCLUDING NO WARRANTY OF

MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 
iii 

Preface 

This document is the result of a technical review of the BEES 2.0b software and user manual, 

developed by the National Institute of Standards and Technology (NISTIR 6520). BEES measures 

the life-cycle environmental and economic performance of building products using methods based 

on consensus standards. For environmental performance measurement, the BEES approach is based 

on the International Organization for Standardization (ISO) 14040 series of life-cycle assessment 

(LCA) standards. These standards require a peer review for LCA studies whose results are used to 

support comparative assertions. This document constitutes the required peer review. 

The review was conducted at the request of the BEES Developer, Ms. Barbara Lippiatt of NIST. 

Ms. Mary Ann Curran of the U.S. EPA National Risk Management Research Laboratory served as 

review chair. Comments were received from four external peer reviewers and then compiled by the 

review chair in this report. Reviewers were selected by their backgrounds and areas of technical 

expertise in order to form a diverse team that would be able to evaluate all aspects of the BEES tool. 

Short biographical sketches of the reviewers can be found at the end of the report. One of the 

reviewers, representing the industrial sector, asked for his review to remain anonymous. 

While the LCA field has advanced dramatically over the last decade, valid disagreements and 

limitations remain. Indeed, not all reviewers agreed with all the issues raised in this report. Since 

reviews were done independently, there are unavoidable inconsistencies among some of the 

comments. To resolve or give NIST guidance on some of the key issues, the peer review team held a 

meeting with the BEES team at NIST in January 2001. The consensus resulting from this meeting is 

noted where appropriate throughout the report. The intent is to shed light on the issues in an effort to 

help BEES users place BEES results in their proper context, and to help LCA researchers and tool 

developers struggling with the same concerns. 

NIST sincerely thanks the entire peer review team for generously volunteering their time and 

expertise for this effort. 

Mary Ann Curran and Barbara Lippiatt 
iv 

Contents 

Abstract........................................................................................................................................... ii 

Preface............................................................................................................................................iii 

1. Introduction................................................................................................................................. 1 

2. General Comments...................................................................................................................... 3 

3. Installation................................................................................................................................... 5 

4. User-Friendliness ........................................................................................................................ 7 

5. Format......................................................................................................................................... 9 

6. Presentation of Results.............................................................................................................. 11 

7. Modeling................................................................................................................................... 13 

7.1 Selection of Impact Categories.....................................................................................................................14

7.2 Characterization Models ..............................................................................................................................15

7.3 Normalization and Weighting ......................................................................................................................21

8. Data and Data Sources.............................................................................................................. 25 

9. User Support ............................................................................................................................. 31 

10. User Manual............................................................................................................................ 33 

11. Other Recommendations......................................................................................................... 35 

Biographical Sketches of Reviewers ............................................................................................ 36 

References..................................................................................................................................... 37 
1 

1. Introduction 

The BEES peer review team members were provided with information on how to download the 

BEES software and user guide from the Internet. Reviewers did not inspect primary data or review 

the TEAM (Tool for Environmental Analysis and Management) model used to develop cradle-togate data. 

Specific review comments follow and are grouped by the following topical areas: 

- General Comments 

- Installation 

- User-Friendliness 

- Format 

- Presentation of Results 

- Modeling (Specifically Impact Modeling) 

- Data and Data Sources 

- User Support 

- User Manual 

- Other Recommendations 

Any comments that require a response from NIST are followed by comments prepared by Ms. 

Lippiatt. Her responses are given in Italics. 
2 
3 

2. General Comments 

BEES 2.0b is a very important tool in the toolbox of an environmentally-conscious designer, builder, 

or product manufacturer. It goes beyond a limited environmental life cycle assessment to include 

life cycle costs. Such life-cycle based cost information is usually lacking in standard cost calculation 

tools. The program is simple in its application and does not require information that is not readily 

available to the user. It makes a good attempt at simplifying a complicated process. If the user is 

aware of the limitations and assumptions made in the model and applies this to their decision 

process, the model has added a tool to the building designer's toolbox. 

The BEES tool takes into account that the user may need to use highly aggregated information in 

order to find the best option. The capability of weighing different environmental impacts as well 

as weighing economic and environmental criteria is a strong feature of the tool and should be 

retained (ISO 14042 may suggest the opposite, but this would “kill” the tool). The opportunity, 

as presently provided, to get single scores without aggregation needs to be maintained for 

transparency reasons. Another reviewer commented that environmental impact categories 

should not be grouped into a single number, saying that this simplifies a complicated decision 

process and would leave the user with a false impression. 

NIST Response: To accommodate those who believe that impact categories should not be 

grouped into a single environmental performance score, BEES offers a “no weighting” option.

The life-cycle assessment (LCA) principles and terminology used in the model are appropriate. 

The first step in any LCA involves setting the boundaries of the product system under study. In 

BEES, system boundaries are defined by three decision criteria: mass, energy, and cost 

contributions. While some may disagree with these criteria, they do define the boundaries NIST 

wants to use for the study. 
4 
5 

3. Installation 

Reviewers commented that initially downloading the software was easy to accomplish, as was 

installation. One reviewer noted having problems with initial installation because the registration page 

on the Internet stayed on the screen after being submitted. A while later he realized that the download 

page appears behind this page. After that the installation worked perfectly! 

Another reviewer noted that he had to search for the primary executable file to start the program. 

(“Maybe I didn't read something that I should have, but there should be something that tells me more 

specifically what file to open so I don't have to go search for it.”) 

NIST Response: The easiest way to run BEES is to follow the installation instructions provided 

on the download site and in the user manual: “Once installation is complete, you are ready to 

run BEES 2.0 from your program group BEES.” These instructions will be clarified before 

releasing BEES 3.0 by adding that, to run BEES, you simply select 

Start¤Programs¤BEES¤BEES30. 

The software is simple to use, requiring roughly 15 minutes to review the help information to start 

using the tool and produce easily reviewable results. The graphs and data provided do give the 

information that they say they will provide, supplying primarily sets of tables and graphs that show 

you the results that you selected to see from the analysis you ran. 
6 
7 

4. User-Friendliness 

- While BEES is computing scores, if I try to switch over to another application on my computer, 

the 'computing...' window remains on top of my other applications. It would be nice if it would 

simply stay within the confines of the BEES window/software. 

NIST Response: NIST will fix this bug before releasing BEES 3.0. 

- I like that the “Welcome” page explains the temporal and spatial applicability of the data, 

particularly stating that generic product groups are being compared within the tool, not specific 

products from specific manufacturers. The careful reader will be well informed about what kind of 

analysis they are doing in BEES. Also about the “Welcome” page, I like that it comes up when 

opening BEES. You should 1) add some text at the top of that page that highly recommends the user 

read this 11⁄2 - 2 pages of info (fairly short, as many first time users don't want to do that!), and 2) 

make an option somewhere for turning off its opening every time, after the user has read what he/she 

needs to read [Having an options area will certainly allow for the addition of more flexibility as the 

tool progresses and reaches a wider audience.] 

NIST Response: NIST will add text highly recommending reading the “Welcome” page to BEES 

3.0, and plans to incorporate option settings in future versions of BEES. 

- It is good that the user can also use the Tab and Alt keys to move within the system (a Windows 

OS plus), in addition to 'mousing.' However, in some interfaces it appears that the tabbing order 

could be improved. 

NIST Response: NIST will check tabbing order on all windows and improve where needed before 

publishing BEES 3.0. 

- While the instructions do inform the user how to select multiple products within the “Select 

Product Alternatives” window, it couldn't hurt to simply place the following text at the top of that 

window: “Use the Control key to select multiple products.” 

NIST Response: It is not possible to use the Control key to select multiple products, and the 

instructions do not inform the user to that effect. 

- Why limit the Zoom In/Zoom Out system to two levels? It would seem nice to allow for at least a 

few more considering the different monitor sizes, resolutions, and visions (eye sight quality) that 

could be at work in trying to review the results (and also considering the limited graphics that are 

supplied). 

NIST Response: Unfortunately, NIST cannot change the two-level Zoom In/Zoom Out system. 

BEES uses a graphics tool to display its graphs, and while many graphics settings may be 

changed, this is not one of them. As more powerful graphics tools become available, NIST will 

explore using them in future versions of BEES. 

- The print feature for the results works well. 
8 

- If I am in the middle of reviewing some results (like a graph) and choose the export option but 

then choose to Cancel, I don't get taken back to the graph, but get booted back to the “Select BEES 

Reports” window. Same with Print. 

NIST Response: This is a bug in the graphics tool NIST uses that has been reported to tool 

developers, and will hopefully be fixed before NIST releases future versions of BEES. 

- It is a bit odd to have execution items as the only listings in the main menu (when viewing graphs, 

“Export” and “Print”). It would seem advantageous to make an options (or the like) menu choice so 

that exporting and printing can pull-down off of this choice. This would be more consistent with 

what folks see in other applications within Windows. [Not that Windows is the best thing ever, but 

it certainly does appear to be the benchmark by which other systems and functionality sets are 

compared.] 

NIST Response: NIST agrees. Unfortunately, NIST cannot change the menu items offered by the 

graphics tool it uses. As more powerful graphics tools become available, NIST will explore using 

them in future versions of BEES 

- BEES does not allow the user to select, evaluate and compare many individual product 

specifications found within the generic product classifications. There is a need to compare the 

environmental performance and cost of products with similar applications, but having distinctly 

different green attributes. As such, the BEES Program is not a particularly useful tool for “fine 

tuning” the green material selections that are available within most generic product specifications. 

NIST Response: To respond to this need, NIST will include manufacturer-specific product data 

in BEES 3.0. 
9 

5. Format 

- The BEES PROGRAM format is very user-friendly and logical. In fact, it was easy to 

discover most of the program features and reporting possibilities BEES offers even without 

reading the manual. The multi-color, graphical display makes it possible to quickly compare 

multi-faceted data for the distinct building products types, however, a numerical display is also 

available. The reviewer especially liked the feature that allows the user to see how building 

products compare based on a single attribute environmental impact, such as global warming, as 

opposed to reporting a score for combined environmental impacts. 
10 
11 

6. Presentation of Results 

- It would be nice if the user knew beforehand that multiple windows of information were going to 

open once he/she selects certain type of results to review and clicks the “Display” button. Granted, 

he/she will find out soon enough, but isn't it the job of software designers to make software intuitive 

and easy to use and understand? A simple line of text at the top of the “Select BEES Reports” 

window stating that “multiple windows will open displaying the individual results you have chosen 

to see” should prepare the user for what is to come. Also, without any description, I thought the “All 

Tables in One” checkbox was going to give me all the results in one pop-up page/window – 

incorrect. 

NIST Response: For BEES 3.0, NIST will cascade multiple results windows so they don’t open 

entirely on top of one another. In addition, to further explain the “All Tables in One” option, 

NIST will add to BEES 3.0 a detailed description that will display when the mouse moves over its 

checkbox. 

- As far as I can tell, the only graphing you get are the columnar charts that are shown in the results. 

There are no options for other graphs or other tabular presentations of those same results (flexibility 

in presentation limitation). 

NIST Response: NIST plans to offer a range of graph types in future versions of BEES. 

- The one graph that is being used doesn't really help any more than the scores themselves. It would 

be nice to see a graph that allowed me to see more of what the results are trying to show... 

NIST Response: More than one graph is used, and different levels of detail are provided. Don’t 

understand this comment. 

- It would seem easier for me to see differences between products if the 'by life-cycle stage' results 

presentation did not stack up the life-cycle stage results, but placed them in different columns which 

compared each product side-by-side in the column. (In the life-cycle stage breakdown, I am not so 

much concerned with the total life impacts anymore as I am trying to burrow down within each 

life-cycle stage. I can go back to the whole life results if I want the aggregate breakdown.) 

NIST Response: NIST plans to offer a range of graph types in future versions of BEES. 

- Considering all the room left on most graphic pages, why not show the entire product names 

somewhere (like a legend), in lieu of only showing a shortened version? 

NIST Response: NIST will lengthen product names on the graphs before releasing BEES 3.0. 

- Although the printout of all provided result sheets is very transparent and includes a lot of insights, 

BEES2.0b does not include a sensitivity and uncertainty analysis. This omission is in disagreement 

with ISO 14043. 

NIST Response: At present, incorporating uncertainty analysis is problematic due to lack of 

uncertainty data. For this reason, at its January 2001 meeting, the BEES Critical Review Team 
12 

advised NIST not to incorporate uncertainty analysis into BEES in the short run. NIST will note 

the disagreement with ISO 14043 in its automated help system and printed documentation. In the 

long run, however, one aspect of uncertainty may be addressed: the representativeness of 

industry averages. That is, once the BEES Please program extensively populates the BEES 

database with manufacturer-specific data, the variation of manufacturer-specific data around its 

industry average data will become available. 
13 

7. Modeling 

Several comments were submitted regarding the selection of the modeling approach as well as 

selection of the impact categories that are used within the BEES program. Since these comments 

lead to similar responses, all the comments regarding these issues were grouped together and 

followed by a single response from NIST. 

- Apparently, the impact assessment methodology is set, and not changeable/flexible (excluding 

weighting). What if I wanted to compare two methodologies across a certain set of building 

products? As far as the user knows, the already selected methodology is the best one. With the 

option of them choosing a methodology, they could decide for themselves (or for their business) 

which methodology they think is best. 

 - Regarding selection of the LCIA method on pages 8-10 of the user manual, a review of LCIA 

methods is provided by Dr. Hofstetter, an LCIA expert: 

 The Critical Volumes Approach was indeed used until the early 90s but abandoned now 

because (i) fate and exposure is not considered (ii) the underlying assumption that the residual 

risk at thresholds is the same for all substances does not hold (iii) and, as you note, legal limit 

values are available only for certain chemicals and pollutants. The early applications of this 

approach go back to Jansen et al. (1972) and Basler & Hofmann (1974). 

- The Ecological Scarcity Approach has been updated in the meantime (BUWAL 1998) and has 

also been developed for Sweden, Belgium, The Netherlands, and Germany. If the residual risks 

at the level of target flows would be comparable, then this would be a revealed preference 

approach and fulfill the requirements set by SETAC. However, this assumption is in reality not 

true and therefore a major point of criticism. 

- The EPS has been recently updated, see Steen (1999) 

- Classification/Characterization are the names that have been suggested by SETAC to describe 

the general methodology of LCA. This description applies to most LCIA methods and is 

therefore inadequately used. You may want refer to the “Environmental themes approach” or 

“Environmental problem approach.” Due to the theoretical and practical importance of the 

document CML (1992), you may mention this reference. An update of this method will be 

available soon; major elements are published on the homepage of CML. Drawbacks of this 

method are the unclear environmental relevance of the category indicators that make the final 

weighting step very challenging/difficult. 

- Another widely used method has been ignored in this overview, the Eco-indicator’99 (Goedkoop 

et al. 1999). The main disadvantages are the assessment gaps (e.g., ecosystem impact due to global 
14 

warming) and its focus on Europe. Its improvements in the weighting step may outweigh these 

disadvantages. 

- Based on this evaluation it is not really clear which method fits best. One may want to include 

even more than one method to test the sensitivity of the outcome. 

NIST Response: The ability to compare two impact assessment methodologies is beyond the 

scope of the BEES project. Rather, BEES assesses impacts based on the best available science 

and the methodologies most accepted in the United States.

7.1 Selection of Impact Categories 

 

The only justification for the selection of impact categories was the loose reference on an 

unpublished and non peer-reviewed report (FRED). Neither the environmental relevance of the 

chosen categories nor the irrelevance of the non-chosen categories is shown. Udo de Haes (1996) 

suggests a list of impact categories that may serve as a starting point. 

- The selection of the impact assessment method and the subsequent selection of impact 

categories leads to problems in the interpretation. The weighting step is misleading and the core 

set of 6 impact categories covers only a subset of potentially important environmental problems. 

The ranking of product alternatives based only on these 6 impact categories cannot be 

recommended.

- The assessment models behind the 10 impact categories are in many cases not state-of-the-art 

and need an update (see specific comments). Among the 6 chosen categories, “solid waste 

impacts” and “indoor air quality” appear in addition to what Udo de Haes (1996) or Udo de Haes 

et al. (1999) suggest. “Indoor air quality” is a very relevant impact for the evaluation of buildings 

and was therefore a good addition. Another reviewer added that the consideration of indoor air 

quality is very sensible and meaningful since human beings tend to spend more than 80 % of 

their lifetime in buildings and since the dose rates are higher indoors than outdoors. Further 

research in this area should be encouraged. 

- Solid waste is usually not considered to be an output of the product system. This is because the 

incinerator or landfill is part of the technosphere and only its effluents and resource use 

(including land use) are considered as inventory flows. Therefore, the treatment of waste has to 

be added in the LCI. 

- If we believe that either the SAB or Harvard weighting scheme is applicable to BEES, then 

this would show that the added 4 categories (only included for driveways and parking lots) have 

a total relevance of 42 % and 52 %, respectively. This confirms the correctness of their inclusion 

but also questions the relevance of the results for all other analyzed elements. 

- Based on these major points, one has to assume that results generated with the present version 

of BEES2.0b are not robust against the mentioned biases and the ranking results may be 

methodological artifacts. A pragmatic way to improve the tool would be the immediate use of 
15 

state-of-the-art impact assessment, and establishment of a reporting strategy for updated LCI 

data that will lead to more credible modeling results in the long run. 

NIST Response: The ability to compare two impact assessment methodologies is beyond the 

scope of the BEES project. Rather, BEES assesses impacts based on the best available science 

and the methodologies most accepted in the United States. For version 2.0, the published EPA 

FRED (Framework for Responsible Environmental Decisionmaking) set of U.S. impact 

assessment methodologies met these criteria. There is general agreement that the FRED impacts 

are relevant. FRED has since been eclipsed by the forthcoming EPA TRACI (Tool for the 

Reduction of Chemical and Other Impacts) methodology set, which will be integrated into BEES 

3.0. At its January 2001 meeting, the BEES Critical Review team agreed with this approach. 

NIST does not claim that non-chosen impacts are irrelevant. Rather, the BEES 2.0 

documentation and online help system discuss the two categories for which valid U.S. impact 

assessment methodologies did not exist—land and water use—and gives guidance for 

interpreting BEES results in this context. The new TRACI land and water use impact assessment 

methodologies are being incorporated into BEES 3.0. 

At its January 2001 meeting, the BEES Critical Review Team agreed that modeling the fate of 

solid waste is an important topic for long-term research. Until research yields the understanding 

necessary to develop a more refined assessment methodology for this impact, however, the 

volume of solid waste to the landfill is a satisfactory proxy for evaluating the solid waste impact. 

(Note that in BEES 3.0, the solid waste assessment is further refined through use of the TRACI 

habitat alteration impact.) 

At the time BEES 2.0 was published, the science and data underlying measurement of the added 

four impacts were not as agreed upon and available as that for the original six impacts, and the 

added four impact measures (which are also available for two floor coverings) were qualified 

accordingly in BEES. Indeed, new science will be used to assess most of these added impacts in 

the upcoming BEES 3.0. The BEES 2.0 documentation and online help system give guidance for 

interpreting BEES results based on an incomplete set of impacts. It is better to scientifically 

assess an incomplete set of impacts rather than no impacts at all, thereby allowing users to focus 

their judgment on missing impacts rather than making everything a matter of judgment. The 

issue of updating BEES data to enable assessment of all 10 impacts for all products was raised 

at the January 2001 meeting of the BEES Critical Review Team. Since national standard, 

“backbone” life-cycle inventory data will likely be developed over the next several years, it was 

agreed that NIST should hold off expanding its existing data to cover all 10 impacts until these 

data become available.

7.2 Characterization Models 

- Global Warming: The integration time horizon is neither mentioned nor justified. The 

exclusion of a large number of further greenhouse gases was not mentioned at all (especially, 

CFCs, H-CFCs, N2O). 

NIST response: In BEES 3.0, NIST will mention and justify the 100-year time horizon it uses, as 

is common LCA practice. Life-cycle inventory data collected for BEES 2.0 include the additional 
16 

greenhouse gases cited, but for consistency reasons, NIST chose not to include these gases in its 

global warming assessment until it could update BEES 1.0 inventory data to include them as 

well. BEES 3.0 will evaluate global warming for all products based on the expanded list of 

greenhouse gases. 

- Acidification: Does the word “hydrogen” imply that it is a H+ ? 

NIST Response: Yes. NIST will clarify that these are hydrogen ions in BEES 3.0. 

- Nutrification: Is the value for ammonia correct? Own calculation suggests 0.35. 

NIST Response: Yes, the value for ammonia in water, 0.42 is correct. Note, however, that the 

TRACI eutrophication impact assessment methodology is being incorporated into BEES 3.0, so 

this value will change. 

- Natural Resource Depletion: The reasoning for including resource depletion does not support 

the choice of the model. While it is argued that this is a long-term sustainability issue, it is 

calculated with economically and sub-economically extractable resources rather than a more 

future-oriented measure (see Guinee et al. 1995, Mueller-Wenk 1998, Goedkoop et al. 1999, and 

Steen 1999). It is also unclear why biotic resources including land use are excluded. 

NIST Response: In BEES 3.0, NIST will clarify the discrepancy between selection of impact and 

choice of model. A valid assessment technique for land use was not available for inclusion in 

BEES 2.0. Note, however, that the TRACI habitat alteration impact assessment methodology 

being incorporated into BEES 3.0 addresses land use. 

- Indoor Air quality: It is excellent that this category is added, and due to the lacking state-ofthe-art method, one can understand the particular problems. However, there are some potential 

improvements that may be considered: (i) the unit in Table 2.6 should read mg/m2

/h rather than 

mega-gram, (ii) the chosen key to add emissions from the floor coverings and the installation 

adhesives is unnecessarily wrong. The chosen key implies that 1.667 mg/m2/h at 24 h is the same 

as a total release of 3.2732 kg/m2

. If this were true and if we make the unrealistic assumption that 

the emission rate at 24 h stays stable, then the floor covering has to last 224 years before the two 

emissions are indeed equal. Instead, it is suggested to integrate an exponential function that 

shows after 24 h the given rate and approaches a negligible rate after a couple of years. This 

integrated amount can then be added to the installation adhesives (and will be in general much 

smaller). It is also not clear why it was necessary to add all different VOCs 1:1. Later, in the 

humantox paragraph, a very sophisticated method is chosen to account for a large number of 

VOCs. The toxicological potencies used there (without fate, only for inhalation) could also be 

used for the indoor situation. What is the source for coat reapplication of every 4 years over 50 

years? (This is much more frequent than what I experience and would equal 12 layers, which 

makes it necessary to scrub the walls in-between. This scrubbing is not included.) 

NIST Response: For BEES 3.0, NIST will turn off its automatic (and erroneous in this case) 

capitalization setting within MS Word to correct the error cited in item i. Emissions from floor 

coverings and adhesives are not added together. Rather, because they are expressed in different 

units, they are each first normalized by dividing by the corresponding emissions for the worstperforming product, then the normalized values are averaged. Different VOCs are added 1:1 for 
17 

data availability reasons, not for assessment technique availability reasons. In BEES 3.0, the 

suggested approach for adding emissions from floor coverings and installation adhesives is 

taken. The source for coat reapplication every 4 years is Whitestone Building Maintenance and 

Repair Cost Reference and reflects commercial building practice. In future versions of BEES, 

NIST will address the scrubbing issue. 

- Smog formation: The present text does not provide a MIR for NOx. What is the source for 

footnote 33? How do MIRs compare with the POCPs used by OECD and introduced in CML 

(1992)? 

NIST Response: For BEES 3.0, NIST will be updating its smog assessment by replacing the EPA 

FRED approach with the forthcoming TRACI approach. At the time of BEES 2.0 publication, 

FRED offered the best available science. FRED has since been eclipsed by the forthcoming EPA 

TRACI approach, so NIST will be using the TRACI smog assessment technique, which includes 

an MIR for NOx, in BEES 3.0. The source for footnote 33 is the FRED report. 

- Ecological toxicity: From the given description the source for the method is unclear. Based on 

the provided information and own interpretation the following comments can be made: 

1. Step 1 makes sure that only clear differences between emissions are included. This step 

can be criticized. First, the 15 % minimum spread is only meaningful if the measurement errors 

are larger than that. Second, the effect of this procedure is that the relative ecotoxic difference 

between alternatives will increase. Within the chosen weighting scheme this is entering a bias to 

“overweight” ecotox impacts. 

2. In step 2, it is unclear which benchmarks have actually been chosen. LCA deals with 

optimization beyond compliance. Therefore, acute effects would not occur. 

3. Steps 4 and 5 will lead to a compression of the toxicity factors towards the upper end, 

which makes relative differences between substances smaller than they are. 

4. Steps 6-8: In the meantime, multi-media models have been developed that may produce 

better fate and exposure information than the suggested approach. This is especially true 

because those multi-media results have been used for the human toxicity potentials. 

This first analysis suggests that the chosen approach may be flawed and enter biases. The most 

obvious step would be to combine the fate and exposure information available from Hertwich 

(1999) - the humantox model for TRACI - with the chronic toxicity factors suggested by RTI. 

Considering the fact that the present approach does not include any U.S.-specific data, one may 

also use the data provided by Huijbregts (2000) before TRACI (the U.S.EPA LCIA method) 

provides American factors. 

NIST Response: The source for the ecological toxicity method is the EPA FRED report. NIST 

will be updating its ecological toxicity approach by replacing the FRED approach with the 

forthcoming TRACI approach in BEES 3.0. At the time of BEES 2.0 publication, FRED offered 

the best available science. 

- Human toxicity: This paragraph refers to an unpublished secondary source. It is suggested to 

refer directly to the latest version of the model (Hertwich 1999). This approach does not follow 

the ILSI recommendations because no weighting between different effects was included. The 

suggested equal weighting of carcinogenic and non-carcinogenic effects is unclear. Does this 
18 

mean that Benzene equivalents are added with Toluene equivalents? If this is the case, then it 

implies that those two substances are equally hazardous. A more reasonable assumption would 

be that the used benchmark doses for the two metrics have a similar meaning which allows one 

to add the non-normalized scores. 

NIST Response: In BEES 3.0, NIST will refer directly to Hertwich 1999. No, benzene and toluene 

equivalents are not added. Rather, each is first normalized by dividing by the corresponding 

equivalents for the worst-performing product, then a simple average of the normalized values is 

taken. For BEES 3.0, NIST worked with Dr. Hertwich to obtain the suggested benchmark doses 

for toluene and benzene equivalents, negating the need to normalize toluence and benzene 

equivalents and permitting the suggested adding of non-normalized scores.. 

- The effects of criteria air pollutants on human health are only partly covered. In particular, the 

large impact of fine particles – which may outweigh the impacts of all other pollutants that 

contribute to human toxicity – is fully omitted. It may be most efficient to work together with 

Jane Bare, U.S.EPA, to implement TRACI (Tool for the Reduction of Chemical and Other 

Impacts) in BEES as soon it becomes available. TRACI is supposed to deal with these problems. 

NIST Response: Yes, NIST is currently working with Jane Bare to implement TRACI in BEES 

3.0, including a new Criteria Air Pollutants impact category. At the time of BEES 2.0 

publication, FRED offered the best available science. 

- The offered transport distances from the manufacturer to the user are, in the example of wall 

insulation (I checked only this), probably too short. This would imply that you have a large 

number of manufacturers. 

NIST Response: NIST originally had longer travel distances for insulation. It shortened them 

based on comments received from the insulation industry during its Beta review of BEES 1.0. 

Insulation is such a voluminous product that long travel distances are not cost-effective, and thus 

not common industry practice. 

- The user manual does not document energy and transportation processes – the backbones of 

most LCAs. When you refer to energy use you do not specify the assumed technology. The type 

of burners assumed should be relevant as well. 

NIST Response: Reliable U.S. backbone data are, at present, only available from commercial 

tools. BEES does not publish transparent backbone data because none are publicly available 

and BEES is not intended to compete with private sector tools. Rather, NIST contracts with 

Environmental Strategies and Solutions and PricewaterhouseCoopers/Ecobalance, a world-class 

LCI tool developer, for aggregated backbone data, thus protecting their intellectual property. 

However, publicly available U.S. backbone data will likely be developed over the next several 

years. Once available, NIST will incorporate these more transparent data into future versions of 

BEES. 

- Functional unit: The definition provided on page 6 suggests that the compared alternatives are 

“true substitutes.” However, only footnote 104 gives an idea what “true” means. Due to the 

importance of this point it is suggested that a table be added where for each individual element 

the “true” is specified, i.e., which additional characteristics next to surface and lifetime have 

been adjusted to correct for different properties and which characteristics may still differ and be 

relevant for the interpretation. 
19 

NIST Response: NIST expanded the discussion of functional units in its BEES 3.0 documentation. 

- It should be mentioned that occupational health was excluded from the present analysis. 

NIST Response: NIST will mention the exclusion of occupational health in BEES 3.0. 

- Due to the lack of uncertainty and sensitivity analysis an interpretation in accordance with ISO 

14043 is not possible. 

NIST Response: This issue was discussed at the January 2001 meeting of the BEES Critical 

Review Team. At present, incorporating uncertainty analysis is problematic due to lack of 

uncertainty data. For this reason, the BEES Critical Review Team advised NIST not to 

incorporate uncertainty analysis into BEES in the short run. NIST will note the disagreement 

with ISO 14043 in its automated help system and printed documentation. In the long run, 

however, one aspect of uncertainty may be addressed: the representativeness of industry 

averages. That is, once the BEES Please program extensively populates the BEES database with 

manufacturer-specific data, the variation of manufacturer-specific data around its industry 

average data will become available. 

- The following statement was noted on page 15 of the BEES 2.0 user manual: 

“Equivalency factors have not been developed to consider the ultimate fate of the non-recyclable 

solid waste (e.g., landfill leachate, gas or incinerator emissions, and ash). Thus, the Direct Use 

of Inventories Approach, described at the beginning of this subsection, is used, with solid waste 

volume representing the solid waste impact of the product.” 

Of course, a measure of solid waste generation (volume) alone is not a meaningful indicator of 

the solid waste’s potential to do environmental damage. In particular, an inert waste, such as 

concrete, is much less likely to create measurable environmental problems (leachate, landfill gas, 

ash) as compared to organic waste and metals that tend to undergo complex chemical reactions 

under aerobic, anaerobic, and acidic landfill conditions. This model would tend to either overestimate or under-estimate the importance of the waste, depending on the type of waste. 

Some “solid waste equivalency factors” might be derived from existing research. For example, 

there are probably studies that estimate ash/landfill gas/leachate production by different fractions 

of the waste stream based on chemical stoichiometry. In particular, there are numerous studies 

available on methane generation from solid waste. The Los Angeles County Sanitation District 

and the University of Wisconsin are reportedly at the forefront of this research. 

Other important indicators of solid waste’s impact on the environment include the percentage of 

landfills meeting federally-mandated minimum standards for landfill design and operation 

(including lining requirements and leachate and gas recovery), the existence of state and local 

recycling laws for various construction waste constituents, average rainfall, and state recycling 

rates for specific solid waste fractions. Of course, many “environmental indicators” for solid 

waste would tend to vary by region. As such, BEES programming may be improved by 

incorporating geographical variables in calculating various environmental assessments. It is 

noted that the current goal of BEES is to establish a scope based on U.S.-average results. 
20 

NIST Response: At its January 2001 meeting, the BEES Critical Review Team agreed that 

modeling the fate of solid waste is an important topic for long-term research. Until research 

yields the understanding necessary to develop a more refined assessment methodology for this 

impact, however, an interim approach will be taken in BEES 3.0 by coverting solid waste volume 

to surface area, then evaluating the “impact” of the surface area use with the new TRACI 

habitat alteration impact. 

- The BEES Program allows the user to set a transportation distance parameter to account for 

shipping finished commercial carpet from the manufacturing plant to the project site. However, 

this feature does not appear to have range setting that is appropriate for the West Coast. While 

the highest transportation setting is only 1000 miles, most commercial nylon carpet shipped to 

California originates in Georgia---a significantly longer distance than the 1,000-mile setting in 

the BEES program. It is unclear to the reviewer whether or not this longer transportation 

distance is accounted for in BEES. It may prove necessary to program a wider range of distances 

for the nylon commercial carpet selections to account for the significant transportation impact of 

shipping this product to the West Coast. 

NIST Response: For BEES 3.0, NIST lengthened its assumed transportation distances for carpet 

products. 

- Early on in LCA development it was suggested that organic compounds not be grouped 

together as a single inventory item. BEES still aggregates organic compounds as one group. 

Potential environmental impact will vary depending on the compound. 

NIST Response: This limitation is documented in the BEES manual (pp. 18-20). Total VOCs are 

used to assess indoor air quality because data permitting speciation of individual organic 

compounds are not available. NIST is encouraging manufacturers to begin providing speciated 

data. 

- The flows for a product are reduced to those that have a significant contribution to potential 

impacts. Who made this decision? Was an environmental background required? 

NIST Response: Yes, the NIST LCI data contractors, Environmental Strategies & Solutions and 

PricewaterhouseCoopers/Ecobalance, have world-class environmental backgrounds and 

experience. Moreover, flows having a significant contribution to potential impacts are in fact 

predefined by the state-of-the-art impact assessment methods BEES uses. For all BEES products, 

flows identified by these methods are collected. NIST will clarify its BEES 3.0 documentation to 

this effect. 

- The BEES model only considers ten environmental impacts. There are other impacts that could 

be considered in the process. EPA has evaluated six of the impacts used in the BEES model and 

verified their scientific validity (Global Warming Potential, Acidification Potential, 

Eutrophication Potential, Natural Resource Depletion and Indoor Air Quality). While EPA has 

reviewed these six, two are still somewhat controversial: Natural Resource Depletion and Indoor 

Air Quality. The reader needs to be aware of this disagreement. Also, Ecological Toxicity and 

Human Toxicity still have controversial assessment methodologies. The main users of BEES 

will have a limited background in Environmental Science and the BEES Model simplifies a 
21 

complicated process. The user needs to be made aware that some impact models are still in the 

development stages. 

NIST Response: At its January 2001 meeting, the BEES Critical Review Team agreed that BEES 

should assess the impacts for which EPA’s TRACI is developing state-of-the-art U.S. 

methodologies. These impacts include 8 of the 10 BEES environmental impacts, as well as an 

additional 3 impacts (habitat alteration, water use, and criteria air pollutants). Further, the 

BEES Critical Review Team endorsed the inclusion of indoor air quality. See discussion above 

regarding the inclusion of the solid waste impact. Assessment techniques have evolved such that 

the earlier controversies are now less serious, and while techniques will continue to evolve for 

some time, they are well beyond the development stages. For BEES 3.0, NIST will update its 

Limitations documentation to inform the user of these issues. See discussion above regarding 

other impacts, primarily land use (aka habitat alteration), not included in BEES 2.0. 

7.3 Normalization and Weighting 

There was disagreement among the reviewers regarding the application of weighting to the 

impact categories: 

 

- The weighting system, including the user-definable portion, works very nicely. And the 

checkbox for “No weighting” is well executed. 

- Depending on the location, some environmental categories would carry more weight than 

others. While the Multiattribute Decision Analysis (MADA) is an accepted methodology, it 

should not be applied here. 

NIST Response: NIST does not understand the conflict between the location-specific issue and 

the applicability of MADA. Indeed, the ability to change weights based on location is a strength 

of BEES. 

- The weighting step is not congruent with the modeled category indicators. The resulting 

weighted scores are therefore arbitrary and shall not be used for ranking product alternatives.

Major Response # 1 to MADA Critique 

NIST Response: While NIST agrees that applying the ASTM standard MADA approach is a 

second-best solution, it strongly maintains that at the time of BEES 2.0 publication it was the 

most appropriate approach, given the lack of U.S. normalization data. Indeed, MADA is 

intended as a technique for application in situations in which normalization data are not 

available. Further, the MADA limitations are clearly discussed in the BEES 2.0 online and 

printed Limitations documentation. At a meeting between BEES and TRACI developers following 

the January Critical Review Team meeting, EPA noted that U.S. normalization data would be 

available for BEES 3.0. This will permit BEES to evolve from a relative to an absolute 

measurement system, and will be an important advance that will be incorporated into BEES 3.0. 

- According to Keeney et al. (1976), multi-attribute utility analysis requires utility 

independence. This condition is not strictly fulfilled, as scores for humantox and ecotox and for 

acidification and eutrophication tend to be highly correlated. Therefore, the statement on page 

28 that MADA is the most appropriate technique is not so clear. However, this assumption may 
22 

be considered to be state-of-the-art. As a first step an internal normalization is suggested. 

Indeed, internal normalization is one of the scaling techniques used in MAUT. However, its 

application in the LCA context violates the congruence required between the weighting step and 

the category indicators. 

Major Response #2 to MADA Critique 

NIST Response: MADA does not belong to the family of multi-attribute utility theory (MAUT) 

approaches. Rather, MADA is a more pragmatic approach than the theoretically appealing, yet 

impractical, MAUT techniques. The arguments for MADA versus MAUT are extensive and date 

back to the 1950’s. The arguments against MADA by utility theorists have not “killed” this 

approach. Indeed, MADA is today applied extensively by Fortune 500 companies and national 

governments worldwide. Also, since MADA is not a MAUT approach, it has different conditions. 

In particular, congruence between the weighting step and the category indicators is not 

required. 

- The two provided default weighting schemes from SAB and Harvard both refer to the 

importance of environmental problems if the total threat to the U.S. is considered. This threat is 

primarily caused by U.S. emissions and extractions, plus some imports minus exports. If these 

weights should be applied to the normalized scores of BEES this means that you have to assume 

that first the building material sector causes exact proportional shares of environmental problems 

as observed on a total economy level, and second that the worst indicators of the actuallycompared building components are again proportional to the average emissions and resource 

extractions of the building industry. If we would believe this to be true, then the BEES analysis 

could be dramatically simplified (the analysis of one impact category would be sufficient). The 

use of the SAB and Harvard weights also implies that they correspond with the implied meaning 

of the decision at hand. Building product decisions will change the marginal burden to the 

environment. Therefore, the weights should reflect the importance of increases or decreases of 

the different effects, rather than their present total magnitude. 

NIST Response: This is a specious argument. In BEES, product flows contributing to 

environmental impacts are penalized in proportion to the severity of the impacts from a U.S. 

perspective. NIST is simply assuming a constant marginal damage function. 

-The offer to weight normalized scores equally means in practice that the exchange rate between 

kg CO2-equivalents and kg PO4-equivalents depends on whether I compare wall insulations or 

flooring materials or whether the comparison of flooring materials includes linoleum or not. 

Such a dependence on the number and nature of alternatives is considered to be a flaw of a 

procedure. 

NIST Response: See Major Responses #1 and #2 to MADA Critique above. 

-In the fourth weighting alternative - the setting of user-defined weights - congruence is not 

possible because the user does not know what s/he is weighting. The magnitude of the weighted 

scores is not provided. 

NIST Response: See Major Response #2 to MADA Critique above. 
23 

- Von Neumann and Morgenstern (1944) introduced some conditions for expected utility theory 

and included the condition (among others) of transitivity. As you admit on page 35 ‘rank 

reversal’ occurs in the chosen method. Therefore, one of the most basic conditions for the 

expected utility theory is violated and the approach flawed. 

NIST Response: See Major Response #2 to MADA Critique above. Transitivity is not required by 

MADA. Indeed, while utility theorists argue that people’s decisionmaking must be absolutely 

consistent, MADA theorists argue that absolute consistency is neither necessary nor desirable. 

- Based on the above, the BEES normalization procedure has to be changed to establish 

congruence between the weights and the category indicators. The most common approach uses 

normalization values that reflect the contribution of a certain region during a defined time to the 

problems (environmental impacts) at hand. However, the congruence criterion requires that the 

actually-used reference system be reflected, which may require different normalization sets for 

the different suggested weighting sets. 

- Jane Bare (U.S. EPA) may be able to provide U.S. data for the normalization step that are 

being developed for her TRACI project. 

NIST Response: See Major Responses #1 and #2 to MADA Critique above. 

- Hofstetter et al. (2000) provide a graphical method that helps to support decisions when tradeoffs have to be made with a minimum of weighting information. (In some cases the 

environmentally-best alternative is also the one with the lowest LCC. In such cases, the user 

should know that the weighting of costs and environmental burden have no relevance to the 

ranking outcome). 

- The transformation of the SAB and Harvard verbal importance rankings to weights on cardinal 

scales includes a lot of assumptions that are not made explicit. The AHP process is controversial 

and the interpretation in this case arbitrary. Why didn’t you choose factors of 3 and 7? It has to 

be assumed that members of both panels would discourage the use of these weights. 

NIST Response: The factors were chosen through an iterative process based on NIST’s 

background and experience in applying the AHP pairwise comparison process. The pairwise 

comparison process is intended for translation of verbal importance rankings to numerical 

importance weights, so panel member support, while desirable, is not a prerequisite. Also note 

that BEES permits users to adjust any of the weights. 

- The rounding of values used in Tables 2.13 and 2.15, and for the equal weights, is appropriate 

with regard to its uncertainty but is very confusing because the equal judgments on some 

problems are not evident. 

NIST Response: For BEES 3.0, NIST will clarify the rounding of weights. 
24 
25 

8. Data and Data Sources 

-A strong feature of the tool is that it includes a reasonably large set of LCI data. This is of 

course the (expensive) heart of the tool. The quality of the data was not reviewed due to the 

lacking documentation. 

- The presented alternatives for individual building elements may cover the most frequently-used 

products, but do not include eco-leaders. This means that additional alternatives, like highlyinsulated walls (R-value higher than R-25 or R-30) and roofs, should be included. Also, unpaved 

parkways that allow retention of water should be added as viable alternatives (these are small 

concrete blocks that protect soil from compression but allow for vegetation). Only the addition of 

these eco-leaders may motivate designers to go for a better alternative. Eco-leaders should 

always appear as benchmarks in comparisons, i.e., even if not selected. 

Major Response to Data Suggestions 

NIST Response: To date, BEES products have been selected on the basis of sponsor interest, 

intriguing product comparisons, and extensive building element coverage. It is not possible to 

identify eco-leaders a priori because weights can change and tradeoffs among multiple impacts 

are not always as expected. Presumably, the BEES Please program will encourage perceived 

eco-leading manufacturers, including those marketing the products cited above, to submit their 

products for inclusion in BEES. Further, the incorporation of U.S. normalization data into BEES 

3.0 will offer more consistent benchmark comparisons, because eco-leaders will change based 

on user-defined weights. 

R-11 through R-15 wall insulation do not really motivate environmentally-relevant 

improvements. Such building practice means that the buildings are using far too much energy in 

the use phase for another 50 years to100 years! What are the assumptions on the remaining wall 

structure when you calculate the difference in heating use? By setting future costs due to energy 

supply to zero for the best insulation material, you underestimate the effective LCC and give 

little incentive to reduce total LCC. 

NIST Response: See Major Response to Data Suggestions above. BEES assumes identical 

remaining wall structures when evaluating use energy differences among insulation alternatives. 

However, as mentioned in the Future Directions section of the BEES documentation, in future 

versions of BEES, NIST intends to begin combining building products to permit comparative 

analyses of entire building components and assemblies, a necessary first step permitting 

analyses of different wall structures in combination with insulation alternatives. The best 

insulator’s use energy is set to zero in line with BEES’ design as a product selection tool. 

Otherwise, BEES would be overestimating the effect on use energy of insulation product choices 

by penalizing insulation products for the building’s entire heating load. 

- It is mentioned on page 8 of the user manual that the data have been checked by industry. Is 

this review documented? Based on which information were the data checked? This could be

helpful when the whole database is reviewed in the future. 
26 

NIST Response: Industry reviewers were all sent Beta versions of BEES 2.0 and its 

documentation. NIST resolved and incorporated valid industry comments, involving both LCI 

assumptions and models, and in some cases involving face-to-face meetings, before publishing 

BEES 2.0. For confidentiality reasons, and to encourage future industry participation in the 

BEES project, it is not possible to publish written industry comments. However, as noted in its 

BEES Please announcement, NIST plans to automatically update its LCI data periodically, and 

invites industry to submit earlier updates as technology changes warrant. 

- Cement production is known for high heavy metal emissions. Are they reported? The 

production of nylon is supposed to have significant N2O emissions. Are they included? 

NIST Response: Heavy metals are reported and evaluated in the ecological and human toxicity 

assessments included for parking lot paving alternatives. The other concrete alternatives 

evaluate only 6 impacts and so do not include these impacts. Yes, nitrous oxide emissions from 

nylon production are reported and included in the BEES global warming assessment. 

- The information provided on the LCI data is insufficient to check whether the analysis is in 

accordance with ISO 14041. Therefore, the transparency criterion is violated. 

Chair Note: It has already been stated at the beginning of this review document that the 

reviewers did not inspect data; the review is limited to the modeling and presentation of results 

Major Response to Transparency Comment 

NIST Response: Reliable, transparent U.S. LCI data are, at present, only available from 

commercial tools. BEES does not publish fully transparent data because none are publicly 

available and NIST does not intend to compete with private sector tools by placing these data in 

the public domain. Rather, NIST contracts with Environmental Strategies and Solutions and 

PricewaterhouseCoopers (PwC)/Ecobalance, a world-class LCI data developer, for aggregated 

LCI data, thus protecting PwC’s intellectual property as well as manufacturers’ trade secrets. 

The PwC commercial LCI data product, DEAM, from which much of the BEES “backbone” 

data are taken, strictly follows the ISO 14041 transparency criterion. Users seeking full 

transparency are free to purchase DEAM for complete documentation on its upstream data 

sources and calculation procedures. NIST seeks to provide a public service by making available 

at no charge LCI data that are transparent in an aggregated sense. However, once U.S. 

backbone LCI data become available in the public domain, NIST will readily incorporate these 

fully transparent data into BEES. Finally, NIST’s transparency concern is primarily focused on 

permitting the user to reproduce all the BEES scores by providing the algorithms and all the 

aggregated data that are used in them. 

- Cut-off rules: Figure 2.1 provides a good overview of the chosen cut-off criteria. The chosen 

criteria do not address the low-weight, low-energy, low-cost inventory flows that have a high 

environmental contribution. Processes with high (fugitive) process-specific emissions tend in this 

framework to be excluded (a violation of cut-off criteria in ISO 14041) 

NIST Response: The BEES contracting team does not agree with this strict interpretation of the 

ISO cut-off rules, and thus does not believe the BEES cut-off rules violate ISO 14041. Section 

5.2.4 (Criteria for initial inclusion of inputs and outputs) in ISO 14041 seems to generally 

support the cut-off rules described under Section 2.1.1, Goal and Scope Definition, of the BEES 

Technical Manual. ISO 14041 states: 
27 

“... During the scope definition, the initial set of inputs and outputs are selected for the inventory. This 

process recognizes that it often not practical to model every input and output into the product system. It is 

an iterative process to identify the inputs and outputs which should be traced to nature, i.e. which unit 

processes producing the inputs or which unit processes receiving the outputs, should be included in the 

product system under study. The initial identification will typically be made using available data, and 

inputs and outputs should be more fully identified when additional data are collected during the course of 

the study, and subject to a sensitivity analysis. ... The criteria and assumptions on which they are 

established shall be clearly described. . .... There are several criteria that are used in LCA practice to 

decide which inputs will be studied, including 1) mass, 2) energy, and 3) environmental relevance. Making 

the initial identification of inputs based on mass contribution alone may result in important inputs being 

omitted from the study. Accordingly, energy and environmental relevance should also be used as criteria 

in this process.” 

In BEES, energy and cost (a proxy for environmental relevance) criteria are used in addition to 

the mass criterion. Finally, the cut-off rules used in BEES cover the majority of flows; the 

example raised by the peer reviewer is more the exception than the rule. With additional funding, 

NIST could refine its cut-off rules to include sensitivity analyses to determine which flows are the 

most environmentally relevant. 

- One of the most significant limitations of the BEES Program is that it does not incorporate a 

number of commonly-specified generic building products that collectively offer numerous 

opportunities to incorporate green materials. Porous paving products, recycled-content aggregate 

base and sub-base, recycled asphalt concrete, rubberized asphalt paving, exterior cladding, 

curtain wall systems, re-locatable interior wall systems, toilet partitions, plastic lumber, 

acoustical ceiling tile, standard gypsum board, synthetic gypsum board, high impact gypsum 

board, finished decorative wall panels, and structural engineered wall systems are among the 

generic material and product classifications that are not addressed in BEES 2.0. The reviewer 

notes that the National Institute of Standards and Technology (NIST) has already recognized this 

limitation and will address the need to expand the product selections in future work. 

NIST Response: Agreed. Collecting BEES LCI data is a very expensive undertaking for NIST. 

NIST asks for patience as it continues to incorporate these important generic products into 

future versions of BEES. 

 

- Another important limitation of the BEES software is that the results mostly reflect industry 

averages compiled for generic product types. Professional designers and other decision-makers 

may consider BEES a useful tool for evaluating very basic material selections made during the 

conceptual design and design development phases of a project. The software can identify 

relevant cost and environmental distinctions between these generic material/product 

classifications. 

Major Response to Industry-Average Data Limitation 

NIST Response: To address this limitation, NIST has already launched its BEES Please program 

soliciting brand-specific LCI data from manufacturers for incorporation into BEES 3.0. 

- Design professionals often need product-specific information during the bidding phase of the 

project to educate the building team and building owners about green material choices and to 

process substitution requests for green building materials. This is important because significant 

differences in both cost and environmental performance characteristics often exist between many 

individual products with the same application. And it is clear that environmental differentiation 
28 

among individual products is becoming a trend. This trend is market-driven and measured by the 

growing number of companies that produce green products to meet the demand for green 

building and green construction. It is noted that the BEES manual addresses the limitations 

related to the lack of individual product selections and has already identified the need to “add 

more products to currently covered elements.” To illustrate the point made above, an interior 

designer evaluating flooring options might use BEES to compare conventional vinyl composition 

tile with ceramic tile. However, if the interior designer selects the VCT flooring option, BEES 

does not permit the user to evaluate product choices that fall within the “VCT classification” or 

reasonable alternatives to VCT based on the performance, cost, and application requirements. 

For instance, the user cannot compare standard VCT with an alternate resilient flooring product 

made from a chlorine-free, mineral-filled ethylene co-polymer backing (made by Amtico). The 

latter product has relatively low VOC emissions (4 week rating is < 10 ug/m2

/hr) and is 

reportedly twice as resistant to abrasion than VCT. The indications are that the latter product has 

significantly better environmental performance than conventional VCT. By the same token, the 

development of detailed product selections in the BEES Program would be very beneficial for 

differentiating cost and environmental performance within the generic “nylon commercial carpet 

classification.” There are only four general selections available in BEES for nylon commercial 

carpet: a) nylon carpet tile with traditional glue, b) nylon carpet tile with low-VOC glue, c) nylon 

broadloom carpet with traditional glue, and d) nylon broadloom carpet with low-VOC glue. 

More detailed product selections are warranted to incorporate critical performance criteria for 

indoor air quality, material re-use, recycled content, carpet recycling and product design life. To 

illustrate this point, the reviewer proposes more detailed nylon carpet specifications, which are 

listed below: (1) For both broadloom and tile, nylon carpet with integral backing, installed with 

low-VOC glue, certification that product meets CRI emission guidelines for indoor air quality, 

certification that recycled content is at least 45 % by total product weight, warranty for surface 

wear not more than 10 % by mass fraction in 15 years, and certification that manufacturer has 

carpet reclamation program in place to provide for recycling at end of its useful life and (2) For 

both broadloom and tile, refurbished nylon carpet with integral backing, installed with low-VOC 

glue, certification that product meets CRI emissions guidelines for indoor air quality, warranty 

for surface wear not more than 10 % by mass fraction in 10 years, and certification that 

manufacturer has carpet reclamation program in place to provide for recycling at end of its useful 

life. 

NIST Response: See Major Response to Industry-Average Data Limitation above. 

- The BEES Program includes a number of product choices for reinforced concrete applications, 

including foundations, basement walls, concrete beams, and columns. For most of these building 

elements made from concrete, the primary selections are for concrete materials having varying 

levels of fly ash content by weight of Portland cement. In the opinion of the reviewer, the BEES 

Program would benefit from additional selections for structural and non-structural concrete 

building elements, in particular pre-cast concrete, cast-in-place concrete, autoclaved aerated 

concrete, and concrete containing recycled aggregates. For example, a comparison of pre-cast 

versus cast-in-place products (e.g., for support columns, structural beams, concrete pipe, and 

other products in the same generic classification) would disclose some interesting environmental 

performance tradeoffs, in particular differences resulting from the need to construct temporary 

formwork (required for cast-in-place), generation of construction waste (less in pre-cast), 

transportation energy impacts (probably higher in pre-cast), weight-to-strength ratio (some pre-
29 

cast products are pre-stressed) and potential fly ash content (use of high volume fly ash content is 

probably more suitable in cast-in-place applications). The BEES approach would be useful in 

helping design professionals understand how these differences coalesce in terms of overall 

environmental performance and cost. 

NIST Response: Agreed. Collecting BEES LCI data is a very expensive undertaking for NIST. 

NIST asks its BEES users for patience as it continues to incorporate these important generic 

products into future versions of the tool. 

- Environmental assessments for specific environmental impacts can vary significantly 

according to regional conditions and circumstances. However, as stated in the BEES Manual, 

“the goal of BEES LCA assessment is to generate U.S. average results.” In the opinion of the 

reviewer, this is not a desirable simplification because it limits the usefulness the BEES results. 

(The goal is probably due to data limitations and the complexity of programming the LCA 

models). The reviewer notes that the BEES manual identifies a goal to “refine all data to permit 

U.S. region-specific BEES analyses” as a proposed data enhancement for future development. 

NIST Response: Yes, to date, data availability and collection cost limitations have been the 

basis for the U.S. scope. However, as noted, NIST intends to refine all data to permit U.S. 

region-specific analyses in future versions of BEES. 

- There is an apparent discrepancy between the BEES results and actual California-specific 

project cost information for incorporating fly ash in concrete for various building elements, such 

as foundations, paving, and basement walls. For example, the BEES results indicate that 

foundations made from fly ash are significantly less expensive on a first cost basis than concrete 

foundations containing no fly ash. Unfortunately, the BEES results may not reflect the true cost 

of high-volume fly ash foundations (and probably other concrete building elements) constructed 

in California. This observation is based on case study information known to the reviewer. For 

example, a 6116 m3

 (8,000 yd3), 40 % fly ash slab foundation poured in Sacramento costs $2.63/ 

m3

 ($2/yd3) more than the projected cost for the same foundation without fly ash. The exact 

reasons for the California cost discrepancy are unclear, but may be partly attributable to the fact 

that fly ash from coal-fired power plants is not a locally available resource in California. This 

discrepancy suggests that the BEES Program may benefit from additional cost research in this 

area. If a significant regional cost anomaly for fly ash is documented, the problem could be 

corrected by creating a variable in the BEES program to allow the results to reflect regional cost 

differences in this classification. The reviewer realizes that the current goal of BEES is to 

establish a scope based on U.S. average results. 

NIST Response: Yes, local availability most likely drives relative first costs for fly-ash-content 

versus 100 % cement- content concretes. When refining environmental data to permit U.S 

region-specific analyses as noted above, NIST will also refine economic data to account for 

regional price differences . 

- Over 90 % of the data used in the assessments come directly from industry sources. Since a 

majority of LCA databases are European, how was the conversion to U.S. operations done? 

NIST Response: For about 99 % of the BEES LCI data, conversion from European LCI 

databases was not necessary. From the very beginning of the BEES project, the NIST LCI data 
30 

contracting team, Environmental Strategies & Solutions and 

PricewaterhouseCoopers/Ecobalance, has contacted appropriate U.S. industry representatives 

for each and every product incorporated into BEES. Based on input from these U.S. contacts, the 

team conducts new research for each product permitting development of gate-to-grave LCI data 

sets based primarily on U.S. industry data. In addition, the expert contracting team applies 

Ecobalance’s commercial U.S. LCI data tool, DEAM, to develop much, if not all, of the cradleto-gate LCI data that are then combined with the gate-to-grave industry data to derive complete, 

cradle-to-grave BEES LCI data sets. Ninety percent of the cradle-to-gate DEAM data come 

directly from industry sources. Industry representatives are again contacted, this time by NIST, 

once LCI data for their products have been incorporated into BEES, for a second opportunity to 

provide input before LCI data for their products are published in BEES. While this is an 

expensive route to take, it is the only way to ensure data quality. 
31 

9. User Support 

- If someone needs to ask the BEES developers a question (or supply a good idea for a future 

version), it would help if contact info (email, phone) were placed either in the “About...” 

window, or in a separate “Technical Support” - type window. 

NIST Response: NIST will add contact information to its BEES 3.0 documentation. 

After 1 month of ordering the material, it has still not arrived. 

NIST Response: At the time of the peer review, there was an extensive delay in shipping the 

printed materials. It has since been corrected. Our apologies. 
32 
33 

10. User Manual 

- The BEES Manual (page 15) does not state which specific solid waste fractions (generated 

from various stages and segments of the building industry) are considered “non-recyclable.” In 

fact, a material that is deemed “non-recyclable” in one region (perhaps due to lack of processors, 

low tipping fees, no mandatory recycling laws) may be considered “recyclable” in another region 

with more favorable conditions. This information needs to be clarified in the BEES manual. 

NIST Response: In future versions of BEES, NIST will add a table specifying the U.S. average 

non-recyclable fractions used, and may parameterize this factor to accommodate regional 

variations. 

- Throughout the manual comments are made about the limitations of the model. These need to 

be strong statements. Even though the limitations are mentioned, will the user consider them in 

their decision process? The model is easy to use, just point and click. Will the user read the 

manual or just go to the tutorial and learn how to use the tool and start their assessments? 

- The comment made above also applies to the assumptions made in each LCA. Will the user 

determine if their process is similar to the assumptions made and then continue with the analysis, 

or will the user just start using the model and not consider the assumptions? 

NIST Response: Limitations are also clearly given in the software to inform users not inclined to 

read the manual: a maximized help window with the Limitations discussion clearly marked 

automatically opens each and every time the user opens BEES. NIST is open to suggestions as to 

how to further address this issue. In addition, the BEES Please program for incorporating 

brand-specific data will help by accounting for variation among products within an industry. 
34 
35 

11. Other Recommendations 

All suggestions that would lead to a broadening of the user group should lead to separate 

software tools rather than a “one size fits all” tool that is as difficult to use as the commercial 

LCA software on the market. 

NIST Response: Agreed. 
36 

Biographical Sketches of Reviewers 

Jonathan Overly, University of Tennessee

M.S. and B.S., Engineering Science, University of Tennessee. As a researcher for the Center 

for Clean Products and Clean Technologies, Jonathan applies his knowledge of the energy, 

environmental, and production aspects of manufacturing to diverse projects involving 

reduced energy use, pollution prevention, and environmental impact assessment. Before 

joining the Energy, Environment and Resources Center, Jonathan spent four years working 

with the Industrial University of Tennessee-Knoxville, performing energy, environmental, 

and productivity assessments for manufacturing facilities in Tennessee and five surrounding 

states. Through his work at the Industrial Assessment Center, Jonathan visited 35 

manufacturing facilities, spanning nearly all of the manufacturing Standard Industrial 

Classification groups. 

Patrick Hofstetter, Ph.D., ORISE Research Fellow at the U.S. EPA

1989, master degree in mechanical engineering ETH Zürich; 1989 LCA of light bulbs 

including an evaluation of methods for LCIA; 1989-1999 own consultancy "Büro für 

Analyse & Ökologie" in the field of industrial ecology, processes for waste treatment, LCA, 

and eco-design. 1990-1994 research for the report and database "Ökoinventare für 

Energiesysteme" (Life Cycle Inventories for Energy Systems), at the Energy Systems 

Laboratory, ETH Zürich. 1994-2000 member of the LCA-Steering Committee of SETACEurope; 3. - 6. 1994 research stay at the Centruum voor Milieukunde (CML), University of 

Leiden; since 1994 research on Life Cycle Impact Assessment. 1998 Ph.D. thesis on 

"Perspectives in Life Cycle Impact Assessment; A Structured Approach to Combine Models 

of the Technosphere, Ecosphere, and Valuesphere" at the Chair of Environmental Sciences: 

Natural and Social Science Interface, ETH Zürich, since 9. 1999 ORISE Research Fellow at 

U.S. EPA, Cincinnati (Systems Analysis Branch, National Risk Management Research 

Laboratory), since October 1999 Visiting Scientist at Harvard School of Public Health, 

Boston. 

Rick Muller, California Integrated Waste Management Board

Integrated Waste Management Specialist in the Waste Prevention and Market Development 

Division, Recycling Technologies Branch, Sustainable Building Section 

Functional Assignments: Newsprint -- Certification 

 Regulations -- Newsprint 

 Private ‘Buy Recycled’ Programs 

 Construction Products 
37 

References 

Basler & Hofmann Ingenieure und Planer, Studie Umwelt und 

Volkswirtschaft, Vergleich der Umweltbelastung von Behältern aus PVC, 

Glas, Blech und Karton, Eidgenössisches Amt für Umweltschutz (BUS), Bern 

1974 

BUWAL, Methode der ökologischen Knappheit - Ökofaktoren 1997, 

Schriftenreihe Umwelt Nr.297, ÖBU/BUWAL Bern 1998 

CML, Environmental Life Cycle Assessment of Products: Background, Leiden, The 

Netherlands, October 1992. 

Goedkoop M. and Spriensma R. The Eco-indicator’99: A damage oriented method for Life 

Cycle Impact Assessment, VROM Zoetermeer, Nr. 1999/36A/B, 2nd edition April 2000 

(http://www.pre.nl) 

Guinée J., Development of a Methodology for the Environmental Life Cycle 

Assessment of Products with a Case Study on Margarines, PhD Thesis, 

Utrecht 1995 

Hertwich EG. (1999). Toxic Equivalency: Accounting for Human Health in 

Life-Cycle Impact Assessment. Berkeley, CA: University of California. 

Hofstetter P., Braunschweig A., Mettier Th., Mueller-Wenk R. and Tietje 

O. (2000). The mixing triangle: Correlation and graphical decision 

support for LCA-based comparisons. Journal of Industrial Ecology, 

3(4):97-115 

Huijbregts MAJ. (1999). Priority Assessment of Toxic Substances in the 

Frame of LCA: Development and Application of the Multi-Media Fate, 

Exposure and Effect model USES-LCA, University of Amsterdam, 

http://www.leidenuniv.nl/interfac/cml/lca2/index.html (11/10/00). 

ISO Environmental Management--Life-Cycle Assessment—Life Cycle Impact Assessment, 

International Standard 14042, 2000. 

ISO Environmental Management--Life-Cycle Interpretation—Life Cycle Impact Assessment, 

International Standard 14043, 2000. 

Jansen P., Jordan S, Schikarski W., Vergleichende Modelltheorie der 

atmosphärischen Schadstoffbelastung durch Kernkraftwerke, Vortrag 

gehalten am 10. Colloque IRCHA sur les atmosphères polluées, 3.-5- Mai 

1972 in Paris, zitiert in Unger H. Atomkraftwerke. Eine 

umweltfreundliche Energieerzeugung in Alt C., Weber F. (Hrsg.) 
38 

Reinhaltung der Luft, Karlsruhe 1973 

Keeney R.L., Raiffa H., Decisions with Multiple Objectives: Preferences 

and Value Tradeoffs, New York 1976 

Lippiatt, Barbara C., “BEES 2.0,” documented in BEES 2.0: Building for Environmental and 

Economic Sustainability Technical Manual and User Guide, NISTIR 6520 (Gaithersburg, 

MD: National Institute of Standards and Technology, June 2000). 

Müller-Wenk R., Depletion of Abiotic Resources Weighted on Base of 

'Virtual' Impacts of Lower Grade Deposits Used in Future, 

IWÖ-Schriftenreihe Nr. 57, St. Gallen 1998 ( 

http://www.unisg.ch/iwoe/index.html) 

Steen B., A systematic approach to environmental priority strategies in 

product development (EPS). Version 2000, CPM Report 1999:4 and 5, CPM, 

Chalmers University, Göteborg 1999 

Udo de Haes, H.A. (ed.), Towards a Methodology for Life Cycle Impact 

Assessment, SETAC-Europe Working Group on Impact Assessment, Brussels 

1996 

Udo de Haes, H.A et al., “Best Available Practice Regarding Impact Categories and Category 

Indicators in Life Cycle Impact Assessment – Part 1,” International Journal of Life Cycle 

Assessment. Vol. 4, pp. 66-74, 1999. 

Udo de Haes, H.A et al., “Best Available Practice Regarding Impact Categories and Category 

Indicators in Life Cycle Impact Assessment – Part 2,” International Journal of Life Cycle 

Assessment. Vol. 4, pp. 167-174, 1999 

United States Environmental Protection Agency, Framework for Responsible Environmental 

Decisionmaking (FRED): Using Life Cycle Assessment to Evaluate Preferability of Products, 

by Science Applications International Corporation, Research Triangle Institute, and 

EcoSense, Inc, EPA 600/R-00/095, 2000. 

Von Neumann J & Morgenstern O. (1943). Theory of games and economic 

behavior. Princeton: Princeton University Press, (3rd edition and second 

printing by Science Editions, John Wiley & Sons, New York, 1967). 

Whitestone Research, The Whitestone Building Maintenance and Repair Cost Reference 

1999, 5th Annual Edition, Seattle, WA, 1999.