Planning Report 09-1 

Retrospective Economic 

Impact Assessment of 

the NIST Combinatorial 

Methods Center 

Prepared by:
 

RTI International
 

for
 

National Institute of 

Standards & Technology 

April 2009 
April 2009 

Retrospective Economic Impact 

Assessment of the NIST 

Combinatorial Methods Center 

Final Report 

Prepared for 

Gregory Tassey 

National Institute for Standards and Technology 

100 Bureau Drive, Mail Stop 1060 

Gaithersburg, Maryland 20899-1060 

Prepared by 

Alan C. O’Connor 

Howard J. Walls 

Dallas W. Wood 

Albert N. Link 

RTI International 

3040 Cornwallis Road 

Research Triangle Park, NC 27709 

RTI Project Number 0210921.000 
Contents 

Section Page 

Executive Summary ES-1
 

1. Study Introduction 1
 

1.1 Overview of Polymers Science.................................................................................1-2
 

1.2 Traditional Methods of Polymers Research .............................................................1-5
 

1.3 Introduction of Combinatorial Methods to Polymers Science...................................1-6
 

1.3.1 Key Concepts in Combi ................................................................................1-7
 

1.3.2 Technical Challenges to Applying Combi in Polymers Research .................1-9
 

1.4 NIST’s Entry into Combi: Addressing Technical and Market Barriers ....................1-10
 

1.5 Report Organization ...............................................................................................1-11
 

2. Overview of the NIST Combinatorial Methods Center 2-1
 

2.1 NCMC’s Consortium Approach ................................................................................2-1
 

2.2 NCMC Structure .......................................................................................................2-2
 

2.2.1 NCMC Staff and Operations .........................................................................2-2
 

2.2.2 General Membership ....................................................................................2-3
 

2.2.3 Focus Projects ..............................................................................................2-4
 

2.3 Nontechnology Outputs............................................................................................2-6
 

2.3.1 Human Capital Development: Postdoctoral and Hosted Researchers .........2-6
 

2.3.2 Conference and Symposium Sponsorship ...................................................2-7
 

2.3.3 Papers and Proceedings ..............................................................................2-7
 

3. NCMC Technical Accomplishments 3-1
 

3.1 Combi Using Continuous Gradient Approaches for Thin Polymer Films ..................3-1
 

3.2 Combi Using Robotics for Discrete Sample Libraries ..............................................3-4
 

3.3 Combi Using Microfluidics Libraries .........................................................................3-6
 

3.4 Informatics for Combi ...............................................................................................3-7
 

4. End Users, Industries, and Applications 4-1
 

4.1 Advanced Materials Industry ....................................................................................4-2
 

4.2 Paints, Coatings, and Adhesives Industry ................................................................4-3
 

iii 
5-1 

4.3	 Personal Care and Household Products Industry ....................................................4-5
 

4.4	 Academic and Government Research Laboratories ................................................4-5
 

4.5	 Equipment and Software Vendors............................................................................4-6
 

5.	 Study Methodology 

5.1 Conceptualizing NCMC’s Role and Economic Benefits ...........................................5-1
 

5.2 Theoretical Approach to Estimating Economic Benefits ...........................................5-2
 

5.2.1 Modeling Economic Impacts of the NCMC’s Contributions to Combi...........5-2
 

5.2.2 Accelerating Combi Adoption .......................................................................5-4
 

5.3 Specifying the Counterfactual Scenario without the NCMC .....................................5-6
 

5.4 Taxonomy of Quantifiable Economic Benefits and Costs ........................................5-7
 

5.4.1 Value of Consortium Experience..................................................................5-7
 

5.4.2 Value of Knowledge Base Expansion.........................................................5-10
 

5.4.3 Value of Technology Adoption and Adaptation ..........................................5-11
 

5.4.4 Technology Acquisition Costs ....................................................................5-12
 

5.4.5 NCMC and Tool Development Costs .........................................................5-12
 

5.5 Time Period of Analysis..........................................................................................5-13
 

5.6 Primary Data Collection..........................................................................................5-13
 

5.7	 Extrapolation to National Impact Estimates............................................................5-14
 

5.8	 Measures of Economic Return ...............................................................................5-15
 

6.	 Net Economic Benefit Estimates 6-1
 

6.1 Respondent Profile and Combi Usage History .........................................................6-2
 

6.2	 Economic Benefits of the NCMC Consortium Experience .......................................6-4
 

6.3	 Economic Benefits of NCMC’s Expansion of the Combi Knowledge Base ..............6-7
 

6.4	 Economic Benefit of NCMC Combi Technology and Accelerating Combi’s 

Adoption ...................................................................................................................6-9
 

6.5	 NCMC Program Costs and End User Technology Acquisition Costs ....................6-12
 

6.6	 Summary Economic Benefit Estimates and Measures of Return ...........................6-14
 

7.	 Concluding Remarks on the Effectiveness of the NCMC Consortium Model 7-1
 

References	 R-1
 

Appendix A: List of Firms Comprising Extrapolation Base for National Impact 

Estimation A-1
 

iv 
Appendix B: Member Survey B-1
 

Appendix C: Nonmember Survey C-1
 

v 
Figures 

Number Page 

1-1. Combinatorial Workflow Cycle ........................................................................................1-7
 

1-2. Discrete Sample, Continuous Gradient Thin Film, and Microfluidics Libraries ...............1-8
 

2-1. Yearly Citation Analysis for Major NCMC Publications ..................................................2-9
 

3-1. Gradient Thin-Film Sample.............................................................................................3-2
 

3-2. Library of Epoxy Droplets with Varying Concentration of Epoxy Resin Generated 

by Computer-Controlled System (Robot) .......................................................................3-5
 

3-3. Failure Map for Varying Epoxy Concentration................................................................3-5
 

3-4. Microfluidics Device for Making Organic Phase Droplets ...............................................3-7
 

3-5. Percentage Conversion of Monomer in a Dental Material ..............................................3-8
 

4-1. Simplified Flow of Benefits through the Combi-Enabled Value Chain ............................4-2
 

5-1. Firm-Level Acceleration of Costs and Benefits ...............................................................5-5
 

5-2. NIST’s Acceleration Effect on Combi Adoption ..............................................................5-6
 

6-1. Survey Participation as a Proportion of Total Industry Publicly Reported R&D 

Expenditures...................................................................................................................6-2
 

6-3. Estimated FTEs Developing and Applying Combi in the Advanced Materials, 

Coatings, and Personal Care and Household Products Industries ................................6-5
 

vi 
Tables 

Number Page 

1-1. Common Office Items Containing Synthetic Polymers ...................................................1-4
 

2-1. NCMC Semiannual Meetings: Topics and Private-Sector Attendance Counts ..............2-4
 

2-2. Chronology of NCMC Membership .................................................................................2-5
 

2-3. NCMC-Sponsored or -Organized Symposia at Major Scientific Meetings .....................2-8
 

2-4. List of Five NCMC Papers Most Frequently Cited in the Literature ................................2-9
 

3-1. NCMC Platforms for Library Creation .............................................................................3-2
 

3-2. Materials Property Measurement for NCMC Library Platforms ......................................3-3
 

4-1. Descriptive Statistics for Key Combi-Adopting Industries, 2006 .....................................4-4
 

4-3. Combi Equipment and Software Vendors ......................................................................4-9
 

5-1. Summary Taxonomy of Quantifiable Benefits, Acquisition Costs, and NIST Costs .......5-8
 

5-2. Benefits Metrics to Capture the Value of the Consortium Experience............................5-9
 

5-3. Benefits Metrics to Capture Value of Knowledge Base Expansion ..............................5-11
 

5-4. Benefits Metrics to Capture the Value of Technology Adoption ...................................5-12
 

6-1. Respondents’ Combi and HTE Use................................................................................6-3
 

6-2. Timing of Respondents’ Combi Adoption .......................................................................6-3
 

6-3. NCMC Members’ Perceptions of the Benefits of Membership .......................................6-5
 

6-4. Economic Benefits of the NCMC Consortium Experience..............................................6-7
 

6-5. Respondents’ Adoption and Awareness of NCMC Combi Technology ..........................6-8
 

6-6. Economic Benefits of Expanding the Combi Knowledge Base ......................................6-9
 

6-7. Respondents’ Perceptions of Efficiency and Effectiveness Gains from NCMCDeveloped Combi Technology .....................................................................................6-10
 

6-8. NCMC Acceleration of and Estimated Productivity Gain from Combi ..........................6-11
 

6-9. Acceleration and Adoption Benefits of NCMC Combi Technologies ............................6-12
 

6-10. NCMC Program and Technology Acquisition Costs .....................................................6-13
 

6-11. Net Quantified Economic Benefits of the NCMC ..........................................................6-14
 

6-12. Performance Measures ................................................................................................6-15
 

vii 
EXECUTIVE SUMMARY 

The National Institute for Standards and Technology (NIST) launched the NIST Combinatorial 

Methods Center (NCMC) in 2002 to develop and transfer to industry and academia technology 

that would assist researchers in significantly increasing the efficiency of their polymers research 

programs. Materials research is the basis for a wide range of polymer-related product (e.g., 

plastics, coatings, formulations), which, in turn, drive the global supply chain for a wide range of 

final goods and services. 

Given this critical role, technology infrastructure that makes R&D processes more effective and 

efficient and that enables the discovery of enhanced materials has the potential to improve 

economic, environmental, and public health welfare. Increasing research output per dollar of 

input, shrinking development times, and avoiding needless research in the upstream portion of 

the product supply chain hasten the introduction of new products and the benefits these 

products offer consumers over their predecessors. 

Combinatorial methods, referred to collectively as “combi”, offer scientists an alternative to 

manual “one-at-a-time” research techniques for discovering new materials. They enable the 

creation of a large number of related chemical samples (known as libraries) and the rapid 

analysis of those samples’ properties, known as “high-throughput screening” or “high-throughput 

experimentation”. 

In its initial foray into combi for polymers, NIST developed a simple but elegant approach for 

studying and screening new material compounds. The provision of continuous gradient libraries 

enabled researchers to vary material thickness and processing temperature, for example, on a 

substrate, and then analyze the entire gradient for combinations of interest at one time instead 

of preparing and analyzing individual samples. 

The acclaim that the results from this research received coupled with the stated needs from 

industry and academia for a community dedicated to combi provided the rationale for NIST to 

launch the NCMC, a public–private research consortium dedicated to combi’s development. 

ES.1 NIST’s Objectives for the NIST Combinatorial Methods Center 

Three overarching objectives characterized the NCMC when it was launched: 

	 Provide technology to overcome technical barriers to combinatorial and high-throughput 

experimentation for organic polymers. 

	 Disseminate research findings rapidly, offer instruction and best practices, and 

proactively engage the materials science community to raise awareness of the 

advantages of combinatorial and high-throughput experimentation for materials 

discovery. 

ES-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

	 Establish a forum wherein NIST, industry, and academics could exchange ideas and 

techniques, identify technology needs, and set research needs. 

The NCMC was conceived as an “open-source” community supported by NIST and industry 

members’ annual fees. All NCMC innovations enter the public domain. The Center offers novel 

technology solutions that minimize costs and maximize throughput and information capture. The 

NCMC adopted a consortium model to maximize technology transfer from the NCMC to industry 

and academia. NIST supports this transfer-efficiency objective by releasing tools, data, and 

methods as they are developed. Strategic planning is also enhanced by industry participation in 

the consortium, which assists NIST in ensuring that the optimal portfolio of infratechnologies is 

developed in response to industry’s needs. 

The NCMC has also become a nucleus for coordinating and driving symposia and national and 

international meetings, the interface with the American Chemistry Society (ACS) and the 

Materials Research Society (MRS), and interactions with other leading professional associations 

for materials science. 

As an additional output from its efforts, the NCMC has published more than 60 papers in the 

science and technology literature.

 In addition, the NCMC has 

	 sponsored more than 20 sessions and conferences nationally and internationally about 

advances in combi for polymers, 

	 hosted researchers for over 455 training days at NIST between 2002 and 2007, 

	 trained more than 25 new experts in combi through its postdoctoral research program, 

	 organized 14 semiannual meetings with presentations, technology demonstrations, and 

round-table discussions of technology infrastructure advances and needs, and 

	 presented more than 200 invited talks at key international meetings, industry sites, 

workshops, and universities, including Gordon Research Conferences, ACS,MRS, GE, 

3M, PPG, Bayer, IBM, Dow, Air Products, BASF, MIT, and Cornell. 

In late 2007, NIST contracted with RTI International to perform a retrospective economic impact 

assessment of NIST’s activities from the launch of its pilot continuous gradients project in 1998 

through 2007. The study’s goal was to quantify economic impacts, assess the effectiveness of 

the consortium approach, and chart the NCMC’s influence in encouraging and accelerating the 

introduction of combi for polymers. 

ES.1 NCMC’s Contributions to Combi for Polymers R&D 

The three characteristics of materials discovery—that materials are tailored for specific 

applications, that materials are formulated from many components, and that each component 

has a structure and intricate behavior—are the root of the challenge of using combinatorial 

approaches for materials. 

ES-2
 
Executive Summary 

In recent years, property specifications for materials mandated by end users have 

simultaneously increased and become more complex. The increasing difficulty in developing 

new materials led Bayer Materials Science (a NCMC member organization) to declare that a 

“revolution” was needed in coatings science, and this “revolution” would be provided in the form 

of combinatorial methods (Wicks and Bach, 2002). 

Combi can improve each stage of the experimentation cycle to create a seamless 

“combinatorial workflow” process (see Figure ES-1). Screening a fabricated library rarely 

presents researchers with a simple yes/no answer. Because typically more than one property is 

of interest and because material properties are a continuum of values (versus discrete reacts or 

does not react), the set of desired properties may be physically or chemically incompatible. 

Materials scientists must make trade-offs between properties such as hard (high modulus) yet 

brittle (low strength) versus soft but elastic. Combi and materials R&D in industry is a continuum 

of phases starting with discovery, transitioning to optimization, and ending with a product. 

Building a high-throughput workflow based on robotics is capital intensive, requiring start-up 

investments ranging between $8 and $20 million (Symyx, 1999). An Advanced Technology 

Program (ATP) white paper published in 2000 argued that “discontinuous innovation in generic 

and/or modular hardware and software technologies will be necessary to drive down costs and 

Figure ES-1. Combinatorial Workflow Cycle

A robust combinatorial system for materials research includes an informatics system to handle data and guides the 

research process. 

facilitate its implementation in industrial sectors that have lower returns on R&D investment, 

such as exist in the chemical and materials industries” (Hewes, 2000). 

ES-3 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

The NCMC made significant contributions to the development of less-expensive research 

capital and supporting technology infrastructure for the fabrication and testing of combi polymer 

libraries, thereby reducing costs associated with the application of combi methods to polymer 

materials research and stimulating U.S. industry to adopt combi and think innovatively about 

applying the concept of combi to polymers research. 

In summary, using NCMC’s approaches, researchers are able to rapidly map out structure– 

property relationships in systems with multiple parameters. This enables developers to more 

rapidly and efficiently develop new materials, optimize a material for a particular application, and 

respond more quickly to changes in the prices of raw materials and market demand. Additional 

benefits may include reduction in materials consumed and waste generated while developing a 

new polymer material because of reduced sample size and development of more robust 

materials because of the ability to test over a much broader range of conditions or explore a 

larger combination of components. 

With respect to scope of impact, NCMC’s work touched on each phase of the combi workflow 

and three different platforms for library creation: gradient thin films, discrete libraries created by 

robotic dispensing systems, and microfluidics. Testing of polymer libraries frequently requires 

new or modified equipment to speed up the measurement process creatively. As summarized in 

Table ES-1, NCMC innovated more than six measurement methods and made contributions to 

several more for library measurement. By design, combi generates large volumes of data that 

have to be captured, processed, and reported in a meaningful manner. This informatics aspect 

of combi was also a research area for the NCMC. 

NCMC advances were particularly important to key economic stakeholders active in polymers 

R&D: 

 advanced materials manufacturers (e.g., resins, rubber, specialty chemicals, synthetic 

fibers) that produce and market material components to other industrial consumers; 

	 paints, coatings, and adhesives manufacturers (collectively referred to as “coatings”); 

	 personal care and household products firms that use combi to study formulations; 

	 academic and government laboratories that employ combi in academic research 

settings; and 

	 laboratory equipment, software, and service providers that participate in the consortium 

to connect with customers, participate in the technical dialog, and leverage NCMC’s 

methods, software, concepts, and manuals. 

In the views of interviewees in these sectors, equally as important to NCMC’s technical 

accomplishments are the NCMC’s contributions to the development of human capital and a 

technical community focused on combi. Human capital development consists largely of NCMC 

postdocs who offer industry and academia talented, multidisciplinary researchers and host 

researchers from NCMC member organizations and international standards bodies. A significant 

ES-4
 
Executive Summary 

proportion of the Center’s mission has been focused on outreach, and the NCMC organized and 

led symposia at MRS and ACS meetings, among others, that demonstrated the possibilities with 

combi to large numbers of researchers. 

Organizing tracks and symposia at major MRS and ACS conferences as well as at more 

targeted audiences for Knowledge Foundation and Gordon Conference meetings have offered 

the staff access to audiences outside of the immediate membership. Indeed, the NIST staff and 

their projects were central to starting the Gordon Conference series on combi. Certainly, the 

staff maintain professional relationships with other researchers, but organizing symposia offered 

the Center the opportunity to present formally and provide evidence of the consortium’s 

technical accomplishments. 

ES.3 Methodology for Estimating Economic Benefits of the NCMC 

Our approach to valuing economic impacts was to prepare a series of counterfactual analyses 

that linked hypotheses about the NCMC’s benefits to relevant technical and economic metrics. 

These metrics then informed processes for collecting data and estimating measures of 

economic benefit. Benefits were estimated against a scenario in which the NCMC did not exist 

and key advances in thin films and microfluidics libraries were not made. 

Table ES-1. NCMC Platforms for Library Creation 

Platform and Library Properties NIST Original 

Gradient thin film libraries for polymers 

Thickness gradient Yes 

Surface energy gradient (UV ozone) Yes 

Temperature gradient Yes 

Composition gradient Yes 

Graded chemically patterned substrates Yes 

Discrete sample libraries 

Composition gradients for solutions 

Composition gradients for films and “dots” (blobs of polymer) 

Temperature gradient in position or time 

Microfluidics libraries 

Composition gradients, discrete library by droplet 

Composition gradients with reaction in organics, discrete library by droplet Yes 

Solution-deposited films (brushes) with chemical/composition gradient, Yes 

continuous sample (film) 

Solution-deposited copolymer films with chemical/thickness gradient, Yes 

continuous sample (film) 

ES-5 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

The total economic impacts of infratechnology development and consortium programs such as 

the NCMC can be broken out into five distinct categories: 

	 technology adoption/adaption: the benefits of adopting new technologies or adapting 

them to suit an organization’s purpose 

	 knowledge base expansion: the benefits of acquiring technical information and data that 

improve an organization’s knowledge base 

	 consortium experience: the benefits of membership in narrowly defined forums in which 

all respondents are focused on a single technical topic 

	 downstream product cost and quality: improved quality of goods and services in which 

those materials are an input and that potentially reduce the costs of those goods and 

services 

	 time acceleration effects: accelerated the accrual of cost savings or product quality 

benefits for end users 

The first three bullets pertain to R&D efficiency: acquiring infratechnology and knowledge that 

permit researchers to do more with less. NCMC methods and “how-to’s” for adapting common 

laboratory apparatus to enable them to perform high-throughput experiments and to enable end 

users to accrue labor, material, and capital cost savings over the life of the project, relative to 

the defending manual one-at-a-time methods. 

In economic analyses such as this one, each component of the total economic impact will 

generally have elements that are more tangible than others. For example, stakeholders could 

not quantify downstream product quality benefits or end-user acceleration benefits. 

Quantification is very difficult, although it is likely that benefits do exist. 

Today some research laboratories are using combi that otherwise would not because of NIST’s 

program. NIST’s contributions to microfluidics and thin-films research were particularly 

significant with respect to impact. Symyx and other vendors were offering comparable and, in 

some cases, more sophisticated technologies for discrete sample libraries, but NIST was the 

leader in thin films and microfluidics for polymers. Interviewees believe it would have taken from 

5 to 10 years before comparable, inexpensive infratechnologies for thin films would have been 

devised without NIST’s contribution. 

The counterfactual scenario against which benefits were measured specified that 

	 the novel suite of techniques for gradient approaches to thin films and elements of 

microfluidics and discrete sample library combi would not have been available before 

2008 and firms would have incurred greater capital, materials, and labor expenses as a 

consequence; 

	 the overall adoption rate of combi for organic polymers would have been lower, meaning 

that some users’ adoption likely would not have occurred until years later; 

ES-6
 
Executive Summary 

	 the costs of adopting combi would have been higher, and some laboratories would not 

have been using combi at all; and 

	 the absence of the NCMC would have hampered information diffusion and offered no 

coordinated forums for discussion of the development of the combi toolkit for organic 

polymers, particularly for thin films and microfluidics. 

Data informing this analysis were collected from semistructured interviews, survey forms 

completed via e-mail and returned to RTI, and an Internet survey of materials discovery 

scientists. Most responding scientists were employed by NCMC members or alumni, research 

universities, or large chemical companies that did not join the NCMC. All were directly involved 

in polymers R&D. 

Results from the survey sample were extrapolated to national estimates using publicly reported 

R&D expenditures for firms in the advanced materials, paints and coatings, and personal and 

household care product industries that are likely to be using combinatorial methods in their 

research. 

ES.4 Net Economic Benefit Estimates 

ES.4.1 Economic Benefits of Consortium Experience 

The consortium offered valuable demonstration and training opportunities that enabled 

companies to keep more readily abreast of the latest in combi-related research. They were also 

able to acquire information on novel approaches and inexpensive strategies for incorporating 

them into their workflows (see Table ES-2). 

On an annualized basis, 78% of members and alumni stated that the benefit of participating was 

equal to or greater than the membership fees paid. Half indicated that the NCMC validated 

internal research strategies or invalidated strategies that would not have born fruit. A former 

NCMC member commented that “the greatest benefit NCMC provided was a regular forum for 

combi materials ... which helped provide technology and competitive awareness [as well as] 

discussion and networking opportunities within the combi materials community.” 

Another member noted that “the biggest benefit we have gained from NCMC is the ability to see 

the internals of complete ‘combi’ procedures, not just the finished product. So, by being a 

member of NCMC we have been able to see a project being defined, problems being identified 

and solutions being developed—equally the handling of samples and information from the 

beginning to the end of an experimental process—unlike the snapshots seen via commercial 

interactions.” 

ES-7 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Table ES-2. NCMC Members’ Perceptions of the Benefits of Membership 

Survey Question Yes No 

Was the annual benefit your organization accrued from participation in 78% 22% 

the NCMC equal to or greater than the annual membership fees paid? 

Did participation in the NCMC 83% 17% 

... offer more productive marketing or networking opportunities, 

relative to other conferences or combi meetings? 

... validate or invalidate internal research projects and permit more 50% 50% 

efficient and effective resource allocation? 

... help your organization avoid any research activities, or enable your 70% 30% 

organization to acquire needed research more quickly or costeffectively? 

... offer valuable researcher training and demonstrations beyond what 67% 33% 

would have been available elsewhere? 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” Survey. 

ES.4.2 Economic Benefits of Knowledge Base Expansion 

The second component of estimated economic benefits was NCMC’s contributions to the 

expansion of the body of knowledge essential to implementing and using combinatorial 

approaches. Data provided by respondents indicate that the economic benefit of NCMC’s 

contributions to the combi knowledge base totaled nearly $24.4 million over the period from 

2001 to 2007 (see Table ES-3). 

These information acquisition benefits reflect two distinct advantages that accrued to end users. 

First, NCMC’s papers and presentations invalidated some firms’ research strategies, enabling 

them to avoid expending resources on projects that either would have ultimately failed or that 

would not have been as effective and efficient as those the NCMC published. Second, 

respondents stated that they acquired valuable knowledge about combi in general and about 

specific approaches. 

One researcher noted that although his lab had not implemented any of the methods developed 

by the NCMC, the overall body of knowledge that the consortium generated offered valuable 

insights into implementing and employing combinatorial approaches. 

ES.4.3 Economic Benefits of Technology Adoption and Acceleration 

The final component of economic benefits was the benefit of adopting NCMC combi technology 

and the economic benefit from accelerating firms’ combi adoption. In the case of methods 

developed by NCMC, the study valued the introduction of novel technology that otherwise would 

not have been introduced within this study’s period of analysis. 

ES-8
 
Executive Summary 

Table ES-3. Respondents’ Adoption and Awareness of NCMC Combi Technology 

Respondents’ Use of the Following Methods and 

Technologies Developed by the NCMC and its Use or Aware of but Unaware of 

Industry Members (% of respondents): Adapted Not Relevant Technology 

Continuous thin films with gradients in temperature, 45% 55% 0% 

composition, thickness, surface energy, or on 

chemically patterned substrates 

Discrete libraries used with gradients in temperature, 32% 55% 5% 

surface energy, or thickness 

High-throughput measurement of morphology 27% 41% 23% 

Buckling method of modulus measurement for thin 23% 64% 9% 

films and soft materials 

Edge lift-off test for interfacial adhesion 23% 64% 9% 

Microfluidics produced organic solvent-based libraries 14% 64% 14% 

Multilens contact test for adhesion 5% 77% 9% 

Interfacial tension measurement via microfluidics 5% 77% 9% 

Integrated metrology (morphology, composition, 0% 73% 18% 

extent of reaction) on microfluidics produced organic 

solvent-based libraries 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” survey. 

Some firms also reported accruing benefits due to accelerated adoption attributable to the 

NCMC. R&D efficiency benefits accrued to firms who adopted combi earlier than they would 

have in the absence of NCMC’s industry outreach work and publications. Where firms reported 

that their adoption of combi was accelerated, the firms’ net benefits from the acceleration effect 

were included in the benefits estimate. 

First, economic benefits from adopting NCMC-developed technology were quantified by 

measuring the cost savings from using NCMC approaches rather than the next best alternative 

(“defending”) method. Productivity gains were defined as increases in the number of equivalent 

samples analyzed per day per researcher. Economic benefits were the value of the increase in 

throughput and the volume of information acquired for equivalent or lower labor, materials, or 

capital expenditures. 

As illustrations of how NCMC’s research generated economic benefits: 

	 One researcher told us that without NCMC-developed approaches her company would 

not have undertaken what proved to be a highly successful project because it would 

have taken 10 times longer and have been 10 times more expensive. 

	 Another indicated that using NCMC’s temperature gradient approach she can 

accomplish in 30 minutes what would have otherwise taken her 375 minutes. 

	 One firm credited NCMC’s research with assisting in the development of advanced 

formulation systems used in their coatings research. The formulation system was 

ES-9 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

installed in each of the firms’ R&D labs, and they estimate savings to date of several 

million dollars compared to the cost if they had installed commercially available 

formulators. 

	 The work performed by NCMC on microfluidics for measuring interfacial tension helped 

another firm develop its own microfluidic equipment. This equipment enabled its lab to 

perform tests up to 3 times faster compared to the previous method while using fewer 

materials. Working with smaller samples had an important environmental benefit 

because the smaller samples size reduced waste, which, in turn, reduced hazardous 

material disposal costs. 

	 The edge lift-off test for interfacial adhesion was credited for enabling another firm to 

analyze 5 times more combinations of parameters than had been possible with previous 

methods. Although this firm did not cite significant gains in throughput, the method did 

permit them to study their samples more thoroughly. As a consequence, the firm 

believes that their customers have more robust products. 

Over 90% of respondents reported that the NCMC technologies offered efficiency benefits over 

the technologies that they had been using before, and a slightly higher percentage reported that 

these technologies also enabled them to test across a broader range of samples. NCMC 

technology also improved the effectiveness of their R&D: 92% of respondents stated that NCMC 

technology improved the quality of their research and/or products that their company may 

produce. 

Table ES-4 presents the average reported labor productivity gains for using combi in polymers 

R&D in general and for using the NCMC-developed technology. The mean labor productivity 

gain reported from combi overall (i.e., not from only NCMC approaches) was about 8.5 fold, with 

individual responses ranging between 1.5 and 24 fold. Driving this overall productivity gain 

estimate is the gains from NCMC-developed technology. 

The mean reported labor productivity gain for NCMC-developed approaches was 5.2 fold, with 

individual responses ranging between 1.2 and 12.5 fold. Follow-up interviews to explore the 

wide variation in reported gains suggested that the variation was attributable to differences in 

the techniques applied by various labs, properties of targeted materials, and required rate of 

automation, for example. 

ES-10
 
Executive Summary 

Table ES-4. NCMC Acceleration of and Estimated Productivity Gain from Combi 

Measure Value 

Mean overall net labor productivity gain from using combi for polymers 

R&D, where applicable 

Mean reported net labor productivity gain from using NCMCdeveloped technologies for polymers R&D, where applicable 

Percentage of respondents reporting that NCMC accelerated their 

adoption of combi for polymers R&D 

Mean number of years respondents reported that NCMC accelerated 

their combi adoption 

8.5 fold 

(Range: 1.5 to 24 fold) 

5.2 fold 

(Range: 1.2 to 12.5 fold) 

50% 

2.3 years 

(Range: 1 to 5 years) 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” survey. 

Half of all respondents reported that NCMC an accelerated their adoption of combi. The 

average adoption acceleration was 2.3 years. Some reported that their adoption of combi was 

accelerated by 4 or 5 years. A recurring theme was that interaction with NIST researchers gave 

firms more confidence in their decision to proceed with investments in combi programs. 

The publicity and attention NCMC drew to combi made it easier for some researchers to 

convince senior executives within their organizations to invest in combinatorial approaches 

because NCMC and NIST demonstrated that a reputable institution had successfully performed 

combi for polymers. 

ES.4.4 Net Economic Benefits 

Table ES-5 assembles the complete time series of quantified costs and benefits for 1998 

through 2007. Total benefits were estimated to be $210.4 million for the advanced materials, 

coatings, and personal and household care products industries: 

	 $1.1 million from adoption and demonstration benefits from participating in the 

consortium, 

	 $24.4 million from expanding the knowledge base supporting combi, leading to more 

informed combi R&D research project selection, and 

	 $184.8 million through product R&D efficiency realized from applying NCMC technology 

and through accelerating many firms adoption of combi and high-throughput research 

methods. 

Total NIST costs from 1998 to 2007 were approximately $14.5 million. These costs were 

augmented by NCMC membership and focus project fees of around $1 million and technology 

acquisition costs for NCMC-developed technology of about $7.5 million, including capital and 

initial labor expenditures. Total costs were therefore about $23 million. 

Inclusive of NIST, NCMC member, and technology acquisition costs net benefits were $187.4 

million. The vast majority of benefits accrued from using NCMC-developed technologies, but the 

ES-11 
 

 

 

 

 

 

 

 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

knowledge base expansion benefits equivalent to more than 10% of total benefits are certainly 

significant. These less tangible benefits are essentially a slice of the minimum alternative 

development cost for the technologies that NCMC developed for the equivalent of $23 million, 

and part of that $23 million included costs for administration and outreach work. 

The net present value (NPV) of net benefits was $118.0 million applying the OMB-approved 

discount rate of 7% (see Table ES-6). The benefit-to-cost ratio, which is the ratio of the NPV of 

total benefits to that of costs, was estimated to be 8.55. In other words, for every $1 that NIST 

and its partners invested in the NCMC at least $8.55 in benefits accrued to the three industries. 

The internal rate of return (IRR) was estimated to be 161%. Because the results of NCMC 

activities are widely used by many companies and other organizations, they have what 

economists call “public-good” content. In such cases, the IRR is called the “social rate of return”. 

Based on reviews of many economic studies, the hurdle rate for rationalizing such public-good 

investments is in the 30−50% range (Tassey, 2003). Thus, the NCMC returned at least three 

times what would be considered the minimum acceptable IRR. 

Table ES-5. Net Quantified Economic Benefits of the NCMC 

Knowledge Technology 

Consortium Base Adoption & Total 

Experience Expansion Acceleration Benefits Total Costs Net Benefits 

Year ($thousands) ($thousands) ($thousands) ($thousands) ($thousands) ($thousands) 

1998 (286) (286) 

1999 (495) (495) 

2000 (556) (556) 

2001 103 103 (2,266) (2,164) 

2002 124 558 10,912 11,594 (2,535) 9,059 

2003 294 4,768 31,349 36,411 (3,166) 33,245 

2004 266 4,981 41,344 46,591 (3,409) 43,182 

2005 151 4,806 37,366 42,323 (4,120) 38,203 

2006 186 4,912 33,575 38,673 (3,806) 34,867 

2007 113 4,306 30,293 34,713 (2,418) 32,295 

Total $1,134 $24,434 $184,839 $210,408 (23,057) $187,351 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. Sums may not add to 

totals due to rounding. 

ES-12
 
Executive Summary 

Table ES-6. Performance Measures 

Value 

Measure (2007$) 

Total quantified benefits $210.4 million 

Total quantified costs $23.1 million 

Net present value of net benefits (NPV) (Base year = 1998) $118.0 million 

Benefit-to-cost ratio (BCR) 8.55 

Internal rate of return (IRR) 161% 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. NPV was calculated 

using the 7% real social discount rate recommended by OMB. 

ES.5 Remarks on the Effectiveness of the Consortium Approach 

Through the NCMC’s work, researchers acquired both technology and evidence of how 

investments in combi could increase R&D efficiency and hasten the development of more robust 

polymeric materials and the products whose performance or quality is enabled by those 

materials. 

The technology NCMC developed between 1998 and 2007 enabled organizations to reap 

significant R&D efficiency benefits by integrating and adapting for their use methods such as 

continuous gradient thin films, microfluidics libraries, and the edge lift-off test for interfacial 

adhesion. As a recognized leader in metrology, NIST assumed the role of filling technology 

gaps left unaddressed because of market and technical barriers. NIST was also able to 

leverage its reputation for scientific excellence and independence into a leadership role for the 

technical community. 

If it were not for these attributes – leadership in metrology, impartiality, and a reputation for 

scientific excellence– the creation of a consortium like the NCMC as a precompetitive forum 

whose members were all from the private sector would have been highly unlikely. Beyond 

developing and then rapidly transferring technology into the public domain, NCMC’s outreach 

work, research, and publications demonstrated what was possible with combi. The combination 

of these activities had a pronounced effect on industrial R&D efficiency—a critical factor in an 

increasingly technology-based global economy. The NCMC offers an excellent example of 

much of the best of what NIST has to offer in science & technology development and outreach: 

	 overcoming technology gaps through the development of infratechnology, 

	 convening researchers in independent, precompetitive forums to disseminate research 

and best practice, 

	 partnering with government and academia to develop and execute research projects of 

particular relevance to industry, 

ES-13 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

	 demonstrating what is possible and advocating for novel approaches for rapidly 

identifying research foci, and 

	 championing the adoption and development of new approaches that offer R&D and 

production efficiency benefits as well as the opportunity to discover groundbreaking new 

materials. 

In summary, the consortium model proved to be particularly effective. The results of this study 

provide evidence of how industry participation in the research program combined with 

government (NIST) support can allow a strategic focus on areas of systematic private-sector 

underinvestment, specifically generic technologies and infratechnologies essential to a 

technology’s development and use. NIST benefited from industry’s feedback, and industry 

benefited from access to information on a prepublication basis as well as from hands-on 

demonstrations of novel approaches. The feedback loop inherent in this approach integrated 

NIST’s mandate to expand the frontier of measurement science with industry’s desire for 

research outcomes to be relevant to their needs. 

ES-14
 
1. STUDY INTRODUCTION 

The science of materials discovery and development is such a fundamental function within the 

global supply chain for goods and services that the question of what commodities and finished 

goods are not affected by it in some way becomes difficult to answer. In the modern economy, 

early in the planning stages of production emerge the critical questions of: From what materials 

shall a product be made? What criteria must those materials meet in order to be acceptable for 

the desired product? How can we enhance key materials’ properties to improve the product’s 

performance? 

These questions compel U.S. industry to expend billions of dollars in research and development 

(R&D) annually on materials discovery, development, and optimization. The scale of materials 

R&D is so large because, by definition, materials are an intermediate good. Just as products are 

of a seemingly infinite variety, so too are the materials of which they are composed. The 

performance of a product purchased from a store shelf largely depends on the properties, 

characteristics, and interactions of its constituent compounds. 

A large proportion of the value-added manufacturing sector consumes as inputs in their 

production processes the output of large materials, specialty chemicals, and coatings 

companies. Many vertically integrated firms and industrial conglomerates (e.g., General Electric, 

Procter & Gamble, and 3M) maintain research units that develop materials for use by sister 

companies and other divisions under their corporate umbrella. In addition to their counterparts 

within corporate R&D centers, research groups in academia and the public sector engage in 

basic and applied research to build the knowledge base of phenomena in materials and to 

develop novel materials to meet national priorities. 

Given the critical role materials science plays in the economy, technology infrastructure that 

makes R&D processes more effective and efficient and that enables the discovery of enhanced 

materials has the potential to improve economic and environmental welfare. Increasing research 

output per dollar of input, shrinking development times, and avoiding needless research in the 

upstream portion of the product supply chain hasten the introduction of new products and the 

benefits these products offer consumers over their predecessors. 

The National Institute for Standards and Technology (NIST) launched the NIST Combinatorial 

Methods Center (NCMC) in 2002 to develop and transfer to industry and academia technology 

that would assist researchers in significantly increasing the throughput of their research 

programs. Materials research in the key area of polymers (e.g., plastics, coatings, formulations) 

is a time-consuming and resource-intensive process. The pharmaceutical industry addressed 

the same throughput challenge via combinatorial chemistry—the systematic synthesis of a large 

variety of chemicals and screening the results for drug candidates. In theory, the same 

approach was possible for polymers science, but the reality was that the obstacles posed by 

1-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

polymeric materials were more challenging than those faced by the drug industry from 

chemicals suspended in purified water. Polymers are often viscous, solvent-based, and 

processed at high temperatures. The dearth of available research and technical expertise in 

high-throughput experimentation for polymers meant that even the most basic infrastructure for 

combinatorial methods in many applications was nonexistent. The same equipment employed 

by the pharmaceutical industry would be quickly destroyed by the industrial solvents and heat 

needed for polymers. 

In its initial foray into combi for polymers NIST developed a simple but elegant approach to 

using combinatorial and high-throughput experimentation, collectively referred to as “combi,” for 

thin polymer films. The acclaim with which the results from that project were met coupled with 

the stated needs from industry and academia for a community dedicated to combi provided the 

rationale for NIST to launch the NCMC, a public–private research consortium dedicated to 

combi’s development. 

In late 2007, NIST contracted with RTI International to perform a retrospective economic impact 

assessment of NIST’s activities from the launch of its pilot thin films project in 1998 through 

2007. The study’s goal was to quantify economic impacts, assess the effectiveness of the 

consortium approach, and chart the NCMC’s influence in encouraging and accelerating the 

introduction of combi for polymers. This report is the study’s final deliverable. 

This introductory chapter offers a foundational discussion on key subjects necessary for 

describing the NCMC’s scientific accomplishments and economic impacts. The following 

sections help illustrate why combi is important and why the NCMC was formed to bridge 

technical gaps: 

	 brief introduction to materials science, 

	 discussion and definition of polymers and their pervasiveness, 

	 review of traditional methods for materials research, 

	 introduction of combinatorial methods to materials science, and 

	 the rationale for NIST’s creation of the NCMC. 

1.1 Overview of Polymers Science 

Materials science is an interdisciplinary field that studies the structure, properties, and synthesis 

of materials (Smith, 2004). A “material” is broadly defined as any physical substance that is 

useful for human purposes, such as plastic or steel. Materials are often grouped into four 

principal categories: 

	 Metals: inorganic substances consisting primarily of metallic elements (e.g., gold, 

copper, and steel) 

	 Ceramics: inorganic substances consisting of metallic and nonmetallic elements (e.g., 

porcelain, glass, and rocks) 

1-2 
Chapter 1 — Study Introduction 

	 Semiconductors: inorganic substances composed of metalloid elements (e.g., silicon 

and germanium) 

	 Polymers: typically organic (i.e., containing carbon atoms) substances composed of 

nonmetallic elements that form long molecular chains consisting of many repeating 

atomic units (from Latin, poly = many, mer = unit). Common synthetic polymers include 

plastics and rubber (Chung, 2007). Most naturally occurring materials like plant fibers 

are also polymers. 

Materials are building blocks for products because of their properties—the way they respond to 

their environment. Copper’s electrical property (conductivity) makes it a useful material for 

electrical wiring. Other important properties include mechanical (strength, elasticity), thermal 

(transmission of heat, heat capacity), chemical stability (corrosion), and optical properties 

(absorption, transmission, and scattering of light) (Bargolia, 2004). 

Two factors govern the properties a material will exhibit: chemical composition and molecular 

structure. The importance of chemical composition in determining a material’s properties is 

similar to the importance of ingredients in determining a cake’s properties. If one bakes a cake 

with salt instead of sugar, its taste properties will be dramatically different (Chung, 2007). 

However, even materials that have identical compositions can display different properties 

because of variations in how molecules are arranged or structured. A material’s structure largely 

depends on how the material is created or its processing conditions. Both graphite and diamond 

are composed of pure carbon, yet they display different properties. Graphite is soft, while 

diamond is hard. These differences in properties are the result of differences in structure. The 

carbon atoms in graphite are organized in the form of sheets stacked on top of one another like 

a stack of paper. As a result, when force is applied to graphite, the sheets of carbon slip past 

each other. This is why it is easy to break the lead at the end of a pencil. In contrast, carbon 

atoms in diamond are bonded together in arrangements that resemble a pyramid that does not 

give as easily when force is applied (Chung, 2007). 

The properties of a material are also affected by the structure of molecules in more subtle ways. 

In particular, when two chemicals are mixed together, the resulting mixture’s properties will not 

only depend on the molecular structure of each individual chemical, but also on how the 

structures interact. One can intuitively understand the importance of how molecular structures 

interact by imagining each molecule as a toy block. The properties of a house built with these 

blocks, such as its appearance and stability, will not only depend on the shape of the individual 

blocks, but also on how they interact. A house made with square and rectangular blocks will 

have different properties than a house made with square and round blocks. 

Molecular composition, structure, and structure at other length scales are important. For 

example, a material can be composed of the same molecules but be in an amorphous or 

crystalline state depending on the processing conditions of the material and temperature at end 

use. The molecules (molecular length scale) are the same, but the structure that arises from the 

1-3 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

intermolecular interactions (meso- or nano-length scale) determines if it is crystalline. The 

resulting differences in structure are reflected in the material’s flexibility. 

Among the largest output categories for the chemical industry, “polymers” is a term that refers to 

synthetic materials composed of carbon (organic) backbones. Polymers are molecules 

composed of carbon and other atoms covalently bonded together to form very long chains. 

Atomic repeat units are present that repeat themselves many times to form chains that can have 

molecular weights of 10,000 g/mol to 1,000,000 g/mol or more compared to small molecules like 

water (18 g/mol) or caffeine (194 g/mol). The molecular composition of a polymer, the number of 

repeat units (i.e., the molecular weight or how long the chain is), and the inter- and 

intramolecular interactions of the polymer chains determine the structure of the polymer. The 

structures, in turn, provide the polymer with its properties. 

Polymers are commonplace; all plastics, paints, adhesives, coatings, rubber, and many other 

formulated products are polymeric. Table 1-1 lists 20 likely polymers that can be found in a 

typical office. Although the NCMC did not exclude siloxanes (silicon-containing materials) 

containing elements other than carbon, oxygen, and hydrogen, this report follows common 

usage and refers to organic polymers simply as polymers. 

Table 1-1. Common Office Items Containing Synthetic Polymers 

1. Writing pen (outer structure, rubber grip, container inside holding the ink, possibly polymers in the ink) 

2. Computer (case work, circuit boards, packing of electronic components, adhesives, covering on all 

power and data lines, keyboard, mouse) 

3. Phone (case work, electronics packaging, display screen, wire insulation) 

4. Polyurethane coating desktop 

5. Tape holder, the tape, and the adhesive on the tape 

6. Carpet (threads, backing, adhesive attaching it to the floor) 

7. Baseboard around the office 

8. Office chair (foam padding, cloth covering, vinyl coverings) 

9. Latex paint on the wall 

10. Cover/diffuser on the room light 

11. Light switch 

12. All the insulation on the wires and data lines into the office 

13. Wall and data sockets 

14. Coatings on paper (e.g., shiny finish on wall calendar) 

15. Plastic water bottle 

16. Coating on coffee cup 

17. Lunch bag, containers holding food in the lunch bag, guar gum as additive in food 

18. Stain-resistant coating on clothing 

19. Rubber soles on shoes 

20. Coat made from nylon fabric 

1-4 
 

Chapter 1 — Study Introduction 

Consider poly(methyl methacrylate), or PMMA, commonly referred to as Plexiglas, which is 

renowned for its durability. Plexiglas’s molecular composition and chain interaction make 

Plexiglas a substitute for glass that is strong, more flexible, and shatter resistant. 

Polymers can be used in large solid forms, as thin coatings or films, or as additives to solutions 

to modify flow properties. Different polymers can be blended together. Additives to polymer 

blends, such as dyes or plasticizers, enhance polymer formulations. Fibers of polymer materials 

can be created: stronger and lighter materials than steel, such as for Kevlar fibers for bulletproof vests; soft and pliable for a child’s toy; thin and sticky for use as an adhesive; or thin, hard, 

and shiny for use as a protective coating on a piece of furniture. 

The diversity of possible chemical compositions and combinations allows polymers to have a 

wide variety of properties for use in many applications. Yet, the large parameter space inherent 

with polymers (e.g., molecular composition, molecular weight, combination of components, 

processing conditions) also makes developing new polymer materials complex, expensive, and 

time consuming. 

1.2 Traditional Methods of Polymers Research 

The research activities involved with developing new polymer materials can be divided into two 

phases: lead discovery and lead optimization. Lead discovery is the identification of a 

material compound that is believed to possess desirable properties. The period during which 

that compound is further tested and improved before being released as a commercial product is 

referred to as lead optimization. 

Until recently, lead discovery most often occurred by accident or through a slow trial-and-error 

process. Discovery by accident, though exciting and revolutionary at times, is unreliable, and 

duplication or reliable process development can be challenging as a consequence. For 

instance, the discovery of Teflon by Roy Plunkett of DuPont occurred by accident while he was 

attempting to develop a new refrigerant compound. Plunkett’s new material exhibited 

extraordinary properties, including high melting temperature, resistance to chemicals, and low 

surface adhesion, making it excellent for a wide variety of applications such as making 

cookware. But it took DuPont researchers 3 years to develop a reliable process for 

manufacturing it (Strauss, 2002). 

More deliberate methods of materials discovery follow a cycle of designing, making, testing, 

optimizing, and analyzing candidate compounds. As an example of this materials discovery 

process, consider the development of a new plastic material with improved strength and heat 

resistance for making engine parts. Researchers would first consider existing materials that 

come close to meeting the requirements and improvements to them. If researchers concluded 

that they needed to develop a new material, like a new polymer or combination of existing 

materials, they would then design experiments. Samples would be individually prepared and 

tested for the properties of interest, such as softening temperature, strength, and hardness. 

1-5 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

After these tests are complete, researchers analyze output data to determine whether any of the 

samples meet the performance criteria. If the sample displays the desired characteristics, it 

becomes a “lead.” If the substance does not meet the criteria, the researcher returns to the 

beginning of the cycle and iterates this process. Once a lead is discovered, it enters the 

optimization phase to determine whether the material can be improved upon to exhibit the 

desired property or properties more strongly. This phase may include mixing the material with 

other materials or improving on the production process used to create the material itself. 

Traditionally, all of these activities were performed manually. Teams of scientists were required 

to select and synthesize sample materials, test them for desirable properties, and then attempt 

to optimize those properties. Given that lead discovery and optimization often involve hundreds 

of samples and several tests per sample, materials R&D can be an expensive and slow 

process. The R&D for bringing a new product to market can take 2 to 10 years and cost in 

excess of $20 million (De Lue, 2001). 

In a 1970 publication, Dr. Joseph Hanak, then an industrial chemist at RCA Laboratories and an 

early thought leader in combi for materials science, was frustrated by the slow process of 

materials research: “[the current] approach to the search for new materials suffers from a 

chronic ailment, that of handling one sample at a time in the processes of synthesis, chemical 

analysis and testing of properties. It is an expensive and time consuming approach, which 

prevents highly trained personnel from taking full advantage of their talents and keeps the 

tempo of discovery of new materials at a low level” (Hanak, 1970). 

1.3 Introduction of Combinatorial Methods to Polymers Science 

The slow and inefficient character of materials research in which samples were individually 

manually prepared was the driving force behind developing combi for polymers. Scientists 

began to envision processes where many different samples with different compositions could be 

made at one time. Combi offered materials scientists an alternative to traditional research 

techniques because they enable the creation of a large number of related chemical samples 

(known as libraries) and the rapid analysis of the properties [known as “high-throughput 

screening” or “high-throughput experimentation (HTE)”]. 

Combi was first successfully used in the pharmaceutical industry in the early 1990s as a means 

to increase the speed of drug discovery and, therefore, lower the cost of bringing new drugs to 

market (PhRMA, 2007). In combinatorial chemistry, a large number of related chemical 

compounds are made through automated synthesis and are then screened quickly using simple 

tests for appropriate activity. For example, does compound A react or not react with enzyme B 

as indicated by a color change? 

An early illustration of the potential benefit of applying combi came in 1995 when researchers at 

Pfizer declared that the drug company had a compound in clinical trials that would not have 

been discovered without combinatorial methods (Thayer, 1995). According to the company’s 

1-6 
Chapter 1 — Study Introduction 

research staff, the breakthrough did not arrive until they had synthesized nearly 900 variations 

of the compound, a number they would never have achieved using manual methods. 

Inspired by the success of combi in pharmaceutical research, in the late 1990s, researchers 

began to seek avenues for applying the combinatorial and HTE philosophy to materials science. 

Foremost among the challenges they faced was fundamental: when combinatorial techniques 

are applied to drug research, screening compounds can be fairly straightforward and based on 

yes-or-no results. The range and diversity of properties, along with the range of factors affecting 

them, make combi for materials science far more difficult than with “classical” combinatorial 

chemistry. 

1.3.1 Key Concepts in Combi 

Combi can be applied to improving each stage of the experimentation cycle described in 

Section 1.3 to create a seamless “combinatorial workflow.” This workflow comprises four 

steps, all of which are integrated using an informatics system (Figure 1-1). 

Researchers begin by using their existing knowledge of materials science to design an 

experiment to identify material exhibiting properties X and Z. This involves identifying materials 

that are likely to exhibit the properties of interest and then determining how these materials 

Figure 1-1. Combinatorial Workflow Cycle

A robust combinatorial system for materials research includes an informatics system to handle data and guides the 

research process. 

Design Library 

Fabricate Library 

Characterize Library 

Collect Data 

No Promising Results? Hit 

Define Target 

Yes 

Design Library 

Fabricate Library 

Characterize Library 

Collect Data 

Promising Results? No 

Lead 

Yes 

Validate & Optimize 

Informatics 

1-7 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

should be synthesized, how they should be tested, and finally how the results of the tests will be 

analyzed. 

Library fabrication is typically automated using robotic systems that are coordinated by the 

informatics system. The design and fabrication of this library typically depends on the 

experiment being conducted. Two examples of libraries used in polymer science are discrete 

libraries and continuous gradient libraries. Discrete libraries have tiny amounts of material 

samples arranged into individual wells or arrays of miniature specimens. Each well may contain 

a different material sample that will be subjected to the same test or each well may contain the 

same type of material sample that will be subjected to different tests in each well. Examples of 

continuous gradient libraries include continuous films of material spread onto a plate where 

properties such as a temperature, thickness, or composition are varied along each axis. 

Microfluidics libraries permit researchers to analyze variations in polymer formulations over a 

variety of processing conditions or compositions. An illustration of each type of library is 

provided in Figure 1-2. 

After sample libraries have been fabricated, the samples are analyzed with automated systems 

that rapidly test each sample in the library for the desired properties. The test in question 

typically depends on the property being analyzed. For example, peel-off tests might be used to 

determine the adhesion properties of material in the library if researchers are interested in 

identifying a material of a specific stickiness. 

Figure 1-2. Discrete Sample, Continuous Gradient Thin Film, and Microfluidics 

Libraries 

A computer-controlled system deposits samples of varying composition into individual wells arranged in a matrix to 

form a discrete library of samples. A flow coating technique is used to make a polymer film that varies in thickness in 

one direction and a temperature gradient stage is used to vary temperature in another direction, thus creating a whole 

range of conditions in a single sample. A microfluidics system creates variations in formulation in a reactor. 

Temperature 

1-8 
Chapter 1 — Study Introduction 

The informatics system records the test results for each chemical compound that researchers 

then use the informatics system to analyze these results. If the experiment in question is being 

conducted to discover a new material, these results will help researchers determine whether 

particular material samples are leads. If the experiment is being conducted as part of the lead 

optimization process, these results will help researchers determine how they can strengthen the 

properties exhibited by the lead materials. 

Analytical results can also be used to inform subsequent experiments because they may help 

researchers better understand what materials (or combination of materials) exhibit which 

properties. As a result of this feedback into experiment design, the combinatorial workflow takes 

on a circular flow quality that is meant to foster improvements in future experiments while giving 

faster results across a broader parameter space today. This combinatorial workflow aims to 

significantly lower cost and time per lead in developing a new product. 

1.3.2 Technical Challenges to Applying Combi in Polymers Research 

Screening a fabricated library rarely presents researchers with a simple yes/no answer. 

Because typically more than one property is of interest, and because material properties are a 

continuum of values (versus reacts or does not react), the set of desired properties may be 

physically or chemically incompatible. Materials scientists must make trade-offs between 

properties such as hard (high modulus) yet brittle (low strength) versus soft but elastic. Combi 

and materials science in industry is a continuum starting with discovery, transitioning to 

optimization, and ending with a product. 

Whereas in combinatorial chemistry in the drug industry, molecular composition of a onecomponent system is the standard focus, the focus in materials development is on 

multicomponent systems. Formulation is standard; more than one type of polymer may be 

mixed together in combination with additives to obtain the desired combination of properties. 

Furthermore, materials scientists face the challenge of working with material that is typically 

viscous and is processed with solvents or at temperatures that would damage or destroy the 

equipment pharmaceutical companies use for their combi workflows. 

The three interrelated challenges of materials discovery—that materials are tailored for specific 

applications, materials are formulated from many components, and each component has a 

structure and intricate behavior—are the root of the challenge of using combinatorial 

approaches for materials. In recent years, property specifications for materials mandated by end 

users have simultaneously increased and become more complex (Wicks and Bach, 2002). This 

increasing difficulty in developing coatings formulations led Douglas Wicks and Hermann Bach 

of the Bayer Corporation (a current NCMC member organization) to declare that a “revolution” 

was needed in coatings science (Wicks and Bach, 2002). In a paper presented at the WaterBorne & Higher Solids and Powder Coatings Symposium, Wicks and Bach argued that this 

“revolution” would be provided in the form of combinatorial methods. 

1-9 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

A small number of firms emerged to produce and market combinatorial workflows for HTE for 

discrete sample libraries, offering solutions based on robotics and automaton. But these 

providers were not fully addressing key challenges for thin films and microfluidics libraries, 

including 

	 library design across multivariate parameter spaces; 

	 library fabrication, including materials handling, production, mixing, deposition, and 

variation of processing conditions; 

	 high-throughput measurement methods and metrics; 

	 data capture, analysis, and management tools; and 

	 informatics for instrument automation, data mining, analysis, and visualization (Hewes, 

2000). 

1.4 NIST’s Entry into Combi: Addressing Technical and Market Barriers 

Despite the promise combi held for materials discovery and the downstream products enabled 

by new materials, technical barriers obstructed combi’s practical introduction. The equipment, 

measurement infrastructure, and analytical tools were inadequate. Indeed, systems capable of 

fabricating gradient and microfluidics laboratories for organic polymers were nonexistent. The 

concepts and philosophy were present, but the technology infrastructure, culture, and 

perception that combi could be applied to polymers were not. Compounding the problem was a 

lack of high-throughput measurement systems to acquire data rapidly from libraries that were 

more than a simple yes-no solution space. 

It is unknown to what extent private-sector organizations successfully implemented combi in 

their R&D and process engineering programs in advance of the NCMC, but those that did likely 

incurred high costs. It is doubtful that they would have disclosed their efforts beyond exchanges 

with contract engineering and equipment vendors. (Research groups in academia, such as the 

Center for Nanoscale Science and Engineering at North Dakota State University, were in the 

early stages of launching programs using combi for discrete libraries when NCMC launched in 

2002.) 

Building a high-throughput workflow based on robotics is capital intensive, requiring start-up 

investments ranging between $8 and $20 million (Symyx, 1999). An Advanced Technology 

Program (ATP) white paper published in 2000 with the assistance of public- and private-sector 

researchers argued that “discontinuous innovation in generic and/or modular hardware and 

software technologies will be necessary to drive down costs and facilitate its implementation in 

industrial sectors that have lower returns on R&D investment, such as exist in the chemical and 

materials industries” (Hewes, 2000). 

NIST’s involvement in combi initially took the form of a research project undertaken by NIST 

researchers Eric Amis, Alamigir Karim, and Carson Meredith that developed gradient thin films 

1-10 
Chapter 1 — Study Introduction 

libraries. The results of NIST’s research were first presented in 1999 at the Gordon Conference 

on Reactive Polymers, Ion Exchange, and Adsorbents (Meredith et al., 1999) and were 

ultimately published in a series of papers in 2000 (Meredith et al., 2000; Meredith, Karim, and 

Amis, 2000). This initial research was leveraged to garner NIST funding (2001) in order to build 

a modest program in combinatorial methods for polymer thin films. 

The enthusiastic response with which the thin films research was met, ATP’s white paper series 

identifying technical challenges, and comments NIST received from stakeholders suggested 

that NIST could serve a role. The combi community was fractured; there was no independent 

coordinating body to advance combi’s science and raise its profile as a more effective and 

alternative research paradigm for advanced materials. NIST conducted an assessment and 

solicited comments via conference circuits cosponsored by the Knowledge Foundation, which 

permitted NIST scientists to further gauge interest, identify needs, and strategize a role for the 

emerging NIST effort. 

The potential impact of developing effective equipment and techniques in combinatorial 

polymers materials research and the sizable challenge to developing workflows for materials 

research led the NIST Polymers Division to develop the NCMC program. Following a brief 

planning period, NIST launched the NCMC in early 2002 to provide a leadership role by 

sponsoring and coordinating an “open-source,” precompetitive forum to accelerate combi in 

polymers. 

1.5 Report Organization 

This report is organized as follows: 

	 Chapter 2 presents an overview of the NCMC with an emphasis on the use of the 

consortium approach. 

	 Chapter 3 reviews the Center’s technical contributions. 

	 Chapter 4 discusses how combi impacts the R&D activities of the key industries where it 

is being used or has potential to be used in the future. 

	 Chapter 5 presents the methodology for conceptualizing economic impacts and 

quantifying economic benefits.
 

	 Chapter 6 presents the analytical results from economic modeling. 

	 Chapter 7 concludes with remarks about the broader implications of the NCMC’s 

impacts and the effectiveness of NIST’s use of a consortium approach as the research 

program model. 

1-11 
 

2. OVERVIEW OF THE NIST COMBINATORIAL METHODS CENTER 

The NCMC’s mission is to promote industrial adaptation of combinatorial and high-throughput 

measurement methods that serve to accelerate the discovery, development, and optimization of 

innovative materials products. The Center furthers its mission by building the technology 

infrastructure supporting combi, partnering with industry on research priorities, transferring 

technology to the public domain, and lending thought leadership to the research community. 

This chapter explores the NCMC’s consortium approach, operational structure, and 

nontechnology outputs. Details on NCMC’s technology contributions are discussed in Chapter 

3. 

2.1 NCMC’s Consortium Approach 

Three overarching objectives characterized the NCMC when it was launched in January 2002: 

	 Provide technology to overcome technical barriers to combinatorial and high-throughput 

experimentation for organic polymers. 

	 Disseminate research findings rapidly, offer instruction and best practices, and 

proactively engage the materials science community to raise awareness of the 

advantages of combinatorial and high-throughput experimentation for materials 

discovery. 

	 Establish a forum wherein NIST, industry, and academics could exchange ideas and 

techniques, identify technology needs, and set research needs. 

At the time of the NCMC’s creation, NIST Polymer Division’s leadership and leading academics 

believed that the existing knowledge base for combi was locked within silos at major corporate 

R&D centers and technology vendors. The commercial infrastructure for HTE using discrete 

sample libraries was more established, particularly through the efforts of Symyx and other 

automated systems vendors. The absence of any precompetitive forum for discussing combi 

inhibited even the discussion of general topics.1

 Comments received following NIST’s 

solicitations at Knowledge Foundation meetings suggested that an open model where industry 

and academia could drive the research agenda and share equally in all knowledge gains, with 

no time advantage, would be most effective. 

The NCMC adopted a consortium model to maximize technology transfer from the NCMC to 

industry and academia. The Center would offer novel technology solutions that minimized costs 

and maximized throughput. NIST aspired to feed research by academic and industrial 

researchers by releasing tools, data, and methods as they are developed. Thus, the NCMC was 

conceived as an “open-source” community supported by NIST and industry members’ annual 

fees. The NCMC would develop infratechnology in response to stated needs and disseminate 

1

 Precompetitive forums offer researchers at competing organizations the opportunity to discuss and jointly address 

or collaborate on issues arising during the early stages of a technology’s development and adoption. 

2-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

them through workshops, papers, and presentations. A condition of membership is that any 

technology developed within the Center would be published and freely available. All NCMC 

innovations enter the public domain. The NCMC would also be a nucleus that would coordinate 

and drive symposia and national and international meetings and interface with the American 

Chemistry Society (ACS) and the Materials Research Society (MRS), the leading professional 

associations for materials science. 

2.2 NCMC Structure 

The NCMC has a two-tier membership structure built on a foundation of NIST expertise in 

materials science and chemical engineering. Given that the Center focuses on combi for 

polymers, most members are specialty chemicals, coatings, or advanced materials companies 

(see Figure 2-1). However, the NCMC leadership partners with and invites to NCMC meetings 

industrial and academic researchers in other materials domains given that the development and 

application of combi and high-throughput experimentation are priorities for materials research in 

general, not just polymers. 

2.2.1 NCMC Staff and Operations 

The NCMC is staffed with a core research and administrative staff of NIST employees from the 

Polymers Division. In addition, researchers from other NIST units lend their expertise to NCMC 

projects and activities as needed. The NCMC is not the only group at NIST engaged in research 

into combi methods or applying combi. NIST’s ceramics and semiconductor research groups 

Figure 2-1. NCMC Members and Alumni, by Industry 

Personal Care and 

Household Products, 6 

Vendors, 5 

Other, 4 

Government and Academic 

Labs, 2 

Specialty Chemicals, 

Materials, and Coatings, 17 

0 

2 

4 

6 

8 

10 

12 

14 

16 

18 

Specialty Chemicals, 

Materials, and Coatings 

Personal Care and 

Household Products 

Vendors Other Government and 

Academic Labs 

Source: NIST Combinatorial Methods Center 

2-2 
Chapter 2 — Overview of the NIST Combinatorial Methods Center 

are also developing and using high-throughput experiments in their work. Thus, there is an 

element of cross fertilization (economies of scope) among the several materials domains. 

The majority of NCMC’s full-time staff are postdoctoral researchers (“postdocs”). Since 2002, 

more than 25 postdocs have left the Center following their 2-year positions to join universities 

and corporate research divisions, many of which were NCMC members. The benefits of this 

approach are that the NCMC regularly draws fresh talent and routinely graduates talented 

researchers with hands-on training and a diversified skill set in these new and exciting 

techniques. 

NIST staff and NCMC postdocs are complemented by hosted researchers from universities, 

member companies, and national research laboratories. Hosting researchers is a key 

component of the education pillar of NCMC’s mission and vital to the direct transfer of 

technology to the private sector. 

2.2.2 General Membership 

Two NCMC membership levels are available for industry: general membership and focus project 

participation. Members at the nonproprietary level pay approximately $10,000 per year to 

participate in the NCMC. This membership includes 

	 participation in semiannual meetings organized in response to stated needs or emerging 

issues in high-throughput experimentation (see Table 2-1); 

	 access to electronic resources through the NCMC Web site, including guides, protocols, 

strategies, and measurement techniques; 

	 access to workshops, demonstrations, and prepublication technical papers; and 

	 on-site demonstrations of new techniques and apparatus for HTE. 

Organizations that are not members of the NCMC have access to the consortium’s 

accomplishments through workshops, journals, and conferences, but they do not have access 

to them on a prepublication basis. Thus, they incur the lag between writing papers, acceptance, 

and publication. 

Current and former NCMC members characterized the benefit of membership in terms of 

impacts within their organization and not in terms of the activities of membership. In the words 

of one former member: “One of the advantages of working with NIST is [that] you’re working 

with a group of very smart people [who] are far more advanced in terms of where you are on the 

[combi] learning curve.” Many members joined the consortium as part of their program for 

implementing HTE. Common benefits cited by members include 

	 demonstration of what was possible with combi; 

	 inspiration for new research approaches and the creation of new methodologies; 

2-3 
 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Table 2-1. NCMC Semiannual Meetings: Topics and Private-Sector Attendance Counts 

Meeting Topic Year 

Industry 

Attendees 

NCMC-1 Library Design and Calibration 2002 16 

NCMC-2 Adhesion and Mechanical Properties 2002 42 

NCMC-3 Combinatorial Informatics 2003 24 

NCMC-4 Polymer Formulations 2003 54 

NCMC-5 Combinatorial Processing & Characterization 2004 37 

NCMC-6 Advanced Materials Forum 2004 27 

NCMC-7 Adhesion and Mechanical Properties II 2005 24 

NCMC-8 Polymer Formulations II 2005 24 

NCMC-9 Combinatorial Methods for Nanostructured Materials 2006 29 

NCMC-10 Completing the Combi Loop: Examining Persistent 2006 26 

Challenges in Implementing Combinatorial Materials Science 

NCMC-11 Complex Interfaces: Library Design & Performance Measures 2007 32 

NCMC-12 Data Acquisition, Management, and Handling 2007 20 

NCMC-13 Advances in Library Fabrication 2008 14 

NCMC-14 DOE-EERE/NIST Joint Workshop on Combinatorial Materials 2008 8 

Science for Applications in Energy 

Source: NIST Combinatorial Methods Center. 

	 validation of research strategies and invalidation of others that, if pursued, would likely 

have failed; 

	 accelerated combi adoption and scale-up; and 

	 knowledge benefits of working with researchers with similar interests but different 

perspectives because of different materials domains.
 

Annual NCMC membership fluctuated and largely corresponded with the research community’s 

interest in combi in general as well as the interest in topics the NCMC was exploring at any 

given time. Table 2-2 presents a chronology of NCMC membership by company. Membership 

was lowest in 2002 and 2003, which was a period when many companies became 

“disillusioned” with combi because of persistent challenges integrating combi into their workflow. 

Membership peaked in 2004 with 23 members and, in subsequent years, was around 15 

companies per year. Specialty chemical and materials companies Air Products, Atofina, BASF, 

Bayer Materials, and National Starch have been members in each year of the NCMC’s 

existence. 

2.2.3 Focus Projects 

NCMC focus projects offer members the additional opportunity to sponsor and collaborate with 

NCMC staff on projects aligned with their particular research interests and needs. The benefits 

of focus project participation are threefold; members’ participation enables them to 

2-4 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Chapter 2 — Overview of the NIST Combinatorial Methods Center 

Table 2-2. Chronology of NCMC Membership 

Company 2002 2003 2004 2005 2006 2007 2008 

3M Company █ █

Accelrys Inc. █ █

Air Force Research Laboratory █ █ █ █

Air Products and Chemicals, Inc. █ █ █ █ █ █ █

Akzo Nobel N.V. █ █ █

Arkema Inc. (formerly Atofina) █ █ █ █ █

ATRP Solutions █ █

Avon Products, Inc. █

BASF SE █ █ █ █ █ █ █

Bayer AG █ █ █ █ █ █ █

Boston Scientific █ █

BP PLC █ █

Chemistry Innovation █ █

Dow Chemical Company █ █ █ █ █

Eastman Chemical Company █ █

Exxon Mobil Corporation █ █ █ █ █ █

Gillette (now part of Procter & Gamble) █

Honeywell International, Inc. █ █ █ █ █

Hysitron, Inc. █ █ █ █

ICI/National Starch & Chemicals █ █ █ █ █ █ █

Insight Faraday █

Intel Corp █ █ █

L’Oréal SA █ █ █ █

LORD █

Michelin █ █

PPG Industries, Inc. █ █ █ █

Procter & Gamble Co. █ █ █ █ █

Rhodia █ █ █ █ █ █

Rohm and Hass Co. █

Sealed Air Corp. █

Symyx Technologies, Inc. █ █ █ █ █

Unilever PLC █ █ █

University of Southern Mississippi █ █ █ █

Vecco Instruments Inc. █ █ █

Vistakon █ █

Source: NIST Combinatorial Methods Center. 

 help guide and scope focus projects; 

 gain direct, ongoing knowledge in the project area; and 

 acquire information and insights in “real time” as opposed to waiting for NCMC 

semiannual meetings or publications. 

2-5 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

The advantage members gain through participation in focus projects is simply that their 

researchers acquire the expertise and tacit knowledge benefits of NCMC postdocs and staff 

involved in the project as they were executing it. They are then able to transfer the technology in 

real time to their home research labs. As with general membership, there is no cooperative 

research and development agreement (CRADA), and all knowledge and technology developed 

as part of the focus project are nonproprietary and fully publishable. 

The NCMC members’ time advantage over competitors can be illustrated this way: if a focus 

project lasted 2 years, and it took an additional 6 to 9 months to have a paper accepted for 

publication, and 3 to 6 months before the article appeared, then member companies have a 

multiyear time advantage in addition to the tacit knowledge experience over their nonmember 

competitors. 

Although costs vary by project, on average, members pay approximately $70,000 to participate 

in focus projects in addition to their annual membership fee and labor and travel expenses for 

their employees. Focus projects last 2 to 3 years, and the annual participation fee is usually 

$20,000 to $30,000. NCMC’s four focus projects included 

 High-Throughput Interfacial Tension Measurements, 

 High-Throughput Methods for the Evaluation of Adhesive Properties, 

 High-Throughput Flammability Test Methods for Compositionally Graded Samples, and 

 High-Throughput Modulus Measurements of Hydrated Polymer Gels. 

2.3 Nontechnology Outputs 

In the views of interviewees, equally as important to NCMC’s technical accomplishments are the 

Center’s contributions to the development of human capital and a technical community focused 

on combi. Human capital development consisted largely of NCMC postdocs who offered 

industry and academia talented, multidisciplinary researchers and hosted researchers from 

NCMC member organizations and international research groups. A significant proportion of the 

Center’s mission was focused on outreach, and the NCMC organized and led symposia at MRS 

and ACS meetings, among others, that demonstrated the possibilities with combi to large 

numbers of researchers. 

2.3.1 Human Capital Development: Postdoctoral and Hosted Researchers 

Many research groups are in the early years of adopting combi and HTE, and the NCMC 

postdocs are a talent pool that offers expertise as groups adopt combi workflows. More than 25 

researchers have completed postdocs with the NCMC and are working in academia, private 

industry, and government. Most postdocs bring their employers connections, hands-on 

knowledge, and research program management skills beyond those of newly minted PhDs. The 

additional value NCMC postdocs offer is hands-on experience developing new approaches in 

an industry-focused, dynamic research environment. Several NCMC postdocs were hired 

2-6 
Chapter 2 — Overview of the NIST Combinatorial Methods Center 

explicitly because they had the broad experience from their NIST years as well as a deep 

understanding of the issues and challenges in moving to a high or higher-throughput research 

environment. The comparable amount of time for training a non-NCMC postdoc to reach the 

same knowledge and technical sophistication level was estimated by their employers to be 8 

months to a year. 

2.3.2 Conference and Symposium Sponsorship 

Interviewees’ remarks suggest that demonstrating what was possible with combi was equally as 

important as the actual methods and tools. The NCMC engaged nonmembers through two 

primary mechanisms: technical papers and conference proceedings, and organization and 

sponsorship of conferences and symposia. 

Organizing tracks and symposia at major MRS and ACS conferences as well as at more 

targeted audiences for Knowledge Foundation and Gordon Conference meetings offered the 

staff access to audiences outside of the immediate membership (see Table 2-3). Indeed, the 

NIST staff and their projects were central to starting the Gordon Conference series on combi. 

Certainly the staff maintain professional relationships with other researchers, but organizing 

symposia offered the Center the opportunity to present formally and provide evidence of the 

consortium’s technical accomplishments. 

Many attendees with whom we spoke did not adopt the technology as presented; their principal 

take-away point was that the Center demonstrated what was possible with combi and that 

combi could be applied efficiently to polymers without necessarily investing in costly automated 

workflow systems. Yet, there are also cases where NIST’s validation of combi aided in 

companies deciding the capital investment was worth the investment to achieve their particular 

goals. 

2.3.3 Papers and Proceedings 

Technical papers published in the peer-reviewed literature facilitated the transfer of knowledge 

generated by the NCMC into the public domain for use by the entire materials science 

community. RTI performed a brief citation analysis for all 146 papers published by NCMC staff 

between 2000 and December 2007 which collectively received nearly 1,400 citations (as of 

January, 2009). Table 2-4 lists five of the most frequently cited papers published. 

The paper that generated the most citations was written by Christopher Stafford and colleagues 

on buckling-based metrology for measuring the elastic moduli of polymeric thin films (Stafford et 

al., 2004). This paper generated 123 citations as of February 2009. The next three most-cited 

papers—Meredith et al. (2000), Meredith, Karim, and Amis (2000), and Ragavan et al. 

(2000c)—addressed the fabrication and use of continuous polymer libraries. These early NCMC 

papers received a combined total of 295 citations by researchers at a variety of universities as 

2-7 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Table 2-3. NCMC-Sponsored or -Organized Symposia at Major Scientific Meetings 

Meeting 

1. 5th International Conference on Combinatorial and High-Throughput Materials Science, Kloster 

Seeon, Germany, October 2008 

2. Combinatorial and Informatics Methods in Materials Science IV, Boston, MA, 

3. Combinatorial and High-Throughput Polymer Chemistry, Chicago Il, ACS Spring Meeting 2007, 

March 2007 

4. 4th International Workshop on Combinatorial Materials Science and Technology, San Juan, Puerto 

Rico, December 2006 

5. Combinatorial and Informatics Methods in Materials Science III, Boston MA, MRS Fall Meeting 2005 

(Published Proceedings) 

6. Combinatorial Approaches to Materials II, Washington DC, ACS Fall Meeting 2005 

7. APS March Meeting 2005, Los Angeles CA, Parallel and High-Throughput Experimentation for the 

Physical Sciences 

8. Combinatorial Approaches to Materials, Anaheim CA, ACS 2004 Spring Meeting 

9. Combinatorial Methods in Adhesion, Wilmington, NC, Adhesion Society 2004 

10. 3rd International Workshop on Combinatorial Materials Science and Technology, Okinawa, Japan, 

December 2004 

11. Knowledge Foundation COMBI 2004: 6th Annual International Symposium on Combinatorial 

Approaches for New Materials Discovery, Arlington, VA 

12. Combinatorial and Artificial Intelligence Methods in Materials Science II, Boston, MA, MRS Fall 

Meeting 2003 (Published Proceedings) 

13. Knowledge Foundation COMBI 2003: 5th Annual International Symposium on Combinatorial 

Approaches for New Materials Discovery, San Jose, CA 

14. New Gordon Conference: Combinatorial and High Throughput Materials Science, New Hampshire, 

2002 

15. Knowledge Foundation COMBI 2002: 4th Annual International Symposium on Combinatorial 

Approaches for New Materials Discovery, San Diego, CA 

16. Combinatorial and Artificial Intelligence Methods in Materials Science I, Boston, MA, MRS Fall 

Meeting 2001 (Published Proceedings) 

Source: NIST Combinatorial Methods Center 

2-8 
Chapter 2 — Overview of the NIST Combinatorial Methods Center 

Table 2-4. List of Five NCMC Papers Most Frequently Cited in the Literature 

Number of 

Publication Information Citations 

Stafford, C.M., C. Harrison, K.L. Beers, et al. 2004. “A Buckling-Based Metrology 

for Measuring the Elastic Moduli of Polymeric Thin Films.” Nature Materials 

3(8):545-550. 

123 

Meredith, J.C., A.P. Smith, A. Karim, et al. 2000a. “Combinatorial Materials Science 

for Polymer Thin-Film Dewetting.” Macromolecules 33(26):9747-9756. 

104 

Raghavan D, Gu X, Nguyen T, et al. 2000. “Mapping Polymer Heterogeneity Using 

Atomic Force Microscopy Phase Imaging and Nanoscale Indentation.” 

Macromolecules 33(7):2573-2583. 

91 

Meredith, J.C., A. Karim, and E.J. Amis. 2000. “High-Throughput Measurement of 

Polymer Blend Phase Behavior.” Macromolecules 33(16):5760-5762. 

85 

Wu, T., Y. Mei, J.T. Cabral, et al. 2004. “A New Synthetic Method for Controlled 

Polymerization using a Microfluidic System.” Journal of the American Chemical 

Society 126(32):9880-9881. 

47 

Note: Citation counts obtained through Web of Science academic database, February, 2009. 

well as private institutions such as GE, BASF, Bayer, and Procter & Gamble. Another frequently 

cited paper (48 citations) focused on microfluidics for polymers in organic solvents (Wu et al., 

2004). The number of times each paper was cited each year is illustrated in Figure 2-1. 

Figure 2-1. Yearly Citation Analysis for Major NCMC Publications 

0 

5 

10 

15 

20 

25 

30 

35 

40 

2000 2001 2002 2003 2004 2005 2006 2007 2008 

Number of Citations 

Meredith et al, 2000a Meredith et al, 2000b Stafford et al, 2004 

Wu et al, 2004 Raghavan, 2004 

2-9 
3. NCMC TECHNICAL ACCOMPLISHMENTS 

The NCMC made significant contributions to the fabrication and testing of combi polymer 

libraries, validating the application of combi methods to polymer materials research and 

stimulating U.S. industry to adopt combi and think innovatively about applying the concept of 

combi to polymers research. Using combi, researchers are able to rapidly map out structure– 

property relationships in systems with multiple parameters. This enables developers to more 

rapidly and efficiently develop new materials, optimize a material for a particular application, and 

respond to changes in raw materials and the market. Additional benefits may include reduction 

in materials consumed and waste generated while developing a new polymer material because 

of reduced sample size and development of more robust materials because of the ability to test 

over a much broader range of conditions or explore a larger combination of components. 

NCMC’s work touched on each phase of the combi workflow and three different platforms for 

library creation (see Table 3-1): gradient thin films, discrete libraries created by robotic 

dispensing systems, and microfluidics. Testing of polymer libraries frequently requires new or 

modified equipment to speed up the measurement process creatively. As summarized in Table 

3-2, NCMC innovated more than six measurement methods and made contributions to several 

more for library measurement. By design, combi generates large volumes of data that have to 

be captured, processed, and reported in a meaningful manner. This informatics aspect of combi 

was also a research area for the NCMC. The discussion in this section is organized by library 

type. 

3.1 Combi Using Continuous Gradient Approaches for Thin Polymer Films 

The most mature and frequently cited research in the academic literature is NCMC’s gradient 

thin-film technology. The basic concept of creating and using gradients is NCMC’s most broadreaching technical contribution. NCMC developed methods to create gradients in thickness, 

temperature, surface energy (e.g., water repellency), and composition. 

An example helps illustrate the gradient thin-film concept and the ingenuity of NCMC staff. 

Consider that a new coating is being developed that requires the blending of two different 

polymers: polystyrene (PS) and polyvinylmethylether (PVME). One problem with blending 

polymers is that they may phase separate. That is, rather than having one nice film with both 

polymers mixed intimately together, a cloudy film is formed that has domains rich in PS and 

domains rich in PVME. Imagine a hypothetical situation in which oil and water mix together at 

high temperature into one clear solution (or phase), but upon cooling the oil and water separate 

(phase separate), forming a cloudy solution. In our polymer thin-film example, phase separation 

is a function of composition and annealing temperature, and the variations in the library can be 

used to assess variations in composition or processing temperature. 

3-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Table 3-1. NCMC Platforms for Library Creation 

Platform and Library Properties NIST Original 

Gradient thin film libraries for polymers 

Thickness gradient Yes 

Surface energy gradient (UV ozone) Yes 

Temperature gradient Yes 

Composition gradient Yes 

Graded chemically patterned substrates Yes 

Discrete sample libraries 

Composition gradients for solutions 

Composition gradients for films and “dots” (blobs of polymer) 

Temperature gradient in position or time 

Microfluidics libraries 

Composition gradients, discrete library by droplet 

Composition gradients with reaction in organics, discrete library by droplet Yes 

Solution-deposited films (brushes) with chemical/composition gradient, Yes 

continuous sample (film) 

Solution-deposited copolymer films with chemical/thickness gradient, Yes 

continuous sample (film) 

Traditionally, a researcher would prepare individual samples, each with varying amounts of PS 

and PVME in them and then vary the temperature of each sample. Instead of this slow process 

of individual samples, a continuous sample that varies in composition in one direction and 

temperature in another can be made. This is a continuous gradient sample library. Figure 3-1 

shows such a gradient sample where polymer PS content is varied along the x-axis and 

annealing temperature along the y-axis (Meredith, Karim, and Amis, 2002). 

Figure 3-1. Gradient Thin-Film Sample 

A polymer thin film with a gradient composition of 

polystyrene and polyvinylmethylether in the x-axis 

and temperature gradient in the y-axis provides a 

library of samples that took 2 days to make 

instead of weeks. The phase separation boundary 

of PS and PVME is clearly seen. White circles are 

traditionally collected data indicating the phase 

boundary (Meredith, Karim, and Amis, 2002). 

3-2 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Chapter 3 — NCMC Technical Accomplishments 

Table 3-2. Materials Property Measurement for NCMC Library Platforms 

Type of Library 

Material Property Measurement 

Continuous Thin FilmDiscrete Thin Film (sectioned) MicrofluidicsDiscrete Dro

plets 

Continuous Fluidics SampleDiscrete Samples via Other ProcessNIST Original 

Adhesion Multilense contact (MCAT) z z Yes 

Adhesion of PSA Adhesive peel test z z

Adhesion-bonding Edge delamination test z z Yes 

Chemical Ion mass spectroscopy, TOF- z z z

composition SIMS 

Composition, rxn FTIR z z z z z

conversion 

Composition, rxn Raman spectroscopy z z

conversion 

Craze resistance HT craze testing on copper grids z Yes 

Interfacial tension Fluidics interfacial tension device z Yes 

Modulus Film buckling (SIEBIMM) z Yes 

Modulus, soft films Reverse buckling (reverse z Yes 

SIEBIMM) 

Morphology Automated AFM/SPM z z

Morphology Near field polorimetry z z z z z

Morphology Light scattering; DLS, SALS z z Yes 

Morphology SANS/SAX z

Morphology Electron Microscopy z z z Yes 

Morphology Automated microscopy z z z z z

Surface energy Automated contact angle z z z

measurement 

Translation rate Automated microscopy 

Thickness FTIR z

Thickness Interferometery z z z

Thickness AFM z z

Thickness Elipsometry z z

Viscosity Fluidics viscometer z Yes 

Viscosity/rheology Multisample rheometer z Yes 

3-3 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Once the library is fabricated it is ready for measurement. In the above example, the phase 

behavior is directly observable by viewing the clouding of the polymer film at conditions where 

phase separation occurs. Part of the ingenuity of gradient libraries is that they can be selfreporting. In other words, the solution space is directly observable. Measurement can include 

color change and light scattering. The boundary between conditions providing for one phase or 

two is clearly seen in Figure 3-1. The white circles are data points from traditional samples and 

light-scattering measurements overlaid on the coordinate system created by the gradient 

sample (PS content x-axis and temperature y-axis). NCMC’s novel approach of preparing this 

gradient sample reduced preparation and measurement from weeks to just 2 days. NCMC’s 

successes with gradients stimulated a number of industrial and academic researchers to 

develop gradient methods for their own applications. 

NCMC also developed several innovations in combi measurements for thin films and gradient 

thin films (see Table 3-2), including rapid and accurate measurement of film modulus (a 

measure of a material’s ability to deform, such as its elasticity). Traditionally, measuring the 

modulus in thin films was capital and labor expensive, leading to limited measurement of this 

important property of polymer thin-film materials. NCMC researchers developed a film-buckling 

method where applied strain causes the film sample under test to buckle and create periodic 

wrinkles that are related to the local modulus of the polymer film. This modulus measurement 

technique used readily available and much less expensive materials than the traditional method 

(nano-indentation) and provided results quickly. The NCMC measurement technique is 

applicable to gradient thin films and traditional thin-film samples and can be used on soft or hard 

films, leading it to be adopted rapidly by many academic labs doing research on polymer thin 

films. Researchers previously unable to or severely restricted in measuring thin-film modulus 

are now able to measure modulus inexpensively over a larger number and variety of samples. 

3.2 Combi Using Robotics for Discrete Sample Libraries 

The NCMC used two methods of making discrete libraries: generating a gradient film sample 

followed by sectioning the film to create a discrete library, and using robotic dispensing systems. 

Commercial systems for discrete libraries already existed at the time NCMC was founded, and 

commercial instrument makers were continuing to develop systems useful for viscous polymer 

systems. Because of the existing commercial effort, NCMC tended to not focus on discrete 

library generation. However, discrete libraries are important to industry given their compatibility 

with industry’s existing toolsets. This need for discrete libraries led industrial researches in 

combi to adapt NCMC’s efforts in continuous libraries to the development of discrete libraries or 

to simply use NCMC’s results with continuous libraries as inspiration for developing their own 

combi methods relevant to discrete libraries. 

An example of discrete library fabrication and testing at NCMC is a study of the effect of epoxy 

resin and curing agent concentration on bond strength between the epoxy and a glass 

substrate. A matrix of droplets with varying concentrations of epoxy resin is generated like the 

3-4 
Chapter 3 — NCMC Technical Accomplishments 

one in Figure 3-2 using the computer-controlled system that varies concentration, mixes the 

components, and deposits a specified amount at each location in the matrix. Alternatively, a 

researcher would measure the amount of resin and curing agent for each sample, mix it by 

hand, and deposit it on the glass substrate. 

Next the bond strength between the cured epoxy and glass is measured. NCMC developed a 

method of testing for edge delamination by introducing minute edge defects and cooling the 

substrate. Failure is observed with a digital camera and recorded. Figure 3-3 shows the type of 

failure map this process can generate rapidly. An application of this technology is the 

development of bonding adhesives in attaching chips to computer boards with the requirements 

of reducing material use and reducing component size while preventing failure during a range of 

operating conditions. 

Figure 3-2. Library of Epoxy Droplets 

with Varying Concentration of Epoxy 

Resin Generated by ComputerControlled System (Robot) 

Samples were prepared by an automated system 

of varying component composition, inline mixing, 

and deposition into a matrix. Source: NIST 

Combinatorial Methods Center. 

Figure 3-3. Failure Map 

for Varying Epoxy 

Concentration 

Epoxy samples with varying 

amounts of resin and curing 

agent were tested for bonding 

with glass and silicone (Si) 

substrates. A failure map is 

readily prepared digital images 

taken of the library after 

introducing small edge defects 

and cooling the library. 

Source: NIST Combinatorial 

Methods Center. 

3-5 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

3.3 Combi Using Microfluidics Libraries 

Microfluidic devices are a means to manipulate small volumes of fluids in channels that are 

millimeters or micrometers in diameter. The advantage of using microfluidics libraries is to 

measure formulations across a series of changing reaction conditions, either continuously or 

discretely. The ability to make and handle small liquid samples reduces the amount of materials 

needed and produces less waste. Additionally, chemical reaction times can be reduced and 

heat and mass transfer times can be reduced as a result of the small scale of the devices. The 

small samples generated can then be analyzed for chemical composition or physical properties 

like viscosity and interfacial tension. The integrated discipline of making and testing samples in 

small flow-through devices has the goal of developing “lab on a chip” technology. That is, the 

process that required several researchers to have a large lab and a variety of lab equipment can 

be reduced to a small device that does it all. 

Microfluidics is best known and widely used in the biochemistry and pharmaceutical fields as a 

way of making and handling small-volume aqueous samples in a continuous fashion (Sia and 

Whitesides, 2003). The application of microfluidics to polymer-based solutions is challenging, 

given that they typically require organic solvents and are more viscous than water-based 

systems. The high viscosity makes mixing more difficult, while the organic solvents tend to 

dissolve or swell materials used in making aqueous-based microfluidics systems. For example, 

the silicone (PDMS) devices developed by Whitesides’ (Sia and Whitesides, 2003) and Quake’ 

(Quake and Scherer, 2000) research groups are degraded by solvents typically used with 

polymers such as acetone or toluene. Additionally, the properties to measure are broader than 

those traditionally done in the biochemistry/pharmaceutical disciplines using microfluidics. 

NCMC pioneered concepts in fabricating inexpensive microfluidics devices with solvent 

resistance and using them to make gradients in samples with inline/online testing of materials 

properties. Currently, uptake by industry of NCMC’s microfluidics for polymer combi libraries is 

minimal because of the incompatibility with their existing toolsets and the need to generate 

larger samples to meet the testing and development needs of the polymer-chemical industry. 

However, one industrial representative indicated that developing new formulations and polymer 

reactions with microfluidics is attractive because of the reduction in hazardous materials 

handling and waste generation, potentially making it greener and cost competitive. 

NCMC developed methods to rapidly fabricate solvent-resistant fluidic devices that can produce 

discrete libraries of materials contained in droplets (Cygan et al., 2005). The same technology 

can be used to produce continuous gradient samples where the sample is either the effluent 

from the device or where the sample is formed/deposited on a surface contained within the 

device (e.g., gradient film formation). 

A demonstration of the microfluidics platform to make and test a library is the UV-curing of a 

dental material containing monomer (reactant used to make polymers) and a crosslinker (a 

crosslinker connects polymers into a network, thus increasing material strength). Residual 

3-6 
Chapter 3 — NCMC Technical Accomplishments 

monomer in the cured compound is undesirable (could leach out in one’s mouth), so curing as a 

function of composition needs to be studied (Barns et al., 2006). Researchers use a 

microfluidics device to create droplets of organics in a water-surfactant solution to make a 

composition library with monomer content that varies from about 10% to 100%. The droplets are 

then cured with UV light, and the monomer content in each droplet is measured. Figure 3-4 

illustrates this type of device. 

The reaction conversion (monomer remaining) needs to be measured for each droplet 

composition. NCMC innovated Raman spectroscopy (inelastic scattering of laser light by 

chemical bonds in molecules) with fiber optic collection of the data for use with a microfluidic 

device. Figure 3-5 illustrates the conversion of a monomer as a function of composition. 

3.4 Informatics for Combi 

Central to a successful combi workflow is managing the data. The large volumes of information 

from the contents of the library created to the results for each measurement on the library must 

be analyzed and correlated. Furthermore, all the data created across many libraries may 

provide unique results and direct future experiments. This examination of existing data across 

many experiments (often unrelated at the time of conception) is called data mining. The 

handling and analysis of the data sets from combinatorial experiments require an informatics 

infrastructure that integrates, manages, and helps direct the combinatorial workflow. 

The NCMC developed a flexible informatics system based on open-source languages. The 

NCMC system addresses the challenge of handling a diverse data set of input parameters (e.g., 

molecular composition, structure, formulation, processing conditions) and output material 

properties and performance (e.g., modulus, optical properties, resistance to wear) (Cebon and 

Figure 3-4. Microfluidics 

Device for Making Organic 

Phase Droplets 

A discrete composition gradient of 

droplets is made at junction B. The 

monomer (benzyl methacrylate) is 

mixed with the crosslinker 

(hexanediol dimethacrylate), and 

the concentration of the monomer 

is varied by droplet as a function of 

distance from point B. Each 

droplet can function as a micro 

reactor for studying the effect of 

composition on reaction 

conversion. The region between A 

and C is the area for UV curing 

and Raman spectroscopy to 

measure conversion of the 

monomer. Source: NIST 

Combinatorial Methods Center. 

3-7 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Figure 3-5. Percentage 

Conversion of Monomer in 

a Dental Material 

A microfluidics device was used to 

prepare a gradient in a monomer 

(benzyl methacrylate) composition 

with a crosslinker. Addition of a 

crosslinker decreases conversion 

as measured in a device using 

Raman spectroscopy with fiber 

optic collection of data. Source: 

NIST Combinatorial Methods 

Center. 

Ashby, 2006; Zhang et al., 2005). Although a good informatics system can be central to the 

success of a combinatorial effort, the handling of large and diverse data sets was already of 

interest to the commercial software community. A number of commercial companies have 

developed software packages for data handling and analysis. Since commercial software 

developers depend heavily on their intellectual property (IP) in software and data handling 

concepts, they were uninterested in endangering their IP by entangling it with public-domain 

work done by the NCMC. Industrial users of informatics systems require more mature and 

ready-to use software than that developed by the NCMC, so they purchase commercial 

software to meet their informatics needs. However, the Center did publish papers describing the 

essential aspects of informatics and held two educational workshops on their development. 

3-8 
4. END USERS, INDUSTRIES, AND APPLICATIONS 

As an approach to screening for materials properties combi’s potential crosses industries and 

disciplines. Research groups testing for combinations of materials properties or discovering new 

ones may benefit from the combinatorial or high-throughput approach if the scale of materials 

samples they seek to screen is sufficiently large. A thin-film temperature gradient library may 

allow a researcher to identify the correct thickness or materials composition, for instance, but 

may be in an industry wholly unrelated to those counted among NCMC members and alumni, 

such as someone in the R&D division of a boiler manufacturer. 

To quantify economic benefits, however, we must define the end-user population along the lines 

of industrial classifications to extrapolate accurately the economic benefits from survey 

responses to national-level impact estimates. Interviews as well as a review of the science and 

technology literature indicated that NCMC’s impacts have affected five stakeholder groups: 

 advanced materials manufacturers (e.g., resins, rubber, specialty chemicals, synthetic 

fibers) that produce and market material components to other industrial consumers; 

	 paints, coatings, and adhesives manufacturers (collectively referred to as “coatings”); 

	 personal care and household products firms that use combi to study formulations; 

	 academic and government laboratories that employ combi in academic research 

settings; and 

	 laboratory equipment, software, and service providers that participate in the consortium 

to connect with customers, participate in the technical dialog, and leverage NCMC’s 

methods, software, concepts, and manuals. 

Although this section reviews each stakeholder group individually, it is important to note that 

there is significant overlap between stakeholder groups at the firm level. For example, 

companies such as Dow Chemical produce materials components for use in other industries as 

well as coatings and adhesives. As a result, Dow would be considered a part of both the 

advanced materials and coatings stakeholder groups. 

The NCMC has worked to lower the technical and financial barriers to implementing combi, but 

combi still requires large investments to restructure R&D workflows, train researchers, and 

purchase needed equipment. As a result, internal combinatorial research programs tend to be 

located in larger organizations. During interviews with industry representatives, it was 

repeatedly stated that combi tends to be concentrated in firms with net sales exceeding $1 

billion per year. 

Any improvements inspired by the NCMC also have the potential to affect other stakeholders 

through improved knowledge, services, or equipment and software. The specific form of these 

benefits will differ from case to case. For example, if an equipment manufacturer develops a 

4-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

new tool based on one of NCMC’s accomplishments, then purchasers of that equipment will 

benefit from increased efficiency just as the equipment manufacturer itself benefits from 

revenues generated through a new product line. 

Improvements in the infrastructure supporting advanced materials R&D also affect companies 

that use polymers as inputs in their production processes. These benefits are created because 

polymers are used as inputs into the production of both coatings and personal care products. 

For example, NCMC’s techniques may allow a firm in the advanced materials industry to 

develop a new resin that allows some coatings manufacturers to formulate coatings with 

improved or new properties. The benefits of the NCMC continue to flow into a broad range of 

consumer goods industries that are too numerous to list (see Figure 4-1). Investments in the 

R&D technologies and supporting infratechnologies (including combi) offer significant 

economies of scope in the form of large net economic benefits across many user industries and 

ultimately consumers. 

4.1 Advanced Materials Industry 

Counted among the names of firms in the advanced materials industry—a cross-section of 

specialty chemicals, resins, fiber, materials, and fiber companies—are some of the most familiar 

names in industrial chemistry: Air Products, Arkema, Bayer Materials, BASF, Dow Chemical, 

and Rohm & Haas, among them. Smaller firms in this intensely competitive industry are few 

because of the high capital intensity and high production volumes required to earn competitive 

returns. These firms’ customers and suppliers are distributed globally as are their corporate 

Figure 4-1. Simplified Flow of Benefits through the Combi-Enabled Value Chain 

4-2 
 

Chapter 4 — End Users, Industries, and Applications 

research divisions. Indeed, like their American counterparts, foreign companies have substantial 

investments in corporate research centers outside their home countries; NCMC members BASF 

and Bayer Materials each have large R&D centers in the United States, for instance. 

Firms in the advanced materials stakeholder group manufacture a diverse range of products 

such as resins, rubbers, composites, and synthetic [North American Industry Classification 

System (NAICS) code 3252]. The products produced by firms in this stakeholder group are 

typically not sold to consumers. Rather they are sold to other industries where they serve as 

inputs into the production of other products. As a result, the products manufactured by 

advanced materials firms are pervasive in modern life. For example, resins are used to create 

plastic products ranging from water bottles to industrial-grade components. Advanced materials 

manufacturers provide customers with products that meet exacting properties and 

specifications, As a result, these firms must engage in R&D to improve existing products and 

discover new materials that may possess properties desired by consumers. 

The pervasiveness of advanced materials products helps explain why this stakeholder group is 

the largest in terms of both sales and employment. In 2006, the advanced materials stakeholder 

group employed 78,600 workers and generated shipments valued at $95 billion (Table 4-1). 

Manufacturers of plastic materials and resins accounted for the vast majority of this activity, 

employing 69% of all workers and generating over 84% of the total value of shipments. A 

significant portion of the total value of shipments of advanced materials manufacturers is 

accounted for by sales to overseas customers. In 2006, these firms generated approximately 

$32 billion in exports or 34% of the total value of shipments that year. The vast majority of these 

exports, approximately 80%, were made by plastic material and resin manufacturers. 

Chapter 1 described how the discovery, formulation, and optimization of such polymers are 

difficult tasks because a large number of parameters impact the properties these materials 

exhibit. By providing researchers with a means to synthesize polymer samples quickly across a 

wide parameter space, combi may allow advanced materials manufacturers to develop more 

quickly new or improved materials with properties that are more desirable to their customer 

base. In 2004, researchers at Dow published an article in Macromolecular Rapid 

Communications that described how using high-throughput equipment enabled them to evaluate 

these interactions in a much shorter period of time than traditional methods. The analysis time 

was measured in hours instead of weeks (Peil et al., 2004). 

4.2 Paints, Coatings, and Adhesives Industry 

Firms in the paints, coatings, and adhesives stakeholder group produce coatings that are 

applied to objects to improve one or more of their surface properties (e.g., appearance, 

adhesion, water resistance, scratch resistance). Like advanced materials, coatings are 

encountered frequently in everyday life, often without the knowledge of most individuals. 

Although coatings may be sold directly to consumers (common examples would be latex paint 

4-3 
 

 

 

 

 

 

 

 

 

 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Table 4-1. Descriptive Statistics for Key Combi-Adopting Industries, 2006 

Total Value U.S. 

of U.S. Total General Trade 

NAICS Industry Description 

Shipments 

($millions) 

Number of 

Employees 

Exports 

($millions) 

Imports 

($millions) 

Balance 

($millions) 

3252 Resin, Synthetic Rubber, 

and Artificial Synthetic 

Fibers and Filaments 

$94,373 78,557 $32,044 $14,504 $17,540 

Manufacturing 

325211 Plastics Material and $78,995 54,042 $25,751 $10,433 $15,318 

Resin Manufacturing 

325212 Synthetic Rubber $6,889 8,570 $4,040 $1,600 $2,440 

Manufacturing 

32522 Artificial and synthetic $8,488 15,945 $2,253 $2,471 −$218 

fibers and filaments 

manufacturing 

3255 Paints, Coatings, and 

Adhesives Manufacturing 

$32,911 59,628 $3,286 $1,335 $1,952 

32551 Paints and Coatings $23,358 39,006 $1,987 $852 $1,135 

Manufacturing 

32552 Adhesives Manufacturing $9,553 20,623 $1,299 $482 $817 

3256 Soap and Other Detergent 

Manufacturing 

$84,802 99,868 $9,707 $6,318 $3,389 

32561 Soap & Cleaning $40,467 44,684 4,565 2,123 $2,442 

Compound Mfg 

32562 Toilet Preparation Mfg $44,335 55,184 5,143 4,196 $947 

Sources: U.S. Department of Commerce, Bureau of the Census. 2007. 2006 Annual Survey of Manufactures. 

Obtained through American Fact Finder Database <http://factfinder.census.gov/home/saff/main.html?_lang=en>. 

U.S. International Trade Commission (USITC). 2008. Interactive Tariff and Trade DataWeb. Available at
 

<http://dataweb.usitc.gov/scripts/user_set.asp>. As obtained on May 1, 2008.
 

or car wax), they are more often sold to other industries as inputs into their manufacturing 

process. As a result, coatings are used in products individuals encounter everyday from the 

paints on automobiles to nonreflective coatings on television screens. 

In 2006, this industry employed over 59,000 workers and generated approximately $32.9 billion 

in total value of shipments (Table 4-1). The vast majority of these shipments (71%) were 

generated by paints and coatings manufacturers, while adhesives manufacturers represented 

the remainder. A relatively small amount of shipments ($3.2 billion or 10% of the total value of 

shipments) were directed toward overseas customers. 

All coatings are formulated by mixing chemical ingredients such as resins, catalysts, pigments, 

pigment dispersants, solvents, and other additives. Formulation is a complex process that 

historically has depended on experimentation for developing or improving products, thus making 

it a natural candidate for combi methods. 

Evidence that coatings manufacturers are already adopting combinatorial methods can be found 

in the press releases of large chemical companies. For example, in 2006, BASF and Robert 

4-4 
Chapter 4 — End Users, Industries, and Applications 

Bosch GmbH’s began developing a high-throughput screening plant to speed up the testing of 

pigments, resins, and additives in their coating formulations. According to Dr. Thomas Brinz, 

head of Bosch’s “Lab Systems” division, “The ultra-modern robotic facility designed jointly with 

BASF allows for the first time to examine complex coating recipes from the formulation of the 

raw materials through to the finished coating product thoroughly in an integrated facility. Thus, 

depending on the requirement, up to 100 different coating systems can be processed in a very 

short time” (BASF, 2006). 

Data on the amount of domestic R&D being conducted by coatings manufacturers is collected 

by NSF and aggregated with data collected for other chemical manufacturers, which in addition 

to coatings manufacturers included manufacturers of agriculture chemicals (NAICS 3253) such 

as pesticides, manufacturers of personal care and household cleaning products (NAICS 3256), 

and other chemicals (NAICS 3259). 

4.3 Personal Care and Household Products Industry 

Personal care and household products include cosmetics, perfumes, soaps, shampoos, and a 

variety of household cleaning solutions (NAICS code 3256). Major personal care product 

manufacturers include Procter & Gamble, Unilever, and Avon, all of which are current or former 

NCMC members. As one might expect from the familiar presence of soaps, perfumes, and other 

products in everyday life, personal care and household product manufacturers are the second 

largest of the three chemical manufacturing stakeholder groups. 

In 2006, this industry generated a total value of shipments of approximately $86 billion and 

employed over 99,000 workers (Table 4-1). That same year, exports for this industry were 

valued at approximately $9.7 billion (or 11% of the total value of shipments that year). 

Similar to coatings, personal care products are formulated from a variety of components that are 

mixed together to create their desired attributes. As a result, combinatorial methods could also 

increase the pace of product development. Evidence of the potential impact of combi on the 

development of personal and household products can be found in a 2002 interview between 

Chemical & Engineering News and Parmond Reddy (a chemist at NCMC alumnus Procter & 

Gamble). Dr. Reddy indicated that the work NCMC was doing on creating high-throughput 

methods for measuring interfacial tension was of great interest to developers of shampoos, 

detergents, and other cleaning products. Dr. Reddy said that interfacial tension was something 

researchers at Procter & Gamble “measure day in and day out.” Procter & Gamble participated 

in NCMC’s interfacial tension focus project, which NCMC initiated in response to industry 

comments such as Dr. Reddy’s. 

4.4 Academic and Government Research Laboratories 

Unlike the previous stakeholder groups we have discussed, academic and government 

laboratories are not generally driven by a profit motive, which may lead them to pursue different 

types of research than research pursued by other stakeholder groups. These labs are not 

4-5 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

primarily focused on developing products but on generating basic or applied scientific 

knowledge for general publication or for use by their funding organizations. The benefits that 

these institutions receive from combi differ very little from those received by other stakeholder 

groups: 1) time-saving methods for synthesizing and analyzing compositions more quickly than 

if they did so by hand, and 2) a means to broaden the scope of research questions they are able 

to study. 

Correspondence with individuals at NCMC and interviews with academic researchers identified 

at least 17 U.S. academic labs at research universities that have implemented combi workflows 

and likely benefited from NCMC’s technical accomplishments. NSF surveys academic 

institutions annually to determine the size and scope of their research activities. A list of the 17 

institutions and their 2006 R&D expenditures in the physical sciences and engineering 

disciplines are provided in Table 4-2.. The list primarily includes institutions that are known to 

have implemented significant combi workflows in their labs. This information is presented for 

informational purposes only; it is not known how much R&D is making use of combi at these 

institutions. 

Other institutions without combi workflows may also be using NCMC technical methods, 

especially gradient thin-film or microfluidics techniques. Similarly, we are also not likely aware of 

all the labs that have adopted combi workflows. 

In addition to academic institutions, other institutional laboratories may be affected by the 

NCMC. One prominent example is NCMC alumnus Wright-Patterson Air Force Base (formerly 

known as the Air Force Research Laboratory), which is dedicated to leading the discovery, 

development, and integration of military technologies. Also, the Lawrence Berkeley National 

Laboratory in California conducts nonclassified materials research that may have been affected 

by NCMC’s technical achievements. 

4.5 Equipment and Software Vendors 

Combinatorial research techniques require the use of tools that can create large amounts of 

sample compounds and quickly analyze their properties. These tools include physical 

equipment used for synthesizing samples and performing tests, as well as the software used to 

run these machines and to analyze the results they generate. Manufacturers of these tools 

represent an important factor in making combinatorial techniques accessible to researchers. 

As stated previously, equipment vendors like Symyx that were members of the consortium may 

benefit less from technical accomplishments generated by the NCMC because their market is in 

discrete sample libraries. Vendors may also focus on other kinds of materials, but even if 

polymers were not their primary interest, participating in the center assisted in their endeavors 

related to combi. However, manufacturers of discrete libraries may still benefit from the NCMC 

because it offers a venue for interacting with potential clients and staying up to date on research 

and market trends. 

4-6 
 

 

 

 

 

 

 

 

 

 

Chapter 4 — End Users, Industries, and Applications 

Table 4-2. Research Institutions Conducting Combinatorial Materials Research, 2006 

Institution 

Total R&D 

Funding 

($thousands) 

Physical 

Science R&D 

Funding 

($thousands) 

Engineering R&D 

Funding 

($thousands) 

Metallurgical/ 

Materials 

Engineering 

R&D Funding 

($thousands) 

Arizona State University 201,955 19,051 71,479 5,082 

Clarkson University 16,143 N/A N/A N/A 

Clemson University 179,840 10,376 81,309 14,815 

Georgia Institute of 440,898 35,570 294,466 14,286 

Technology 

Iowa State University 221,998 11,010 51,726 4,976 

Massachusetts Institute of 600,748 98,367 220,043 17,320 

Technology 

North Carolina State 330,936 22,189 96,066 8,237 

University 

North Dakota State 103,778 14,355 10,557 N/A 

University 

Princeton University 188,165 28,357 55,548 20,796 

Rutgers University 308,204 39,420 25,245 5,957 

Texas A&M University 492,955 38,430 161,735 5,750 

University of Colorado, all 512,794 63,594 36,721 N/A 

campuses 

University of 136,057 29,363 28,301 N/A 

Massachusetts, Amherst 

University of Pittsburgh 530,162 19,109 21,254 3,071 

University of Wisconsin, 831,895 55,697 92,933 5,272 

Madison 

University of Maryland, 354,244 60,990 88,321 6,040 

College Park 

University of Southern 39,163 9,907 1,313 N/A 

Mississippi 

Source: National Science Foundation. 2007. "Academic Research and Development Expenditures: Fiscal Year 

2006." Available at <http://www.nsf.gov/statistics/nsf08300/>. As obtained on May 11, 2008. 

Manufacturers of other types of equipment, such as equipment used for high-throughput 

measurements, might use NCMC technology in certain aspects of product development. For 

example, equipment manufacturers apply coatings to certain parts of their equipment. NCMC 

techniques could be used to determine how thickly they should apply the coating if another 

parameter such as temperature is involved. 

4-7 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Equipment and software vendors are spread across several NAICS codes and make up only a 

small portion of each. To determine which firms were active in the equipment and software 

stakeholder group, RTI searched the published literature. Table 4-3 lists the equipment and 

software vending companies RTI identified. This table reports each company’s sales, 

employment, R&D expenditures, and whether they provide equipment, software, research 

services, or a combination of the three. 

4-8 
 

 

 

 

 

 

 

 

 

 

 

Chapter 4 — End Users, Industries, and Applications 

Table 4-3. Equipment and Software Vendors for Combinatorial Workflows 

R&D 

Equipment 

(E), Software 

Company Location 

Sales 

($millions) Employees 

Expenditures 

($millions) 

(S), Services 

(V) 

Accelrys Inc. San Diego, CA $81 372 S 

Amtec GmbH Chemnitz, E 

Germany 

Anton Paar GmbH Graz, Austria E 

Avantium Amsterdam, E, S, V 

Technologies The 

Netherlands 

Biotage AB Uppsala, $1,175 332 $629 E 

Sweden 

Brookfield Engineering Middleboro, MA E 

Laboratories, Inc. 

Bruker Corporation Billerica, MA $548 1,905 $50 E,V 

CambridgeSoft Cambridge, MA $9 65 S, V 

Corporation 

Chemspeed August, E,V 

Technologies Switzerland 

HEL Group Hertfordshire, E, V 

UK 

hte Aktiengesellschaft Heidelberg, $7 60 P 

Germany 

Hysitron, Inc. Eden Prairie, $4 57 E 

MN 

J-KEM Scientific St. Louis, MO $1 18 E 

Jobin Yvon Inc Edison, NH $50 211 E 

Mettler-Toledo Greifensee, $1,595 9,100 $83 E 

Switzerland 

nScrypt Orlando, FL E 

Symyx Technologies, Santa Clara, $125 380 $66 E, S, V 

Inc. CA 

Tecan Inc. Maennedorf, $333 1,087 E 

Switzerland 

Varian Inc Palo Alto, CA $835 3,700 $60 E, V 

Veeco Instruments Inc Woodbury, NY $441 1,279 $63 E 

Waters Corporation Milford, MA $1,280 4,687 $77 E, S 

Zinsser Analytic GmbH Frankfurt, $6 33 E 

Germany 

Source: Company Forms 10-k and Hoovers.com. 

4-9 

5. STUDY METHODOLOGY 

The NCMC introduced novel combi technologies and demonstrated combi’s utility, invigorating 

interest in and accelerating combi’s adoption for R&D in organic polymers. In some cases, 

interviewees believe these advances would not have occurred for 5 or more years (if at all), and 

the technologies that would have ultimately come to market might have been substantially more 

expensive. 

The most common comment during stakeholder interviews was that the NCMC demonstrated 

what was possible with combi, which then catalyzed innovation within corporate research 

divisions and academic labs. Some NCMC technologies were adopted with little additional 

customization; others were adapted or influenced the development of custom systems that 

otherwise would not have existed. The consortium offered a coordinated technical community— 

both with semiannual members’ meetings and at large ACS and MRS conferences—in which 

researchers tasked with leading their firms’ efforts in high-throughput R&D strategy were able to 

interact with their peers and learn from NIST’s experts. 

Our approach to valuing economic impacts was to prepare a series of counterfactual analyses 

that linked hypotheses about the NCMC’s benefits to relevant technical and economic metrics. 

These metrics then informed processes for collecting data and estimating measures of 

economic benefit. Benefits were estimated against a scenario in which the NCMC did not exist 

and key advances in thin films and microfluidics libraries were not made. 

5.1 Conceptualizing NCMC’s Role and Economic Benefits 

The total economic impacts of infratechnology development and consortium programs such as 

the NCMC can be broken out into five distinct categories: 

	 technology adoption/adaption: the benefits of adopting new technologies or adapting 

them to suit an organization’s purpose 

	 knowledge base expansion: the benefits of acquiring technical information and data that 

improve an organization’s knowledge base 

	 consortium experience: the benefits of membership in narrowly defined forums in which 

all respondents are focused on a single technical topic 

	 downstream product cost and quality: improved quality of goods and services in which 

those materials are an input and that potentially reduce the costs of those goods and 

services 

	 time acceleration effects: accelerated the accrual of cost savings or product quality 

benefits for end users 

5-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

In economic analyses such as this one, each component of the total economic impact will 

generally have elements that are more tangible than others. The first three bullets pertain to 

R&D efficiency: acquiring infratechnology and knowledge that permit researchers to do more 

with less. NCMC methods and “how-to’s” for adapting common laboratory apparatus to perform 

high-throughput experiments and enable end users to accrue labor, material, and capital cost 

savings over the life of the project, relative to the defending manual one-at-a-time methods. 

Cost savings from the introduction of a new technology in a process can be calculated by 

comparing total costs for one period with those observed in a previous period. Those benefits 

are realized, meaning that they have accrued and are measurable. In this retrospective 

economic impact analysis, the study’s focus was on the NCMC’s realized benefits, which we 

compared with historical observed costs. Some NCMC technology has yet to be adopted and 

there is the possibility that this technology may hold benefits for end users in the future. 

Stakeholders also could not quantify downstream product quality benefits or end-user 

acceleration benefits. Quantification is very difficult, although it is likely that benefits do exist. 

Combi has directly affected key materials in many products, but these ingredients are combined 

with many others in a wide variety of products from shampoos to semiconductor devices to the 

hulls of cargo ships. These products’ performance is affected by work flowing out of the NCMC. 

What proportion of their net economic value is attributable to or was accelerated by combi was 

not sufficiently distinct to be quantifiable. 

For example, one interviewee said that “several” products her company markets were affected 

by combi in some way, but she declined to articulate how they were affected and to what extent 

NCMC innovations may have played a role in their development because the information is 

proprietary. Given that a portion of the NCMC’s economic value was not quantified, but most 

costs were captured, the net economic benefits and measures of economic return are 

conservative. 

5.2 Theoretical Approach to Estimating Economic Benefits 

This study quantified R&D efficiency benefits and acceleration benefits for organizations using 

combi. We estimated both the economic impact of combi in general and the portion of that 

economic impact attributable to the NCMC’s introduction of novel NCMC technologies. 

Acceleration benefits include the benefits for firms adopting combi earlier than they otherwise 

would have. 

5.2.1 Modeling Economic Impacts of the NCMC’s Contributions to Combi 

A recurrent theme in this report is that NIST’s contributions to combi are characteristic of 

generic technologies and infratechnologies. The technologies are generic because the suite of 

systems presented in Chapter 3 present functional concepts of how common laboratory 

instrumentation can be adapted or new apparatus can inexpensively assembled to implement 

combi. NCMC’s contributions in terms of technical approach, fabrication techniques, and a 

5-2 
 

 

 

 

Chapter 5 — Study Methodology 

system of measurement are infratechnologies because they are foundational to the use of 

combi for thin films and microfluidics. 

The increase in economic welfare from using combi is the benefits accruing to combi adopters 

from more efficient and effective R&D and the benefits accruing to end users from producing 

products whose features, cost, and/or quality are enhanced by materials emerging from combienabled R&D. End users would be companies purchasing a polymer product for input in their 

production process. 

The preceding section highlighted the challenges to measuring materials end-users benefits 

accurately. However, by developing representative firm R&D and production cost functions for 

combi adopters under the current and counterfactual conditions, we can measure the benefits 

accruing to them to develop a lower-bound estimate of combi’s impacts. Understanding the 

mechanism through which costs are incurred and benefits may accrue is essential to developing 

the taxonomy of costs and benefits underlying this analysis. It should be noted here that many 

combi end users are academic and government laboratories, but the discussion of how NCMCdeveloped technology benefits firms is also applicable to them. 

Combi’s immediate impacts are on the combi-adopting firm’s R&D costs: changes in the 

proportions of and costs of capital, labor, materials, and technology consumed during materials 

research. Industry-level impacts are the sum of impacts accruing to all firms composing the 

industry. Identifying the appropriate technical and economic impact metrics, measuring them 

relative to a counterfactual scenario, and extrapolating results from a survey sample to the 

industry population yields economic benefits. These benefits must, however, be reduced by the 

programs’ costs. 

A time series of net economic benefits would therefore be: 

Δ net economic benefitst = (∑Δ R&D capital savingsit + 

∑Δ R&D labor savingsit + 

∑Δ R&D materials savingsit) – 

(∑Δ technology acquisition costsit + 

∑NCMC program costsit) 

It is also possible that one R&D output is a new polymer product that is more cost-effective to 

produce. Thus, production costs may change. The new polymer product may also be of higher 

quality or have a suite of novel properties, in which case it could possibly command a higher 

price ‘in the market or sell more units’. If the firm was a price taker, as is common in bulk 

commodity industries, there may be no change in price. Beyond this conceptual view, and in 

practice, the model must reflect comments from scoping interviews with industry: appreciable 

5-3 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

differences in price, quantity, and production costs, though possible, were not disclosed by the 

real-world firms participating in the study. 

5.2.2 Accelerating Combi Adoption 

The above discussion of modeling firm-level economic benefits did not consider the time 

dimension. The time value of money means that a dollar is worth more today than tomorrow and 

accelerating the time series of economic benefits generates value. Thus, measuring the 

economic impact of combi overall was necessary to accurately assess the extent to which 

NCMC influenced the adoption of combi, regardless of whether a firm adopted any of the 

NCMC’s generic technology. 

The top portion of Figure 5-1 illustrates a stylized flow of costs and benefits from the perspective 

of the combi-adopting firm relative to not using combi. Costs accrue for implementing a combi 

workflow, but those costs are expected to be recouped by cost savings or profits over time. 

Traditional investment theory states that if the net present value (NPV) of the flow of benefits is 

greater than zero, then the company will undertake the investment project subject to a budget 

constraint. The NCMC’s mission was to accelerate combi adoption in addition to developing 

novel, cost-effective tools for combi adopters. 

The potential impact of NIST on users’ costs and benefits is shown in Figure 5-1. NIST’s 

contributions have 

(a) reduced combi workflow development and acquisition costs and time: these cost 

reductions are shown by the shaded square area. Time reductions are shown as a shift 

in the curve to the left. 

(b) reduced combi workflow implementation costs and time: the cost reductions are 

reflected in the nonparallel shift in the implementation stage.
 

Combined, the two time reductions shift the entire life-cycle curve to the left by time “t” resulting 

in the acceleration of benefits shown in the second shaded area of the lower portion of Figure 

5-1. 

Figure 5-2 illustrates the adoption of combi over time. The vertical axis indicates the cumulative 

share of adoptions through a given year, and the horizontal axis is time. Because adopting 

combi is not a discrete event and adoption grows over time, market adoption is modeled as a 

continuous diffusion curve. The S-shaped diffusion curve approaches the full industry potential 

asymptotically. 

5-4 
Chapter 5 — Study Methodology 

Figure 5-1. Firm-Level Acceleration of Costs and Benefits 

Cash Flow 

Time 

Development 

Z 

Implementation Use 

Benefits 

Costs 

Cash Flow 

Time 

Z 

(a) 

Reduced 

development 

costs 

(b) 

Reduced 

implementation 

costs 

Accelerated 

benefits 

5-5 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Figure 5-2. NIST’s Acceleration Effect on Combi Adoption 

Forecasting combi’s rate of diffusion is challenging because it is in the early stages of adoption. 

It is a function of the number of current adopters, the number of potential adopters, and the rate 

at which information and knowledge pass from one agent to another. This study forecasts 

diffusion using an S-shaped logistic curve. Originally, only a small number of firms adopt this 

technology. As more firms observe the benefits realized by initial adopters, they too adopt the 

technology. 

5.3 Specifying the Counterfactual Scenario without the NCMC 

Today some research laboratories are using combi that otherwise would not because of NIST’s 

program. NIST’s contributions to microfluidics and thin-films research were particularly 

disruptive. Symyx and other vendors were offering comparable and, in some cases, more 

sophisticated technologies for discrete libraries, but NIST was the leader in thin films and 

microfluidics for polymers. Interviewees believe it would have taken from 5 to 10 years before 

comparable, inexpensive strategies for thin films would have been devised without NIST’s 

contribution. 

The counterfactual scenario against which benefits were measured specified that 

 the novel suite of techniques for gradient approaches to thin films and elements of 

microfluidics and discrete sample library combi would not have been available before 

2008 and firms would have incurred greater capital, materials, and labor expenses as a 

consequence; 

 the overall adoption rate of combi for organic polymers would have been lower, meaning 

that some users’ adoption likely would not have occurred until years later; 

5-6 
 

Chapter 5 — Study Methodology 

	 the costs of adopting combi would have been higher, and some laboratories would not 

have been using combi at all; and 

	 the absence of the NCMC as a consortium hampered information diffusion and offered 

no coordinated forums for discussion of the development of the combi toolkit for organic 

polymers, particularly for thin films and microfluidics. 

While NCMC did not invent combi, it accelerated combi’s adoption through its outreach work in 

the technical community, demonstrating what was possible with combi and reducing and/or 

eliminating cost barriers, thus making it more cost-effective on an ongoing basis. To ease the 

information collection burden on survey respondents, we “packaged” all of NCMC’s technical 

accomplishments into a general combi toolkit that was a subset of the broader combi toolkit. We 

assessed acceleration and efficiency benefits for both combi overall and the NCMC toolkit. 

Exceptional research centers for combi development and combi applications in discrete libraries 

existed at universities such as North Dakota State University. However, these centers relied on 

corporate funding and were subject to CRADAs. Thus, in addition to their focus on combi for 

discrete libraries, the free flow of information outside of these centers could not have been a 

substitute for the NCMC. 

The value NCMC members, visitors, and alumni derived from the consortium experience, such 

as networking and marketing among peers, would not have occurred. In addition to these 

informal exchanges, training and productivity benefits from more formal exchanges, such as 

meetings and demonstrations, would not have occurred. The broader community would not 

have benefited from papers, how-to’s, proceedings, and demonstrations emerging from the 

Center. 

5.4 Taxonomy of Quantifiable Economic Benefits and Costs 

In this section, we offer the specific categories of benefits and costs as well as the technical and 

economic metric pairs used to measure and monetize them. Table 5-1 presents an overview of 

the taxonomy of costs and benefits analyzed in this study. 

5.4.1 Value of Consortium Experience 

NCMC members often cited the consortium experience as one of the key benefit categories for 

RTI to investigate. The consensus view among interviewees was that one of the primary 

benefits of the NCMC was networking and engaging with other researchers in a precompetitive 

environment sponsored by a governmental group. Researchers stated that company policies 

and the presence of competitor firms often make it difficult for them to engage in substantive 

conversations in a group environment. At NCMC meetings researchers were able to connect 

with other researchers working on similar problems one on one. This was possible because the 

NCMC focused on precompetitive research and topics. In other words, participation in the 

NCMC, and the amount of feedback and information NCMC collected as a consequence, was 

possible in part because of NCMC’s choice to center discussion on generic and emerging 

5-7 
 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Table 5-1. Summary Taxonomy of Quantifiable Benefits, Acquisition Costs, and NIST 

Costs 

Benefit Description Benefit Components 

Adoption Cost 

Components 

NIST Cost 

Components 

Consortium NCMC hosts NCMC Members: NCMC Members:  Program 

experience researchers and 

provides an environment 

for networking and 

informal exchange with 

leading firms applying 

combi on a 

precompetitive basis 

 Marketing benefits 

 Transactions 

benefits 

 Recruitment 

benefits 

 Preferential access 

 Researcher training 

 Demonstrations 



 Membership fees 

 In-kind expenses 

 Labor participation 

 Travel costs 

administration 

 Technical program 

costs 

 Facilities investment 

 Conference 

participation and 

organization 

 Miscellaneous costs 

Knowledge 

base 

NCMC presentations, 

technical documentation, 

All Beneficiaries: 

 Validation of 

expansion data, and analytical 

results 

research strategies 

 Avoided essential 

research 

• New research 

methods 

Technology NCMC-developed or NCMC Members: NCMC Members: 

adoption -brokered techniques 

and strategies enable 

efficiencies in R&D and 

accelerate product 

development 

 Focus project 

research 

All Beneficiaries: 

 Workflow design 

 Reduced capital 

expense 

 Labor efficiency 

 Materials efficiency 

 Accelerated 

product 

development 

 Focus project fees 

All Beneficiaries: 

 Equipment and 

materials 

 Toolkit 

development 

 Installation 

 Training 



materials classes and methods development rather than exact materials types and specific 

measurement solutions. 

Many members interviewed believed that the NCMC improved technology transfer among 

researchers as well as from the NCMC to researchers. One interviewee noted that “one of the 

biggest advantages of working with NIST is that you’re working with a group of very smart 

people that are far more advanced in terms of where you are on the learning curve from 

measurement aspects.” What respondents also found to be compelling was that the Center was 

more practically focused than most academic consortia. They viewed the costs of participation 

as low, the reputation of NIST as a neutral partner as a key success factor, and being part of the 

community as having substantial tacit benefits. 

We identified four broad categories to capture the benefits from the exchanges within the 

consortium model (see Table 5-2): 

5-8 
Chapter 5 — Study Methodology 

Table 5-2. Benefits Metrics to Capture the Value of the Consortium Experience 

Benefit 

Category Specific Cost Technical Metric Economic Metric 

NCMC meeting 

researcher 

training and 

demonstrations 

Labor costs Labor hours for acquiring 

comparable technical knowledge 

from secondary resources 

Loaded hourly wage rate for 

scientists participating in the 

consortium 

Labor costs Labor hours expended on 

developing comparable training 

sessions 

Loaded hourly wage rate for 

scientists participating in the 

consortium 

Marketing and 

networking 

benefits 

Labor costs Labor hours for achieving 

comparable visibility and 

awareness among peers 

Loaded hourly wage rate for 

senior scientists 

participating in the 

consortium 

Travel expenses Estimated associated travel 

needs to achieve visibility and 

Estimated travel expenses 

awareness 

Conference fees Estimated comparable 

conference attendance for 

Estimated conference fees 

achieving the same 

Recruitment 

benefits 

Labor costs Labor hours expended to 

identify similarly qualified 

employment candidates 

Loaded hourly wage rate for 

senior scientists 

participating in the 

consortium 

Labor costs Labor hours to bring non-NCMC 

postdoc candidate to equivalent 

skill level 

Loaded hourly wage rate for 

recent postdoc hires 

Transactions 

benefits 

Labor costs Incremental labor hours 

otherwise expended to facilitate 

transactions among NCMC 

members in the absence of the 

Loaded hourly wage rate for 

senior scientists 

participating in the 

consortium 

consortium 

Note: RTI used wage-rate estimates from the Bureau of Labor Statistics’ (BLS’) Occupational Employment Survey to 

1) enhance repeatability of our results and 2) reduce the information collection burden on interviewees. According 

to the BLS, the average annual salary, excluding benefits, for a materials scientist engaged in scientific research 

and development services was $84,790 in 2007. The hourly wage estimate was multiplied by 2 to account for 

fringe, administrative, and overhead expenses. 

	 training benefits from hands-on demonstrations, presentations from leading researchers, 

and discussion series; 

	 marketing and networking benefits stemming from participation in an industry consortium 

dedicated solely to combi for organic polymers; 

	 recruitment benefits associated with hiring an NCMC postdoc with hands-on NCMC 

training as opposed to researchers without that experience and investing in their training; 

and 

5-9 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

	 transactions benefits associated with interactions among potential customers, teaming 

partners, and collaborators. 

5.4.2 Value of Knowledge Base Expansion 

Members often mentioned that although some concepts and methodologies developed in the 

NCMC did have a specific application within their organization, the knowledge they gained from 

observing these advances was beneficial. For example, research projects or presentations from 

invited lecturers at semiannual meetings may have offered insights that enabled the researchers 

to overcome a technical obstacle similar to what was discussed in the consortium or invalidated 

an internal technology proposal that would not have born fruit. 

There is a training benefit to the NCMC that is difficult to duplicate elsewhere. Most members 

stated that the training value of being party to the Center was alone worth the annual 

membership fee. Similar benefits included learning new methods and viewing demonstrations of 

novel combi approaches (including apparatus design and protocols). 

The NCMC assists the combi community by providing essential research, offering insights into 

key focus projects, and validating research strategies. Access to NCMC prepublications permits 

members to learn about new techniques before they enter the public domain. The broader 

technical community benefits from NCMC innovations (how-to’s, technical papers, software, 

data), but members have access to this information more than 1 year before the broader 

community does. 

In the words of one member, the NCMC has a “ripple effect” within R&D labs: the research they 

perform yields knowledge that validates or invalidates research strategies. The member said 

that they learned that one of their research projects would fail before they started the project 

and, thus, assigned the resources to more promising work. 

We identified three broad categories to capture the benefits from the formal exchanges within 

the consortium model (see Table 5-3): 

	 researcher training benefits stem from the NCMC’s core mission to educate researchers 

on adopting and implementing a combinatorial workflow in their R&D labs through 

publications and presentations; these benefits accrue to the consumers of the NCMC’s 

technical output who may be NCMC members or nonmembers; 

	 avoidance of basic research expenditures that accrue from leveraging NCMC 

knowledge, thereby avoiding labor, capital, and materials spending; and 

	 validation or invalidation of research strategies, which leads to greater R&D efficiency. 

5-10 
Chapter 5 — Study Methodology 

Table 5-3. Benefits Metrics to Capture Value of Knowledge Base Expansion 

Benefit 

Category Specific Cost Technical Metric Economic Metric 

Researcher 

training 

Labor costs Labor hours for acquiring 

comparable technical knowledge 

from secondary resources 

Loaded hourly wage rate for 

scientists participating in the 

consortium 

Labor costs Labor hours expended on 

developing comparable training 

sessions 

Loaded hourly wage rate for 

recent postdoc hires 

Avoided 

research 

Labor costs Avoided labor hour expenditure 

on foundational research 

Loaded hourly wage rate for 

senior scientists 

participating in the 

consortium 

Materials costs Avoided materials purchases Estimated materials costs 

Capital costs Avoided equipment purchases 

and depreciation on basic 

research 

Estimated equipment costs 

Validation of 

research 

strategies 

Labor costs Avoided labor hour expenditures 

on strategies that the NCMC 

disproved; efficiency gains from 

identifying the correct approach 

initially 

Loaded hourly wage rate for 

scientists participating in the 

consortium 

Materials costs Avoided materials purchases Estimated materials costs 

Capital costs Avoided equipment purchases 

and depreciation on basic 

research 

Estimated equipment costs 

5.4.3 Value of Technology Adoption and Adaptation 

The NCMC introduced novel technologies in combi workflows and accelerated others. 

Respondents in our scoping interviews stated that although “off-the-shelf” solutions to several 

NCMC innovations existed, the cost advantages of the NCMC approaches enabled a wider 

variety of laboratories to adopt combi. NCMC technologies have one-time cost advantages 

(e.g., equipment and set-up) and ongoing savings. 

Combi allows researchers to miniaturize experiments (less material, less waste) and reduce the 

number of tests performed (less time). Moreover, continuous libraries permit nearly infinite 

internal analysis of a sample space, something that could not be done otherwise. Scoping 

interview respondents stated that combi offers a 10- to 25-fold improvement in time efficiency 

versus hand preparing and analyzing individual samples, and that cost savings mostly stem 

from time savings. The benefits categories for technology adoption and adaptation are in the 

same structure as the preceding two benefits discussions: labor, capital, and materials. We 

5-11 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

want to capture the economic value of less material, less waste, and less equipment expense 

(see Table 5-4). 

5.4.4 Technology Acquisition Costs 

All costs borne by all non-NIST groups fall into a broad category called technology acquisition 

costs. Technology acquisition cost is a contra account to benefits because the benefits firms 

accrue are offset by costs for consortium fees, training and travel expenses, and equipment 

expenditures, for example. While the surveys may request end users to provide measures of 

benefit net of cost, an accurate taxonomy of adoption cost categories is needed to illustrate the 

magnitude of investment needed to derive quantified benefits. 

Technology adoption cost categories encompass 

 NCMC membership fees and focus project cost shares; 

 labor expenses for NCMC meeting attendance and focus project research; 

 in-kind labor, materials, and equipment expenses in lieu of other NCMC contributions; 

 travel and conference expenses; and 

 direct labor, materials, and equipment expenditures to adopt or adapt NCMC technology. 

5.4.5 NCMC and Tool Development Costs 

MSEL management provided financial data on the NCMC for the program dating from early 

2002 when the Center was first created. These data were supplemented with information on the 

costs from the pioneering research in thin-films libraries. 

Table 5-4. Benefits Metrics to Capture the Value of Technology Adoption 

Benefit 

Category Specific Cost Technical Metric Economic Metric 

Productivity Labor costs Incremental labor expenditure to 

acquire equivalent data volumes 

from other sources 

Loaded hourly wage rate for 

scientists participating in the 

consortium 

Training Labor costs Avoided training on dedicated 

equipment that may have 

otherwise been purchased 

Loaded hourly wage rate for 

scientists participating in the 

consortium 

Materials usage 

optimization 

Materials costs Incremental materials consumed 

to acquire equivalent data 

volumes 

Estimated materials costs

 Materials 

disposal costs 

Avoided hazardous waste 

disposal costs 

Estimated disposal costs 

Avoided capital 

expenditures 

Capital costs Avoided dedicated equipment 

purchases and installation 

Estimated equipment costs 

5-12 
 

 

Chapter 5 — Study Methodology 

5.5 Time Period of Analysis 

Choosing a time frame for estimating the economic impacts of such technological improvements 

can be difficult because there are potentially long and variable lags between commencing a 

research activity and generating an improvement in technology (Alston et al., 1995). Even after 

research leads to a technological improvement, several years may pass before the technology 

begins generating benefits through its adoption. A simplified illustration of this process of 

development and adoption is provided in Figure 5-1. 

When thin-films library research began at NIST in 1998, applying combinatorial methods to 

polymers was a relatively new field of research—the first paper describing such an application 

was published only the previous year in 1997 (Brocchini et al., 1997). As a result, there were no 

commercially available instruments or published techniques for creating libraries of polymer 

samples, yet these methods had great potential for industry applications. NIST’s Polymers 

Division responded to this need by investing in fundamental research that centered on creating 

a foundation for practical applications of combi for materials discovery. 

The period of analysis begins in 1998, the year in which Eric Amis, Alamigir Karim, Carson 

Meredith, and others at NIST began developing gradient thin-film libraries, and ends in 2007, 

the last calendar year for which full-year NCMC financial data are available. Scoping interviews 

with end users suggested that thin-film libraries account for a significant proportion of economic 

benefits. It is important to capture the costs associated with those technical advances; therefore, 

the period of analysis begins 4 years prior to the official launch of the NCMC in 2002. 

The results of NIST’s research were first presented in 1999 at the Gordon Conference on 

Reactive Polymers, Ion Exchange, and Adsorbents (Meredith et al., 1999) and were ultimately 

published in a series of papers in 2000 (Meredith et al., 2000; Meredith, Karim, and Amis, 

2000). This marked the beginning of when stakeholders in private companies and academic 

research labs could begin adopting or adapting the techniques NIST developed in their own 

research. 

5.6 Primary Data Collection 

Data informing this analysis were collected from semistructured interviews, survey forms 

completed via e-mail and returned to RTI, and an Internet survey of materials discovery 

scientists. Most responding scientists were employed by NCMC members or alumni, research 

universities, or large chemical companies that did not join the NCMC. All were directly involved 

in polymers R&D. 

Primary data collection efforts were conducted in three phases:2 

2 The formal data collection surveys in this analysis were approved by OMB as well as RTI’s Institutional Review 

Board. 

5-13 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

	 Phase I consisted of scoping interviews over a wide range of topics with researchers 

from the NCMC, its member and alumni companies, and academic researchers. The 

purpose of the scoping interviews was to explore combi and its impacts to inform 

hypotheses about the NCMC’s economic benefits. RTI researchers also attended the 

NCMC-13 meeting. 

	 Phase II was the distribution of the survey forms that could either be completed on the 

Internet (https://combisurvey.rti.org) or via e-mail using an electronic form. We used two 

forms: one for NCMC members and alumni (Appendix B) and one for the broader 

community (Appendix C). To maximize distribution of our survey announcements, we 

partnered with the MRS, which placed survey announcements in their members’ 

newsletter, “Materials360,” and distributed a link via ACS’s Polymers Group Listserv. 

	 Phase III was predominantly a series of follow-up interviews with respondents to either 

the member or the nonmember survey. A limited number of supplemental scoping 

interviews were also conducted to verify our assessment and synthesis of data from 

Phase II. 

This study respected the sensitive nature of the information provided by respondents needed to 

quantify economic benefits. As a philosophy of high-throughput experimentation, combi is 

central to firms’ R&D strategy. We endeavored to work with companies to get approvals for their 

researchers’ participation in this analysis given that our focus was on the benefits of combi and 

not the substantive content of their research strategies. Respondents were promised 

confidentiality and that only aggregated data would be presented in the report. Respondents to 

the Internet survey had the option to respond anonymously. 

5.7 Extrapolation to National Impact Estimates 

Results from the survey sample were extrapolated to national estimates using publicly reported 

R&D expenditures for firms in the advanced materials, paints and coatings, and personal and 

household care product industries that are likely to be using combinatorial methods in their 

research. Sales and employment data are traditionally used to extrapolate benefits, but as 

combi resides in the R&D function, and the percentage of R&D effort related to combi is the 

most accurate measure of activity. 

The challenge to extrapolation presented by combi is that it is an approach applicable to 

multiple research settings in multiple industries. For the period of interest, however it is 

generally believed that adoption by industry has been highly concentrated in a small number of 

industries and by firms of a certain size class within those industries. As described in Section 4, 

an industry representative revealed that the firms most likely to be using combi in these 

industries have net sales exceeding $1 billion per year. Therefore, to create an extrapolation 

base RTI used the Hoovers industry database to identify all public companies with either direct 

or secondary business lines in manufacturing advanced materials, paints & coatings, and 

personal care products with sales exceeding $1 billion per year in 2007. We also included all 

current and former NCMC members. After the first pass at assembling the basket of 

5-14 
Chapter 5 — Study Methodology 

representative companies that would serve as the core R&D base, we added key foreign and 

domestic companies that both have US R&D operations and that fit the profile of a likely target 

combi adopter. The inclusion of some foreign firms in the extrapolation base was important 

given that they maintain significant R&D operations in the U.S. The firms comprising the 

extrapolation base are listed in Appendix A. 

RTI thoroughly quality checked this list of companies to ensure that the companies included 

were relevant for the purposes of this study. After this quality checking was complete, RTI 

utilized the COMPUSTAT North American database to collect annual revenue and R&D 

expenditure data for these companies from 1998 to 2007. 

Given the sensitivity in querying companies about their R&D activities and expenditures, the 

survey requested firms to provide their stock ticker or their publicly-report R&D budgets. RTI 

aggregated benefits for all survey respondents and then used the ratio of this benefits total to 

their aggregated R&D budgets to generate national impact estimates. Chapter 6 provides a 

summary table of the extrapolation base. 

Note that a firm’s inclusion in R&D base does not mean that they use combi or participated in 

this analysis. Rather, the extrapolation base was created first with current or former NCMC 

members and then augmented according to the screening criteria outlined above. It is also 

possible that a survey respondent outside of the given criteria was not included as part of the 

extrapolation base because the extrapolation base was to be publicly reported. 

RTI did not extrapolate benefits from consortium-specific measurements to national impact 

measurements. These benefits accrued only to current and past NCMC members and were 

measured solely from a census of all members and alumni. RTI did, however, extrapolate 

benefits pertaining to technology adoption, adaption, and knowledge base expansion, which 

were not limited to NCMC members. 

5.8 Measures of Economic Return 

Three benchmark measures—benefit-to-cost ratio (BCR), NPV, and internal rate of return 

(IRR)—were used to evaluate the time series of quantified benefits and costs. 

Benefit-to-Cost Ratio 

The BCR calculated in this analysis is the ratio of the NPV of benefits to the NPV of costs, which 

accounts for differences in the timing of cash flows (which has implications for the real value of 

$1 in one time period versus another). 

Letting Bt be the benefits accrued in year t by firms and Ct the total costs for the project in year t 

by NIST, then the BCR for the program is given by 

5-15 
 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

B1(t+i) B2(t+i)

∑

n

i +∑

n

i (1+ r) (1+ r) i=0 i=0 (BCR) = (5.1) C(t+i

∑

n ) 

i (1+ r) i=0 

where 

t is the first year in which benefits or costs occur, 

n is the number of years the benefits and/or costs occur, and 

r is the social discount rate. 

In this study, r was set at 7%, the Office of Management and Budget (OMB)-specified level.3 

Because benefits and costs occur at different time periods, both are expressed in present-value 

terms before the ratio is calculated. Essentially, a BCR greater than 1 indicates that quantified 

benefits outweigh the calculated costs. A BCR less than 1 indicates that costs exceeded 

benefits, and a BCR equal to 1 means that the project broke even. 

Net Present Value 

The NPV of the investment in a project is calculated as 

n ⎡ B1 B2 C ⎤ ( t + i ) ( t + i ) ( t + i ) NPV = ∑ ⎢ + − ⎥ , (5.2) i i i i = 0 ⎢(1 + r ) (1 + r ) (1 + r ) ⎥ ⎣ ⎦

where the terms have the same meanings as identified for Equation (5.1). Any project that 

yields a positive NPV is considered economically successful. Projects that show a positive NPV 

when analyzed using OMB’s 7% real discount rate are socially advantageous. A negative NPV 

would indicate that the costs to society outweigh the benefits, and an NPV equal to zero would 

indicate a breakeven point. 

Internal Rate of Return 

The IRR on an investment should be interpreted as the percentage yield on an R&D project 

over the life of the project, often multiple years. In mathematical terms, the IRR is the value of r 

that sets the NPV equal to zero in Equation (5.2) or results in a benefit-cost ratio of 1 in 

Equation (5.1). 

The IRR’s value can be compared with conventional rates of return for comparable or 

alternative investments. Risk-free capital investments such as government bonds can be 

expected to yield rates of return under 5% in real terms, while equities seldom return more than 

10% over an extended period of time. In academic studies of the diffusion of new technologies, 

however, real rates of return of 100% or more have been found for significant advances with 

3

See OMB Circular A-94. 

5-16 
 

Chapter 5 — Study Methodology 

broad social benefits (Tassey, 2003). It should be noted that, in cases for which costs exceed 

benefits, an IRR cannot be calculated. 

5-17 
 

 

6. NET ECONOMIC BENEFIT ESTIMATES 

This chapter presents quantified net economic benefit estimates attributable to the NCMC of 

 participating in the NCMC consortium, 

 expanding the combi knowledge base, 

 introducing novel technologies, and 

 accelerating many firms’ adoption of combi and HTE. 

The research divisions of many coatings, materials, and personal care and household products 

firms provided the data and contextual information that made it possible to quantify estimates 

and assess the NCMC’s contribution to combi and materials science. Almost all private-sector 

survey respondents fell into these industry groups. Their responses were augmented by 

contributions from academic researchers, equipment and software vendors, and government 

researchers. Some respondents completed survey instruments and participated in multiple 

rounds of informational interviews. More than 70 researchers participated via surveys, 

telephone interviews, or in-person discussions. 

Figure 6-1 compares study respondents’ publicly reported R&D expenditures to the total 

estimated R&D expenditures for firms in the affected industry segments with annual sales 

greater than $1 billion.4

 As discussed in Chapters 4 and 5, combi is still in the early stage of 

adoption for polymers R&D, and most firms currently using combi are large publicly traded firms. 

Only two firms with annual sales below $1 billion responded to the survey. Respondents to our 

survey had publicly reported R&D expenditures of $29.2 billion, which was equivalent to 72% of 

the total R&D expenditure estimated for combi users in the industries covered by this analysis. 

Quantified benefits in this chapter are conservative for four principal reasons. First, we were 

only able to reliably quantify impacts for early combi adopters and adopting industries. Adoption 

of NCMC combi technology outside of the materials, coatings, and personal care and household 

products industries could not be estimated. Economic benefits accruing to other industries were 

not quantified. Second, as presented in Chapter 5, benefits for industries that use combienabled polymers and benefits for the final end user that accrues quality or performance 

benefits from polymer-containing products could not be quantified. Third, benefits for two 

stakeholder groups, equipment vendors and academic laboratories, could not be reliably 

estimated. 

4

 As stated in Chapter 5, relevant R&D expenditure data used for extrapolating estimates to national benefit estimates 

was the sum of R&D expenditures by all firms in the affected industries with revenues greater than $1 billion. We 

refer to this group of firms as the affected industry segment because industry experts predicted that firms in 

smaller size classes are not likely to be using combi at this time. The decision to use publicly reported R&D 

budgets as the extrapolation measure was in response to the extreme care necessary when exploring the R&D 

activities of firms in highly competitive industries. 

6-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Figure 6-1. Survey Participation as a Proportion of Total Industry Publicly Reported 

R&D Expenditures 

$0 

$5,000 

$10,000 

$15,000 

$20,000 

$25,000 

$30,000 

$35,000 

$40,000 

$45,000 

Publicly-Reported R&D Expenditures

($ billions) 

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 

Participants Estimated Total Industry 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. Appendix A lists the firms 

whose R&D expenditures compose the total estimated expenditures for firms with sales greater than $1 billion in 

materials, coatings, and/or personal and household care products industries. Also included in the estimate are 

current and former NCMC members. The rationale underlying firm selection is presented in Chapter 5. 

Many of the same types of benefits accruing to industry researchers also accrue to academic 

ones. Several academic researchers cited cost and time savings relative to earlier sample 

preparation and analysis methods when applying NCMC combi technology. Although academic 

respondents stated that they used the NCMC approaches infrequently, when applied, those 

approaches offered efficiency benefits. Fourth, combi’s adoption is ongoing, and in addition to 

unquantified benefits detailed above, there are benefits beyond 2007 that have yet to accrue 

and were also not estimated. 

6.1 Respondent Profile and Combi Usage History 

This section complements Figure 6-1 and profiles survey respondents and their reported combi 

usage. Seventy-four percent of respondents reported that they used combi or that they had 

used it in the past. Of those who reported not using combi, 60% expect to use it in the future. 

Respondents who have not used combi were exited from the survey after being offered the 

opportunity to comment on why they did or did not expect to use combi in the future. Those who 

expected to use combi cited that lower costs and a maturation of how to use combi polymers 

made the approach more attractive relative to other methods. Respondents who did not expect 

6-2 
Chapter 6 — Net Economic Benefit Estimates 

to use combi almost uniformly stated that combinatorial approaches were not well suited for 

their research. 

Two respondents reported suspending their combi program because of difficulties in applying 

these methods or because they believed the costs were too high relative to the benefits they 

were accruing or had expected to accrue. Other interviewees reported that several firms 

encountered challenges with adopting and integrating combinatorial workflows in the early 

2000s. Indeed, some NCMC members’ decision to join the consortium was in part to learn from 

other firms’ and NIST’s experiences. Eighty percent of members and alumni joined the NCMC 

as part of their combi adoption strategy. The remaining 20% joined to interact and network with 

researchers and to stay abreast of trends in combi (see Table 6-1). 

On average, respondents reported becoming aware of the successful, practical use of combi in 

late 2001 and began using it between 5 and 6 years ago (see Table 6-2). Members and 

nonmembers reported learning about advances in combi for polymers through presentations at 

MRS, ACS, Gordon Conferences, and the Knowledge Foundation. They also reported learning 

about combi through the technical literature and through direct interactions with NIST 

researchers. A small number of firms were developing their own combi techniques and 

programs for discrete sample libraries in the late 1990s, making them very early adopters. Their 

research coincided with NIST’s early research into thin films libraries. 

Table 6-1. Respondents’ Combi and HTE Use 

Survey Question Yes No 

Does your company or university research laboratory(-ies) currently use 74% 26% 

combi or HTE for polymers R&D, or has it used combi in the past? 

If no, do you expect to use combi in the future? 60% 40% 

Did your company join the NCMC as part of a combi adoption plan 80% 20% 

(NCMC members survey only)? 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” Survey. 

Table 6-2. Timing of Respondents’ Combi Adoption 

Survey Question: “In which year(s) would you estimate that Mean 

the following occurred?” Range 

Researchers in our lab became aware of the successful, 2002 1997–2007 

practical use of combi for polymers R&D 

My lab(s) implemented and began using combi for the first time 2003 1997–2008 

Combi became a part of the regularly-used methods in our 2004 1997–2009 

lab(s) 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” Survey. 

6-3 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

During interviews several members stated that they would have preferred the NCMC to do more 

research and toolkit development in discrete sample libraries since this library type was most 

common. Yet, as mentioned earlier in this report, discrete sample libraries benefited from an 

established commercial infrastructure, and NIST’s goals were to expand and demonstrate the 

viability of combi for new materials systems using low-infrastructure gradient thin film and 

microfluidic methods. Thus, there was some degree of disconnect between the stated interests 

of industry in desiring greater NCMC research on specific systems, and NIST’s mission of 

bridging technology gaps by focusing on more general demonstrative cases. NCMC members 

without interests in future research directions ceased participation and transitioned to alumni 

status. Despite the apparent disconnect, NCMC continued to attract new members, and alumni 

stated that they derived value from their membership but that their research interests no longer 

aligned with NCMC’s agenda. 

Figure 6-3 presents a time series of full-time equivalents (FTEs) respondents reported tasking 

with developing and applying combi.5

 The data in this figure provide insight into the pattern of 

combi adoption. The data suggest that the amount of person effort expended to develop combi 

among respondents is declining and that a greater amount of effort is devoted to applying 

combi. Overall, the effort expended developing and adopting combi workflows exceeded the 

labor expended applying those methods in ongoing research activities until 2005. The run up in 

combi development activities was largely between 2000 and 2005, which is consistent with the 

combi adoption timing reported in Table 6-2. 

6.2 Economic Benefits of the NCMC Consortium Experience 

The first major component of the NCMC’s economic benefit we sought to quantify was that of 

participating in the NCMC consortium itself. During early interviews, members indicated that 

their participation offered more productive marketing and networking opportunities with other 

researchers in combi than other conference opportunities. Consequently, the survey asked 

NCMC members and alumni to reflect on the benefit of partnering with NIST and participating in 

the NCMC. 

Results show that participating in the consortium, whose meetings were attended by 30 to 50 

industry researchers plus invited guests and NIST staff, was reported as being of significant 

benefit (see Table 6-3). The consortium offered valuable demonstration and training 

opportunities that enabled them to keep more readily abreast of the latest in combi-related 

research. They were also able to acquire information on novel approaches and inexpensive 

strategies for incorporating them into their workflows. On an annualized basis, 78% of members 

and alumni stated that the benefit of participating was equal to or greater than the membership 

fees paid. Half indicated that the NCMC validated internal research strategies or invalidated 

strategies that would not have born fruit. 

5

 One FTE is equal to 2,000 person-hours of effort (i.e., not one person). 

6-4 
Chapter 6 — Net Economic Benefit Estimates 

Figure 6-3. Estimated FTEs Developing and Applying Combi in the Advanced 

Materials, Coatings, and Personal Care and Household Products Industries 

120 

100 

80 

60 

40 

20 

0 

1999 2000 2001 2002 2003 2004 2005 2006 2007 

FTEs Engaged in Combi Development 

FTEs Applying Combi in Polymers Research 

Note: Data are in FTEs for stated industries only. The data represent FTEs and not the number of persons who may 

use combi overall within these industrial sectors. Equivalent data for other industries and for academia could not be 

estimated reliably. 

Table 6-3. NCMC Members’ Perceptions of the Benefits of Membership 

Survey Question Yes No 

Was the annual benefit your organization accrued from participation in 78% 22% 

the NCMC equal to or greater than the annual membership fees paid? 

Did participation in the NCMC 83% 17% 

... offer more productive marketing or networking opportunities, 

relative to other conferences or combi meetings? 

... validate or invalidate internal research projects and permit more 50% 50% 

efficient and effective resource allocation? 

... help your organization avoid any research activities, or enable your 70% 30% 

organization to acquire needed research more quickly or costeffectively? 

... offer valuable researcher training and demonstrations beyond what 67% 33% 

would have been available elsewhere? 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” Survey. 

6-5 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

A former NCMC member commented that “the greatest benefit NCMC provided was a regular 

forum for combi materials outside of commercial vendors, which helped provide technology and 

competitive awareness [as well as] discussion and networking opportunities within the combi 

materials community.” 

Another member noted that “the biggest benefit we have gained from NCMC is the ability to see 

the internals of complete ‘combi’ procedures, not just the finished product. So, by being a 

member of NCMC we have been able to see a project being defined, problems being identified 

and solutions being developed—equally the handling of samples and information from the 

beginning to the end of an experimental process—unlike the snapshots seen via commercial 

interactions.” 

Two consortium benefits were quantified: (1) demonstration and training benefits from 

attendance at semiannual meetings and (2) hosted researcher training benefits. Interviewees 

told us that these semiannual meetings offered more productive networking and marketing 

opportunities than other combi conferences. However, we were unable to quantify these 

benefits. 

Each of the 12 semiannual meetings held between 2002 and 2007 had a theme, and noted 

researchers were invited to present their research pursuant to that theme (see Chapter 2). Two 

additional NCMC meetings in 2008 were excluded because they fell outside of the analysis 

period. Common among all meetings were technology demonstrations and discussion series 

about emerging technical challenges. 

Meetings were attended by 355 member representatives. Members estimated the typical 

additional person-hour effort that would have been required to derive the same value as they 

actually accrued from these meetings to be between 25 and 32 hours per meeting per 

attendee.6

 All reported hours in this analysis were quantified using the mean hourly wage for 

materials scientists in research and development ($40.76 [BLS, 2007])), which was then 

multiplied by 2 to estimate fringe and administrative expenses. The total benefit between 2002 

and 2007 was estimated to be about $850,000. 

NCMC also hosted visiting researchers from member and partner organizations. The benefit of 

the training was estimated in a similar fashion as the consortium participation benefits above— 

savings of the additional person-effort that would have been required to achieve the same level 

of knowledge and sophistication. Interviews indicated that the additional training investment 

would have been equivalent to the length of stay at NCMC. Thus, annual benefits were the sum 

of the product of the wage rate above and the estimated number of days NCMC hosted a 

6

 Individual members responded for their estimated person-hour benefits for the meetings their representatives 

attended. Data were extrapolated to all attendees using the ratio of responding members who attended the 

meeting to all NCMC member attendees. Consequently, there is some variation in the annual benefit per 

attendee. 

6-6 
 

 

 

 

 

Chapter 6 — Net Economic Benefit Estimates 

researcher. NCMC records indicate that the Center offered hosted researchers 455 training 

days, which therefore amounted to a benefit of $286,000. 

In addition to the benefit stream presented in Table 6-4, NCMC graduated postdocs to industry 

and academia who received 2 years of specialized combi training and were integral to the 

NCMC’s efforts to develop the combi knowledge base. One former postdoc working with a 

NCMC alumnus posited that the net benefit accruing to their hire was equivalent to about 8 

months to 1 year of additional training (see Chapter 2). In all, more than 25 postdocs have 

completed 2-year positions and are now working in industry, academia, and government. 

6.3 Economic Benefits of NCMC’s Expansion of the Combi Knowledge Base 

The second component of economic benefits quantified was NCMC’s contributions to the 

expansion of the body of knowledge foundational to implementing and using combinatorial 

approaches. The survey presented respondents with a list of technologies developed by the 

NCMC and its industry partners and asked whether they 

	 directly used or adapted the technology; 

	 were aware of the technology, but the technology was not relevant to their research 

agenda; or 

	 were unaware of the technology. 

Recall that because NCMC technology is generic and infrastructural, some researchers may 

adapt it to suit their particular needs. Thus, rather than ask users whether they adopted a 

technology, the survey asked whether they used or adapted it. 

Table 6-4. Economic Benefits of the NCMC Consortium Experience 

NCMC Meeting Hosted Semiannual Meeting Hosted 

Industry Researchers Demonstration and Researcher 

Attendees (hosted Training Benefits Training Benefits Total 

Year (persons) days) ($thousands) ($thousands) ($thousands) 

2002 58 130 124 — 124 

2003 78 157 209 85 294 

2004 64 58 164 102 266 

2005 48 85 114 38 151 

2006 55 8 130 55 186 

2007 52 7 108 5 113 

Total 355 persons 445 days $849 $286 $1,134 

Note: All dollar values are real 2007 dollars. Sums may not add to totals due to independent rounding. 

6-7 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Respondents’ views are presented in Table 6-5. The data show that gradient combinatorial 

methods for polymer thin films, NIST’s leading and most substantial contribution to the field, 

were reported as being both the most used or adapted technology as well as the one with which 

respondents were most familiar. There also appears to be a high level of awareness of NCMC’s 

contributions to the combi knowledge base. 

Data provided by respondents indicate that the economic benefit of NCMC’s contributions to the 

combi knowledge base totaled nearly $24.4 million over the period from 2001 to 2007 (see 

Table 6-6). These information acquisition benefits reflect two distinct advantages that accrued to 

end users. First, NCMC’s papers and presentations invalidated some firms’ research strategies, 

enabling them to avoid expending resources on projects that either would have ultimately failed 

or that would not have been as effective and efficient as those the NCMC published. Second, 

respondents stated that they acquired valuable knowledge about combi in general and about 

specific approaches. One researcher noted that although his lab had not implemented any of 

the methods developed by the NCMC, the overall body of knowledge that the consortium 

generated offered valuable insights into implementing and employing combinatorial approaches. 

Table 6-5. Respondents’ Adoption and Awareness of NCMC Combi Technology 

Respondents’ Use of the Following Methods and 

Technologies Developed by the NCMC and its Use or Aware of but Unaware of 

Industry Members (% of respondents): Adapted Not Relevant Technology 

Continuous thin films with gradients in temperature, 45% 55% 0% 

composition, thickness, surface energy, or on 

chemically patterned substrates 

Discrete libraries used with gradients in temperature, 32% 55% 5% 

surface energy, or thickness 

High-throughput measurement of morphology 27% 41% 23% 

Buckling method of modulus measurement for thin 23% 64% 9% 

films and soft materials 

Edge lift-off test for interfacial adhesion 23% 64% 9% 

Microfluidics produced organic solvent-based libraries 14% 64% 14% 

Multilens contact test for adhesion 5% 77% 9% 

Interfacial tension measurement via microfluidics 5% 77% 9% 

Integrated metrology (morphology, composition, 0% 73% 18% 

extent of reaction) on microfluidics produced organic 

solvent-based libraries 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” Survey. 

6-8 
 

 

 

 

Chapter 6 — Net Economic Benefit Estimates 

Table 6-6. Economic Benefits of Expanding the Combi Knowledge Base 

Labor Benefits Materials Benefits Capital Benefits Total 

Year ($thousands) ($thousands) ($thousands) ($thousands) 

2001 — 103 103 

2002 419 30 109 558 

2003 725 27 4,016 4,768 

2004 859 29 4,094 4,981 

2005 354 28 4,424 4,806 

2006 658 26 4,228 4,912 

2007 423 24 3,859 4,306 

Total $3,437 $164 $20,833 $24,434 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. Sums may not add to 

totals due to rounding. 

The majority of knowledge-base benefits were reported to be capital expenditure savings. In 

interviews with researchers, we posed questions about why benefits from consuming NCMC 

research were largely capital savings. Interviewees told us that they were readily able to 

quantify avoided expenditures from implementing the NCMC approach, which often used 

existing or less expensive equipment. That is, researchers reported saving thousands of dollars 

on equipment whose purchase was precluded by published NCMC research. Those same 

researchers generally could not quantify approximately how much labor effort was also saved 

because most labor savings were considered from actually using the NCMC methods.7 

6.4 Economic Benefit of NCMC Combi Technology and Accelerating Combi’s 

Adoption 

The final component of economic benefits was the benefit of adopting NCMC combi technology 

and the economic benefit from accelerating firms’ combi adoption. In the case of methods 

developed by NCMC, the study valued the introduction of novel technology that otherwise would 

not have been introduced within this study’s period of analysis. Some firms also reported 

accruing acceleration benefits attributable to the NCMC. R&D efficiency benefits accrued to 

firms who adopted combi earlier than they would have in the absence of NCMC’s industry 

outreach work and publications. Where firms reported that their adoption of combi was 

accelerated, the firms’ net benefits from the acceleration effect were included in the benefits 

estimate. 

First, economic benefits from adopting NCMC-developed technology were quantified by 

measuring the cost savings from using NCMC approaches rather than the next best alternative 

(“defending”) method. Table 6-5 lists the technologies that were presented to respondents, who 

7 The reported average labor benefit was about 0.7 FTE (Range: 0 to 2 FTE) distributed over a 2- to 4-year period. 

6-9 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

were then asked to compare how use of these approaches offered R&D efficiency benefits 

compared to alternative approaches. Productivity gains were defined as increases in the 

number of equivalent samples analyzed per day per researcher. Economic benefits were the 

value of the increase in throughput and the volume of information acquired for equivalent or 

lower labor, materials, or capital expenditures. 

As an illustration of how NCMC’s research generated economic benefits, one researcher offered 

the following anecdote. Her laboratory developed a new apparatus for testing the mechanical 

properties of small samples that was heavily influenced by methods NCMC had developed. The 

apparatus and its associated techniques enabled her lab to complete the research faster and at 

lower cost. The researcher speculated that they would not have undertaken what proved to be a 

highly successful project without the NCMC technology because it would have taken 10 times 

longer and have been 10 times more expensive. 

Another respondent said that using the temperature gradient approach for thin films permitted 

her to do in one experiment what would normally take 25 experiments. She said that, assuming 

each experiment used to require 15 minutes, she can now accomplish in 30 minutes (an 

additional 15 minutes was required for sample preparation using the NCMC method) what 

would have taken her 375 minutes. 

Over 90% of respondents reported that the NCMC technologies offered efficiency benefits over 

the technologies that they had been using before, and a slightly higher percentage reported that 

these technologies also enabled them to test across a broader range of samples (see Table 

6-7). NCMC technology also improved the effectiveness of their R&D: 92% of respondents cited 

that NCMC technology improved the quality of their research and/or products that their company 

may produce. 

One firm credited NCMC’s research with assisting with the development of advanced 

formulation systems they use in their coatings research. While it was not a direct transfer of 

technology from NCMC to the firm, they did say that they would not have been able to develop 

the formulation system or something similar without having had the benefit of NCMC research. 

Table 6-7. Respondents’ Perceptions of Efficiency and Effectiveness Gains from 

NCMC-Developed Combi Technology 

Survey Question	 Yes No 

Did the NCMC-developed technologies 

... offer efficiency benefits or cost savings over what your lab had been 91% 9% 

using heretofore? 

... enable you to test samples across a broader range of conditions? 92% 8% 

... improve the quality of your research or any products that your 92% 8% company may produce? 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” Survey. 

6-10 
Chapter 6 — Net Economic Benefit Estimates 

The formulation system was installed in each of the firms’ R&D labs, and they estimate savings 

to date of several million dollars relative to if they had installed commercially available 

formulators. 

Table 6-8 presents the average reported labor productivity gains for using combi in polymers 

R&D in general and for using the NCMC-developed technology. The mean labor productivity 

gain reported from combi overall (i.e., not from only NCMC approaches) was about 8.5 fold, with 

individual responses ranging between 1.5 and 24 fold. The mean reported labor productivity 

gain for NCMC-developed approaches was 5.2 fold, with individual responses ranging between 

1.2 and 12.5 fold. Follow-up interviews to explore the wide variation in reported gains suggested 

that the variation was attributable to differences in the techniques different labs applied, 

properties and materials of interest, and required rate of automation, for example. 

The work performed by NCMC on microfluidics for measuring interfacial tension helped another 

firm develop its own microfluidic equipment. This equipment enabled his lab to perform tests up 

to 3 times faster compared to the previous method while using fewer materials (1 microliter of 

fluid using this new method was equivalent to the 100 ml of fluid required by traditional 

interfacial tension tests). Less labor was required because sample preparation was largely 

automated. Working with smaller samples had an important environmental benefit because the 

smaller samples size reduced waste, which, in turn, reduced hazardous material disposal costs. 

The edge lift-off test for interfacial adhesion was credited for enabling another firm to analyze 5 

times more combinations of parameters than had been possible with previous methods. 

Although they did not cite significant gains in throughput, the method did permit them to study 

their samples more thoroughly. As a consequence, the firm believes that their customers have 

more robust products, which they went on to explain meant that even if their customers did not 

follow their instructions to the letter, the material would still give them the desired behavior. 

Many labs reported using combi approaches within some stages of their research projects and 

did not use a “classical” combinatorial workflow such as that presented in Chapter 1. This 

Table 6-8. NCMC Acceleration of and Estimated Productivity Gain from Combi 

Measure Value 

Mean overall net labor productivity gain from using combi for polymers 

R&D, where applicable 

Mean reported net labor productivity gain from using NCMCdeveloped technologies for polymers R&D, where applicable 

Percent of respondents reporting that NCMC accelerated their 

adoption of combi for polymers R&D 

Mean number of years respondents reported that NCMC accelerated 

their combi adoption 

8.5 fold 

(Range: 1.5 to 24 fold) 

5.2 fold 

(Range: 1.2 to 12.5 fold) 

50% 

2.3 years 

(Range: 1 to 5 years) 

Source: “Economic Impacts from the Adoption of Combinatorial Methods for Organic Polymers” Survey. 

6-11 
 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

finding, taken in conjunction with the technical and economic measures discussed above, is 

consistent with the view that combi’s benefits are best conceptualized as the knowledge that 

combi generates and not just on the number of additional samples that can be analyzed 

(Fasolka and Amis, 2007). 

Half of all respondents reported that NCMC accelerated their adoption of combi by an average 

of 2.3 years. Some reported that their adoption of combi was accelerated by 4 or 5 years. A 

recurring theme was that interaction with NIST researchers gave firms more confidence in their 

decision to proceed with investments in combi programs. The publicity and attention NCMC 

drew to combi made it easier for some researchers to convince senior executives within their 

organizations to invest in combinatorial approaches because NCMC and NIST demonstrated 

that a reputable institution had successfully performed combi for polymers. 

Acceleration benefits were quantified by applying respondents’ reported productivity gains from 

combi overall to their reported FTEs using combi, subtracting adoption costs associated with 

FTEs developing combi and other expenses as well as any benefits from using NCMC 

technology. The net benefits for the reported period of acceleration were attributed to NCMC. 

The data in Table 6-9 illustrate that the NCMC spurred adoption most during 2003 and 2004 

and again in 2006 and 2007. 

Total technology adoption benefits were nearly $185 million for the materials, coatings, and 

adhesives industries alone. These benefits are conservative because 

	 not all impact categories were quantified, such as end-user benefits from improved 

materials; 

Table 6-9. Acceleration and Adoption Benefits of NCMC Combi Technologies 

Benefits of Benefits of NCMC-Developed Technologies 

Accelerating 

Combi Adoption Labor Materials Capital Total 

Year ($thousands) ($thousands) ($thousands) ($thousands) ($thousands) 

2002 10,814 98 10,912 

2003 3,857 27,246 92 154 31,349 

2004 8,345 32,753 92 151 41,344 

2005 1,532 35,536 86 212 37,366 

2006 2,206 31,220 112 37 33,575 

2007 3,142 27,046 105 30,293 

Total $19,082 $164,616 $585 $556 $184,839 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. Sums may not add to 

totals due to rounding. 

6-12 
 

 

Chapter 6 — Net Economic Benefit Estimates 

	 the extrapolation methodology was based on a profile of likely combi adopters within 

three industries; 

	 benefits for academic laboratories could not be reliably quantified; and 

	 combi’s adoption is ongoing, and in addition to benefits in other industries, benefits 

beyond 2007 have yet to accrue and therefore remain unquantified. 

6.5 NCMC Program Costs and End User Technology Acquisition Costs 

Total NIST costs from 1998 to 2007 were approximately $14.5 million (see Table 6-10). These 

costs were augmented by NCMC membership and focus project fees of around $1 million and 

technology acquisition costs for NCMC-developed technology of about $7.5 million, including 

capital and initial labor expenditures. One respondent commented that in their experience the 

cost of implementing NCMC techniques was on par with some of their internally developed 

techniques and about half that of commercial combi vendors. Another respondent “...found the 

NCMC to be a good place for high-throughput ideas or approaches. The challenge was to 

modify their approaches to accommodate industrially-relevant materials.” Low technology 

adoption costs relative to benefits may have been reported because many NCMC approaches 

used equipment that was already available in industrial laboratories. Additional capital 

expenditures were reported to be as little as $5,000 from some firms and as high as $300,000 

for others. 

Table 6-10. NCMC Program and Technology Acquisition Costs 

Technology 

NIST Costs Membership Fees Acquisition Costs Total 

Year ($thousands) ($thousands) ($thousands) ($thousands) 

1998 286 286 

1999 495 495 

2000 556 556 

2001 2,266 2,266 

2002 1,548 19 968 2,535 

2003 1,793 153 1,219 3,166 

2004 1,759 313 1,336 3,409 

2005 2,420 157 1,543 4,120 

2006 2,062 183 1,561 3,806 

2007 1,339 214 865 2,418 

Total $14,525 $1,040 $7,492 $23,057 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. Sums may not add to 

totals due to rounding. 

6-13 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

6.6 Summary Economic Benefit Estimates and Measures of Return 

Table 6-11 assembles the complete time series of quantified costs and benefits for 1998 

through 2007. Total benefits were over $210.4 million for the advanced materials, coatings, and 

personal and household care products industries. Inclusive of NIST, NCMC member, and 

technology acquisition costs net benefits were $187.4 million. The vast majority of benefits 

accrued from using NCMC-developed technologies, but the knowledge base expansion benefits 

equivalent to more than 10% of total benefits are equally as significant. These less tangible 

benefits are essentially a slice of the minimum alternative development cost for the technologies 

that NCMC developed for the equivalent of $23 million, and part of that $23 million included 

costs for administration and outreach work. 

The net present value (NPV) of net benefits was $118.0 million applying the OMB-approved 

discount rate of 7% (see Table 6-12). The benefit-to-cost ratio, which is the ratio of the NPV of 

total benefits to that of costs, was estimated to be 8.55. In other words, for every $1 that NIST 

and its partners invested in the NCMC at least $8.55 in benefits accrued to the three industries. 

The internal rate of return (IRR) was estimated to be 161%. Because the results of NCMC 

activities are widely used by many companies and other organizations, they have what 

economists call “public-good” content. In such cases, the IRR is called the “social rate of return”. 

In academic studies of the diffusion of new technologies, IRRs of 100% or more have been 

found for significant advances with broad social benefits. Based on a variety of economic 

studies, the hurdle rate for rationalizing such public-good investments is in the 30−50% range. 

Thus, the NCMC returned at least three times what would be considered the minimum 

acceptable IRR (Tassey, 2003). 

6-14 
 

 

 

 

 

 

 

 

 

 

Chapter 6 — Net Economic Benefit Estimates 

Table 6-11. Net Quantified Economic Benefits of the NCMC 

Knowledge Technology 

Consortium Base Adoption & Total 

Experience Expansion Acceleration Benefits Total Costs Net Benefits 

Year ($thousands) ($thousands) ($thousands) ($thousands) ($thousands) ($thousands) 

1998 (286) (286) 

1999 (495) (495) 

2000 (556) (556) 

2001 103 103 (2,266) (2,164) 

2002 124 558 10,912 11,594 (2,535) 9,059 

2003 294 4,768 31,349 36,411 (3,166) 33,245 

2004 266 4,981 41,344 46,591 (3,409) 43,182 

2005 151 4,806 37,366 42,323 (4,120) 38,203 

2006 186 4,912 33,575 38,673 (3,806) 34,867 

2007 113 4,306 30,293 34,713 (2,418) 32,295 

Total $1,134 $24,434 $184,839 $210,408 (23,057) $187,351 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. Sums may not add to 

totals due to rounding. 

Table 6-12. Performance Measures 

Value 

Measure (2007$) 

Total quantified benefits $210.4 million 

Total quantified costs $23.1 million 

Net present value of net benefits (NPV) (Base year = 1998) $118.0 million 

Benefit-to-cost ratio (BCR) 8.55 

Internal rate of return (IRR) 161% 

Note: All dollar values were inflation-adjusted to 2007 using the real GDP deflator, chained. NPV was calculated 

using the 7% real social discount rate recommended by OMB. 

6-15 
7. CONCLUDING REMARKS ON THE 

EFFECTIVENESS OF THE NCMC CONSORTIUM MODEL 

Before NIST’s entry into the development of technology infrastructure for combinatorial methods 

and high-throughput experimentation, the knowledge base was uneven and proprietary. Large 

chemical firms and their engineering vendors had developed sophisticated solutions for combi 

using discrete sample libraries, but not for continuous thin films or microfluidics libraries. These 

firms’ knowledge was essentially locked within the silos of corporate research divisions. Given 

that firms’ R&D programs and strategies are a source of competitive advantage, in the absence 

of a neutral, precompetitive forum, firms had a disincentive to communicate and exchange best 

practices in combi’s development, implementation, and use. This status quo hampered combi’s 

development and uptake as well as a broader awareness of the potential combi holds for 

increasing throughput and the quantity and quality of data captured per material sample 

analyzed. A commercial infrastructure for combi for discrete sample libraries based on robotics 

and automation had emerged, but little technology infrastructure for combi in continuous thin 

films and microfluidics had emerged. 

As a recognized leader in metrology, NIST assumed the role of filling technology gaps left 

unaddressed because of market and technical barriers. NIST was also able to leverage its 

reputation for scientific excellence and independence into a leadership role for the technical 

community. Consequently, benefits from NCMC’s novel technologies and of an organization 

assuming the mantle of leadership of a technical community would not have accrued. The 

technology NCMC developed between 1998 and 2007 enabled organizations to reap significant 

R&D efficiency benefits by integrating and adapting for their use methods such as continuous 

gradient thin films, microfluidics libraries, and the edge lift-off test for interfacial adhesion. 

If it were not for these attributes – leadership in metrology, a reputation for scientific excellence, 

and independence – the creation of a consortium like the NCMC as a precompetitive forum 

whose members were all from the private sector would have been highly unlikely. Beyond 

developing and then rapidly transferring technology into the public domain, NCMC’s outreach 

work, research, and publications demonstrated what was possible with combi. Several firms 

stated that this body of knowledge accelerated their incorporation of combinatorial approaches 

into their research programs. The publicity and attention NCMC drew to combi made it easier for 

some researchers to convince senior executives within their organizations to get invest in 

combinatorial approaches because NCMC and NIST demonstrated that combi for polymers had 

been successfully performed by a reputable institution. 

Through the NCMC’s work researchers acquired both technology and evidence of how 

investments in combi could increase R&D efficiency and hasten the development of more robust 

polymeric materials and the products whose performance or quality is enabled by those 

materials. The value generated by the NCMC is evident in the results: firms that adapted NIST’s 

7-1 
 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

approaches for their needs accrued economic benefits of $165.8 million between 2002 and 

2007. Notably, irrespective of whether they adopted any NCMC technology, 50% of survey 

respondents indicated that NCMC’s work accelerated their adoption of combi by an average of 

2.3 years. Including benefits from participating in the NCMC ($1.1 million), expanding the combi 

knowledge base ($24.4 million), and accelerating some firms’ combi adoption ($19.0 million), 

total economic benefits were $210.4 million for the polymeric materials, coatings, and personal 

care and household products industries. Total costs, inclusive of technology acquisition costs, 

were $23.1 million, translating into a net economic benefit of $187.4 million. The BCR was 8.55 

and the IRR over the analysis period was 161%. 

The NCMC offers an exemplary example of much of the best of what NIST has to offer in 

science & technology development and outreach: 

	 overcoming technology gaps through the development of infratechnology, 

	 convening researchers in independent, precompetitive forums to disseminate research 

and best practice, 

	 partnering with government and academia to develop and execute research projects of 

particular relevance to industry, 

	 demonstrating what is possible and advocating for novel approaches for rapidly 

identifying research foci, and 

	 championing the adoption and development of new approaches that offer R&D and 

production efficiency benefits as well as the opportunity to discover groundbreaking new 

materials. 

The consortium model approved to be particularly effective. The results in Chapter 6 evidence 

how industry participation in the research program meant that NCMC was developing generic 

technologies and infratechnologies that responded to infrastructure gaps discovered as 

industrial research agenda evolved and experience with combi grew. NIST benefited from 

industry’s feedback, and industry benefited from access to information on a prepublication basis 

as well as from hands-on demonstrations of novel approaches. The feedback loop inherent in 

this approach balanced NIST’s mandate to expand the frontier of measurement science with 

industry’s desire for research outcomes to be relevant to their needs. 

7-2 
REFERENCES 

Alston, J., G. Norton, and P. Pardey. 1995. Science Under Scarcity: Principles for Agricultural 

Research Evaluation and Priority Setting. Ithaca, NY: Cornell University Press. 

Amis, E., X-D Xiang, J-C Zhao. 2002. “Combinatorial Methods for Investigations in Polymer 

Materials Science.” MRS Bulletin 27(4):295-300. 

Bains, W. 2004. Biotechnology from A to Z. New York: Oxford University Press, Inc. 

Borman, S. April 6, 1998. “Combinatorial Chemistry: Industry Is Embracing the Technology 

‘Totally’ as Researchers Continue to Advance the Art of Rapid Synthesis and 

Screening.” Chemical & Engineering News. 

<http://pubs.acs.org/hotartcl/cenear/980406/comb.html>. As obtained on January 15, 

2007. 

Barnes, S.E., Z.T. Cygan, J.K. Yates, K.L. Beers, and E.J. Amis. 2006. “Raman Spectroscopic 

Monitoring of Droplet Polymerization in a Microfluidic Device.” Analyst 131(9):1027­

1033. 

BASF. 2006. “Joint Development by BASF and Bosch: Ultra-modern High-Throughput 

Screening Plant for Testing Coating Raw Materials in Formulation Systems.” News 

release. <http://www.performancechemicals.basf.com/ev-wcmsin/internet/en_GB/function/conversions:/publish/upload/pr/2006/060427-ev1-P229.pdf>. 

As obtained on February 20, 2009. 

Brocchini, S., K. James, V. Tangpasuthadol, and J. Kohn. 1997. “A Combinatorial Approach for 

Polymer Design.” Journal of the American Chemical Society 119(19):4553-4554. 

Bureau of Labor Statistics. 2007. Occupational Employment and Wages, may 2007: 19-2032 

Materials Scientists. <http://www.bls.gov/oes/2007/may/oes192032.htm>. As obtained 

December 1, 2008. 

Busch, J. 2001. “Combinatorial Analysis—How Much Is It Worth?” Research Technology 

Management 2(1):38-45. 

Cebon, D., and M.F. Ashby. 2006. “Engineering Materials Informatics.” MRS Bulletin 

31(12):1004-1012. 

Chastek, T.Q., K.L. Beers, and E.J. Amis. 2007. “Miniaturized Dynamic Light Scattering 

Instrumentation for Use in Microfluidic Applications.” Review of Scientific Instruments 

78(7):072201. 

Chung, Y-W. 2007. Introduction to Materials Science and Engineering. Boca Raton, Florida: 

CRC Press. 

Cygan, Z.T., J.T. Cabral, K.L. Beers, and E.J. Amis. 2005. “Microfluidic Platform for the 

Generation of Organic-Phase Microreactors.” Langmuir 21(8):3629-3634. 

R-1 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Dagani, R. March 8, 1999. “A Faster Route to New Materials.” Chemical & Engineering News. 

<>. As obtained on January 15, 2007. 

De Lue, N. 2001. “Combinatorial Chemistry Moves Beyond Pharmaceuticals.” Chemical 

Innovation 31:33-39. 

Fasolka, M., and E. Amis. 2007. “Combinatorial Material Science: Measures of Success.” In 

Combinatorial Material Science, Balaji Narasimhan, Surya K. Mallapragada, and Marc D. 

Porter, eds., pp. 1-20. New York: John Wiley & Sons, Inc. 

Hanak, J. 1970. “The ‘Multiple Sample Concept’ in Materials Research: Synthesis, 

Compositional Analysis and Testing of Entire Multicomponent Systems.” Journal of 

Materials Science 5(11):964-971. 

Harrison, C., J. Cabral, C.M. Stafford, A. Karim, and E.J. Amis. 2004. “A Rapid Prototyping 

Technique for the Fabrication of Solvent-Resistant Structures.” Journal of 

Micromechanics and Microengineering 14(1):153-158. 

Hewes, J. 2000. “High Throughput Methodologies for Chemicals and Materials RD&E.” 

Advanced Technology Program White Paper. <http://www.atp.nist.gov/www/ 

ccmr/ccmr_off.htm>. 

Hewes, J. 2003. “Economic Impact of Combinatorial Materials Science.” In High-throughput 

Synthesis, Radislav Potyralio and Eric Amis, eds., pp. 15-29. New York: Kluwer 

Academic Publishers. 

Hoogenboom, R., M. Meier, and U. Schubert. 2003. “Combinatorial Methods, Automated 

Synthesis and High-Throughput Screening in Polymer Research: Past and Present.” 

Macromolecular Rapid Communication (24):15-32. 

Iden, R., W. Schorf, J. Hadeler, and S. Lechman. 2003. “Combinatorial Materials Research in 

the Polymer Industry: Speed versus Flexibility.” Macromolecular Rapid Communication 

(24):63-72. <http://www.chemspeed.com/files/publications/sa/msp/ 

2003_Combinatorial%20Materials%20Research.pdf>. As obtained on February 6, 2007. 

Iden, R., W. Schorf, J. Hadeler, and S. Lechman. 2003. “Combinatorial Materials Research in 

the Polymer Industry: Speed versus Flexibility.” Macromolecular Rapid Communication 

(24):63-72. <http://www.chemspeed.com/files/publications/sa/msp/ 

2003_Combinatorial%20Materials%20Research.pdf>. As obtained on February 6, 2007. 

l’Abee, R.M.A., L.J.M. Cornelissen, M.C. Diaz, M. Ligthart, M.J. de la Mar, S. Wijnans, B.G.G. 

Lohmeijer, U.S. Schubert. 2003. “High-Throughput Screening and Combinatorial 

Methods: The Future of Polymer Research in Industry and Academia? A Feasibility 

Study.” e-Polymers no. T_001. <>. 

Jandeleit, B., D. Schaefer, T. Powers, H. Turner, and H. Weinberg. 1999. “Combinatorial 

Materials Science and Catalysis.” Angewandte Chemie International Edition 38:2494­

2532. 

Meredith, C., A. Karim, and E. Amis. July 20, 1999. Combinatorial Mapping of Polymer Thin 

Film Dewetting, Reactive Polymers, Ion Exchange, and Adsorbents Gordon Conference. 

R-2 
References 

Meredith, J.C., A. Karim, and E.J. Amis. 2000. “High-Throughput Measurement of Polymer 

Blend Phase Behavior.” Macromolecules 33 (16):5760-5762. 

Meredith, J.C., A.P. Smith, A. Karim, et al. 2000. “Combinatorial Materials Science for polymer 

Thin-Film Dewetting.” Macromolecules 33(26):9747-9756. 

Meredith, J.C., A. Karim, and E.J. Amis. 2002. “Combinatorial Methods for Investigations in 

Polymer Materials Science.” MRS Bulletin 27(4):330-335. 

National Institute of Standards and Technology (NIST). 2002. “NIST Launches Combinatorial 

Methods Center.” <http://polymers.nist.gov/combi/kickoff.htm>. As obtained on January 

15, 2007. 

National Science Foundation. 2007. “Academic Research and Development Expenditures: 

Fiscal Year 2006.” <http://www.nsf.gov/statistics/nsf08300/>. As obtained on May 11, 

2008. 

National Research Council. 2001. “Energy Research at DOE: Was It Worth It?” Washington, 

DC: National Academies Press. 

Pavia, M. 2001. “High-Throughput Organic Synthesis: A Perspective Looking Back on 10 Years 

of Combinatorial Chemistry.” In High-throughput Synthesis: Principles and Practices, 

Irving Sucholeiki, ed., pp. 43-53. New York: Marcel Dekker, Inc. 

Peil, K., D. Neithamer, D. Patrick, B. Wilson, and C. Tucker. 2004. “Applications of High 

Throughput Research at the Dow Chemical Company.” Macromolecular Rapid 

Communications 25(1):119-126. 

Pharmaceutical Research and Manufacturers of America (PhRMA). March 2007. 

Pharmaceutical Industry Profile 2007. Washington, DC: PhRMA. 

Quake, S.R., and A. Scherer. 2000. “From Micro to Nano Fabrication with Soft Materials.” 

Science 290:1536-1540. 

Raghavan, D., X. Gu, T. Nguyen, M. VanLandingham, and A. Karim. 2000. “Mapping Polymer 

Heterogeneity Using Atomic Force Microscopy Phase Imaging and Nanoscale 

Indentation.” Macromolecules 33(7):2573-2583. 

Sia, S.K., and G.M. Whitesides. 2003. “Microfluidic Devices Fabricated in 

Poly(Dimethylsiloxane) for Biological Studies.” Electrophoresis 24(21):3563-3576. 

Smith, W. 2004. Foundations of Materials Science and Engineering. McGraw-Hill. 

Stafford, C.M., C. Harrison, K.L. Beers, et al. 2004. “A Buckling-Based Metrology for Measuring 

the Elastic Moduli of Polymeric Thin Films.” Nature Materials 3(8):545-550. 

Strauss, S.D. 2002. The Big Idea How Business Innovators Get Great Ideas to Market. Chicago: 

Dearborn Trade Publishing. 

Sysmyx Technologies. 1999. Form S-1/A, General form for registration of securities under the 

Securities Act of 1933. Filed with the Securities Exchange Commission November 17. 

R-3 
Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

Tassey, G. 2003. Methods for Assessing the Economic Impact of Government R&D. NIST 

Planning Report 03-01. Gaithersburg, MD: National Institute of Standards and 

Technology. 

Thayer, A. 1996. “Combinatorial Chemistry Becoming Core Technology at Drug Discovery 

Companies.” Chemical & Engineering News. <http://pubs.acs.org/hotartcl/ 

cenear/960212/core.html>. As obtained on January 15, 2007. 

U.S. Bureau of Labor Statistics (BLS). 2006. Occupational Employment Statistics: Material 

Scientists. Available at <http://www.bls.gov/oes/home.htm>. As obtained on May 20, 

2008. 

Wicks, D., and H. Bach. 2002. “The Coming Revolution for Coatings Science: High Throughput 

Screening for Formulations” <>. As obtained on May 10, 2008. 

Wu, T., Y. Mei, J.T. Cabral, et al. 2004. “A New Synthetic Method for controlled Polymerization 

Using a Microfluidic System.” Journal of the American Chemical Society 126(32):9880­

9881. 

Zhang, W.H., M.J. Fasolka, A. Karim, and E.J. Amis. 2005. “An Informatics Infrastructure for 

Combinatorial and High-Throughput Materials Research Built on Open Source Code.” 

Measurement Science & Technology 16(1):261-269. 

R-4 
 

 

 

 

 

APPENDIX A: LIST OF FIRMS COMPRISING 

EXTRAPOLATION BASE FOR NATIONAL IMPACT ESTIMATION
 

Colgate–Palmolive Imperial Chemical 

3M Company Company Industries RPM International Inc. 

Air Products and Cytec Industries Inc. Intel Sealed Air 

Chemicals, Inc. 

Albemarle Corporation E. I. du Pont de International Flavors & Sensient Technologies 

Nemours and Company Fragrances Inc. Corporation 

Alberto–Culver Eastman Chemical Johnson & Johnson Solutia Inc. 

Company Company 

Arch Chemicals, Inc. Ecolab Inc. Kimberly–Clark Stepan Company 

Corporation 

Ashland Inc. Elizabeth Arden, Inc. L'Oréal SA The Clorox Company 

Avery Dennison Exxon Mobil Corporation LVMH Moët Hennessy The Dow Chemical 

Corporation Louis Vuitton SA Company 

Avon Products, Inc. Ferro Corporation New Market Corp The Estée Lauder 

Companies Inc. 

BASF SE FMC Corp Nu Skin Enterprises, Inc. The Lubrizol Corporation 

Bayer AG General Electric PolyOne Corporation The Procter & Gamble 

Company Company 

Boston Scientific Georgia Gulf PPG Industries, Inc. The Sherwin–Williams 

Corporation Company 

BP p.l.c. H.B. Fuller Company Praxair, Inc. The Valspar Corporation 

Cabot Corporation Hercules Incorporated Rayonier Inc. W. R. Grace & Co. 

Celanese Corporation Hexcel Corporation Revlon Akzo Nobel N.V. 

Chemtura Corporation Honeywell International Rhodia Arkema 

Corp 

Church & Dwight Co., Huntsman Corporation Rockwood Holdings, Inc. Michelin Corporation 

Inc. 

Ciba Specialty Illinois Tool Works Inc. Rohm and Haas Unilever PLC 

Chemicals Holding Inc. Company 

A-1 
APPENDIX B: MEMBER SURVEY 
Economic Impacts from the Adoption of 

Combinatorial Methods for Organic Polymers

A Survey Sponsored by the National Institute of Standards and Technology (NIST)
 

Survey Instrument for Current and Former NCMC Members
 

Combinatorial and high-throughput methods, or “combi,” have been credited with accelerating the pace and 

quality of polymers R&D within both the private and academic sectors. By enabling material scientists to create 

large “libraries” of samples and to quickly analyze their properties, combi speeds the discovery of new 

polymeric materials. 

NIST has commissioned RTI International, a not-for-profit research institute, to conduct an evaluation of the 

adoption of combinatorial methods for organic polymers R&D, the economic impacts associated with this 

adoption, and the NIST Combinatorial Methods Center (NCMC). 

The goals of the study are to: 

• learn the extent to which NIST combi technologies have been adopted, 

• estimate the impact combi has had on researchers’ productivity, and 

• provide guidance on how a NIST-industry consortium model has benefited the technical community. 

The following survey is an important effort that seeks to provide insights into the effectiveness of NISTsponsored, industry-directed programs. Such insights will help shape future strategic directions. The survey is 

voluntary and is estimated to take between 15 and 20 minutes to complete. It requests: 

• estimates of your familiarity with combi and whether you use it, 

• when you may have adopted a combinatorial workflow in your lab, 

• data on your experience with the NCMC and its work to spur combi’s adoption, and 

• measures of how combi may have improved the materials discovery process. 

NIST will use the results of this study in its strategic planning for the NCMC, and specifically for its materials 

science programs. NCMC’s technical contributions to combi for organic polymers have largely been in library 

fabrication, testing, and analysis of: continuous gradient thin film libraries, discrete libraries, and microfluidics 

for polymers in organic solvents. Skip questions that are outside your area of expertise. 

Responses to this survey will be kept strictly confidential. At no time will any individual’s name, any 

company or university name, their participation, or identifiable response be released by RTI to any 

third party, including NIST and the NCMC. The data that survey respondents provide will only be used 

to present aggregate analytical findings to the NCMC in the form of a final report that will be publicly 

released by the end of this calendar year following a peer review process. 

Questions about the survey should be directed to Dallas Wood at (919) 541-8743 or [US Eastern Time], or 

Alan O’Connor at (415) 848-1316 or [US Pacific Time]. 

OMB Control Number 0693-0033, expiration date 7/31/09 

This survey is authorized under Executive Order 12862, “Setting Customer Service Standards.” Your response is voluntary and all data collected will be 

considered confidential. Public reportings for this collection of information is estimated to average 20 minutes per response, including the time of 

reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of 

information. Send comments regarding this estimate or any other aspects of this collection of information, including suggestions for reducing the length 

of this questionnaire, to the National Institute of Standards and Technology, 100 Bureau Drive, Stop 3220, Gaithersburg, MD, 20899-3220 and the Office 

of Management and Budget Information and Regulatory Affairs, Office of Management and Budget, Washington, DC 20503. 

B-1 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

1. Respondent Contact Information

Responses to this survey are strictly confidential. At no time will any individual’s name, any company name, 

participation, or identifiable response be released by RTI to any third party, including the NIST and the NCMC. 

Respondent name (optional): 

Affiliation (optional):
 

Title (optional): 

Telephone number (optional) 

Email (optional): 

Geographic location, if not USA: 

Are you willing to participate in a short, confidential follow-up telephone discussion about your [Select One] 

response? 

2. Combinatorial and/or High Throughput Methods (“Combi”) Usage. 

Does your company or university research laboratory (-ies) currently use any combinatorial or high-throughput methods 

(referred to as “combi”) for polymers R&D, or has it used combi in the past? 

Yes, we are currently using combi in my lab. 

Approximately how many years has combi been used in your lab?
 

Did your company join the NCMC as part of a combi adoption plan?
 

No, we are not using combi in my lab. 

If no, do you expect to use combi in the future? 

did your lab suspend using combi?
 

Comments? 

3. Combi Technology Adoption Timeframe.

The following questions ask about your lab’s adoption of combi for polymers R&D. Our goal is to track the diffusion of 

the combi technology and methods. Please answer the following questions to the best of your ability. Your best 

approximation will suffice. 

3a. In which year(s) would you estimate that the following occurred? 

Researchers in our lab became aware of the successful, practical use of combi for polymers 

R&D 

My lab(s) implemented and began using combi for the first time 

Combi became a part of the regularly used methods in our lab(s) 

Comments? 

3b. Reflect upon your familiarity with the NCMC’s papers, workshops, proceedings, and/or conference presentations. 

Did the research offered at conferences or in the technical literature accelerate your lab’s adoption of combi 

overall? 

Yes, this research likely accelerated our adoption of combi by years. 

No, this research likely had little impact on the timing of our combi adoption. 

B-2 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Appendix B — Member Survey 

4. Laboratory Personnel Working in Combi. 

The purpose of this section is to estimate how many researchers may be using combi for R&D and how that estimate 

changed over time. This information will be combined with other responses to estimate the number of people working in 

combi today. If you are unsure of an estimate for any given year, simply leave the cell empty. 

4a. Approximately how many full-time equivalent employees (FTEs) worked in combi in your lab during the following 

years? Please separate those FTEs that were primarily involved in developing/adopting/adapting new combi 

methods from those FTEs that were primarily involved in the regular use of established combi methods. 

Year 1999 2001 2003 2005 2007 

FTE(s) primarily involved in 

developing/adopting/adapting new combi methods 

FTE(s) primarily involved in regular use of 

established combi methods 

4b. What percent of your combi activity currently falls within each of the following library types? 

2003 Current 

Thin films libraries % %

 Discrete libraries % %

 Microfluidics libraries % % 

Comments? 

5. Technology Adoption of Combi Methods and Technologies. 

5a. What magnitude of labor productivity increase (tests per day per researcher) do [Select One] 

you estimate that combi in general offers your lab overall? 

5b. Please indicate whether you use or adapted any of the following methods and technologies 

developed by the NCMC and its industry members. 

Continuous thin films with gradients in temperature, composition, thickness, surface 

energy, or on chemically patterned substrates 

Discrete libraries used with gradients in temperature, surface energy, or thickness 

Microfluidics produced organic solvent-based libraries 

Buckling method of modulus measurement for thin films and soft materials 

Edge lift-off test for interfacial adhesion 

Multi lens contact test for adhesion 

Interfacial tension measurement via microfluidics 

High throughput measurement of morphology 

Integrated metrology (morphology, composition, extent of reaction) on microfluidics 

produced organic solvent-based libraries 

5c.	 If you use or adapted any of the NCMC technologies listed in 5b, did they offer efficiency [Select One] 

benefits or cost savings over what your lab had been using heretofore? 

Use or adapted this 

technology 

Aware of, but not

relevant to your research

Unaware of this

technology 

B-3 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

If yes, please estimate the combined benefits adoption of these NCMC technologies had on the following factors: 

Increase in research labor productivity: [Select One] 

(tests per day per researcher) 

Reduced materials expenses, per 10 samples processed:	 US$ 

Savings on research equipment expenditures (excluding US$

one-time adoption costs): 

 Time Savings: [Select One] 

 Comments? 

5d. Did the NCMC technologies you adopted or adapted from 5b improve the quality of your [Select One]

research or any products that your company may produce? 

 Comments? 

5e. Did the NCMC technologies you adopted or adapted from 5b enable you to test samples over a [Select One] 

broader range of conditions? 

5f. Please estimate the one-time costs you incurred when adopting or adapting any of the NCMC technologies listed 

in Question 5b. Would you characterize these expenses as a large investment in labor, research equipment, or 

instillation relative to other technologies your lab has adopted?

 Comments: 

6. Benefits of Consortium Membership. 

The following questions ask you to reflect upon the economic benefits your organization accrues from participating in the 

NCMC. 

6a. Did participation in the NCMC offer more productive marketing or networking opportunities, [Select One] 

relative to other conferences or combi meetings? 

If yes, how would you rank the productivity of these NCMC meetings relative to other 

conferences or combi meetings?
 

Not Much More 

Productive 

Slightly More 

Productive 

Significantly More 

Productive 

6b. Did participation in the NCMC validate or invalidate internal research projects and permit more 

efficient and effective resource allocation? 

[Select One] 

Did participation in the NCMC help your organization avoid any research activities, or enable 

your organization to acquire needed research more quickly or cost effectively? 

[Select One] 

If yes, please estimate the costs your firm likely saved and the amount of time 

Labor (in FTEs or person-months): 

 Materials Expense 

 Research equipment 

Time 

[Select One] 

US$

US$

[Select One] 

Over what time period did you accrue these benefits: 

B-4 
 

 

Appendix B — Member Survey 

6c. Did the NCMC offer valuable researcher training and demonstrations beyond what would have 

been available elsewhere? 

[Select One] 

If yes, approximately how many training hours were likely saved, per meeting, 

had researchers attempted to acquire the same amount and quality of 

research via readings and/or conference attendance? 

hours 

6d. 

6e. 

Was the annual benefit your organization accrued from participation in the NCMC greater than, 

equal to, or less than the annual membership fees it paid? 

If you would like to offer comments about the NCMC, its effectiveness and usefulness 

consortium, or offer suggestions for its improvement or expansion, please do so below. 

[Select One] 

Comments: 

B-5 
APPENDIX C: NONMEMBER SURVEY 
Economic Impacts from the Adoption of 

Combinatorial Methods for Organic Polymers

A Survey Sponsored by the National Institute of Standards and Technology (NIST)
 

Combinatorial and high-throughput methods, or “combi,” have been credited with accelerating the pace and 

quality of polymers R&D within both the private and academic sectors. By enabling material scientists to create 

large “libraries” of samples and to quickly analyze their properties, combi speeds the discovery of new 

polymeric materials. 

NIST has commissioned RTI International, a not-for-profit research institute, to conduct an evaluation of the 

adoption of combinatorial methods for organic polymers R&D, the economic impacts associated with this 

adoption, and the NIST Combinatorial Methods Center (NCMC). Respondents do not need to be using combi 

in order to participate. 

The goals of the study are to: 

• learn the extent to which NIST combi technologies have been adopted, 

• estimate the impact combi has had on researchers’ productivity, and 

• provide guidance on how a NIST-industry consortium model has benefited the technical community. 

The following survey is an important effort that seeks to provide insights into the effectiveness of NISTsponsored, industry-directed programs. Such insights will help shape future strategic directions. The survey is 

voluntary and is estimated to take between 15 and 20 minutes to complete. It requests: 

• estimates of your familiarity with combi and whether you use it, 

• when you may have adopted a combinatorial workflow in your lab, 

• information on your familiarity with the NCMC and its work to spur combi’s adoption, and 

• measures of how combi may have improved the materials discovery process. 

NIST will use the results of this study in its strategic planning for the NCMC, and specifically for its materials 

science programs. NCMC’s technical contributions to combi for organic polymers have largely been in library 

fabrication, testing, and analysis of continuous gradient thin film libraries, discrete libraries, and microfluidics 

for polymers in organic solvents. Skip questions which are outside your area of expertise. 

Responses to this survey will be kept strictly confidential. At no time will any individual’s name, any 

company or university name, their participation, or identifiable response be released by RTI to any 

third party, including NIST and the NCMC. The data that survey respondents provide will only be used 

to present aggregate analytical findings to the NCMC in the form of a final report that will be publicly 

released by the end of this calendar year following a peer review process. 

Questions about the survey should be directed to Dallas Wood at (919) 541-8743 or [US Eastern Time], or 

Alan O’Connor at (415) 848-1316 or [US Pacific Time]. 

OMB Control Number 0693-0033, expiration date 7/31/09 

This survey is authorized under Executive Order 12862, “Setting Customer Service Standards.” Your response is voluntary and all data collected will be 

considered confidential. Public reportings for this collection of information is estimated to average 20 minutes per response, including the time of 

reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of 

information. Send comments regarding this estimate or any other aspects of this collection of information, including suggestions for reducing the length 

of this questionnaire, to the National Institute of Standards and Technology, 100 Bureau Drive, Stop 3220, Gaithersburg, MD, 20899-3220 and the Office 

of Management and Budget Information and Regulatory Affairs, Office of Management and Budget, Washington, DC 20503. 
 

 

 

 

 

 

 

 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

1. Combinatorial and/or High Throughput Methods (“Combi”) Usage. 

Does your company or university research laboratory (-ies) currently use any combinatorial or high-throughput methods 

(referred to as “combi”) for polymers R&D, or has it used combi in the past? 

Yes, we are currently using combi in my lab. 

I do not know if we are using combi. 

No, we are not using combi in my lab. 

If approximately how many years has combi been used in your lab? 

yes, 

If no, do you expect to use combi in the future? 

did your lab suspend using combi? 

Comments? 

2. Combi Technology Adoption Timeframe.

The following questions ask about your lab’s adoption of combi for polymers R&D. Our goal is to track the diffusion of the 

combi technology and methods. Please answer the following questions to the best of your ability. Your best 

approximation will suffice. 

2a. In which year(s) would you estimate that the following occurred? 

Researchers in our lab became aware of the successful, practical use of combi for polymers 

R&D 

My lab(s) implemented and began using combi for the first time 

Combi became a part of the regularly used methods in our lab(s) 

Comments? 

2b. Are you familiar with the NIST Combinatorial Methods Center (NCMC), a NIST-sponsored research consortium 

that works with industry members to advance the combi technology, measurement, and analysis needs 

(http://polymers.msel.nist.gov/combi/index.html)? 

No, I am not familiar with the NCMC. 

Yes, I am familiar with or have heard of the NCMC. 

How did you hear about the NCMC? Please select all that apply. 

Conferences, seminars, or workshops 

Technical literature 

Trade journal article(s) 

Word of mouth 

Other (please specify): 

2c. The NCMC made several presentations and sponsored symposia about combi at recent conferences. Have you 

attended any of the following meetings or conferences since 2001? 

American Chemical Society American Physical Society 

Materials Research Society Knowledge Foundation Conferences 

Gordon Conferences Intl Workshop on Combinatorial Materials Science 

NCMC Industry Workshops Other: 

I did not attend any of the above conferences or workshops. 

2d. The NCMC published its research in the technical literature to support academic and industry researchers in their 

adoption of combi. Authors included E.J. Amis, A. Karim, J.C. Meredith, C.M. Stafford, M.J. Fasolka, and K.L. 

Beers, among others. Are you familiar with the research of these authors? 

Yes, I am familiar with one or more of the above authors’ research. 

C-2 
Appendix C — Nonmember Survey 

No, I am not familiar with any of the above authors’ research. 

C-3 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

2e. Reflect upon your familiarity with the NCMC’s papers and/or conference presentations. Did the research offered at 

conferences or in the technical literature accelerate your lab’s adoption of combi overall? 

Yes, this research likely accelerated our adoption of combi by years. 

No, this research likely had little impact on the timing of our combi adoption. 

3. Technology Adoption of Combi Methods and Technologies. 

3a. What magnitude of labor productivity increase (tests per day per researcher) do 

you estimate that combi in general offers your lab overall? 

[Select One] 

3b. Please indicate whether you use or adapted any of the following methods and technologies 

developed by the NCMC and its industry members. 

Use or adapted this 

technology 

Aware of, but not relevant to

your research

Unaware of this technology 

Continuous thin films with gradients in temperature, composition, thickness, surface 

energy, or on chemically patterned substrates 

Discrete libraries used with gradients in temperature, surface energy, or thickness 

Microfluidics produced organic solvent-based libraries 

Buckling method of modulus measurement for thin films and soft materials 

Edge lift-off test for interfacial adhesion 

Multi lens contact test for adhesion 

Interfacial tension measurement via microfluidics 

High throughput measurement of morphology 

Integrated metrology (morphology, composition, extent of reaction) on microfluidics 

produced organic solvent-based libraries 

3c.	 If you use or adapted any of the NCMC technologies listed in the previous question, did they [Select One] 

offer efficiency benefits or cost savings over what your lab had been using heretofore? 

If yes, please estimate the combined benefits adoption of these NCMC technologies had on the following factors: 

Increase in research labor productivity: [Select One] 

Reduced materials expenses, per 10 samples processed:	 US$ 

Savings on research equipment expenditures (excluding US$

one-time adoption costs): 

 Time Savings: [Select One] 

 Comments? 

3d. Did the NCMC technologies you adopted or adapted (from 3b) improve the quality of your [Select One]

research or any products that your company may produce? 

 Comments? 

3e. Did the NCMC technologies from 3b enable you to test samples over a broader range of [Select One] 

conditions? 

C-4 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Appendix C — Nonmember Survey 

3f. Please estimate the one-time costs you incurred when adopting or adapting any of the NCMC technologies listed 

in Question 3b. Would you characterize these expenses as a large investment in labor, research equipment, or 

instillation relative to other technologies your lab has adopted? 

 Comments: 

4. Laboratory Personnel Working in Combi. 

The purpose of this section is to estimate how many researchers may be using combi for R&D and how that estimate 

changed over time. This information will be combined with other responses to estimate the number of people working in 

combi today. If you are unsure of an estimate for any given year, simply leave the cell empty. 

4a. Approximately how many full-time equivalent employees (FTEs) worked in combi in your lab during the following 

years? Please separate those FTEs that were primarily involved in developing/adopting/adapting new combi 

methods from those FTEs that were primarily involved in the regular use of established combi methods. 

Year 1999 2001 2003 2005 2007 

FTE(s) primarily involved in 

developing/adopting/adapting new combi methods 

FTE(s) primarily involved in regular use of 

established combi methods 

4b. What percent of your combi activity currently falls within each of the following library types currently and in 2003? 

2003 Current 

Thin films libraries % %

 Discrete libraries % %

 Microfluidics libraries % %
 

Comments: 

5. Organization or Industry Classification. 

Please indicate which of the following best describes your lab. 

Advanced materials lab (e.g. resins, synthetic rubbers, 

artificial synthetic fibers) 

University research laboratory 

Coatings, adhesives, paints, and pigments lab Government or institutional research laboratory 

Personal care and household products lab Other: 

6. Activity and Size Measures. 

RTI has obtained industry-level measures of economic activity, and would like to aggregate your responses with those of 

others to extrapolate to industry-level measures. The measures below will only be used to combine your responses with 

those of others. Again, your best approximation will suffice. 

6a. For private-sector respondents: 

Stock trading symbol (“Ticker”): 

Approximate 2007 company sales revenue: 

Approximate 2007 company R&D expenditures: 

million US$ 

million US$ 

6b. For academic and public sector respondents: 

Approximate 2007 laboratory or department funding: 

Approximate 2007 full-time equivalent staffing: 

Approximate number of student staff (all levels): 

million US$ 

FTEs 

students 

C-5 
 

 

 

 

 

 

Retrospective Economic Impact Assessment of the NIST Combinatorial Methods Center 

7. Respondent Contact Information

Responses to this survey are strictly confidential. At no time will any individual’s name, any company name, participation, 

or identifiable response be released by RTI to any third party, including the NIST and the NCMC. 

Geographic location, if not USA: 

Respondent name (optional): 

Affiliation (optional):
 

Title (optional): 

Telephone number (optional) 

Email (optional): 

Are you willing to participate in a short, confidential follow-up telephone discussion about your [Select One] response? If so, please ensure that you provided your email address or telephone number above. 

C-6