Planning Report 12-1

The Economic Impacts of

Documentary Standards:

A Case Study of the Flat

Panel Display Measurement

Standard (FPDM)

David P. Leech

TASC, Inc.

John T. Scott Ph.D.

Consultant

Dartmouth College

December 2011
The Economic Impacts of Documentary Standards: 

A Case Study of the Flat Panel Display Measurement 

Standard (FPDM)

David P. Leech

John T. Scott Ph.D.

GCR G2012-0299
GCR G2012-0299

The Economic Impacts of Documentary Standards: A Case 

Study of the Flat Panel Display Measurement Standard 

(FPDM)

Prepared for

National Institute of Standards and Technology, Standards Coordination Office

This publication was produced as part of contract SB134109NC0286 with the National Institute of Standards 

and Technology. The contents of this publication do not necessarily reflect the views or policies of the National 

Institute of Standards and Technology or the US Government

David P. Leech

TASC, Inc.

John T. Scott Ph.D.

Consultant

Dartmouth College

December 2011

 U.S. Department of Commerce

John E. Bryson, Secretary

National Institute of Standards and Technology

Patrick D. Gallagher, Director
ii

ES. Executive Summary

One of the functions of NIST and its Standards Coordination Office (SCO) is to assist

U.S. industry with the standards-related tools and information it needs to compete more

effectively in the global marketplace. This analysis is one of a series of NIST assessments

that focus on examining the economic impacts of documentary standards. Its purpose is 

to assist NIST in more effective leveraging of the expert resources that it devotes to the

development of voluntary consensus standards.

Each year as many as four hundred NIST experts are involved in more than 1000 private 

sector-led documentary standards committee activities. With this high level or resource 

commitment, NIST expertise and resources should be focused on areas that have the 

highest potential impact and likelihood of achieving NIST‘s goals and objectives. This 

economic impact assessment contributes to understanding how that is achieved.

The selection of the Video Electronics Standards Association‘s (VESA‘s) Flat Panel 

Display Measurement (FPDM) standard was based on pragmatic criteria namely, 

availability of information, but was, in retrospect, fortuitous. The history of the FPDM 

project, and the investigation of the way in which NIST‘s know-how is developed and 

enters the stream of industry value creation provides many lessons learned. The 

development of similar case studies may well provide the foundation for a richly 

informed understanding of why such engagements with industry succeed and fail. Not 

only was the FPDM a success from an economic impact perspective, but it is also the 

story of the role documentary standards can play in a global knowledge-based growth 

industry and how a NIST ―core competency‖ (measurement technology), in partnership 

with industry, can hold important keys to innovation and economic growth.

In 1992, just a few years after the Omnibus Trade and Competitiveness Act of 1988

changed the name of the National Bureau of Standards (NBS) to the National Institute of 

Standards and Technology (NIST), the Flat Panel Display Laboratory (FPDL) was 

established at the Gaithersburg campus of NIST1. Much like today, global 

competitiveness and innovation was of foremost concern and helped provide a catalyst 

for the US investment in flat panel technology. But the ultimate success of the FPDL and 

the FPDM standard is not centrally a story about competitiveness. Rather, it is a story of 

how innovation was fostered from the close collaboration of industry and NIST, solving 

difficult technical and commercial problems using NIST‘s unique resources, in exactly 

those areas where such collaboration is least controversial — standards development.

Acting alone, industry would have faced difficult proprietary and technical barriers. In 

this case, NIST was able to help by exercising its essential roles as an honest broker and a 

conduit for the highest international standards based on good measurement practice. As 

part of VESA‘s FPDM Workgroup, NIST experts worked hand-in-hand with industry 

display measurement leaders. This collaboration helped accelerate the transfer of 

measurement know-how, and the application of flat panels to a wide array of industries at 

a cost that, working alone, industry would have been highly unlikely to achieve.

 

1 The laboratory was moved to the Boulder campus in 2003.
iii

While not without its critics, the FPDM Workgroup produced a measurement standard 

unlike any other by most accounts. And all agree that NIST‘s technical staff made a great 

difference. In addition to technical expertise, and probably largely because of it, NIST 

forged a close and effective relationship with its industry counterparts. The result of this 

collaboration included: expansion of markets, increased product quality for a rapidly 

expanding array of products, and lower costs of production. In the words of a prominent 

industry spokesman, ―displays are becoming pervasive throughout today‘s information 

society, and having a uniform and consistent means of measuring and evaluating them is 

essential.‖

Through the collaborative efforts of VESA‘s FPDM Workgroup, with NIST as a partner, 

the project developed and transferred measurement technology; codified display 

measurement know-how in a comprehensive measurement standard (FPDM); enabled

annual savings of thousands of hours of metrology operations within firms and additional 

hundreds of hours in SDO consensus-making efforts (not only for VESA but for other 

SDOs that adopted FPDM as part of their own related work); and effected the application 

of flat panel technology to ever-widening uses to the benefit of the consuming public. 

This economic impact assessment estimates only selected areas of the FPDM‘s economic 

impact: industry metrology labor savings that resulted from the adoption of FPDM, and

the consensus-making efficiencies resulting from NIST‘s participation in VESA‘s FPDM 

Workgroup. As only these impacts are captured in this analysis, the estimates of 

economic impact shown in Table ES-1 are highly conservative. 

Table ES-1. Estimate of Economic Impact

Performance Metric Value

Net Present Value in 1992 $15,573,930

Net Present Value in 2010 $56,323,545

Real Social Rate of Return 48%

Benefit-to-Cost Ratio 4

Even as a very conservative estimate of the project‘s outcome, the FPDM‘s economic 

impact is impressive. With the availability of better data the estimate of the impact of the 

FPDM would be considerably higher and would capture: 

• FPDMs impact on the quality of products that use FPDs and the 

consumer benefits that accrue from such quality improvements

• The value to industry of measurement device designs that were a product 

of NIST‘s role in the FPDM Workgroup

• Extra benefits to industry of releasing the FPDM sooner than it would 

have been released, due to NIST‘s participation

• The codified know-how carried forward into the International 

Committee on Display Measurement‘s (ICDM‘s) forthcoming Display 

Measurement Standard (DMS). 
iv

Table of Contents

ES. Executive Summary ...........................................................................................ii

List of Figures ...............................................................................................................v

List of Tables.................................................................................................................v

1 Introduction .............................................................................................................1

1.1 What is a Documentary Standard? .........................................................................................1

1.2 NIST‘s Role in the Documentary Standards Process .............................................................1

1.3 NIST‘s Standards Coordinating Office ..................................................................................3

1.4 Assessing the Economic Impact of Documentary Standards at NIST....................................3

1.5 Flat Panel Display Measurement Standard (FPDM) ..............................................................4

1.6 ICDMs Display Measurement Standard (DMS).....................................................................6

2 Background .............................................................................................................7

2.1 Development and Take-Off of the Flat Panel Display Industry .............................................7

2.2 Flat Panel Display Metrology...............................................................................................10

2.3 Establishment of NIST‘s Flat Panel Display Laboratory (FPDL)........................................11

2.4 Video Electronics Standards Association (VESA) FPDM Workgroup................................14

2.4.1 VESA....................................................................................................................... 14

2.4.2 FPDM Workgroup................................................................................................... 14

3 Economic Analysis Framework............................................................................17

3.1 Documentary Standards as ―Infratechnology‖ .....................................................................17

3.2 An Economic Perspective on the Genesis of a the FPDM ...................................................19

3.3 The Economic Implications of FPDM Workgroup Inputs and Outputs...............................26

3.3.1 FPDM: Codified Measurement Know-How............................................................ 26

3.3.2 FPDM-Related Metrology Equipment Design ........................................................ 28

3.3.3 SDO Process Efficiencies........................................................................................ 29

3.4 Affected Organizations: Industry Structure..........................................................................31

4 Assessment Framework and Approach...............................................................33

4.1 Expected Outcomes..............................................................................................................33

4.2 Barriers to Innovation...........................................................................................................34

4.3 Comparison Scenario............................................................................................................36

4.4 Economic Impact Timeframe (1992-2010) ..........................................................................38

4.5 Economic Impact Variables..................................................................................................38

5 Survey Findings ....................................................................................................39

5.1 Survey Strategy, Population, and Sample.............................................................................39

5.2 Qualitative Findings .............................................................................................................40

5.3 Quantitative Findings ...........................................................................................................42

6 Quantitative Analysis ............................................................................................44

6.1 Benefits.................................................................................................................................44

6.2 SDO Costs and NIST Expenditures......................................................................................45

6.3 Measures of Economic Impact .............................................................................................46

7 Conclusion.............................................................................................................50

Appendix A: Types of Documentary Standards ISO/IEC Guide 2: 2004.................53

Appendix B. Flat Panel Display Laboratory Publications and Presentations .........54

Appendix C. FPDM Standard Economic Impact Survey Instrument........................67

Appendix D. Economic Impact Metrics......................................................................73
v

List of Figures

Figure 3-1. Flat Panel Display Industry Value Chain ................................................................... 31

List of Tables

Table ES-1. Estimate of Economic Impact .................................................................................... iii

Table 2-1. Market Growth by Selected FPD Applications.............................................................. 9

Table 3-1. Documentary Standards Economic Benefit Matrix ..................................................... 18

Table 3-2. Quality Dimensions for Documentary Standards ........................................................ 29

Table 4-1. Sources of Cost-Reduction Benefits from FPDM........................................................ 34

Table 6-1. FPDM-Related Benefits (Nominal $) .......................................................................... 45

Table 6-2. FPDM Workgroup Costs and NIST Expenditures (Nominal $) .................................. 46

Table 6-3. Constant 2010 Dollar Benefits and Costs (1992 - 2010) ............................................. 47

Table 6-4. Estimate of Economic Impact ...................................................................................... 48
vi

Acknowledgments

The authors want to acknowledge the guidance and judgment provided by Erik Puskar 

and Mary Donaldson of NIST's Standards Coordination Office over the entire course of 

study effort, from conceptualization to final editing. Extensive communications with 

knowledgeable technical experts and industry participant-observers were indispensible 

for the authors' understanding of the flat panel industry and display measurement. While 

input from many technical experts was received, we wish to specifically thank the 

following individuals for their contributions without whose guidance this study would not 

have been completed; Paul A. Boynton (NIST Quantum Measurement Division), Edward 

F. Kelley, Joe Miseli, Thomas P. Murtha, Mark Fihn, and Phil Downen. We are also 

grateful to Greg Tassey for his review of the paper.
1

1 Introduction

1.1 What is a Documentary Standard?

There are two different types of standards -- physical measurement standards and 

documentary standards. 2

The National Institute of Standards and Technology (NIST) is responsible for developing, 

maintaining and disseminating national physical measurement standards for basic 

measurement quantities (such as mass, time and frequency), which are traceable to the 

International System of Units (SI). Together they promote order, efficiency, and fairness 

in the marketplace, facilitate technological progress, and ultimately enhance U.S. 

competitiveness.

Documentary standards are written agreements among producers and/or users of products 

and services containing technical specifications or other precise criteria that may contain 

rules, guidelines, or definitions of characteristics. These standards ensure that materials, 

products, personnel qualifications, processes, and services are adequate for their purpose, 

compatible and/or interchangeable, if necessary; ensure public health and safety; protect 

the environment; and/or improve economic performance. 

Documentary standards3can specify product characteristics, establish accepted test 

methods and procedures, characterize materials, define processes and systems, or specify 

knowledge, training and competencies for specific tasks.

Frequently (perhaps optimally), documentary standards act as a bridge connecting 

national representations of international physical measurement standards to the day-today operations of industry in their research and development (R&D) efforts; in the 

application of this R&D to new technologies and innovations; and in transitioning new 

products and services into growing markets. 

1.2 NIST’s Role in the Documentary Standards Process

NIST has a variety of roles in the private sector-led U.S. voluntary standards system. 

First, NIST frequently provides research and measurements that establish the 

underpinning for standards, ranging from materials test methods to standards for building 

performance, and for a range of technologies, from information and communications 

technologies to nano- and bio-technologies. Second, NIST‘s staff members often 

participate in the preparation of the standards documents through their work on private 

sector-led standards committees and, in support of those committees, develop and 

organize workshops, seminars, and conferences.

 

2 Maureen A. Breitenberg, The ABC’s of Standards Activities (NISTIR 7614), August 2009, National 

Institute for Standards and Technology.

3

ISO/IEC Guide 2: 2004, Standardization and related activities -- General vocabulary, provides general 

terms and definitions concerning standardization and related activities. See Appendix A for types of 

documentary standards.
2

Third, under the National Technology Transfer and Advancement Act (NTTAA), NIST is 

responsible for coordinating federal, state and local activities in voluntary standards and 

working with industry and government to develop and apply technology, measurements 

and standards. Fourth, NIST is responsible for chairing the Interagency Committee on 

Standards Policy (ICSP), which helps to ensure effective participation by the federal 

government in domestic and international standards and conformity assessment activities 

and promote the adherence to uniform policies by federal agencies in the development 

and use of standards and in conformity assessment activities. 

In addition, NIST is responsible for the following efforts:

• National Inquiry Point on Technical Barriers to Trade4

• National Voluntary Laboratory Accreditation Program (NVLAP)5, 

• Maintaining the fundamental physical standards, such as length, time 

and frequency and units of mass, which underlie measurements 

contained in standards.6

NIST coordinates its standards role with that of the private sector through a 

Memorandum of Understanding (MoU) with the American National Standards Institute 

(ANSI).7The MoU is intended to improve domestic communication and coordination 

 

4 The National Inquiry Point on Technical Barriers to Trade provides research services on standards, 

technical regulations, and conformity assessment procedures for non-agricultural products to assist in 

carrying out the U.S. government‘s responsibilities under the World Trade Organization (WTO) Agreement 

on Technical Barriers to Trade (TBT).

5 The NVLAP provides third-party accreditation to testing and calibration laboratories in response to 

Congressional mandates or administrative actions by the Federal Government or from requests by privatesector organizations. The NAVLAP operates in conformance with International Organization for 

Standardization (ISO) and the International Electrotechnical Commission (IEC) standards, including 

ISO/IEC 17025 and ISO/IEC 17011.

6 These fundamental physical standards are national realizations of standards of measurement agreed to by 

the International Committee for Weights and Measures (CIPM) under the Treaty of the Meter. The CIPM 

establishes, maintains, and disseminates the International System of Units (SI). These SI definitions are 

stated without uncertainties. To be useful, SI definitions require physical artifacts or experimental 

apparatuses that generate realizations of the SI units. Generating and maintaining such realizations is the 

business of NIST and similar national metrology institutes in other countries. For an example of how this 

system works in a specific instance — the volt — See, D. Leech, Economic Impact Assessment of the 

NIST’s Josephson Volt Standard Program, (Planning Report 01-1), NIST, July, 2001, pp. 4-7.

7 ANSI has served as administrator and coordinator of the United States private sector, voluntary 

standardization system for almost 90 years. ANSI is a private, not-for-profit membership organization 

supported by a diverse constituency of private and public sector organizations including government 

agencies, companies, academic and international bodies, and individuals. ANSI accredits U.S. standards 

developers using criteria based on international requirements. ANSI has accredited over 200 standards 

developers in the private and public sectors. These accredited Standards Developing Organizations (SDOs)

develop standards based on consensus and other principles, and can choose to publish such standards as

American National Standards (ANS). ANSI is the sole U.S. representative and dues-paying member of the 

two major non-treaty international standards developing organizations, the International Standards 

Organization (ISO) and the International Electrotechnical Commission (IEC). Through ANSI, the U.S. has 

immediate access to the ISO and IEC standards development processes. ANSI administers many key ISO 

and IEC committees and subcommittees. 
3

among both private and public sector parties in the United States on voluntary standards 

issues and increases the effectiveness of U.S. government agency participation in the 

national and international voluntary standards-setting process.

1.3 NIST’s Standards Coordinating Office

One of NIST‘s central missions is to assist U.S. industry by providing standards-related 

tools and information it needs in order to compete more effectively in the global 

marketplace. The Standards Coordination Office (SCO) within NIST supports this 

mission by conducting standards-related programs, and providing knowledge and 

services including the monitoring of global standards developments and conformity 

assessment activities.

Under the aegis of the National Technology Transfer and Advancement Act of 1995 

(NTTA), SCO manages the coordination of federal, state, and local technical standards 

and conformity assessment activities, as well as coordinating with those in the private 

sector. SCO coordinates the activities of the Interagency Committee on Standards Policy, 

and participates in the Commerce Standards Committee and operates the Global 

Standards Information website.

8

To execute its responsibility, SCO offers training, publications, policy analysis, and 

research and information services and works closely with U.S. industry, standards 

developers, other government agencies in the global standards community to maintain 

and improve a technological infrastructure that supports innovation and economic 

growth.

1.4 Assessing the Economic Impact of Documentary Standards at NIST

Promoting U.S. innovation and industrial competitiveness is part of NIST‘s mission. One 

of NIST‘s core competencies is to contribute to the development and use of sound 

technical standards. To form a comprehensive picture of NIST‘s documentary standards 

development activities, in 2008 NIST labs and centers were canvassed to answer the 

question: How well is NIST doing in this area? The laboratories and centers were asked 

to identify instances where NIST has played an active role in the development or 

implementation of documentary standards.9 At the time of the canvassing, SCO identified 

404 NIST personnel involved in 1398 documentary standards committees. Seventy-eight 

of those canvassed reported their involvement in what they regard as ―high-impact‖ 

documentary standards efforts. 

Over time NIST wants to develop a reliable approach to understanding and assessing its 

influence on standards development processes so as to better monitor performance and 

 

8 The Interagency Committee on Standards Policy (ICSP) advises the Secretary of Commerce and other 

Executive Branch agencies in standards policy matters, and reports to the Secretary of Commerce through 

the Director of the National Institute of Standards and Technology (NIST). The Committee's authority is set 

out in Section 13 b of OMB Circular No. A 119. 

9 Erik Puskar, Selected Impacts of Documentary Standards Supported by NIST (NISTIR 7548), U.S. 

Department of Commerce, National Institute of Standards and Technology (NIST), 2008.
4

assess impacts. SCO wants to gain an understanding of the best way to allocate NIST 

resources to documentary standards development projects and the best circumstances in 

which to engage the standards development community. Several of the self-reported 

―high-impact‖ documentary standards projects were selected for further analysis and two 

were selected as potential candidates for quantitative economic impact assessment. The 

Flat Panel Display Measurement standard development project was selected for initial 

quantitative impact assessment. Unlike other impact assessments conducted by NIST, one 

of the goals of this assessment project is to discern attributes of the documentary standard 

development process that appear to affect impact. In the future, a collection of such 

attributes from across impact assessments will be tested for their potential as ―leading 

indicators‖ of the economic value of NIST‘s participation in the development of specific 

documentary standards.

1.5 Flat Panel Display Measurement Standard (FPDM)

Flat-panel displays (FPDs) are a class of advanced display technologies that have 

emerged to replace traditional cathode ray tubes because of advantages in weight, power 

needs, resolution, and high information content. There are several technologies 

competing for market share, and a number of niche applications. (See section 2.1 below, 

for a more detailed discussion of the evolution of FPD technology.)

In the vernacular of ISO/IEC Guide 2: 2004, the FPDM is a ―testing standard.‖(See 

Appendix A.) According to industry metrologists, the FPDM standard has become the de 

facto ―standard of standards,‖ serving the display industry with a comprehensive catalog 

of versatile optical measurements and informative technical discussions well grounded in 

metrology. According to one participant observer in the FPDM development process,

―No other standard offers so much practical information organized so 

well.‖10,11

In May, 1998, the FPDM was released by the FPDM Workgroup of the VESA Display 

Committee (discussed in section 2.3 below). It was developed to fill a void in FPD 

metrology. Other standards spoke to the issue of what to measure but nothing had been 

published on how to measure and how to avoid the pitfalls of inadequate metrology.12

In 1994, prior to the publication of FPDM 1.0 (1998), a NIST workshop on display 

standards concluded that few companies could afford to follow all the display standards 

 

10 Phil Downen, ―A Closer Look at Flat Panel-Display Measurement Standards and Trends.‖ Information 

Display, No, 1, 2006.

11 Not all well-informed observers agree with this accolade. One expert voiced the opinion that the FPDM 

was not even a ―standard‖ in the ordinary sense that it focused on one way to measure ―X.‖ And in this 

expert‘s opinion, the FPDM introduced some inefficiencies into the display industry as a result. The 

preponderance of opinion seems to be closer to Downen‘s view, Ibid. That said, the view that FPDM is not 

an ordinary standard will be used to support the evaluation strategy explained in section 4.3 of this report.

12 Ibid., p.16. Other relevant standards emphasized: physical aspects of the human-display interface, and 

classified displays in terms of viewing direction, contrast and color, reflection performance and pixel faults 

to determine the suitability of a display for the office environment (ISO 13406-2, ―Ergonomic 

Requirements for Work with Visual Display based on Flat Panel‖); compliance mandates
5

activities; that it would be helpful for industry metrologists to have access to the latest 

measurement developments; and that users of displays (companies that integrate displays 

into their products) need a bedrock of measurement standards to enable their choice of 

the best display for their application. 

Metrologists from NIST and industry‘s David Sarnoff Research Center National 

Information Display Laboratory (NIDL) initiated a survey of existing display standards.

13

The authors — all seasoned display metrologists — reported a blossoming of display 

applications and technologies and the growing interdependence of users and 

manufacturers of display devices.14 This, in turn, was driving a proliferation of standardsmaking efforts characterized, unfortunately, by ―a great deal of overlap among the 

various display standards.‖ And yet the methods of measurement among the various 

standard were not the same.‖ According to the report, ―one finds a wide range of detail 

describing precisely how any measurement is to be made and reported.‖15

Prior to the development of FPDM, a producer or buyer of flat panel displays (FPDs) 

would consult any (or all) of a number of existing standards, depending on the specific 

application.

16 In the absence of FPDM (1998), a producer and/or buyer would have to 

expend considerable resources working out which measurements best suited a given 

need; working out the numerous technical measurement inconsistencies among the 

various measurement standards; and negotiating agreement on chosen metrics and 

measurement procedures to be used in qualifying a FPD for future use, or in assessing the 

quality of the FPDs provided by manufacturers to integrators or users. 

The FPDM resolved many of these technical difficulties by offering a measurement 

standard with a set of measurement procedures that is unambiguous, that applied to 

multiple display technologies, and that was practical and cost-effective to utilize. 

 

13 The National Information Display Laboratory (NIDL) was part of the David Sarnoff Research Center 

(itself an early pioneer in liquid crystal display materials and flat panel display technology during the 1960s 

and early 1970s) and conducted ―third party evaluations‖ (like an ―Underwriter‘s Laboratory) for 

technically sophisticated government and corporate clients. (Personal communication with Michael Grote, 

April 5, 2010, former NIDL employee who was also active and VESA‘s FPDM Workgroup.)

14 This observed ―growing interdependence of users and manufacturers of display devices‖ signaled an 

important industrial transition that, it is hypothesized, opened the door for NIST to play an important role in 

a booming global industry. See the discussion in section 3.2 below.

15 Edward Kelley et al., ―A survey of components of display-measurement standards, Journal of the SID, 

No. 3/4, 1995, pp. 219-222.

16 The array of standards a display metrologist would consult included ISO 9241 (Parts 3, 7, 8) and ISO 

13406 (draft 2); ANSI HSF-100 (1988) and IT7.215 (1992); EIA TEB (27) and TEP (105); VESA Display 

Specifications and Test Procedures (for CRTs); NIDL‘s Procedures for Evaluation and Reporting the 

Capabilities of High Performance Display Monitors for Imagery Applications; SAE ARP 1782 and ARP 

4260; MRP 1990:8 (1990:10);USAF AFGS 87213A; and IEC SC 47C. ISO – International Standards 

Organization; ANSI – American National Standards Institute; EIA – Electronic Industries Association; EC 

– International Electrotechnical Commission; SAE – Society of Automotive Engineers; MRP – Swedish 

National Board of Measurement and Testing; NIDL - National Information Display Laboratory; USAF –

United States Air Force. See, Edward F. Kelley, George R. Jones, Paul A. Boynton, Michael D. Grote, and 

Dennis J. Bechis, ―A Survey of the Components of Display Measurement Standards,‖ Journal of the Society 

for Information Display, Vol. 3, No. 4, December 1995, pp. 219-222.
6

1.6 ICDMs Display Measurement Standard (DMS)

Display products have evolved at a fast pace. To keep up with those changes further 

revisions of FPDM have been necessary. In 2007, the FPDM Workgroup efforts, 

including the NIST representatives, were transferred from VESA to the Society of 

Information Display (SID) and the International Committee for Display Metrology 

(ICDM) was formed as part of SID‘s Definitions and Standards Committee. The ICDM 

included the core group of individuals who wrote the existing versions of FPDM with 

expanded participation of world experts from within the SID membership in the area of 

display evaluation. Experts of the Committee Working Group include display engineers, 

metrologists, color scientists, vision scientists, university researchers, human-factor 

specialists, and ergonomists. The third version of what was the FPDM — now, the 

Display Measurement Standard (DMS) — is expected to be released in 2011.
7

2 Background

2.1 Development and Take-Off of the Flat Panel Display Industry17

Flat-panel displays are a class of advanced display technologies that have emerged to 

replace traditional cathode ray tubes (CRTs) because of advantages in weight, power 

needs, high resolution, and high information content. There are several technologies 

competing for market share, and a number of niche applications. During the 1980s, 

pushed by leading Japanese companies, liquid crystal displays (LCDs) became the 

dominant commercial display technology. LCDs in turn come in two types: active and 

passive matrix LCDs. The more advanced (active matrix) AMLCDs, consist of a thin 

film transistor (TFT) attached to each of hundreds of thousands of pixels (short for 

picture elements) that together form an image on a television screen).

FPD technologies include plasma display panels (PDPs), in which ionized gases produce 

the light; field emission displays (FEDs), an improved cathode ray technology that 

combines with semiconductor technology; and electroluminescent displays (ELDs), 

which generate light from phosphorus, sandwiched between electrodes. All of these 

technologies are high-information content displays, capable of containing large amounts 

of information required, for example, in high-definition television and computer displays. 

Not all LCDs are high information; there is still a large market for low-information LCDs 

in such items as wristwatches, calculators, thermometers, and appliances. Though U.S. 

companies pioneered the technology, today Japanese, Korean, and, increasingly, Chinese 

companies control the lion‘s share of the world market. 

Beginning in the 1960s, and throughout the 1970s and 1980s, scientists and product 

developers in the U.S. (at RCA, IBM, Westinghouse, Texas Instruments, Hughes 

Aircraft, Rockwell, ATT, to name some familiar U.S. corporate R&D powerhouses), and 

their Japanese partners and competitors (prominently Sharp, Seiko Epson, SuwaSeikosha, and Hitachi) worked to perfect display technologies that would eventually 

make giant, wall-hanging, flat TV sets possible —and affordable—for any household. 

Along the way, Japanese, U.S., and European firms developed products and 

commercialized a dizzying array of underlying material, component, and device 

technologies such as instrument read-outs, wristwatch displays, calculator displays, 

personal digital assistants, and, eventually, small TV screens (primarily in the 3-5 inch 

range).18

 

17 This section is based on Thomas Murtha, et. al., Managing New Industry Creation: Global Knowledge 

Formation and Entrepreneurship in High Technology, Stanford University Press, 2001; Claude Barfield, 

High-Tech Protectionism, AEI Press, December 2003; Joseph Castellano, Liquid Gold: The Story of Liquid 

Crystal Displays and the Creation of an Industry, World Scientific Publishing Company, 2005; and John 

A. Mathews, ―Strategy and the Crystal Cycle,‖ California Management Review, Vol. 47, No. 2, Winter

2005, pp. 6-32.

18 The first flat panel TV was is attributed to John vaan Raalte of RCA. He demonstrated the first off-air 

moving TV on an LCD in 1967. Sanyo Electric & and Sanritsu Electric together build the first 3-inch color 

AMLCD TV using alpha-Si:H in 1983. In 1987, GE‘s R&D Center in New York built the first 88-inch 

color TV utilizing an AMLCD with alpha-Si:H TFTs and Hitachi introduced a 5-inch color TV using 
8

Throughout the 1980s flat panel computer displays were also being developed. In 1988, 

IBM and Toshiba developed a prototype for what would prove to be the first ―killer app‖:

14-inch working prototypes of color liquid crystal displays.19 According to industry 

observers, by the early 1990s, the FPD industry stood on the brink of taking off. In 

early1992, IBM Personal Computer Company had received orders for more than 100,000 

units of the Thinkpad (which utilized a 10.4-inch DTI color display). ―The 14-inch thinfilm-transistor (TFT) LCD demonstrations sounded the starting bell in the knowledge 

race to create high-volume manufacturing processes for large-format, color FPD 

production.‖20

And yet the 14-inch prototypes did not establish the underlying TFT LCD technology as 

a standard. Development continued in a number of alternative FPD technologies. The 

demonstrations did not establish a dominant design for TFT LCDs either. ―The products 

continued to evolve on multiple dimensions, including size, video response rates, color 

subtlety, viewing angle, power consumption, and weight, just to name a few.‖21

Many technical performance limitations on TFT LCDs could only be solved by the 

accumulation of manufacturing process knowledge, as well as its embodiment in 

successive generations of manufacturing equipment and materials.22 But as these 

 

AMLCD with alpha-Si:H TFTs. In 1988, the Japanese government sponsored a consortium (chartered by 

MITI) with the goal of developing a 40-inch wall-hung TV. Developing a 40-inch HDTV by 1998 was the 

goal of the Japanese High-Vision PDP Consortium launched in 1994. In that same year, NIST‘s FPDL 

published papers on measuring noise in TV receivers and TV displays. In 1996 Mitsubishi acquired 

Plasmaco for the promise of its PDP technology for HDTV. Samsung built the first 30-inch color TFTLCD for TV in 1997. In 1998, the large PDP TV that was the goal of Japan‘s High-Vision PDP 

Consortium, launched in 1994, was demonstrated at Nagano Olympic Games and Sharp announced its first 

complete line of flat HDTVs. Plasma TVs started to significantly penetrate the high-end consumer market 

by 2000, the same year that Planar Systems (the leading U.S. manufacturers of EL displays) exited the 

market for military displays. By 2003, a 52-inch and a 55-inch color TFT LCD display was available from 

LG Philips for TV to market, and 54-inch and 58-inch color TFT LCD display was available from 

Samsung. That year (2003) was also the year that FPDs eclipsed CRTs in their share of world market value 

and the year that Samsung and Sony formed the S-LCD panel production alliance that propelled Sony 

Bravia to the global TV market share lead in 2006. These historical citations are taken from Mutha, et. al., 

op cit.; and Castellano, op cit.

19 Seiko Epson and Grid Systems introduced portable computers to the U.S. in 1982, the later with a display 

supplied by Sharp. In 1983, Kyocera built the first portable computers with passive LCDs. These were used 

in models sold by Tandy, NEC, and Olivetti. Sharp began high-volume production of STN-LCDs for 

notebook computers in1986. In 1988, IBM and Toshiba jointly developed a 14.3 flat panel display using 

active matrix technology, and, in 1989, NEC, Toshiba, Sharp, Mitsubishi introduce portable computers 

with color screens. Mutha, et. al., op cit.; and Castellano, op cit.

20 Murtha, et. al., op. cit., p. 11.

21 Ibid.

22 Ibid. Generation 2 manufacturing lines were in place by mid-1994. The AKT-1600, a four-chamber 

production system, sold for about $5 million and could process approximately forty substrates per hour. 

High-volume fabs needed about four of them. Murtha, et. al., op. cit., p. 113.
9

problems were being addressed by a new generation of manufacturing equipment, ―the 

competition to differentiate products to meet users' rapidly evolving needs had already 

rekindled.‖23

A respected industry analyst, writing in 1997, estimates that the FPD market grew from 

$100 million to $14 billion from 1980 to 1997.24,25 The market was driven first by 

consumer electronics, then portable computers. In 1997, the next wave of production was 

expected to include tens of millions of notebook computers, desktop monitors and TVs.

The explosive market growth of selected markets is shown in Table 2-1. According to 

industry observers, it was the market for notebook PC displays that really drove the 

market take-off. PC manufacturers couldn‘t get all the displays they needed from the 

market, much less from a single supplier. In the early 1990‘s one leading PC maker 

qualified the products of eight display manufacturers, and led the notebook display 

standardization charge — along with several other PC manufacturers — so the PC 

makers could obtain the same displays from multiple FPD manufacturers.26

Table 2-1. Market Growth by Selected FPD Applications

Market $M (1997) $M (2003)

Portable Computers $7,443 $10,360

Desktop Monitors $397 $7,483

Handheld Computers $119 $715

Camcorder $379 $421

Digital Still Camera $131 $313

GPS (aftermarket) $145 $290

Financial Terminals $136 $255

This growth has continued to be driven by the expanding applications of panels made 

with the dominant TFT-LCD technology. In addition to flat screens for notebook 

computers, new applications for these panels keep being developed including desktop 

computer monitors (displacing traditional cathode ray tube displays), LCD TVs for the 

consumer television market, small displays for cell phones, digital still cameras, and 

personal digital assistants, as well as very large format displays such as the electronic 

billboards that adorn the facades of buildings.

 

23 Murtha, et. al., op. cit., p. 113.

24 David E. Mentley, ―Market Opportunities for Advanced LCDs,‖ Stanford Resources, Inc., 1997 Flat 

Information Displays Conference.

25 It is assumed that these market value numbers are nominal, not adjusted for inflation. The source (Ibid.) 

provides no indication.

26 Mark Fihn, personal communication, November 5, 2010. The PC makers developed the Standard Panels 

Working Group (SPWG) standard. According to the SPWG web site, the organization was formed in May 

of 1999 by Compaq, Dell Computer, Hewlett-Packard, IBM, and Toshiba for the purpose of establishing 

mechanical and interface specifications for the displays used in notebook PCs. Widespread adoption of 

SPWG-compliant panels — encompassing specifications for 13.3", 14.1" and 15.0" displays or 

approximately 60% of all displays used in notebook PCs —has served to simplify design-in cycle times, to 

reduce component and manufacturing costs, and to better assure a steady source of supply in a volatile 

market. SPWG estimates that its adoption has saved the notebook PC industry more than $100 million in 

costs annually. The SPWG references the FPDM.
10

According to one estimate the industry as a whole generated $62 billion in gross revenues 

in 2004 with an anticipated yearly growth rate of approximately 40%. Writing in 2005, 

the analyst predicted that within the next decade the value of industry shipments would 

come to $100 to $200 billion, making it comparable to semiconductors.27 In 2010 the 

total revenue from the sale of TFT LCDs was approximately $90 billion.28

While Japanese firms dominated the product markets during the industry‘s ―take off,‖

Korean and Taiwanese firms have rapidly entered, as have firms from Singapore, China, 

and India. U.S. and European firms, by contrast, appear to be pinning their strategic 

aspirations on the development of new applications and new technologies, having 

apparently abandoned the effort to manufacture active-matrix TFT-LCDs in mass 

quantities. U.S. firms, such as Applied Materials, Corning, Photon Dynamics, and 

Applied Films Corporation have strong positions in the equipment and materials markets 

that supply FDP manufacturers. FPD manufacturers spend billions of dollars annually 

updating fabrication and test equipment.29

2.2 Flat Panel Display Metrology

Display metrology is concerned with the measurement of the optical properties of 

electronic displays. Display metrology is practiced in order to provide physical data as an 

objective basis for rating of the visual performance of electronic-display devices; e.g., 

luminance is measured in order to estimate the brightness perceived by a human 

observer. Display metrology and the resulting data are of significant commercial interest 

and thus often are subject to manipulations. Hence, the stakes for understanding and 

properly exercising the practice of display metrology can be high.30

In the early phases of the exploding market for FPDs there was a confusion of terms and 

definitions and measurement approaches. According to a leading industry metrologist, 

―This confusion is often exploited by marketers by selecting such measurements that are 

yielding the  ̳best numbers‘ for the product data sheet. In line with that mentality of 

specsmanship (i.e., abuse of technical data to establish putative superiority of one device 

over another) [an example] is the measurement of contrast in a dark room, yielding 

 

27 John A. Mathews, ―Strategy and the Crystal Cycle,‖ California Management Review, Vol. 47, No. 2, 

Winter2005, pp. 6-32. (Data update provided by John Mathews, personal communication, June 2, 2011.)

28 Estimated for 12 months from September 2009 to August 2010 from data provided by DisplaySearch at 

http://www.displaysearch.com/cps/rde/xchg/displaysearch/hs.xsl/resources_paneltrack.asp, November 4, 

2010.

29 Mathews, op. cit.

30 Michael Becker, ―Display Metrology: What Is It (Good for)?,‖ Information Display, February 2008 Vol. 

24, No. 2, p. 4, 52. Becker observes, ―In a 2003 court case (NEC Mitsubishi Display of America vs. 

ViewSonic Corp., Illinois Federal Court, Case No.: 02 C 08304), for example, NEC Mitsubishi charged 

that ViewSonic was misleading or confusing customers with the way they specified the contrast of 

ViewSonic LCD monitors. The case was between competitors but similar cases involving consumer 

advocacy groups have, over the years, taken manufacturers to task for issues of exaggerated technical 

claims.‖
11

(very) high numbers, especially for emissive displays, but under conditions that do not 

represent the actual application situation (about 99% of all display application cases are 

under ambient illumination).‖31

Display metrologists spend their entire careers pursuing two goals: 

• Making the most precise and accurate measurement of light from 

displays

• Finding the best correlation possible between what they measure and 

what users of the displays actually see. 

This is no simple task. ―There are an astounding number of commercial instruments on 

the market today that even when used as rigorously as possible are not better than ±10% 

in absolute accuracy. Whether that performance is good enough for the intended 

application is a real problem with which display metrologists regularly struggle.‖32

NIST‘s Flat Panel Display Laboratory (FPDL) was established in 1992, attuned to the 

fact that addressing these ―real problems‖ was a key to the proliferation and growth of 

FPD applications.

2.3 Establishment of NIST’s Flat Panel Display Laboratory (FPDL)

In the late1980s and early 1990s the U.S. economic policy community was engaged with 

concerns about U.S. international competitiveness.33 A NIST planning report from the 

period focused on the measurement facets of competitiveness:

At present, U.S. industry is experiencing a major shortfall in the 

measurement capability needed for competitiveness in electronic products. 

This document identifies the measurement needs that are most critical to 

U.S. competitiveness, that would have the highest economic impact if met, 

and that are the most difficult for the broad range of individual companies 

to address.34

According to one of its authors, the report is evidence of the institutional context in which 

NIST‘s FPDL was established.35 The National Bureau of Standards (NBS) had recently 

been reorganized and renamed the National Institute of Standards and Technology

(NIST) by the Omnibus Trade and Competitiveness Act of 1988. Two chapters of the 

report (Chapter 11,Video; and Chapter 12, Electromagnetic Compatibility) focus 

especially on flat panel displays, a focus explained by:

 

31 Ibid.

32 Stephen Atwood, ―You Might Be a Display Metrologist If,‖ Information Display, February 2008 Vol. 24, 

No. 2, p. 2.

33 See, for example, R. D. Norton, ―Industrial Policy and American Renewal,‖ Journal of Economic 

Literature, March 1986; and Otis Graham, Loosing Time: The Industrial Policy Debate, Harvard 

University Press, 1992.

34 Measurements for Competitiveness in Electronics (NISTIR 4583), U.S. Department of Commerce, 

National Institute of Standards and Technology, Electronics and Electrical Engineering Laboratory, April 

1993.

35 Communication with Herbert Bennett, November 29, 2010.
12

―The emphasis given in this chapter to flat-panel displays is driven largely 

by the intense competition already evident in this area. Nations, especially 

Japan, are investing very large amounts of resources to manufacture flatpanel displays since the markets for products containing displays are 

expected to be enormous. Japanese companies are investing over $2.0 

billion in flat-panel-display plants costing from $45 million to $300 

million each for the first stage. Additional stages may bring the total 

investment to over $900 million for some individual plants. The 

realization of the key role that displays play, and the investments that 

Japan and the U.S. have already made, have produced much interest in 

display technologies.‖

While NIST‘s expectations for U.S. industry‘s production of flat panel displays never 

came to fruition (as discussed in section 2.1, today the U.S. share of the worldwide 

display manufacturing industry is small), this FPDM impact assessment demonstrates

that the pay-off from NIST‘s focus on U.S. companies that incorporate flat panel displays 

in their products and services has been significant.

The FPDL was founded in 1992 on NIST‘s Gaithersburg, Maryland campus36, under the 

leadership of Dr. Edward Kelley. Kelly, Paul Boynton, and others, soon demonstrated 

that industry offered little in the way of standard tests to evaluate the quality of FPD 

products. Many of the people performing flat panel display measurements were doing it 

in a manner that would produce erroneous results. The researchers found a situation in 

which published specifications from one manufacturer's display couldn't be directly 

compared with the specifications from another manufacturer's display. The FPDL 

adopted a mission to create a battery of test methods that all manufacturers could use to 

determine the actual performance of their displays so that the end user could accurately 

compare flat-panel display performance. The technical situation facing display 

metrologists at the time has been described as follows: 

The two most important performance categories for any display device are 

contrast and color accuracy. Because of the nature of flat-panel devices, 

your viewing environment makes a major difference in the perceived 

contrast range of the display's picture. For years display manufacturers 

used the "bright room test" to determine a display's contrast under normal 

room conditions. This test consisted of placing a bank of fluorescent lights 

that put out 100 Lux of light above the display and then measuring the 

black and white boxes of a standard checkered display pattern. The

difference between the black box and the white box was the contrast ratio, 

although this test proved to be fundamentally flawed. First, the ambient 

light level in an average room is well above 100 Lux (closer to 300 to 400 

Lux). More importantly, depending on the placement of the light and 

measuring device, measurements could vary as much as 1,000 percent. 

Even minute changes in the location of the light, measuring device, and 

 

36 The FPDL was moved to the NIST campus in Boulder, Colorado in 2003. 
13

even the size of the testing room can cause radically differing results, 

making this test virtually worthless for comparing displays made by 

different manufacturers, as each company conducted tests in their own 

unique facility. There had to be a better way.

NIST‘s Flat Panel Display Lab developed a standard way to measure not 

only the reflectance of front-panel-display materials, but also a display's 

contrast under different ambient light levels. The methodology involves 

using a closed chamber with critically even illumination and a carefully 

calibrated light level that technicians can accurately reproduce day in and 

day out. Trained testing personnel can set up a chamber anywhere, and 

once it's properly calibrated, their tests will correlate with tests performed 

in other chambers anywhere in the world. This codification of testing 

procedures forms the backbone of the Flat Panel Display Lab's work.37

At the time, the best of the commercial measurement laboratories (for example, the David 

Sarnoff Research Center‘s National Information Display Laboratory (NIDL)), had a 

number of procedures written out that were self-contained, but they suffered from 

redundancy. Some of the metrics that became part of the FPDM standard were in 

common practice in the cathode ray tube (CRT) world and used by the leading 

laboratories in the U.S. and Japan. But often the documentation for the leading 

laboratories was sparse. Some related standards, ISO 13406-2 (an ergonomic standard),

for example, suffered from insufficient details on the methods and hidden interpretive 

problems. Old methods were designed for CRT displays, but were being applied to LCDs 

and other new technologies, resulting in serious measurement errors.

In the context of the VESA Workgroup, NIST experts worked with industry metrologists 

to remove redundancies and better organize their information. NIST experts participated 

in an inter-laboratory comparison to verify their metrics. Representatives from the 

leading commercial laboratories became part of VESA‘s FPDM Workgroup. 

With NIST‘s expertise, explanations were expanded and warnings were added. NIST 

contributed rigor and explanation. Over time, the work of the FPDL moved from the 

complicated modeling required to address basic metrology needs to correcting the many 

errors in most existing measurement procedures. According to Kelley, ―We focused the 

metrics on lab people making good measurements. ... We pulled it all together and made 

it robust, reproducible, unambiguous, and solid.‖38 There were four main tracks to the 

FPDL‘s efforts:

• Metrology efforts in support of the FPDM Workgroup

• Stray light management

• Reflection metrology

 

37 Mike Kahn, ―Flat Panel Display Lab,‖ January 2008, http://www.hometheatermag.com/hookmeup/108flathook. 

38 Communication with Ed Kelley, November 22, 2010.
14

• Diagnostics.39

FPDL activities attracted visiting scientists from national metrology institutes and 

governmental laboratories, including the United Kingdom (NPL), South Korea (KRISS), 

and Brazil (INMETRO). NIST‘s FPDL experts established a four-day display metrology 

―short course‖ to provide scientists and engineers extensive hands-on experience in 

display metrology. The course attracted over one hundred display professionals from 

thirteen countries.40

2.4 Video Electronics Standards Association (VESA) FPDM Workgroup

2.4.1 VESA

VESA‘s origins are rooted in the early evolution of the personal computer industry. The 

phenomenal growth of graphics capabilities for personal computers in the 1980s and the 

subsequent proliferation of non-compatible products led to the formation of VESA in 

1989. The VESA Flat Panel Display Interface (FPDI) Committee was formed in February 

1994 to develop a standardized interface between graphics controllers and flat panel 

displays used in integrated environments such as portable PCs.41

VESA‘s first standard allowed computer users to take full advantage of the advanced 

graphics capabilities offered on the market at that time.

2.4.2 FPDM Workgroup

In April 1995, the VESA FPDI Committee established an initiative to develop voluntary 

standards for the measurement of flat panel displays and formed the Flat Panel Display 

Measurement (FPDM) Workgroup. The Workgroup's purpose was to:

• Ensure consistent, relevant display measurements

• Foster display improvements and new developments

• Aid in display system design and procurement.42

The Workgroup was formed at the instigation of FPDI Committee members Joseph 

Miseli (Sun Microsystems) and Doug Baker (Compaq). Kelley, of NIST‘s FPDL, 

assumed the Workgroup chair in 1995, following Dennis Bechis (NIDL), and was the

editor of FPDM 1.0 and 2.0.43

The FPDM Workgroup built upon the metrology work of the NIDL and the work of 

Japan‘s Electronic Industries Association of Japan (EIAJ). The core of the FPDM is 

 

39 See Appendix B for a breakdown of published papers and presentations by FPDL staff that reflects their 

focus over the history of the FPDL. In addition to the four main tracks of research noted, FPDL staff 

conducted FPD metrology tutorials and developed display metrology for specific applications.

40 Undated NIST, SID Fellow Award nomination document. 

41 http://www1.ecs.ru.acad.bg/kp/less/dispalys/VD-site/vesa/ve00005.html

42 http://www1.ecs.ru.acad.bg/kp/less/dispalys/VD-site/vesa/ve00005.html

43 Communication with a source who requested anonymity, May 26, 2011.
15

metrology, the science of measurements, not only what to do but how and why do it a 

certain way. According to Kelley, ―All other standards lacked that element, and we filled 

the niche.‖44

As an example of the situation display metrologists faced, and the order and rigor that the 

VESA FPDM standard provided, FPDL‘s former director, recalls the following:

NIDL had a number of procedures written out. They were self-contained, 

but they suffered from a lot of redundancy .... We removed the redundancy 

and organized ... things better. There were common measurements made 

on CRTs. We even participated in an inter-laboratory comparison with 

[NIDL] to verify their metrics. All of them became part of the FPDM, I'd 

guess about 35% of the metrics in the FPDM were in common practice in 

the CRT world and used by NIDL. Another bunch, probably 15%, was

added from the EIAJ. The EIAJ document was very sparse. We added the

explanations and warnings. The NIST contribution to all of this was rigor 

and explanation where needed. We focused the metrics on lab people 

making good measurements. There was also the ISO 13406-2, an 

ergonomic standard that suffered from insufficient detail on the methods 

and ―gotchas,‖ and it was very hard to read. There were a lot of standards 

on CRTs, some good, like the SAE standards (in part), some bad. We 

pulled it all together and made it robust, reproducible, unambiguous, and 

solid.45

A representative of an equipment manufacturer that incorporates displays in sophisticated 

computer systems, similarly recalled:

In the early days of developing a product [specification] it took weeks and 

weeks of discussion and discovery to figure out how to make 

measurements, especially for ―contentious parameters‖ such as set-up 

conditions (emphasized in the FPDM). Part of this discussion was with the 

customers (for whom the system with the display was being designed) and 

part was with the display manufacturer. The more contentious the 

parameter the longer the discovery and negotiation process took. FPDM 

changed that. It provides a unified  ̳buffet‘ of measurements and set-up 

conditions that anyone can use and combine in any way they need but with 

the underlying metrology clearly established.46

VESA‘s FPDM avoided compliance language and offered a range of display measures 

from which manufacturers and users could pick and choose according to their 

requirements. According to the FPDM 2.0 introduction, ―What people do with the results 

is their own business, but we wanted to make sure that it was measured correctly.‖47

 

44 Communication with Ed Kelley, November 22, 2010.

45 Ibid.

46 Communication with an anonymous industry source, April 6, 2011.

47 FPDM,Version 2.0, 2001.
16

In this respect, the FPDM was unique. According to one close participant-observer, 

―Looking back, in 1992, [NIST‘s technical staff] were visionary. The market for the 

notebook PC and the desktop flat panel display had not exploded (until1998-2000) but 

they knew what was needed on the metrology front and they knew what they wanted to 

do.‖48

Released initially in 1998 (as FPDM Version 1.0), and re-released in June 2001 by a 

newly formed Display Metrology Committee of VESA, Version 2.0 incorporated 

corrections and clarifications and expanded test-item coverage. FPDM 2.0 was most 

frequently applied to consumer active matrix LCDs such as notebooks, monitors, TVs, 

etc., but its methods and principles extended across industrial, military/aerospace, and 

medical applications, regardless of display technology, with only occasional caveats. 

 

48 Communication, Mark Fihn, November 5, 2010.
17

3 Economic Analysis Framework

3.1 Documentary Standards as “Infratechnology”

Quantitatively assessing the economic impact of documentary standards requires their 

conceptualization in a manner that is conducive to economic analysis, that is, in terms of 

microeconomic categories and logic of market failures.

49

A recent NIST publication categorizes documentary standards according to a schema that 

is familiar to participants in documentary standards development processes. The 

publication categorizes documentary standards by type (terminology standards, product 

standards, service standards, data provision standards,) and describes the market 

functions that the types of standards perform.50

Assessing economic impact requires further categorizing these market functions in 

microeconomic terms: First, what are the economic concepts that capture these market 

functions? Second, to what extent do markets need non-market assistance (the assistance 

of standards-making organizations and the organizations, like NIST, that provide the 

technical basis for standards) in performing these market functions? Both questions are 

addressed in the literature concerning market failures.51 A categorization of standards that 

stresses their economic function (rows) is combined with a categorization of documentary 

standards by type (columns) to produce the documentary standard‘s economic impact 

matrix shown in Table 3-1. The nature of the chief economic benefits (cells) associated 

with each documentary standards economic function and type is indicated in the table.

Not all types of standard perform all economic functions but some economic functions 

are performed by all types of standard. Quality and reliability assessment, for example, is 

an economic function primarily associated with testing standards, so only the intersecting 

cell identifies the economic benefits associated with testing types of standards. On the 

 

49 For a thorough discussion of ―market failures‖ and their causes, see Gregory Tassey, The Economics of 

R&D Policy, Quorum Books, 1997; John Roberts, The Modern Firm, Oxford University Press, 2004; and 

Oliver Williamson, The Economic Institutions of Capitalism, Free Press, 1985.

50 The ABCs of Standards, (NISTIR 7614), August 2009. For example, terminology standards permit 

parties to a transaction to use a common language; product and service standards establish fitness for use; 

and process standards help establish the compatibility of products or systems at their points of 

interconnection. 

51 While a large literature exists addressing problems of market failures, little of it addresses the ―nuts and 

bolts‖ of performing impact assessments. Sparser still is the literature that addresses market failures as they 

relate to documentary standards and standards-related activities in a manner that clearly informs the 

practice of impact assessments. The economic literature useful for evaluation purposes includes, Gregory 

Tassey, The Roles and Economic Impacts of Technology Infrastructure, mimeo, March, 2008; 

―Standardization of Technology-Based Markets,‖ Research Policy, 29, 2000, pp. 587-602; Stephen Martin 

and John Scott, ―The Nature of Innovation Market Failure and the Design of Public Support for Private 

Innovation,‖ Research Policy, 29, 2000, pp. 437-447; David Leech, Albert Link, and John Scott, The 

Economics of a Technology-Based Service Sector, NIST Planning Report, 98-2, 1998; Gregory Tassey, The 

Economics of R&D Policy, 1997; and Oliver Williamson, The Economic Institutions of Capitalism, Free 

Press, 1985.
18

other hand, all types of standards are believed to foster variety reduction, so all the 

intersecting cell in the variety reduction row are associated with the economic benefits of 

economies of scale for producers and reduced search costs for buyers.

Table 3-1. Documentary Standards Economic Benefit Matrix

Documentary Standards Type

Economic

Function

Terminology, 

Service, and 

Product 

Standards

Data 

Standards

Interface 

Standards

Testing 

Standards

Harmonized 

Standards

Provision of 

measurementrelated 

information

• Lower 

transaction costs

• R&D 

efficiencies

•Quality/Process 

control 

improvements

• Lower 

transaction costs

• R&D 

efficiencies

•Quality/Process 

control 

improvements

• Lower 

transaction costs

• R&D 

efficiencies

•Quality/Process 

control 

improvements

Compatibility/

Interoperability

• Increased 

innovation at the 

component level

• Optimized 

system design

• Increased 

innovation at 

the component 

level

• Optimized 

system design

Quality/ 

Reliability

Assessment

• New basis for 

price/quality 

competition

• Lower costs 

and/or higher 

quality products 

and services

Variety 

Reduction

• Economies of 

scale

• Reduced buyer 

search cost

• Economies of 

scale

• Reduced buyer 

search cost

• Economies of 

scale

• Reduced buyer 

search cost

• Economies of 

scale

• Reduced buyer 

search cost

• Economies of 

scale

• Reduced buyer 

search cost

The term ―infratechnology‖ was coined to describe the ―technical tools‖ that are 

ubiquitous in a high-tech economy and provide substantive foundation for many of the 

documentary standard types identified in Table 3-1. Infratechnologies include 

measurement and test methods, standard artifacts (such standard references materials or 

weights and measures artifacts), scientific and engineering reference databases, process 

models, and the technical basis for both physical and functional interfaces between the 

components of systems technologies.52

For infratechnolgies to effectively perform their functions, they must be shared. For 

example, for a buyer and seller to agree that a product conveyed by the seller meets the 

buyer‘s performance specifications, the buyer and the seller must agree on how to

measure performance. For a buyer to measure the relative performance of a range of 

competing sellers, the buyer must have a basis for making ―apples-to-apples‖ 

comparisons of the competing sellers‘ performance claims. To take advantage of the 

competitive process, the basis for apples-to-apples measurement must be equally 

 

52 Tassey, Ibid., 1997.
19

available to all sellers and buyers. In the jargon of economics, infratechnologies have 

high ―public-good‖ content because buyers and sellers share the know-how to evaluate 

products.

53

NIST personnel serve at least two important economic functions when they are engaged 

in the documentary standards development process, an “honest broker” function and a 

“conduit” function.

A documentary standard embodying an honest broker function is one in which NIST‘s

participation serves to reduce the cost of reaching an industry consensus on some facet of 

a standard because NIST is viewed as committed to the highest standards of metrology 

and devoid of any proprietary interest. Standards are often regarded as efficiency 

reducing to the extent that standard-making organizations are unduly influenced by a 

subset of companies trying to co-opt the consensus-making process for their own gain at 

the expense of other member companies or the consumers of their products. The potential 

for NIST personnel to play important coordinating and support roles should enhance the 

efficiency of the standards making process. 

In addition to the honest broker function, documentary standards in which NIST 

personnel are involved are presumed to serve as a conduit for the effective ―traceability‖

of international standards of physical measurement through national representations of 

international standards (maintained by NIST), all the way to the working standards of 

measurement employed in industry. From there, ultimately, documentary standards affect 

the transactions among buyers, sellers, and competitors. In this ―conduit‖ role, 

documentary standards form a strong ―bridge‖ that connects international agreements 

about measurement standards to a ―local,‖ product market-specific context. That bridge 

will be strongest — and NIST‘s ―conduit function‖ most effective — where documentary 

standards are based on the most technically-defensible, widely-recognized

infratechnologies.

3.2 An Economic Perspective on the Genesis of a the FPDM 

Documentary standards are vehicles for the transmission of ―infratechnology,‖ 

measurement practices and know-how that must be shared to be useful. If two parties —

for example, a buyer and a seller — are going to agree on the measurement criteria by 

which a product or process will be assessed, they must agree on and share the underlying 

measurement practices for the assessment to be performed effectively. 

 

53 ―Public‖ in this context refers to the availability of, or access to, the basis for making measurements to 

confirm performance, not to the distinction between social estates or sectors, as when reference is made to 

public and private sectors. The private sector can provide public goods. See, Harold Demsetz, ―The Private 

Production of Public Goods,‖ Journal of Law and Economics, Vol. 13, October 1970, pp. 293-306. While 

the private sector can provide public goods like infratechnologies, their provision, as indicated in Figure 1, 

is regarded as one of the barriers to innovation; a barrier that public-private barrier mitigation strategies —

such as consensus-based documentary standards development efforts — are intended to address. So, the 

public sector often plays a significant role in the provision of goods and services with high ―public‖ content 

even though, logically speaking, it need not.
20

Economic logic suggests that where multiple parties must share measurement know-how, 

the incentive to invest in the development and dissemination of that information will 

likely confront free-rider problems. In addition, policing problems are likely to arise 

when complex technologies make the identification of product quality dimensions hard to 

specify and measure uniformly, all the more when parties act with guile to advance the 

position of their products at the expense of others who are attempting to come to a 

multiparty arrangement.54 When the number of participants in a business arrangement is 

small, private negotiations (contracts) can address the free rider problem as well as the 

measurement specification and policing issues. When the number of parties is large and 

varied, the cost of developing and policing satisfactory contracts could become

prohibitive and some form of collective decision-making (often involving a government 

institution) may be sought to keep transaction costs within acceptable boundaries.55

Standards Development Organizations (SDOs) often arise to address measurement issues 

such as these.56

Requirements for new measurement technologies continuously emerge as product and 

process technologies develop. To the extent that new product and process technologies 

emerge mostly from private sector initiatives, the measurement technologies used to 

demonstrate their performance are initially developed internally and communicated, for 

example, between organizations within the same firm or between corporate partners 

engaged in collaborative arrangements. To the extent that private sector firms guard their 

new product and process developments from competitors, they likely develop and guard 

the associated measurement technologies. As discussed in section 3.1, measurement 

technologies (infratechnologies) have the greatest economic impact to society if they are 

widely disseminated. For many industries, historically, this dissemination was thought to 

have come about by the gradual change in industry structure. In the case of the FPD 

industry, the traditional explanation is probably insufficient. Rather, NIST, working with 

industry through the FPDM Workgroup, played an important role in disseminating FPD 

infratechnologies. 

Traditionally, the distinction between product and process technologies has been 

important for explaining how the structure of an industry evolves over time. Accordingly, 

in the early stages of a product life cycle, innovative product features are regarded as 

more important to buyers of new products than manufacturing process control and cost 

 

54 E. F. Kelley, "What Do the Specifications Mean?" Proceedings of the First Americas Display 

Engineering and Applications Conference (ADEAC 2004), Society for Information Display, Ft. Worth, 

TX, USA, pp. 15-18, October 26, 2004.

55 In the context of oligopoly coordination it has been suggested that the difficulty of coordination rises 

nearly exponentially with the number of parties involved. Unless there is a central coordinator, each firm 

must tacitly or overtly communicate with every other over matters of concern. The number of two-way 

communications required is given by the combinatorial expression N(N-1)/2. With two participates the 

number of two-way communications is 1. With six participants it rises to 15. See, F.M Scherer, Industrial 

Market Structure and Economic Performance, 1980, pp.199-200.

56 The focus of this report is measurement technology standards. A similar logic applies to other kinds of 

standards, such as terminological standards, data standards, interface standards, testing standards, and 

harmonized standards. For a discussion of the various kinds of standards see, The ABCs of Standards,

(NISTIR 7614), August 2009.
21

control. In time, when the scale of the industry expands, and competitors enter the 

market, cost control and process technology come to dominate firms‘ strategies, they spin 

off what are now well-controlled internal processes to manufacturing process specialists

who compete with other process specialists and drive down process costs. This is one

view of how process technology (and undifferentiated measurement infratechnology)

comes to be disseminated.57,58

However, this product-process distinction is inadequate for understanding the evolution 

of the FPD industry and the role of VESA‘s FPDM Workgroup in that process. Rather, a 

new perspective explicitly emphasizes the central role of measurement know-how, 

especially in technology-driven industries. This new perspective asserts that enterprises

are not free to dis-integrate and re-integrate competitive capabilities on the basis of the 

maturity of their process technologies if the interface between stages of value-added in an 

industry is characterized by ―unstructured technological dialogue.‖

59 According to this 

perspective, unstructured dialog between stages or elements of the value-added chain 

must be transformed into ―structured dialog‖ for those value chain elements to be 

deployed strategically (either as independent spin-offs or as internal specialized or cospecialized assets in support of customers). For structured dialog to occur, three 

conditions must be met: 

• The customer that procures or uses an input must understand and be able 

to specify to its supplier which attributes or parameters of the product or 

service must be provided, and to what tolerances; 

• Metrics for those attributes must exist, and the technology to measure 

those attributes must be available, reliable and unambiguous; and

• The procuring company must understand the interactions or 

interdependencies between the attributes of what is provided and the 

performance of the system in which the procurer will use it. If there is 

 

57 See, G. Stigler, ―The Division of Labor is Limited by the Extent of the Market,‖ The Organization of 

Industry, University of Chicago Press, 1968, pp. 129-141. 

58 Improving on Stigler‘s conceptualization of how business strategy and industry structure change over the 

product life cycle, David Teece distinguished between tight and loose ―appropriability regimens‖ (that 

make it easier or harder to control the leakage of product innovations) and emphasized how tacit product 

and process know-how make it hard for innovators to communicate with collaborators. As an alternative to 

the view that process technologies are typically gottern under control and spun off, Teece called attention to 

the role that deploying specialized and co-specialized process technology assets (such as specialized repair 

facilities) could have on profitability once innovative product designs were stablized. (See, D. Teece, 

―Profiting from technological innovation: Implications for integration, collaboration, licensing and public 

policy,‖ Research Policy, 15 (1986) 285-305.) Teece envisioned a result similar to that envisioned by 

Stigler: ―Islands of specialized capital ... [in a] sea of general purpose manufacturing equipment.‖ Whether 

these specialized assets include measurement technology on Teece‘s account is hard to say. How tightly 

measurement technologies are tied to the product and process improvements they authenticate is an issue to 

be discussed momentarily.

59 Clayton M. Christensen, Matt Verlinden, and George Westerman, ―Disruption, Disintegration, and the 

Dissipation of Differentiability,‖ Industrial and Corporate Change, Vol. 11, No. 5, 2002, pp. 955-993.
22

any variation in what is provided, the procurer needs to understand 

how, when and why it will affect the performance of the system.‖ 

(Emphasis added.) 60

The traditional distinction between product and process technology is insufficient for 

understanding this formulation. Only by introducing measurement technology 

(infratechnology) as a separate and essential complement to the traditional productprocess distinction can unstructured technological dialog be transformed into structured 

technological dialog. This three-part categorization of technology — productmeasurement-process61 — is necessary for understanding the genesis of the FPDM in the 

context of the fast-paced evolution of the FPD industry in which product, measurement, 

and process technology development were tightly bound.

Some analysts claim that the FPD industry is exemplary of new globally competitive, 

knowledge-driven manufacturing industry. In such industries, price competition occurs 

before the technology settles down to a dominant design so that companies face relentless 

pressures to reduce costs, while investments in R&D and updating of plant, equipment, 

materials, and processes continue to rise.62 An additional attribute of this early phase of a 

technology‘s life cycle is the extra costs incurred in assuring customers that the 

performance of competing designs is being articulated and measured accurately. 

Otherwise, the additional substantial transaction costs can create a competitive 

disadvantage. 

In the FPD industry the fast pace of dialectical change among product, measurement, and 

process technologies might not have allowed the vertical dis-integration process to 

transpire as expected because the ―feedback from a continuous person-to-person and 

person-to-equipment interaction at the operational level play[ed] a vital role in 

accumulating and transmitting knowledge for new equipment generations.‖63 Making the 

point somewhat differently, and retrospectively, computer notebook industry participants 

observed,

In the early years, standards were sorely needed [but unavailable] to 

reconcile the emerging requirements for new display applications with 

new design specifications. This was largely accomplished by countless 

proprietary requirements specs exchanged between panel makers and the 

notebook contract manufacturers. Early panel specifications referenced 

VESA timing standards (originally established for CRT monitors) and 

Japanese optical measurement methods (composed [merely] of a page or 

two in the back of the spec). There was no urgency to optimize 

relationships between price, volume, lead time, quality, and 

 

60 Ibid., p. 958

61 ―Infratechnology‖ could be substituted for ―measurement‖ in this triad but the emphasis here is on the 

transformation from ―nascent infratechnology‖ (proprietary measurement technology) to ―effective 

infratechnology‖ (commonly used (i.e., standardized) measurement infratechnology. 

62 Thomas Murtha, Stefanie Lenway, and Jeffery Hart, [hereafter, Murtha, et. al.] Managing New Industry 

Creation: Global Knowledge Formation and Entrepreneurship in High Technology, 2001, pp. 8-9.

63 Murtha, et. al., op. cit., p.113.
23

interchangeability; ODM-Supplier partners where just trying to get the flat 

panel technology to work, fit in the lid, and not look too bad – all while 

keeping up with increasing volume demands.

Today, industry standards such as the VESA Notebook Standard Panel 

standards (defining timing, electrical interface, and mechanical outline) 

and VESA Flat Panel Display Measurement standard (defining 

measurement methods) are critical to the display industry because they 

enable discussion of display electrical, optical, and mechanical attributes 

in a common language up and down the food chain. [Emphasis added.] 

These standards played key roles in quantifying the current definition of 

panel quality.64,65

As late as 2008 analysts were observing that a high percentage of ―in-house‖ value added

(above 80%) still dominated FPD manufacturers and OEMs in some important sectors.‖66

So the traditional ―process specialization‖ explanations of industry evolution and growth 

didn‘t explain the dynamics of the FPD industry. Rather, the ―discussion‖ of technical 

attributes enabled by the FPDM is a clear example of unstructured technological dialog

being transformed into structured technological dialog. In the FPD industry it wasn‘t 

vertical dis-integration of the industry, and process specialization, that opened up 

opportunities for industry growth and development. Other forces were at work that 

centrally involved measurement technology: the explosion of FPD applications, on the 

one hand, and the creation of NIST‘s FPDL and the SDO consensus-based standardsmaking process on the other hand.

First, the range of industries finding applications for relatively large FPDs was expanding 

and these were industries into which the FPD manufacturers had not generally integrated 

forward. Industry analysts contend that the companies holding the preponderance of 

market share for products that incorporated the most advanced displays were not 

Japanese. Three of the global top-four notebook sellers (typically Toshiba, IBM, Dell, 

and Compaq) were consistently U.S.-based. Once the notebook market emerged, in the 

U.S. especially, it evolved rapidly. That evolution was driven, on the demand side, by the 

increasing size and sophistication of a mobile workforce demanding laptops, rapid 

 

64 P.A. Downen, S. Peana, J. Yasukawa, E. Munteanu, and J. Conway [hereafter, Downen, et. al.], 

―Meeting the Product Qualification Needs of a Rapidly Growing Industry,‖ ADEAC 05/Society for 

Information Display, 2005.

65 According to a participant in the development of the Standard Panels Working Group (SPWG) standard, 

the SPWG had already developed the basis for the VESA Notebook standard and the VESA Notebook 

standard, as such, was not as widely adopted as the SPWG standard. Personal communication with Mark 

Fihn, November 11, 2010.

66 Brian Chen, ―2008 LCD TV Panel Makers‘ Production Plan and Panel Roadmap,‖ Display Search,‖ 

January 16, 2008, especially, ‖LCD TV Outsourcing Status & Future Outlook,‖ p. 47, and ―LCD TV Brand 

In-House vs. OEM,‖ p. 48.
24

progress in personal computing technology, the emergence of the World Wide Web, the 

spread of high-speed Internet access, and the rapid advancement of the digital 

infrastructure.67

What was nascent in the early 1990s was clear by 2005. By then, the flat panel display 

industry served a wide variety of applications with display diagonal sizes from 1 inch to 

80 inches. The five largest shares, by revenue, were desktop monitors (31%), mobile

telephones (18%), LCD televisions (14%), notebook PCs (12%), personal digital 

assistants (7%). Other significant applications included pagers, MP3 players, home 

appliances, games and digital cameras, calculators, automotive applications, and toys.68

Many professional and consumer products now depend on electronic displays. 

Looking back, a market participant made the following observation:

[T]he changing notebook market has steadily shifted ownership of the 

definition of display quality from the original design manufacturer (ODM) 

– read as ―notebook maker‖ – to the collective customer comprised of 

corporate, governmental, academic, and private organizations as well as 

individual consumers.(Emphasis added.) 69

Driving that shift in ownership of the definition of display quality were demand-side 

factors that came into focus over a decade:

For the flat panel display industry, the notebook computer has been the 

undisputed king of the killer applications having fueled research and 

development that is now reemerging in all mobile devices, flat panel 

monitors, and LCD TVs. ... The notebook flat panel display has become a 

commodity item characterized by high volume production, a customerdriven quality threshold, and more price than performance discrimination. 

One leading notebook maker, for example, utilizes eight different 

AMLCD panel vendors in Korea, Japan, and Taiwan to feed their 

notebook production demand.

 

67 Murtha, et. al., op. cit., p. 4.

68 David Hsieh, ―Flat Panel Display Market Outlook,‖ Display Search Company, November 5, 2005, p. 8.

69 Phil Downen, ―A Closer Look at Flat Panel-Display Measurement Standards and Trends.‖ Information 

Display, No, 1, 2006. Close market observers differ on the specifics. Mark Fihn, for example, argues that 

until about 1990, display quality for the notebook market was defined primarily by the LCD maker; only 

from1990 until about 2000 was it defined primarily by the notebook OEM; and more recently it has been 

primarily defined by the notebook manufacturer (Quanta, Compal, Wistron, Pegatron, etc), who built the 

vast majority of notebooks for their notebook OEM customers. Personal communication with Mark Fihn,

November 18, 2010. Regardless of these specifics, Downen‘s ―collective consumer‖ emphasizes the 

―demand side‖ of the dialectical relationship between the supplier and the buyer wherever it occurs in the 

supply chain. The argument in this section is that the possibility of such a shift was enabled by what 

Christenson calls ―structured dialog‖; that the FPDM reflects the three conditions that enabled structured 

dialog to occur; and that the role of NIST‘s FPDL (as ―honest broker‖ and ―conduit‖ of international 

standards of display metrology) was fundamental to that process.
25

Commodity status, however, in no way guarantees that the process of 

designing, producing, integrating and delivering flat panel display 

products is optimized. The panel supply chain alone is complex and 

fraught with challenges for ODMs (original design manufacturer – read 

as “notebook maker”) and panel maker alike. Concerns over panel costs 

and time to market are just now supplanting yesterday‘s worries about 

viewing angle, color gamut, response time, form-fit-function, and 

reliability. Still, managing panel quality cannot be taken for granted; it 

requires a portable and scalable solution that is owned by both the ODM 

and display maker.70

The supply side of the flat panel market was expanding rapidly as well as, first, Korean 

companies, LG and Samsung, joined the Japanese as key global suppliers in 1993, and 

then Taiwanese companies, Acer Display Technology (ADT – now AUO), Chi Mei 

Optoelectronics (CMO), Chunghwa Picture Tubes (CPT), HannStar, and Quanta Display 

Incorporated (QDI), joined the ranks of global suppliers from 1998 to 2001.

71

The second major enabler of the FPD industry take-off was that NIST‘s FPDL came on 

the scene. Its participation in the VESA FPDM Workgroup allowed all three conditions 

for structured dialog to be fulfilled. Arguably, before NIST set up the FPDL:

• The ―customer that procures‖ was unable to specify and compare 

parameters at a reasonable cost

• ―Technology to measure‖ was unavailable, unreliable, and ambiguous; 

and

• ―Variation in what was provided‖ by various FPD producers limited the 

procurers‘ abilities to understand interdependencies between attributes 

and performance.

As long as these measurement issues were unresolved, attempts to reach consensus in so 

technically challenging and complicated a field would be bogged down in 

―specsmanship‖: ―reporting a measured value that is deliberately intended to mislead or 

where the display is measured in a configuration in which it would never be used but 

provides better reporting values. ... to hide a deficiency for competitive purposes.‖

72

To summarize, the ―take off‖ of the flat panel industry that followed the initial 

development of the industry‘s 14-inch notebook screen ―killer app,‖ led to a rapid 

expansion of applications in industries into which, by and large, FPD manufacturers had 

not integrated. The measurement technology required by FPD applications developers 

could not be liberated from its proprietary sources within the integrated FPD 

manufacturers — where, due to the fast-paced, dialectical nature of product-process-

 

70 Downen, et. al., op. cit., 2005

71 Ibid.

72 E. F. Kelley, "What Do the Specifications Mean?" Proceedings of the First Americas Display 

Engineering and Applications Conference (ADEAC 2004), Society for Information Display, Ft. Worth, TX, 

USA, pp. 15-18, October 26, 2004.
26

measurement technology development patterns it had the quality of ―unstructured 

technological dialogue‖ — until it could be translated into the ―structured dialog‖ that 

would allow FPD users to compare FPD parameters at a reasonable cost, implement 

unambiguous measurements, and understand the relationship between the FPDs‘

attributes and the performance of the users‘ applications. NIST‘s FPDL proved essential 

to achieving the transition as reflected in VESA‘s FPDM standard.

3.3 The Economic Implications of FPDM Workgroup Inputs and Outputs

The FPDM was initially created to solve a set of problems associated with the 

characterization, specification, qualification, and assessment of flat panel displays for a 

broad range of uses that could not be adequately addressed by a range of existing related 

standards. The information required for the resolution of these problems was a primary 

output of the Video Electronics Standards Association (VESA) FPDM Workgroup.73

This primary output took at least two forms: the codified measurement infratechnology 

represented by the FPDM itself, on the one hand, and metrology equipment designs by 

NIST‘s FPDL, largely in support of the FPDM Workgroup on the other hand. 

Also, due in large part to NIST‘s involvement in the FPDM Workgroup, the standard 

development process was more effective and efficient than it otherwise would have been.

This was a secondary output of the FPDM Workgroup configuration. This ―process 

efficiency‖ took two forms: the time it took for the FPDM Workgroup and VESA (as an 

SDO) to reach consensus was significantly reduced; and other SDO‘s process times were 

reduced to the extent that the FPDM could be incorporated into their documentary 

standards. These SDO process efficiencies are a secondary output of the configuration of 

the FPDM Workgroup.

Finally, as the discussion in section 3.3 indicates, the FPDM also represented the 

culmination of a process that played an important part in the take-off of the FPD industry 

and the utilization of FPDs in a wide range of applications. The FPDM represented the 

transformation from ―unstructured technological dialogue‖ to a ―structured dialog‖ 

between manufacturers and users. This was a significant, tertiary output of the FPDM 

Workgroup‘s efforts. This last aspect of the FPDM Workgroup‘s efforts was thoroughly 

discussed in section 2.2 above. The following four sub-sections (3.4.1-3.4.4) further 

describe the primary and secondary outputs of the FPDM Workgroups efforts.

3.3.1 FPDM: Codified Measurement Know-How

The FPDM is a document consisting of approximately 300 pages of measurement 

procedures, discussions of measurement problems and difficult concepts, an approach 

and short template for reporting measurement results, and conventions for naming and 

reporting parameters that characterize flat panel screens. Flat panel measurement 

 

73 In 2007, the VESA FPDM Workgroup was reorganized into the International Committee on Display 

Metrology (ICDM) within the Society for Information Display (SID), a professional organization 

representing technical and business disciplines that relate to display research, design, manufacturing, 

applications, marketing, and sales.
27

technology is codified and transferred to users of the document. The first order utility of 

the FPDM is associated with the metrology cost saving experienced by manufacturers 

and integrators of flat panel displays. 

Prior to the development of FPDM, a producer or buyer of flat panel displays (FPDs) 

would consult any (or all) of a number of existing standards (depending on the specific 

application)— ISO 9241 (Parts 3, 7, 8) and ISO 13406 (draft 2); ANSI HSF-100 (1988) 

and IT7.215 (1992); EIA TEB (27) and TEP (105); VESA Display Specifications and 

Test Procedures (for CRTs); NIDL‘s Procedures for Evaluation and Reporting the 

Capabilities of High Performance Display Monitors for Imagery Applications; SAE ARP 

1782 and ARP 4260; MRP 1990:8 (1990:10);USAF AFGS 87213A; and IEC SC 

47C.74,75 Utilizing any or all of these non-FPDM standards presented the following 

challenges for buyers and suppliers of specific FPDs:

• Alternative standards are ―requirements based‖ not metrology based. In 

other words, they describe what the FPD is to do, not how it is to be 

tested or whether it succeeds or not.76

• Alternative assorted standards were not coordinated so that it would take 

time to figure out how they were/were not aligned for any specific 

purpose.

• At best they provided a partial solution (perhaps 60%) to the 

characterization, specification, or qualification task and, therefore, 

required the buyer or supplier to develop consolidated and integrated 

proprietary specifications/qualifications, a very time-consuming process 

without FPDM.

• In many cases, metrology was specified, but as the writers were not 

metrology experts, the methods were inadequate; however, few were 

aware of the inadequacy.

• Addressed old technology (CRTs) and assumed measurement techniques 

and procedures transferred well to new technologies (e.g., Liquid Crystal 

Displays, LCDs) – which proved an erroneous assumption.

According to NIST researchers, the quality of FPD metrology was extremely low when 

the project was launched. The extent of low product quality, product failure, and rework, 

while unknown, was most likely substantial.77

 

74 See, Edward F. Kelley, George R. Jones, Paul A. Boynton, Michael D. Grote, and Dennis J. Bechis. ‖A 

Survey of the Components of Display Measurement Standards,‖ Journal of the Society for Information 

Display, Vol. 3, No. 4, December 1995, pp. 219-222.

75 ISO – International Standards Organization, ANSI – American National Standards Institute, EIA –

Electronic Industries Association, EC – International Electrotechnical Commission, SAE – Society of 

Automotive Engineers, MRP – Swedish National Board of Measurement and Testing, NIDL - National 

Information Display Laboratory (a private sector laboratory associated with Sarnoff Laboratories), USAF –

United States Air Force.

76 Procurement standards and product-acceptance standards are not substitutes..
28

An anecdote provided by an industry participant in the early development of the FPDM 

captures a primary purpose of this documentary standards activity:

In the early days of developing a product specification, it took weeks and 

weeks of discussion and discovery to figure out how to make 

measurements, especially for  ̳contentious parameters‘ such as  ̳set up 

conditions‘ (on which the FPDM focuses — See FPDM 2.0, section 301). 

Part of this discussion is ongoing discussions with either the customer —

for whom the display was being made — or the display manufacturer. The 

more contentious the parameter the longer the discovery and negotiations 

process took. FPDM changed that. It provides a unified  ̳buffet‘ of 

measurements and set up conditions that anyone can use and combine in 

any way they need but with the underlying metrology clearly established.

The utility of the FPDM changes subtly depending on a firm‘s position in the industry 

value chain (discussed in section 3.5 below). The FPDM is used by original equipment 

manufacturers (OEMs) — providers of systems or subsystems that incorporate FPDs —

when the OEMs have a requirement that needs to be specified in a subcontract. They also 

employ the FPDM as the basis for qualifying competing displays to ensure that the 

displays specified for use in their products are neither over- or under-specified (both of 

which entail unwanted costs). Display and display component manufacturers use FPDM

to characterize their products and to specify their performance unambiguously. For 

optical measurement equipment makers, the FPDM has been a valuable roadmap for 

R&D activities over the past 10 years. In the words of one such manufacturer, ―the 

structure, clarity, and pragmatism of the standard has served all of us in the equipment 

business by defining a set of meaningful measurements (with back-up discussions on 

methodology and metrology) that could be implemented with practical and robust 

solutions. ... We've promulgated the standard in our sales discussions with panel makers, 

brand name leaders, [display manufacturers], and component makers.‖ The FPDM has 

also helped equipment manufacturers to explain instrument limitations and good 

metrology practices to customers.78,79

3.3.2 FPDM-Related Metrology Equipment Design

In support of the FPDM Workgroup, NIST researchers also developed and, through

professional publications, disseminated the design of several items used in the process of 

FPD measurement, including: 

 

77 P. A. Boynton and E. F. Kelley, "Measuring the Contrast Ratio of Displays: It's Not Trivial." Information 

Display, Vol. 12, No. 11, pp. 24-27, November 1996; and G. Jones, E. Kelley, T. Germer, "Specular and 

Diffuse Reflection Measurements of Electronic Displays," Society for Information Display International 

Symposium Digest of Technical Papers, Vol. 27, San Diego, CA, pp. 203-206, May 12-17, 1996.

78 Personal communication with Paul Boynton (NIST), April 18, 2010. 

79 In addition to the projects undertaken in support of the FPDM Workgroup, NIST conducted metrology 

projects focused on specific applications, such as projection displays, medical displays, first-responder 

imaging cameras, micro-displays, and sunlight readability. These projects will inform the next generation 

display measurement standard to be published by the SID‘s ICDM.
29

• Stray-Light Frustum

• Stray-Light Elimination Tube (SLET) 

• Sampling Sphere, and

• Test Patterns.

A stray-light frustum is a specially designed, three-dimensional black plastic cone with 

the apex cut off, used to make accurate measurements of luminance by reducing glare 

effects during display testing. The frustum fits within a stray-light elimination tube 

(SLET), a device that measures luminance by rejecting stray-light corruption even in 

well-lit rooms.80 A sampling sphere (or integrating sphere) is a device that allows the 

power of an optical source to be accurately measured irrespective of the direction of the 

source.81 Finally, test patterns are image patterns with features of various sizes that serve 

in place of image artifacts. Some pattern features are much larger (and thus easier to 

measure) than typical noise features, others scaling down to small clusters of pixels. The 

larger pattern features are easier to measure and errors that appear when measuring larger 

features are likely to be even worse with smaller features. With a range of sizes of 

features, it can be determined how measurement errors scale with feature size.82

3.3.3 SDO Process Efficiencies

The FPDM Workgroup developed a set of criteria for assessing the quality of a 

documentary standard. These criteria are identified in Table 3-2, The FPDM strove to 

meet these criteria which were considered to be an extremely high hurdle of quality. 

Table 3-2. Quality Dimensions for Documentary Standards

Feature Definition

Reproducible Everyone can get the same results on the same display 

using appropriate instrumentation. 

Robust Insensitive to small changes in the measurement apparatus 

that will affect the ease with which reproducibility is 

attained.

Unambiguous The method is clearly stated and easily understood.

Important details that are required for success are not left 

out. 

Extensible Applicable to as many different technologies as possible 

permitting inter-comparisons of technologies.

Distinct The name of a measurement method must be chosen so that 

it is not confused with another metric.

Honest The measurement method is not devised to hide an obvious 

deficiency; redefining familiar terms to cloak a problem.

Accommodating Enables as broad a range of apparatus as possible.

 

80 John Roberts and Edward Kelley, ―Measurements of Static Noise in Display Images,‖ Flat Panel Display 

Technology and Display Metrology II, Edward Kelley and Tolis Voutsas, Editors, Proceedings of SPIE 

Vol. 4295 (2001).

81 Encyclopedia of Laser Physics and Technology, http://www.rp-photonics.com/integrating_spheres.html, 

May 29, 2011.

82 Roberts and Kelley, Op. Cit.
30

Table 3.2, continued

Feature Definition

Accessible Requiring the use of highly specialized, or otherwise 

arcane, apparatus or methods would be avoided unless it is 

necessary (e.g., for people who influence written standards 

in order to sell their apparatus when it is not necessary to 

do so).

Simple Procedures should be made as uncomplicated as possible, 

avoiding deliberate obscuration from elitism or exploitation 

(e.g., deliberately making the standard so difficult to use 

that only a few experts can use it).

Meaningful Properly captures the visual experience for task and 

environment. Measuring what the eye appreciates should 

not be sacrificed for some related esoteric measurement 

method of limited use.

The FPDM Workgroup produced a standard that conformed to these exacting dimensions 

of quality, and, because they employed NIST‘s special technical expertise in compiling 

industry technical input, the Workgroup avoided the ―specsmanship‖ that presumably 

slows the process of standards development. (See section 2.2 above.) NIST‘s expertise 

was used to make technical corrections to submissions from other FPDM workgroup 

members. NIST staff analyzed, tested, reviewed, composed, and edited the FPDM drafts 

under the auspices of NIST‘s Flat Panel Display Laboratory.83

In addition to addressing problems concerning the characterization, specification, 

qualification, and assessment of FPDs for display manufacturers, their suppliers, and their 

users, the FPDM was also used as source language for documentary standards being 

developed by other standards organizations concerned with related equipment. In other 

cases, the FPDM was referenced by other standards.84 This diffusion of FPDM 

measurement technology represents another FPDM Workgroup outcome, one that did for 

other standards organizations what it did for display manufacturers and their value chain 

participants. In the words of an industry expert, ―No other [similar] standard was so 

profound; so wide-ranging.‖85,86

 

83 These corrections could be simple technical errors or they could be the result of ―specsmanship.‖ See, E. 

F. Kelley, "Understanding Specsmanship," 2007 Display Applications Conference, Society for Information 

Display, Invited Paper 7.1, San Francisco, CA, October 23-25, 2007.

84 ISO, IEC, SAE, ITU all reference or use material from the FPDM. Personal communication with Paul 

Boynton, May 18, 2010.

85 Communication with a respected industry expert wishing to remain anonymous, April 6, 2010.

86 The Softcopy Exploitation Display Hardware Performance Standard (SEDHPS), Version 2.1 (28 August 

2006), is an example of a standard that references FPDMS. This standard, used across Federal Agencies, is 

published by the National-Geospatial Intelligence Agency (NGA), Image Quality and Utility (IQ&U) 

Program. When the need arises, a manufacturer responsible for providing equipment to a Federal agency 

will refer to this standard in its subcontracts. Another example of this documentary standard diffusion 

function can be found in the medical display community. According to an FDA representative, the FPDMS 

directly affected recommendations from the AAPM (a professional organization) and is incorporated as a 

reference in the IEC Standard Group on Medical Image Display (TC62BWG36). 
31

3.4 Affected Organizations: Industry Structure

The FPDM Workgroup‘s outputs effected outcomes across the tiers of the FPD industry. 

Figure 3-1 depicts the structure of the FPD industry value chain showing examples of 

producers and consumers within the tiers.87

Figure 3-1. Flat Panel Display Industry Value Chain

The first-order beneficiaries of the FPDM are likely to be the measurement equipment 

manufacturers and testing laboratories (both vendor testing laboratories as well as testing 

capabilities internal to, especially, display manufacturers, and OEMs).88 The ubiquity of 

metrology equipment and testing services is depicted as the vertical elements to the left in 

Figure 3-1. For example, the notebook PC OEMs that drove the take-off of the FPD 

industry in the mid-1990s, discussed in section 2.1 above, maintained sophisticated 

internal testing laboratories that scrambled to qualify multiple display vendors to meet 

rapidly rising demand. Similarly today, sophisticated displays for national securityrelated users employ sophisticated internal capabilities to specify and qualify vendors of 

high-quality FPDs. 

 

87 ―Component manufacturers‖ are shown for the sake of completeness. Interviews with FPD industry 

representatives suggest that the FPDM has little direct utility for component manufacturers.

88 These industry segments themselves may exhibit a microstructure but this information could not be 

ascertained from the survey conducted for this assessment nor does it appear available from available 

market intelligence studies. 
32

The scale and shape of testing equipment and services, itself a superset of the smaller 

market for metrology equipment and services, is hard to ascertain. For 2009-2010, an 

estimate of TFT-LCD display manufacturers‘ revenues was $90 billion worldwide.89

A respected market intelligence analyst estimates that for the last several years the total 

market for TFT-LCD manufacturing equipment market has fluctuated around $8-10 

billion annually. Analysts say that ―yield management services‖ account for less than ten 

percent of that, and perhaps one percent of that could be allocated to testing and 

metrology. This rule-of-thumb estimate, coupled with several market niche estimates 

received from survey respondents, suggest that the current market for FPD metrologyrelated equipment and services could be in the neighborhood of $10 million annually.90

 

89 http://www.displaysearch.com/cps/rde/xchg/displaysearch/hs.xsl/resources_paneltrack.asp, November 4, 

2010.

90 Communication with Charles Annis, DisplaySearch, Inc., December 10, 2010.
33

4 Assessment Framework and Approach

4.1 Expected Outcomes

Ideally, an impact assessment would be able to enumerate and estimate all the facets of 

the events in question. As discussed briefly in section 3.2 above, economists have 

traditionally distinguished between product and process technologies and fashioned the 

conceptualization of economic impact accordingly.91 Economic impact assessments 

conducted for NIST have tended to treat the events or technologies assessed as process 

technologies, conceptualizing their impact as reductions in the costs of production. The 

benefits of the technology to users of a product that has higher quality or a new feature 

are not usually assessed quantitatively. 

For this assessment it was originally intended that benefits to users of products with 

improved features and higher quality due to the availability of the FPDM would be 

estimated and an approach to collecting relevant data was developed. It was hypothesized 

that higher quality FPDs would increase some customers‘ willingness to pay and the ratio 

of the price the typical customer would be willing to pay to the actual price of the product 

(under different counterfactual scenarios) could be estimated by knowledgeable product 

producers. 

In addition, during the course of conducting background research for this assessment it 

became apparent that the FPD market ―take off‖ of the mid-1990s (discussed in section 

3.2) was significantly enabled by the codification of measurement know-how represented 

in the FPDM — a precondition for the industry‘s transition from unstructured to 

structured technological dialog. This too was a hypothesized outcome of the FPDM 

Workgroup‘s efforts but it was developed too late in the project to be quantitatively 

assessed.

On the basis of initial interviews with industry representatives, Table 4-1 characterizes 

hypothesized sources of cost-reduction benefits that accrue to firms from the utilization 

of FPDM across the product life cycle and how the locus of these benefits may vary by 

industry value chain tier. The columns indicate the major functions metrology serves in 

the product innovation and commercialization process. 

 

91 M. Trajtenberg, Economic Analysis of Product Innovation, Harvard University Press, 1990, especially 

pp. 14-16 and pp. 35-40; A. Link and J. Scott, Public Goods, Public Gains, Oxford University Press, 2011.
34

Table 4-1. Sources of Cost-Reduction Benefits from FPDM

Product Life Cycle

Sources by

Industry Tier R&D

Qualification

Test Production*

Acceptance

Test

Complaint 

Adjustment 

& Allowances

FPD End User

1 X

OEM2 X X X X X

Third Party

Test Laboratories X X X X

FPD Manufacturers X X X X X

FPD Measurement

Equipment

Manufacturers X X X

* Includes costs related to scrap, rework, retest, re-inspection, warranty field engineering, field failure, returned 

material.

1. FPD End Users include general consumers and ―pro-sumers‖ (professional consumers) of ―high-end‖ displaycentered equipment, such as medical diagnosis equipment, geospatial data analysis equipment, and military equipment.

2. OEMs (Original Equipment Manufacturers) include manufacturers of TVs, computers of most sizes, and integrators 

equipment systems and subsystems (e.g., medical imaging equipment, geospatial analysis equipment and military 

equipment system integrators and subsystems manufacturers). Equipment not covered includes front-projection 

displays, micro-displays (including head-mounted displays), and 3-D displays (including stereoscopic, holographic, and 

volumetric).

It was also expected that the transfer of FPD measurement-related device designs —stray 

light frustum, stray light elimination tube, sampling sphere, and test patterns — were 

developed and transferred more efficiently and effectively than they would have been if 

designed by the companies themselves or designed and transferred by conventional 

commercial means. 

Finally, it was posited that the SDO consensus-making process was made more efficient 

due to the fact that the composition of the FPDM Workgroup included NIST‘s FPDL 

staff and that the FPDM documentary standard itself was of a higher quality than similar 

standards due to NIST‘s participation.

4.2 Barriers to Innovation

From an economic perspective, efforts at private collaboration and public-private 

collaboration arise from barriers to innovation. In the case of the FPDM Workgroup‘s 

objectives, a number of such barriers were hypothesized. The FPDM Workgroup‘s 

outputs are believed to have mitigated those barriers and an economic impact assessment 

of the value of the outputs to industry would reflect the net benefits to society of the 

FPDM Workgroup‘s outputs (described in section 3.3 above).

The primary barrier to innovation in measurement technology arises from the nature of 

metrology itself. The costs of metrology are high, not least because it is painstakingly 

exact and the resources required to develop it are highly specialized. The more widely 

measurement technologies and techniques are shared, the lower the per-unit measurement 

cost and the broader the market of suppliers who can compete and be assessed on the 
35

basis of ―apples-to-apples‖ product quality comparisons. While this is optimal from the 

buyer perspective, it can be seen as less than ideal from the perspective of individual 

suppliers seeking increased market power and, perhaps, status as a de facto standard. 

Proprietary measurement techniques can confer market advantage but at the same time 

limit the extent to which the measurement techniques and results are shared. It was 

hypothesized that:

• VESA FPDM Workgroup was formed to address barriers to more open 

collaboration in terms of shared display quality and performance 

measurement as the industry evolved; 

• The primary form of the barriers-mitigating output is the measurement 

know-how codified in FPDM-1 and FPDM-2 (that, following 

Christenson, cited above, enabled the transformation of the quality of 

FPD measurement practices from unstructured technological dialog to 

structured technological dialog); 

• The types and extent of the barriers differ by industry tier and product 

life cycle stage; and 

• Cumulatively the mitigation of these barriers produce benefits associated 

with the FPDM Workgroup‘s outputs. 

In addition, other barriers-mitigation would include:

• The transferred metrology software and hardware designs developed by 

NIST as a member of the FPDM Workgroup, that some firms have 

neither the in house resources to design or assess; 

• The educational/training workshops made available to industry as a 

result of NIST‘s membership on the FPDM Workgroup, like the 

embodied software and hardware artifacts, is valued in terms of the cost 

to industry of developing and conveying similar material.

Some industry representatives claimed that they did not have the specialized expertise to 

develop or assess the metrology software and hardware designs developed by NIST, but, 

even if they had, assuring the wide utilization of these artifacts would be difficult in the 

absence of NIST‘s reputation for the highest standards of impartiality. Similarly, 

regarding the FPDM Workgroup‘s educational/training output, given the ―thin market‖ 

for this type of highly specialized, practical, and applied display measurement know-how, 

it seemed unlikely that a private organization would have developed such a curriculum, 

or that normal academic institutions would develop courses to fill such a narrow 

educational niche.

Finally, it seems likely that information asymmetries among the competitors and rivals 

that form SDOs drive the costs associated with consensus formation (part of which are 

due to what has been described above as ―specsmanship‖) and that NIST‘s ―honest 

broker‖ role dramatically reduced those barriers. Once the FPD metrology was codified 

in FPDM the barriers to transferring that measurement know-how to related SDOs 
36

(barriers presumably similar to those of the originating organization (VESA)) were no 

doubt reduced and secondary SDO process efficiencies would accrue to the adopting 

SDO‘s.

4.3 Comparison Scenario

In general, the benefits of the FPDM Workgroup outputs are assessed using a 

counterfactual technique, posing the question, ―How would industry events have 

unfolded in the absence of VESA‘s FPDM Workgroup?‖92 There are at least two ways to 

formulate a counterfactual question in this case. First, it might be hypothesized that in the 

absence of VESA‘s FPDM Workgroup an alternative organization would have risen to 

the occasion and the economic benefits attributable to the FPDM Workgroup are weighed 

against the relative costs of the alternative SDO. This assessment does not formulate the 

issue that way. Rather it asserts that the industry would not have developed an alternative 

to FPDM and that industry would have borne higher costs, and, perhaps, would have 

expanded less rapidly, in the absence of the FPDM. Before explaining the rationale for 

this approach to constructing the comparison scenario, a brief description of the 

evaluation method is in order.

Within the context of comparative evaluation scenarios, there are essentially three 

different evaluation methods that can employ the counterfactual technique. The 

evaluation method employed for this assessment in a variation of the ―traditional method‖ 

(also known as the ―Griliches/Mansfield method‖).

93 It estimates the benefits and costs 

that accrue to innovators and the users of the innovation; sums the net benefits (benefits 

minus costs) across all beneficiaries; and reports the ―social rate of return‖ where ―social‖ 

refers to the sum of all the beneficiaries — those who invested in the underlying 

innovation as well as those who benefited from the innovation without making 

investments (so-called ―free-riders‖). When the ―traditional evaluation method‖ is used to 

evaluate public expenditures on R&D projects, for example, the social rate of return is 

implicitly compared to private rates of return to justify the government investment role

and it is assumed that the private sector would not have undertaken the project in 

question. In the FPDM impact assessment, private collective (SDO) expenditures plus

public expenditures are being assessed by posing a variation of the traditional evaluation 

question: ―What costs and benefits accrue from the combined resources of the public and 

private organization and what was the economic significance of the public sector 

contribution?‖

To answer this question in the most straightforward manner, it is asserted that absent 

NIST‘s involvement, the industry would not have developed a true alternative to FPDM 

 

92 To avoid confusion with the discussion of a ―counterfactual evaluation method” discussed below, the 

word ―technique‖ is employed here to indicate a more general practice of reasoning, employed, for 

example, by professional historians (see, http://en.wikipedia.org/wiki/Counterfactual_history) and 

theologians (see, http://en.wikipedia.org/wiki/Molinism). For evaluative purposes, an alternative to the 

counterfactual technique is a ―dosage‖ — ―before and after‖ — technique but adequate ―before‖ and ―after‖ 

information can be hard to ascertain.

93 See Link and Scott, Op. Cit., 2011 for a discussion of the Counterfactual Evaluation Method (not to be 

confused with the counterfactual investigative technique) and the Spillover Evaluation Method. 
37

and that in the absence of the FPDM the industry would have borne higher costs, and, 

perhaps, would have expanded less rapidly. In effect, this comparison scenario (―no NIST 

means no FPDM‖) requires the application of the traditional evaluation method. It is 

hypothesized that there would be no comparable private sector alternative to the FPDM 

in the absence of NIST‘s unique capabilities as ―honest broker‖ and ―measurement 

standards conduit.‖ The anticipated economic benefits stream — the widening of the gap 

between buyers‘ willingness to pay a higher price and actual market price (what 

economists refer to as ―consumer surplus‖); faster growth of FPD applications; the 

transfer of measurement device designs; and the cost reductions in company operations 

and the SDO consensus-making process — would not have been generated absent NIST‘s 

involvement.

Three observations justify this comparison scenario. First, the FPDM is an unusually 

comprehensive measurement standard, so much so that some regard it more as a 

―guidebook‖ than a standard. (―It‘s not a standard because it doesn‘t recommend one 

thing,‖ in the words of one industry observer.) It provides the user with choices rather

than requiring compliance with ―one right way‖ to measure. The FPDM features its 

agnosticism on what to measure and focuses on how to measure: 

The standard is a measurement standard, not a compliance standard, nor a 

prescription for calibration or adjustment. ... What people do with the 

results is their own business, but we wanted to be sure it was measured 

correctly. The goal of a measurement standard is to provide unambiguous 

methods so that everyone would get the same result on the same display.94

This aspect of the FPDM, alone, suggests that reproducing the relationship between the 

FPDM Workgroup and NIST in another standards organization might not have been at all 

likely. 

Second, many close observers suggest that NIST‘s FPDL was the ―secret ingredient‖ in 

the development of the FPDM; that the standard simply would not have emerged in 

anything like its current form absent the vision and knowledge of NIST‘s staff 

Workgroup representatives.

95

Coupled with the unusual scope of the FPDM, it is difficult to imagine that the FPDM 

Workgroup-FPDL collaboration could have been replicated in an alternative SDO 

existing at the time.

Perhaps the most important reason for the chosen counterfactual comparison scenario is 

that it became apparent that positing an alternative SDO and assessing the effectiveness 

with which it might have achieved tasks actually accomplished under the auspices of 

 

94 Flat Panel Display Measurements Standard (FPDM) Version 2.0, Video Electronics Standards 

Association, Display Metrology Committee, June 1, 2001, p. 3.

95 The ―acknowledgements‖ section of the FPDM states, ―Most of the credit for creating both versions of 

the standard should go to ... Dr. Edward F. Kelley. His experienced insights, originality, and hard work 

have been an inspiration to us all.‖ Ibid., p. 4.
38

VESA would entail an implicit evaluation of VESA relative to an alternate SDO. That 

evaluation is beyond the scope of this assessment.

4.4 Economic Impact Timeframe (1992-2010)

The significant FPDM timeline events are as follows:

• 1992 — NIST Flat Panel Display Laboratory (FPDL) established.

• 1995 — VESA organizes the FPDM Workgroup at the instigation of 

Joseph Miseli (Sun Microsystems) and Douglas Baker (Compaq). 

Dennis Bechis (NIDL) was the first chairman followed soon thereafter 

by Ed Kelley, of NIST‘s FPDL, who was also the editor of FPDM 1.0 

and 2.0.

• 1998 — FPDM 1.0 released.

• 2001 — FPDM 2.0 released.

• 2007 — VESA‘s FPRM Workgroup reorganized within the Society for 

Information Display (SID) as the International Committee for Display 

Metrology (ICDM) and continues the development of the Display 

Measurement Standard (DMS), a follow-on to FPDM 2.0 with Ed 

Kelley as editor.

• 2010 — NIST discontinues Display Metrology project funding.

4.5 Economic Impact Variables96

Two kinds of cost-avoidance impact variables were identified:

• Company metrology-related cost avoidance due to the availability of the 

FPDM (annual hours x fully burdened hourly compensation x years 

utilized)

• SDO consensus-making cost avoidance due to NIST‘s participation in 

the FPDM Workgroup (annual hours x fully burdened hourly 

compensation x publication acceleration (years) due to NIST 

participation).

See Appendix C for FPDM Standard Economic Impact Survey instrument.

 

96 As discussed in Section 5.1, the survey data elements ultimately obtained were significantly fewer than 

originally anticipated. Only the economic impact variables finally estimated are identified here. 
39

5 Survey Findings

5.1 Survey Strategy, Population, and Sample

Two overlapping groups served as the population for the survey and follow-up interviews 

conducted for this impact assessment: 

• Members of the International Committee for Display Metrology —

ICDM (a committee under the Society for Information Display — SID), 

including former members of VESA‘s FPDM Workgroup

• Organizations that purchased FPDM 2.0 from VESA.

The survey population was chosen because of their interest and familiarity with the 

subject matter and their first-hand knowledge of the documentary standard development 

process. They represent all the value chain tiers of the flat panel display industry 

including government organizations, certification agencies, universities, measurement 

standards organizations, FPD integrators, measurement equipment manufacturers, display 

manufacturers, national measurement laboratories, and related standards groups.97,98

A detailed survey was posted on the ICDM website and members were encouraged to 

complete the survey on numerous occasions.99 Despite continuous encouragement of 

ICDM committee members, the number of responses remained low. In order to increase 

the number of respondents, a second survey phase was launched, focused first on other 

standards organizations with a history of interest in the FPDM, and secondly on 

purchasers of VESA‘s FPDM 2.0. In the second phase of the survey, the survey was 

dramatically reduced to reflect the questions that respondents appeared most likely to 

answer.

100 With this significant additional effort a total of sixteen responses were 

obtained. 

 

97 Joseph Miseli, ICDM. Personal communication, September 20, 2010.

98 Systematic information on the industry composition of FPDM purchasers was not readily available. It is 

assumed that the industry composition of the entire survey population (ICDM and FPDM 2.0 purchasers) 

reflects the industry composition of the ICDM.

99 Unfortunately, as a matter of ICDM policy the authors were not granted access to individual ICDM 

members‘ contact information.

100 The ―reduced survey‖ approach required that several potentially important facets of economic impact be 

abandoned. These included, in order of priority: willingness to pay benefits (a novel approach to making 

this estimate was being tested), measurement device design transfer benefits, educational/tutorial workshop 

benefits, and an assessment of the quality dimension of the FPDM (discussed in section 3.3.3 of this 

report). At least one significant FPD manufacturer refused to participate because the respondent regarded 

the framing of some of the survey questions as flawed, arguing, essentially, that the periodization used in 

the survey instrument (1992-1998 and 1999-2007) was too broad and that this would result in erroneous 

results. However, using a more precise periodization would have made even greater demands on potential 

respondents. It is typical for detailed microeconomic impact assessments to face difficult trade-offs 

between the number of survey respondents, on the one hand, and the depth of information required for 

insightful details about company operations and resource use on the other hand. Due to perceived risks to 

proprietary information, organizations are often reticent to share information that is essential for the 

assessment microeconomic impact. Famously, one of the most important and seminal articles in R&D 
40

5.2 Qualitative Findings

The impact of the FPDM appears far greater than this impact assessment is able to 

quantify. As suggested in the discussion of the emergence of the FPDM at a key juncture 

in the ―take off‖ of the PC notebook market (the so-called ―killer app‖), discussed in 

section 3.2 above), and the ever-widening and rapidly growing markets for FPD 

applications, an industry measurement expert describes the sweep of the FPDMs impact, 

even though he was unable to provide supporting data:

Before [FPDM], display characterization was the wild west of deceptive 

advertising and claims. ... NIST, realized that the emerging flat panel 

display technology created new problems for device characterization and 

performance assessment. ... One goal of [the FPDL] was to establish 

benchmarks for performance that could be applied uniformly and without 

bias to characterize these emerging technologies. ... Flat panel display 

technology over the past 20 years has emerged as the dominant visual 

interface. Its importance as core enabler of so many products today cannot 

be underestimated. ... There were many technical and manufacturing 

hurdles that had to be overcome to enable these devices to match the older 

technologies of optical projection using film and CRT based television. 

Not one but several types of flat panel display core technologies emerged. 

... Each of these technologies employed unique system architectures for 

light creation and or spatial and temporal modulation of the video signal. 

Each due their unique method of creating imagery created unique 

challenges for performance characterization. ... As these technologies 

emerged and continue to emerge into the marketplace, new challenges are 

constantly being created.

While systematic data on this facet of the FPDM‘s impact are not available, one test 

equipment manufacturer with average annual sales of $700,000 (2005-2010) estimated 

that, without FPDM, sales would have been much lower, in the range of only $200,000 -

$500,000 annually.

Intense competition in the FPD industry forces prices below some consumers‘ 

willingness to pay. Economist call this benefit, ―consumer surplus.‖ While survey data 

was insufficient to systematically estimate what are undoubtedly very large economic 

benefits, a few survey respondents indicated that FPDM increased consumer surplus 

(increased willingness to pay above competitive market prices) by an average of 36 

percent, 36 cents worth of value for each dollar actually spent by consumers for FPDMsupported products.

 

evaluation literature, and one published in a leading academic journal, not only used a small sample, but 

explains as why those doing practical, real-world economic evaluations of investments are often forced to

use small samples. See, Edwin Mansfield, et al., ―Social and Private Rates of Return from Industrial 

Innovations,‖ The Quarterly Journal of Economics, Vol. 91, No. 2 (May 1977), pp. 221-240. For the 

continued relevance of Mansfield‘s evaluation work, see Link and Scott, Op. Cit., 2011, pp. 28-29.
41

A representative of the FPD-optical-measurement-equipment-segment of the industry 

value chain, and much closer to the direct impacts of the FPDM, observed:

The FPDM has been a very valuable roadmap for our R&D activities over 

the past 10 years. From inception, the structure, clarity, and pragmatism of 

the standard has served all of us in the equipment business by defining a 

set of meaningful measurements (with back-up discussions on 

methodology and metrology) that could be implemented with practical and 

robust solutions. ... It's impossible to quantify the savings to our R&D 

effort, but I can say with confidence the FPDM standard has given us a 

solid script/roadmap to which we developed test solutions (options). 

We've promulgated the standard in our sales discussions with panel 

makers, brand name leaders, ODMs, and component makers. ... The 

FPDM's value to us equipment makers & solution-providers is immense. 

However for the broader industry, and for those on either side of a display 

value-exchange (i.e., up and down the food chain), the FPDM is even 

more important. It provided the vernacular, language, sound metrology, 

well thought-out & robust (repeatable and sufficiently accurate) test 

methods, without endorsement of a particular solution or equipment 

maker.

Equipment calibration services have also been affected by the FPDM. One representative 

expressed the importance of the FPDM for sophisticated FPD users and speculates that it 

also had effects on adjacent markets over and above its direct FPD market impacts:

[The FPD] industry is now ... able to vet their products more uniformly 

and with greater understanding. Also, U.S. government users of displays 

have a much better handle on which displays will serve their mission 

objectives, based on the methods in FPDM. Finally, the medical-imaging 

community is using FPDM concepts to test displays for acceptability and 

also to calibrate them. ... As for the impact of FPDM on our market, [our 

product] hit the market just about the time Version 2.0 of FPDM emerged. 

I could speculate that the success of [that product line] (and ... of our 

competitors) is partially the result of calibration-consciousness imparted 

by the FPDM. 

Regarding the FPDM‘s potential for reducing the cost of the consensus-making process 

in related SDOs, a medical instruments community representative wrote:

The FPDM has influenced the way we perform measurements in the 

medical display community. It has directly affected recommendations 

from professional organizations like the AAPM [American Association of 

Physicists in Medicine] and was extremely beneficial and a reference for 

work done by the IEC Standard group on medical image display 

(TC62BWG36). I expect the new version under final revisions will be 

even more beneficial, since it is seen as a useful and needed reference.

In addition, two international standards committees used text from the FPDM with formal 

permission from VESA:
42

• ISO 9241-302:2008 Ergonomics of human-system interaction -- Part 

302: Terminology for electronic visual displays

• ISO 9241-305:2008 Ergonomics of human-system interaction -- Part 

305: Optical laboratory test methods for electronic visual displays.

101

No estimate of the time saved by the ISO technical committees is available but, based on 

the experience of the FPDM Workgroup discussed below, it is prudent to suggest that the 

availability of the FPDM‘s language saved the relevant ISO technical committees many 

hours of consensus-making time worth thousands or hundreds of thousands of dollars.

Referring to the forthcoming ICDM/SID revision of the FPDM (the DMS), another datapoor respondent, representing a large FPD manufacturer, articulates a perspective that 

reflects the importance of the flat panel display measurement to the long-growing 

applications market:

Dollarizing the value of DMS and its successive revisions may be 

impossible, but we know it will be very important.... Displays are 

becoming pervasive throughout today‘s information society, and having a 

uniform and consistent means of measuring and evaluating them is 

essential.

Finally, a defense-related OEM representative also believes strongly in the costeffectiveness of the original FPDM and its successors:

The lack of uniform standards for display measurement [would be] is a 

real handicap, especially for defense and military programs, where we rely 

on the data to assess the ergonomics, human factors, and hence mission 

readiness of display systems. Using a well-defined measurement process 

as well as the well-understood specifications is the key to getting those 

deployments right the first time.

As these qualitative observations indicate, industry representatives believe that the 

economic impact of the FPDM was, and will continue to be, broad and deep. Despite an 

attempt to measure many facets of the FPDM‘s economic impact alluded to above, only a 

thin sliver of these benefits proved possible to estimate quantitatively resulting in a very 

conservative quantitative estimate of the true economic impact of the FPDM.

5.3 Quantitative Findings

Survey respondents were requested to provide information identifying the markets within 

which they operate, data concerning their market share, the company resources devoted 

to FPDM-related measurements before and after FPDM 1.0 and 2.0. Survey respondents 

also provided estimated savings in labor time devoted to the SDO consensus-making 

process due to the presence of NIST staff in the FPDM Workgroup activities, the 

 

101 Paul Boynton, personal communication, August 8, 2011.
43

acceleration of the consensus-making process due to NIST‘s role, and the extent to which 

the technical substance of the FPDM carries over to the forthcoming FPDM revision —

the ICDM‘s DMS.

The estimates of the annual FPDM-labor time saved by a company due to the availability 

of the FPDM (referred to in the language of the survey as ―do-it-yourself metrology labor 

avoidance‖) ranged from fifty (50) hours for small companies to fourteen thousand 

(14,000) hours for large aerospace firms. Double-checking the latter number, the 

respondent confirmed, ―the FPDM is very valuable to [our company]. We get a lot out of 

it. We invested a lot of time and effort helping to develop it.‖ For the period 1998-2010, 

net benefits from this source alone are estimated at approximately $2.4 million annually. 

The estimates of annual SDO consensus-making process time avoided by companies due 

to the active involvement of NIST staff in FPDM Workgroup activities ranged from 

twenty (20) hours for companies only peripherally involved in the core activities of the 

Workgroup, to one thousand (1000) hours for companies who took the deliberations of 

the Workgroup most seriously. It was estimated by a knowledgeable independent 

observer of the FPDM Workgroup‘s activities, that ten companies participated in 

Workgroup activities regularly and the majority dedicated fewer than one hundred (100) 

hours to the process annually. He confirmed that a few companies dedicate three hundred 

to one thousand (300-1000) hours annually, an estimate that squares with the information 

provided by survey respondents. For the period 1995-2001, net benefits from this source 

are estimated at approximately $640,000 annually. In addition, several survey 

respondents estimate that the FPDM was published relatively fast due to NIST‘s 

participation, an average of 4.5 years faster in the case of FPDM 1.0 (1998) and an 

estimated 7 years faster for FPDM 2.0 (2001).

Fortunately most of the respondents who provided full survey responses, including 

market share estimates, represented different horizontal segments of the FPD value chain 

so that extrapolating their labor savings to their respective value chain segment is 

straightforward.102

Finally, it is anticipated that the benefits associated with the FPDM will continue to yield 

benefits to industry beyond the study period. Survey respondents estimate that sixty-five 

percent of the measurement cost-avoidance attributed to the content of the FPDM will 

carry over to the revised DMS standard expected to be released in 2011. That standard 

will address a wide array of additional measurement challenges resulting from the 

continuing expansion of FPD applications. None of those carry-over reductions in cost 

are measured as a part of the time series of FPDM benefits used for the evaluation 

metrics estimated here, thus providing another significant source of the conservative, 

lower-bound nature of evaluation metrics presented in this study.

 

102 Where market share estimates were not provided, or were ambiguous, on-line sources were consulted to 

estimate a scaling factor based on published market share estimates.
44

6 Quantitative Analysis

6.1 Benefits

Broadly speaking, only two types of benefits were estimated for FPDM users and FPDM 

Workgroup participants:

• ―Do-it-yourself‖ metrology labor avoidance

• SDO consensus-making labor time avoidance.

Estimates of ―do-it-yourself metrology labor cost avoidance‖ benefits reflect the 

respondents‘ observations that prior to the promulgation of the FPDM, weeks and weeks 

of discussion and discovery could be dedicated to developing approaches to making

measurements, especially for  ̳contentious parameters‘ such as  ̳set up conditions‘ (a 

focus of the FPDM). The development of the approach included ongoing discussions 

with customers — for whom the displays were being made — or with display 

manufacturers. The metrology cost avoidance estimates reflect the extent to which the 

FPDM changed that. 

Similarly, estimates of SDO consensus-making cost avoidance benefits reflect the 

respondents‘ counterfactual estimates of the additional time they would have to have 

invested in the absence of support from NIST‘s FPDL staff, to produce a measurement 

standard of similar quality. It provides a unified ―buffet‖ of measurements and set up 

conditions that anyone can use and combine in any way they need but with the 

underlying metrology clearly established.

A time series of these benefits estimates for the period 1992-2010 is provided in Table 6-

1. Benefit estimates for the 1992-1997 period include only the SDO process cost 

avoidance attributable to NIST‘s staff contributions to the FPDM Workgroup established 

in 1995. Thereafter, the benefits ramp up by the amount of added benefits that accrue to 

industry in terms of do-it-yourself metrology cost avoidance. The metrology-cost 

avoidance benefits cover the period from the publication of FPDM 1.0, in 1998, to 2010 

because the future revision of the VESA FPDM 2.0 (ICDM‘s DMS) has not yet been 

released so those annual benefit estimates attributable to FPDM 2.0 continue to accrue to 

industry. The SDO consensus process benefits begin in 1995 and continue to 2001. 

Thereafter, the consensus process cost avoidance benefits accrue to the future DMS, the 

revision of FPDM 2.0.
45

Table 6-1. FPDM-Related Benefits (Nominal $)

Year Metrology Labor 

Avoidance

(Nominal $)

Consensus Labor 

Avoidance

(Nominal $)

1992 0 0

1993 0 0

1994 0 0

1995 0 1,128,483

1996 0 1,149,936

1997 0 1,170,296

1998 2,427,106 1,183,487

1999 2,462,842 573,265

2000 2,516,263 585,700

2001 1,964,988 598,953

2002 1,996,612 0

2003 2,039,660 0

2004 2,097,534 0

2005 2,167,568 0

2006 2,238,296 0

2007 2,304,147 0

2008 2,353,936 0

2009 2,376,045 0

2010 2,398,718 0

6.2 SDO Costs and NIST Expenditures

Two categories of costs are estimated for this assessment:

• SDO costs of two kinds (VESA administrative costs and the labor time 

donated to the FPDM Workgroup by member companies)

• NIST expenditures in support of the FPDL and the FPDM Workgroup.

VESA‘s FPDM Workgroup administrative costs of ~$9,000 (2010 dollars) annually were 

estimated by a senior VESA administrator familiar with the activities of the Workgroup. 

Private sector costs of participating in the Workgroup were estimated from survey 

responses scaled to reflect the estimated number of companies that regularly participated 

in Workgroup activities, estimated by active members.

NIST expenditures are estimated from historical NIST budget data projected back to the 

relevant years, 1992-2001. Due to NIST budgeting practices in the early period of the 

assessment timeframe (1992-2000) project budgets were ―pieced together‖ from a wide 

array of sources making it difficult to clearly identify the composition of the FPDL 
46

budget. In 2001, the budgeting system underwent a significant change, and then in 2004 

there was another significant administrative change. For the period 2004 -2009 the FPDL 

budget is consolidated and easily tracked, even though these NIST costs are not pertinent 

to the evaluation of the FPDM since it was released in 2001. Consultation with former 

NIST project staff confirmed that the actual level of funding for the FPDL was more or

less stable over the entire study timeframe; that the historical budget practices make it 

difficult to capture that stable level without a considerable expenditure of staff time; and, 

therefore, that back-casting the 2004-2009 average budget would reasonably reflect the 

actual historical pattern of NIST expenditure. That is the procedure that was followed for 

constructing the NIST expenditure time series in Table 6-2.

Table 6-2. FPDM Workgroup Costs and NIST Expenditures (Nominal $)

Year NIST Expenditures

(Nominal $)

FPDM Workgroup 

Labor

(Nominal $)

Workgroup 

Administrative

(Nominal $)

1992 434,663 0 0

1993 444,232 0 0

1994 453,574 0 0

1995 463,036 183,925 6,631

1996 471,838 187,422 6,757

1997 480,192 190,740 6,877

1998 485,605 192,890 6,954

1999 492,755 186,078 7,056

2000 503,443 190,114 7,209

2001 514,835 194,417 7,373

2002 0 0 0

2003 0 0 0

2004 0 0 0

2005 0 0 0

2006 0 0 0

2007 0 0 0

2008 0 0 0

2009 0 0 0

2010 0 0 0

6.3 Measures of Economic Impact 

Table 6-3 transforms the nominal costs and benefits reported in Tables 6-1 and 6-2,

above, into a series of constant 2010 dollars and provides the basis for the summary 

economic impact estimates reported below: social rate of return (SRR), net present value 

(NPV), and benefit-to-cost ratio (B/C). (For a explanation and discussion of these 

metrics, see Appendix D.)
47

Table 6-3. Constant 2010 Dollar Benefits and Costs (1992 - 2010)

Year Constant Dollar 

Benefits

($2010)

Constant Dollar 

Costs

($2010)

Constant Dollar 

Net Benefits

($2010)

1992 0 628,474 -628,474

1993 0 628,474 -628,474

1994 0 628,474 -628,474

1995 1531680 887,114 644,566

1996 1531680 887,114 644,566

1997 1531680 887,114 644,566

1998

FPDM 1.0

4672864 887,114 3,785,750

1999 3872344 874,804 2,997,540

2000 3872344 874,804 2,997,540

2001

FPDM 2.0

3129878 874,804 2,255,074

2002 2398718 0 2,398,718

2003 2398718 0 2,398,718

2004 2398718 0 2,398,718

2005 2398718 0 2,398,718

2006 2398718 0 2,398,718

2007 2398718 0 2,398,718

2008 2398718 0 2,398,718

2009 2398718 0 2,398,718

2010 2398718 0 2,398,718

As discussed in section 5.1 of this report, the economic impact assessment estimates 

selected areas of the FPDM‘s economic impact: industry metrology labor savings that 

resulted from the adoption of FPDM and consensus-making efficiencies due to NIST‘s 

participation in VESA‘s FPDM Workgroup. Only these impacts are captured in the 

conservative estimates of economic impact shown in Table 6-4. 
48

Table 6-4. Estimate of Economic Impact

Performance Metric Value

Net Present Value in 1992 $15,573,930

Net Present Value in 2010 $56,323,545

Real Social Rate of Return 48%

Benefit-to-Cost Ratio 4

With better data the full impact of the FPDM would be considerably higher and would 

capture: 

• FPDMs impact on the quality of products that use FPDs

• The value to industry of measurement device designs that were a product 

of NIST‘s role in the FPDM Workgroup

• Extra benefits to industry of releasing the FPDM sooner than it would 

have been released, due to NIST‘s participation

• The codified know-how carried forward into the International 

Committee on Display Measurement‘s (ICDM‘s) forthcoming Display 

Measurement Standard (DMS). 

The first impact metric shown in Table 6-4 (net present value in 1992 — NPV 1992) uses 

the year NIST‘s FPDL was founded as the base year and calculates the net present value 

of the project from the perspective of 1992. The NPV is the inflation-adjusted (real) value 

of the net benefits (benefits – cost) generated by the project over the course of the study 

period (1992-2010). If, in 1992, NIST project managers were trying to judge which of 

two or more projects would yield the highest economic return this is the calculation they 

would have made. From an economic perspective, if only one of the projects could have 

been chosen, the chosen project would have been the one with the highest NPV.

The second impact metric (net present value in 2010) is intended to interpret the NPV of 

the net benefits that actually occurred as a result of the FPDM effort from a somewhat 

different perspective. If the net benefits that actually accrued to the effort (NPV 1992, 

~$15.6 million) were invested in 1992 and annually earned the cost of capital to the U.S. 

government (seven percent), the value of those benefits today would be ~$56 million.

103

The third impact metric (real social rate of return — SRR) is similar to an internal rate of 

return calculation, another corporate finance technique used to judge the worthiness of an 

investment project. The modifier ―social‖ indicates that the value of this performance

metric accounts for the benefits and costs that accrue to all beneficiaries, not just the 

project investors. The SRR is the interest rate (also called the ―discount rate‖) that would 

reduce the NPV 1992 of the project to zero and reduce the benefit-cost ratio to one —

 

103 The government cost of capital is stipulated as seven percent in Office of Management and Budget 

(OMB) Circular no. A-94: Guidelines and Discount Rates for Cost-Benefits Analyses of Federal Programs, 

1992. For a discussion of the rationale of the seven percent discount rate, see, Link and Scott, Op. Cit., 

2011 and (OMB) Circular no. A-4: Regulatory Analysis, 2003. 
49

hence, the project would breakeven. As a guide to making a decision on an investment 

project (private or public), if the SRR is higher than the discount rate, the project is 

acceptable.

Finally, the benefit-to-cost ratio (BCR) is simply the ratio of the NPV of benefits to the 

NPV of costs. This value indicates that the real value of the benefits of the FPDM effort 

to all beneficiaries exceeded the costs by a ratio of 4:1.104

 

104 For a comparison of the economic impact of the FPDM effort to other efforts assessed by NIST, see 

http://www.nist.gov/director/planning/studies.cfm. Note that the nature of the FPDM project impacts is 

somewhat different than those listed on the NIST website because the source of a portion of the FPDM 

impact benefits derives from SDO operational efficiencies associated with the NIST-VESA‘s FPDM 

Workgroup collaboration. Whether the scale of these operational efficiencies is relatively large or small is 

unknown. 
50

7 Conclusion

NIST devotes considerable resources to the support of industry standards. One of its 

missions is to assist industry with the standards-related tools and information to compete 

more effectively in the global marketplace.

In 2010, approximately 350 NIST experts were involved in more than 1300 documentary 

standards committees. Optimally, these expert resources should go where they are most 

capable of achieving NIST‘s goals and objectives. By examining the role of NIST‘s 

measurement expertise in a specific case, this economic impact assessment contributes to 

understanding how the allocation of those expert resources might be optimized.

The selection of the Video Electronics Standards Association‘s (VESA‘s) Flat Panel 

Display Measurement (FPDM) standard from among a short list of potential projects was 

based on pragmatic considerations. As it turned out, the history of the FPDM project, and 

the investigation of the way in which NIST‘s know-how is developed and enters the 

stream of industry value creation, is a treasure trove of lessons learned. For example:

• Documentary standards have significant economic impacts, similar to the 

economic impact of other types of standards based on various categories of 

―infratechnologies‖

• NIST collaboration with SDOs is a significant technology transfer platform

• NIST involvement in SDOs improves the efficiency of the SDOs operations by 

mitigating ―specsmanship‖ (reducing consensus-making time) and speeding 

standard release date

• NIST‘s measurement know-how played a critical role in the dynamics of this 

global, knowledge-driven industry, enabling the ―structured dialog‖ that lead to 

the proliferation of FPD applications.

Reflecting on the history of the FPDM standard may also hold some clues about 

attributes of the standard development process that indicate if and how NIST should 

participate in specific SDO projects. The history of NIST‘s engagement with VESA‘s

FPDM Workgroup suggests that the following organizational attributes are worthy of 

management consideration when choosing to engage with an SDO:

Does NIST take a technical leadership role at Workgroup level? (+), 

Is industry participation wide and substantial at the Workgroup level? (+)

Is NIST‘s leadership at the Workgroup level anticipatory of industry trends 

and well supported financially (indicating a NIST strategic priority)? (+)

Are other SDOs proposing similar or competing standards? (-)

Even though data limitations prevented the estimation of the FPDM‘s full economic 

impact, it is clear that the benefits are substantially higher than those measured. While 

this study‘s impact metrics account for the full costs of the FPDM‘s development, only 

two benefit streams — metrology labor savings and SDO consensus-making efficiencies 

— are estimated. Other, significantly large benefits streams associated with FPDM 

Workgroup outputs include:
51

• FPDMs impact on the quality of products that use FPDs and the 

consumer benefits that accrue from such quality improvements

• The value to industry of measurement device designs that were a product 

of NIST‘s role in the FPDM Workgroup

• Extra benefits to industry of releasing the FPDM sooner than it would 

have been released, due to NIST‘s participation

• The codified know-how carried forward into the International 

Committee on Display Measurement‘s (ICDM‘s) forthcoming Display 

Measurement Standard (DMS). 

The impact of the FPDM on the quality of products that use FPDs, alone, is no doubt

very considerable. FPD applications in the markets for portable computers, desktop 

monitors, televisions, handheld computers, camcorders, digital still camera, GPS devices, 

and financial terminals have grown while fierce global competition has driven prices 

lower and lower. It is likely that professional and general consumers‘ ―willingness to 

pay‖ higher prices than prevail in the market would be considerable. These ―consumer 

surplus‖ benefits probably dwarf the cost savings captured in the economic impact 

measures presented in this report. 

On the account presented here, the scope of those applications was enabled by a 

fundamental transformation of the quality of the measurement know-how available to 

sellers and buyers of FPD. That transformation was brought about by the collaboration of 

NIST‘s FPDL and VESA‘s FPDM Workgroup. 

Quantitative economic impact metrics, while important, are only one feature of the

important insights that come from close, detailed examination of cases like this one.

This case study highlights a facet of the dynamics of global competition that is vitally 

important to the innovation that drives U.S. prosperity. NIST emphasizes the importance 

of ―infratechnology‖ to the growth and development of industries and, going forward, to 

an economic policy that strives to foster U.S. economic growth in a globally competitive 

world.105 The collaboration between NIST and VESA may be exemplary from this 

perspective. 

In one of its most concrete forms, ―infratechnology‖ is measurement know-how. As 

discussed throughout this report, from an economic perspective documentary standards 

can be vehicle for such know-how and NIST‘s ―honest broker‖ role can play a significant 

role in diffusing this measurement know-how. Moreover, this case highlights how the 

traditional distinctions between product and process technology alone could not account 

for important developments in the industry. The role of measurement know-how —

infratechnology, a NIST core competency — played a central role, embodied in the 

FPDM.

 

105 Gregory Tassey, The Technology Imperative, Edward Elgar, 2007.
52

FPDM Workgroup produced a measurement standard unlike any other by most accounts. 

NIST‘s technical staff made a great deal of difference to the outcome, but not all the 

difference. In addition to technical virtuosity of the FPDL‘s staff, probably largely 

because of it, NIST forged a close and effective relationship with its industry counterparts

that resulted in expanding markets, increased product quality for a rapidly expanding 

array of products, and lowered costs of production. In the words of a prominent industry 

spokesman, ―displays are becoming pervasive throughout today‘s information society, 

and having a uniform and consistent means of measuring and evaluating them is 

essential.‖

The impact assessment demonstrates that documentary standards have had significant 

economic impact. Even as a very conservative measure of economic impact, the 

performance metric values estimated for the FPDM are similar to the economic impact of 

other ―infratechnologies.‖

106

Moreover, it appears that NIST‘s collaboration with SDOs has significant potential for 

supporting the innovation process that fuels global competition. In the case of the global

FPD industry that support took the form of facilitating the application of technologies 

initially developed in the U.S. but manufactured elsewhere. Still, the value to FPD 

supplier industries and to consumers of devices and services that incorporate FPD 

technology is significant. 

In addition to benefiting industry operations directly by reducing considerable metrology 

costs, NIST involvement in VESA improved the efficiency of that organization by 

reducing a considerable barriers to consensus making — the mitigation of

―specsmanship.‖ NIST‘s close collaboration with VESA and other related SDOs also 

appears to have served as a platform for the transfer and diffusion of measurement device 

designs to industry as well.

The Standard Coordinating Office (SCO) aims to support the development of qualitative 

and quantitative impact assessments of documentary standards and, in time, begin to 

accumulate and translate ―lessons learned‖ into a strategic management advice about the 

likely economic impact of NIST staff‘s SDO collaborations.

 

106 See, http://www.nist.gov/director/planning/studies.cfm
53

Appendix A: Types of Documentary Standards 

ISO/IEC Guide 2: 2004 

A basic standard has a wide-ranging coverage or contains general provisions for one 

particular field, such as a standard for metal that can affect a wide range of products from 

cars to fasteners. 

Terminology standards are concerned with terms, usually accompanied by their 

definitions. The standards define words that permit industries or parties entering into a 

transaction to use a common, clearly understood language. 

Testing standards are concerned with test methods, sometimes supplemented with other 

provisions related to testing, such as sampling, use of statistical methods, or the sequence 

of tests. They are generally used to assess the performance or other characteristics of a 

product. 

Product standards specify requirements to be fulfilled by a product (or a group of 

products) to establish its fitness for purpose. Such standards can also address other issues, 

including packaging and labeling or processing requirements. 

Process standards specify requirements to be fulfilled by a process to establish its fitness 

for purpose. For example, a process standard could cover requirements for the effective 

functioning of an assembly line operation. 

Service standards, such as for servicing or repairing a car, establish requirements to be 

fulfilled by a service to establish its fitness for purpose. 

Interface standards, such as requirements for the point of connection between a 

telephone and a computer terminal, specify requirements concerned with the 

compatibility of products or systems at their points of interconnection. 

Standards on data to be provided contain a list of characteristics for which values or 

other data are to be stated for specifying the product, process or service. This type of 

standard generally provides a list of data requirements for a product or service for which 

values need to be obtained. 
54

Appendix B. Flat Panel Display Laboratory Publications 

and Presentations

STANDARDS

E. F. Kelley, "Understanding Specsmanship," 2007 Display Applications Conference, 

Society for Information Display, Invited Paper 7.1, San Francisco, CA, October 23-25, 

2007.

P. Boynton, ―Display Metrology Concerns in International Standards,‖ Proceedings of 

the Third Americas Display Engineering and Applications Conference (ADEAC ‘06), 

October 2006 (2006)

P. Boynton, ―The Challenge to Display Metrology and Standards: Whom Do You Trust?‖ 

Digest of the IEEE Broadcast Technology Society/IBC2006, Amsterdam, September 7, 

2006 (2006) (invited paper and talk)

P.A. Boynton, "Display Metrology Concerns in International Standards," Council for 

Optical Radiation Measurements annual conference, Boulder, Colorado, May 11, 12, 

2005

E. F. Kelley and S. Pala, "Challenges in Automotive Display Standards," Council for 

Optical Radiation Measurements annual conference, Boulder, Colorado, May 11, 12, 

2005.

P. A. Boynton, "The goals of good display metrology," The Display Standard, May 2005

E. F. Kelley, "Plotting the Course of the Next VESA Flat Panel Display Measurements 

Standard," J. Society for Information Display, Vol. 13, No. 1, pp. 67-79, January 2005. 

E. F. Kelley, "What Do the Specifications Mean?" Proceedings of the First Americas 

Display Engineering and Applications Conference (ADEAC 2004), Society for 

Information Display, Ft. Worth, TX, USA, pp. 15-18, October 26, 2004.

P. Boynton, ―Display Metrology Concerns in International Standards,‖ Presentation 

Materials of Special Sessions of the IMID 2nd Flat Panel Display Standardization 

Session, Daegu, Korea, August 2004 

P. Boynton, ―The Challenge to Display Metrology and Standards: Whom Do You Trust?‖ 

Workshop and Special Evening Session of the SID 24th International Display Research 

Conference, pp. 46-57, August 2004.

E. F. Kelley, "Plotting the Course of the Next VESA Flat Panel Display Measurements 

Standard," 2004-SID International Symposium Digest of Technical Papers, Society for 

Information Display, Invited Paper 7.1, Vol. 35, Book 1, pp. 78-81, Seattle, WA, May 25, 

2004.
55

P. A. Boynton, and E. F. Kelley, "ISO and VESA Standardization for FPDs", Proceedings 

of the Special Session on Standardization of Flat Panel Displays, The 3rd International 

Meeting on Information Display, Daegu, Korea, July 2003

E. F. Kelley, "Vehicular Display Metrology Seminar," Society for Information Display 

Metropolitan Detroit Chapter, 8th Annual SID Symposium on Vehicle Displays, Detroit, 

MI, October 16-18, 2001.

E. F. Kelley, "SAE J1757 Committee Report on Proposed Reflection Measurements," 8th 

Annual SID Symposium on Vehicle Displays, October 16-18, 2001.

P. Boynton and C. Fenimore, ―Characterization of Projection Systems for the MPEG-4 

Digital Cinema Compression Scheme Evaluation,‖ NIST IR 6792, 2001.

E. F. Kelley, "Flat Panel Display Measurements and Standards," Applications Seminar A4, 2000-SID International Symposium, Society for Information Display, Long Beach, 

California, May 17, 2000.

Edward F. Kelley, George R. Jones, Paul A. Boynton, Michael D. Grote, and Dennis J. 

Bechis. "A Survey of the Components of Display Measurement Standards." Journal of the 

Society for Information Display, Vol. 3, No. 4, pp. 219-222, December 1995.

Edward F. Kelley, George R. Jones, Paul A. Boynton, Michael D. Grote, and Dennis J. 

Bechis. "A Survey of the Components of Display Measurement Standards." Society for 

Information Display International Symposium Digest of Technical Papers, Vol. 26, 

Orlando, FL, pp. 637-640, May 23-25, 1995.

Dennis J. Bechis, Michael D. Grote, David P. Bortfeld, Lawrence H. Hammer, Michael J. 

Polak, Edward F. Kelley, George R. Jones, and Paul A. Boynton. "Display Measurement 

Round Robin," Society for Information Display International Symposium Digest of 

Technical Papers, Vol. 26, pp. 641-644, Orlando, FL, May 23-25, 1995.

STRAY LIGHT MANAGEMENT

E. F. Kelley, J. Penczek, and P. A. Boynton, "Character Contrast under Uniform Ambient 

Conditions," 2009-SID International Symposium Digest of Technical Papers, Society for 

Information Display, San Antonio, Texas, USA, paper 24.1, pp. 321-324, May 31 - June 

5, 2009.

E. F. Kelley, "Character Contrast," Information Display Magazine, January 2009. 

P.A. Boynton and E.F. Kelley, "Comparing Methodologies for Determining Resolution 

from Contrast in Projection Display Systems," Projection Displays XI, Proceedings of the 

SPIE, Vol 5740, January 2005. 
56

P. Boynton, ―Tools and Diagnostics to Assess and Improve Projection Display Metrology 

Capabilities,‖ Proceedings of the First Americas Display Engineering and Applications 

Conference (ADEAC 2004), Society for Information Display, Ft. Worth, TX, USA, pp. 

15-18, October 26, 2004

P. Boynton, ―Methods to Eliminate Stray Light for Display Characterization,‖ 

Proceedings of the USDC High Information Content Display Systems Conference, 

Arlington, VA, September 2004 (2004)

P. A. Boynton, , E. F. Kelley, and J. M. Libert ―Projection Display Metrology at NIST:

Measurements and Diagnostics,‖ Liquid Crystal Materials, Devices, and Applications X 

and Projection Displays X, Proceedings of the SPIE, Vol. 5289, pp. 302-313, January 

2004 (2004)

E. Kelley and P. Boynton, ―Seamless Frusta Creation for Stray-Light Management in 

Optical Devices,‖ NIST IR 6637. December 1, 2003. (2003)

Edward F. Kelley, Paul A. Boynton, and John M. Libert, "Evaluation of Luminance Probes 

Used for Accurate Contrast Measurements in Medical Displays," Council for Optical 

Radiation Measurements Conference, Stanford University, Menlo Park, California, June 

19-20, 2003.

A. Badano, S. Pappada, E. F. Kelley, M. J. Flynn, S. Martin, and J. Kanicki, "Luminance 

probes for contrast measurements in medical displays," 2003-SID International Symposium 

Digest of Technical Papers, pp. 928-931, Society for Information Display, Baltimore, MD, 

May 18-23, 2003.

P. Boynton and E. Kelley, "A Liquid-Filled Camera for the Measurement of High-Contrast 

Images," Cockpit Displays X, Proceedings of the SPIE, Vol. 5080, (April 2003).

P. A. Boynton, ―Evaluation of a Liquid-Filled Camera for Measurement of Shadow Detail. 

Part I: Sensor Evaluation,‖ NIST IR 6900, August 2002.

P. Boynton and E. Kelley, ―Stray Light Elimination Tube Prototype,‖ NIST IR 6861, 

March 2002

P. A. Boynton and E. F. Kelley, "Stray Light Compensation in Small Area Contrast 

Measurements of Projection Displays," Projection Displays VIII, Proceedings of the SPIE, 

Vol. 4657, pp. 122-130 (January 2002).

P. Boynton and C. Fenimore, ―Characterization of Projection Systems for the MPEG-4 

Digital Cinema Compression Scheme Evaluation,‖ NIST IR 6792, 2001.

P. A. Boynton and E. F. Kelley, "Compensation for Stray Light in Projection-Display 

Metrology," 2001-SID International Symposium Digest of Technical Papers, Society for 

Information Display, Vol. 32, pp. 334-337, San Jose, CA, June 4-8, 2001.
57

J. W. Roberts and E. F. Kelley, "Measurements of Static Noise in Display Images," 

Proceedings of the SPIE V4295B-27, Electronic Imaging Symposium, San Jose, CA, pp. 

211-218, January 23, 2001.

P. A. Boynton, ―Tools and Diagnostics for Projection Display Metrology,‖ NISTIR 6591: 

Digital Cinema 2001 Conference Proceedings, Gaithersburg, MD, January 11-12, 2001

P. A. Boynton and E. F. Kelley, "Stray Light Elimination in Making Projection Display 

Measurements," Flat Panel Display Technology and Display Metrology, Proceedings of the 

SPIE Vol. 3636, pp. 232-239 (1999).

E. F. Kelley, "Simulated-Eye-Design Camera for High-Contrast Measurements," 

International Society for Optical Engineering (SPIE) Electronic Imaging Conference, San 

Jose, CA, January 29, 1999, Proceedings of SPIE, Flat Panel Display Technology and 

Display Metrology, Vol. 3636, pp. 184-190.

P. A. Boynton and E. F. Kelley, "Is the Measurement of Front-Projector Characteristics an 

Impossible Task?", Information Display, Vol. 14, No. 10, pp. 34-37, December 1998.

P. A. Boynton and E. F. Kelley, "Small-Area Black Luminance Measurements on a White 

Screen Using Replica Masks," 1998-SID International Symposium Digest of Technical 

Papers, Society for Information Display, Vol. 29, pp. 941-944, Anaheim, CA, May 17-22, 

1998.

P. A. Boynton and E. F. Kelley, "Accurate Contrast-Ratio Measurements Using a Cone 

Mask," 1997 Society for Information Display (SID) Symposium, Digest of Technical 

Papers, May 11-16, 1997, Boston, MA, Sec 32.1, pp. 823-826 (May 1997). 

P. A. Boynton and E. F. Kelley, "Measuring the Contrast Ratio of Displays: It's Not 

Trivial." Information Display, Vol. 12, No. 11, pp. 24-27, November 1996.

DIAGNOSTICS

P. A. Boynton, J. M. Libert, E. F. Kelley, C. Wall, C. Campbell, R. Baribeau, "A Color 

Gamut Assessment Standard: An Inter-Laboratory Measurement Comparison," Council 

for Optical Radiation Measurements annual conference, Boulder, Colorado, May 11, 12, 

2005 

P. Boynton, ―Methods for Assessing Display Measurement Capabilities,‖ Cockpit 

Displays XI, Proceedings of the SPIE, Vol. 5543, September 2004 (2004) 

J. Libert, E. Kelley, P. Boynton, S. Brown, C. Wall, and C. Campbell, "A Color Gamut 

Assessment Standard: Construction, Characterization, and Inter-Laboratory Measurement 

Comparison," Fourth Oxford Conference on Spectrometry, SPIE vol. 4826 (August 2003).
58

P. A. Boynton and E. F. Kelley, "Display Metrology and ISO and VESA Standardization 

for FPDs", Display Asia, September 2003.

J. M. Libert, E. F. Kelley, P. A. Boynton, S. W. Brown, C. Wall, and C. Campbell, "A 

Color Gamut Assessment Standard: Construction, Characterization and Inter-Laboratory 

Measurement Comparison," Fourth CORM Oxford Conference on Spectrometry," 

Davidson, NC, June 9 - 13, 2002. 

E. F. Kelley, ―Electronic Display Metrology--Not a Simple Matter,‖ Proc. of SPIE, Vol. 

4450, pp. 44-53, NIST/SPIE Centennial Symposium/Celebration, San Diego, CA, August 

1, 2001.

E. F. Kelley and J. V. Miseli, "Setup Patterns for Display Measurements - Version 1.0," 

NISTIR 6758, 14 pp., June 2001.

J. M. Libert, P. A. Boynton, E. F. Kelley, S. Brown, Y. Ohno, and F. Manoocheri, ―Display 

Measurement Assessment Standard: Program Update and Preliminary Colorimetric 

Measurement Inter-comparison,‖ Council for Optical Radiometric Measurement (CORM) 

Annual Conference Presentation Materials, Session III: Inter-Comparisons, Gaithersburg, 

MD, May 14-16, 2001 (May 2001).

P. A. Boynton and E. F. Kelley, "Light Measuring Device Diagnostics for the Photometric 

and Colorimetric Measurement of Flying-Spot Displays," Proceedings of the SPIE V4295, 

Electronic Imaging Symposium, San Jose, CA, January 23, 2001.

J. M. Libert, Paul A. Boynton, Edward F. Kelley, Steven W. Brown, Yoshihiro Ohno, and 

Farshid Manoocheri, "A Standard Illumination Source for the Evaluation of Display 

Measurement Methods and Instruments," Proceedings of the SPIE V4295, SPIE Electronic 

Imaging Symposium, San Jose, CA, January 23, 2001.

J. M. Libert, P. A. Boynton, E. F. Kelley, S. Brown and Y. Ohno, "An Assessment 

Standard for the Evaluation of Display Measurement Capabilities," Proceedings of the 

Society for Imaging Science & Technology (IS&T) / Society for Information Display (SID) 

Eighth Color Imaging Conference: Color Science and Engineering, Scottsdale, AZ, 

November 7-10, 2000.

P. A. Boynton, E. F. Kelley, S. Highnote, R. Hurtado, ―Diagnostics for light measuring 

devices in flying-spot display measurements,‖ Projection Displays 2000: Sixth in a Series, 

Proceedings of the SPIE Vol. 3954, pp. 42-51 (2000)

E. F. Kelley, "Polystyrene Box Uniform Light Sources," Council for Optical Radiation 

Measurements, May 4-6, 1999.

P. A. Boynton, ―Evaluation of Light Measuring Devices for Flying-Spot Display 

Measurements,‖ Council for Optical Radiometric Measurement (CORM) Annual 
59

Conference Presentation Materials, Session IV: Optical Metrology of Displays, 

Gaithersburg, MD, May 3-6, 1999 (May 1999).

P. A. Boynton and E. F. Kelley, "Assessment of Colour Measurement Systems Using 

Interference Filters," Proc. CIE Expert Symposium '97 on Colour Standards for Imaging 

Technology, Nov 21-22, 1997, Scottsdale, AZ, publ. Commission Internationale de 

L'Eclairage (CIE), Vienna, Austria, pp. 22-27 (1998).

P. Boynton, Y. Ohno, and E. Kelley, "Interference Filter Characterization of 

Spectroradiometers and Colorimeters," Society for Information Display International 

Symposium Digest of Technical Papers, Vol. 27, San Diego, CA, pp. 207-210, May 12-17, 

1996.

REFLECTION METROLOGY

E. F. Kelley, J. Penczek, and P. A. Boynton, "Character Contrast under Uniform Ambient 

Conditions," 2009-SID International Symposium Digest of Technical Papers, Society for 

Information Display, San Antonio, Texas, USA, paper 24.1, pp. 321-324, May 31 - June 

5, 2009.

S. Kim, E. F. Kelley, and J. Penczek, "Robustness of Display Reflectance Measurements: 

Comparison between BRDF and Hemispherical Diffuse Reflectance," 2009-SID 

International Symposium Digest of Technical Papers, Society for Information Display, 

San Antonio, Texas, USA, paper 24.2, pp. 325-327, May 31 - June 5, 2009.

E. F. Kelley, "Character Contrast," Information Display Magazine, January 2009. 

J. Penczek, E. F. Kelley, and S. Kim, "Robustness of Display Hemispherical Reflectance 

measurement Apparatus," Invited Paper 41-1, IMID/IDMC/Asia Display 2008 Digest, 

pp. 1355-1357, Ilsan, Korea, October 13-17, 2008.

E. F. Kelley, "Diffuse Clarification," Information Display Magazine, Vo. 24, No. 2, p. 8 

ff., February 2008.

E. F. Kelley, "Metrology and Robustness of Bright-Room Contrast Measurement," 2007-

SID International Symposium Digest of Technical Papers, Society for Information 

Display, Invited Paper 20.1, pp. 1034-1037, Long Beach, CA, May 21-25, 2007.

Edward F. Kelley, Max Lindfors, and John Penczek, "Display Daylight Ambient Contrast 

Measurement Methods and Daylight Readability," J. Society of Information Display, 

Vol. 14, No. 11, pp. 1019-1030, November 2006.

E. F. Kelley, "Diffuse Reflectance and Ambient Contrast Measurements Using a 

Sampling Sphere," Proceedings of the Third Americas Display Engineering and 

Applications Conference (ADEAC 2006), Society for Information Display, Atlanta, GA, 

USA, pp. 1-5, October 24,26, 2006.
60

E. F. Kelley, "Daylight and Sunlight Display Readability Measurement Methods," 

Proceedings of the Second Americas Display Engineering and Applications Conference 

(ADEAC 2005), Society for Information Display, Portland, Oregon, USA, pp. 1-4, 

October 25-27, 2005.

E. F. Kelley, "Variable-Radius Source Method to Separate Specular Component from 

Haze Peak," Council for Optical Radiation Measurements annual conference, Boulder, 

Colorado, May 11, 12, 2005

E. F. Kelley and J. Penczek, "Scalability of OLED Fluorescence in Consideration of 

Sunlight-Readability Reflection Measurements," 2004-SID International Symposium 

Digest of Technical Papers, Society for Information Display, Paper P.54, Vol. 35, Book 

1, pp. 450-453, Seattle, WA, May 25, 2004.

E. F. Kelley, "Sensitivity of Display Reflection Measurements to Apparatus Geometry," 

2002-SID International Symposium Digest of Technical Papers, Society for Information 

Display, Boston, MA, pp. 140-143, May 19-24, 2002.

E. F. Kelley, "SAE J1757 Committee Report on Proposed Reflection Measurements," 8th 

Annual SID Symposium on Vehicle Displays, October 16-18, 2001.

E. F. Kelley, "Impediments to Reproducibility in Display Metrology," Digital Cinema 2001 

Conference, NIST, pp. 91-101, January 11, 2001. http://www.itl.nist.gov/div895/docs/DCinema2001Proceedings.pdf

S. Pala and E. F. Kelley, "Haze Complications in FPD Performance Reflection 

Measurements," SAE, 6th Annual Strategic and Technical FPD Symposium - Vehicular 

Applications of Displays and MEMS, Session 2 on Vehicular FPD-Requirements & 

Market, September 22, 1999.

E. F. Kelley, G. R. Jones, and T. A. Germer, "The Three Components of Reflection," 

Information Display, Vol. 14, No. 10, pp. 24-29, October 1998.

E. F. Kelley, G. R. Jones, and T. A. Germer, "Display Reflectance Model Based on the 

BRDF," Displays, Vol. 19, No. 1, pp. 27-34, June 30, 1998.

G. R. Jones and E. F. Kelley, "Reflection Measurement Problems Arising from Haze," 

1998-SID International Symposium Digest of Technical Papers, Society for Information 

Display, Vol. 29, pp. 945-946, Anaheim, CA, May 17-22, 1998.

G. R. Jones and E. F. Kelley, "Bidirectional Reflectance Distribution Factor Measurements 

on Flat Panel Displays," Proc. CIE Expert Symposium '97 on Colour Standards for Imaging 

Technology, Nov 21-22, 1997, Scottsdale, AZ, publ. Commission Internationale de 

L'Eclairage (CIE), Vienna, Austria, pp. 28-33 (1998).
61

E. F. Kelley and G. R. Jones, "Utilizing the Bidirectional Reflection Distribution Function 

to Predict Reflections from FPDs," 1997 Society for Information Display (SID) 

Symposium, Digest of Technical Papers, May 11-16, 1997, Boston, MA, Sec 32.3, p. 831-

834 (May 1997). 

G. Jones, E. Kelley, T. Germer, "Specular and Diffuse Reflection Measurements of 

Electronic Displays," Society for Information Display International Symposium Digest of 

Technical Papers, Vol. 27, San Diego, CA, pp. 203-206, May 12-17, 1996.

E. F. Kelley and G. Jones, ―Flat Panel Display Reflection Measurements at NIST,‖ 

Invited Presentation at Photo Research sponsored seminar on Reflection Measurements at 

the EDEX'96 (Electronic Display Exhibition) in Yokohama Japan, May 1996.

TUTORIAL/OVERVIEW

M. Dowell and E. F. Kelley, "Tips for Buying Flat Panel Display Televisions," 

Metrologist, pp. 28, 29, January 2008.

E. F. Kelley, "Understanding Specsmanship," 2007 Display Applications Conference, 

Society for Information Display, Invited Paper 7.1, San Francisco, CA, October 23-25, 

2007.

P. Boynton, ―The Challenge to Display Metrology and Standards: Whom Do You Trust?‖ 

Digest of the IEEE Broadcast Technology Society/IBC2006, Amsterdam, September 7, 

2006 (2006) (invited paper and talk)

E. F. Kelley, "Display Measurements for Flat Panel Displays," Application Tutorial A-6, 

Society for Information Display, 2006 International Symposium, San Francisco, 

California, June 9, 2006.

E. F. Kelley, "Flat-Panel-Display Measurement Techniques and Concerns," 2005-SID 

International Symposium, Application Tutorial, Seminar Lecture Notes, Society for 

Information Display, Boston, MA, May 26, 2005

P. A. Boynton, "The goals of good display metrology," The Display Standard, May 2005

E. F. Kelley, "What Do the Specifications Mean?" Proceedings of the First Americas 

Display Engineering and Applications Conference (ADEAC 2004), Society for 

Information Display, Ft. Worth, TX, USA, pp. 15-18, October 26, 2004.

P. Boynton, ―The Challenge to Display Metrology and Standards: Whom Do You Trust?‖ 

Workshop and Special Evening Session of the SID 24th International Display Research 

Conference, pp. 46-57, August 2004. 
62

E. F. Kelley, "Flat-Panel-Display Measurement Techniques and Concerns," 2004-SID 

International Symposium Seminar Lecture Notes, Society for Information Display, 

Seminar F-1, Vol. 2, pp. F-1/1-61, Seattle, WA, May 28, 2004.

P. A. Boynton, E. F. Kelley, and J. M. Libert, "Current Projects In Display Metrology At 

The NIST Flat Panel Display Laboratory," Fourth Oxford Conference on Spectrometry, 

SPIE vol. 4826 (August 2003).

P. A. Boynton, E. F. Kelley, and J. M. Libert, "Current Projects in Display Metrology at the 

NIST Flat Panel Display Laboratory," Fourth CORM Oxford Conference on 

Spectrometry," Davidson, NC, June 9 - 13, 2002.

E. F. Kelley, "Fundamentals of Display Metrology," 2002 Society for Information Display 

International Symposium Short Course S-3, 2002 SID International Symposium, Boston, 

Massachusetts, 205 pp., May 19-24, 2002

E. F. Kelley, "Vehicular Display Metrology Seminar," Society for Information Display 

Metropolitan Detroit Chapter, 8th Annual SID Symposium on Vehicle Displays, Detroit, 

MI, October 16-18, 2001.

E. F. Kelley, ―Electronic Display Metrology--Not a Simple Matter,‖ Proc. of SPIE, Vol. 

4450, pp. 44-53, NIST/SPIE Centennial Symposium/Celebration, San Diego, CA, August 

1, 2001.

E. F. Kelley, "Display Metrology Seminar," Society for Information Display (SID), 

Metropolitan Detroit Chapter, 7th Annual SID Symposium on Vehicle Displays, Dearborn, 

Michigan, 18 pages (66 slides) October 3, 2000.

P. A. Boynton, E. F. Kelley, J. M. Libert, "Meeting the Metrology Needs of the 

Microdisplay Industry," Proceedings of the United Engineering Foundation Microdisplay 

2000 Conference, Boulder, CO, August 7-9, 2000. 

E. F. Kelley, "Flat Panel Display Measurements and Standards," Applications Seminar A4, 2000-SID International Symposium, Society for Information Display, Long Beach, 

California, May 17, 2000.

E. F. Kelley, "Electronic Display and Imaging at NIST," OIDA Advanced Electronic 

Imaging Workshop, SPIE Electronic Imaging Symposium & Photonics West, San Jose, 

CA, 38 pages, January 26, 2000. 

E. F. Kelley, ―Flat Panel Display Metrology -- Less than Meets the Eye,‖ National 

Conference of Standards Laboratories, Charlotte, North Carolina, July 11-15, 1999.

E. F. Kelley, "Flat Panel Display Measurements and Standards," Applications Seminar, 

1999-SID International Symposium, Society for Information Display, San Jose, CA, May 

17-21, 1999.
63

E. F. Kelley, "Flat Panel Display Measurements and Standards," Applications Seminar, 

1998-SID International Symposium, Society for Information Display, Anaheim, CA, May 

17-22, 1998.

Edward F. Kelley, ―Measurements for Optical Imaging Applications,‖ Optoelectronics 

Metrology Roadmapping Workshop, NIST, Boulder, Colorado, February 3, 1998

E. F. Kelley, "What Is Behind Good Display Metrology?", Information Display, Vol. ?, No. 

?, pp. ?, 1998.

Paul Boynton. "Flat Panel Display Laboratory at NIST". Proceedings of the Military 

Display Workshop, Long Branch, NJ, July 11-13, 1995. (July 1995)

P. A. Boynton, G. Jones, and E. Kelley, "Flat Panel Display Lab at NIST," abstract and 

viewgraphs from invited talk published in Workshop Proceedings of Third Annual Military 

Display Workshop, Army Research Laboratory, Appendix G-1, Jul 11-13, 1995, Ft. 

Monmouth, NJ (Jul 1995). 

H. S. Bennett, C. Fenimore, B. F. Field, and E. F. Kelley, "Making Displays Deliver a Full 

Measure," Information Display, Inst. for Research Services, New York, NY, 11, Issue 1, 

pp. 20-27 (Jan 1995).

B. F. Field, E. F. Kelley, C. Fenimore, and H. S. Bennett, "Research on Flat Panel Display 

Measurements at the National Institute of Standards and Technology," Proc. Intl. Society 

for Optical Engineering (SPIE) Symposium on Advanced Flat Panel Display Technologies, 

February 7-8, 1994, San Jose, CA, SPIE Vol. 2174, pp. 30-34 (Jul 1994).

METROLOGY FOR SPECIFIC APPLICATIONS

Projection Displays

E. F. Kelley, K. Lang, L. D. Silverstein, and M. H. Brill, "Projector Flux from Color 

Primaries," 2009-SID International Symposium Digest of Technical Papers, Society for 

Information Display, San Antonio, Texas, USA, Late News Paper 17.5L, pp. 224-227, 

May 31 - June 5, 2009.

P.A. Boynton and E.F. Kelley, "Comparing Methodologies for Determining Resolution 

from Contrast in Projection Display Systems," Projection Displays XI, Proceedings of the 

SPIE, Vol 5740, January 2005. 

P. Boynton, ―Tools and Diagnostics to Assess and Improve Projection Display Metrology 

Capabilities,‖ Proceedings of the First Americas Display Engineering and Applications 

Conference (ADEAC 2004), Society for Information Display, Ft. Worth, TX, USA, pp. 

15-18, October 26, 2004
64

P. A. Boynton, , E. F. Kelley, and J. M. Libert ―Projection Display Metrology at NIST:

Measurements and Diagnostics,‖ Liquid Crystal Materials, Devices, and Applications X 

and Projection Displays X, Proceedings of the SPIE, Vol. 5289, pp. 302-313, January 

2004 (2004)

P. Boynton and E. Kelley, ―Stray Light Elimination Tube Prototype,‖ NIST IR 6861, 

March 2002

P. A. Boynton and E. F. Kelley, "Stray Light Compensation in Small Area Contrast 

Measurements of Projection Displays," Projection Displays VIII, Proceedings of the 

SPIE, Vol. 4657, pp. 122-130 (January 2002).

P. Boynton and C. Fenimore, ―Characterization of Projection Systems for the MPEG-4 

Digital Cinema Compression Scheme Evaluation,‖ NIST IR 6792, 2001.

P. A. Boynton and E. F. Kelley, "Compensation for Stray Light in Projection-Display 

Metrology," 2001-SID International Symposium Digest of Technical Papers, Society for 

Information Display, Vol. 32, pp. 334-337, San Jose, CA, June 4-8, 2001.

P. A. Boynton, ―Tools and Diagnostics for Projection Display Metrology,‖ NISTIR 6591: 

Digital Cinema 2001 Conference Proceedings, Gaithersburg, MD, January 11-12, 2001

P. A. Boynton, E. F. Kelley, S. Highnote, R. Hurtado, ―Diagnostics for light measuring 

devices in flying-spot display measurements,‖ Projection Displays 2000: Sixth in a 

Series, Proceedings of the SPIE Vol. 3954, pp. 42-51 (2000)

P. A. Boynton, ―Evaluation of Light Measuring Devices for Flying-Spot Display 

Measurements,‖ Council for Optical Radiometric Measurement (CORM) Annual 

Conference Presentation Materials, Session IV: Optical Metrology of Displays, 

Gaithersburg, MD, May 3-6, 1999 (May 1999).

P. A. Boynton and E. F. Kelley, "Stray Light Elimination in Making Projection Display 

Measurements," Flat Panel Display Technology and Display Metrology, Proceedings of 

the SPIE Vol. 3636, pp. 232-239 (1999).

P. A. Boynton and E. F. Kelley, "Is the Measurement of Front-Projector Characteristics 

an Impossible Task?," Information Display, Vol. 14, No. 10, pp. 34-37, December 1998.

Daylight Readability 

E. F. Kelley, J. Penczek, and P. A. Boynton, "Character Contrast under Uniform Ambient 

Conditions," 2009-SID International Symposium Digest of Technical Papers, Society for 

Information Display, San Antonio, Texas, USA, paper 24.1, pp. 321-324, May 31 - June 

5, 2009.
65

S. Kim, E. F. Kelley, and J. Penczek, "Robustness of Display Reflectance Measurements: 

Comparison between BRDF and Hemispherical Diffuse Reflectance," 2009-SID 

International Symposium Digest of Technical Papers, Society for Information Display, 

San Antonio, Texas, USA, paper 24.2, pp. 325-327, May 31 - June 5, 2009.

E. F. Kelley, "Character Contrast," Information Display Magazine, January 2009. 

J. Penczek, E. F. Kelley, and S. Kim, "Robustness of Display Hemispherical Reflectance 

measurement Apparatus," Invited Paper 41-1, IMID/IDMC/Asia Display 2008 Digest, 

pp. 1355-1357, Ilsan, Korea, October 13-17, 2008.

Edward F. Kelley, Max Lindfors, and John Penczek, "Display Daylight Ambient Contrast 

Measurement Methods and Daylight Readability," J. Society of Information Display, 

Vol. 14, No. 11, pp. 1019-1030, November 2006.

E. F. Kelley, "Daylight and Sunlight Display Readability Measurement Methods," 

Proceedings of the Second Americas Display Engineering and Applications Conference 

(ADEAC 2005), Society for Information Display, Portland, Oregon, USA, pp. 1-4, 

October 25-27, 2005.

Medical Displays

A. Saha, E. F. Kelley, and A. Badano, "Accurate color measurement methods for medical 

displays," Med. Phys., Vol. 37, No. 1, January 2010, to be published.

Anindita Saha, Hongye Liang, Aldo Badano, and Edward F. Kelley, "Accurate color 

measurement methods for medical displays," SPIE Medical Imaging, Conference 6516 

PACS and Imaging Informatics, San Diego, CA, USA, 17-22 Feb. 2007.

Edward F. Kelley, Paul A. Boynton, and John M. Libert, "Evaluation of Luminance 

Probes Used for Accurate Contrast Measurements in Medical Displays," Council for 

Optical Radiation Measurements Conference, Stanford University, Menlo Park, 

California, June 19-20, 2003.

A. Badano, S. Pappada, E. F. Kelley, M. J. Flynn, S. Martin, and J. Kanicki, "Luminance 

probes for contrast measurements in medical displays," 2003-SID International 

Symposium Digest of Technical Papers, pp. 928-931, Society for Information Display, 

Baltimore, MD, May 18-23, 2003.

First-Responder Imaging Cameras

Joshua B. Dinaburg, Francine Amon, Anthony Hamins, Paul Boynton, ―Performance of 

liquid-crystal displays for fire-service thermal-imaging cameras,‖ Journal of the Society 

for Information Display, June 2008 (2008).
66

Joshua B. Dinaburg, Francine Amon, Anthony Hamins, Paul Boynton, ―LCD display 

screen performance testing for handheld thermal imaging cameras‖ Proceedings of SPIE, 

the International Society for Optical Engineering, Infrared imaging systems (design, 

analysis, modeling, and testing XVII) (19-20 April, 2006) (2006)

Microdisplays

P. A. Boynton, ―Microdisplay Metrology Research at NIST,‖ Proceedings of the Scottish 

Optoelectronics Association (SOA) / Society for Information Display (SID) Conference 

and Workshop on Microdisplays and Their Applications, Edinburgh, UK, September 13-

14, 2001 

P. A. Boynton, E. F. Kelley, J. M. Libert, "Meeting the Metrology Needs of the

Microdisplay Industry," Proceedings of the United Engineering Foundation Microdisplay 

2000 Conference, Boulder, CO, August 7-9, 2000. 

Viewing Angle

T. G. Fiske, L. D. Silverstein, C. Penn, E. F. Kelley, "Viewing Angle: A Matter of 

Perspective," 1998-SID International Symposium Digest of Technical Papers, Society for 

Information Display, Vol. 29, pp. 937-940, Anaheim, CA, May 17-22, 1998.

Noise Perception 

Stephen Herman, Bruce Field, Paul Boynton, and Gerard de Haan. "The Perception of 

Clamp Noise in Television Receivers." Conference Record of the Society for Information 

Display International Display Research Conference, Monterey, CA, October 10-13, 1994, 

pp. 317-320 (Oct 1994).

Stephen Herman, Bruce Field, and Paul Boynton, "Quantification of Temporal Noise 

Threshold in Television Displays", Society for Information Display International 

Symposium Digest of Technical Papers, San Jose, CA, June 14-16, 1994, vol. XXV, pp. 

869-872 (May 1994)

Modeling

Edward F. Kelley, Bruce Field, George R. Jones, and Paul A. Boynton, "Display 

Modeling and Active Matrix Liquid Crystal Display Model on a Video Supercomputer," 

Society for Information Display International Symposium Digest of Technical Papers, 

Vol. 25, pp. 779-782, San Jose, CA, June 14-16, 1994.
67

Appendix C. FPDM Standard Economic Impact Survey 

Instrument 

Survey 

FPDM Standard (1.0, 2.0) Economic Impact 

Introduction

NIST is conducting an impact assessment of a documentary standard. NIST has conducted numerous 

economic impact assessments over the years. For examples of such assessments, go to 

< http://www.nist.gov/director/planning/study_info.cfm>. 

Survey respondents were chosen as the survey population because of their interest and familiarity with 

the subject matter and their first-hand knowledge of the documentary standard development process.

The Flat Panel Display Measurement (FPDM) standard (FPDM 1.0 and FPDM 2.0), (developed by the 

FPDM Working Group of the Video Electronics Standards Association (VESA)) has been selected as the 

basis for this assessment of documentary standards. The choice of FPDM has no strategic significance. It 

was deemed the best candidate of a small number of projects from an impact assessment perspective. 

BECAUSE THIS SURVEY CONCERNS THE PAST, AND BECAUSE WE ARE SENSITIVE TO THE BURDEN 

PLACED ON INDUSTRY RESPONDENTS, WE ARE NOT SEEKING “ACCOUNTING QUALITY” ANSWERS.

WE EXPECT ROUGH-ORDER-OF-MAGNITUDE ANSWERS BASED ON YOUR SEASONED JUDGEMENT; 

ESTIMATES THAT WOULD “MAKE SENSE” TO OTHER EXPERIENCED INDUSTRY PARTICPANTS GIVEN 

THE TIME CONSTRAINTS OF 30 MINUTES OR SO.

TASC Inc., an independent analytical services company, is conducting this assessment on NIST’s behalf. 

All the answers you provide will be held in the strictest confidence. All data in the economic impact 

assessment will be reported in aggregated form, as averages and ranges, so that no individual 

person, company, or establishment data will be discernable.

The impact assessment will be based on data collected for this survey and employs a present discounted 

value approach to organizing time series estimates of benefits and costs provided by you, the survey 

respondents. The data will be compiled to calculate several measures of economic impact.

We need you to provide your best estimates to all questions. Where these take you past your comfort 

zone, consider that there is likely no one in a better position to formulate a response. If, in addition to 

your response, you would like to suggest a point of contact within your organization whose estimate we 

would also benefit from obtaining, please provide us with a name, phone number, and e-mail address. 

We will contact that person and solicit their estimates as well. We welcome this opportunity.

As a token of appreciation for participating in this survey effort, the final report will be available from 

NIST in early 2011 and you and your company will be listed in the acknowledgements. Your full 

participation in the survey assures that the report will be based on the best information available.

_____________________________________________________________________

NOTE: This survey contains collection of information requirements subject to the Paperwork Reduction Act. Notwithstanding 

any other provision of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to 

comply with, a collection of information subject to the requirements of the Paperwork Reduction Act, unless that collection of 

information displays a currently valid OMB control number. The estimated response time for this survey is 30 minutes. The 

response time includes the time for reviewing instructions, searching existing data sources, gathering and maintaining the 

data needed, and completing and reviewing the collection of information." OMB Number: 0693-0033; Expiration: 

10/31/2012.
68

Background Information

1. The flat panel display industry consists of multiple tiers or facets. Please 

indicate the industry tiers that best characterize your company’s role in the 

industry.

End user (e.g., general or professional consumer)

Original equipment manufacturer (e.g., Dell, Sun Microsystems, HP, Apple, 

Sony, Samsung or Boeing, Lockheed-Martin, Raytheon, Northrop-Grumman)

Display Manufacturers (e.g., Samsung, LG, AUO, CMO, CPT, HannStar or 

aerospace display manufacturers such as Honeywell, Rockwell-Collins, 

American Panel Corp.)

Display component/material manufacturers

Equipment manufacturers (including measurement instruments)

Testing laboratories

Other (Please specify and offer an explanation of your role.) 

2. For the segment(s) of the tier(s) of the FPD industry in which your company 

operates, please estimate your company’s average annual shares – of the 

worldwide market and U.S. market for products and services that are 

significantly affected by FPDM – over time for each of two periods—1992-

1998 and 1999-2007. 

1992-1998 1999-2007

For Worldwide Sales:

Average Annual Company Share (%)

For U.S. Sales: 

Average Annual Company Share (%) 

Please provide explanatory notes about your market segment(s) and tier(s) if 

necessary:

3. What facets of your company’s operations are most affected by the 

measurement data and techniques represented in the FPDM.

R&D

Qualification of displays for use in complementary products/services

Manufacturing process quality control

Acceptance testing

Complaint adjustment

Other (Please specify and explain for a non-expert.) 
69

4. In what year did your company adopt FPDM as its measurement standard?

FPDM 1.0 

FPDM 2.0 

Costs and Benefits Estimates

FPDM Working Group Participation

For the purposes of this assessment, VESA’s FPDM Working Group was constituted in 

1992 and continued through 2007 (when the Working Group’s activities were 

transferred to SID’s, ICDM). In the analysis of survey data, we will distinguish two 

periods, 1992-1998 (that includes the release of FPDM 1.0 in 1998) and 1999-2007 

(that includes the release of FPDM 2.0 in 2001).

5. Estimate the average annual number of hours your company employees or 

consultants actually dedicated to the FPDM Working Group, 1992-2007.

Average Annual Hours, 1992-1998: 

Average Annual Hours, 1999-2007: 

6. In 2010 dollars, estimate the value of the fully burdened (i.e., including 

benefits such as retirement and health) annual compensation for a full-time 

equivalent (FTE) employee with the requisite expertise to participate in the 

efforts of the FPDM Working Group.

Total annual compensation for one FTE in 2010 dollars: $ 

Absent FPDM

Economic impact assessments are often conducted on the basis of a “counterfactual 

scenario” that posits how things would have been in the absence of the event being 

assessed. Prior to the release of FPDM 1.0, producers or buyers of flat panel displays 

would consult any (or all) of a number of existing standards (depending on the specific

application) and develop their own methodologies for assessing display quality, often 

in consultation and coordination with their suppliers and buyers.107

 

107. According to an authoritative source, the following standards were available for consultation: ISO 9241 (Parts 3, 

7, 8) and ISO 13406 (draft 2); ANSI HSF-100 (1988) and IT7.215 (1992); EIA TEB (27) and TEP (105); VESA 

Display Specifications and Test Procedures (for CRTs); NIDL‘s Procedures for Evaluation and Reporting the 

Capabilities of High Performance Display Monitors for Imagery Applications; SAE ARP 1782 and ARP 4260; MRP 

1990:8 (1990:10); USAF AFGS 87213A; and IEC SC 47C. Consulting these various uncoordinated standards 

presented measurement difficulties because they described what a flat panel display was to perform, not how it was to 

be tested; the various standards were not aligned for any specific purpose; they provided only partial solutions to 

characterization, specification, or qualification tasks and required the buyer and/or supplier to develop consolidated and 

integrated proprietary specifications and qualifications; often provided inadequate measurement methods; and often 
70

For evaluation purposes, we posit a counterfactual scenario with two phases labeled, 

“do-it-yourself” and “find another home.”

7. For the period 1992-1998 (release of FPDM 1.0, 1998) estimate the average 

annual number of hours expended by your company (and, in parenthesis, its 

suppliers and buyers) in “do-it-yourself” solutions to problems and issues for 

which the information in FPDM 1.0 provided an alternative solution.

Average Annual Hours, 1992-1998: (Suppliers/Buyers Hours: )

Please provide some typical examples of the types of problems and issues you 

have in mind in your response to Question #7. 

8. In 2010 dollars, estimate the value of the fully burdened (i.e., including 

benefits such as retirement and health) annual compensation for a full-time 

equivalent (FTE) employee with the requisite expertise to develop “do-ityourself” solutions to problems and issues for which the information in the 

FPDM provided alternative solutions. 

Total annual compensation for one FTE in 2010 dollars $ 

9. Please identify an organization (“another home”) that, in your view, would 

likely have developed an alternative to FPDM, had VESA not undertaken the 

effort, and the year the FPDM alternative would have emerged in that 

scenario.

Alternative organization: 

Year an FPDM alternative would have emerged: 

Effectiveness of that FPDM alternative (% of FPDM quality): %

If you would like to elaborate, please provide your rationale for the three

responses:

 

assumed that measurement techniques for older technologies (e.g., CRTs) transferred well to new technologies (e.g., 

LCDs). See, Edward F. Kelley, George R. Jones, Paul A. Boynton, Michael D. Grote, and Dennis J. Bechis. ‖A Survey of 

the Components of Display Measurement Standards,‖ Journal of the Society for Information Display, Vol. 3, No. 4, 

December 1995, pp. 219-222.
71

10. For the period from the time that your company adopted FPDM 1.0 until 

the time that it adopted FPDM 2.0—indicated in your answer to Question #4—

estimate the average annual number of hours expended by your company 

(and, in parenthesis, its suppliers and buyers) in “do-it-yourself” solutions to 

problems and issues not addressed in FPDM 1.0 but for which FPDM 2.0 did 

provide solutions. 

Average Annual Hours, from adoption of FPDM 1.0 until adoption of FPDM 2.0:

 (Suppliers/Buyers Hours: )

Please provide some typical examples of the types of problems and issues you 

have in mind in your response to Question #10. 

11. After the organizational change from VESA to SID in 2007, development of 

the forthcoming ICDM Display Measurements Standard (DMS) began. The new 

ICDM DMS is expected to enable more do-it-yourself cost avoidance, because it 

will include standard performance measurements in FPDM and hence the 

resulting do-it-yourself cost avoidance, but also there will be development of 

the standard, extending coverage to measurements not covered in FPDM, 

allowing even more substitution for do-it-yourself activities. The do-ityourself cost-avoidance will be what was obtained with FPDM and then more. 

Moreover, the new ICDM DMS is expected to accomplish in new ways some of 

the measurements that FPDM substituted for do-it-yourself solutions. To the 

extent that new ICDM DMS procedures are substituted for previously existing 

FPDM procedures, some of the do-it-yourself cost-avoidance that had been 

due to FPDM would be due to the new ICDM DMS. The question here is what 

percentage of the previous do-it-yourself cost-avoidance from FPDM is still 

anticipated to be obtained from the content of FPDM rather than having been 

replaced by new content in ICDM DMS. The answer could be close to 100%, or 

if the ICDM DMS is expected to substitute new approaches for much of what is 

in FPDM 1.0 and 2.0, the answer could be substantially less than 100%. 

11a. What percentage of the benefits (the costs avoided in do-it-yourself 

solutions) realized by your company because of FPDM do you anticipate will 

still be realized and still be due to the content of FPDM rather than the new 

content in ICDM DMS? 

Percentage of benefits due to FPDM:

< 10% 10%-20% 21%-30% 31%-40% 41%-50%

51%-60% 61%-70% 71%-80% 81%-90% >90%
72

11b. What do you anticipate to be the commercial lifetime (in years) of the 

forthcoming ICDM Display Measurement Standard (DMS)? 

 Years

Benefits of NIST Participation in the FPDM Working Group

12a. Given your answers to Question #5, estimate the average annual 

additional (beyond those reported in Question #5) number of hours your 

company employees or consultants would have dedicated to the FPDM 

Working Group, 1992-2007, had NIST not participated in the effort and each of 

the Working Group participants cooperatively increased their time dedicated 

to the effort to the extent needed to ensure the quality and the same time of 

development of the existing FPDM.

Average Annual Additional Hours, 1992-1998: 

Average Annual Additional Hours, 1999-2007: 

12b. Estimate the years that FPDM 1.0 (1998) and FPDM 2.0 (2001) would 

have been released, if NIST had not participated in the effort and the average 

annual number of hours your company employees or consultants (and other 

participants) dedicated to the FPDM Working Group remained the same as 

provided in your response to Question #5. 

“Absent NIST” FPDM 1.0 release year: 

“Absent NIST” FPDM 2.0 release year: 

Thank you for taking the time to provide your best estimates for the answers to 

the questions.

We look forward to providing you with the results of our analysis.
73

Appendix D. Economic Impact Metrics108

The economic impact metrics in this report are calculated from a time series of costs and 

benefits in constant dollars. They represent "real" rates of return. In contrast, "nominal" 

rates of return would be based on time series of current dollars (the dollars of the year in 

which the benefits were realized or the costs were incurred).

Social Rate of Return (SRR)

The SRR is the value of the discount rate, i, that equates the net present value (NPV) of a 

stream of all net benefits associated with an investment project to zero. The time series 

runs from the beginning of the project, t = 0, to a milestone terminal point, t = n. Net 

benefits refer to total benefits (B) less total costs (C) in each time period. Mathematically,

(1) NPV = [(B0 - C0) / (1 + i)0] + ... + [(Bn - Cn) / (1 + i)n] = 0

where (Bt - Ct) represents the net benefits associated with the project in year t, and n 

represents the number of time periods (years in most cases) being considered in the 

evaluation. 

For unique solutions of i, from equation (1), the SRR can be compared to a value, r, that 

represents the opportunity cost of funds invested by the technology-based public 

institution. Thus, if the opportunity cost of funds is less than the social rate of return, the 

project was worthwhile from an ex post social perspective.

Benefit-to-Cost Ratio

The ratio of benefits-to-costs is precisely that, the ratio of the present value of all 

measured benefits to the present value of all costs. Both benefits and costs are referenced 

to the initial time period, t = 0, as:

(2) B / C = [ t=0 to t=n Bt /(1 + r)t] / [ t=0 to t=n Ct / (1 + r)t]

A benefit-to-cost ratio of 1 implies a break-even project. Any project with B / C > 1 is a 

relatively successful project. Fundamental to implementing the ratio of benefits-to-costs 

is a value for the discount rate, r.

While the discount rate representing the opportunity cost for public funds could differ 

across a portfolio of public investments, the calculated metrics in this report follow the 

 

108 The characterization of the three metrics follows A. Link and J. Scott, Public Accountability: Evaluating 

Technology-Based Institutions (Boston: Kluwer Academic Publishers) 1998.
74

guidelines set forth by the Office of Management and Budget: Constant-dollar benefitcost analyses of proposed investments and regulations should report net present value and 

other outcomes determined using a real discount rate of 7 percent.109

Net Present Value (NPV)

The information developed to determine the benefit-to-cost ratio can be used to determine 

net present value as:

(3) NPV = B – C.

 

109 Office of Management and Budget (OMB), Circular No. A-94 Revised, ―Guidelines and Discount Rates

for Benefit-Cost Analysis of Federal Programs,‖ October 29, 1992.