OFFICE OF 

PERSONNEL 

MANAGEMENT

Agency Needs to 

Improve Outcome 

Measures to 

Demonstrate the 

Value of Its Innovation 

Lab

Report to Congressional Requesters

March 2014

GAO-14-306

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-306, a report to 

congressional requesters

March 2014

OFFICE OF PERSONNEL MANAGEMENT

Agency Needs to Improve Outcome Measures to 

Demonstrate the Value of Its Innovation Lab

Why GAO Did This Study

Organizations from around the globe 

are emphasizing that strategies 

promoting innovation are vital to 

solving complex problems. To try to 

instill a culture of innovation in its 

agency, OPM followed the lead of a 

number of private sector companies, 

nonprofit organizations, and 

government bodies by creating an 

innovation lab. GAO was asked to 

examine the lab. 

Specifically, GAO 1) described the 

lab’s start-up costs, staffing and 

organization, activities, and policies 

governing the lab’s use, and 2) 

assessed how OPM’s innovation lab 

compares to other organizations’ 

innovation labs, including how it uses 

benchmarks and metrics and how it 

addresses challenges to innovation. 

GAO reviewed cost, staffing, and 

performance information. GAO also 

reviewed relevant literature on 

innovation and interviewed officials 

from public, private, and nonprofit 

organizations with innovation facilities 

similar to OPM’s lab.

What GAO Recommends

Among other things, GAO 

recommends that the Director of OPM 

should direct lab staff to 1) develop a 

mix of performance targets and 

measures to help them monitor and 

report on progress toward lab goals, 

and 2) build on existing efforts to share 

information with other agencies that 

have innovation labs. OPM generally 

concurred with GAO’s 

recommendations; in addition, they 

described the steps being taken and 

planned to refine their ongoing 

evaluation efforts and to further 

leverage other federal innovation labs.

What GAO Found

In March 2012, the Office of Personnel Management (OPM) opened its 

innovation lab, a distinct physical space with a set of policies for engaging people 

and using technology in problem solving. The goals of OPM’s innovation lab are 

to provide federal workers with 21st century skills in design-led innovation, such 

as intelligent risk-taking to develop new services, products, and processes. 

OPM’s lab was built at a reported cost of $1.1 million, including facility upgrades 

and construction, equipment and training, and other personnel costs. The lab 

employs approximately 6 full-time equivalents, including a director, and in fiscal 

year 2013, the lab’s operating costs were approximately $476,000, including 

salaries.

View of OPM’s Innovation Lab

OPM’s innovation lab is similar in mission and design to other innovation labs 

GAO reviewed, and OPM has incorporated some of the prevalent practices that 

other labs use to sustain their operations. Specifically, OPM is using its lab for a 

variety of projects, including as a classroom for building the capacity to innovate 

in the federal government. Lab staff indicated that they plan to begin long-term 

immersion projects—complex projects with diverse users—within a few months. 

OPM plans to develop and implement evaluation plans specific to each 

immersion project that will help them track cost benefits or performance 

improvement benefits associated with the projects.

Starting in March 2013, OPM lab staff began work on a program evaluation 

framework to more systematically measure the lab’s progress toward meeting its 

overarching goals. In addition, lab staff members are tracking lab activities, such 

as classes and workshops, and are surveying lab users about the quality of their 

experience in the lab. However, they have not developed performance targets or 

measures related to project outcomes, and without a rigorous evaluation 

framework that can help OPM track the lab’s performance, it will be hard to 

demonstrate that the lab is operating as originally envisioned. 

While labs provide a physical space where innovators can convene, federal 

agencies are not fully aware of their growing community. However, OPM is taking 

steps to ensure work done in the lab is shared across OPM and with other 

federal innovators—for example, by hosting weekly training sessions in the lab 

on best practices. Studies show that information sharing and interorganizational 

networks can be a powerful driver supporting innovation.

View GAO-14-306. For more information, 

contact Seto J. Bagdoyan at (202) 512-4749 

or bagdoyans@gao.gov.
Page i GAO-14-306 OPM's Innovation Lab

Letter 1

Background 3

OPM Lab Activities Are Intended to Build Capacity for Innovation 

and Support Project-Based Problem Solving 8

OPM’s Lab Is Similar in Mission and Design to Other Innovation 

Labs, but OPM Needs to Systematically Evaluate the Lab’s 

Performance 13

Conclusions 24

Recommendations for Executive Action 24

Agency Comments and Our Evaluation 25

Appendix I Objectives, Scope, and Methodology 27

Appendix II Innovation Labs Share Similar Design Elements and Problem-Solving 

Methods 30

Appendix III Office of Personnel Management’s Innovation Lab Non-Design 

Programmed Activities, Calendar Years 2012-13 32

Appendix IV Comments from the Office of Personnel Management 33

Appendix V GAO Contact and Staff Acknowledgments 35

Tables

Table 1: Breakdown of OPM’s Lab Start-Up Costs (Fiscal Year 

2012) 8

Table 2: Description of OPM’s Lab Staff for Fiscal Year 2013 10

Table 3: OPM’s Innovation Lab Sessions with Human-Centered 

Design Programming, Calendar Years 2012 and 2013 12

Table 4: List of Innovation Labs with Facilities Similar to OPM 28

Contents
Page ii GAO-14-306 OPM's Innovation Lab

Figures

Figure 1: OPM’s Lab 5

Figure 2: Innovation Labs Share Similar Design Elements and 

Problem-Solving Methods 7

Figure 3: Five-Step Framework for Guiding Innovation in the 

Census and HUD Labs 14

Abbreviations

CAT Center for Applied Technology

Census U.S. Census Bureau

CFPB Consumer Financial Protection Bureau

FCAT Fidelity Center for Applied Technology

FDA Food and Drug Administration

FTE Full-time equivalent

GSA General Services Administration

HUD Housing and Urban Development

KSC Kennedy Space Center

NASA National Aeronautics and Space Administration

NERD New England Research and Development

OPM Office of Personnel Management

STEM Science, technology, engineering, and mathematics

TACSI Australian Center for Social Innovation

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-306 OPM's Innovation Lab

441 G St. N.W.

Washington, DC 20548

March 31, 2014

The Honorable Darrell Issa

Chairman

Committee on Oversight and Government Reform

House of Representatives

The Honorable Blake Farenthold

Chairman

Subcommittee on Federal Workforce, U.S. Postal Service and the 

Census

Committee on Oversight and Government Reform

House of Representatives

Organizations from around the globe are emphasizing that strategies 

promoting innovation are vital to solving complex problems and that future 

growth will increasingly depend on productivity-raising innovation.1 To try 

to instill a culture of innovation in its agency, in March 2012, the Office of 

Personnel Management (OPM) followed the lead of a number of private 

sector companies, nonprofit organizations, and U.S. and foreign 

government bodies and created an innovation lab. An innovation lab is 

generally defined as a distinct physical space with a set of policies for 

engaging people and using technology in problem solving. OPM’s lab, 

located in the sub-basement of its Washington, D.C. headquarters, uses 

a problem-solving approach called “human-centered design,” which is 

described as the discipline of generating concrete solutions and ideas 

driven by the needs, behaviors, and context of the people for whom they 

are designed. This approach also uses iterative testing of new services, 

products, and processes and is intended to encourage faster learning at 

the front end of the design process rather than large, expensive failures 

upon implementation.2 The premise behind OPM’s lab was that the new 

space and a new approach to problem solving would support efforts to 

 1

The top 1,000 global innovation companies, including Intel, Microsoft, Samsung, and 

Volkswagen, spent an estimated $638 billion in 2013 on their research and development 

activities, a key input driving innovation. See Barry Jaruzelski, John Loehr, and Richard 

Holman, “The 2013 Global Innovation 1000 Study: Navigating the Digital Future,” Booz & 

Company, Issue 73 (New York, New York: Booz and Company Inc., Winter 2013).

2

An iterative process arrives at a decision or a desired result by repeating rounds of 

analysis or a cycle of operations. The objective is to bring the decision or result closer to 

the desired outcome with each repetition. 
Page 2 GAO-14-306 OPM's Innovation Lab

address some complex issues and eventually lead to a more innovative 

agency culture.

You requested that we review OPM’s innovation lab. This report (1) 

describes the OPM innovation lab’s start-up and operating costs, staffing 

and organization, activities, and policies governing the lab’s use, and (2) 

assesses how OPM’s innovation lab compares to other organizations’

innovation labs, including how it uses benchmarks and associated metrics 

and how it addresses potential challenges to innovation.

To address our first objective, we reviewed documentation on OPM’s 

start-up and operating costs, staffing, and policies regarding the lab and 

interviewed OPM staff overseeing the lab and its activities. To address 

the second objective, we conducted a comprehensive literature search—

including research conducted by global consulting firms and academic 

institutions—and interviewed representatives familiar with innovation 

facilities. We reviewed documentation and interviewed officials from the 

U.S. Census Bureau (Census), the Department of Housing and Urban 

Development (HUD), and the National Aeronautics and Space 

Administration (NASA) innovation labs. In addition, we met with an official 

from the Consumer Financial Protection Bureau (CFPB). While CFPB 

lacks a dedicated innovation lab, the agency has a reputation among 

federal agencies as a leader in innovative website development. We also 

interviewed representatives from other government, private sector, and 

nonprofit organization innovation facilities, such as Denmark’s MindLab, 

the Fidelity Center for Applied Technology, and UNICEF’s innovation 

labs.3 To select organizations for comparison, we identified those with 

innovation facilities similar to OPM’s lab, including those with a dedicated 

physical space and using similar problem-solving methods. We examined 

documentation and interviewed OPM lab staff regarding identifying any 

outcomes—such as cost reductions, performance improvements, or other 

results from projects—undertaken by the lab since its inception. We also 

conducted a detailed literature search to identify benchmarks and 

associated metrics applicable to the development and use of innovation 

facilities in the public and private sectors. Based on our literature search, 

we identified common challenges that can hamper organizations’ efforts 

to use labs as innovation vehicles and prevalent practices that can 

support labs’ success and sustainability. In addition, we reviewed our 

 3

See figure 2 for a complete list of the innovation labs we surveyed.
Page 3 GAO-14-306 OPM's Innovation Lab

interview records to identify commonly recurring challenges and prevalent 

practices that can support labs’ success and sustainability. We verified 

that the challenges and prevalent practices we identified during our 

literature search were also those more often cited during the interviews.

We conducted this performance audit from July 2013 to March 2014 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives. More detailed information on 

our scope and methodology appears in appendix I.

Innovation is a dynamic process through which problems and challenges 

are defined, new and creative ideas are developed, and new solutions are 

selected and implemented.4 It is also a complex process that involves 

taking iterative steps to solve problems. Innovation requires an 

environment that encourages participants to challenge traditional 

practices without fear of repercussions. Ideally, innovation participants 

are empowered to be creative and make mistakes, and appropriate risktaking is not only tolerated but encouraged.

Some federal leaders are trying various innovation tools, including on-line 

idea submission programs, competitions, and prizes, as ways of 

unleashing employee creativity. For example, in a memorandum issued in 

March 2010, the administration urged federal agencies to use challenges 

and prizes to crowdsource5 innovative approaches to government 

initiatives and programs.6 At relatively low costs, crowdsourcing initiatives 

can garner valuable and creative solutions that may not have come 

through traditional means. As another example, the Presidential 

 4

OPM defines innovation as the process of improving, adapting, or developing a product, 

system, or service to deliver better results. 

5

Crowdsourcing is the practice of obtaining needed services, ideas, or content by soliciting 

contributions from a large group of people—especially from the online community—rather 

than from traditional employees or suppliers.

6

Office of Management and Budget, Guidance on the Use of Challenges and Prizes to 

Promote Open Government, OMB Memorandum M-10-11 (Washington, D.C.: 2010). 

Background
Page 4 GAO-14-306 OPM's Innovation Lab

Innovation Fellows program pairs top innovators from the private sector, 

nonprofit organizations, and academia with top innovators in government 

to collaborate during focused six- to thirteen-month periods. The program 

aims to develop solutions that can save lives, save taxpayer money, and 

fuel job creation. For example, the goal of one of the program’s projects is 

to identify information critical to saving lives and mitigating damage in a 

disaster.

Even with the efforts of some federal leaders to encourage innovation, 

federal government-wide scores tracking how agencies foster and reward 

employee innovation dropped in 2013 for the second year in a row. 

OPM’s 2013 Federal Employee Viewpoint Survey, released in November 

2013, found that only 35 percent of federal workers believe that creativity 

and innovation are rewarded, with positive responses in this area showing 

a steady decline of six percentage points over the past three years. 

Research suggests that half of all innovations are not initiated by 

organizational leaders. Instead, research shows that it is important to 

have processes for gathering stakeholders’ and front-line workers’ views 

to identify areas for possible improvement.7

As an innovation tool, labs are based on the idea that the competencies 

needed for systematic innovation—such as intelligent risk-taking to 

develop new services, products and processes—are not the same as 

those required for daily operations. Innovation labs seek to provide 

approaches, skills, models, and tools beyond those that most employees 

are trained in and use to do their work. In addition, public sector

innovation labs can be viewed as attempts to create an organizational 

response to a range of challenges to innovation, as innovation efforts face 

unique obstacles in the public sector. For example, funding for new public 

ventures is limited, and the risks of innovation are high in government. A 

defining characteristic of the public sector is that it is subject to broad 

scrutiny, so that when an innovation fails or is less than a complete 

success, there is the prospect of political consequences.

With constrained budgets expected to continue into the foreseeable 

future, innovation in our public services is a necessity. In the last decade, 

many public sector organizations around the globe have set up facilities 

 7

Geoff Mulgan and David Albury, Innovation in the Public Sector, Ver.1.9 (London, 

England: Strategy Unit, October 2003) p. 13.
Page 5 GAO-14-306 OPM's Innovation Lab

with the explicit purpose of supporting innovation efforts. For example, 

Denmark’s MindLab, started in 2002, is a cross-governmental innovation 

unit that is part of the country’s Ministry of Business and Growth, the 

Ministry of Education, the Ministry of Employment, and Odense 

Municipality, and which collaborates with the Ministry for Economic Affairs 

and the Interior. The group covers broad policy areas including areas 

such as entrepreneurship, digital self-service, education, and 

employment. OPM’s lab was modeled, in part, on Denmark’s MindLab.

OPM officials, consistent with other innovation lab representatives we 

interviewed, maintained that, unlike a typical conference room, innovation 

labs can be easily reconfigured for large groups and smaller breakout 

sessions. They allow users to write on walls and preserve visual artifacts 

more easily than typical cubicles and traditional office space. This can be 

done with very low-tech tools such as markers and a whiteboard. Figure 1 

shows a view of OPM’s lab.

Figure 1: OPM’s Lab
Page 6 GAO-14-306 OPM's Innovation Lab

Organizations with different missions are pursuing a lab-based strategy to 

foster innovation. For example, organizations—including OPM—use their 

labs as a space where participants can conceptualize and prototype new 

products or processes outside their normal environment. Many also use 

their labs as a teaching space where participants can exchange ideas 

and information through classes, workshops, presentations, or other 

events. Figure 2 shows how innovation labs we surveyed from the public, 

private, and nonprofit sectors share common design elements, and how 

these different organizations generally use their labs for multiple and 

similar purposes. 
Interactive graphic Figure 2: Innovation Labs Share Similar Design Elements and Problem-Solving Methods 

Print instructions To print text version of this graphic, go to appendix II.

CLICK each lab organization's PHOTO to see more information regarding how labs are used then 

CLICK the highlighted NAME to close.

Directions:

Sources: GAO analysis of lab documents and interviews with lab representatives. Photos provided by lab representatives; photo of Harvard i-lab taken by Neal 

Doyle, photo of MindLab taken by MindLab staff, photo of Microsoft New England Research & Development (NERD) Center taken by Dana J. Quigley Photography, photo of the Australian Center for Social Innovation Lab taken by Margaret Frazer, and photo of UNICEF Innovation Lab in Uganda taken by UNICEF CC/A/

SA.

Page 7 GAO-14-306 OPM's Innovation Lab
Page 8 GAO-14-306 OPM's Innovation Lab

Based on OPM documents, the innovation lab’s start-up costs totaled 

approximately $1.1 million including facility upgrades and construction, 

equipment, and training and other personnel costs. (See table 1 for a 

breakdown of costs.) In building the lab, OPM worked with the General 

Services Administration (GSA) and contracted with both design and 

architectural firms to renovate a former storage room in the sub-basement 

of its headquarters building.8 The 3,000 square foot renovated space 

presents an open layout with a meeting area for up to two dozen people 

and is surrounded by breakout areas and team rooms. The physical 

renovation of the facility was completed in March 2012, after the 

installation of final technology equipment, asbestos abatement, and 

enhancements to ventilation and life-safety systems. According to OPM, 

to make the space useful for any purpose, much of the funding for the 

improvements and construction of the space would have been required.

Table 1: Breakdown of OPM’s Lab Start-Up Costs (Fiscal Year 2012)

Item Cost

Facility upgrade/space construction (includes asbestos abatement, etc.) $428,226a

Concept design/construction/furniture 299,712a

Project fee paid to GSA 25,315

Equipment/information technology 247,867

Travel 14,533

Training 75,000

Salaries and benefits 30,000

Supplies 14,410

Total $1,135,063 

Source: OPM.

a

GSA funded $457,820 of the total of these start-up costs.

 8

OPM leases its entire headquarters building and pays GSA at a rate based on square 

footage.

OPM Lab Activities 

Are Intended to Build 

Capacity for 

Innovation and 

Support ProjectBased Problem 

Solving
Page 9 GAO-14-306 OPM's Innovation Lab

OPM officials said that in fiscal year 2013, the lab’s total operating 

budget, including all contracting costs, was $476,000, which supported a 

build up to 5.5 full-time equivalent (FTE) employees over the last seven 

months of fiscal year 2013.9 Officials expect this amount will remain 

stable in the coming fiscal years, proportional to a full fiscal year.

Operational responsibility for the innovation lab has been assigned to 

OPM’s Employee Services Division and is managed by the agency’s 

Deputy Associate Director of Strategic Workforce Planning.10 According to 

OPM officials, since February 2013, the lab has grown from 1 FTE to 

roughly 6 FTEs. Specifically, as of the end of summer 2013, day-to-day 

operations in the lab are carried out by 4 FTE staff members, 1 FTE 

intern, 1 part-time intern, and 1 part-time staff member whose time is 

divided between the innovation lab and OPM’s Resource Management 

Office. A core group of staff from OPM’s Employee Services Division 

have been trained in human-centered design and also contribute up to 15 

percent of their time in the lab. According to OPM officials, the lab 

reached its maximum fiscal year 2013 funding level of 5.5 FTEs in July 

2013. A brief description of each position is provided in table 2.

 9

An FTE is the number of total hours worked divided by the maximum number of 

compensable hours in a work year. For example, if the work year is defined as 2,080 

hours, then one worker occupying a paid full-time job all year would consume one FTE. 

Two persons working for 1,040 hours each would consume one FTE between the two of 

them. 

10OPM’s Employee Services Division provides policy direction and leadership in 

designing, developing, and implementing government-wide human resources systems and 

programs for recruitment, pay, leave, performance management and recognition, 

employee development, work/life/wellness programs, and labor and employee relations. 

They also provide technical support to agencies for human resources management 

policies and practices, including veterans’ employment and evaluation of agencies’ human 

resources programs; the division is also responsible for managing the operation of OPM’s 

internal human resources program. 

OPM’s Lab Is Managed 

and Operated By a Core 

Group of Staff
Page 10 GAO-14-306 OPM's Innovation Lab

Table 2: Description of OPM’s Lab Staff for Fiscal Year 2013

Lab position Job description

Lab director Delivers on and ensures the integrity of the innovation lab’s activities, outputs, and outcomes. Directs the 

program’s strategies, in consultation with the lab team and with the executive director of the lab (the 

Deputy Associate Director of Strategic Workforce Planning).

Lead program analyst Leads lab program evaluation efforts; provides course instruction, design support on projects, and 

facilitation as needed; prepares reports; and assists on managing lab use and other activities.

Graphic designer Produces designs for lab projects and for groups that engage with the lab, including executive staff 

leadership.

Program analyst Manages lab calendar, supports delivery of courses, conducts some facilitation, and supports the lab 

director and program analyst in ongoing review of lab activities.

Student intern Provides ongoing support for lab operations and services, with a focus on program evaluation support 

and outreach to the OPM community. Conducts research, provides administrative support, and 

transcribes outputs.

Student intern Provides ongoing support for lab operations and services, with a focus on supporting lab technology and 

graphic designing. Conducts research, provides administrative support, and transcribes outputs.

Business analyst Supports project scoping as well as delivery of courses, operations, individual projects, and lead 

facilitators; manages budget and other financial-related reporting.

Other staffing A core group of staff from OPM’s Employee Services Division have been trained in human-centered 

design and also contribute up to 15 percent of their time in the lab.

Source: OPM.

OPM has taken a phased approach to developing the lab programming 

(activities taking place in the lab) and the policies governing lab use (such 

as priority-setting policies for lab projects). According to OPM documents, 

each phase has incorporated an element of experimentation, review, and 

a shift in strategy based on lessons learned.

Phase I lasted from March through June 2012. During these first 4 

months after the lab was built, OPM made the space available to the 

OPM workforce for meetings and events. OPM leadership also used this 

time to investigate an appropriate problem-solving approach to pair with 

the lab that would be consistent with approaches used by other labs; they 

determined that a human-centered design approach and curriculum 

would complement OPM employees’ technical expertise and analytic 

competencies. OPM also began to recruit interested staff from the 

Employee Services Division to be trained in human-centered design 

fundamentals: this staff would then support project sessions in the lab as 

part of their collateral duties.

Phase II lasted from July 2012 through March 2013 and consisted mostly 

of facilitated sessions with OPM project teams. During these sessions, 

OPM Employed a Phased 

Approach to Lab 

Development Initially 

Focusing on Capacity 

Building
Page 11 GAO-14-306 OPM's Innovation Lab

Employee Services staff worked with project teams to generate ideas to 

long-standing problems through exercises such as project or strategic 

planning, brainstorming sessions, or stakeholder mapping aimed at 

discussing and testing potential solutions. Topics discussed during these 

sessions involved a variety of initiatives directed at improving OPM 

processes and addressing government-wide human resources 

challenges. OPM officials noted that innovation lab projects have 

included, among others, designing an implementation plan with other 

federal agencies to collect valid, accurate, and timely data on the federal 

cyber security workforce; updating the government-wide strategy for 

veterans recruitment; attracting and retaining individuals with talent in

science, technology, engineering, and mathematics disciplines; and 

specific challenges unique to individual agencies.

Phase III lasted from April through November 2013. In Phase III, OPM 

continued to provide facilitated design sessions and in some cases, 

follow-on coaching to program offices from within OPM and OPM-led 

projects. For example, a facilitated design session included lab staff 

working with the Food and Drug Administration’s (FDA) Battery Working 

Group to more effectively engage with the group’s external stakeholders. 

According to an OPM case study about the lab, eight FDA employees 

attended the Fundamentals of Human-Centered Design course. Following 

the course, lab staff provided planning support for a public workshop with 

over 200 participants from stakeholder groups including medical device 

and battery manufacturers, other regulatory groups, and hospital staff; lab 

staff also attended the Battery-Powered Medical Device workshop to 

support the FDA team in their use of design methods. According to an 

FDA participant, their collaboration with the lab helped them engage in 

stakeholder dialogue that would not have been otherwise possible. OPM 

lab staff also began to offer classes in the lab designed to develop 

mission-critical skills federal workers need to become better problem 

solvers. The OPM lab offers courses such as Human-Centered Design 

Fundamentals, Prototyping in the Public Sector, and Communicating 

Visually, among other topics. These classes are available to OPM staff 

and other federal workers. Staff also made the lab available for federal 

communities of practice to convene. According to lab staff, the lab is 

becoming a hub for a number of standing meetings of a growing 

community of federal innovators and innovation communities of practice. 
Page 12 GAO-14-306 OPM's Innovation Lab

Table 3 presents a summary of OPM’s human-centered design lab 

activities since its inception.11

Table 3: OPM’s Innovation Lab Sessions with Human-Centered Design Programming, Calendar Years 2012 and 2013

Quarter 

(calendar year)

Approximate 

number of 

sessions

Approximate number 

of participating 

organizationsa

Number of

participants per 

meeting (ranges) Examples of topics included

Q1 2012 NAb NAb NAb

Q2 13 25 8-40 Attracting and retaining science, technology, 

engineering, and mathematics (STEM) talent; 

Presidential Management Fellows re-design; 

cyber security

Q3 33 27 8-35 Domestic violence policy; veterans’ hiring and 

retention; human capital reports consolidation 

Q4 12 29 6-40 FDA’s Entrepreneurs in Residence program; 

retirement benefits; closing critical skills gaps 

government-wide

Q1 2013 43 16 4-95 USAJOBS; Chief Financial Officers’ action 

planning; international affairs communications

Q2 66 24 8-40 Interagency community of practice on innovation; 

usability training; new employee on-boarding 

Q3 57 7 8-40 Diversity and inclusion; human-centered design 

workshops; knowledge sharing

Source: GAO analysis of OPM data.

Note: Data as of September 30, 2013. OPM made the lab available to other agencies for a variety of 

purposes, such as targeted design support, classes, and follow-on coaching. Some organizations 

used the lab more than once over the six quarters reported in the table.

a

Participating agencies for both human-centered design and non-design activities included the 

following: Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human 

Services, Homeland Security, Housing and Urban Development, Interior, Justice, Labor, State, 

Transportation, Treasury, and Veterans Affairs; Chief Human Capital Officers Council; Director of 

National Intelligence; Environmental Protection Agency; Executive Office of the President; Federal 

Aviation Administration; Food and Drug Administration; Forest Service; General Services 

Administration; Institute of Museum and Library Services; Internal Revenue Service; National 

Aeronautics and Space Administration; National Institute of Health; National Institute of Standards 

and Technology; National Oceanic and Atmospheric Administration; National Science Foundation; 

Nuclear Regulatory Commission; Office of Management and Budget; Office of Science and 

Technology Policy; Small Business Administration; Social Security Administration; Transportation 

Security Administration; U.S. Agency for International Development; United States Postal Service; 

and the White House Office for Science and Technology. Not all participating organizations were at 

all the meetings.

b

The lab was opened in March 2012, but no programmed human-centered design activity was 

reported during this period.

 11See appendix III for a summary of OPM’s non-design lab activities, which include more 

traditional meetings such as Presidential Innovation Fellows coaching sessions, since the 

innovation lab’s inception.
Page 13 GAO-14-306 OPM's Innovation Lab

Lab staff report that in the future they intend to expand from their sessionbased work and targeted design support projects, such as those 

consultative sessions that took place in the lab during Phase III. While 

some of these more episodic projects may continue to occur, the lab’s 

focus will be on creating and establishing large-scale projects typically 

involving stakeholders from either wholly within OPM or across different 

agencies that are working on crosscutting issues. As discussed later in 

the report, projects appropriate for this design method would have diverse 

users, be more complex, and be called immersion projects. These would 

be the most structured activities undertaken in the lab, characteristically 

being longer-term activities that could take up to six months of intense 

collaboration with project owners and a diverse group of stakeholders.

We identified a common set of challenges that can undermine 

organizations’ efforts to use innovation labs and a set of prevalent 

practices that the organizations employ to address these challenges and 

support their labs’ success and sustainability. OPM has incorporated 

some of these practices, such as pairing a distinct space with a structured 

approach to problem-solving, but has not implemented others, such as 

developing meaningful performance measures. Although OPM has begun 

reaching out to other federal innovators, the agency has not fully 

leveraged the experience of other agencies employing similar 

approaches.

As a prevalent practice for encouraging and supporting greater 

innovation, both the literature and representatives from the organizations 

we reviewed stressed the benefits of pairing a dedicated physical space 

with a structured framework rooted in design-thinking principles. Many of 

the lab representatives said building or establishing a distinct space 

carries an important symbolic value as it signals an organization-level 

commitment to a culture that supports innovation. However, simply 

building a lab is not sufficient to change an organization’s culture; it is 

necessary to also introduce a new framework for problem solving. 

Although these organizations use different terminology to describe their 

selected frameworks, such as agile development and human-centered 

OPM’s Lab Is Similar 

in Mission and Design 

to Other Innovation 

Labs, but OPM 

Needs to 

Systematically 

Evaluate the Lab’s 

Performance

OPM Has Dedicated 

Physical Space, a 

Problem-Solving 

Approach, and Plans to 

Undertake Long-Term 

Immersion Projects
Page 14 GAO-14-306 OPM's Innovation Lab

design, the general principles are similar.12 They include placing users at 

the center of the desired solution—research on successful innovation 

practices shows the importance of engaging customers and 

understanding their needs. Further, they include extensive collaboration 

with relevant stakeholders, experimentation, prototyping, and iterative 

steps to find a solution. A primary objective of this approach is to allow for 

failure in the beginning of the design cycle, so that organizations can 

manage and learn from early mistakes, rather than try to recover from an 

expensive, comprehensive failure upon implementation.

For example, Census and HUD have similar problem-solving frameworks 

for lab use. As figure 3 shows, the Census Center for Applied Technology 

and the HUD Innovation Lab rely on a five-step framework to guide 

innovation in their labs.

Figure 3: Five-Step Framework for Guiding Innovation in the Census and HUD Labs

 12For example, agile software development focuses on keeping software code simple, 

testing often, and delivering functional bits of the application as soon as they are ready. 

The goal of agile development is to build upon small parts as the project progresses, as 

opposed to delivering one large application at the end of the project. See GAO, Software 

Development: Effective Practices and Federal Challenges in Applying Agile Methods, 

GAO-12-681 (Washington, D.C.: Jul. 27, 2012).
Page 15 GAO-14-306 OPM's Innovation Lab

As another example, although CFPB does not have a dedicated physical 

space, it used a similar framework to develop its on-line mortgage 

disclosure form. According to CFPB’s Creative Director for Technology 

and Innovation, designers interacted with end-users including mortgage 

applicants, prototyped different forms, and made refinements based on 

continual feedback before launching the new form. On its website, CFPB 

describes its design process in detail, including prototyping and feedback 

sessions with consumers, lenders, and mortgage brokers.13

OPM’s lab provides a menu of design services to meet the specific needs 

of various projects. The lab’s larger-scale immersion project work will 

involve taking a complex problem through OPM’s problem-solving 

framework, which encompasses steps for problem framing to learning 

about users to analysis to concept development, testing, and rapid 

iterative steps. This problem-solving framework is similar to those 

employed by other innovation labs.

As discussed earlier in this report, similar to other organizations with labs, 

OPM is using its lab for a variety of purposes including as a learning 

space for classes on human-centered design principles and techniques 

and as a meeting space for interagency task teams and communities of 

practice. As originally envisioned in its strategic and performance plans, 

the lab was designed to host a mix of activities rooted in the humancentered design approach, including longer-term design challenges. 

Moreover, OPM lab staff asserted that to gain an organization’s 

confidence and to instill a culture of innovation, it is necessary for a 

successful innovation lab to have an array of sufficiently compelling 

projects that demonstrate how the lab approach can lead to performance 

improvements.

Based on our interviews with public and private sector organizations with 

similar innovation facilities, larger-scale problem-solving projects were 

common activities in their innovation labs’ service portfolios. As an 

example, Denmark’s MindLab has contributed to tackling several pressing 

social issues including simplifying the process for managing claims 

related to industrial accidents and shortening the time before injured 

workers return to the market. Opportunities to showcase a new approach 

to problem solving reduce the likelihood that the lab might lose its 

 13See http://www.consumerfinance.gov/knowbeforeyouowe/#2. 
Page 16 GAO-14-306 OPM's Innovation Lab

distinction as different from a traditional meeting space or classroom 

usually associated with training facilities.

Consistent with what staff from other labs told us, OPM officials said they 

needed the past two years to first introduce agency staff to humancentered design concepts and applications before they could initiate an 

immersion project. Lab staff said this phased approach was necessary for 

several reasons. Targeted design support sessions allowed lab staff to 

expose lab users to design methods and provided opportunities for 

collaboration. These sessions also allowed lab staff to quickly show value 

for program offices in response to a specific need. For example, OPM lab 

staff members were able to help FDA staff plan and engage with over 200 

different stakeholders at a conference. They also said targeted design 

support is a critical way for emerging design practitioners to develop and 

hone their own skills before applying them to a longer term, and higher 

stakes, project engagement. Lab staff said overall these sessions 

benefited both the users of the lab, who developed new skills to take to 

their home offices, such as problem framing and engaging with 

stakeholders, as well as lab employees, who continue to grow and refine 

their human-centered design skills. In a December 2013 document, OPM 

staff stated they intend to create and establish these longer-term 

immersion projects and evaluate their impact during the next phase of the 

lab’s development.

Measuring the long-term outcomes of innovation labs is a prevalent 

practice for building acceptance and demonstrating the value of the labs. 

Consistent with our literature review, several representatives we 

interviewed from other innovation labs concurred with the director of 

innovation at Denmark’s innovation lab, MindLab, who said that 

innovation labs need to know how much they are spending and their 

outcomes. According to the director, the labs must also be able to 

attribute where the change happens based on their work. In addition, lab 

staff must be prepared to present a narrative of their work. He 

acknowledged that innovation labs are risky because they look different,

and they have a different focus than other government entities. The 

director said that, as a result, innovation lab officials need to show where 

the funds are going along with the benefits and results of those 

investments.

Representatives from newer labs—i.e. those operating less than three 

years—stated they primarily rely on output measures to gauge their initial 

efforts such as number of users, ways in which the lab is being used, 

OPM Has Not Fully 

Developed Meaningful 

Performance Measures
Page 17 GAO-14-306 OPM's Innovation Lab

classes or events held in the lab, and anecdotal evidence. Developing 

outcome measures is more challenging for several reasons. Appropriate 

outcome measures are often not obvious at the onset of a project. 

Moreover, agencies may not have appropriate measures or baseline data 

when they start using an innovation lab as a problem-solving tool, and the 

role of the lab in driving a successful innovation may not always be clear.

Given these challenges to accurately measuring innovation and the value 

of an innovation lab, lab managers from labs that have been operating for 

a longer period of time told us they focus on developing meaningful 

milestones and measures applicable to different phases of the innovation 

lifecycle, such as problem generation, idea generation, and skills 

development. For example, the UNICEF Innovation Unit—which has been 

helping member-country offices set up innovation labs since 2006—and 

several European initiatives developed a set of benchmarks intended to 

help them measure the value of public sector labs and identify ways in 

which the lab’s performance can be improved.14 The benchmarks 

UNICEF developed span across six categories, such as problem 

definition and idea generation, internal and external collaboration, and 

secondary effects. Within each category, they include a list of questions 

intended to assess their strengths and weaknesses. For example, they 

want to know whether labs are helping employees define problems and 

generate ideas, strengthen internal collaborations, and build external 

partnerships. They also measure the extent to which work done in the 

labs results in new team or staff capacity, excitement and goodwill toward 

the organization, and an increase in leverage and influence in their field. 

OPM is undertaking a similar effort to establish benchmarks that will help 

lab staff gauge the extent to which lab users are learning and applying 

many of these same skills, but the lab is not mature enough to have 

results.

OPM documents state that the goal of OPM’s innovation lab is to provide 

federal workers with 21st century skills in design-led innovation, and the 

intended purpose of the lab is to provide a physical space for projectbased problem solving. The documents also note that the value of the lab 

 14For more information on the work that Nesta, an innovation organization in the United 

Kingdom, and the Danish Agency for Science, Technology and Innovation are doing to 

evaluate innovation in the public sector, see Hugo Thenint, “Innovation in the Public 

Sector,” Mini Study 10 (INNO Global Review of Innovation Intelligence and Policy Studies,

Feb. 2010).
Page 18 GAO-14-306 OPM's Innovation Lab

can be measured, in part, by how well it helps develop the mission-critical 

competencies to improve the federal workforce’s ability to solve problems 

and deliver results.

In its strategy document, OPM laid out the following high-level goals for 

the innovation lab:

• Employees assigned to the innovation lab should go back to their 

home organization with an understanding of, and an appreciation for, 

the power of innovative approaches to problem solving.

• Employees should be equipped to implement similar methodologies in 

their home organizations on future projects.

• As the innovation lab matures, and as more and more projects are 

completed, the notion of using innovation to tackle complex problems 

will gain traction across the organization.

• Eventually, leaders and employees across OPM will vie to get their 

issues sent to the innovation lab for resolution.

• This in turn will contribute to a decrease in organizational silos, and a 

concurrent increase in cross-organizational teams addressing one 

organization’s issues.

In the same document, OPM officials also described an evaluation 

strategy resembling an agile approach.15 Specifically, OPM described 

these goals as moving targets which would be achieved through an 

evolving and self-correcting process. Lab staff immediately started to 

track lab activities and outputs, such as number of participating people 

and agencies, and how participants used the lab, such as consultative 

sessions, follow-on coaching, training classes, or as a meeting space. 

Five months after the lab opened, they also started to survey users who 

participated in day-long facilitated sessions. For example, there was a 

one-page evaluation that asked respondents to rate the appropriateness 

 15The agile approach was first articulated in 2001 and is still used today. With this 

approach, project managers establish their higher-level, strategic goals but are more likely 

to respond to change rather than follow a set plan. This approach recognizes that longterm project plans are not adaptable, whereas short-term plans provide more flexibility in 

responding to change. It is considered more effective to devise a detailed plan for a 

shorter period and a general plan for a longer period.
Page 19 GAO-14-306 OPM's Innovation Lab

of the environment and quality of the facilitators. The surveys also asked 

whether users would recommend the lab to colleagues and whether 

human-centered design problem-solving tools can be used as an effective 

tool government-wide. The responses were generally positive—about 82 

percent of respondents (84 out of 103) said they would recommend the 

lab to someone else, providing a baseline for subsequent survey findings. 

According to lab staff, they periodically reviewed the available data and 

adjusted their strategy for operating the lab.

Starting in March 2013, a year after the lab opened, OPM lab staff began 

work on a program evaluation framework to more systematically measure 

the lab’s progress toward meeting their overarching goals. To evaluate 

the extent to which lab participants are learning and applying innovative 

approaches, lab staff intends to measure the lab’s performance along 

three overarching categories: service experience, skill development, and 

project outcome. According to the framework, resources dedicated to 

evaluation efforts will reflect the resources needed to host lab-sponsored 

events. Episodic events such as consulting sessions will correspond to a 

“light-touch” follow-up effort, such as immediately surveying all 

participants on their session experience and skills development. More 

long-term, resource-intensive efforts such as immersion projects will 

employ a more robust follow-up effort that, in addition to assessing the 

session experience and skills development, will also address projectspecific outcomes. Collection of assessment data in all three areas will 

include the administration of surveys to participants both before and right 

after a session, and some services will involve the administration of 

surveys to participants before a service and subsequent periodic checkins. Depending on the nature of the lab session, information on skill 

development and outcomes will also be obtained from session clients in 

pre-session scoping conversations and periodic, post-session check-ins 

using either surveys, or interviews. For one type of session, assessment 

of participant skill development will also include a survey of participant 

supervisors.

Lab staff has used a series of surveys to measure participant experience

and skills development, and to capture specific project-related outcomes 

for the different services they offer. However, the survey instruments are 

unlikely to yield data that would be of sufficient capacity, credibility, and 

relevance to indicate the nature and extent to which the lab is achieving 

what it intends to accomplish or its value to those who use the lab space. 

Although there are several items across all surveys that are reasonably 

aligned with generally accepted questionnaire and item design principles, 

there are limitations associated with many items where language is 
Page 20 GAO-14-306 OPM's Innovation Lab

ambiguous, where the intent of the question is not clear, and directions 

are lacking. For example, phrases such as “changed behavior” or 

“tangible outputs you can move forward” are open to numerous 

interpretations and are likely to engender an array of responses that 

range from being relevant to not at all relevant or relatable to the 

purposes or objectives of the session. In addition, some of the items may 

be more likely to engender responses with a greater likelihood of being 

subject to a respondent’s social desirability bias. For example, the 

respondent may want to provide answers that are socially desirable, 

maintain the status quo, or make a good impression.

While some customization is to be expected, the surveys did not indicate 

any approach to evaluate some core aspects of the lab and its value 

using a consistently presented set of the same questions. For example, 

the question asking participants about the likelihood that they would 

recommend the lab to someone else is the type of item that could, with 

revision, be incorporated in all of the surveys. Analyses of a core set of 

items by type of lab event or service would enable lab staff to discern and 

compare where participants were more and less engaged in lab activities 

and curricula. Consequently, these survey instruments and the items on 

them may be susceptible to various types of question and respondent 

bias and could, when the responses are analyzed, produce results that 

would be difficult to interpret or link to expected participant effects, or to 

the intent or activities of the workshop session. Moreover, lab staff has 

not developed outcome measures or milestones related to customer 

experience and skills development.

The evaluation framework being developed by OPM does not include 

interim performance targets or measures. Best practices state that new 

initiatives benefit when managers set time-bounded, quantifiable interim 

goals, establish related performance measures, collect data, and use that 

information to assess and adjust their performance.16 To evaluate the 

overall performance of MindLab, the director said he develops an annual 

work plan, which describes the number and types of projects and other 

activities the lab will undertake, as well as the relative resource allocation 

 16GAO, The Results Act: An Evaluator’s Guide to Assessing Agency Annual Performance 

Plans, GAO/GGD-10.1.20 (Washington, D.C.: April 1998) and GAO, Streamlining 

Government: Questions to Consider When Evaluating Proposals to Consolidate Physical 

Infrastructure and Management Functions, GAO-12-542 (Washington, D.C.: May 23, 

2012).
Page 21 GAO-14-306 OPM's Innovation Lab

to those projects and activities. He said his staff also conducts an annual 

review of the budget and actual expenditures with the board. OPM lab 

staff has been tracking outputs—such as number of participants and 

number and type of activities—meaning that they have baseline data 

which could inform realistic, meaningful targets and measures related to 

lab use and activities for the upcoming year. Although they continue to 

refine their surveys, they could use the results from earlier versions to 

establish targets and measures related to customer experience and skills 

development. Meaningful measures or milestones could help them 

assess their progress toward improving participants’ ability to solve 

problems and accurately measure the effect of working in the lab on 

services, products, and processes.

As mentioned previously, the lab plans to host the more resourceintensive immersion projects. To demonstrate that the lab is operating as 

originally intended, evaluation plans will be needed for specific immersion 

projects that can help track cost-benefits and performance improvement 

outcomes. OPM stated that evaluation plans will be prepared for each 

immersion project to account for project outcomes. They indicated that 

they wanted to host their first immersion project within the next several 

months.

Another prevalent practice we identified included leveraging other 

innovation labs’ efforts to try to increase the value of the lab approach. 

Studies show that information sharing and interorganizational networks 

can be a powerful driver supporting innovation. One study showed that 

interorganizational networks of innovators help members develop new 

products at a faster rate with lower investment commitments, due in large 

part to the information sharing that takes place.17 Sharing information can 

help mitigate the risks and uncertainty that typically characterize 

innovation ventures. Best practices state the importance of establishing 

channels of communication and other mechanisms that facilitate 

knowledge-sharing and building networks of like-minded communities to 

 17Jennifer Lewis Priestly and Subhashish Samaddar, “Information Sharing in Innovation 

Networks,” Encyclopedia of Information Science and Technology, Second Edition 

(Hershey, PA: IRM Press, 2006). 

OPM Has Started Efforts 

to Connect with the 

Federal Innovation 

Community, but Has Not 

Fully Leveraged Other 

Agencies’ Efforts
Page 22 GAO-14-306 OPM's Innovation Lab

help agencies achieve crosscutting objectives.18 For example, the Census 

Bureau’s Chief Technology Officer suggested a way in which innovation 

leaders could share information and pool resources. Specifically, instead 

of each agency creating its own technology innovation lab with its own 

hardware, software, and associated maintenance, they could use a 

common innovation infrastructure service in the public cloud.19 Every 

agency could still have their own branded offering and could still provide 

access in their own facility or at their regional offices. However, an 

outside vendor could provide the infrastructure. For example, if an agency 

wanted to experiment with some unique visual analytic tools, they could 

purchase what they need on a subscription service; this would eliminate 

agencies buying all the tools themselves.

While labs provide a physical space where innovators can convene, 

federal agencies are not fully aware of their growing community. As of 

June 2013, OPM was unaware that other agencies such as Census, 

HUD, and NASA were pursuing a lab approach to promote innovation. 

Moreover, the lab directors at these agencies were not aware or only 

marginally aware of OPM’s lab and its resources or other federal 

innovation labs. OPM’s efforts to develop an innovation lab occurred 

around the same time or pre-dated those of other agencies we 

interviewed.

According to OPM officials, during its first year of operations, OPM lab 

staff focused their efforts on promoting awareness of the lab and its 

resources internally to OPM staff. In their second year, OPM lab staff 

planned more activities intended to promote the lab and its resources 

externally to connect with federal agencies’ innovation efforts. Staff noted 

the OPM lab is the hub of various interagency networks of innovation 

practitioners. For example, an interagency community of practice on idea 

generation meets in the lab on a monthly basis. OPM’s lab staff also 

 18GAO, Managing for Results: Key Considerations for Implementing Interagency 

Collaborative Mechanisms, GAO-12-1022 (Washington, D.C.: Sept. 27, 2012); 

Interagency Collaboration: State and Army Personnel Rotation Programs Can Build on 

Positive Results with Additional Preparation and Evaluation, GAO-12-386 (Washington, 

D.C.: Mar 9, 2012) and Results-Oriented Government: Practices That Can Help Enhance 

and Sustain Collaboration among Federal Agencies, GAO-06-15 (Washington, D.C.: Oct. 

21, 2005). 

19A public cloud is a set of computers and computer network resources based on the 

standard cloud computing model, in which a service provider makes resources, such as 

applications and storage, available to the general public over the Internet.
Page 23 GAO-14-306 OPM's Innovation Lab

reported that they host weekly trainings in the lab on best practices, 

including webinars about measuring the success of enterprise-level 

design efforts and the value of visualizing information. These training 

sessions include case study presentations from other federal agencies, 

such as GSA, and non-federal entities. In addition, OPM has shared best 

practices with other public sector design labs across the globe by 

participating in a number of conferences. OPM is also collaborating with a 

current Presidential Innovation Fellow, who is building an innovation 

toolkit. Although projects in the lab are currently managed and for the 

most part delivered by OPM employees, staff noted that they are 

increasingly looking to leverage detailees, short-term assignments, and 

other ways to harness the potential of talent from other agencies. OPM 

staff said they also give regular tours of their innovation lab for other 

government entities already supporting innovation initiatives or 

developing them. In addition to these activities, OPM hosted a convening 

of federal innovators to compare various agencies’ innovation 

communication efforts across the agencies.

Several federal officials we interviewed said they would welcome the 

opportunity to communicate as the need arose with a community of peers 

to exchange information and ideas and trouble-shoot problems related to 

the start-up and maintenance of their labs. For example, CFPB’s Creative 

Director for Technology and Innovation said it would be helpful to find out 

what other bureaus and departments are doing to incorporate design 

principles, so that she could exchange ideas and information. An official 

from NASA’s Swamp Works noted that it would be beneficial to show that 

others in the federal sector are also looking at innovation labs. To that 

end, simply knowing the innovation community exists and how agency 

staff leading innovation efforts can initiate a conversation related to a 

specific topic would likely be beneficial and would help avoid the risk of a 

fragmented innovation community.

Because innovation necessarily entails culture change, experimentation, 

periodic setbacks, and often resource investments, another prevalent 

practice necessary to sustain a lab includes leadership support. 

Innovation labs are one tool that agencies can use to foster innovation. 

Agency officials and lab directors we interviewed said leaders must be 

willing to embrace experimentation within the lab and understand that 

smart failures—failures that result from trial and error, where the 

alternative would be to do something truly risky due to lack of evidence—

are part of the design process. For example, the Census Bureau’s Chief 

Technology Officer noted that support by Census Bureau leadership is 

OPM’s 2014 through 2018 

Strategic Plan Indicates It 

Intends to Use the Lab to 

Advance OPM Priorities
Page 24 GAO-14-306 OPM's Innovation Lab

critical to ensure staff participation and the continued availability of funds 

to drive innovation in its Center for Applied Technology lab. Other lab 

directors highlighted several strategies they use to balance the risks and 

failures that accompany a problem-solving methodology rooted in a more 

experimental approach. These include accelerated timelines of three to 

six months, which allows organizations to quickly shelve projects that are 

without merit. Some lab leaders also noted that a quick win or early 

success can give new labs the underlying support they need to take on 

riskier projects.

In March 2014, OPM released its 2014 through 2018 strategic plan, which 

states that the agency plans to seek new, innovative ways to accomplish 

its work of advancing human resource management in the federal 

government. In the strategic plan, OPM indicates that, among other 

things, it intends to use the innovation lab and human-centered design 

methods to address OPM’s operational challenges.

For OPM and the rest of the federal government, finding more efficient 

and effective ways of doing business to help meet rising citizen demands 

for public services is critical, particularly in an era of continued fiscal and 

budgetary constraints. OPM’s innovation lab is one such tool intended to 

give rise to solutions of complex problems facing the federal government.

Consistent with other innovation labs, development of performance and 

outcome measures, tools to assess performance, and further leveraging 

the experience of other organizations undertaking similar efforts will also 

be critical. Having clear and specific outcome measures will help OPM 

track and evaluate the extent to which the lab is meeting its original intent 

and over time, to make any necessary adjustments. Otherwise, OPM’s 

innovation efforts may not be able to demonstrate the types of results 

initially envisioned.

We recommend that the Director of OPM take the following actions to 

help substantiate the lab’s original goals of enhancing skills in innovation 

and supporting project-based problem solving:

• Direct lab staff to develop a mix of performance targets and measures 

to help them monitor and report on their progress toward lab goals. 

Output targets could include number and type of lab activities over the 

next year. Outcome targets and measures should correspond to the 

Conclusions

Recommendations for 

Executive Action
 

Page 25 GAO-14-306 OPM's Innovation Lab

lab’s overarching goals to build organizational capacity to innovate 

and achieve specific innovations in concrete operational challenges.

• Direct lab staff to review and refine the set of survey instruments to 

ensure that taken as a whole, they will yield data of sufficient 

credibility and relevance to indicate the nature and extent to which the 

lab is achieving what it intends to accomplish or is demonstrating its 

value to those who use the lab space. For example, lab staff should 

consider the following actions:

• Developing a standard set of questions across all service 

offerings.

• Revising the format and wording of existing questions related to 

skills development to diminish the likelihood of social desirability 

bias and use post-session questions that ask, in a straightforward way, about whether, or the extent to which, new 

information was acquired.

• Replacing words or phrases that are ambiguous or vague with 

defined or relevant terminology (e.g., terms actually used in the 

session) so that the respondent can easily recognize a link 

between what is being asked and the content of the session.

• Direct lab staff to build on existing efforts to share information and 

knowledge within the federal innovation community. For example, 

OPM lab staff could reach out to other agencies with labs such as 

Census, HUD, and NASA’s Kennedy Space Center to share best 

practices and develop a credible evaluation framework.

We provided a draft of this report to the Director of OPM for review and 

comment. The director provided written comments, which we have 

reprinted in appendix IV. In summary, OPM generally concurred with our 

recommendations and described ongoing and planned steps to refine 

evaluation efforts and further leverage other federal innovation labs.

For the recommendations on evaluating performance, the director 

described a competency-based skills gap pilot the lab is undertaking, 

based on targets from pre- and post-testing of participants in lab 

activities. We acknowledge that this is an important step in developing 

performance measures, and OPM will also need targets and measures to 

demonstrate the lab’s value in achieving specific innovations in concrete 

operational challenges. 

Agency Comments 

and Our Evaluation
Page 26 GAO-14-306 OPM's Innovation Lab

For the recommendation on leveraging other federal agency innovation 

efforts, the director noted OPM’s work seeking out information and 

contacts from other innovation endeavors, including lab-based ones. We 

acknowledge OPM’s more recent emphasis in this area, including 

participating in an interagency community of practice on innovation. 

Federal officials we interviewed said they would welcome the opportunity 

to communicate as the need arose with a community of peers.

To clarify, the report recognizes sustained organizational leadership as a 

prevalent practice for the success of innovation labs. However, this was 

not a specific report recommendation, but an acknowledgement regarding 

the general role leadership plays in ensuring the success of innovation 

labs. In her response, the director stated that OPM recently released its 

2014 through 2018 strategic plan, which she said demonstrates OPM

leadership’s commitment to the advancement of work in the lab. 

Accordingly, we updated the report to reflect the most current information 

available at the time of our publication.

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of this report to the Director 

of OPM and appropriate congressional committees. This report will also 

be available at no charge on our website at http://www.gao.gov.

If you or your staff members have any questions about this report, please 

contact me at (202) 512-4749 or bagdoyans@gao.gov. Contact points for 

our Offices of Congressional Relations and Public Affairs may be found 

on the last page of this report. GAO staff members who made major 

contributions to this report are listed in appendix V.

Seto J. Bagdoyan

Acting Director

Strategic Issues 
Appendix I: Objectives, Scope, and 

Methodology

Page 27 GAO-14-306 OPM's Innovation Lab

This appendix provides information on the scope of work and the 

methodology used to (1) describe the Office of Personnel Management 

(OPM) innovation lab’s start-up and operating costs, staffing and 

organization, activities, and policies governing the lab’s use, and (2) 

assess how OPM’s innovation lab compares to other organizations’

innovation labs, including how it uses benchmarks and associated metrics 

and how it addresses potential challenges to innovation.

To address the first objective, we reviewed documentation and met 

several times with OPM staff overseeing the lab and its activities. We 

reviewed the lab’s construction and operations’ budget, including the 

funding sources for the lab, and interviewed agency officials 

knowledgeable about the lab’s budget. Based on interviews and e-mail 

exchanges with knowledgeable OPM and General Services 

Administration staff and reviewed documents, we found OPM’s lab 

expense data to be sufficiently reliable for the purposes of our report. We 

reviewed spreadsheets maintained by lab staff tracking lab outputs, such 

as workshops hosted in the lab and number of attendees.1 We also 

reviewed lab performance materials such as the lab’s performance plan, 

user surveys, and the results of those surveys. Survey specialists in our 

Center for Design, Methods, and Analysis reviewed the lab user surveys 

using internal review guidance that is typically performed on draft GAO 

surveys as part of our development process and required before 

deployment of a survey. In addition to reviewing with lab staff the 

documents they provided us, we interviewed them about OPM’s process 

for identifying and selecting a lab strategy, lab staff’s approach to 

implementing a human-centered design curriculum, and their goals for the 

lab.

To address the second objective, we conducted a detailed literature 

search of material from academic institutions, global management 

consultants, professional associations, think tanks, news outlets, and 

various other organizations. We also reviewed literature documenting 

public, private, and academic innovation efforts and associated positive 

and negative outcomes. Our literature search helped us identify 

benchmarks and associated metrics applicable to the development and 

use of innovation facilities in the public, private, and nonprofit sectors. We 

 1

Outputs as used here include measurement of the lab’s services and level of participation 

in lab activities. 

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 28 GAO-14-306 OPM's Innovation Lab

also interviewed OPM lab staff on how they intend to identify outcomes—

such as cost reductions, performance improvements, or other results—

from projects undertaken by OPM since the inception of the lab.

We used the findings from our literature review to identify organizations 

with innovation facilities having a dedicated physical space and using 

problem-solving methods similar to OPM’s lab. We selected a mix of 11 

public, nonprofit, and private organizations to visit or interview. In 

addition, we met with an official from the Consumer Financial Protection 

Bureau (CFPB). While CFPB lacks a dedicated innovation lab, the 

agency has a reputation among federal agencies as a leader in innovative 

website development. Table 4 lists the organizations we visited in person 

or interviewed their representatives by telephone.

Table 4: List of Innovation Labs with Facilities Similar to OPM

Innovation lab Sector

Census Bureau’s Center for Applied Technology Public

Housing and Urban Development’s Innovation Lab Public

National Aeronautics and Space Administration Kennedy Space Center’s 

Swampworks

Public

Denmark’s MindLab Public

UNICEF’s Innovation Labs Nonprofit

Harvard University’s Harvard i-lab Nonprofit

The Australian Centre for Social Innovation (TACSI) Nonprofit

Microsoft’s New England Research & Development (NERD) Center Private

Fidelity Investment’s Fidelity Center for Applied Technology (FCAT) Private

Staples’ Velocity Lab Private

Deloitte’s Govlab Private

Source: GAO.

At every lab we visited or contacted, we interviewed lab representatives 

about the history of the lab, including why they decided to pursue a lab 

strategy; how the lab is used; the protocols for engaging participants; how 

lab directors measure the performance of the lab; challenges to 

promoting innovation within the organization; and practices for addressing 

those challenges. Based on our literature search, we identified common 

challenges that can hamper organizations’ efforts to use labs as 

innovation vehicles and prevalent practices that can support labs’

success and sustainability. In addition, we reviewed our interview records 

to identify commonly recurring challenges and prevalent practices that 
Appendix I: Objectives, Scope, and 

Methodology

Page 29 GAO-14-306 OPM's Innovation Lab

can support labs’ success and sustainability. We verified that the 

challenges and prevalent practices we identified during our literature 

search were also those more often cited during the interviews.

We also interviewed representatives from two management consultancies 

that promote problem-solving approaches rooted in design-thinking 

principles, IDEO and Luma. OPM contracted with Luma to help design 

the lab and implement human-centered design programming. We spoke 

with their representatives to understand the challenges their clients face 

in changing organizational culture and the benchmarks and metrics they 

advise their clients to adopt to measure the performance of new labs and 

problem-solving methods.

In addition, we interviewed officials from three public-sector 

organizations—San Francisco Mayor’s Office of Civic Innovation, 

Canada’s Public Policy Forum, and United Kingdom Behavioural Insights 

Team—that are pursuing strategies to promote innovation in their 

organizations but opted not to build innovation labs. They spoke to us 

about the challenges that prevented them from building labs and the 

steps they are taking to incorporate human-centered design-like problemsolving methods without a physical lab.

We conducted this performance audit from July 2013 to March 2014 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.
Appendix II: Innovation Labs Share Similar 

Design Elements and Problem-Solving 

Methods

Page 30 GAO-14-306 OPM's Innovation Lab

This table contains the same text portrayed in figure 2 and shows how 

innovation labs we surveyed from the public, private, and nonprofit 

sectors generally use their labs for multiple and similar purposes.

Organization Sector Purpose of lab How is it used

Lab@OPM Public To help deliver on President 

Obama’s vision of a more 

effective and efficient government 

for the American people by 

supporting a government-wide 

community of innovators.

Dedicated space where OPM and interagency 

teams can take a problem through a full 

design cycle. This includes problem framing, 

learning about users, analysis, concept 

development, testing, and rapid iteration.

Capacity building—lab hosts classes and 

workshops.

Meeting space for OPM staff and other 

federal workers and interagency communities 

of practice.

Census Center for Applied Technology 

(CAT) 

Public To provide a “safe zone” where 

Census staff can explore new 

technology solutions without 

impact to production operations.

Dedicated space where lab and other Census 

staff can develop, test, and implement ideas 

into production outside the standard 

production environment

Capacity building—lab hosts presentations of 

new technologies and solutions.

NASA Kennedy Space Center’s (KSC)

Swampworks

Public To provide a dedicated space 

where NASA KSC engineers can 

quickly resolve problems related 

to deep-space exploration. 

Space where 20 NASA engineers and 

scientists prototype and test emerging 

technologies.

Swamp Works has contracts with other NASA 

centers worth about $7 million a year.

Housing and Urban Development 

(HUD) Innovation Lab

Public Provide a space that looks 

different from the traditional office 

environment, where HUD 

employees can accelerate the 

development of solutions more 

efficiently than other available 

approaches.

Dedicated space where HUD lab staff and 

mission area leads can develop, test, and 

implement ideas into production within a 

compressed timeframe.

Denmark’s MindLab Public Provide a neutral space for 

government ministries to work

with citizens and businesses to 

create new solutions for society.

Ministry officials from Danish Ministries of 

Business and Growth, Education, and 

Employment use the MindLab space and 

resources to take a problem through the full 

design cycle.

Capacity building—lab hosts classes, 

conferences, and workshops.

UNICEF Innovation Labs Nonprofit Help the organization become 

more flexible, agile, and better 

prepared for global changes by 

providing spaces where 135 

country offices can collaborate 

with local partners.

Dedicated spaces where UNICEF country 

office staff and their local partners can take a 

problem through a full design cycle.

Appendix II: Innovation Labs Share Similar 

Design Elements and Problem-Solving 

Methods
Appendix II: Innovation Labs Share Similar 

Design Elements and Problem-Solving 

Methods

Page 31 GAO-14-306 OPM's Innovation Lab

Organization Sector Purpose of lab How is it used

Harvard i-lab Nonprofit Provides space, practical skill 

building, and programming for 

Harvard students, faculty, staff, 

alumni, and others engaged in 

new ventures, nonprofit creation, 

product or service innovation, 

small business development, and 

related educational and research 

activities.

Incubator, workspace, and programming for 

start-up ventures involving Harvard students, 

and their partners.

Teaching space for Harvard students.

Meet-up space for local community.

The Australian Centre for Social 

Innovation (TACSI) 

Nonprofit To tackle tough social issues, 

such as family breakdown and 

social inequality, by building 

Australia’s social innovation 

capability.

Laboratory for co-designing new social 

programs for vulnerable populations—this 

includes generating ideas, conducting 

ethnographic research, and prototyping 

potential solutions.

Meet-up space: TACSI hosts events, 

workshops, and conferences for social 

change community.

Capacity building for other social innovators.

Staples Velocity Lab Private Provide a dedicated physical 

space in the heart of the 

Cambridge, MA tech sector that 

can accelerate the speed of 

product development, increase 

collaboration, and attract new 

talent.

Office space for moving new and emerging 

technologies through the development 

pipeline.

Meet-up space for local tech community.

Recruitment tool for top tech talent.

Microsoft’s New England Research & 

Development (NERD) Center

Private To build a strong and permanent 

research and development 

presence in Cambridge, MA

where Microsoft researchers and 

programmers can build 

relationships with local 

universities, biotech, and 

healthcare companies.

Office space for moving new and emerging 

technologies through the development 

pipeline.

Meet-up space for local tech community.

Recruitment tool for top tech talent.

Fidelity Center for Applied Technology 

(FCAT)

Private Provide a dedicated physical 

space where Fidelity executives 

explore how emerging 

technologies can improve 

products and processes for 

internal business units. 

Laboratory where FCAT staff can identify 

solutions and develop new products and 

processes for Fidelity business units.

Hosts conferences, workshops, and social 

events related to technological and social 

innovation.

Deloitte GovLab Private Provide a dedicated physical 

space that exemplifies how 

environment can promote 

creativity and collaboration where 

Deloitte’s future leaders can 

hypothesize, research, and test 

new ideas.

Leadership development institute for 

Deloitte’s highest performing consultants

A think tank where fellows develop innovative 

yet practical strategies governments can use 

to transform the way they deliver their 

services and prepare for the challenges 

ahead

Capacity building—GovLab educates Deloitte 

account teams on these emerging trends.

Source: GAO analysis of lab documents and interviews with lab representatives.
Appendix III: Office of Personnel 

Management’s Innovation Lab Non-Design 

Programmed Activities, Calendar Years 2012-

13

Page 32 GAO-14-306 OPM's Innovation Lab

Quarter 

(calendar 

year)

Approximate 

number of 

meetings

Approximate 

number of 

participating 

organizationsa

Number of

participants per 

meetingb (ranges) Examples of topics included

Q1 2012 7 12 9-25 Recruitment policy and outreach; civil service innovation 

cohort; Chief Learning Officers’ meeting

Q2 37 8 8-30 Acquisition planning; retirements calculator challenge; 

leadership and talent management

Q3 7 2 unknown Performance improvement review; Presidential 

Innovation Fellows coaching session

Q4 21 4 unknown MyGov platform; RFP-EZ; health exchanges 

Q1 2013 14 4 5-47 HealthCare.gov summit; retirement service summit; 

Science, Technology, Engineering, and Mathematics 

(STEM) event 

Q2 4 3 10-15 State e-Diplomacy; speaker series; U.S. Department of 

Agriculture brainstorming

Q3 21 6 5-25 Graphic Facilitators Community of Practice; State’s 

global partnership initiative retreat; USAJOBS training 

session with Environmental Protection Agency students 

Source: GAO analysis of OPM data.

Note: Data as of September 30, 2013.

a

Participating agencies for both human-centered design and non-design activities included the 

following: Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human 

Services, Homeland Security, Housing and Urban Development, Interior, Justice, Labor, State, 

Transportation, Treasury, and Veterans Affairs; Chief Human Capital Officers Council; Director of 

National Intelligence; Environmental Protection Agency; Executive Office of the President; Federal 

Aviation Administration; Food and Drug Administration; Forest Service; General Services 

Administration; Institute of Museum and Library Services; Internal Revenue Service; National 

Aeronautics and Space Administration; National Institute of Health; National Institute of Standards 

and Technology; National Oceanic and Atmospheric Administration; National Science Foundation; 

Nuclear Regulatory Commission; Office of Management and Budget; Office of Science and 

Technology Policy; Small Business Administration; Social Security Administration; Transportation 

Security Administration; U.S. Agency for International Development; United States Postal Service; 

and the White House Office for Science and Technology. Not all participating organizations were at 

all the meetings.

b

The ranges provided are estimates based on OPM data.

Appendix III: Office of Personnel Management’s 

Innovation Lab Non-Design Programmed 

Activities, Calendar Years 2012-13
Appendix IV: Comments from the Office of 

Personnel Management

Page 33 GAO-14-306 OPM's Innovation Lab

Appendix IV: Comments from the Office of 

Personnel Management
Appendix IV: Comments from the Office of 

Personnel Management

Page 34 GAO-14-306 OPM's Innovation Lab
Appendix V: GAO Contact and Staff 

Acknowledgments

Page 35 GAO-14-306 OPM's Innovation Lab

Seto J. Bagdoyan, (202) 512-4749 or bagdoyans@gao.gov.

In addition to the contact named above, Thomas Gilbert, Assistant 

Director, and Judith Kordahl, Analyst-in-Charge, supervised the 

development of this report. Jessica Nierenberg and Anthony Patterson 

made significant contributions to all aspects of this report. Other important 

contributors included Thomas Beall, Karin Fangman, Donna Miller, and 

Robert Robinson.

Appendix V: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(451049)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.