appl.nasa.gov

ASK Magazine • Issue Thirteen

AUGUST 2003

INTERVIEW

Dr. Charles Pellerin

Academy Sharing Knowledge

The NASA source for project management
appl.nasa.gov

Table of Contents

ASK 13 August 2003

STORIES

6 Smart Buying

How do you buy the best technology, when you don’t even know what technology is out there?

BY STEVEN A. GONZALEZ 

10 As Small as Possible

I showed the prototype to the customer and said, “This is the smallest switch we can make”

BY SCOTT TIBBITTS

16 Off the Charts

We came up with two basic concepts for landing: one was a traditional approach, the other was

a wild idea BY BRIAN K. MUIRHEAD 

18 Dropping in on Mars 

Airbags may seem like simple, low-tech products, but it was eye-opening to discover just how

little we knew about them BY TOMMASO P. RIVELLINI

22 Supplying the Giant

We partnered with a small company making baseball bats and golf club shafts. They had never

built a military product BY LARRY A. LAWSON

26 Proof of Concept

Before anyone would consider a design incorporating inflatable wings, you had to know the

concept was sound BY JEFFREY BAUER

FEATURES

28 Prototyping “Rope-a-dopes” and Other Pitfalls

It’s OK for a prototype to fail. In fact, if there isn’t a non-trivial likelihood of a failure,

then why build a prototype at all? BY TERRY LITTLE

30 An Old Dog and New Tricks

The young project manager was eager to experiment. I had never tried out this particular

concept before, but it intrigued me BY W. SCOTT CAMERON

SPECIAL FEATURE: THE IDEO WAY 

12 Right - Rapid - Rough

Instead of spending your time and resources speculating solutions and analyzing the problem,

spend your time solving it BY DR. CRAIG LAWRENCE

PRACTICES

24 The Storyboard’s Big Picture

Graphic, sequential depictions give a quick project overview while breaking down the product

into its major components BY CHERYL A. MALLOY AND WILLIAM COOLEY

INTERVIEW

32 ASK Talks with Dr. Charles J. Pellerin

A former Director of Astrophysics for NASA considers what the Hubble Space Telescope failure

taught him about leadership

IN THIS ISSUE • 3

Learning About ASK

BY TODD POST

REVIEW BOARD • 4

FROM THE DIRECTOR’S DESK • 5

Playing with Prototypes

BY DR. EDWARD HOFFMAN

LOOP • 38

Book Reviews

LETTER FROM THE 

EDITOR-IN-CHIEF • 40

Letting Go of 

“Once and For All”

BY DR. ALEXANDER LAUFER

ON THE COVER Engineers test a prototype 

of airbags used for landing on Mars.
2 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

WELCOME TO THE ACADEMY OF PROGRAM AND PROJECT

Leadership (APPL) and ASK Magazine. APPL helps

NASA managers and project teams accomplish today’s

missions and meet tomorrow’s challenges by providing

performance enhancement services and tools, supporting

career development programs, sponsoring knowledge

sharing events and publications, and creating opportunities for project management collaboration with universities, professional associations, industry partners and

other government agencies.

ASK Magazine grew out of APPL’s Knowledge

Sharing Initiative. The stories that appear in ASK are

written by the ‘best of the best’ project managers,

primarily from NASA, but also from other government

agencies and industry. These stories contain genuine

nuggets of knowledge and wisdom that are transferable

across projects. Who better than a project manager to

help another project manager address a critical issue on a

project? Big projects, small projects—they’re all here in ASK.

Please direct all inquiries about ASK Magazine editorial

policy to Todd Post, EduTech Ltd., 8455 Colesville Rd.,

Suite 930, Silver Spring, MD 20910, (301) 585-1030; or

email to tpost@edutechltd.com.

APPL DIRECTOR AND MANAGING EDITOR 

Dr. Edward Hoffman 

ehoffman@hq.nasa.gov 

EDITOR-IN-CHIEF

Dr. Alexander Laufer 

allaufer@askmagazine.org

www.edutechltd.com/NASA/Laufer/index.htm 

EDITOR

Todd Post

tpost@edutechltd.com

ASSOCIATE EDITOR

Jody Lannen Brady

jbrady@edutechltd.com

CONTRIBUTING WRITERS

W. Scott Cameron 

Terry Little 

KNOWLEDGE SHARING MANAGERS 

Denise Lee

dlee@edutechltd.com 

Therese Salmon

tsalmon@edutechltd.com

EDITORIAL ASSISTANT

Michael Derocher

mderocher@edutechltd.com 

DESIGN

Hirshorn Zuckerman Design Group, Inc.

www.hzdg.com 

SENIOR PRINTING SPECIALIST

Dave Dixon

ddixon@hq.nasa.gov 

STAFF
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 3

The ASK Magazine prototype appeared in January 2001,

and was released to a small group of readers, less than a hundred 

of some of the most reflective project managers we knew at the time

THE FORMAT WE CHOSE—STORIES—WAS BOLDLY

different than any other project management publication we were aware of. This was truly an innovation, and so a prototype seemed like a good way of

testing this new concept to see if it was legitimate. We

wanted our readers to tell us if we were on the right

track, and by producing a prototype we could gain

that knowledge quickly in case we needed to rethink

our approach.

All I had to start was a stack of transcribed

presentations from the APPL Masters Forum of

Project Managers from September 2000. It was my

job to pull out the best stories and contact the

authors to fill in some areas that the editorial team

(Ed Hoffman and Alex Laufer and myself) thought

deserved more attention.

Let me tell you a little story within this story. I

learned a tremendous amount about project management at NASA while working on this prototype issue—

for instance, when I conducted my first interview with a

NASA project manager, Elizabeth Citrin of Goddard

Space Flight Center. I began the interview by asking her

some generic questions about scheduling and budgets.

As an interviewer, you know when you’ve got good

material by the excitement you get back from the interviewee. It felt like a stiff interview, until I tossed aside my

prepared questions and simply asked her, “So what is it

that gets you excited about a project?”

And there the interviewed tipped. Liz told me that

it was the science, so I asked her to tell me more about

the science on the mission she was project manager on

then. Her eyes lit up even brighter, and suddenly we

were having a wonderful time talking about the project

and its science objectives and how those intersected

with her management.

Whenever I do an interview with a NASA project

manager, I’m always looking for that tipping point.

NASA project managers will speak passionately with me

about the work they do and why they love it, but the

onus is primarily on me to get them to open up.

Back now to the ASK prototype. We came out with

the prototype in January ‘01 as intended, and the advice

we got from our audience was critical in helping us think

through what we wanted to accomplish with ASK. We

continue to learn from our readers and adapt the

magazine with their help. In that way, the ASK project

still feels a bit like a prototype.

In this issue of ASK, you’ll find examples of how

prototyping has been used to benefit a range of projects.

We reach a much larger audience now than we did

when we released the ASK prototype, but we still look

forward to hearing from our readers as to how to do a

better job. As always, we invite you to share your

thoughts with us. •

Learning About ASK

IN THIS ISSUE Todd Post
4 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

JOHN BRUNSON of the Marshall Space Flight Center is a

member of the NASA Program Management

Council Working Group. He served as project

manager for three separate microgravity

payloads that flew on various Spacelab

missions. His career in the space industry began

in 1980 as a technician working on the first Space Shuttle.

DR. MICHELLE COLLINS works in the Spaceport Engineering

& Technology Research Group at Kennedy

Space Center. She has over 20 years experience

in aerospace spanning engineering, R&D and

project management. She is on the Florida

Tech Engineering Accreditation Board, the

National Fire Protection Association’s Technical Committee for

Halon Alternatives, and the United Nations Environmental

Programme Halon Technical Options Committee.

HECTOR DELGADO is Division Chief of Process Tools and

Techniques in the Safety, Health and

Independent Assessment Directorate at the

Kennedy Space Center. In 1995, he served as

Senior Technical Staff to the NASA Chief

Engineer at NASA Headquarters in Washington,

D.C. He has received many honors and awards including the

Exceptional Service Medal, Silver Snoopy Award, and various

achievement awards.

DR. OWEN GADEKEN is a Professor of Engineering

Management at the Defense Acquisition

University where he has taught Department of

Defense program and project managers for

over 20 years. He retired last year from the Air

Force Reserve as a Colonel and Senior

Reservist at the Air Force Office of Scientific Research. He is a

frequent speaker at project management conferences and

symposia.

DR. MICHAEL HECHT has been with NASA since 1982 at the

Jet Propulsion Laboratory (JPL). He is project

manager and a co-investigator for the Mars

Environmental Compatibility Assessment

(MECA). In his previous assignment with

NASA’s New Millennium Program, he was

instrumental in defining the “microlander” that was adopted as

NASA’s New Millennium Program Deep Space 2.

JODY ZALL KUSEK is a Senior Evaluation Officer at the World

Bank. She is currently involved in supporting

the efforts of seven governments to move to a

focus of performance-based management. She

has spent many years in the area of public

sector reform, serving the Vice President of the

United States, the U.S. Secretary of the Interior, and the U.S.

Secretary of Energy in the areas of Strategic Planning and

Performance Management.

DONALD MARGOLIES of the Goddard Space Flight Center was

Project Manager for the Advanced Composition Explorer (ACE) mission, launched in

1997 and still operating successfully. He

received the NASA Medal for Outstanding

Leadership for his work on ACE and a NASA

Exceptional Service Medal for the Active Magnetospheric

Particle Tracer Explorers (AMPTE) mission.

DR. GERALD MULENBURG is the Manager of the Aeronautics

and Spaceflight Hardware Development

Division at the NASA Ames Research Center.

He has project management experience in

airborne, spaceflight, and ground research

projects with the Air Force, industry, and NASA.

He also served as Executive Director of the California Math

Science Task Force and as Assistant Director of the Lawrence

Hall of Science.

JOAN SALUTE is the Associate Director of Aerospace at Ames

Research Center. She has managed many

NASA projects including those involving flight

testing of thermal protection materials,

commercial technology, commercial applications of remote sensing, and remote sensing

science projects. She has been at Ames for 20 years, and was

awarded the Sloan Fellowship to attend Stanford Graduate

School of Business.

HARVEY SCHABES is currently assigned to the Systems

Management Office at the Glenn Research

Center. He started his career with NASA in

icing research, and since then has served in

numerous organizations in support of the

Space Station Program.

CHARLIE STEGEMOELLER is Manager of the Johnson Space

Center (JSC) Human Space Life Sciences

Programs Office. He is responsible for the

programmatic and tactical implementation of

the lead center assignments for Space Medicine,

Biomedical Research and Countermeasures,

and Advanced Human Support Technology. He began his

career at NASA in 1985 with JSC Comptroller’s Office as a

technical program analyst.

HUGH WOODWARD is a Program Manager for Global Business

Services with the Procter & Gamble Company.

He served as the Chairman of the Project

Management Institute (PMI) for consecutive

terms in 2000 and 2001. He was elected to the

Board of Directors in 1996, and before being

elected as the chair, served terms as vice chair and in several

other key leadership roles.

REVIEW BOARD
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 5

I FREQUENTLY RETURN HOME FROM MY TRIPS TO NASA

Centers with fun little things for them. The urge is

irresistible, and the truth is the stuff is as much for me as

it is for the kids.

One of the toys I brought home on this trip was

a gyroscope. When I showed it to my daughter

Amanda, she was as smitten with

it as I was when I spotted it in the

JPL gift shop.

For the next hour we laughed

and played. We placed it on its 

side, right side up, upside down,

on different surfaces and changed

speeds. We discussed ways to

improve the gyroscope. We talked

about the science. It was just 

plain fun.

I have always liked gyroscopes.

I’m not exactly sure why, but I know it has something to

do with how they feel in my hands. The whole time

Amanda and I were playing, I couldn’t stop thinking of

the power of letting people touch and play with

something, and the learning experience that provides.

In his outstanding book, Serious Play, Michael

Schrage suggests that play should be a critical core

competency of any modern organization. Coupled with

the serious nature of work is the joy that comes from

playing with a prototype or model. A fascinating premise

of the book is that prototypes and models have their

power in allowing people to play with difficult concepts.

If this is not proof, after an hour or so Amanda

announced, “I’d like to make my own gyroscope.”

The power of models and prototypes has always been

a part of my life. I remember as a kid, not much older than

Amanda, playing football on East 9th Street in Brooklyn.

We used primitive forms of prototypes by drawing

pictures and plays on the street with chalk. If it was raining

or we were concerned about the other side stealing our

plays, we would gather the team

around someone’s back and draw

the play with an index finger.

Here again, I am reminded of

Serious Play and the notion that a

prototype, model, or simulation is

important because it creates interactions between people involved

with the outcome. Our drawings

opened up a space to discuss and

explore the execution of the play.

Sometimes our discussions were so

lively and went on so long that the other side would start

counting down aloud to make a point of their impatience.

Back at the scrimmage line, we’d run our play and

see what happened. Sometimes it worked, sometimes

not. Incomplete passes, sweeps that ran out of bounds—

afterwards we’d regroup in the huddle and attempt to

figure out what went wrong, drawing the play again and

learning from our mistakes. A couple of downs later,

we’d try again and see if we could muster better yardage.

Each time, we learned to read the defense a little better.

That, too, is the nature of playing with prototypes:

They are a constant source of learning. This issue of

ASK is devoted to prototyping, and I hope you will find

plenty to learn from inside... •

Playing with Prototypes

On a recent trip to the Jet Propulsion Laboratory,

I brought home some toys for my children

FROM THE DIRECTOR’S DESK Dr. Edward Hoffman

“...It’s not enough to have

brilliant ideas; you have to 

be able to demonstrate them.

You have to get people to 

want to play with them.”

—Michael Schrage, Serious Play
6 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 7

$ MART BUYING

STEVEN A. GONZALEZ

When it came time to buy the next-generation data storage system for the Mission Control

Center at Johnson Space Center, we asked our contractor who provides Control Center support

to come up with a solution that would consolidate three current storage systems, as well as

provide additional capability and functionality—all without spending vast amounts of money.

An artist’s conception of completed

International Space Station, with its 356-foot

wingspan and 1-million-pound mass.
8 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

EVENTUALLY, THE CONTRACTOR’S REPORT ARRIVED AT MY

office. To my great disappointment, the proposed

system came along with a multi-million dollar price

tag. And, even more disappointing, the system relied on

the same technology we already had in place and

wouldn’t deliver much additional functionality. It was

clear that we needed to come up with a better

solution—the best we could buy. But how do you buy

the best technology, when you don’t even know what

technology is out there? 

Technology changes often and staying aware of the

latest technological developments is always a challenge.

In this case, we needed to invest in an in-depth evaluation of potential solutions.

I realized that we had to learn first-hand to be

better buyers, so I came up with the idea of inviting

storage area network vendors to come on site and

show us their capabilities and products. I

hoped that by “test driving” the latest,

greatest technology, our civil servants

would be smarter buyers when it came time

to choose a system.

We cleared out two rooms, reached

agreements with several companies, and

then, one-by-one, put their storage systems

through the paces that would enable them to be

installed at Mission Control—in essence, testing out a

series of prototypes of the system we hoped to acquire.

Why would a company expend their own resources

to temporarily install more than a million dollars of

equipment at our technology lab? It allowed them to say

that they had helped create a Mission Control Center

prototype, and to tell potential clients that NASA was

evaluating their equipment.

Our prototype project allowed us to better

understand our requirements, before investing in a

system. One of the things we learned about was

clustering capabilities that would enable us to better

support the Space Station’s 24-hour operation. We

had to have a storage system that could be reconfigured to a reduced environment so that the rest of the

Control Center could be updated. We needed to be

able to quickly move from one configuration to

another, but didn’t know how we could reduce the

four hours required to do this. We discovered a clustering capability associated with

some of the systems we tested that

provided that capability and reduced the

time dramatically.

While our people were brought up to

speed on the latest technologies available,

the companies got a heads-up on our

requirements. We used the prototypes to learn, and we

told the companies that NASA and its contractor

support would create the RFP that would go out for the

new storage system based on what we had learned. We

couldn’t promise them anything, but it would give them

a chance to see how their systems could be adapted to

work in our particular environment. As it turned out,

one of them did get work from the Mission Control

Center contractor using the prototype concept they

presented to us.

In the end, NASA got a better system for less money

than had been thought possible. Instead of spending

about $3 million, we spent $750,000 on a state-of-theart system ideally suited to meet our configuration

requirements. And in the process, we became smarter

customers and smarter buyers of new technology. •

LESSONS

•Prototyping can be a key management and communication

tool. Prototypes can increase the active participation of users

in project definition.

• Using the products of different vendors allows the user

to refine his or her objectives.

QUESTION

What would it take on your projects to be a smarter buyer? 

INSTEAD OF SPENDING ABOUT $3 MILLION, WE SPENT

$750,000 ON A STATE-OF-THE-ART SYSTEM IDEALLY

SUITED TO MEET OUR CONFIGURATION REQUIREMENTS.

I HOPED THAT BY ‘TEST DRIVING’ THE LATEST, GREATEST

TECHNOLOGY, OUR CIVIL SERVANTS WOULD BE SMARTER

BUYERS WHEN IT CAME TIME TO CHOOSE A SYSTEM.
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 9

Two years ago, STEVEN A. GONZALEZ’s story, “It’s All About Passion” (ASK 2), described founding the

Qualification and Utilization for Electronics Systems Technology (QUEST) lab with his fellow engineers at

Johnson Space Center.The QUEST group sought to reinvigorate their passion for work by mapping out a path

to NASA’s future. About the lab’s founding ideals, Gonzalez wrote, “The farthest reaches of the galaxy would

forever be expanding as long as we had the imagination to see a way there.... Our mission, as we saw it, was to

come up with a plan to achieve the infrastructure and technology that would make this vision a reality.”

Today, Gonzalez serves as Chief of the Operations Research & Strategic Development Branch at Johnson, and his team continues

to support the QUEST vision. Now called the Quest Innovations Lab and under the leadership of Tony Bruins, the group

partners with private industry to bring cutting-edge technology to NASA at low cost.

Clouds form the backdrop for the connected 

Zarya and Unity modules after their 1998 release

from the Space Shuttle Endeavour’s cargo bay.
10 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

AS SMALL AS POSSIBLE

BY SCOTT TIBBITTS
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 11

THIS STORY BEGINS WITH A BIT OF SERENDIPITY: I WAS ON

a trip to see a Shuttle launch and I happened to sit next

to a guy who was in charge of batteries for Space

Systems/Loral. He told me that they needed to create a

new battery bypass switch, the device that takes a

battery out of commission if it goes bad.

After discussing the conversation back at my

company, we decided that we could create the switch.

We contacted the folks at Loral and they said, “Okay,

let’s see what you can come up with. We need it as small

as possible.” We asked, “How small?” They said, “We

need it as small as you can possibly make it.”

I called in my lead design engineer and said,

“Dave, I need you to make this switch as small as

humanly possible.”

Dave went to work and he created a prototype for

the bypass switch that we were pleased with. We

showed the prototype to the customer and we said,

“This is the smallest bypass switch we can make.”

And they asked, “Are you sure?” And we said, “Yes,

that’s it.” They told us that they weren’t too happy

about the size, but if that was as small as we could

make it, they would consider it.

A month later, we went to a trade show and saw a

prototype for the same switch that a competitor had

created. It was half the size of ours. I brought Dave over

to take a look. We looked and looked at it. A week later,

Dave came to me with a new prototype—this one was

less than half the size of his first prototype.

Now, I can’t tell you how many times I harped on

Dave before he designed the first prototype that it had

to be “as small as possible.” But it wasn’t until he saw

the dimensions of what someone else had come up with

that he created his second, smaller design. One thing

going on was personal pride: If another designer could

do something, then, by God, Dave could do it.

I’ve come to call this “harnessing the power of the

sun.” You don’t want to use this in a manipulative way,

but if you have the opportunity to take someone who

has a self-righteous pride that they can do something,

couple it with talent and point them in the right

direction, magic can come from that.

Another lesson that I’ve learned is that—whether

you’re talking to a designer about a prototype or to a

team about an entire project, good managers don’t say,

“I need it as small as possible,” or “I need it as inexpensive as possible,” or “I need it as soon as possible.” They

set an ambitious but realistic goal, and they drive their

team to it by saying, “Hey, I need this. How can we do

it?” That’s when the creativity kicks in and people start

thinking outside the box.

When you’ve agreed on such a goal, it’s amazing

sometimes how clever people can get. •

LESSONS

• Setting ambitious but realistic goals for a project is a

key to succeeding in today’s competitive environment.

• Competition is a powerful motivator—as is personal

pride—which project managers may use to elicit creative

contributions from individuals.

QUESTION

When setting a goal for a project, how do you know whether it

is ambitious enough?

“The people who run small technology

companies are bold and fearless,” says 

Dr. Michael Hecht of the Jet Propulsion

Laboratory. When Hecht started up his 

$5-million project to fly an instrument to Mars, he looked to small

business and found his way to SCOTT TIBBITTS, President of

Starsys Research in Boulder, Colorado. “I remember very well a

spring morning when I first came and sat opposite Scott at a

conference table and I told him about my project. He listened

patiently, and then he told me, ‘If we mess up your project, we

might as well take the shingle down from in front of the door.’

That was my first introduction to the Starsys philosophy. I

learned far more from them than they ever could have learned

from me,” says Hecht. Since 1988, Tibbitts’s company has

grown from a garage-based, two-person business building

thermal actuators, to a 90-person firm with a world-class

reputation for producing more than 2,000 mechanisms flown on

more than 200 spacecraft.

A W EEK LATER, DAVE CAME TO ME WITH A NEW PROTOTYPE — THIS ONE WAS LESS THAN HALF THE SIZE OF THE FIRST PROTOTYPE.
12 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

Right -

SPECIAL FEATURE: THE IDEO WAY
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 13

RIGHT

A LEADING MANUFACTURER OF PERSONAL COMPUTERS

came to IDEO to design a new laptop. One of the many

areas they wanted us to improve was the design of the

door that covers and protects the connectors on the

back of the product. Why were they so interested in this

door? It turns out that one of the most common failures

in a laptop is the connector door. This little feature is

constantly used and abused, and inevitably breaks or

falls off, causing great annoyance to the user. Our

customer wanted an innovative solution that was

reliable and easy to use.

The team brainstormed hundreds of alternate

solutions, and quickly narrowed down the field to

BY DR. CRAIG LAWRENCE

- Rapid - Rough

IDEO (pronounced “eye-dee-oh”) is an international design, engineering, and

innovation firm that has developed thousands of products and services

for clients across a wide range of industries. Its process and culture have

attracted the attention of academics, businesses, and journalists around

the world, and are the subject of a bestselling book, The Art of

Innovation by Tom Kelley (reviewed on page 39). One of the keys to

IDEO’s success is its use of prototyping as a tool for rapid innovation.
14 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

several that seemed promising. How to select the best

concept? Analysis was not enough; we needed to know

how the doors functioned when used, and this clearly

called for prototyping. At this point in the program, the

overall design of the laptop had not yet been

developed. Regardless, the team pressed ahead and

built a series of prototypes that focused exclusively on

the connector door.

Models of the door concepts were machined from

plastic that would simulate the actual performance

and feel of a real production solution. These

models were then attached to blocks of

wood that roughly approximated the

size and weight of the final laptop

design. Great attention to detail

was placed on the areas around

the door and hinge, while no

effort was made to simulate

any other aspect of the laptop.

Some of the concepts

were complex. For example,

one concept used a clever

geared hinge to open a set of

double doors, revealing the

hidden connectors in an appealing

way. Others were simple; one even

“borrowed” from the common metal tape

measure to create a sliding door. In fact, the

team literally used a tape measure in the

prototype—a quick method of testing the concept. These

prototypes were taken to potential users and tested. They

were pulled, pushed, squeezed, dropped, and cycled until

the flaws in the concepts were revealed and a clear winner

could be selected.

This example illustrates an important concept in

prototyping. A prototype should be designed to answer

a specific question. The key is to ask the right question

(the right question is the one you really need answered),

and target your prototypes to answer it.

We find that if we try to answer too many questions

with a single prototype, the prototypes become more

expensive, take longer to create, and often provide less

value than a series of quicker, cheaper prototypes

targeted at individual questions. Prototyping in this

manner keeps the cost of failure low.

RAPID

Prototyping does not have to be painful. Prototypes

don’t have to break your project budget and devour all

of your project resources. A prototype can (and should)

be built in a few minutes.

IDEO developed a small digital camera that plugs

into the expansion port of a PDA (Personal Digital

Assistant). While this product provided serious

engineering challenges, one of the toughest challenges

was developing the software interface that the user

encounters when operating the product. A poorly

designed interface can ruin a user’s experience with a

technically perfect product.

Before any software code was

written, IDEO’s interaction

designers prototyped the

user interface using the

quickest method they

could think of: Post-It®

notes. They created a

series of handwritten

Post-Its that represented the on-screen

menus and windows

the user might

encounter while using

the product. The team

could quickly simulate

and test different interface

screens and menu logic by

simply peeling away successive

layers of Post-Its as a user navigated

through the “software.” If something did not make sense,

they could toss the Post-It and quickly create another.

Again, this prototype was taken to users and tested,

refined, and iterated. By the end of the prototyping

process, the team had a solid structure they could use to

develop the actual software. All without ever turning on

a computer.

The main value of this sort of rapid prototyping is

the ability to quickly evaluate your concept, and refine it

through a series of iterative prototypes. Instead of

spending your time and resources speculating solutions

and analyzing the problem, spend your time solving it.

Fail early in order to succeed sooner.

One of my favorite tools for rapid prototyping is

Lego®

. These ubiquitous children’s toys are a great way

to prototype fairly complex mechanisms. During the

development of a medical instrument, an IDEO team

used Legos to prototype several concepts for a

mechanism to convert continuous rotary motion into

reciprocating linear motion. These prototypes clearly

By limiting 

the time, energy,

blood, sweat, and tears

you put into 

a prototype, you reduce 

your bias towards 

the concept.

SPECIAL FEATURE: THE IDEO WAY CONTINUED
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 15

could not be used for surgery, but they are great for

allowing an engineer to visualize a concept and work

through some of the complex details rapidly.

There are many great materials and tools available for

rapid prototyping. Often, these are things you can find

laying around: wood, plastic, tape, hot glue, coat hangers,

boxes, plastic tubes, and toys. For more complex

problems, tools like fused deposition modeling (FDM)

and stereo lithography (SLA) are often useful. Feel free to

mix high-tech and low-tech components and techniques.

Use your prototype to get feedback, and then move on.

ROUGH

A prototype does not have to be pretty.

Sleep apnea is a significant problem

facing millions of people. While a sufferer

of this condition sleeps, the muscles at

the base of the throat relax and obstruct

the airway. This results in a drastic

slowdown or even stoppage of breathing

that can cause hypoxia or even death.

One solution to this problem is to provide

positive pressure to the person’s airway to

keep it open. The user wears a mask over the

nose connected by a tube to a pump to provide

the needed pressure.

IDEO worked closely with a medical device

manufacturer to develop a product designed to help

people with sleep apnea. Several products existed on

the market, but all shared a common flaw: They were

uncomfortable to wear while sleeping, and many

sufferers refused to use them. IDEO set out to develop

a product that exceeded the performance of existing

products on the market, and that would let a user

sleep comfortably.

One concept that quickly emerged was to mount

the air tube over the user’s head. This idea seemed

attractive since it addressed a number of complaints

from users. It could locate the tube in a predictable

place, keep it from moving around as the user shifted

during sleep, and it relieved the weight of the tube,

preventing it from pulling at the mask.

A series of head-mounted concepts were generated

and quickly prototyped at the engineers’ desks using

commonly found objects and a little

creativity. Engineers put prototypes together using the

lining of a bicycle helmet,

stereo headphones, and

pieces of hand-cut

plastic in order to try

out different ideas.

They took them

home and slept

with them on,

waking up to modify

them as they encountered problems.

These prototypes

weren’t attractive; however,

they did the job and allowed

the team to focus in on a winning

solution. The final product incorporated a number of

concepts from the various prototypes, including a

unique cantilever design allowing it to accommodate

various head sizes and shapes. The final product was

beautiful, but it was a beauty that came from humble

beginnings.

One of the benefits of creating rough prototypes is

that you reduce your emotional attachment to a

concept. By limiting the time, energy, blood, sweat, and

tears you put into a prototype, you reduce your bias

towards the concept, and are more likely to make

objective conclusions and decisions about its value. •

LESSONS

• Prototyping is a technique that embraces failure as a

means to ultimate success.

• The Right-Rapid-Rough approach fosters innovation

by forcing you to use all of your senses to attack a

problem.

QUESTION

• How do you learn from small failures on a project?

After earning a Ph.D. at Stanford University,

CRAIG LAWRENCE joined IDEO in 1999 as

a mechanical engineer and project manager

in the Smart Products studio. Focusing his

efforts on developing electromechanical

products, Lawrence has been a technical contributor and

manager on such projects as a portable fuel cell battery for

consumer electronics and a handheld medical instrument to

measure human metabolism. 

Lawrence is a frequent instructor at IDEO workshops, helping

clients understand how to work with innovation tools. In

addition, he is a regular instructor at APPL’s Advanced Project

Management course, and he has participated in other NASA

forums for the knowledge sharing community.

Fail 

early 

in order to 

succeed 

sooner.
We came up with two basic concepts 

for landing Pathfinder on the surface of Mars. One was a traditional

approach—propulsive descent—just like Viking had done in 1976. 

The other concept was a wild idea—using giant airbags to cushion 

the lander’s impact, then letting it bounce and roll to a stop.

16 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP
C

ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 17

NASA BASICALLY LOOKED AT THE TWO OPTIONS AND SAID,

“Well, propulsion...that’s the old way of doing business.

You guys will never get this job done if you do it that

way. It’s too expensive.” And so we said, “Okay, let’s go

make this airbag thing work.”

The airbags idea was clearly eccentric. Off the

charts. When you think of an airbag, you think of the

automobile design, about twice the size of a pillow,

which took many years to develop. But what we needed

would have to be about 19 feet in diameter, designed to

tolerate a head-on collision with a very rocky Mars

surface at 60 miles per hour or more. And not just once,

but multiple times, as it bounced and rolled to a stop.

The only thing in common between our design and an

automobile airbag was the name. Another very eccentric

aspect of this was the idea of using fabrics in outer space.

We were used to dealing in aluminum and titanium, but

this needed to be the stuff of bulletproof vests...

advanced polymer cloth. We’d worked with software in

space, but not “softgoods.”

The young man who had come up with the kernel of

the airbag concept was Tomasso Rivellini. Tom had never

done a flight hardware engineering job before, but he had

the right energy and creative instincts. So we gave him the

job. Of course, he knew he needed help. He went to Bob

Bamford and Bill Layman, two of JPL’s intellectual giants,

for help in developing the basic design. But once Tom

started working the details, he alone was responsible for

figuring out a way to build and test this behemoth. Tom

knew that JPL didn’t have the expertise in working with

fabrics and sewing—with so-called “softgoods.” So he

sought out and found people at Sandia National

Laboratory in Albuquerque and ILC Dover in Dover,

Delaware, to help build a scale model followed by fullscale prototypes. This job took a lot of trial and error. Tom

started with a 1/20th scale model, and worked up to full

scale. It turned out that the only way to really understand

how an airbag works is to test it full scale.

Every time we showed the video of the first fullscale test, in which the airbags were dropped about 120

feet onto a flat surface, people laughed. It did look

comical seeing a giant beachball bounce like a superball.

But our early attempts were discouraging. Our first

drops on a rocky surface simulating expected Martian

terrain were complete failures. We weren’t sure if this

thing was going to work. But we kept working the

details, improving the design, and going back into test. It

was a very iterative process. We tried an analytical

approach, but we spent over a week of Cray computer

time to get only a few seconds of data on the impact. The

problem was just too complex for state-of-the-art

analysis tools at that time. So we had to rely on Tom and

his team’s ability to design, build, and test their way to a

design that would work. And they did.

The manager of NASA’s Viking mission to Mars—

the legendary Jim Martin—was, at best, skeptical that the

airbag idea would work. He chaired the formal review

boards that oversaw the project’s progress throughout

its three-year development. He knew about all the trials

and tribulations of the airbag development, and that the

proof would only come on landing day. On July 4th,

1997, Jim and I were standing next to each other shortly

after the landing. Jim turned to me and said, “You know,

Brian, I think these airbags ought to be the required

technology, the technology of choice, for any mission

that is going to land where the terrain is unknown.” Our

eccentric idea had just become mainstream. •

FFTHE HAR

by Brian K. Muirhead

TS

“I like to do things that people consider impossible

missions,” says BRIAN K. MUIRHEAD, who led the

design, development, and launch of the flight system

for the Pathfinder Mission to Mars. “There were many

people who thought we would not be able to land on

Mars. [This mission] attracted innovators and some renegades. It was a

major challenge, so it hooked the risk-takers and people with a competitive spirit.” For his achievements on Pathfinder, Muirhead was awarded

NASA’s Outstanding Leadership medal. He was also named Engineer of

the Year for 1997 by Design News and 1997 Laureate for Space by

Aviation Week & Space Technology. In 1998, he achieved another

milestone of sorts, when he was awarded his very own “star” in the sky.

Asteroid Muirhead is a Mars-crossing asteroid, between 5 and 9 kilometers (about 3 to 6 miles) in diameter, and reported to be traveling in a

highly inclined, eccentric orbit.

EXCERPTED FROM THE MARS PATHFINDER APPROACH TO “FASTER-BETTER-CHEAPER,”

BY PRICE PRITCHETT AND BRIAN MUIRHEAD. USED WITH PERMISSION OF PRITCHETT, LP. 

ALL RIGHTS RESERVED
D

Here I was: 26 years old, I had

never worked on a flight project before, and all

eyes were on me. Every time I walked by the Pathfinder

project office, Tony Spear, the project manager, would throw his arm

around me and announce, “Hey everybody, the whole mission is

riding on this guy right here.”

Our task was to design and build airbags for Pathfinder’s landing on Mars—

an approach that had never been used on any mission. Airbags may seem like a

simple, low-tech product, but it was eye-opening to discover just how little we

knew about them. We knew that the only way to find out what we needed to learn

was to build prototypes and test them. We just didn’t know how ignorant we were

going to be.

Airbags seemed like a crazy idea to a lot of people. Nobody ever said that,

mind you, but there seemed to be a widespread feeling that the airbags weren’t

going to work. “We’ll let you guys go off and fool around until you fall flat on your

faces.” That was the unspoken message I received day after day.

18 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

DRR

IN

OO

ON P

PIN

MA

P

R

PIN

S

G

by Tommaso Rivellini

G
We knew that the only way to find out 

what we needed to learn was to 

build prototypes and test them. 

We just didn’t know how ignorant 

we were going to be.

ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 19
20 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

EVERYONE’S MAIN FEAR ABOUT USING

these giant airbags was that the lander

would be buried in an ocean of fabric

when the airbags deflated. I began the

search for a solution by building scale

models of the airbags and lander, and I

played with them in my office for a

couple of months.

I built the models out of cardboard

and plastic, and taped them up with

packing tape I got from the hardware

store and ribbon from the fabric store. I

used a small raft inflator that I had at

home to pump up my model airbags. Over and over

again, I filled the miniature airbags and then let them

deflate, watching what happened.

I fooled around with a dozen or more approaches

before I finally came up with something that I thought

worked. Slowly but surely, I came up with the idea of

using cords that zigzag through belt loops inside the

airbags. Pull the cords a certain way, and the cords would

draw in all of the fabric and contain it. Wait to open the

lander until after all of the airbags had retracted, and the

fabric would be tucked neatly underneath.

Testing on another scale

Once we built large-scale models to conduct drop tests,

we started by doing simple vertical drops, first at 30 feet,

and then up to 70 feet. The bags performed well,

although the way they bounced like a giant ball was

interesting to observe. People began to realize that the

concept might just be reasonably sound. But we still had

our doubters. Even after we had the mechanics figured

out for the airbags, a big question remained: What about

the rocky Martian terrain? 

Landing on Mars, we had to accept whatever

Mother Nature gave us. The Pathfinder wouldn’t have a

landing strip. To simulate conditions on Mars, we

brought in large lava rocks the size of a small office desk.

They were real lava rocks that our geologists had gone

out and picked; if you tried to handle one of them, you

would cut up your hands.

The more landscape simulations we tested, the more

we started tearing up the airbags. Things were not looking

good. Once again, we realized that this was an area that we

just didn’t understand. The challenge was to protect the

bladder layer, essentially the inner tube of the airbag

system, with as little fabric as possible because the project

could not afford to just throw mass at the problem. We

tried material after material—

heavy duty Kevlars and Vectrans

among them—applying them in

dozens of different configurations to the outside of the airbag.

Ultimately, we knew that

we could just throw on more

and more material and come up

with a reasonably performing

airbag system, but the weight of

that solution would have come

at the expense of something

else—another component of

Pathfinder would have to be sacrificed. We weren’t,

however, going to Mars just to land there and take a few

pictures. We wanted to go there and do science—and we

needed instruments to do that science. So there was a lot

of motivation to come up with the lowest-mass, highestperformance airbag system that we could.

5, 4, 3, 2, 1

Each test became like a ritual, because it took between

eight and ten hours to prepare the system—including

transporting the airbags into the vacuum chamber,

getting all of the instrumentation wired up, raising the

airbags up to the top of the chamber, making sure all the

rocks were in the right place, and preparing the nets.

The vacuum chamber where we did the drop tests

used so much power that we were only able to test in the

middle of the night. Once the doors of the vacuum

chamber were closed, it took three or four hours just to

pump down the chamber. At that point, everybody either

broke for dinner or went to relax for a while, before

coming back at midnight or whatever the appointed

hour was. Then we had another 45 minutes of going

over all of the instrumentation, going through checklists, and then ultimately the countdown.

The last 30 seconds of the countdown were excruciating. All of that anticipation, and then the whole

impact lasted less than one second.

When we finished a drop test, we knew right away

whether it was a success or failure. Brian Muirhead, the

flight systems manager, was always insistent that I call

him immediately—no matter how late it was. At 4 a.m.,

I would call him at his home and have to give him the

news, “Brian, we failed another test.”

Each test was followed by a high-pressure rush to

figure out what went wrong, what test to run next,

how to fix the extensively damaged bags, and how to

I fooled around with 

a dozen or more

approaches before I finally

came up with something

that I thought worked.
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 21

simultaneously incorporate whatever new “experi- tests. In spacecraft terms, this is supposed to be the last

mental fix” we came up with. As a team, we agreed test that you run in order to qualify your final design. By

upon a course of action, usually in a surly, sleep- the time you get to that point, there is supposed to be no

deprived mood over a greasy breakfast at a local diner. question whatsoever that you have a fully functioning

Then the ILC Dover folks would figure out any new system that meets all of the mission requirements. It is

patterns that needed to be generated as well as the supposed to be a check-the-box process that the system

detailed engineering to ensure the seams and stitch is ready for flight. The problem was that at that point we

designs could handle the test loads. Our hero was our had still only experienced partial success; we’d never had

lead sewer, who incidentally sewed Neil Armstrong and that A+, 100% grade on any of our drop tests.

Buz Aldren’s moon suits. She worked under less-than- Flying in to watch that last drop test, my plane was

ideal conditions while we slept and turned our delayed. One of my colleagues at the test facility called

sometimes unusual ideas into reality. Usually by the and asked me, “Do you want us to wait for you?” I told

next day we were ready to do it all over again. him, “No, go ahead.”

Tony Spear and Brian understood the challenges we When I got to the facility, the test crew wasn’t

were facing. They knew we had a solid team working on there. I went into the control room and ran into the guy

this, and I always kept them informed on the technical who processes the videotapes. “So what happened?” I

progress. They were always understanding, but that’s asked him. “Did you guys do the test?” He pointed at a

not to say they were always happy. VCR and said, “The video is in there. Just go ahead and

press play.”

Back to the drawing board So, I hit play. Down comes the airbag in the video—

We said, “Okay, let’s start doing analysis, computer it hits the platform and explodes catastrophically. My

modeling of the airbags and the impact against the heart sank. We weren’t going to make it. But then I

rocks.” At the same time, we expanded our test program realized that there was something strangely familiar about

to understand how to optimize this airbag abrasion layer. the video I had just watched. In an instant it came to me;

It turned out that the time, money, and effort we they had put in the videotape from our worst drop test.

expended on the computer modeling didn’t pay off. The practical joke could mean only one thing: We had

Though we ran the most sophisticated programs available had a successful drop test, and were finally good to go. •

back in 1993 and 1994, the results didn’t help us design

the abrasion layer. We had to rely on our prototypes. LESSONS

After doing dozens of drop tests, looking at the • Prototypes focus attention on the most essential

data, and studying what was happening, we started to characteristics of a problem.

realize that a single layer of heavy material wasn’t the • To develop innovative products, you must be

solution. Multiple layers of lightweight material might tenacious in the face of failures.

prove stronger.

We were forced to decide on the final abrasion layer QUESTION

design in order to meet our scheduled Qualification drop How do you know whom to assign innovative tasks on a project?

DROPPING IN ON MARS, AGAIN

Much like his role as Systems Engineer on the 1997 Mars Pathfinder mission, TOMMASO (TOM) P. RIVELLINI

led the Entry, Descent and Landing System team that worked on the 2003 Mars Exploration Rover (MER) missions.

Scheduled to arrive on Mars in early 2004, the landing of each of the two MER spacecraft will resemble that of

Pathfinder, including deployment of inflatable airbags. 

But it was by no means a straightforward effort adapting Pathfinder technology to MER. “To try to squeeze out more

performance, we had a lot of problems. It was like Pathfinder all over again,” explains Rivilleni. “On Pathfinder, we never had a

chance to explore in-depth the physics of why things behaved the way they did. We came to realize that we made mistakes on

Pathfinder that we needed to fix. All aspects of the system required rework compared to what we expected. Slowly but surely, our

team pushed through it all.” 
22 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

S UPPL YING THE GIANT

BY LARRY LAWSON

When I was program manager for Lockheed

Martin on the Joint Air Surface Standoff Missile

(JASSM), the government-stated objective for this

stealthy high-performance cruise missile was a

unit price of $400,000. The predecessor program,

which was cancelled, had cost four times more.
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 23

“Before acquisition reform, the government said to its contractors, ‘Follow these military standards and everything will be

okay,’” remarks LARRY LAWSON, Vice President of Systems Integration and Business Development for Lockheed Martin

Corporation. “From a contractor’s point of view, that was a comfortable place to be. You knew that if you followed the

handbook you were in good shape. Suddenly, we found ourselves in a position where our customer was saying, ‘Throw

out all the standards. You don’t have to follow them. I don’t want you to reference a single military standard.’” At the time, Lawson served

Lockheed Martin as Vice President of Strike Weapons, which included the Joint Air-to-Surface Standoff Missile (JASSM). The Office of the

Secretary of Defense honored JASSM with the David Packard Award for acquisition excellence. Mr. Lawson has received the Inventor and

Manager of the Year awards from Lockheed Martin and holds patents in Advanced Discrimination Technology. 

IT WAS CRITICAL FOR OUR TEAM TO FIND A RADICALLY

different way of doing business. Deciding to build the

airframe out of composites was the first step, refining

processes from the boat building industry was second,

and the final step was choosing a supplier.

Lockheed Martin built the first prototypes at our

Skunk Works facility in Palmdale, California. These

units were hand-built and used early prototypical

tooling. They looked great but were not affordable.

We had to focus on minimizing touch labor and cycle

time and reducing material costs. We needed a company

to produce the composite quilts we would use to avoid

hand lay-ups.

The company we found surprised a lot of people.

We partnered with a small company outside of Boston

whose primary business was making baseball bats and

golf club shafts. They had never built a military product

but they knew how to weave carbon fiber and build

basic composite parts. Their experience in the commercial market had forced them to learn to build these parts

to final shapes with little labor, and they could control

material price because they bought fiber as a commodity.

We began our efforts with them by building prototypes and eventually came up with fuselages that were

usable with some rework. We began testing these bodies

for material and structural properties and then using

them in flight test articles. Once we qualified the

integrity of processes we could focus solely on first-pass

quality. Lockheed Martin, the Air Force Mantech office,

and the vendor continued to refine the process to meet

or exceed all our objectives. We brought this small house

from being a baseball bat provider to an aerospace

house, and it has been a remarkable transformation.

Along these same lines, my favorite JASSM story is

the supplier we chose for the wings of the missile. One

really creative individual in our organization knew about

a company that built surfboards and had ventured into

building the blades for windmills. We went down to their

factory in a disadvantaged part of Los Angeles, saw what

we liked and gave them a chance. Today, this technique

is used not only on JASSM but on another missile in our

portfolio, as well.

We had no choice but to operate this way. Our

customer, DoD, told us point blank: “We want a missile in

half the time for half the cost of what we used to be willing

to pay.” We had entered the era of acquisition reform.

Acquisition reform gave us the freedom to become highly

creative in developing solutions that best met the

customer objectives. “Faster, Better, Cheaper” wasn’t just

a NASA concept. The government charter of quick

turnaround at low cost forced us to demonstrate we could

build this thing right and do it for what we said it would

cost. Prototyping was a key component of our strategy. •

LESSON

• To achieve remarkable results from a contractor, you

must demand it unequivocally. However, you must also

release the contactor from beaurocratic constraints.

Most important of all, you must select a contractor who

is willing to take on such risk.

QUESTION

Have you ever considered creating an environment where you

required your contractor or subcontractor to be more innovative? 

WE BROUGHT THIS SMALL HOUSE FROM BEING A BASEBALL BAT PROVIDER TO AN

AEROSPACE HOUSE, AND IT HAS BEEN A REMARKABLE TRANSFORMATION.
BIG P

THE STORYBOARD

IC

’S

TURE

, C O . MALL Y, BILL OOLEY

CHERYL A

(SAIC) PMP (NASA)

OUR TOOL IS A STORYBOARD. THE BASIC INGREDIENTS

include a 12-foot-long tack board strip on the wall, a

pack of thumbtacks, paper, and a writing instrument. We

use our storyboard to create a paper prototype of our

product. Graphic, sequential depictions give a quick

project overview while breaking down the product into

its major components.

Though it could be applied to any type of project,

we have found that the storyboard concept is ideally

suited for software development. For example, many

members of a software development team are specialists

at coding and can get caught up in a particular function

or aspect of the project. The storyboard helps them

conceptualize the relationships between project

tasks and the bigger picture. Seeing the big

picture was a particular problem for us on our

current project, so we posted a copy of a story by

Dr. Michelle Collins, “Lessons From the Great

Masters” (ASK 3), to try to help the team think at

a higher level and ask the right questions: What is

the operational concept of this product? What do

we really need to do first?

Most of the board, however, reflects the major

elements of the project. We tack sheets of paper on the

wall in the sequence that users will likely perform their

tasks. At first, we sketch out ideas with a few words

24 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

PRACTICES

AT SCIENCE APPLICATIONS INTERNATIONAL CORPORATION (SAIC), CAPE CANAVERAL OFFICE, WE’RE

USING A PROJECT MANAGEMENT TOOL THAT FACILITATES TEAM COMMUNICATION, KEEPS OUR

PROJECT TEAM FOCUSED, STREAMLINES WORK AND IDENTIFIES POTENTIAL ISSUES. WHAT DID IT

COST US TO INSTALL THE TOOL? ALMOST NOTHING.
“ANYONE IN 

ABOUT OUR ENTIRE PROJECT BY 

THE ORGANIZAT

DOWN 

ION CAN LEARN 

THE LINE OF P

WALKING 

APERS.”

or graphics; but as our storyboard

progresses, we replace the words with

screen shots and major elements begin to

evolve “down” the storyboard.

The storyboard process helps promote

brainstorming, highlights missing tasks, and

allows the team to incorporate changes prior

to traveling too far down a particular path.

It also helps us to stand back from our work

and ask, “Is this the most logical sequence

for the way we’re doing things?” We physically move pages around and put them in a

different order as we resolve issues. The

number of revisions done to the storyboard is

based on a project’s schedule and budget constraints.

The storyboard also gives us maximum exposure.

During our “graffiti phase,” anyone in the organization

(potential users, customers, and team members alike)

can learn about our entire project by walking down the

line of papers conveniently located in the office hallway.

When they see something that doesn’t make sense to

them or they think of a feature that might be added, they

write down their comments directly on the sheet of

paper on the wall. For example, one person wrote on the

storyboard, “When a procedure is executed where are

the results stored?” Thanks to this comment, we

realized we were so focused on the procedure itself that

we hadn’t thought about where the documentation of

the procedure would be stored. How could we track and

display the information without cluttering the screen?

Would our repository be on individual hard drives or a

shared network? The comment helped us to step back

from our work and look at it from a user’s perspective.

We have a designated keeper of the storyboard

whose job it is to evaluate those comments and meet

with the team to see which ideas should be implemented

into our planning. If we decide to use an idea, it becomes

part of our evolving storyboard. When we recognize

good ideas that are outside our current scope, they are

consolidated and tacked at the end of the storyboard on

a separate page called “Future Features.”

By using the storyboard, we get many people

involved in providing constructive feedback and, most

importantly, we make certain that team members aren’t

going off in different directions. The storyboard keeps us

all working toward the same goal. •

ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 25

TRADING PLACES

As part of NASA’s newly established Industry

Exchange Program (IEP), CHERYL A. MALLOY

began a nine-month assignment with SAIC in

December 2002. IEP promotes the exchange of ideas, best

practices, and operational insights between NASA and its

industry partners by arranging temporary assignment

exchanges. During her tenure at SAIC, Malloy had the

opportunity to work with several program and project

managers at SAIC, including WILLIAM COOLEY, a technical

analyst who specializes in merging software

with physics. Malloy, a 15-year NASA veteran,

previously served as Expendable Launch

Vehicle Mission Integration Manager.
WINGS THAT INFLATE WHEN NEEDED SEEMED LIKE AN IDEAL

solution to the problem. I knew that people had been

working on the concept of inflatable wings for other applications, but the technology was immature and unproven.

Before anyone would consider a design incorporating

inflatable wings, you had to know the concept was sound.

And that’s where prototyping enters this story.

I got together a small team. We learned that a set of

small-scale inflatable wings had been created for a Navy

munitions application through a Small Business

Innovative Research (SBIR) program. At the conclusion

of that SBIR, the prototype wings became the property

of the government, and the Navy program offered to

transfer the hardware over to us. With the wings in

hand, we secured discretionary funding to test our

design concept.

In a sense, this prototype we were going to build

was the product. We were just trying to answer some

fundamental questions before we went out and tried to

advocate for significant funding to do anything real. And

therein lay the beauty of prototyping: it allows you to try

a lot of things in a short period of time, without having

to spend a lot of money to try them. There were lots of

doubters about whether or not this idea had practical

merit, and we didn’t know for sure ourselves whether

any of our ideas would pan out.

[001]

[024

]

PROOF

[056]

OF

[073

]

[104

]

CONCEPT

[By Jeffrey Bauer]

Back in 2000, one of the potential Mars projects

involved delivering and then flying an observation plane

over the planet. Among the challenges of the project

was the small size of the capsule that was going to be

used in order to get the plane to Mars. With the

planet’s thin atmosphere, a plane would need to have

large wings in order to fly efficiently. How could you

package a large-winged plane in a small capsule?
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 27

We had our model shop design a small airplane

around the set of wings. All the testing that had been

done under the SBIR had used pre-inflated wings;

because we needed wings that could be inflated in flight,

we also needed to design an in-flight inflation system.

We came up with a system that used compressed

nitrogen gas to inflate the wings.

Our prototype flew with stable enough dynamics

that it could be controlled and land successfully. We had

proven the principle that, yes, you could inflate wings in

flight that wouldn’t fold back on themselves or fail in

any other mechanics. We had proven the concept to be

a viable option for vehicles with volume constraints.

We shot a video as we ran several drop tests where

we inflated the wings in flight under varying conditions,

and then put together a demo to prove to others that

this concept does seem to have some merit.

At the time we did this demonstration, the interest

for a Mars aircraft program faded, and it wasn’t among the

missions selected to fly. To our great satisfaction, however,

that wasn’t the final word on our prototype project.

We sent the test flight video off to anyone we

thought might be interested in the concept. One of

those people was an associate in the military. At the time,

he was trying to sell a program that relied on the use of

inflatable wings. Whenever he briefed people about the

program, they all told him that it was unproven and too

risky a concept to consider.

View graphs and a lot of conjecture don’t inspire

people to invest in projects. Proof of concept does. Once

he had our video as proof of concept, he successfully

sold a $50-million program based on our low-cost

(roughly $100,000) prototype.

Why use a prototype? If you can validate basic

principles early in the life of a project, you mitigate risks

substantially before you embark on a full-scale development effort. We answered a fundamental question before

we went out and tried to advocate for significant project

funding. Things might not have turned out exactly as we

had wished, but another part of the government was able

to benefit from our work, and we are proud of that. •

LESSONS

• Prototyping is one of the most crucial tools for innovation. Ideas can be tested and verified quickly by prototyping, before an extensive commitment of resources.

• One prototype is worth a thousand words. A good

prototype has the power to communicate more convincingly than any analysis.

QUESTION

How do you develop a knowledge database for prototypes?

Once he had our video 

as proof of concept, 

he successfully sold a 

$50-million program based

on our low-cost (roughly 

$100,000) prototype.

NASA AND 

ITS PARTNERS

“In my project world, there is typically

a good deal of cross-pollination of

projects between NASA and the

military, if not outright collaboration,” says

JEFFREY BAUER of NASA’s Dryden Flight

Research Center in Edwards, California.

Currently, Bauer serves as the NASA project

lead for an activity that will eventually grant

routine national airspace access to highaltitude, long-endurance unpiloted aircraft.

NASA, DoD and FAA will collaborate with

seven Unmanned Aerial Vehicle (UAV)

manufacturers on the program. Bauer also

serves as manager of NASA’s Environmental

Research Aircraft and Sensor Technology

(ERAST) program. ERAST is developing highaltitude long-endurance UAVs for science and

commercial applications.
CAUTION

By Terry Little

My experience, both first-and second-hand,

has been that people have misused prototyping

almost as often as they have used it wisely. 

I will try and cite some of the ways I have seen

people abuse the concept.

PROTOTYPING “ROPE-A-DOPES”

And Other Pitfalls

28 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

BEWARE

DANGER
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 29

“ROPE-A-DOPE” I HAD A BOSS ONCE WHO GAVE ME SOME

advice about how to get support for a new program. He

said that you label some obscure aspect of the program

as “high risk” when, in fact, you know that it is

eminently low risk. Then, he continued, you get

someone to give you limited money to support a riskreduction prototype and, voilà, the prototype demonstration is successful! You then use the prototype’s

success, ideally with videotape and loads of “data,” to

secure funding for a major new program or project.

The sad part about his strategy is that it often works.

Technologists use it all the time as a way of getting

funding that they could not get otherwise. I call the

strategy prototype “rope-a-dope” because it is deliberately misleading. The legitimate use of prototyping is to

find out something you don’t know—not to demonstrate

something you do know. Others may differ with me, but

I fail to see marketing as a legitimate use of prototyping—

at least not when the government is paying the bill.

“Kluging” Another pitfall is a belief that basic system

engineering principles can go out-the-window when

you design and build prototypes. There may be rare

instances where “kluging” together a prototype (like my

high school science fair projects) makes sense, but

usually one should build a prototype with an eye toward

making a smooth transition to beyond the prototype

stage. There is nothing worse than having a successful

prototype demonstration and then having to start again

from scratch to build something that’s affordable and

serves some useful purpose.

Some years ago an Air Force program spent several

hundred million dollars to build missile prototypes for a

competitive “fly-off.” The prototypes worked just fine,

but the designers had to completely redesign the missile

to make it into something that anyone would want to

buy. That redesign had some major cost and technical

problems that almost led to the program’s demise. I am

quite sure that had the prototype design been more

thoughtful and systematic, the transition would have

been much less painful.

Risk-averse Beware It’s OK for

a prototype to fail. In fact, if there

isn’t a non-trivial likelihood of a

failure, then why build a

prototype at all? The purpose of

building prototypes is to reduce

risk and, sometimes, to find

problems that you can only find

from a prototype. “Try-fail-fixtry-fail-fix...” is a legitimate and

sound prototyping strategy, but

hasn’t always been acceptable

where I have worked. Perhaps

NASA is different, but my experience is that “higher-ups” tend to be fine with risky

ventures so long as the ventures succeed. It reminds me

of people who are happy making high-risk investments

so long as they don’t lose any money.

There isn’t much that we can do about others’

attitudes except to make sure that all the higher-ups

understand the risks and to regularly remind them as

the effort unfolds. It is easy to get so mesmerized by a

prototyping project that we lose our objectivity and

become less-than-sober about assessing risk. It’s always

a critical mistake to take a path and underestimate the

number of opportunities to stumble along the way.

Seeing Forests Instead of Trees Finally, I have seen

plenty of instances where someone built a prototype of

the wrong thing—for example, producing a system

prototype when only a subsystem prototype was

necessary. Building a prototype when a model or simulation would have yielded a similar result is also common.

The missile program I mentioned earlier should

have built seeker prototypes and tested them in

hardware-in-the-loop simulations and captive carry.

There was no real need to go to the expense and time to

prototype the entire system, because 90% of the risk was

in the seeker. However, the program succumbed to

external pressures to shoot down an aircraft. The money

and time required to do that stunt would have better

gone to packaging the seeker prototype in a more

production-representative configuration. It was that

packaging challenge that later threatened the program.

The lesson here is to carefully craft any prototyping

effort to address the most salient risks or unknowns.

Spending 90% of the money to address a 10% risk area

is just not good use of taxpayer money.

Knowing Your Tools Overall, I love prototyping as a

tool. As with any tool, it’s important to use it wisely. (You

don’t want to hammer a nail with a screwdriver.) When

prototyping is the right tool, it can be a powerful means

to identify challenges, reduce risk or prove a hypothesis.

It is a superb way to learn what we don’t know.

Prototyping can give us the confidence that there is a

way ahead, or the knowledge that there isn’t—either

way, it’s a worthwhile investment. • 

The legitimate use

of prototyping is to

find out something

you don’t know—

not to demonstrate

something you 

do know.

TERRY LITTLE

is the Director of

the Kinetic Energy

Boost Office at the

Missile Defense

Agency. One of the most seasoned

program managers in DoD, he is also a

regular contributor to ASK Magazine.
30 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

AnOld Dog andNewTricks

BY W. SCOTT CAMERON

As I approach my 55th birthday,, the old adage

“you can’t teach an old dog new tricks” keeps coming to mind. I’m not sure why, because

I don’t feel old and I’m still interested in taking on new challenges and learning new tricks.

However, as I mentor new project managers, I am also aware that others may consider

me an old dog unable to learn new tricks. To the contrary, the people I mentor continue

to teach me new tricks and challenge my assumptions about project management.

THIEN-KIM PHAM
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 31

FOR EXAMPLE, I WAS MENTORING A NEW PROJECT MANAGER

and we were looking for ways to reduce our engineering

and overall project costs. One option we considered was

offshore (non-U.S.-based) design engineering. The

young project manager was eager to experiment, but I

had never tried this concept before and knew other

project managers who had with bad results.

Nevertheless, the concept still intrigued me.

There was a nine-hour time difference

between our office and the offshore engineering

office we were considering—as well as a

language difference. I could just imagine the

nightmare of being that far apart and trying to

clean up a project gone awry. The schedule and

cost implications to the business proposal would be

horrific. Since neither of us wanted to risk the entire

project design, we agreed to prototype this concept on a

specific portion of the project.

We worked with our engineering contractor to

select one element of our design for execution in their

offshore office. Someone on the contractor’s staff had

previously worked in this office and understood their

capabilities. His familiarity with their strengths and

weaknesses was integral to our prototype strategy.

The potential risks of our strategy should not be

understated. The offshore option generally isn’t considered unless the potential savings are high for either the

contractor or client. Working offshore, in this case,

wasn’t going to yield a huge savings for the project or for

the engineering contractor. But we saw its value as a

learning experience. And since we weren’t putting the

entire project at risk and we had a person in place with

strategic knowledge, we could present our idea as a

prototype execution strategy with manageable risk. In

the end, upper management accepted our plan.

We defined the work we wanted designed and

transmitted it electronically. We also had two of the

offshore office’s lead engineers visit our site to understand how we worked and what we needed. They stayed

for three weeks, and this face-to-face time was invaluable. With these personal relationships created and a

better understanding of how each other worked, the

design was completed on schedule and with no

increases in construction costs.

I understand now the key fundamentals of what it

takes to make an offshore project successful, as does

the project manager I was mentoring. Now that I’m a

bit wiser on this subject, I would certainly consider

going offshore again. Though we didn’t save money on

this project, the potential for that is certainly there on

bigger projects.

Here is what I learned: It worked because we were

able to mitigate the perceived/real “offshore risk.” We

worked through someone who already knew their

offshore engineering culture and how to get work done

there, plus we took the time to meet with their

engineers. Good communication is the key to success

on any project. If you’re separated by nine time zones,

you must figure out a way to bridge that gap, same as

you do when you’re separated by nine cities, nine floors

of an office building, or nine doors on the same floor.

I doubt I would have chosen this option had I not

been mentoring someone who was as curious to learn

“something new” as I am. Because I was working with

someone in a learning situation, he stimulated my

curiosity and excitement to learn a new trick. The

mentoring arrangement was a stimulus to experiment

with the prototype.

Mentoring, at its richest, is a two-way street. •

Since neither of us wanted

to risk the entire project design,

we agreed to prototype this concept 

on a specific portion of the project.

W. SCOTT CAMERON is the Capital

Systems Manager for the Food & Beverage

Global Business Unit of Proctor & Gamble. He

is also a regular contributor to ASK Magazine.
IN 1975, PELLERIN LEFT HIS POSITION AS A PRINCIPAL

investigator at Goddard Space Flight Center and transferred to NASA Headquarters. Eight years later, he was

named Director of Astrophysics. During his tenure, he

conceived of the “Great Observatories” strategy—four

large contemporaneous telescopes spanning the observable electromagnetic spectrum.

Pellerin oversaw the launch of a dozen scientific

satellites, including the Hubble Space Telescope. When a

flawed mirror on Hubble prevented the telescope from

performing, he launched a successful space repair

mission for which he was awarded a NASA Outstanding

Leadership Medal. Before retiring from the Agency in

1995, he also received NASA’s highest honor, the

Distinguished Service Medal.

Since the time that the Hubble review board identified failed leadership as the root cause of the telescope’s

failure, Pellerin began an intensive inquiry into the effects

of social factors on project success. As a consultant to

NASA, accounting firms, and aerospace companies, he

has developed a “four-dimensional” approach to

measuring and improving the effectiveness of project

leadership, project culture, and organization interfaces.

Would it be fair to say that since leaving NASA you

have focused your career on trying to understand why

projects fail? 

Yes, why projects fail around social issues and around

leadership—because I saw this same thing coming up

again and again.

Can you explain what you mean by “social issues,” and

how they relate to leadership?

I was the Director of Astrophysics at NASA between

1983 and ‘92. I had a lot of flight projects going on,

perhaps as many as 20 at any one time, and I would try

to watch and pay attention to them all. That was my

primary job, I thought.

I began to see a pattern repeated far too often when

a successful project manager would get promoted or

leave a project for some reason. I would replace him with

someone who looked just as good on paper, but three

months later, all of a sudden, the project started to fall

apart. Milestones got missed. Reserves depleted too fast.

I was frustrated that I couldn’t anticipate and

recognize the difference between project managers who

were going to succeed and project managers who were

doomed to fail. We could predict things like sensor

performance. We could understand the detectors. We

Dr. Charles Pellerin

INTERVIEW 

32 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

Starting out as a bench engineer and later serving for a decade as

Director of Astrophysics, Dr. Charles (Charlie) Pellerin’s NASA career

spanned nearly 30 years
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 33

could understand the power systems. But we couldn’t

understand this one critical, invisible piece: What makes

a good manager? 

So, I pondered this. I read books and thought about

it a lot, but I got very little insight.

Was it the magnitude of the Hubble telescope problems,

launching it with a flawed mirror, which brought this all

to a head? 

Yes, exactly. If you go back to what was happening at the

time, we launched Hubble in 1990 and very soon thereafter we found that a technical person had made an

error. At first we thought, “Now at least we know what

the error was. We can figure out how to fix it.” And that’s

just what we did—we fixed it. This would appear to be a

very happy story for me; I got a NASA medal for the

repair mission.

That’s all well and good, but then I said, “Wait a

minute. We should have had systems in place to find this

kind of thing.” The procedures are written. The engineers

sign them. Safety & Quality Assurance stamps it all to

verify that this is being done properly along the way.

Hubble was the final straw for me. I needed to understand what had happened, because when I looked around

me I realized it was commonplace. I mean, take a look at

Challenger. It was not, in a sense, a technical failure. It was

another human communications failure. I knew a bunch

of those people. They were damn good managers and

engineers, but they got caught in a story. They created an

environment where it wasn’t safe to tell the truth.

That’s interesting how you describe it as people who got

“caught in a story.” How do stories figure into this

leadership quotient?

The stories that you carry affect how you make decisions

in your life. That’s why I’m very interested in the stories

we tell. We all perceive reality through the filter of the

“stories” we believe. We create stories to make sense of

our experience. And, we act within this context as if it

were truth, because to each of us it feels like truth.

I collect and study the impact of organizational and

personal stories. Here’s an example: An entire American

industry clung to their long-held story that “Improving

quality means higher cost” until they had lost over 60%

of domestic market share to a foreign industry. I am

speaking, of course, about the automotive industry.

Similarly, it’s fascinating to help projects discover

some of the stories they carry with them. They tell stories

about their contractors, managers tell stories about

scientists and vice-versa.
What kind of story did you tell when you were Director

of Astrophysics?

The primary story that drove me, for good or bad, when I

was Director of Astrophysics was that “We have a perishable opportunity here.” Never in the history

of man had anyone invested what we were

investing in astronomy.

If you added up all of the ancillary

support, like the Shuttle launches and the

civil service manpower I didn’t have to pay

for, I probably had $2 to $3 billion per year

to look at stars. I told people, “This is

never going to happen again in our

lifetime, and we’re going to make the best

of it. If you’re not dedicated to giving your very best to it

every day, go to work someplace else because this is a

privilege and opportunity we have.” That was the story I

told at that time.

What was it like to work for you?

You’d have to ask the people who worked for me, I

suppose. I tried to be fair and not to confuse “toughness”

for “meanness,” but there were certainly people who were

uncomfortable in my division. I had a reputation for

being tough. I remember that during one performance

appraisal, a boss said to me that I would do better if I

could “suffer fools gladly.”

I can recall people who joined the division who said,

“You’re not the son of a bitch that people say you are.

You are actually a great guy to work for.” I said to them,

“Well, don’t ever tell anybody. I want people who like to

work for SOBs to come here.”

It was this simple. If you shared my story, you would

be comfortable. If not, you wouldn’t. I recall one hire

who believed “Headquarters would be a nice place to

ease into retirement.” He didn’t last long in the

Astrophysics Division.

What are some of the other stories that you’ve told?

I have lots of them. As Howard Gardner says, “Leaders

achieve their effectiveness chiefly through the stories

they relate.” Here’s one: A day’s work for a day’s pay.

This was often said to civil servants during Lyndon

Johnson’s Presidency. The variation I used with my staff

went something like this: At the end of the day, I want

you to stop at some small house outside the Capital

Beltway, preferably one with a car in front that’s worth

INTERVIEW CONTINUED

34 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP
F. Story Musgrave and Jeffrey A. Hoffman (below) prepare to install protective covers

for magnetometers on the Hubble Space Telescope in December 1993.

ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 35
INTERVIEW CONTINUED

36 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

Mission Specialist James H. Newman tests the Portable Foot Restraint

used on the first Hubble Space Telescope servicing mission.
less than my titanium bicycle. Go inside and tell the

people what you did during the day and what you were

paid. If they agree that they got good value for their tax

dollars, then you’re doing good work. If you can’t

imagine passing that test, you should work elsewhere.

Here’s another one: When I was in college and

running out of money, my roommate and I started a

monogrammed sweatshirt business. We made a ton of

money in four months. That was what mattered at the

time. But the story I learned over the years is that making

money is easy. What we need to focus on is making a

contribution, and that’s a lot harder.

Closely related is another story I carry with me:

Those who love you deserve a life of service in return.

And, you know, I tell my wife every day that I live in

service to her, but I don’t think she fully buys it.

What kind of stories are you telling now?

My main story (now) is that “unknown and unnamed”

social undercurrents are at the root of many, if not most,

project difficulties. These can be defined, measured, and

remedied. Helping NASA projects do this is the most

important contribution I can make with my life now.

You said that leadership was at the core of the Hubble

mishap. Do you find evidence of this in other projects?

Sure. Diane Vaughn in her book, The Challenger Launch

Decision, said she was a year into her study before she

realized that then-accepted accounts of what happened

were wrong. Vaughn concluded that the disaster was

caused by an “incremental descent into poor judgment.”

And she went on to say that the technical risks grew out

of social issues. Notice the word “social” again. She

realized that signals of potential danger had been repeatedly “normalized.” That was okay in the context of the

stories their culture supported.

And that’s not the only example. Administrator

O’Keefe cited similar language during recent testimony

on the Columbia disaster. Art Stephenson, following the

back-to-back Mars failures, said that success begins with

leadership and forming a culture where people are

permitted to succeed.

How are you trying to address these issues in your work

with APPL today?

We have been developing a leadership/culture assessment

and learning system called “Four-Dimensional (4-D)

Leadership” since 1995. We began with workshops, and

then added coaching, and now have Web-based diagnostics customized for NASA projects. Simply put, we make

three measurements in each of the social dimensions—

directing, visioning, relating and valuing—that we

believe are fundamental to effective leadership and

efficient cultures.

In a Web-based survey that only takes about 10 to 15

minutes to fill out, we ask respondents about observed

behaviors such as mutual respect, trustworthiness, clear

roles, and responsibilities. We review the data with the

project manager (our “customer”) and the leads for each of

the project teams. If the numbers for leadership or culture

effectiveness are too low, we use a combination of

workshops, telephone coaching, and consulting to improve

performance. Then we re-measure and track progress.

Does it work?

Project managers tell us it does. We’re in the process

now of working with projects at Goddard, JPL, and

Ames, and our database of validated metrics is growing

by the day.

We believe that we’re not far from having conclusive

data that these systems work and that dramatic reduction

in project failures, minor and major, are within our grasp.

I’m talking about preventing high-profile failures like the

Hubble mirror—but just as important are the less-visible

performance shortfalls, budget overruns, and project terminations that managers and teams experience every day.

I truly believe that we can identify and address the

root cause of most project difficulties. That’s my story.

And many of the projects I’m working with are choosing

to run that story, as well—because they see results.

You know, no story is “good” or “bad.” Some just

get you the results you want and some don’t. •

ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 37
38 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

LOOP 

SERIOUS PLAY BY MICHAEL SCHRAGE WILL PROBABLY BE OF

more help to you as a project manager than many of the

other books on your bookshelf. It’s about something

that all NASA project managers do to ensure the success

of their projects. We innovate by creating models,

simulations, and prototypes. But these creations, as

Schrage makes clear, are not the project, which project

managers and team members sometimes forget, and our

management never seems to understand.

The main point of the book is that developing

prototypes (models, simulations, etc.) enhances collaboration because the prototypes act as catalysts to increase

learning and understanding. Schrage points out that the

process of developing a plan (read prototype) leads to

innovation because of the behavioral changes that occur

among the participants. The prototyping process

inspires “clever interactions between people.”

This, therefore, is not a how-to book about prototyping; it is a why-to book about using prototyping as the

means of innovating, and it provides a framework for

understanding the value of prototyping. In Schrage’s

words, “Serious play is not an oxymoron; it is the essence

of innovation, less the product of how innovators think

than a by-product of how they behave.” The book argues

that this approach to prototyping fundamentally transforms how organizations approach innovation challenges.

The debris left on the wayside that results from

prototyping Schrage calls “productive waste” that

“shrinks risk.” He claims that, “The real value of a

model or simulation may stem less from its ability to

test a hypothesis than from its power to generate useful

surprise.” He cautions, however, to beware of relying

too heavily on prototypes to predict the future, but

instead to use the results as “projections” to gain

insight into the future.

The book consists of three parts: Part I sets the stage

with two chapters describing the relationship between

innovation and prototyping. Part II covers the language

of prototypes, models, simulations, and other activities

including computer-aided design and computer-aided

engineering. Part III measures the paybacks of serious

play and how to turn innovation into a way of doing

business in an organization.

In the end, Schrage makes a strong case for the

value of prototyping as the backbone of an organizational approach to innovation. In his forward, author

Tom Peters writes, “Serious Play is simply the best book

on innovation I’ve ever read.” Peters goes on to say that

Schrage’s approach to prototyping as process “is

possibly the most insightful, counterintuitive twist in a

generally insightful book.” •

Books about Prototyping

Serious Play 

Michael Schrage, 2000. Boston MA: Harvard Business School Press

Reviewed by Dr. Gerald Mulenburg

Dr. Mulenburg reviews books regularly for ASK Magazine. His last review appeared in ASK 10 on The Attention Economy.

He is also a member of the ASK Review Board.
ASK 13 FOR PRACTITIONERS BY PRACTITIONERS 39

IN ITS QUEST TO DEVELOP INNOVATIVE PRODUCTS,

California-based IDEO has designed a shopping cart

inspired by diapers, ridden exercise bikes in a meat

locker, and prototyped a computer mouse using a

butter dish. With plenty more stories like these, IDEO

general manager Tom Kelley and co-author Jonathan

Littman flesh out the philosophy behind the cuttingedge work done by America’s self-described leading

design firm.

Indeed, the stories included in Art of Innovation

make it a good read, but what if your next project won’t

involve building a better toothpaste tube or a finned

football? Is there something in this book for the typical

NASA project manager? 

There is. Chapters on brainstorming, team

dynamics, dealing with uncertainty, and creating prototypes transcend project specifics. The chapter on prototyping alone makes the book worth picking up. Kelley

gets to the heart of what a prototype should and

shouldn’t be. “What counts,” he explains, “is moving the

ball forward, achieving some part of your goal. Not

wasting time.” With examples from science, hardware

design, and movie making, Kelley outlines the use of

prototyping to solve problems, speed progress, and

minimize risk.

Throughout, Kelly addresses building and

sustaining creative, motivated teams. His “hot teams”

approach considers everything from selecting team

members to rewarding performance. “Great projects are

achieved by great teams,” writes Kelley. “Though

[projects] are naturally about groups and teamwork, too

often these groups are simply the result of inertia....That

sort of staid, unfocused group bears little or no resemblance to the hot groups we’re talking about.”

Yes, the Art of Innovation reads at times like an

infomercial for the publicity-happy design firm, but

that doesn’t detract from the book’s usefulness as a

stimulus for re-envisioning project work and thinking

outside the box. •

Jody Lannen Brady is the Associate Editor of ASK Magazine.

The Art of Innovation: 

Lessons in Creativity from IDEO, America’s Leading Design Firm

Tom Kelley with Jonathan Littman, 2001. New York: Doubleday

Reviewed by Jody Lannen Brady
LETTER FROM THE EDITOR-IN-CHIEF Dr. Alexander Laufer

40 APPL THE NASA ACADEMY OF PROGRAM AND PROJECT LEADERSHIP

THAT WAS THE CASE FOR ME IN 1988, WHEN I WAS INVITED

to do some work for the Construction Industry Institute

(CII), a research consortium of top American companies

and universities.

I had come to CII’s attention because of my work

regarding project planning. To continue my research, CII

made it possible for me to interview 39 project managers

at 11 companies. I asked each manager a series of

questions about planning at the early phase of a project.

Rather than confirming what I expected to hear, out of

this process came something I didn’t understand. Again

and again, the managers I spoke

with told me that they searched for

potential solutions, i.e. they started

engineering designs, before they

finalized their project objectives.

“Objectives first, means

second.” Define the problem, then

solve it. That is what I had been

taught as a student, and that is

what I had subsequently taught to

my students. But top-notch managers at well-respected

companies were telling me that they didn’t work that

way. In almost all my interviews, I observed the same

discrepancy. The objective formation process is not an

isolated activity, and it is not completed before searching

for alternatives begins.

This astonished me or, to be honest, it shocked me.

For a couple of months, I wrestled with what I had heard.

My wife and children have told me that it was clear to

everyone around me that something was bothering me.

We lived in a duplex, and after my neighbor heard me

pacing back and forth, night after night, he asked if there

was something wrong with me.

Because I had conducted my research accepting the

prevailing assumption (objectives before means), I hadn’t

phrased my questions in a way that could directly

disprove the assumption. Instead, my conclusions had to

be derived indirectly from my data, and this added to my

feeling of unease about the validity of my findings.

It took me a long time before I fully understood

what I had observed, and it required a lot of reinforcement. As I went back through the literature and re-read

pioneering works by highly respected researchers like

James March, Donald Schon and James Thompson,

I found support for my new understanding of 

project planning. As March wrote, “The argument that

goal development and choice are independent behaviorally seems clearly false. It seems

to me perfectly obvious that a

description that assumes goals

come first and action comes later

is frequently radically wrong.”

The old paradigm assumed

implicitly that a manager first

reduces all uncertainty of objectives, and only then begins to

develop the plans or means to

accomplish those objectives. But experienced project

managers were telling me that they simultaneously

reduced the uncertainty of both objectives and 

means. My findings showed that in most capital 

projects, not only is “means uncertainty” (how to do it)

resolved late in project life, but so is “end uncertainty”

(what to do).

So, my research led me to formulate a new

paradigm. Under conditions of uncertainty, it is impossible to finalize project objectives at the outset once and

for all. Rather, in order to set stable project objectives,

one must sometimes first explore the means. That is

what I learned.

We can learn much from research—but very 

often we need to be willing to engage in a little

unlearning first. •

Letting Go of “Once and for All”

Research is about discovery. Sometimes, if we let it,

research can shake our core beliefs

It took me a long time before 

I fully understood what 

I had observed, and it required 

a lot of reinforcement.