Written Contributions to the EHR Advisory Committee

Public Hearing on Employers' Views

*Convened November 1, 1995*

*At the National Science Foundation*

*Arlington, VA*

Invited Speakers:

"Employers' Views on the Preparation of SME&T Undergraduates"

*Listed in order of Testimony. Titles indicate the speakers' positions
at the time of the Hearing.*

+-------------------------------------+--------------------------------+
| **Baltimore Public Schools**        | **Walter G. Amprey**           |
|                                     |                                |
| *Baltimore, Maryland*               | *Superintendent*               |
+-------------------------------------+--------------------------------+
| **Columbia University**             | **Eugene Galanter**            |
|                                     |                                |
| *New York, New York*                | *Professor of Psychology*      |
+-------------------------------------+--------------------------------+
| **New York Hall of Science**        | **Peggy Ruth Cole**            |
|                                     |                                |
| *New York, New York*                | *Director of Program Planning  |
|                                     | and Development\               |
|                                     | *                              |
+-------------------------------------+--------------------------------+
| **GHG Corporation**                 | **Israel J. Galvan**           |
|                                     |                                |
| *Houston, Texas*                    | *President*                    |
+-------------------------------------+--------------------------------+
| **Hewlett Packard Company**         | **Alfred Moyé**                |
|                                     |                                |
| *Palo Alto, California*             | *Manager, University           |
|                                     | Relations*                     |
|                                     |                                |
|                                     | **Robert W. Ritchie\***        |
|                                     |                                |
|                                     | *Director, University Affairs* |
+-------------------------------------+--------------------------------+
| **Boeing Commercial Airplane        | **John H. McMasters**          |
| Group**                             |                                |
|                                     | *Senior Principal Engineer,    |
| *Seattle, Washington*               | Aerodynamics Engineering*      |
|                                     |                                |
|                                     | **James D. Lang\***            |
|                                     |                                |
|                                     | *Director, Technology          |
|                                     | Division, New Aircraft and     |
|                                     | Missile Products, McDonnell    |
|                                     | Douglas Aerospace, St. Louis,  |
|                                     | Missouri.*                     |
+-------------------------------------+--------------------------------+
| **National Alliance of Business**   | **Robert Jones**               |
|                                     |                                |
| *Washington, District of Columbia*  | *Executive Vice President*     |
+-------------------------------------+--------------------------------+
| **Shell Oil Company**               | **John J. Sisler**             |
|                                     |                                |
| *Houston, Texas*                    | *Manager of Exploration &      |
|                                     | Production Training*           |
+-------------------------------------+--------------------------------+
| **Bell Atlantic Corporation**       | **Patrick E. White**           |
|                                     |                                |
| *Arlington, Virginia*               | *Vice President, Strategy*     |
+-------------------------------------+--------------------------------+

*\* Additional author of contributed remarks.*

Employers\' Views on Desired Capabilities

of Undergraduate Students Entering the Workforce

Walter G. Amprey

*Superintendent of Public Instruction, Baltimore City Public Schools\
Baltimore, Maryland*

I thank the National Science Foundation for this opportunity to testify
on behalf of public education concerning the undergraduate preparation
of students entering the workforce. Our \"industry,\" public education,
develops a \"product\" that ultimately must support a regional and
national economy and must be the source of problem-solving ingenuity and
innovation to meet the challenges of change. The university
undergraduate product we depend upon to mold this \"product\" is the new
teacher: either a science, mathematics, engineering, or technology
(SME&T) major who often enters secondary education or a non-SME&T major.
Even though these teachers receive formal instruction in content and
methodology, they are not fully prepared to teach effectively as soon as
they graduate.

Many new teachers arrive at their first assignments lacking
sophisticated skills in writing, speaking, and computing. These are
essential skills for success in any workplace. In addition to these
\"basic\" skills, all graduates should be able to think logically and be
able to show their students how to use higher level thinking skills.

Technological literacy is another basic requirement for successful
citizens of the Information Age. All new teachers should be able to use
technology and adapt to its roles and applications. They also should be
able to appreciate the power of science, mathematics, engineering, and
technology in everyday living and should understand how mathematics and
science support, undergraduate, or clarify concepts in other disciplines
such as art, music, and social studies and in diverse career fields.

Along with these \"basic\" skills, new teachers should understand
organizational \"culture\" and the need for continuous improvement. When
entering the workforce, all new teachers should be able to contribute to
team efforts that focus on an organization\'s goals and respect their
co-workers\' abilities. New teachers should also enter the workforce
seeking more education. They must understand that continuous study and
research, both formal and informal, are essential, and that preparation
for teaching does not end at college graduation or with a master\'s
degree.

SME&T content are also essential for all new teachers. Non-SME&T majors,
especially those, who will become elementary teachers, need broad-based
knowledge of SME&T content, so they can integrate material into any
subject area. They also need to be aware of career opportunities for
their students and be able to direct students to appropriate course work
for their career goals.

While all teachers should be technologically literate, those who become
SME&T teachers should be able to use technology as a pedagogical tool in
their classrooms on a regular basis. SME&T teachers also should be well
grounded in modern pedagogy and be able to apply modern pedagogical
techniques to their content areas.

In their content areas, SME&T teachers should know more about the
subject materials than they are required to teach. For example, a
pre-college mathematics teacher for senior high school should be able to
teach freshman and sophomore courses at the college level. Similarly, a
chemistry teacher should be able to handle undergraduate skills and
concepts in addition to knowing the state-of-the-art equipment.

New SME&T teachers should have the benefit of sufficient
practicum/internship experience before they graduate. After they
graduate, these new teachers should have mentors who are experienced
teachers as well as mentors who are based in business and industry.
SME&T teachers cannot lose focus on the world of SME&T beyond the
classroom.

In general, all SME&T majors must be able to adjust to the changing
demands of the workplace and use the higher level skills of thinking in
mathematics and science to be productive workers. As requirements of
business and industry change and as corporate downsizing becomes a
reality, graduates must have the transferable skills and flexibility
that will provide them with successful livelihoods.

Undergraduate SME&T education for majors and non-majors is influenced by
several trends. First, the increased requirements in mathematics and
science in precollege education will require colleges to introduce more
rigors in college-level courses. Second, the increased number of
students in the pipeline for SME&T at the precollege level will increase
the number SME&T majors at the college level. This will require more
college SME&T faculty. Finally, the increased number of SME&T-literate
precollege students will increase the number of majors in college and
technical schools who will request training in SME&T-related fields.

Ideally, undergraduate SME&T education for potential teachers should be
developed through collaborations of local school districts and
universities. For example, the National Science Foundation funds the
Maryland Collaborative for Teacher Preparation, an alliance of the
University of Maryland system, Morgan State University, Baltimore City
Community College, and three Maryland public school systems. This
project focuses on elementary and middle school science and mathematics
instruction and supports joint efforts to meet the needs of teachers new
to the profession. Many informal collaborations coexist with such formal
arrangements. Thus, individual public school staff members interact
frequently with their colleagues on college faculties because of
numerous projects that tie them together. In addition, a number of
institutions of higher learning assist local districts in meeting their
identified needs.

The standard school district/University interface, however, is the
student teaching program, where young people preparing to be teachers
work in classrooms under the joint guidance of a classroom teacher and a
college professor. Because individual school systems are institutionally
separate from the teacher training institution, school district needs
are not always met as quickly or directly as possible since universities
must meet their own goals and objectives. This \"extended reaction
time\" sometimes keeps the teacher training institutions one step behind
the needs of the school district. What is needed is a process for more
rapid and direct implementation of system needs into college programs.

Improved articulation between school districts and teacher training
programs also would reduce the need for \"remedial\" teacher training.
Because so many new graduates need in-service training to perform
satisfactorily, it appears that the training programs at the college
level lack content and pedagogy or lack understanding of the needs of
the school district. New teachers must be able to apply educational
theory in their daily teaching practices. Too often, the practices
demonstrated by professors in teacher preparation courses are not the
state-of-the-art pedagogical methods that are needed for instruction in
precollege classes.

SME&T curriculum for both majors and non-majors must be broadened and
deepened if we are to improve elementary and secondary instruction.
Preparation of secondary teachers, for example, usually includes almost
as many courses as a SME&T major; however, new teachers should be up to
the standard of all other graduating SME&T majors. Elementary teachers
enter the workforce with very little content background in science and
mathematics. Further, they are not challenged to become more prepared
because "seasoned" administrators whose backgrounds may be lacking in
science and mathematics content supervise them. Thus, a
self-perpetuating cycle of \"science and mathematics weak\" elementary
curricula establishes and maintains itself

Both pedagogy and curriculum can best be improved through the
collaborations of precollege and undergraduate faculties. With such
alliances, curricular and pedagogical innovations could better address
the knowledge and performance bases of students \-- and better prepares
them for jobs and career paths.

Finally, as colleges and universities work with school districts to
update and upgrade teacher preparation programs, states should
re­-evaluate their requirements for teacher certification. Currently, a
limited number of SME&T courses is required, especially for elementary
teachers. New standards should emphasize the integration of mathematics
and science in other disciplines such as art and history where
high-level skills in problem solving and decision making are required.
States also need to increase their requirements for certification of
mathematics and science teachers as more rigor is introduced in college
and precollege courses. In general, certification requirements should
give all potential teachers more practical and comprehensive
\"hands-on\" experience in mathematics and science.

To meet the requirements of the 21st century workplace, we must work
harder to disseminate needed improvements in educational practice. On
the national level, roundtable discussions and other forums involving
school districts and teacher education institutions could address
problems. Direct local interaction on a regular basis can resolve
problems as they occur. Collaboratives already address some of those
needs, but superintendents and college administrators need to interact
more frequently. This will prevent the bureaucratic needs of either
institution from overwhelming the common goal of providing and employing
a competent SME&T workforce in education.

Local districts such as the Baltimore City Public Schools look forward
to continued partnership with the National Science Foundation, other
federal agencies, and other funders. Specially funded programs have
produced many positive outcomes. For example, more minority students
enter the SME&T pipeline and consequently enter lucrative career paths
where they have historically been under-represented. These programs have
also provided the dollars to assist these young people as well as
dollars to assist colleges and universities establish SME&T programs.
NSF grants to colleges and universities have provided equipment \-- and
resulting technological literacy \-- for students training to function
successfully as scientists, engineers, or teachers. Specially funded
projects have developed teacher-training programs, such as \"STARS,\"
featuring state-of-the-art hands-on/minds-on pedagogy. In the future, we
hope organizations such as NSF will develop more collaboratives to help
us meet critical needs.

In recent years, changes in our society have produced changes in the
direction and practices of education. We have become more sophisticated.
We have adopted the principle of ethnic and gender equity. We expect all
students to know more and to perform at higher levels. The growth of
technology and its application in the mathematics and science classroom
require that new teachers be computer literate and that they be
adaptable to new technologies and techniques as they appear. Because we
expect students to know more, we expect new teachers to know more and be
prepared to learn more throughout their careers. Also, as we implement
performance-based instruction, cooperative learning, and
\"hands-on/minds-on\" strategies, teachers must be prepared with an
expanded \"bag of tricks\" so they can deliver a 21st century
curriculum.

***Walter G. Amprey** has been Superintendent of Public Instruction of
the Baltimore City Public Schools since 1991. From 1973 to 1991, he
served in administrative positions in the Baltimore County Public
Schools -- associate superintendent for staff and community relations,
associate superintendent for physical facilities, director of staff
relations, and principal and assistant principal. In 1966, Dr, Amprey
began his career in education in the Baltimore City Public Schools as a
social studies teacher then served from 1971 to 1973 as a school
administrator, He holds a doctorate in education from Temple University,
a master\'s degree in educational administration and supervision from
Johns Hopkins University and a master\'s degree in history/social
science and a bachelor's degree in history/secondary education from
Morgan State University. Dr. Amprey was honored as Maryland
Superintendent of the Year, 1994-95, and served as president of the
Large City Schools Superintendents of U.S. and Canada, 1995-96. Most
recently, Dr. Amprey served with Reverend Jesse Jackson as facilitator
of the Rainbow/PUSH Coalition\'s Public Policy Education Conference,
\"Closing the Gap,\" in Chicago, Illinois.*

Testimony to the NSF Undergraduate Review Subcommittee

Eugene Galanter

*Professor of Psychology, Columbia University\
New York, New York*

Real improvement in science education (SME&T) at the baccalaureate level
will require: a) revisions in instructor attitudes and interests; b)
attention to multiple cognitive and motivational functions of students;
c) revisions in standard modes of instruction; d) new evaluation
procedures; e) augmentation of new technology; and finally f) serious
longitudinal studies. I will attend mostly to items a - d. The last pair
are technical issues that need attention by experts in these fields.

To begin with, instructors (clearly not all) must recognize that
students don\'t think science or math is much fun. They are mainly
forced into science by curricular or career path requirements, e.g.,
chemistry or biology for medicine, psychology for aspects of medicine or
social or technical professions, and physics and math by those with
self-recognized quantitative or computational skills for technical or
scientific program management. Engineering students are self-selected
for these skills, and comprise a less intense problem for engineering
instructors. However, many of these students at research institutions
are more interested in engineering science than real engineering. In my
discipline (experimental psychology), students choose our introductory
courses to satisfy (effortlessly they hope) a requirement, or to find a
less demanding career with the cachet of science. They may then discover
that some of the materials are daunting, or that the substantive
materials barely speak to their interests.

I attack the question of how to advance science education at the
undergraduate level by asking several questions. First: how many
students really want to take science in college? The answer is very few.
Which sciences do the students take? Our data show that except for
pre-meds, they opt for the easiest, the least quantitative, and the most
descriptive. The third question: Do these sciences provide opportunities
for such students to engage seriously in the technical world: The
answer, I am afraid, is maybe not. The fourth question is: What can we
do? And that constitutes the heart of my suggestions. These suggestions
must be tempered by the enormous revolutionary events that are occurring
now in our colleges and universities. My colleague, Eli Noam in the
Columbia Business School, has just published an article in *Science* on
the demise of the university as the mechanism by which information
transfer occurs. His point is that, in the past, the user went to the
source. Now the source is distributed to the user. So what does the user
need our real estate for? The consequence is that this will engender
enormous changes, changes in which instrumentality's that we have never
thought of as the educational base of our activities will become the
instrumentality's by which information does flow to our consumers, if
you will pardon that expression.

Back to our primary topics. Once instructors recognize these default
attitudes of students --requirements or jobs -- they can often loosen-up
the relentless drive to provide technical materials that span a broad
spectrum. They must remember that their own initiation required many
years of hard work. Part of the difficulty is a failure to notice that
our students are taking other courses as well as our own. The demands on
their time can create a climate of anticipated failure. I will say more
about this phenomenon later.

Students have limited experience. The kinds of experiences they have are
not connected to atomic structure, to spiral nebulae, et cetera. They
are connected to why can't my grandfather remember things any more? How
is it that (I have been told) still images on a television screen look
as if they are really people moving around? What is virtual reality? How
can athletes do the stuff they do, and so on.

My own field of experimental psychology is useful for discussion because
it covers an enormous class of phenomena, and also interlocks with
biology, chemistry, and physics through neurosciences and psychophysics
on one side, and the personal and social sciences on the other. As such
it will serve to illustrate and informally substantiate my views on
under­graduate SME&T in the large. To summarize the discussion to follow,
I suggest that the paths to continuing progress in SME&T are:

-   increasing relevance

-   restructuring content

-   modularization

-   motivational design

    In order to make science attractive to students who are not yet
    career driven, but who may be drawn into a career choice by their
    college experiences, our offerings must have *relevance.* This
    much-abused concept refers here to the ability of a course of study
    to raise and answer questions important to (the limited experience
    of) students. In the last ten years many curricular efforts have
    attempted to achieve this relevance, sometimes, unhappily, with a
    loss of precision. In my first attempt to make psychophysics an
    integral part of introductory psychology[^1]. I opened with the
    question of how an airliner is guided to a safe landing during
    weather conditions that defeat direct visual control. Such tasks
    depend on translating visual codes into skilled action. In recent
    text materials[^2] topics in signal delectability theory center on
    issues of radiological diagnoses of breast cancer, and failure of
    memory in recognizing faces. Such questions engage the student with
    problems they can understand. If science informs the answer,
    relevance is insured.

    Are there enough relevant topics? Again in my own field consider a
    spectrum of subjects such as memory loss through aging or accident,
    virtual reality from apparent motion through full-fidelity
    simulations, individual differences in intelligence and other
    personal characteristics, cognitive development, social and
    political consequences of individual attitudes and prejudices, motor
    skills and athletics, sexual preferences and aversions, human error
    and its consequences; the list could continue to a score or more.

    We are *restructuring* the content of early science education in
    psychology to motivate interest and to offer rational and more
    importantly --plausible -- ways to understand and solve real
    problems. I mean to do this by making data collection one of the
    centers of effort to confront the questions of fact and theory.
    Students who collect their own data as compared with students who do
    problems in a book *own* the data in a way that the book never
    provides. When they are really your reaction times to stimuli of
    various kinds, then you may say, "hey, can I respond faster to red
    or green." Having those personal data becomes and intrinsic
    motivator that is critical to the continuing activities of the
    student. And so rather than follow some preferred sequence,
    including prerequisite course work, structure has been changed to
    show relevance. The tested retention of such information in our
    admittedly minimal current evaluations suggests enhancements suggest
    enhancements in understanding by a factor of six.

    A third advance in my own field, and extendible to other
    disciplines, is the tech­nique of *modularization.* This refers to
    the construction of independent sub-parts of a discipline along with
    its own auxiliary technical materials. Units of study do not need to
    be timed to coincide with rigid time frames. If we want to talk
    about memory and about lapses of memory, then, we could have a
    memory module, and that memory module might last for two lectures or
    four lectures or six lectures. It might include experiments to see
    how memory works, how it dissolves and disappears, et cetera.

    Our current NSF support has helped us develop a variety of
    stand-alone quantitative modules on topics in the human sciences
    from biology to sociology. We have modules of varying duration on
    human genetics, evolution, psychophysics and decision theory, space
    and motion perception, and social psychology. We plan to construct
    additional modules on cognitive mechanisms in perception and action,
    motor organization, language and thought, theories of memory,
    virtual reality, human engineering and ergonomics, psycho­metrics,
    animal conditioning, learning and cognition, and quantitative
    sociology.

    Each of these units is supported by quantitative material that
    undergirds the topic. We do not demand prerequisites in, say,
    calculus, statistics, algebra, or probability theory. Rather, we
    incorporate the required techniques in the module itself. Any single
    module is studied for its nominal duration, which currently ranges
    from 4 to 14 fifty-minute lecture/labs. The ancient bookkeeping of
    academe proscribes such plans. Our modules currently fill the
    standard semester time scale, and are given in a "rational\" order.
    The ramifications of these historic constraints are reviewed later.

    Table 3.1:

+--------------+-----------------+---------------+-------------------+
| **Theory of  | **Goal          | **Confidence  | **Behavior        |
| In           | Orientation**   | in Present    | Pattern**         |
| telligence** |                 | Ability**     |                   |
+--------------+-----------------+---------------+-------------------+
| **Entity     | **Performance   | if **High**   | **Mastery         |
| Theory**\    | Goal**\         |               | Oriented**        |
| (IQ is       | gain positive   | if **Low**    |                   |
| fixed)       | judgments       |               | seeks challenge,  |
|              |                 |               | high persistence  |
|              | avoid negative  |               |                   |
|              | judgments       |               | **Helplessness**  |
|              |                 |               |                   |
|              |                 |               | avoid challenge,  |
|              |                 |               | low persistence   |
+--------------+-----------------+---------------+-------------------+
| *            | **Learning      | **High** or   | **Mastery         |
| *Incremental | Goal**          | **Low**       | Oriented**        |
| Theory**     |                 |               |                   |
|              | increase        |               | seek challenge,   |
| (IQ is       | competence      |               | high persistence  |
| malleable)   |                 |               |                   |
+--------------+-----------------+---------------+-------------------+

From Dweck, C. Motivational processes affecting learning. *American
Psychologist,* October 1986.

In psychology the computer has become the required instrument for
experimental research and field data tabulation. It is our microscope,
centrifuge, accelerator, or tele­scope. Few institutions provide adequate
access to these machines for undergraduate use. In my own department of
fifteen tenured faculty, our financially challenged administra­tion has
recognized the importance of such investment, and has provided 16
machines for undergraduate labs. These sequestered microcomputers have
made \"open\" labs possible. Only one graduate (or often an advanced
undergraduate) supervisor is needed, and the lab is available to
students in several courses on an essentially ad lib. basis. There are
*ca*. 600 undergraduates per semester in our course offerings, so the
ratio is quite small. On the other hand only about 150 students need
access to the machines. With increased curricular implementations, we
anticipate that this 1:10 ratio will probably increase to 1:25, a figure
that is probably close to optimum. As with any computer implementation,
we are hampered by software limits. We have developed some experiments
for use on our machines, and will be offering these materials on our
Internet home page.

Just as lab equipment (in our case microcomputers) is often in short
supply, so also are teaching materials. In the modules we are
developing, primary sources are the vehicle of choice for students to
read. However, for many undergraduates it is often necessary to provide
some intellectual resources that may not be part of their apperceptive
mass. To this end we need texts that can provide explicatory material to
permit students to engage the primary source scientific writings that
have cast light on the central issues they study. We are trying to
provide such texts in our curriculum development program, but as any
teacher will recognize, this task is enormously difficult and time
consuming.

The limit imposed by academic bookkeeping prevents us from using new
formats and methods creatively. With free standing modules on a variety
of topics keyed to questions of obvious and direct importance, we need
to allow students to select a group of such modules based on their own
interests. If we intelligently metricize the credit value of each
module, we can allow students to select from a menu of topics a
personal­ized subset for study during a standard academic time period.
Students might be expected to take a variety of admixed modules, any
combination of which would be accepted by a registrar for appropriate
academic credit. In combination with newly designed evaluation
procedures that we are currently exercising, the entire task of
selection (i.e., advising), pedagogy, and credentialing could be
simplified and packaged to minimize student anxiety and time
constraints. These revisions have direct implications for various sorts
of distance learning, and other forms of non-institution based
education.

The previous reference to evaluation needs expansion. In science
education we have commonly used so-called objective test formats to
evaluate student comprehension. Critics have charged that
\"constructive\" tests better measure student knowledge. Indeed, this
topic is quite current[^3], but fails to recognize the cognitive
continuum between, say, a true-false test and a constructive essay. We
have developed several waypoints on this continuum, and is currently
analyzing correlation between double blind scoring of quasi-constructive
answers, and items on objective (four-alternative) examinations. Our
preliminary findings support the view that the correlations are very
high (*ca*. +.85). We score the objective tests by requiring students to
eliminate incorrect answers. Originally we gave unit weight to each
incorrect answer. Now we have installed our examinations on our Web
page, and process the test data by more sophisticated statistics. We
will shortly be able to differently weight the incorrect answers as
"easier\" or \"harder,\" and in this fashion improve the ostensible
validity of the score. Of course, all of these improve­ments are based on
plausibility, and wait real longitudinal validation.

Finally, the fourth topic -- *motivation --* stands at the center of our
efforts to bring science to those who are traditionally absent from the
feast. Of the 186,013 baccalaureate degrees in science and engineering
awarded in 1989, 9325 were awarded to blacks[^4] (the data are similar
in structure for Hispanics and Native Americans). Roughly 5 percent of
the students are drawn from 13 percent of the population. We
characterize this as a motivational deficit, not an intellectual
one[^5].

To attack this problem we must recognize the distinction between
learning and performance. What we mean by the goals of learning are to
increase confidence to act. What we mean by performance goals are to
seek positive personal judgments. That is, performance means you have
gotten such a good score on the test, or you have gotten such a bad
score on the test. Either of those two can lead to problems down the
road. When we can make clear that failure to learn is a natural part of
learning, and if we ensure that the goals of learning are learning
goals, then we may observe mastery after failure. There are now more
than 50 studies that demonstrate that this is the case. We also believe
that an entity theory of intellect can be replaced by an incremental
theory, one where IQ is malleable. The format of our modules tries to
accommodate these principles.

We know that failure in performance goals leads to hopelessness and
rejection of the intellectual content. We must examine how we frame our
material so students understand that it is not their performance that
counts, but rather their ability to grasp the issues and solve the
problem. And if they can't solve the problem, then that failure is
intrinsic to actively searching for more information.

Let me conclude with some tangential remarks about how we might fix some
parts of SME&T; in particular how some aspects of these revisions would
work. Change would occur because students would do things. The teacher
would say as little as possible. That is hard to implement. After all,
because of my competence in language and delivery, I can amuse you, and
it rewards me to see you smile, applaud, and so on. Teachers are mostly
like that everywhere. However, the rewards to the teacher are mostly
irrelevant to the student. What the student wants to know is how to do
something, not to be told about something. In science, doing is
everything. Ideas are cheap. It is work that is costly. We have
concentrated too much on the transmission of ideas, and not enough on
the transmission of effort, activity, and the way to do it.

Unless we are proactive in respect of the changes that have to occur, we
are going to be left in the dust. But there are things we have to know,
e.g., whether any of these revisions have consequence. I can talk about
students being happier, students getting better grades, students
inviting others to join the course, et cetera. But if we don't know how
many students go to the right jobs or whatever, then all of this is just
talk.

We need to plan major longitudinal studies to assess the consequences of
science education. We must answer the hard questions: Do the things we
are trying to do now have real consequences? To find such answers will
require a coherent and well-organized research enterprise to see what
happens 5, 10, and 15 years after these changes go into effect. In order
to do this I think we must mount a federal enterprise with continuity in
design and support. Management has to be administratively structured in
such a way that students can be tracked into graduate school, the market
place, and their later lives. Do we change from 3 percent entering
technical work to 7 percent because of the changes we make now or not?

Finally what about our input? What is K-12 science education today? We
know that an enormous problem is the turnoff that has occurred in the
middle and senior high schools in science. Now, attempts to rectify that
by talking about imitation science -- I don't mean that in any
disparaging way -- but things like nature studies and fanciful ecology
are not going to increase our entry quality, because the first time
these kids take a college course that is serious about these issues,
they are going to have to look at population densities and distribution
functions of animals, and genetic structures, and they will say, "hey,
where is the ecology." And we must answer it is in there somewhere, but
we have to work to get to it. K-12 education, as you know, is run in a
political environment that is essentially inviolate when it comes to
attempts by people in collegiate education to impose anything. We have
to suck them into it. We can't push it down. They will simply walk away.
They may not have exactly the training that is needed to accommodate the
materials we suggest they provide. In which case they are going to be
very tough about saying, "well, I think the stuff I am teaching is much
more important than what you are doing, and my kids love it, and they
are doing very well, thank you, and furthermore 70 percent of the class
got A's." To reach the common schools will be a much harder job than
getting our own act together.

*Professor Galanter received his Ph.D. from the University of
Pennsylvania in 1953, and advanced from Instructor to Professor from
1952 to 1959. He was appointed a Research Fellow at Harvard from 1955 to
1957. He was a Fellow at the* Center for Advanced Study in the
Behavioral Sciences *in 1958-59. His work with George A. Miller and Karl
Pribram on a new cognitive psychology was published in 1960 as* Plans
and the Structure of Behavior*. Following his return from the* Center*,
he began a collaboration that resulted in the three-volume* Handbook of
Mathematical Psychology*, published during 1963-65. He completed his*
Textbook of Elementary Psychology *in 1966. In addition to these books,
and his three popular books in the* Kids and Computers *series published
in 1982-84, he has contributed more than 100 reports, journal articles,
and book chapters in the fields of psychophysics, mathematical
psychology, aviation psychology, and utility measurement. Professor
Galanter was made Chair of the Department of Psychology at the
University of Washington in 1962. In 1966 he went to Columbia University
as Joseph Klingenstein Professor of Social Psychology and Race
Relations. He was named Professor of Psychology and Director of the
Psychophysics Laboratory in Columbia University in 1967.*

Testimony Delivered to the National Science Foundation\
Review of Undergraduate Education Hearings on

Employers Views on Desired Capabilities of\
Undergraduate Students Entering The Work Force

Peggy Ruth Cole

*Director of Program Planning and Development, New York Hall of Science\
Flushing-Meadows Corona Park, New York*

Before coming to the New York Hall of Science 11 years ago, I was
principal of an independent school in New York City, and before that a
graduate faculty member of Bank Street College of Education, a teacher
training institution also in New York City. I have 15 years of classroom
teaching experience at both the elementary and secondary levels.

While I will limit my examples to issues related to our particular
science center, I am also speaking on behalf of the 300
science-technology centers around the country serving over 76 million
annual visitors including children, families, school groups, and
teachers. Informal science institutions offer training to 150,000
teachers annually. All these institutions have an informal approach to
science, mathematics, and technology in common. Unlike schools, they are
discretionary environments where visitors chose to come and select their
experiences based on interest, attraction, and a variety of other
factors. To quote the late Frank Oppenheimer, founder of the
Exploratorium in San Francisco, \"Nobody flunks a museum."

Science centers have the potential to improve the undergraduate
preparation of new teachers. We have an externally evaluated program in
operation for 9 years, which is a model for other science centers. Our
Science Teacher Career Ladder, a program within our Science Career
Ladder, serving mostly women and minority youth, involves cooperation
among the science center, colleges and universities and schools.
Together, we provide a support and training package which colleges and
universities cannot supply alone.

The program design is effective. A survey of almost 25 percent of our
Career Ladder alumni showed that 34 percent of them are now teaching.
And our record of attracting women and minorities to teaching is high.
Of our Science Career Ladder alumni now teaching, 21 percent are
African-American compared with a national average of 5 percent of
current teachers; 14 percent are Asian compared with a national average
of 1 percent; and 13 percent are Hispanic compared with a national
average of 3 percent.

How does the program work? We employ 40 high school and 60 college youth
annually as our floor staff. They work part time in a science- and
technology-rich environment, and receive both pay and school credit for
the experience.

Their job is to explain science to a broad public, from pre-schoolers
through senior citizens. The stimuli for both Explainers and visitors
are l70 hands-on interactive exhibits, ten on-floor demonstrations, and
science activities delivered through workshops and special events. Our
youth staff learn science and they learn how to teach science in an
environment free from grades and exams, but rich with science phenomena.
As a result, many discover both an interest in science and pleasure in
sharing it with others. Consequently, the majority decides to become
teachers.

These students have, on the whole, come through a school system that
teaches science through the traditional avenues of textbooks,
lecture/demonstrations, and structured laboratory experiences with a
reward system centered on high grades. Most never see the real-world
application of science and math and they rarely get to meet and interact
with people who work in these fields while they are still in decision
making positions that will effect their entire futures. The result, as
we all know, is an early turnoff to math and science by the very
population of women and minorities that will, by the year 2000, be the
majority entering the workforce.

The Science Career Ladder is almost 10 years old. It has been a model
for 12 other science centers around the country that have adapted
aspects of the program. In the past decade we have worked with 20
colleges and universities and 14 local high schools. Almost without
exception, we have been able to develop meaningful partnerships with
Cooperative Education Departments who do not offer content course
credit. In some cases we have developed relationships with Schools of
Education but rarely with departments of academic science: physics,
biology, chemistry, etc. The culture of undergraduate institutions is
curriculum driven, departmentally structured, grade oriented -- formal.
The culture of science centers is content driven, discretionary,
intergenerational, exploratory, visitor driven -- informal. We are
working hard to bridge this gap and to help colleges see our program as
relevant and meaningful to their enterprise, but the academic structure
that informs course development and credit makes it very hard.

As employers of youth, in a rapidly expanding industry, with a
significant shortage of well-trained minority youth, we have learned to
"grow our own." We identify women and minority youth who traditionally
are not represented in science and technology fields, and offer them
science-related employment to both serve our public and to prepare them
for the workforce. And we are succeeding. In addition to producing
science teachers, the 1994 survey showed that 9 percent are working in
museums, and 15 percent are working in science and technology fields.
The majority of our Career Ladder alumni are African American, Latino,
or Asian, and the minority of white students includes recently
immigrated ethnic groups. They are poor, often first generation college
students.

What are the workplace literacy skills we are developing in our youth
staff? Because these positions are authentic, paid, and essential to the
functioning of the museum, our demands are those of any other employer.
We expect punctuality, responsibility, appropriate dress and
professional demeanor, teamwork, the ability to meet performance
standards, and above all, knowledge of science. We pay for training and
we expect students to learn the science needed to work with the public.

The science content we teach is directly related to our permanent,
temporary and traveling exhibitions, and school and public programs.
Interpreting our 170 permanent exhibitions demands a knowledge of
anatomy, psychology of perception, optics, wave theory, resonance,
atomic structures, physics of light and color, feedback systems, quantum
physics, audio-technology, microbiology and chemistry. Delivering
programs and interpreting traveling and temporary exhibits involves
knowledge of astronomy, information technology***,*** navigation, earth
science, manufacturing technology, architecture, and the biology of
AIDS. We also expect our youth staff to use instruments from scanning
electronic microscopes to a functioning steam engine. Students tell us
the skills they need to work at the New York Hall of Science test and
exercise what they learn in the university in new and more interesting
ways.

Career Ladder participants learn questioning techniques, how to adapt
demonstrations for kindergartners and high school seniors, how to
present the same content in different ways to meet different learning
styles, how to use manipulatives to illustrate principles, how to
attract and keep attention, and how to control large groups. Our
Preschool Science Place is a laboratory for learning about early
childhood development and how to work with very young children and their
caretakers. Unlike traditional undergraduate pre-service students who
get one or two student teaching placements, our Science Career Ladder
participants interact with students at every grade level from preschool
through graduate school, students with disabilities and learning
problems, students who are intellectually gifted, and everyone in
between. Many Career Ladder participants discover an affinity with a
particular grade level or learning style as a result of their
experiences at the Hall of Science. Our Career Ladder students get to
observe teachers handling groups on class trips and they form ideas
about teachers and teaching from a perspective unavailable in any other
setting.

Some schools of Education offer field credit for the experience, one or
two offer student teaching credit. Our experienced pre-service Career
Ladder students work in our teacher training programs, interacting with
classroom teachers as museum staff. But most importantly, our students
are rewarded for exploration, discovery inquiry, experimentation,
flexibility; the very attributes essential for good science teaching and
learning and, unhappily rarely modeled in high schools and colleges.

In sum, the Science Career Ladder and similar programs around the
country, are providing students with workplace skills and science,
technology and math know-how within informal settings that supplement
and enhance undergraduate education. The evidence is rapidly
accumulating and it indicates that work experiences explaining science
in an informal setting have a positive impact on career choice, science
knowledge, and a sense of self-confidence and self esteem.

The major obstacle science centers face is neither designing nor
implementing successful work-study programs for undergraduates. It is,
rather, convincing the formal education community that experiences
provided by science centers are legitimate and valuable ways to provide
young people with skills, motivation, and know-how in science,
mathematics and technology. At the moment, science center staff are
included in reform efforts as advisors, committee members, and
representatives of community-based organizations. But they are not taken
seriously as an integral part of undergraduate teacher preparation
despite the advantages I have just described.

These hearing are designed to help you assess what further needs and
opportunities we see in undergraduate education reform. I would strongly
recommend a National Science Foundation program that provides major
funding for a few model programs which, like our Science Career Ladder,
offer undergraduates carefully constructed, content rich experiences in
informal science centers. If requirements for successful proposals
included: 1) academic credit in the disciplines for participants; 2)
college level administrative commitment to such programs; and 3) a
planning team of college and science center staff; we could show the
formal education community ways they might work with the informal
science community to reach our mutual goals, which are the preparation
and employment of a scientifically literate, and science- and
technology-oriented workforce.

***Peggy Ruth Cole** is a senior staff member at the New York Hall of
Science responsible for the planning and implementation of new programs
and for the development department and its fundraising activities. She
is the founder of the Science Career Ladder, the signature, minority
access education program of the New York Hall of Science which is
replicated nationally. Before joining the Hall, Dr. Cole was special
project coordinator at the Chrysler Museum in NorfoL4 , Virginia and the
conceptual advisor to the Staten Island Children\'s Museum\'s
exhibition, \" Once Upon An Island.\" She was Principal of the Fieldston
Lower School in New York City from 1981-1983. Prior to this, for 12
years, Dr. Cole was a graduate faculty member of the Bank Street College
of Education in New York City. She serves on the editorial board of*
Curator*, the journal of the American Museum of Natural History. Dr.
Cole is nationally active as a reviewer, author, and speaker in the
field.*

Math, Science, and Technology Undergraduate Education in America: One
Small Business Perspective

Israel Joseph Galvan

*President, GHG Corporation\
Houston, TX*

Introduction

These are the personal views of one small business owner deeply involved
in current and emerging technologies. However, I wish to emphasize that
I claim no special expertise on education.

My name is Israel Joseph Galvan and I am President of GHG Corporation, a
computer systems engineering company based in Houston, Texas. We have
been in business since 1979 and employ about 200 people.

Criticism of the American educational system comes from a variety of
directions. It has increased in volume and intensity and, worse, it has
become a permanent issue in every political campaign. Whether the
fundamental role and purpose of a university education is in need of a
serious national debate is problematical. However, there appears to be
some confusion and lack of a unifying principle in our educational
systema lack of clarity.

Arguably, the last serious debate on the role of the university occurred
in the 1930\'s between the camps of Robert Maynard Hutchins, the
traditionalist, and John Dewey, the progressive. These two giants
proposed ideas that were fundamentally different, but their views were
clear and unambiguous. Hutchins believed that the university should be a
community of scholars, a place to train the intellect and to transmit a
Great Tradition of culture. Dewey saw education as an experience and a
process for coming to grips with and solving real world problems, a
bridge between school and society. Hutchins opposed professional
schools; Dewey embraced them. It is safe to say that a hybrid of
Dewey\'s ideas won.

As it should be in any democracy, the debate on the nature and function
of higher education continues, but the current issues are not as
fundamental. The debate is less on the role and more on quality,
efficiency, and economics. It is centered principally on delivery and
implementation issues. The core of contemporary criticisms, I believe,
validates our present educational system. Our educational system is
sound. It is only in need of minor adjustments. Yet there are some
aspects of our educational system that do require further clarification.

What is the role of our educational system in training our work force,
as opposed to educating our work force? What is the role of our
community college system? What is the relationship between our
university system and our community college system?

These questions are beyond my simple musings. Contrasted to the complex
problems visited on our educational system, I fear that this modest
presentation may be too narrow and parochial. My intent is to simply
share the personal observations and views of a small \"high-tech\" firm.

The Environment

The world is undergoing a transformation not seen since the Industrial
Revolution, but at an increasingly faster rate. The time from idea to
development to implementation has virtually disappeared. The forces
driving this change began in the early 1960's and were not anticipated.
Our finest research institutions and our finest minds thought that the
1960's were going to be the decade of nuclear energy. It was going to
satisfy our every need for energy. Energy was going to be so cheap it
would not have to be metered.

They were wrong, of course; it turned out to be the decade of the
computer, with IBM as its corporate manifestation. For most of the next
two decades, IBM became the corporate model for the world, emulated for
its technical and business innovation. Little did we know that the heart
of IBM\'s computer, the Central Processing Unit (CPU), consisting of a
dozen circuit boards, would be replaced by another CPU the size of a
dime. Driven by this new CPU, which is embedded in every facet of our
lives, including our psyche, our world and our perceptions of that world
changed, and continue to change before our very eyes. It has created,
and is continuing to create, new industries. It has changed, and
continues to change, existing industries. It has transformed, and is
continuing to transform, every sector of our economy. It has eliminated,
and is eliminating, jobs; created, and is creating, new jobs; and is
contributing significantly to our country\'s anxieties and sense of
uncertainty. Indeed, even IBM feels threatened. It is threatening the
classic American corporate model and creating new ones. It has empowered
the technician and the small business.

Much has been written and said about the economic importance and
significance of the small business community. Small businesses:

-   provided virtually all of the net new jobs from 1987 to 1992;

-   were 99.7 percent of all employers in 1992;

-   employed 53 percent of the private work force in 1992; and

-   are the source of considerable innovation.

    So says the U.S. Small Business Administration. And yet most of our
    national educational and economic policies do not reflect the
    uniqueness, diversity, and role of small business in the current
    transformation.

    These are fast changing times; these are exciting times; and these
    are perilous times. However, there appears to be one universal
    constant: \"Knowledge is Power." For the present, this verity could
    possibly be restated \"Mathematical, Scientific, and Technical
    Knowledge are Power." And it is the role of our educational system
    and this board to insure that all Americans are armed with this
    knowledge. Of course, nothing much is at stake, except the economic
    well being of this country.

    Small businesses, especially \"high-tech\" ones, live, operate,
    succeed, and fail in this environment. Most small businesses share
    common problems and concerns, but \"high-tech\" small businesses, I
    believe, have a number of unique concerns and problems intimately
    tied to undergraduate science, mathematics, and engineering
    education. Generally, the small business community has very little
    direct contact with the university community, except as a source of
    talent. In the university research and development world,
    cooperative agreements and sponsored research is the exclusive
    domain of the large corporation. The overhead cost of a typical
    university is daunting to any small business wishing to fund a
    research project.

    The Real World

    Recently, a colleague said that the three most important concerns of
    his business were: capital, capital, and capital. Hyperbole not
    withstanding, capital is always a problem, but equally important is
    good management and good people. This, of course, is true for any
    business, small or large. We both face a rapid and sustained force
    of technological change, and a diminishing life expectancy and
    relevancy of particular skills. It is challenging our abilities to
    maintain our skill base tuned to this change.

    Unlike the large, well financed corporation, the small \"high-tech\"
    business has minimal redundancy in its skill base and less resources
    to keep that skill base continuously trained. The loss of a key
    employee can be devastating. Thus the recruitment, retention, and
    continuing training of a quality work force, while always important,
    has become critical. In this highly competitive market place, we can
    not afford to lose employees, lose our sense of unity, or disrupt
    the \"team.\" We can not afford to treat employees as expendable
    commodities. We have to nurture them, train them, in short, value
    them. This is not without a significant financial burden and
    increasing pressure to make tough management decisions. Yet, we must
    remain competitive. We are still subject to all of the forces of a
    competitive market place: being raided by highly skilled
    professional recruitment agencies, market demands for increasingly
    narrow job descriptions, competitive compensation packages, and the
    bottom line.

    For example, it is not unusual for us to have to pull a highly
    skilled computer scientist from a revenue generating project in
    order to send him or her to an expensive short course on some very
    specific emerging and proprietary technology. An alternative is to
    establish an internal training program with our own courses and
    instructors, and we have. But this is very expensive. Another option
    is to lay off employees when their substantial skills are not
    considered adequate for more narrowly defined skills. To retain a
    competitive posture, we are forced to make decisions at odds with
    our sense of fairness, our sense of community with our employees,
    and the long-term health of the firm. However, survival is basic and
    immediate. Though necessary, it is no longer sufficient to have a
    solid education in a particular technical discipline., thus
    demonstrating a compelling need for the development, implementation,
    and delivery of continuing education programs that keep pace with
    the changing technical environment.

    This fact is equally true in non-degree technical fields. For
    example, the transition from mainframes to distributed workstations
    is abolishing the need for the traditional computer operator and
    other affiliated paraprofessionals. This same transition is creating
    new jobs such as local area network (LAN) administrators, managers,
    and other related jobs. However, instead of training the current
    work force, businesses are replacing them.

    We expect recent college graduates to arrive at the work place fully
    armed and ready to contribute to the bottom line. Perhaps our
    expectations are misplaced. Perhaps it is the role of the business
    community to provide the necessary training. Additionally, it
    appears that recent college graduates generally lack practical,
    immediate skills; skills demanded by current market forces; skills
    such as being able to work in teams, being familiar with current
    technologies, and a fundamental understanding of systems engineering
    are glaring examples. Unfamiliarity with working in teams and with
    current technologies and systems thinking may be a manifestation of
    dated curriculums. The currency of detailed technical skills is
    important, but graduates also lack fundamental skills. Skills with
    the written and spoken language are deteriorating. In fact, it has
    become a cliché that scientists and engineers can not write. This
    paper is probably a good example.

    In summary, the most glaring deficiencies in Math, Science, and
    Technology Undergraduate Education that I have observed are:

-   Lack of team skills in recent recruits;

-   Lack of communication skills, both oral and written;

-   Lack of a dynamic and current curriculum and continuing education
    program; and

-   Inability of small businesses to access the applied research and
    development capabilities of our educational system.

    Modest Recommendations

    The National Science Foundation should promote greater participation
    between the Small Business community and our educational
    institutions. More specifically, the Foundation should:

-   Encourage the development of curricula allowing undergraduates to
    work on more relevant projects; projects that require multiple
    disciplines and an integrated team effort. This will allow them to
    experience a project through its entire life cycle. This will
    encourage teamwork, systems thinking, and significant practice in
    the development of their oral and writing skills.

-   Encourage the continuous improvement and relevancy of the curricula.

-   Encourage the development of a better, more relevant, and more
    accessible continuing education program. This program should be made
    available to non-degree technical people, post graduates, and even
    post doctoral students. Considerations should be given for small
    businesses to consult in the development of such a continuing
    program. In addition, small businesses should be considered in its
    implementation and delivery.

-   Encourage an active partnership between small business and our
    university system. Undergraduates should be encouraged and allowed
    to work on small business sponsored projects and receive credit.

-   Encourage and sponsor more relevant and applied research projects at
    the undergraduate and masters level. The small business community
    should be allowed and encouraged to participate, share cost, and
    benefit from the research.

-   Encourage the small business community to sponsor application
    research without the burden of the educational institution\'s
    overhead. Perhaps, the Foundation could make overhead moneys
    available to the university community if they could match it with
    appropriate direct cost moneys.

-   Finally, the Foundation should encourage the small business
    community to work more closely with the Community College System in
    the development of curricula more efficacious to small business and
    to the paraprofessional.

***Israel J. Galvan** graduated Cum Laude from the University of
Houston\'s Cullen College of Engineering in 1973; receiving a Bachelor
of Science Degree in Electrical Engineering. He is a member of the
Electrical Engineering Honor Society; received a NASA Space Act Award
1991, received an Honor Award from the Central Intelligence Agency and
the Directorate of Science and Technology in May 1996. In 1973, he
joined IBM as a product development engineer. He worked on a variety of
applications research and product prototype projects. He assisted IBM in
establishing a manufacturing plant in Milan, Italy. He began his own
company, GHG Corporation, in 1979 as a systems integration company.*

Employers Views on Desired Capabilities of\
Undergraduates Entering the Workforce

Alfred L. Moyé

*University Relationships Manager*

Robert W. Ritchie

*Director, University Affairs*

*Hewlett-Packard Company\
Palo Alto, California*

Introduction

We applaud NSF for having these hearings and for focusing on
undergraduate science, math, engineering, and technology (SME&T)
education. Significant change in how we educate in these critical areas
is required, and NSF is uniquely positioned to take the leadership in
encouraging this change. NSF is the Federal agency looked to by the
Academy as having the general health of higher education as its focus,
without the encumbrance of specific, narrowly defined missions, such as
the Departments of Defense, Health, or Commerce.

The organization for this presentation is:

1.  The Environment and Why it Requires Change in Undergraduate SME&T
    Education

2.  The Characteristics Required of Graduates to Succeed in This
    Environment

3.  The Kind of Undergraduate Education for There to be Such Graduates

4.  Specific Programs to Move Towards These Kinds of Undergraduate
    Experiences

5.  The Environment and Why it Requires Change in Undergraduate SME&T
    Education

The Environment and Why it Requires Change in Undergraduate SME&T
Education

Our society is radically different from the one in which the current
educational system developed. Changes in our world have had a major
impact on our sense of what students need to learn and how the learning
should be delivered to them. Increasingly, we are recognizing that
mastering a body of knowledge is simply not enough. Students must
acquire lifetime skills, such as critical thinking, quantitative
reasoning, effective communications, along with such abilities as
finding needed information and interacting well with others. We want
graduates who have the capacity to learn not who know everything.

Some institutions are rethinking and redesigning their curricula to
reflect the shift from teaching content to enabling students to develop
life-long learning skills. Some faculty understands the implications of
the information explosion and has re-engineered their courses
accordingly.

An important, and not so subtle, change has been a shift from life-long
job security to a more realistic hope for career security. This shift is
not greatly appreciated because it places responsibility on employees to
manage their careers; and to continue learning in order to avoid
technological obsolescence.

Today\'s worker has to learn to survive and thrive in a world that is
globally interdependent. Workers must be able to operate and communicate
cross-functionally, cross-culturally and across geography.

Teamwork, group problem solving and collaboration characterize today's
high-performance workplace. Learning in industry shifts from the
classroom to peers and teams. Knowledge creation shifts from the
individual to the organization where problem solving and decision-making
skills are important components.

Characteristics Required of Graduates to Succeed in this Environment

This environment requires that all graduates have skills for continued
learning. Workers must assume responsibility for managing their careers,
for continuous upgrading of their skills and acquisition of new skills
in order to be prepared for emerging opportunities. Managing one\'s
career in this way is the way to guarantee career security\--to position
oneself to have a career for as long as one wants to work. The onus is
on the individual.

To prepare students to be productively employed for life in science,
math, engineering and technology, the undergraduate experience must
consist of a solid foundation in mathematics and science fundamentals.

They must have an understanding of the modern, rapidly changing,
high-performance workplace and knowledge of what is required to succeed
and thrive in such a workplace.

They must have the flexibility to expect and embrace change throughout
their careers, skills for and the commitment to working in teams,
cutting across vastly different areas of expertise whether it is
development, manufacturing, marketing, or whether it is with other
people from around the globe.

Workers must have an appreciation for the global interdependence of our
communities and have the ability to work across geographies. In
particular, students planning for careers in science, math, engineering,
and technology must be prepared to understand the business and social
implications of the activities in which they will be engaged.

The Kind of Undergraduate Education for There to be Such Graduates

To prepare graduates, who have the above skills, schools must
drastically change the undergraduate educational enterprise, especially
in SME&T education.

An expectation of all undergraduates in the next millennium must be
basic science and technology literacy, regardless of area of
concentration. Information technology will increasingly impact our lives
as citizens, voters, politicians, and parents. Without an understanding
of issues arising from science and technology, it will be difficult to
be a fully functioning member of a democracy dominated by technology and
public understanding of science and technology will remain very low.

K-12 teachers must have better preparation in SME&T. K-12 teachers who
are science, math, technology-illiterate, will not be able to encourage
students to consider careers where SME&T skills are needed.

A much higher percentage of our populace must understand science, math,
engineering, and technology to be successful in the workplace, and
hence, to lead financially secure lives. For this reason, ways must be
found to make SME&T education interesting and attractive for a much
broader cross-section of our populace.

To make education interesting and attractive requires a shift to
student-centered learning environments in which the faculty are guides,
in contrast to the current, largely mass production, teacher-centered
ones, in which the faculty are lecturers imparting information. Someone
said it requires a shift from the sage on stage to the guide at your
side.

It requires faculty who are knowledgeable of careers outside the
academic world so that they can make their courses more relevant and
advise students of career opportunities using SME&T knowledge in jobs
that are not closely aligned with the narrow interests of most faculty.

Acquisition and application of a better understanding of the learning
process and of different learning styles is required, especially those
which may be the prevalent styles of segments of our society which have
not traditionally been attracted into mathematical, scientific, and
technological careers. We know a lot about how people learn, but too few
faculty, especially in science and engineering, are using the results of
research on learning in their courses. The knowledge is there, but our
faculty does not use that knowledge.

Individualizing education will lead to much heavier reliance on current
and shortly forthcoming technology, enabling learning experiences to be
delivered on the learner\'s demand across space and time (just-in-time,
anytime, any place learning). Faculty must become proficient at using
the technology for education. They must have both the willingness and
commitment to use technology and also the support and time from the
university for professional development.

Undergraduate education programs must find and apply ways to impart
basic written, visual and oral communication skills and skills in
working in teams. In addition, students aiming for science, math,
engineering careers need to learn about business and societal issues to
be able to understand implications of decisions they make.

At a Hewlett-Packard sponsored conference in October, 1995, attended by
18 faculty involved in SME&T education, the topic of appropriate
characteristics of institutions of higher learning in the year 2000 was
discussed and debated. The conclusion of this group was that the college
or university of five years from now must be learner-centered and
include the following:

-   Anytime, anyplace access

-   Collaborative learning and problem solving

-   Support for life-long learning

-   Significantly improved learning outcomes

-   Equity, access and success across all populations; and

-   Economic viability and globally competitiveness

    They also concluded that this requires a very different kind of
    academic environment: one, which has agile and change-oriented
    institutions. One which applies and integrates appropriate
    technologies, exhibits cooperation and resource-sharing among
    government, industrial, and other educational institutions,
    disseminates and applies best practices from elsewhere, effectively
    evaluates the results of its activities, and has a faculty reward
    structure which encourages these activities.

    We absolutely agree with these conclusions.

    Specific Programs to Move us Towards These Kinds of Undergraduate
    Programs

    The discussion above indicates where the focus of Federal programs
    should be, and they should be designed to move as quickly as
    possible to the academic environment outlined above.

    To reinvent institutions to be more agile and change oriented, those
    individuals within the institution who are change agents should be
    supported and rewarded. For example, a program of summer grants and
    awards to faculty for significant curricular development,
    application of best practices into their behavior, etc., could be
    developed.

    To encourage use of technologies, infrastructure for
    learner-centered anytime, anyplace activities could be mounted.
    Faculty at the HP October 1995, workshop recommended leveraging
    funding by requiring matching funds from the institution. Faculty
    saw this as a way to get institutional commitment and begin to
    institutionalize successful programs. NSF might also require
    institutions to state how they intend to sustain funded programs
    through operating expenses or endowments for technology upgrades and
    enhancements.

    To enhance collaboration, NSF might fund faculty time to work with
    industry to transfer insights from the corporate world to the
    academy. Much of the breakthrough research today is coming out of
    industry, and faculty would benefit from the experience. Similarly,
    encourage faculty to work with K - 12 teachers with, for example,
    summer funding to increase the number of high school graduates ready
    to pursue the undergraduate experience.

    Faculty who attended the HP workshop ranked new models of
    collaboration \-- among institutions, commercial-industrial
    collaboration, as well as collaboration with government -- among
    their highest priorities. They also felt it was important to launch
    some grand experiments; especially grand experiments with
    multi-campus involvement. To disseminate best practices, NSF could
    fund centers for accumulating and distributing best practices much
    in the model of the Synthesis coalitions\' NEEDS project, which is
    funded by NSF.

    To develop and apply evaluation methods, NSF could fund research
    into evaluation of program design to enhance learning, especially
    learning in populations previously under-represented in SME&T
    programs.

    To encourage modified faculty reward structures, NSF might consider
    awarding summer grants to faculty and administrators to create
    alternative ways to recognize faculty involved in educational
    innovation. Finally, on this point, NSF has special leverage with
    its research programs and could require descriptions from
    institutions as to the steps the institutions are taking to modify
    their faculty reward structure as a condition of (or at least as a
    significant factor in) the award of a research grant.

    Conclusion

    Thank you for the opportunity to appear today. Our environment
    mandates changes in SME&T education if we are to produce graduates
    who can succeed in today\'s rapidly changing information society. We
    applaud your interests and believe your focus can have a unique and
    positive influence on American higher education.

    ***Alfred L. Moyé** became Hewlett-Packard\'s Director of University
    Affairs in May 1996. He leads HP\'s worldwide interaction with key
    universities, including the company\'s on­going programs in such
    areas as research, recruiting, marketing, training grants and public
    advocacy. In addition, because of the crucial role of universities
    in developing skills and technologies needed to improve national
    competitiveness, Dr. Moyé leads the formulation and communication of
    HP\'s views on U.S. policies for technology and higher education. He
    was previously Manager of Engineering Education Performance
    Technology at Hewlett-Packard. Prior to joining HP in April 1984,
    Dr. Moyé was national educational sales manager and director of the
    Atari Institute for Education Research He was also vice president
    for academic affairs at Roosevelt University, Chicago, and vice
    chancellor for student affairs and associate professor of chemistry
    at the University of Pittsburgh. During the Carter Administration he
    served as deputy assistant secretary for higher and continuing
    education He has published an organic chemistry laboratory text and
    has authored several articles in chemistry and chemical education
    journals. He is an officer of the Board of Trustees of West Virginia
    Wesleyan College and the National Technological University, and a
    member of the Board of Visitors, Faculty of Arts and Sciences,
    University of Pittsburgh. He was recently elected to the Board of
    Trustees of the University of Pittsburgh. Dr. Moyé received his B.S.
    degree from West Virginia Wesleyan College and his Ph.D. from the
    University of Pittsburgh, both in chemistry. He is also recipient of
    five honorary doctorates.*

    *From January 1990 until his retirement in 1996, **Robert W.
    Ritchie** served as the first Director of University Affairs at
    Hewlett-Packard Company. He led HP's worldwide interaction with
    higher education, including the company's on-going programs in such
    areas as research, recruiting, marketing, training, grants and
    public advocacy. Prior to becoming Director or University Affairs,
    Dr. Ritchie was Director of the Computer Systems Center in HP
    Laboratories from 1988 through 1989. From Science Laboratory of
    Xerox Corporation's Palo Alto Research Center. From 1962 to 1983, he
    was on the faculty of the University of Washington, first in
    mathematics and, subsequently, computer science. He formed the
    Department of Computer Science there in 1966, was its first chair,
    and then chaired the department again from 1977 to 1983, helping it
    achieve its current rating among the top ten in the country. While
    at the university, Dr. Ritchie also served as Associate Graduate
    Dean and Vice Provost of the University. From 1960 until joining the
    faculty at the University of Washington, he was John Wesley Young
    Research Instructor in Mathematics at Dartmouth College. He has
    served as the chair of the Committee on New Areas in Applied
    Mathematics of the Conference Board of Mathematical Sciences, chair
    of the Computing Research Board, and co-chair of NSF Workshop on
    Quality in Engineering Education.*

-   Enhancing Engineering and Manufacturing Education:

    Industry Needs, Industry Roles [^6]

    John H. McMasters

    *Senior Principal Engineer, Aerodynamics Engineering*

    *Boeing Commercial Airplane Group\
    Seattle, Washington*

    James D. Lang

    *Director of the Technology Division, New Aircraft and Missile
    Products*

    *McDonnell Douglas Aerospace\
    St. Louis, Missouri*

    The problem of how to provide quality-engineering education (as
    contrasted with training) in our universities, and the appropriate
    role of industry in this endeavor, has been a topic of controversy
    and debate for decades. Ever since the proliferation of
    research-oriented universities began in the early 1960s, a widening
    gulf has been developing between our colleges of engineering and the
    industries they support. It has become increasingly clear to those
    of us with an interest in industry-university interactions and
    recruiting that the curricula in most of the major universities in
    this country are now badly out of balance, with a too heavy emphasis
    on engineering science (analysis) and competition at the expense of
    design (creative synthesis), manufacturing and cooperative learning
    (teamwork). Despite an increasing number of recent refreshing signs
    of change, the current under-emphasis on the quality of
    undergraduate education is further cause for major concern with
    respect to the impact this may have on the overall competitiveness
    of our future industrial technical workforce. Of particular concern
    is the inadequate exposure to manufacturing issues given to most
    undergraduate engineering students. Design (engineering) and
    manufacturing are inextricably bound together in modern industrial
    practice, and this fact generally is not reflected in current
    engineering education programs in the majority of our research
    oriented universities. This situation poses a significant long-range
    problem for industry in this country unless sustained, cooperative
    actions are taken to restructure and rationalize the present system.
    Furthermore, industry needs a greatly increased level of continuing
    education and training, much of which could be provided by
    developing strong and continuous linkages with our universities and
    community colleges.

    *\"Poor \... education wastes human potential If schools are
    sound\...\[students\] will graduate with the knowledge, skills and
    motivation to help further build our nation, and with the foundation
    needed to help them enjoy full and productive lives.*

    *Frank Shrontz*

    *CEO, The Boeing Company*

    *"A scientist discovers that which exists. An engineer creates that
    which never was.\"*

    *Theodore von Kármán*

    *\"In time, the \... public and possibly even the \'educated class\'
    will come to appreciate that engineering is no more applied (and
    therefore second rate) science, than science is theoretical
    engineering."*

    *C. R. Chaplin*^1^

    *"The mind is not a receptacle; information is not education.
    Education is what remains after the information that has been taught
    has been forgotten. \"*

    *Robert M. Hutchins*^2^

    *(after Benjamin Franklin)*

    The Current Situation

    The state of education in this country, especially in science,
    engineering and technology, has become a matter of increasing
    concern to many of us. The recent National Research Council working
    paper (*Major Issues in Engineering Education*) of the Board on
    Engineering Education, as only one recent example, identifies many
    of the major issues that need to be addressed. While the problems we
    perceive encompass the whole system from pre-elementary through
    post-graduate education and training, the focus of the present paper
    is on needed changes in university undergraduate education programs.
    Our paper is sometimes strongly worded, but we believe that the
    assertions made can be adequately documented and defended. Extended
    discussions with a large number of our colleagues in industry,
    academia (including many students) and government show that while
    there are significant exceptions to what we have written, and
    continued denial that anything very basic is wrong with our current
    system of higher education, there is now an increasing degree of
    admission that the following observations have validity.

-   Too many of the faculty and administrators in our colleges of
    engineering remain complacent. The primary message from academia
    continues to be: Give us more money and we will do more (of the
    same) good things.

-   Improvements in curriculum, etc. are always needed, but nothing
    basic is broken in the educational system we now have. Witness the
    number of foreign students in our graduate programs as only one
    measure of the success of the system.

-   Research money continues to come (from largely government sources)
    to those willing to work hard enough to get it, which is good
    because so many departments are now so heavily dependent on it. The
    fact that too much of this research is of very limited value to our
    industry seems to be irrelevant.

-   From an aerospace industry vantage, as potential employers of the
    universities\' graduates, we see vast opportunities for improvement.
    We are increasingly "dissatisfied customers.\"

-   New hires must serve excessively long apprenticeships (three-to-five
    years) before they become fully productive (i.e., we must fill
    significant gaps in their education's as well as provide
    job-specific training).

-   We see too many new graduates with an inadequate grasp of what
    engineering (as contrasted with engineering science) is and how one
    practices it, particularly in the currently evolving industry
    environment. Too few of our engineering graduates seem to have any
    idea of how to work in teams or how to manufacture anything. Fewer
    still seem to understand the process of large-scale, complex system
    integration, which characterizes so much of what we do in our
    industry.

-   Based on limited data available to us, academic success (as measured
    by test scores and grade point averages) shows little discernible
    correlation with subsequent performance on-the-job (as measured by
    salary growth and perceived value of an employee to a company).
    Those students who are judged the \"best and brightest\" on the
    basis of grade point average are Frequently those who have worked
    hardest in a highly competitive academic environment of separate,
    specialized courses, and are often least prepared to work
    cooperatively in teams to engineer an integrated complex system
    which is economically and operationally viable.

-   Industry continues to be limited, and sometimes-underutilized
    partner in the engineering education process. Merely throwing money
    into the system is inadequate.

-   Industry recognizes its vested interest in education, but is largely
    preoccupied with staying in business while producing a proper return
    on the investment of its shareholders. Improving the national
    educational system is not in most corporate charters.

-   The modern international marketplace is highly competitive and
    volatile (particularly in the aerospace industry). Historically,
    industry has tended to be shortsighted in its planning, while
    longer-range forecasts have generally proved unreliable. The unknown
    consequences of the present era of world political and economic
    uncertainty only exacerbates the difficulty of making sustained,
    long-term commitments to \"lower priority\" supporting activities
    such as generic research and pre-employment education. At best,
    industry has too often been viewed as a \"fair weather friend\" by
    much of academe.

-   Discussing appropriate engineering education programs is made very
    difficult when a higher-level manager from a given company (with a
    background in one discipline, as practiced when he was an engineer
    years earlier) tells a dean of engineering one version of \"what
    industry needs\" while a working lead engineer from another
    discipline tells a department chairman quite another story. In the
    end, both stories get lost or the one closest to what the listener
    wants to hear gets heard. *Industry really would do all involved a
    great favor by developing and delivering a more coherent position to
    academia on this issue\--even if the message is complicated or a
    mere statement of principles.* \[A listing of the "Desired
    Attributes of an Engineering Graduate" is appended as on candidate
    for such an industry-wide message\].

    Much more than the above has been written ^1-13^ on the faults (and
    less frequently, the real virtues) of our educational system in
    general, and a large number of documents have been prepared
    recommending a wide range of reforms. The university system has
    proved remarkably resistant to many of these assaults, however, and
    little real change has been apparent until quite recently -- spurred
    in part by the massive dislocations caused by the end of the Cold
    War and the concomitant \"right-sizing\" (downsizing) in our
    industry ^l4^ which has been further fueled by increasing
    international competition. In defense of our universities it also
    must be pointed out that the increasing complexity of our
    technology, the past availability of large sums of government money,
    the changing economics of maintaining a high quality (largely
    graduate student focused) academic and research program, a lack of
    adequate industry attention, and the perceived weaknesses of K-12
    public education are in part responsible for the higher education
    system we have today -- which in the face of greatly increased
    international competition, is no longer good enough. While it is
    easy enough to point fingers or blame \"outside\" causes for our
    current situation, the problems we now face remain ours to deal
    with. Since we all (industry, government and academe) share some
    measure of culpability for what we have created, we should now be
    able to go beyond the point of deciding \"whose fault it is\" and
    work together in a spirit of enlightened self interest to create a
    new system which better meets the future needs of all of us and our
    nation as a whole. Further studies of the problems we face are not
    needed; what we do need are practical plans and processes for
    cooperative act ion. These plans should be based on rigorous use of
    systems engineering principles to define requirements for and
    develop the design of, the new approaches needed. Appropriate
    metrics must also be developed and applied.

    What We Need - Some Opportunities

    What is needed is a long-term, sustainable and fully cooperative
    effort based on systems engineering principals aimed at producing
    the sort of education (the most valued product of the university
    system from an industry perspective) which will develop future
    engineers with the skills and attributes listed in the Appendix.
    This program would be based on:

-   Recognition of the basic distinction between "engineering science\"
    (analysis) and "engineering\" (design/creative synthesis) in both
    the education and employment of our future engineers. "Design\" and
    \"engineering\" are fundamentally synonymous^1,6^ and thus we must
    accord design education the respect to which it is due.

-   Recognition that what you design is what you have to (attempt to)
    build and therefore that design and manufacturing are inextricably
    bound together -- both in professional practice and in university
    curricula.

-   Recognition that no 4-year (or 5- or l0-year) university program is
    ever going to be able to produce a fully and permanently qualified
    professional engineer, and thus that efforts to cram \"everything a
    student must know\" into this time frame is futile. A student must
    be prepared for life-long learning and major new partnership efforts
    are required to provide continuing education at all stages of an
    individual\'s career.

-   Recognition that our future prosperity depends on
    industry/university teamwork and cooperation at all levels,
    including student course work and in university faculty
    interactions.

-   Development of a modern approach to teaching design and
    manufacturing as a fully integrated, team focused, part of the
    university curriculum (including such exposure in core engineering
    science courses) beginning at the beginning of a student\'s freshman
    year. Intrinsic in this is the recognition of the profound
    importance of systems engineering and information technology (far
    more than mere \"computer literacy\") both in the curriculum and as
    a means of educational program delivery.

-   Providing our students a proper view of the context (the whole
    system) within which engineering is practiced. This context includes
    economics, history, manufacturing as well as environmental, legal
    and ethical aspects of our profession.

-   Providing an appropriate hiring and reward system for engineering
    (design) and manufacturing educators within a given university
    structure, and which at least provides a faculty with sufficient
    experience and dedication to teach these topics well.

-   Recognition of the distinction between education (a primary
    university function) and training, much of which must be viewed as
    an industry responsibility. We must understand the appropriate (and
    perhaps new) relations between our research universities and our
    community colleges in providing both initial and continuing
    educational and training services. The differing requirements and
    available resources of large corporations and smaller companies for
    training must also be recognized and dealt with.

-   Proactive leadership and involvement from all relevant professional
    societies (e.g., AIAA, ASME, ASEE, SAE, etc.) in building processes
    to link industry, academia and government participants. Their
    traditional role should be expanded and integrated across societies
    in recognition of their members\' current needs. Their role, along
    with ABET, in formalizing and enforcing educational standards
    provides strong leverage to develop and implement engineering
    curricula changes and to achieve results on a national level.

    Roadblocks

    It is easy enough to make lists of what we in industry need, but it
    must be clearly recognized that there are some serious stumbling
    blocks to be overcome in accomplishing the changes previously
    advocated. Some of the most obvious are:

-   While many in both industry and academe now recognize the need to
    change, there is a serious discrepancy between our two cultures with
    respect to the sense of urgency we each feel regarding the rate at
    which changes must occur.

-   Too many university administrators and faculty members have little
    or no knowledge of what industry does, how it does it, and what
    their graduates need to know to function effectively in an
    industrial environment.

-   Too few in industry have an adequate understanding of how our
    current universities function, how the faculty reward system works,
    or the constraints under which faculty members must operate in the
    system we have.

-   Design education generates little research revenue and leads to few
    publications in \"the right journals.\" Manufacturing has been held
    in even lower esteem until quite recently since, in academic terms,
    it is perceived to lack "intellectual content.\"

    What We Can Do - Recommendations for Action

    Engineering education has become "big business" and there is much
    inertia and resistance to the sort of changes advocated here, which
    also echo recommendations in several recent national studies (i.e.,
    those prepared under the auspices of the ASEE^15^ and the NRC). Many
    believe that no further studies are needed. We need action. These
    actions include:

-   Establishment of a process to develop an industry-wide position on:
    "What we (industry) want from our colleges of engineering and what
    we are prepared to do to help provide it."

-   A much better focused use of our "aid to education" resources (both
    money and people). This includes a balance between government-funded
    research and educational programs.

-   A change in the university reward system that recognizes that
    quality teaching must be at least on parity with funded research
    with regard to faculty promotion and tenure.

-   Changes in the degree structure to establish engineering as a true
    professional program and which are consistent with future industrial
    technical workforce needs.

-   Greatly expanded faculty - industry exchange programs. The objective
    of such programs should be to provide faculty with first-hand
    knowledge of industry practice and issues which would allow them to
    devise curricula which are truly responsive to industry needs rather
    than to merely allow faculty to continue their usual research at
    another (industry) site. Models for such exchanges exist in NSF
    Grant Opportunities for Academic Liaison with Industry (GOALI) and
    the Boeing-A.D. Welliver Faculty Fellowship programs.

-   No culture change of the magnitude we believe to be required can be
    accomplished in a single meeting or by conducting yet another study.
    Likewise, no single party, working in isolation, can resolve the
    diverse problems we confront What we must do is establish and
    support (at both national and local levels) processes which
    encourage individuals from industry, academe and government to work
    together on a sustained basis to address the practical aspects of
    how we are going to change. A national forum to include relevant
    professional societies, perhaps in the leadership role, should be
    sought to address these issues and to create the continuing process
    for discussion and resolution. Agreements on curricula options
    should be sought on a periodic basis. ABET accreditation criteria
    and the ABET accreditation process can be used as a mechanism for
    now to formalize and enforce the agreements.

Concluding Comments

The central message of this paper has been that the world is changing
dramatically^l4^ in the wake of the Cold War, and just as broad sectors
of industry are changing to meet new challenges and opportunities, our
universities (particularly in technical education) must change as well.
The needed changes we can foresee will not come as swiftly as some would
wish, nor will they come easily. But change must come if we are to
sustain our prosperity in an increasing complex and highly competitive
international environment. Of the changes advocated in this paper,
perhaps the most profound and sweeping must be the rethinking and
implementation of appropriate partnerships between industry and academe
in the education and training of engineers. In our view, it is no longer
tenable (if indeed it ever was) to accept a practice in which a
university \"educates\" a student to some level of competence, and thus
having done its job and \"completed\" its task, turns this \"finished
product\" over to an employer to do with as may be fitting. From now on
we must truly prepare a student to become a \"life long learner\" - an
individual who will require and actively seek continuing education as
well as training opportunities for the entirety of his or her career. In
this process, industry and academe (appropriately aided by government)
must become active partners, each doing what it is best qualified to do
in fully cooperative efforts from the beginning of the educational
process. We all have much to gain if these new partnerships can be
established, and much to lose if they are not.

*Dr. John H. McMasters is an aerodynamicist and airplane design
instructor in the Boeing Commercial Airplane Group and serves as an
Affiliate Professor in the Department of Aeronautics and Astronautics at
the University of Washington in Seattle. He has B.S. and M.S. degrees
from the University of Colorado at Boulder and a Ph.D. from Purdue
University; all in aeronautical engineering. Before joining Boeing in
1976, he served as a Project Officer for air-to-air missile systems at
the Air Force Weapons Laboratory and taught at Arizona State University
and Purdue. He is an Associate Fellow of the AIAA and was an ALAA
Distinguished Lecturer in 1992-94. He is also a member of the Board on
Engineering Education of the ASME and the Boeing University Relations
Coordinating Committee.*

*Dr. James D. Lang is the Director of the Technology Division, New
Aircraft and Missile Products, at McDonnell Douglas Aerospace. He is
responsible for technology development and transition. Dr. Lang is an
AIAA Fellow, a member of ASEE, and a member of the USAF Scientific
Advisory Board. He retired from the U.S. Air Force as the Aeronautical
Systems Division Deputy Commander for Engineering. He also led the
Avionic Laboratory and the Flight Controls Division. He has over 3000
flight hours including 320 combat missions as a Forward Air Controller.
He has a B.S. from the U.S. Military Academy, a M.S. from Stanford
University, and a Ph.D. from the Cranfield Institute of Technology,
England.*

Acknowledgments

This paper is an outgrowth of a series of position papers written for
various audiences over the past several years, the most recent version
having been prepared as part of a study of aeronautics materials and
manufacturing technologies needs conducted in 1994 under the auspices of
the National Center for Advanced Technologies (NCAT) in Washington, D.C.
The authors believe that the issues raised in this paper transcend the
more narrowly focused concerns of the NCAT study, however. We are
greatly indebted to a large number of individuals for providing
thoughtful comments and encouragement in the preparation of this paper
and its predecessors. While too numerous to list comprehensively, a few
key individuals must be cited for their special contributions: Richard
Hartke (NCAT), Drs. Joe Bordogna and Elbert Marsh (NSF), Stephen G.
Moran (NASA), Earll Murman (MIT), Lee Nicolai (Lockheed), and Daniel P.
Schrage (Georgia Tech). In addition, P. M. Condit, A. R Mulally, R. L.
Bengelink, R A. Davis, A.M. Erisman, J. F. McGuire, T. Tobey, B. J.
White, and M. H. Wiese of The Boeing Company provided major input and
inspiration for earlier versions of this paper. The authors remain
solely responsible for all opinions expressed, however, and these do not
\[yet\] necessarily represent the positions of either The Boeing Company
or McDonnell Douglas Aerospace.

References

1.  Chaplin, C.R., \"Creativity in Engineering Design -The Educational
    Function,\" The \[UK\] Fellowship of Engineering, 2 Little Smith
    Street, Westminster, London SWIP 3DL, November, 1989.

2.  Hutchins, R.M., The Learning Society, NY: New American Library,
    1969.

3.  Grinter, L.E. (Chair), *Report on Evaluation of Engineering
    Education,* American Society for Engineering Education, Washington,
    DC, 1955.

4.  Evans, D.L., McNeill, B.W., and Beakley, G.C., "Design in
    Engineering Education: Past Views of Future Directions."
    *Engineering Education* 80 (5): 517-522. July/August, 1990.

5.  Ferguson, E.S., *Engineering and the Mind's Eye*. MIT Press,
    Cambridge, 1992.

6.  McMasters, J.H. and Ford, S.D., "The \[Airplane\] Design Professor
    as Sheepherder (An Industry Role in Enhancing Engineering
    Education),\" AIAA 90-3259 (Invited), ALAA/AHS/ASEE Aircraft Design
    and Operations Meeting, Dayton, OH, September 17-19. 1990.

7.  McMasters, J.H. and Ford, S.D., "An Industry View of Enhancing
    Design Education,\" *Engineering Education*, 80 (5): 526-529.
    July/August, 1990.

8.  McMasters, J.H., \"Paradigms Lost, Paradigms Regained: Paradigm
    Shifts in Engineering Education,\" SAE (Invited) Paper 91-1179,
    April, 1991.

9.  McMasters, J.H., Holt, S.C. and Derwin, W.J., \"Adventures in
    Engineering Education at Boeing (The BCAG Summer Intern Program),\"
    ALAA Paper 92-1093, February 1992.

10. Nicolai, L.M., \"Designing a Better Engineer,\" *Aerospace America*,
    Vol. 30, No. 4, April 1992, pp. 30-33.

11. Hoover, C.W. and Jones, J.B. (Co-Chairs), Improving Engineering
    Design, National Research Council. Washington. D.C., 1991.

12. Dertouzos, M.L., Lester, R.K and Solow, R.M., *Made In America*, M1T
    Press, Cambridge, 1989.

13. Covert, E.E., \"Engineering Education in the 1990s: Back to
    Basics,\" *Aerospace* *America*, 30 (4): 20-23. April 1992

14. Ellis, R.A., \"At the Crossroads (Crisis and Opportunity for
    American Engineers in the

15. 1990\'s),\" Engineering Workforce Bulletin No. 130-131, Engineering
    Workforce Commission of the American Association of Engineering
    Societies, January 1994.

16. Engineering Education for a Changing World, Report of the
    Engineering Dean's Council and Corporate Roundtable of the American
    Society for Engineering Education, October, 1994.

Appendix: DESIRED ATTRIBUTES OF AN ENGINEERING GRADUATE

-   A good grasp of engineering science fundamentals.

-   Mathematics (including statistics)

-   Physical and life sciences

-   Information technology

-   A good understanding of the design and manufacturing process (i.e.,
    understand engineering).

-   A basic understanding of the context in which engineering is
    practiced.

-   Economics

-   History

-   The environment

-   Customer and societal needs

-   Possesses a multi-disciplinary, system perspective.

-   Good communication skills.

-   Written

-   Verbal

-   Graphic

-   Listening

-   High ethical standards.

-   An ability to think both critically and creatively - independently
    and cooperatively.

-   Flexibility - an ability and the self-confidence to adapt to
    rapid/major change.

-   Curiosity and a desire to learn - for life.

-   A profound understanding of the importance of team work

Note: This is a list of basic, durable *attributes* into which can be
mapped specific *skills* reflecting the diversity of the overall
engineering environment in which we in professional practice operate. In
specifying desired attributes (i.e. desired outcomes of *the educational
process*), we avoid specifying how a given university goes about meeting
industry needs. Curriculum development is viewed as a university task to
be done in cooperation with their \"customers,\" and in recognition of
their own local resources and constraints. Industry, as an important
customer, must be an active partner in this process.

Testimony on the Views of Employers on Undergraduate Education in
Science, Mathematics, Engineering, and Technology[^7]

Robert T. Jones

*President and Chief Executive Officer, National Alliance of Business\
Washington, District of Columbia*

Permit me to make three statements that generally characterize business
community positions regarding higher education.

First, higher education should demonstrate greater understanding of the
needs of the private sector. Too often higher education seems unaware of
job requirements and changes in the workplace. I believe that
maintaining a close connection between what goes on in our educational
system and the rapidly changing work environment is essential.

Second, higher education curricula should reflect the changing
conditions in the workplace. More students need to experience the
challenges and rigor of contextualized learning.

Third, higher education should help develop the work ethic that will
sustain students in their eventual careers. Note that many corporate
leaders say that new employees are fine − even more responsive than they
used to be. These same leaders point out that they screen a substantial
number of potential applicants, selecting only the best from that pool.
As long as that pool is large enough and active enough and vital enough,
firms will always manage to hire the kind of people they need. We must
however be concerned about all the rest, those who do not enjoy the
skills needed for employment and advancement.

Corporate leaders tend to believe that the factual knowledge derived
from undergraduate education can quickly become obsolete. Further, the
life expectancy of new entrants in their first job is very short.
Relatively inexperienced employees move on to another company or another
function or task more rapidly than ever. Companies are also increasingly
reluctant to hire recent college graduates. Before they invest in hiring
on a long-term basis, they want to see degree holders with three to six
years of work experience in the modern corporate system who can show an
effective adaptation to the updated demands of the new workplace.

The typical employer no longer defines educational preparation solely on
the basis of scientific knowledge or experience, or mathematical and
technical competencies. Increasingly, employers are looking for the
ability of new scientists to think clearly and communicate their
thoughts, to write clearly, to negotiate, to listen, to work on teams,
and to function effectively as part of a diverse work force. Employers'
perceptions are that schools are not paying enough attention to the
development of these skills. They do not seem to be taking business very
seriously, nor listening to our cues that how we hire is now different.

I urge you to look at the hiring processes that companies are utilizing,
to study carefully the methods of testing and interviewing. What are the
broad and durable skills and competencies they seek? It is very clear
that when applicants don\'t understand the needs of employers, they are
not likely to be hired.

New employees are increasingly expected to demonstrate as one CEO calls
it "boundary-less" skills. This all happens because of the changing
nature of the way we work. Engineers and scientists are now "on the
factory floor." They are not isolated in a lab. Engineers and scientists
now deal with customers and their ability to deal with customers is key
to their being hired in the first place.

There is a growing perception that a college degree per se is of minimal
value to business. Business is more concerned about the level of a job
applicant\'s competencies, the breadth of that applicant\'s
competencies, and the applicant\'s ability to assimilate, manage, and
utilize information. How those skills are acquired \-- whether during
studies for an MBA or a MS in science or math, or as part of some other
educational experience \-- is not of great import.

Related to this focus on competencies rather than credentials is the
increasing demand for certification programs. The demand is growing for
continuing education programs that let employees re-enter the education
community in order to update their skills, to stay current on new
technology and procedures and to do so in short certifiable bites that
employers can recognize. Simply taking people out of employment for two
or three years for an advanced degree for the sake of improving skills
and knowledge doesn\'t work effectively for many jobs. The way we
traditionally boxed up these degrees doesn\'t match well with the needs
of today's work force. We will probably see within the next 24 months a
very rapidly growing discussion about how we create certification
programs where people can engage frequently and efficiently in
continuing education.

In summary, business believes more strongly than ever that
post-secondary and continuing education are crucial for all potential
workers. How to sustain the strengths of our existing system while
responding to the vibrancy of the business environment is a challenge
that higher education must meet.

*A nationally recognized expert in the workforce development field,
**Roberts Jones** has over thirty years of organizational management and
public policy experience in the training and education field. As a
lifelong advocate for investing in people, Mr. Jones is widely credited
with helping place workforce development at the forefront of the
nation's public policy agenda. Prior to joining the Alliance, Mr. Jones
was vice president of RJR Nabisco, Inc. Mr. Jones served both Presidents
Reagan and Bush as Assistant Secretary of Labor, responsible for federal
workforce development and training policy addressing the significant
changes in work and the workplace. Jones played a key role in the Hudson
Institute's landmark research project and report,* Workforce 2000: Work
and Workers for the 21^st^ Century*, and was responsible for the
Department of Labor's SCANS Commission, which for the first time spelled
out the skills necessary for success in the workplace. Mr. Jones was
awarded the Presidential Distinguished Executive Award, the highest
honor in the Federal government, in 1986. Mr. Jones received his B.S. in
Psychology from the University of Redlands in Redlands, California, and
performed graduate work in Public Administration at the American
University in Washington, DC. He also served for four years in the
United States Air Force.*

Testimony to the National Science Foundation

Undergraduate Review Subcommittee:

Employers Views on Desired Capabilities of Undergraduates\
Entering the Workforce

John L. Sisler

*Manager of Exploration and Production Training\
Shell Exploration and Production Company\
Houston, Texas*

Thank you for the opportunity of share views on undergraduate science,
mathematics, engineering, and technology (SME&T) education from the
perspective of an Energy Industry employer of graduates from these
programs. As manager of E&P Training in Shell Exploration and Production
Company, I am increasingly aware of the challenges and demands placed on
BS and MS SME&T graduates in out technical professional workforce. How
well our Company, Industry, and Nation surmounts this challenge is of
critical importance to future success and survival in a global market
place.

While this testimony is from the viewpoint of a large, primarily
domestic oil and gas exploration company, many of the phenomena
discussed appear to extend beyond the upstream portion of Energy
Companies. They are reportedly commonplace throughout other portions of
US Industry.

One characteristic of the upstream Energy Industry's technical
professional workforce is the mixture of scientists and engineers.
Geosciences (Geology and Geophysics) together with Physicists and
Computer Scientists are combined with Electrical, Geological, and
Petroleum Engineers to form the skill pool responsible for finding and
delineating new hydrocarbon deposits. Engineering graduates become
increasingly important in developing, producing, and managing existing
oil and gas reservoirs through economic lives to eventual abandonment.
These include Geological, Petroleum, Chemical, Electrical, Mechanical,
and Civil Engineering graduates. They are integrated with the skill
groups used to find and delineate who are also involved in the later
phases of the E&P cycle. This workforce is predominately BS and MS level
with a lesser number of Ph.D.s who typically transferred from Research
into the operating organizations. The way work gets done by this mix of
SME&T graduates across the spectrum of E&P activities has changed
significantly in the last five years.

Organizational Characteristics

Changes in the job environment associated with organization redesign are
profound. An ongoing process attempts to create work group structures
that are focused, efficient, and responsive to change. Nearly all
upstream US Energy companies, including Shell E&P Company, have
experienced dramatic organizational changes. In most cases these
resulted in somewhat smaller SME&T workforces and significantly fewer
supervisors and managers. Work now being done often exceeds levels prior
to the changes. Further change is very unlikely but a return to the type
of work group structure characteristic of the past is very unlikely.

Integration

Currently, most U.S. Energy companies have emphasized integration at
every organizational level. Formerly organizations were divided into
separate groups based largely on skills and type of work performed.
Exploration was typically separate from Production, and within each
there were subdivisions such as Geology and Geophysics in Exploration,
and similar splitting within Production into Operations and Engineering
that was in turn split into various engineering scientific specialty
groups. Today, multidisciplinary teams that are formed to accomplish a
specific project or manage a group of assets within a defined geographic
area do work. These teams are interlocked to form multi-disciplined
project groups responsible for major business segments.

Today's integrated work team organization has far fewer managers and
supervisors. Typically, supervisors lead large, diverse work teams and
do not exercise the level of detailed control characteristic of past
more hierarchical work structures. Someone will probably lead the SME&T
graduate with a dissimilar educational specialty.

Leadership Style

The style characteristic of successful leaders today is one of coach and
coordinator rather than command and control that was possible when past
supervisors frequently were experts in subordinates' jobs. Some work was
associated with personnel administration and the individuals now do
development formerly done by managers and supervisors themselves. The
SME&T graduate will probably find the leader not as deeply involved in
how work is done, provided it is timely and correct. Today's leader is
far more interested in results than style. Because of the size and
diversity of typical work teams the leader today is rarely as involved
in each individual's personal skill development as were past
supervisors. Individuals are expected to be proactive in defining their
needs and seeking our opportunities to grow. The leader's role is to
enable and encourage, but not to be prescriptive.

Performance Measures

Performance measures for individuals in today's organizations attempt to
focus on business-relevant results. Compensation is more directly tied
to current business results. To a growing extent individual results are
tied directly with Team results. In contrast, past performance measures
too often dealt with style and activity level. These may have had only
an indirect linkage with business objectives. The SME&T graduate is
finding that it is increasingly difficult to prosper personally while
the ship of which he or she is a crewmember sinks. Coming from an
undergraduate system that exclusively recognizes individual achievements
this change may cause some to question "fairness."

Skills

Skills needed to successfully function in today's work team environment
are somewhat different from the past. Each person needs an awareness of
the role and contributions of others on the team. In-depth specialized
knowledge must be accompanied by skill at relating various parts to the
whole and finding solutions to problems that formerly might have "fallen
between the cracks" of the different specialties and departments. Skills
such as communications and teamwork are essential. Unfortunately these
are often given low priority during the SME&T professional's
undergraduate education. Working together interdependently in a team to
achieve a common goal rather than competitively to garner individual
recognition is a very satisfying change for many, but one that does not
come easily to some SME&T graduates who have thrived in their former
arena.

Personal Characteristics

Along with the organizational changes there have been changes in
expected personal characteristics of the Technical Professional
workforce. Skill areas deemed important for success in the current and
future work environments include many areas besides the traditional
SME&T subjects.

Technical

Sold preparation in the relevant sciences and mathematics together with
their application to create beneficial products remains the most
valuable asset that a SME&T graduate can bring into our workforce.
Without this, chances of success are slim. It is particularly important
to gain a broad foundation during the undergraduate period. This will
provide the means to adapt and thrive as changes occur that impact the
various applications used in the workplace.

One characteristic often missing in graduates of US SME&T programs is
ability to understand system versus segments. Too frequently graduates
who have impressive transcripts from highly acclaimed universities
exhibit limited understanding of how their segmented knowledge fits
together. An ability to understand context and to fit pieces into a
larger framework are essential in working in an integrated,
multi-disciplined environment. Critical thinking has greater importance
than problem solving.

Commercial

SME&T graduates are expected to possess more commercial acumen than in
the past. The necessity for improved financial performance has caused
accountability for profit and loss to be at much lower levels,
frequently at the work team level. A vocabulary once restricted to
Business School graduates is now routinely used by technical staff.
Engineers and scientists are no longer as insulated from accounting,
financing, and marketing.

Risk management is particularly important. SME&T graduates typically are
expected to define uncertainty, develop risk assessment, and play a
major role in managing risk.

While SME&T graduates are not expected to possess abilities to perform
specialized commercial tasks, they do need to understand their
importance and how they must be integrated with technical work to
achieve business success. Clearly SME&T graduates need to respect the
importance of the commercial context of their work.

Information Technology

Without question, advances in computing and information technology (IT)
are enablers for many of the changes discussed here today. All SME&T
graduates are expected to have skills that allow application of advances
in IT to their work. In fact, many technical professionals in E&P spend
the bulk of the time at work at workstations or PCs. Most are not
developing new software, but are using increasingly integrated
applications systems provided by others. Major concerns are with data
management and getting work done in complex internal IT systems. Many
find their undergraduate programs did not provide sufficient expertise
in the personal productivity tools.

This is one area in which recent SME&T graduates as well as technicians
have exerted leadership. Without their training that embraces IT as just
another of the natural work tools, progress would have been much slower.

Teamwork

All SME&T graduates must possess teamwork skills. While many factors go
into developing these, one that is partly a consequence of the
undergraduate educational system stands above the rest. Respect for
others who have different educational backgrounds is essential.

Academic divisions between sciences and engineering and further
subdivisions within each too frequently imprint SME&T graduates with
personal biases. These are obstacles to effective teamwork. Rivalry
between academic schools, departments, and even specialties within a
particular field, create stereotyping of various groups. Academic role
models that participate in such activities are sending very wrong
messages to their students. In today's business environment, one's
leader and office mates may well be from one of the groups held in low
regard by his or her academic role models. The need to respect others
extends beyond the SME&T disciplines to include those in business,
humanities, and other groups such as law, which must work together in
teams.

Communication

Good communication skills are even more important today. In addition to
the long-standing expectation that SME&T graduates be able to write and
speak effectively, the challenge today is the audience. Former audiences
were often composed of specialists who all spoke and understood their
specialties' jargon. Today an important audience is the
multi-disciplined work group and its leadership. Increasingly the SME&T
graduate is also communicating with external customers and stakeholders
who generally do not understand the "insider" language of science and
technology. This becomes particularly important to those working with
environmental issues.

Electronic tools are playing a bigger role as a means of communication.
However, the skill needed is to understand the cycle of listening,
understanding, creation and delivery of a clear message to a specific
audience. These are independent of the tools and should be incorporated
into SME&T undergraduate programs.

Leadership

More leadership skills are expected from everyone in flat organizations.
Fewer opportunities exist for formal management jobs, but there are many
more opportunities for individuals to hold ad hoc leadership roles. As
mid-level managerial and coordinational jobs have disappeared,
individuals in teams are doing some of the work. The leadership roles
and skills are very closely associated with teamwork rather than
exercise of the power of an organizational position.

Change Management

The final personal skill in this list is the ability to proactively
participate in the processes used to continuously improve. Change
Management is now a part of every SME&T graduate's responsibilities.
Each must acknowledge that the future is unpredictable. Major changes
and retaining flexibility to alter strategies will probably be needed to
survive both as business and individuals.

Individuals who accept personal responsibility for their own deployment
and take actions to grow personally and professionally will be more
likely to succeed. The employer of SME&T graduates is no longer
providing the level of security and development direction seen in the
past.

Recommendations

1.  Support programs, which are interdisciplinary and integrated. Do not
    restrict this to just the SME&T fields. Opportunities for
    undergraduate SME&T students to become involved must be provided.

2.  Support programs that instill rigor and breadth in BS/MS SME&T
    studies. Focus on teaching and shaping those who constitute the bulk
    of our workforce. Meaningful recognition must be available to
    faculty and institutions that do this well.

3.  Support programs that provide BS/MS students with the
    interpersonal/organizational/ teamwork skills now essential in
    today's workplace.

***John Sisler** is Manager of Exploration and Production Training for
Shell Oil Company in Houston, Texas. He has 36 years of varied Shell
experience as a geoscientist, engineer, and manger. During his career he
has worked on domestic exploration and production projects in Alaska,
California, New Mexico, Texas, Louisiana, the Gulf of Mexico, and
outside the United States in The Netherlands and Iran. Prior to joining
Shell he received a BA in geology from Rice University and a MS in
geology from Stanford University. He is a member of the American
Association of Petroleum Geologists, the Society of Petroleum Engineers,
the Geological Association of America, and the Houston Geological
Society.*

Testimony on the Views of Employers on Undergraduate Education\
in Science, Mathematics, Engineering, and Technology[^8]

Patrick E. White

*Vice President, Strategy, Bell Atlantic Corporation\
Arlington, VA*

I certainly appreciate the opportunity to be here to talk to you about
undergraduate needs, particularly with regards to the telecommunications
industry.

As the telephone industry evolves, one of the first things I think we
need more of is engineering programs designed to educate generalists, to
produce graduates with almost the same kinds of skills that John Sisler
of Shell Exploration and Production Company was just describing.

Very frequently, the kinds of issues that at least our undergraduate
engineers face require participation in multidisciplinary teams, but
more than that, they are often not focused on one specific area of
technology. They cover a variety of different areas. I think that a
program in undergraduate education designed to produce engineering
"generalists" would perhaps better match that need.

Our situation is analogous to the issue of the best strategy for
educating medical doctors. A general practitioner course of study often
meets the greatest need of communities for medical services. General
practitioners can be coupled with the selective use of specialists. In
the past there was a tendency to stress the education of medical
specialists. Now we realize the importance of general practitioners.

We clearly do have that need for specialists in telecommunications, but
for the vast majority of jobs, a general program matches our needs much
better than specialized programs. Where we do need specialists, a
bachelor\'s degree or even a master\'s degree is often not enough to
cover the field in depth. We really need to hire Ph.D. level scientists
or engineers to get the best contributions in specialized areas.

Another aspect that I wanted to bring out is that telephone technology,
like most high‑tech, electronics‑type fields, is undergoing pretty much
a revolution. For example, just look at one aspect of this field over
the last ten years, fiber optic transmission. Ten years ago, engineers
were talking about multi‑mode fiber with LEDs. That has shifted very
quickly through a number of different technology stages. Today, ten
years later, nobody ever talks about multi‑mode fiber, and nobody talks
about LEDs. Instead, the talk is about lasers, fiber amplifiers, and
dispersion‑shifted fibers. In other words, the field is moving so fast
that it is pretty hard for me to imagine how the universities can really
keep up with current technology.

In fact, if you look at most programs in engineering, you typically see
that they are pretty much covering the same old topics, and that is
surprising, given the pace at which the technology seems to be moving
along.

I think that there is certainly several impediments or several areas
within teaching that need to be dealt with. Tenure policies, I think,
need to be looked at to make sure they are really encouraging faculty to
keep abreast of technical areas, and that universities continue to hire
the best‑equipped faculty or the faculty with the most knowledge in some
of these new technology fields.

I also think that the types of research that are being funded also have
an impact. If you pick up almost any journal these days, I would say at
least two‑thirds, maybe three‑quarters, and in some cases even 100
percent of the selected papers seem to represent minor tweaks in
insignificant technical areas, most of which isn\'t all that relevant to
what industry really needs in terms of research.

If you have faculty who are moving in that direction, then you can
easily see why there will be a widening gap between what the
universities are actually teaching young engineers and what industry
needs if we are to remain at the cutting edge.

The last thing I wanted to talk about in my prepared remarks -- related
to the plea I made at the outset for more a general education in
engineering -- is that I believe we should reexamine the lack of a
foreign language requirement in most colleges and universities.
Increasingly there is globalization of just about every aspect of
industry. Too often we go to meetings with the French, Germans, and
Japanese and find that they can speak English plus their own language.
In negotiation, I think we are at a significant disadvantage in only
knowing English.

I also would put in a plea that the types of elective options that are
associated with engineering, currently drawing from a traditional
liberal arts menu, might also sensibly include wider options, e.g.,
business‑oriented courses, finance, economics, intellectual property
law. These are all very vital aspects for a young engineer's education.

***Patrick White** is Vice President of Research and Development at Bell
Atlantic Corporation. Prior to his current assignment, he was Vice
President of telecommunications strategies and before that he held a
number of positions with Bell Communications Research, Inc.
(\"Bellcore\"), including Assistant Vice President of the Network
Architecture and Analysis Research Center and Assistant Vice President,
New Architecture and Service Concepts Planning. Prior to joining
Bellcore in January 1984, Dr. White held various software development
supervisory positions with Bell Laboratories, commencing in 1973. Dr.
White holds a Ph.D. degree in Electrical Engineering/Computer Science
from Northwestern University. He has authored numerous technical papers
and edited technical journals. Dr. White is a member of the Eta Kappa Nu
and Tau Beta Pi national engineering honor societies as well as a member
of the IEEE and the Association for Computer Machinery.*

[^1]: Eugene Galanter, *Contemporary Psychophysics,* in *New Directions
    in Psychology,* ed. by T. Newcomb. Holt, Rinehart and Winston, New
    York, 1962.

[^2]: Eugene Galanter, *Psych Tech Notes V. 2.1* Adams • Bannister •
    Cox, 1994.

[^3]: *Construction vs. Choice in Cognitive Measurement,* ed. by R. E.
    Bennett & W. C. Ward. LEA, Hillsdale, NJ, 1993.

[^4]: Adapted from *National Science Board, Science and Engineering
    Indicators,* Tenth Ed., 1991.

[^5]: Carol Dweck, Motivational processes affecting learning. *American
    Psychologist*. October 1986.

[^6]: Presented at the 1995 American Society for Engineering Education
    Annual Conference and Exposition, Anaheim, CA June 25-28, 1995

[^7]: This text is an edited version of Mr. Jones' verbal testimony to
    the EHR Advisory Committee on November 1, 1995.

[^8]: This text is an edited version of Dr. White's verbal testimony to
    the EHR Advisory Committee on November 1, 1995.