t PUBUCATiONS

fl^

I I^^^V" United States Department of Commerce

I ^1 I National Institute of Standards and Tectinology

NAT L INST. OF STAND 4 TECH R.I.C.

A111D3 733115

NIST Special Publication 730

NBS-INA — The Institute for Numerical

Analysis - UCLA 1947-1954

Magnus R, Hestenes and John Todd

-QC100

.U57

#730

1991

C.2
iI
NIST Special Publication 730

NBS-INA — The Institute for Numerical

Analysis - UCLA 1947-1954

Magnus R. Hestenes

Department of Mathematics

University of California

at Los Angeles

Los Angeles, CA 90078

John Todd

Department of Mathematics

California Institute of Technology

Pasadena, CA 91109

Sponsored in part by: The Mathematical Association of America

1529 Eighteenth Street, N.W.

Washington, DC 20036

August 1991

U.S. Department of Commerce

Robert A. Mosbacher, Secretary

National Institute of Standards and Technology

John W. Lyons, Director
National Institute of Standards U.S. Government Printing Office

and Technology Washington: 1991

Special Publication 730

Natl. Inst. Stand. Technol.

Spec. Publ. 730

182 pages (Aug. 1991)

CODEN: NSPUE2

For sale by the Superintendent

of Documents

U.S. Government Printing Office

Washington, DC 20402
ABSTRACT

This is a history of the Institute for Numerical Analysis (INA) with special emphasis on its research program during the period 1947 to 1956. The Institute for Numerical Analysis was located

on the campus of the University of California, Los Angeles. It was a section of the National Applied

Mathematics Laboratories, which formed the Applied Mathematics Division of the National Bureau

of Standards (now the National Institute of Standards and Technology), under the U.S. Department

of Commerce.

This history of the program at INA is concerned primarily with the development of mathematics pertinent to solving numerical computations. This development could happen only if some

mathematicians were proficient in handling the electronic digital computers. To insure that there

would be some, INA was constituted. It was well funded, and could attract first class mathematicians to take a year off for research at INA. There they were in the midst of people solving

problems of considerable difficulty using the digital computers. They were thus enticed into using

them. When this happened, many important developments emerged. This history is centered around

these people and asks "Who were there?", "What were their interests?", "What did they do?".

Key words: differential equations; digital computers; history; linear programming; matrix; numerical analysis; UCLA

iii

FOREWORD

In the 1930's A. M. Turing conceived the idea of a computer. The electronic technology

needed to construct a working model did not then exist. However, during World War II great

strides were made in electronic technology. Just after World War II there appeared the ENIAC. It was an electronic computer but not of the Turing type. However, the Tiu"ing computer was obviously just around the comer, and would run rings around the ENIAC. It was anticipated that the

Turing computer would give much assistance in various mathematical studies.

This could happen only if some mathematicians were proficient in handling the Turing computer. To assure that there would be some, the Institute for Numerical Analysis (INA) was constituted. It was well funded, and could attract first class mathematicians to take a year off for research

at INA. There they were in the midst of people solving problems of considerable difficialty by the use of the Turing computers. They were enticed into learning to use them. When this happened,

many important developments emerged, as recounted in the history that follows.

Summer 1985 J. Baridey Rosser

Professor Emeritus

Center for Mathematical Sciences

University of Wisconsin, Madison

V
(

I
PREFACE

Over 30 years have passed since the events described in this History took place. We have

recounted them to the best of our recollection and apologize for our sins of omission and commission. We have given references to our sources and the material which we have used will be deposited in appropriate places. The Quarterly Progress Reports and internal memoranda of the

Institute for Numerical Analysis (INA) were particularly useful. Attention is invited to the Proceedings of the 1987 Princeton Conference [131] in which the articles by G. B. Dantzig and ourselves are

particularly relevant.

Our point of view is probably biased towards the West and may not represent the official

Washington attitude. Further, in part because the original sponsor of the writing is the Mathematical Association of America, we have stressed the research and educational aspects of the program,

rather than the machine development and computational service aspects, although they had imdeniable importance. Finally, our selection of topics for more detailed discussion is undoubtedly different from that of others who might have undertaken this work— differential equations, Monte Carlo,

combinatorics, number theory could well have been emphasized more. We are grateful to many of our former colleagues who shared their reminiscences with us, and

we are grateful for the opportunity to communicate to a wider public some account of what was an

exciting time for both of us. The machine development aspect of INA has been well documented. A short history of this

development is reproduced in Appendix C. This history was written by H. D. Huskey, who was

responsible for the building of the SWAC, the National Bureau of Standards Western Automatic

Computer. There is, however, no adequate documentation of the research aspect of INA. Accordingly, we give here a history of the program at INA concerned primarily with the development of

mathematics pertinent to solving problems involving numerical computations. We include abbreviated remarks about other aspects of the program at INA. Our story is centered around people. We

ask "Who were there?", "What were their interests?", "What did they do?". We give only selected answers to the questions with the hope that they are sufficient to give a bird's-eye view of the

program at INA.

In a sense the researchers at INA pioneered in modern numerical analysis. "Graduates" of INA

are scattered all over the United States and elsewhere. They have done much to promote the

development of numerical mathematics pertinent for use of computing machines for scientific and

educational purposes.

We wish to single out for special mention three persons who played a dominant role in the

formation and maintenance of INA. They are E. U. Condon, John H. Curtiss, and Mina Rees.

Condon was the Director of the National Bureau of Standards (NBS) and was an enthusiastic

supporter of the project. He gave the "go ahead" for Curtiss to set up INA as part of the Applied

Mathematics Program of the NBS. Curtiss was the guiding force of the INA project and was

instrimiental in interesting prominent mathematicians to participate in the program. Mina Rees was

the Head of the Mathematics Branch of the Office of Naval Research (ONR). She was one of a group which saw need for the development of "machine" mathematics and encouraged Curtiss to

proceed with his program. She was instrumental in arranging substantial financial support by ONR

for the INA project. In fact, ONR funded the major portion of the mathematical research program

of INA. The machine development program was largely supported by the Air Force. Contributions

to this program were also made by the Army. These governmental agencies played an important

role in the formation of INA.

There are several mathematicians to whom we are especially grateful for their help in the

preparation of this volume. First, Olga Taussky-Todd, who was one of the original crew in 1947-48

and was a consultant to the NBS Applied Mathematics Division (AMD) from 1949 to 1957. Her

vii
wide knowledge of mathematics and mathematicians played an important part in the development

of NBS-INA, as will become apparent in our history. In addition, her outstanding memory and her

collection of INA photographs have been invaluable during the preparation of this history.

We are also indebted to Marion I. Walter, who was at INA in 1952-53 for her extensive

collection of INA photographs.

We also acknowledge the permissions to reproduce published photographs which have been

given by various organizations and individuals.

We appreciate comments on drafts of our manuscripts by E. W. Cannon, C. R. De Prima, and

J. H. Wilkinson.

Finally we appreciate greatly the work of Churchill Eisenhart and Joan Rosenblatt who carefully read our manuscript and tried to make m historians of science and of Shirley Bremer who

collated and checked all the changes and corrections and acted as our representative with the NBS

production staff who, imder the direction of Don Baker, Rebecca Pardee, and Ernestine Gladden,

did a splendid job.

M. R. Hestenes

UCLA

Los Angeles, CA 90024

John Todd

Caltech

Pasadena, CA 91125

viii
CONTENTS

Page

FOREWORD iu

PREFACE V

CHAPTER I The Beginnings 1

The War Years - 1939-1945 2

The Post-War Years 3

CHAPTER II The Period Summer 1947 through Summer 1948 7

CHAPTER III The Period Fall 1948 through Spring 1949 11

CHAPTER IV

The Period Summer 1949 through Summer 1950 17

CHAPTER V

The Period Fall 1950 through Spring 1951 23

CHAPTER VI

The Period Summer 1951 through Spring 1952 27

CHAPTER VII

The Period Summer 1952 through Spring 1953 33

CHAPTER VIII

The Period Summer 1953 through Spring 1954 39

CHAPTER IX

Some Typical Projects 45

Hilbert Matrices and Related Topics 45

Gerschgorin Theorems and Related Results 47

Solutions of Systems of Linear Equations 48

REFERENCES 67

PHOTOGRAPHS AND CARTOONS 73

APPENDIX A. John Hamilton Curtiss 1909-77 A-1

APPENDIX B. Two Papers by John H. Curtiss B-1

APPENDIX C. TheSWAC C-1

APPENDIX D. Symposia and Tables D-1

APPENDIX E. Educational Activities E-1

APPENDIX F. Personnel F-1

INDEX ind-1

ix
(
CHAPTER I

THE BEGINNINGS

This is a story of the Institute for Numerical Analysis (INA) with special emphasis on its research program during the period 1947 to 1954. The Institute for Nimierical Analysis was located

on the campus of the University of California, Los Angeles (UCLA). It was a section of the

National Applied Mathematics Laboratories (NAML), which formed the Applied Mathematics

Division (AMD) of the National Bureau of Standards (NBS), which was a part of the United States

Department of Commerce.

Harry S. Truman was President from 1945 to 1952 and Dwight D. Eisenhower was President

from 1952 to 1960. E. U. Condon was the Director of NBS from 1944 to 1951 and A. V. Astin was

the Director from 1951 to 1969. Although the Directorship of NBS was a Presidential appointment,

it was regarded as non-partisan.

The NBS was established in 1901 and the basic legislation was greatly amended in 1950.

Included in the 1950 amendment was authorization to "operate a laboratory of applied mathematics." This was permissive, not mandatory. In 1951 a Semi-Centennial celebration was held. At that

time a history of NBS was written by Rexmond C. Cochrane [12]. As part of this celebration a Symposium was held at INA on "Simultaneous Linear Equations and the Determination of Eigenvalues" [see Appendix D]. This Symposium was the first of a long series of Symposia on this and

related subjects. The contributions of these Symposia to the development of modem numerical

analysis were significant.

While it is tempting to trace the origins of INA back to Archimedes, or Pythagoras, or to

Leonard da Vinci, or to Peter the Great, or to other scientists who worked for their governments,

it is more realistic to begin with World War I. Governments realized that some scientists could

contribute more in their laboratories than in the field. Accordingly, scientists were recruited to

work on special programs of scientific and mathematical nature. We restrict our remarks to mathematical aspects of these programs. In the United States, for example, F. R. Moulton, G. A. Bliss,

O. Veblen, and others were stationed at the Aberdeen Proving Grounds to study problems in

ballistics. At this time Bliss developed a method of differential corrections for predicting the trajectories of shells fired from a long range cannon. This method proved to be very effective. When

World War II was imminent. Bliss resurrected this method in a graduate course on exterior ballistics

and wrote a book on this subject. The method of differential corrections is a generalized Newton's

method applied to differential equations. It is a quasilinearization method of the variety used later by

R. Bellman.

R. Courant made scientific contributions to the German war effort, even though he spent

considerable time on the front lines. His experiences are vividly described by Constance Reid [79].

In England, J. E. Littlewood was called upon to apply his talents to the study of ballistics. He

was stationed in London, because "The higher brass at Woolwich recognized that Littlewood

should not be subjected to routine chores and petty restrictions. He could 'live out' and make his

home with friends in London. If, in uniform, he carried and used an umbrella, it would not be seen."

His Adventures in Ballistics can be foimd in his Collected Papers [57]. Other mathematicians, who

have made significant contributions to Numerical Analysis, were not so fortunate. A. C. Aitken was

wounded and invalided. An accoimt of Aitken's experiences can be found in his autobiography [3]

and his obituary [3]. L. J. Comrie was also woimded as noted in his obituary [128].

Between World War I and World War II, there was little activity in the development of

National Mathematical Laboratories. However, during the Depression, the Works Progress Administration set up a Mathematical Tables Project in New York, imder the scientific direction of the

National Bureau of Standards. It was staffed largely by unemployed high school mathematicians

1
I

(about 100), supervised by a small group of mathematicians such as A. N. Lowan, Gertrude Blanch,

Ida Rhodes, M. Abramowitz, H. E. Salzer, and Cornelius Lanczos. The story of this group was

given in articles by Lowan [60] and by Blanch and Rhodes [6].

In Italy, M. Picone organized the Istituto Nazionale per le Applicazioni del Calcolo (INAC). It was founded in Naples in 1927 and established in Rome in 1932. Picone [72] characterizes INAC as a "living table" of functions. The general problem of numerical analysis is to compute/(x ) given x, granted reasonably generous interpretations off and x : for instance, x might be a matrix and f the

set of its characteristic values and vectors. In about 1952, Picone [73] discussing the accomplishments of INAC wrote that "It is the place where the marriage between functional topology and

numerical calcxilation has taken place." At this time there were close relations between INAC and

NBS-AMD. Picone and G. Fichera made brief visits and E. Aparo and D. Dainelli longer visits to

NBS-AMD. Fichera and Aparo made extended visits to INA. NBS-AMD returned these visits and

contributed to conferences organized by INAC.

The War Years 1939-1945

World War II began on September 3, 1939 for the United Kingdom and on December 7, 1941

for the United States. VE Day was May 7, 1945 and VJ Day was August 14, 1945.

Initially, it was not clear where scientists could be most usefully deployed. Some were more

adaptable and some were more imaginative and aggressive and found suitable slots for themselves.

In the United Kingdom in the summer of 1940, a census of scientists and engineers was made under

the direction of C. P. Snow. Some of the civil servants involved seemed more interested in the

novelty of compulsory registration announced on radio, in some way going back to the town crier,

than in the actual classification. Many of the academics, who participated in this census, were able

to find for themselves employment suited to their experience.

During these years in the United Kingdom and the United States, existing groups with mathematical strengths were built up and new groups developed around energetic mathematicians. For

example, in the United Kingdom, we note the very specialized cryptographic work at Bletchley,

which had a great impact later. For this se& Annals of the History of Computing 1-, 1979-. We note

also the Differential Analyser Group at Manchester, whose activities have been described by D. R.

Hartree [26]. Another organization was the so-called Admiralty Computing Service (ACS) which

was developed, beginning in 1942, by D. H. Sadler and John Todd. A description of its activities is given in their article [87]. Important contributions were also made by a commercial organization,

namely. Scientific Computer Service Ltd., which was organized by L. J. Comrie and with which J. C. P. Miller and H. O. Hartley were associated [128]. A report on British Intelligence [33] credits

Comrie for locating an important enemy radio installation in France io 1940.

The war-time activities of mathematicians in the United States are well documented in articles

by M. S. Rees [77] and J. B. Rosser [84].

We note that the ENIAC was developed at the Ballistics Research Laboratories at the

Aberdeen Proving Groimds under the supervision of the University of Pennsylvania. The ENIAC

was the first large-scale electronic automatic computing machine built in the United States.

Among the many scientific organizations engaged in war-related activities was an Applied

Mathematics Panel with Warren Weaver as Chief. It was set up by the National Defense Research

Conmiittee. This panel had several groups working at various imiversities and other institutions.

One of us (Hestenes) was attached to the Columbia University Group. Another of the groups was

the Mathematical Tables Project, under Arnold Lowan, which later became the basis of the NBS

Computation Laboratory.

Among the mathematical leaders in Germany were Alwin Walther, Robert Sauer, and Lothar

Collatz. There was developed a National Research Institute at Oberwolfach imder the direction of

Wilhelm Suss. An accoimt of this Institute can be foimd in the articles by Irmgard Siiss and by John

Todd in a book edited by E. F. Beckenbach and W. Walter [4]. Extensive accounts of mathematical

activities in Germany fi-om 1939 to 1946 were issued as Field Information Agencies Technical

Reports [103,123] edited by W. Suss and A. Walther respectively.

2
The Post-War Years

During the time of reco .truction, established organizations, such as the Picone Institute and

the Oberwolfach Institute, de eloped within the then existing limitations.

Before discussing in some detail the beginnings of the national organizations in the United

Kingdom and in the United States, of which we have some personal knowledge, we briefly mention

two European centers, a program in the U.S.S.R., and an international development. We shall see

that both the British and American organizations owed their existence in large part to the respective

naval establishments. For an account of the American contributions we refer to two papers by Mina

S. Rees [75,76].

In 1948 a Mathematisch Centrum was set up in Amsterdam to deal with problems in pure and

applied mathematics, statistics, and the construction and use of computers. Among those active in

this center were D. van Dantzig, A. van Wijngaarden, and J. G. van der Corput. This organization

has made and continues to make notable contributions in all its areas of interest, e.g., in the study of

North Sea floods.

In 1947 the Swedish government set up an organization to develop and use computers. This,

too, has had considerable influence, as indicated, for example, by the relatively high number of

Swedish mathematicians specializing in nimierical mathematics, e.g., C. E. Frdberg, G. G.

Dahlquist, H. O. Kreiss, and V. Thom6e.

The immediate post-war program in the U.S.S.R. is described by S. Vavilov in "Our Five-Year

Plan for Science," Soviet News, 26 September, 1946. We quote:

"Mathematics, which is of vital importance to natural science, to technique,

and to such social sciences as economics, is directly linked up with problems of

philosophy and logic. Much of our five-year plan for mathematics is directed

towards assisting other sciences. For example, we are stressing questions of the

theory of probability —particularly those bearing on the interpretation of observations and research on partial differential eqxiations, particularly those associated with what may be termed "machine mathematics," that is, the solution of

mathematical problems with the help of calculating devices.

Calculating machines have been known for centuries. But never has "machine mathematics" reached such scope as at the present time. New calculators

devised on electrical principles make it possible to solve extremely complex

mathematical problems connected with technique and the various branches of

natural science. So important do we think this side of mathematics is, that we are proposing in the inomediate future to devote to it a special institute.

However, machines can never displace mathematical thought. A characteristic of mathematical thought is its boldness, its imaginative power. Such creative

mathematics, which at times find no immediate application in technical science,

has always found fertile ground in our Academy; and its development must continue on a broad scale. Such subjects as non-euclidean geometry, the tensor calculus, and the theory of groups, which seem to be abstract studies absolutely cut

off from life and reality, nevertheless suddenly assume a decisive significance at definite stages of scientific development.

This explains the inclusion in the Academy's plan of the problems of the

theory of numbers, abstract algebra, topology, and mathematical logic."

Much later an international activity was conceived by UNESCO. During the years of planning

and competition for a permanent International Computing Center (ICC) an ICC Preparatory Committee organized several symposia in the late 1950's (Proceedings of some were published by

BirkhSuser Verlag, Basel). Among the leaders in the negotiations were Stig Comdt, R. de Possel, A.

Ghizzetti, and C. Berge. A Provisional ICC (PICC) commenced operations in Rome in January

1958. The ICC was finally set up in November 1961. Both the PICC and ICC published Bulletins;

16 issues 1958-1962 by PICC and 6 volumes 1962-1967 by ICC.

3
In the United States, a private research organization called the Rand Corporation was formed.

Its mathematics branch had close ties with INA, particularly in the study of Theory of Games,

Linear Programming, Computers, and related fields. One of us (Hestenes) was a consultant to

E. Paxson at Rand from 1947-1950. In his study of time-optimal flights of a fighter plane, he was led

to a formulation of a general Optimal Control Problem and gave basic conditions for its solution. In

a similar study of encoimters of fighter planes, the Russian mathematician L. S. Pontryagin was led

to an Optimal Control problem of the same type. His formulation of the basic conditions that a

solution must satisfy is now called "Pontryagin's Maximum Principle." At Rand, R. Bellman, in the

study of a similar problem, was led to the theory of Dynamical Programming. R. Isaacs at Rand

developed a theory of Differential Games, as did Pontryagin. Studies of these topics were also

pursued at Rand by L. Berkovitz. The group at Rand was computer oriented and sought computer

solutions to their problems when it was feasible to do so. They also made significant contributions

in the field of Mathematical Programming.

We turn now to a general discussion of the foimding of the national mathematics laboratories

in the United States and in the United Kingdom. These laboratories were constituted as units of the

National Bureau of Standards (NBS) in the United States and of the National Physical Laboratory

(NPL) in the United Kingdom. Their location and their immediate successes stemmed from the fact

that the then heads of the NBS, E. U. Condon, and the NPL, C. G. Darwin, were substantial

mathematicians, though technically physicists. These two mathematical laboratories became in a

sense "sister" institutions in that they kept in close contact and staff members made frequent visits,

one to the other.

C. G. Darwin's interest in special functions is evidenced by his paper on "Weber's Function"

[17] and by the tables of J. C. P. Miller [62]. John Todd recalls Darwin telling him that, on his way

to the United States (to head the British Commonwealth Scientific Office) during wartime on one of

the Queens, he told the captain that he had worked out, with primitive observations and primitive

computing equipment, the position of the ship, for which he was promptly reprimanded. Details of

Darwin's life and work can be found in the obituary by Sir George Thompson [113].

In 1945 those concerned with the Admiralty Computing Service arranged, through appropriate

channels, for an invitation for the Department of Scientific and Industrial Research, of which Sir

Edward Appleton was the head, to consider the establishment of a National Mathematics Laboratory (NML), in parallel to the National Physical Laboratory (NPL). To support this suggestion a memorandum was prepared by A. Erd61yi, D. H. Sadler, and John Todd. It was entitled "Memorandum on the centralization of computation in a National Mathematical Laboratory." See also

[130]. Two related articles were also prepared, one by D. H. Sadler and John Todd and one by A.

Erd6lyi and John Todd. After various negotiations, the NML was realized as a Division of NPL

with J. R. Womersley as its first Chief (1945-1954). Several members of the computing staff of the

ACS trained by D. H. Sadler, including L. Fox and F. W. J. Olver, joined the new organization.

Among the other members of the staff of this Division were A. M. Turing, J. H. Wilkinson, and E.

T. Goodwin (Chief from 1954-1972).

Return now to the story of E. U. Condon and the NBS. As remarked above Condon was a

substantial mathematician as well as a distinguished physicist. As a person, he was compassionate,

outspoken, but controversial, especially in the McCarthy era. All significant scientists have international contacts and this led to the characterization of Condon as the weakest link in the chain of

security by Congressman J. Pamell Thomas (who was later jailed for taking kickbacks from his

staff). His responses in hearings were legendary. We recall how, on a visit to a major university, he

encountered an imemployed scientist, living in the Common-Room. On his return he asked us to try

to find an opening for him— which we did and he made notable contributions to the INA.

For details of Condon's activities we refer to the sympathetic obituaries by P. M. Morse [64]

and by Churchill Eisenhart [18]. Eisenhart has been with the Applied Mathematics Division (AMD)

in various capacities since its beginning. Morse of MIT was an enthusiastic supporter of the work of

the AMD from its earliest days and, for example, was chairman of the committee which organized

the famous Handbook of Mathematical Functions, edited by M. Abramowitz and I. A. Stegun [1].

4
We finally come to the story of the establishment of the National Applied Mathematical Laboratories (NAML) and the choice of UCLA as the location for the Institute for Numerical Analysis

(INA).

At the end of World War II farsighted leaders in the U.S. Government deemed it to be

important that scientific research be promoted in order to meet the future needs of society. Accordingly, various agencies were set up in order to promote research. In particular the Navy established

the Office of Naval Research (ONR). In 1946 the Mathematics Branch of ONR was formed and

Mina Rees was selected to be its head. At about the same time John Curtiss was chosen to be the

Assistant to the Director, E. U. Condon, of the National Bureau of Standards (NBS). Mina Rees and

John Curtiss were largely responsible for the formation of INA. In doing so they frequently sought

advice from C. B. Tompkins, a mathematician whose war experience convinced him that high-speed

digital computers were on the horizon and needed to be developed. In 1953 Tompkins became the

Director of INA.

Early in 1946 ONR requested the assistance of NBS in the establishment of a national mathematical computation center. Cooperative studies of this proposal revealed the desirability of a more

fundamental approach. Specifications for a more general facility evolved after lengthy consultations

with scientific representatives of industrial groups, education institutions, and other governmental

agencies. Early in 1947 NBS issued a prospectm for the formation of National Applied Mathematical Laboratories (NAML) to be guided by a committee of representatives of interested outside

groups, to be called the Applied Mathematics Executive Council. (In 1949 this was rechartered as the Applied Mathematics-(4Jvwory Council.) In this prospectus it was proposed that these laboratories consist of four major imits. The first of these was a section devoted primarily to research and

training, to be located in California and to be called the Institute for Numerical Analysis. We are not

clear about the reasons for choosing a location in California rather than a more central one. Perhaps

Condon was remembering his student days at Berkeley. Perhaps the climate was a main attraction.

A certain distance from Washington was certainly desirable, for some mathematicians are imcomfortable with strict dress code and regular hours. The second laboratory was a large Computing

Laboratory to be located in Washington, DC, or in New York. The third unit was a Statistical

Engineering Laboratory, and the fourth was a Machine Development Laboratory devoted to the

development of automatic digital computing machinery. The last two units were to be located at the

NBS in Washington, DC. When these laboratories were formed in 1947, the second was also located

at NBS and the first was scheduled to be located at UCLA.

Late in the fall of 1946 John Curtiss came West on a mission for E. U. Condon to seek a home

for the proposed INA. He first visited Berkeley and Stanford and then came south to UCLA. At the

request of Provost Qarence A. Dykstra, Samuel Herrick of the UCLA Astronomy Department

arranged for Curtiss' UCLA presentation of the proposal to set up an Institute for Numerical

Analysis on the west coast and to set up sister applied mathematical and numerical laboratories at NBS. Edwin F. Beckenbach, in particular, was excited about the possibility of having such an

Institute located at UCLA with a flow of outstanding mathematicians as visitors. He took the

initiative for writing a proposal to NBS for the location of INA on the UCLA campus. In doing so he was ably assisted by Ivan Sokolnikoff. John Curtiss also visited Lee Du Bridge, the President of

Caltech, who said he had no space available for an INA project but would be glad to support the

UCLA proposal. After the proposal by UCLA had been accepted by NBS, Beckenbach became an

early participant of Project INA and Sokolnikoff became a local advisor.

According to the proposal to NBS, the primary function of the INA would be to conduct

research and training in the types of mathematics pertinent to the efficient exploitation and further

development of high-speed automatic digital computing machinery. A secondary function would be

to provide expert computing service for local groups with immediate computational problems, and

to assist, as necessary, in formulations and analysis of difficult problems in applied mathematics. To

carry out these functions INA would be supplied with one general-purpose automatic electronic

digital computing machine of large capacity together with desk calculators and punch-card equipment as needed. More precisely the functions of INA would be the following:

5
(a) Plan and conduct a program of research in pure and applied mathematics directed primarily at developing methods of analysis which will permit the most efficient and general use of

high-speed automatic digital computing machinery.

(b) Conduct training programs for personnel of industry, Government agencies, and educational institutions, in the theory and disciplines needed for the full exploitation of high-speed automatic digital computing equipment.

(c) Study and formulate requirements for the intelligence and internal organization which

high-speed automatic computing machinery should have; develop overall performance specifications for such machinery.

(d) Serve as a center at which competent scholars can explore the usefulness of high-speed

automatic digital computing machinery in their own fields of interest.

(e) Formulate requirements for further mathematical tables and other aids to computation;

review the overall program of the National Applied Mathematical Laboratories with regard to the

production of such objects and advise the Administration and Executive Council accordingly.

(f) Review, analyze, and, as necessary, assist in the mathematical formulation of problems in

applied mathematics of the more complex and novel type arising in outside laboratories.

(g) Provide a computing service containing both standard and high-speed automatic equipment (when available) for local industries, educational institutions, and Government agencies.

(h) Assist and conduct liaison with related programs in local educational institutions.

(i) Maintain a consulting service on special problems in applied mathematics.

(j) Prepare reports of the research described above; also prepare training manuals, bibliographies, and indices.

The development of the AMD and, in particular, the INA is described in the (initially) quarterly reports; "Projects and Publications of the National Applied Mathematics Laboratories," which

were given a quite wide distribution. Projects with managers, objectives, backgroimds, priorities,

and magnitude were set up, continued, and terminated. Progress on these was reported quarterly.

The preparation of these reports was quite arduous, for mathematicians were not yet conditioned to

the preparation of grant proposals (the NSF was not set up until 1950).

The name NAML was dropped in 1954 in deference to the views of other organizations who

aspired to a national title. Related papers on the history of the Institute for Nimierical Analysis have appeared in [134-

136].

6
CHAPTER II

THE PERIOD SUMMER 1947 THROUGH SUMMER 1948

The National Applied Mathematics Laboratories were formally organized as Division 11 of the

National Bureau of Standards on July 1, 1947. The plan of organization followed closely the one set forth in the "Prospectus" and approved by various officially interested agencies. John H. Curtiss

was made their Chief. It was decided to locate the Institute for Numerical Analysis at UCLA.

However, INA was temporarily housed at NBS in Washington, DC, awaiting the completion of

adequate facilities at UCLA. The first appointees to INA were Olga Taussky-Todd, John Todd, and

Albert Cahn. The first two projects sponsored by INA were entitled "Characteristic roots of matrices" and "Applications of automatic digital computing machines in algebra and number theory."

The Todds were their project managers. These projects were the first of many to which the

researchers at INA made significant contributions. In a sense they signalled the beginning of a new

endeavor which later became known as Computer Science. Cahn's duties were largely administrative. Otto SzSsz joined the group in February 1948. His specialty was the theory and applications of

infinite series.

John Todd's fiirst contact with NBS-INA was a letter from Curtiss, dated April 16, 1947,

describing the plans and inviting him to spend the next academic year at the "Institute" performing

research in the field of his choice— the sole condition being that tb ; research should have some

general bearing on the functions of the "Institute" as set forth in ths Prospectus. Olga Todd was

appointed as soon as she arrived in Washington, in the fall of 1947.

SzSsz and the Todds spent the first few months of 1948 at the Institute for Advanced Study in

Princeton, at the invitation of John von Neumann. During that period Olga Todd, in discussion of

questions raised by S. D. Chowla, J. Nielsen, and I. Reiner, saw the relevance of a theorem of

Latimer and MacDuffee [52] on integral matrices. Chowla encouraged her to find a simple proof of

this. This was accomplished at INA in the siimmer of 1948, using the characteristic vectors of the

matrices. This is now accepted as the definitive proof. She mentioned this to Hans A. Rademacher

[74], who spent the summer at INA. He immediately foimd a use for it in his work on Dedekind

simis. Rademacher gave a series of expository lectures on elliptic and modular functions. In addition

he worked on the accumulation of round-off errors in numerical computations.

The new building for the Institute for Numerical Analysis at UCLA was occupied in April

1948. The Todds, Otto SzSsz, and Gertrude Blanch were the initial members. Blanch was an expert

on Mathieu Fimctions. She organized the computational unit. Almost immediately computational

services were performed for the Departments of Astronomy and Engineering. In particular. Rocket

Navigational Tables were constructed at the request of Samuel Herrick of the Astronomy Department. The period April-Jime 1948 was an organizational period in which the scientific and

computational staffs were acquired. Roselyn A. Siegel became the first junior member of the computational unit.

The table-making tradition of the NBS was continued at INA. Gertrude Blanch was largely

responsible for ensuring that the tables issued were of the quality (typographic and accurate) which

the scientific public had been led to expect. The following tables in the NBS Applied Mathematics

Series (AMS) are among those which were almost wholly the work of INA. (Some of the copy was

prepared on a card controlled typewriter at the U.S. Naval Observatory.)

AMS 11 (1951) Tables of Rational Arctangents, by John Todd. Reprinted as a paperback in

1965.

AMS 20 (1953) Tables for Rocket and Comet Orbits, by Samuel Herrick.

AMS 50 (1959) Tables for the Bivariate Normal Distribution Function and Related Functions.

The tables of contents of these publications are given in Appendix D.

7
Initially all computations were carried out on standard desk calculators. The Institute quickly

acquired an IBM Card Programmed Calculator, dubbed a CPC. This calculator was programmed

on a small plug board. Input was accomplished by reading punched cards. It had a typewriter

output. If one wished, one could print out the result of each step in the computation. Doing so was

an education to the user because it clearly demonstrated the effect of roimdoff errors. It showed the

need for suitable scaling of nimibers and the advantages of using floating point arithmetic. Originally it was planned to purchase a high-speed automatic digital computer for use at INA. This was

because NBS was not permitted to compete with industry by building its own computer. However,

noncompetitive experimental models could be constructed. Because no automatic computers would

be available in the near future, it was decided to build an innovative computer at INA based on the use of a Williams tube memory. Harry D. Huskey was in charge of its development. It became

known as the SWAC, the 5tandards Western AvLtomatic Computer. Construction of the SWAC

began in January 1949. It was dedicated in August 1950 and decommissioned in 1967. (For details see Appendix C.)

Douglas R. Hartree, a Mathematical Physicist from Cambridge, England, was invited to be the

first Director of INA. Unfortimately, he could stay only a short time. The invitation of Hartree was

a natural one because he was a pioneer in machine computations. In the 1920's he devised a method

of Self-Consistent Fields to solve problems which emerged with the advent of wave mechanics.

This is basically an iterative method of successive approximations. The numerical labor was enor- mous. Upon learning about the Differential Analyzer (D.A.) being developed by Vannevar Bush at MIT, he realized that the D.A. could be exploited for his problem. Accordingly, he pioneered in the

development of D.A.'s in England for scientific computation. Hartree and his collaborators made

effective use of the D.A. to solve a great variety of problems before and during World War II. During World War II, the ENIAC was developed in the United States at the University of Pennsylvania. Initially, its main function was the determination of trajectories for the Aberdeen Proving

Grounds. Hartree visited the University of Pennsylvania and solved two-point boimdary value

problems on the ENIAC. He also visited the Aberdeen Proving Grounds and lectured on computing instruments like the D.A. and the ENIAC. Upon returning to England, he kept in close touch

with the development, by M. V. Wilkes, of automatic electronic computing machines (the EDSAC

Project) in England.

It was Hartree's experience that, by and large, the mathematical conmiunity was inexperienced

in nimierical analysis and showed very little interest in the subject. Some even belittled it. In 1948

in an INA Symposium (see AMS 15 in Appendix D) Hartree said, "One of the unsolved problems

of Numerical Analysis is how to overcome the attitude of the Mathematical Fraternity on this

subject." Of course, one of the purposes of the NBS Laboratories and, in particular, INA was to

solve this problem. That this problem has been solved is in large measure due to the influence of

John Curtiss and the support given him by NBS and by various governmental agencies under the

leadership of Mina Rees of ONR.

During Summer 1948 the research program of INA was carried out intensely under the direction of D. R. Hartree with the assistance of a number of distinguished mathematicians on temporary

appointments. Supplementing the research program, a set of "Symposia on Modem Calciilating

Machinery and Numerical Methods" was held on July 29-31, 1948. Sponsorship of the Symposia

was undertaken jointly by UCLA and INA. The agenda involved 32 addresses and various other

roimd-table discussions. The topics ranged from reports on the status of machine construction

projects to the statements of mathematical problems awaiting high-speed automatic computing

machinery for solution. The opening address was given by John von Neumann. Douglas R. Hartree

spoke on "Some unsolved problems in numerical analysis." Solomon Lefschetz talked about "Numerical calculations in nonlinear mechanics." Bernard Friedman presented a paper on "Wave propagation in hydrodynamics and electrodynamics." George B. Dantzig discussed "Linear

progranmiing." This symposiimi served as dedicatory exercises for the Institute and the proceedings

were published as AMS 15 (see Appendix Q.

8
SUMMER 1948

The roster of the senior research personnel for Summer 1948 was the following:

Douglas R. Hartree (Cambridge, England; Acting Director)

Edwin F. Beckenbach (UCLA representative)

Gertrude Blanch (Head of Computational Unit)

Robert H. Cameron (Minnesota)

George E. Forsythe ( INA)

Robert E. Greenwood (Texas)

Samuel Herrick (UCLA Astronomer)

William E. Milne (Oregon)

Hans A. Rademacher (Pennsylvania)

Wladimir Seidel (Wayne State)

Otto Sz^z (Cincinnati)

John Todd (London)

Olga Taussky-Todd (London)

A complete roster of INA personnel for Summer 1948 and for other periods is given in Appendix F.

During this period D. R. Hartree gave a series of lectures pertinent to automatic computing

machinery and to related problems in numerical analysis. By so doing he gave direction to the

research to be carried out at INA. He envisioned a program centered around the problem of solving

ordinary and partial differential equations by machine methods. Studies in numerical integration

were carried out by D. R. Hartree, W. E. Milne, and R. E. Greenwood. A "Simpson's Rule" for the

numerical integration of Wiener Integrals was devised by R. H. Cameron. E. F. Beckenbach was

concerned with applications of conformal mapping and with the study of inequalities. H. A.

Rademacher carried out numerical investigations of the zeros of the Zeta-function and studies in

number theory. The Todds and O. SzSsz continued investigations begun earlier. This indicates the

type of programs in which the senior staff were engaged during Summer 1948.

A problem completed by the Todds [106] while at INA was a problem arising in connection

with copyright law. This is related to the theory of error-correcting codes, to the theory of factorial

experiment designs, and to the practice of (British) football pools. Although the original problems

were solved by hand, the next case was not solved imtil much later with the heavy use of a computer by H. J. L. Kamps and J. H. van Lint [41].

The Todds also introduced the study of the Hilbert matrices

H„ = [l/(/+;-l)] (w = l,2,...,n)

for n = 2, 3, ... . These matrices are positive definite synametric matrices whose inverses are given by

explicit formulas. Their eigenvalues are clustered near X = 0. These matrices are therefore highly

ill-conditioned even for relatively small values of n . We found that Hilbert matrices were useful test matrices for testing numerical procedures involving matrix computations. A detailed account of

Hilbert and related matrices is given in Chapter IX.

John Todd also analyzed the behavior of a 5-term difference approximation to the differential

equation y' = —y, which had a pathological behavior. This was an early example of the study of

numerical stability in this area. He later analyzed the convergence and stability of various discretizations of the classical second order partial differential equations using Williamson's theorem on the

characteristic values of certain partitioned matrices.

Otto SzSsz was one of the few visitors who spent several periods at INA. As remarked above,

his specialty was the theory and applications of infinite series. Among his work at INA was the

study of a generalization of Bernstein polynomials which give a construction of a uniform approximation to continuous functions by polynomials, of which an existence proof was given by Weierstrass. SzSsz also collaborated with John Todd in the study of the problem of mmierical evaluation

9
of improper integrals. Qassical quadrature formulas, such as Simpson's Rule, apply to integrals over

a finite interval [a,b]. The problem at issue is the approximate evaluation of the integral off(x) over

[0, 00 ) as the limit of Riemann sums X hf{nh). See his collected papers [100].

George E. Forsythe was the first regular member of research staff of INA. He came to INA

with a strong background in the applications of mathematics to meteorology. He was a imiversalist

in the sense that he collaborated with all members of the research staff and with other members of

INA. He was very active in the computational aspects of the projects pursued at INA. He also

became a leader in the educational program of the Institute. George Forsythe received his Ph. D. at Brown University in 1941. During 1941-42 he was an instructor of mathematics at Stanford University. During World War II he participated in the meteorology program at UCLA, at which time he

collaborated with JOrgen Holmboe and William Gustin in writing a book entitled Dynamic Meteorology. During 1946-47 he was a Research Engineer at the Boeing Aircraft Company in Seattle. He

returned to UCLA in 1947 as an Assistant Professor in Meteorology, before joining INA in 1948.

His background in applied mathematics helped make him an outstanding member of the research

group at INA.

As a part of its educational program INA employed junior members who were candidates for

advanced degrees. They acted as assistants to the senior staff. The first appointees were Raymond

P. Peterson, Jr., Robert H. Sehnert, and Marvin L. Stein.

One of the significant appointments made at this time was the addition of Everett C. Yowell to

the computational unit to be in charge of Machine Computations. Being a mathematical astronomer

he had considerable experience in this area. He quickly became the chief advisor to the research

staff on the use of the CPC and other equipment. He also performed useful computational services

for various governmental agencies.

Among the peripheral activities, which John Curtiss on occasion frowned upon, were studies in

the theory of orthogonal polynomials. For instance, there were several proofs of Turin's inequality

for Legendre polynomials

P„2-P„_aP„+i>0

and its generalizations by SzSsz, Beckenbach, Seidel, and Forsythe. A conjectxire by John Todd on

the extrema of Legendre Polynomials, established asymptotically by R. Cooper, was completely

established by G. Szegd and then established in the Jacobi case by O. Sz^z and in the Laguerre case by J. Todd. Attempts to obtain a unified treatment have not succeeded so far. Detailed references to

the papers mentioned in this paragraph are given in the second paper reproduced in Appendix B.

Such diversions from the main program are unavoidable. When a group of mathematicians

learns of a problem within or near their field of competence, it is only natural for them to attack it by their own methods. These digressions improve communication between members of the organization who learn the expertise of their colleagues. A somewhat similar problem arises in Service

Departments. Arrangements have to be made so that the senior people have some freedom for

research in their field of interest and so that the junior people have educational opportunities to

improve their skills, either at in-house courses and seminars or at local universities. These are mentioned in the course of this history. Generally arrangements were made to hire for Summer

work winners of the Westinghouse Science Talent Search— a notable example is E. C. Dade at NBS-Washington. Later a program of post-doctoral research associates imder the control of the

National Research Comcil was instituted. Among those appointed at NBS-Washington, who made

particularly relevant contributions to numerical mathematics, were Marvin Marcus and John R.

Rice.

10
CHAPTER III

THE PERIOD FALL 1948 THROUGH SPRING 1949

At the end of the Summer Session, several of the investigators returned to their respective

universities. The Todds returned to the University of London. A year later they joined the staff of

the National Applied Mathematics Laboratories at NBS in Washington, DC. John Todd became

Chief of the Computation Laboratory and Olga Taussky-Todd became a Mathematics Consultant in

the Division. They had an active interest in the development of INA and made frequent visits to

INA.

Hartree returned to Cambridge, England, at the end of October. As a result John Curtiss

became Acting Director of INA until a qualified director could be found. The Senior Researchers

during the period October 1948-June 1949 were:

Edwin F. Beckenbach (UCLA Representative)

Gertrude Blanch (Head of Computational Unit)

George E. Forsythe (INA)

Samuel Herrick (UCLA)

Harry D. Huskey (Head of Machine Development)

Cornelius Lanczos (Joined INA in January 1949)

William E. Milne (Oregon)

Alexander M. Ostrowski (University of Basel, Switzerland)

Wladimir Seidel (Wayne State)

The Junior Members of the Research Staff for this period were:

Howard W. Luchsinger Raymond P. Peterson, Jr. Robert H. Sehnert Marvin L. Stein

As the Institute developed, Wilbur W. Bolton, Jr., joined the administrative imit to help with

budget problems. He was the NBS budget chief, 1951-56, and later became the Budget Officer of

the National Science Foundation.

In December 1948 Harry D. Huskey arrived at INA for the purpose of designing and building an Electronic Digital Computer. He received his Ph. D. in mathematics in 1943 under the sponsorship of Tibor Rado at Ohio State University. He was an instructor in mathematics at Ohio State,

1942-43, and at the University of Pennsylvania, 1943-46. He first became interested in computers in

1944 working on the ENIAC project at the University of Pennsylvania. The ENIAC was the first

large scale electronic computer to be constructed. The year 1947 was spent in England at the

National Physical Laboratory where he worked with Turing on the ACE computer project, supervised the design, and began construction of a pilot model of the ACE. In 1948 Huskey joined the

staff of the National Bureau of Standards in Washington, DC, as Chief of the Machine Development

Laboratory of the Applied Mathematics Division. In December 1948 he was transferred to INA as an Associate Director in charge of the design and construction of the SWAC—the National Bureau

of Standards Western Automatic Computer. Huskey and his associates were pioneers in this field.

The story of the SWAC is given in two articles by Harry Huskey [37,38]. We have reproduced the

second of these articles in Appendix C to give the reader an authoritative description of the SWAC

Project.

In January 1949 the UCLA physicists, Alfredo Bafios, Jr. and David S. Saxon became consultants to INA. They made significant contributions to the program at INA.

Cornelius Lanczos became the second regular member of the research staff at INA. He was

bom in Hungary in 1893. In 1921 he obtained his Ph. D. in Physics for a thesis entitled "Function

11
Theoretical Relations of Maxwell's Vacuum Equations" at the University of Szeged under Professor Rudolph Ortvay. Shortly thereafter he went to Germany, first to the University of Freiburg in

Breisgau and then to Frankfurt am Main. In 1926 he published a paper on an interpretation of

quantum mechanics on a continuimi basis in terms of integrals. This was the earliest continuum

theoretic formalism of quantimi mechanics, preceding SchrcJdinger's first conmiunication by about

4 weeks. Lanczos was also interested in the Einsteinian equation of gravity, and, at the invitation of

Einstein, worked with Einstein on the theory of relativity at the University of Berlin during the

Academic year 1928-29. In 1931 he published his first paper on the quadratic action principle in

relativity. Lanczos came to the United States in 1931. He joined the staff at Purdue University

where he remained for 15 years. He became interested in numerical analysis and, in 1938, developed

his theory of economization of polynomials, now known as the Lanczos-Tau method. In 1942 two

papers were published in conjunction with G. C. Danielson on practical techniques of Fourier

Analysis. In 1943-44 Lanczos was on the staff of NBS working on the Mathematical Tables Project.

During the period 1946-48 he was employed by Boeing Aircraft Company in Seattle, where he

obtained industrial experience in numerical methods. Although initially his numerical methods were

designed for use on standard desk computers, they turned out to be very suitable for "automatic"

machine computation. His wide experience enabled Lanczos to make outstanding contributions at INA.

During this period the research program was channeled into the following three lines of

work: development of numerical methods in conformal mapping, the exposition and further development of the numerical solution of ordinary differential equations, and studies in the use of probabilistic methods in nimierical analysis. In addition, certain studies, such as the programs begun

earlier by Herrick and by Beckenbach, were continued. Wladimir Seidel of Wayne State University

initiated a study on numerical methods in conformal mapping by compiling a bibliography of

pertinent references. He also translated into English a collection of Russian papers on conformal

mapping entitled, "Conformal representation of simply and multiply connected regions," by L.

Kantorovitch, V. Kryloff, G. Golouzin, P. Melentief, and others. William E. Milne, a well-known

numerical analyst from the University of Oregon, undertook the preparation of a monograph [63] on

numerical solutions of differential equations. Lanczos sought to develop a practical and economical

method of evaluating the eigenvalues of complex matrices and to develop an economical method for

solving simultaneous linear equations. Raymond P. Peterson, Jr. and John Curtiss began the study

of Monte Carlo methods for solving partial differential equations and integro-differential equations.

Alexander Ostrowski of the University of Basel, Switzerland, became a frequent visitor at INA.

We always looked forward to his visits. During his first visit he developed various formxilas for

evaluating definite integrals and sought to fiind new and better ways of dealing with algebraic

equations. He also participated in the research program on numerical methods in conformal mapping. He was a catalyst in the group.

Ostrowski was one of the most versatile mathematicians of our time. This is evident from his

Collected Papers being published in six voliraies, beginning in 1983, by Birkhauser. He is perhaps

best known for his determination of the valuations of the field of rationals. His appointment to

NBS-INA was suggested by the Todds, who were aware of his earlier work on the stability of the

Newton Process and what is now called complexity. It was he who introduced the "Horner" as a

imit of work: the evaluation of a polynomial of degree n by the Homer process. He later showed

that the Horner method (synthetic division) was (in cases n =3,4) the most efficient way of evaluating a polynomial. He also began the study of symbolic integration. Ostrowski published in 1937 two

classical papers concerning bounds for determinants, connected with the results of Hadamard,

Minkowski, and others. Following a suggestion made to him at INA by Olga Taussky, he made

many contributions to Gerschgorin Theory and to obtaining bounds of eigenvalues of matrices. He

was also concerned with iterative methods for the solution of linear systems. His attention was

invited by John Todd to a popular method for the solution of the algebraic eigenvalue problem

Ax = Xx. This method consists of guessing an estimate of v of an eigenvector of A, obtaining an

estimate of the eigenvalue \ by the Rayleigh quotient of v, and obtaining a new estimate by solving

the nearly singular system Ax = \x. Ostrowski analyzed this process in a series of six papers and said

12
that only his trainin g in analytic number theory in the Landau tradition enabled him to complete the

study. For a survey of his work in Linear Algebra, see the Preface by his student, Walter Gautschi,

to Linear Algebra and Applications, Vols. 52-53, dedicated to him on his 90th birthday in 1983. The

first edition of his book [68] Solutions of Equations and Systems of Equations was based on his 1952

lectures at NBS.

As a public service John Curtiss initiated a Colloquium Series open to the public. A speaker

was free to choose his own topic. Many of the speakers were from INA. Mainly they described the

activities in which they were engaged. Other speakers were visitors of the Institute who spoke on

topics of their own interests. During this period the following visitors gave colloquium lectures:

A "Symposium on the Construction and Applications of Conformal Maps" and a "Symposiimi

on the Monte Carlo Method" were held at INA at the end of June 1949. Sandwiched between these

two symposia was a 2-day condensed course in automatic computation given under the direction of

Harry Huskey. The combined registration for all events totaled about 350 persons.

The purposes of the "Symposium on the Construction and Applications of Conformal Maps"

(June 22-25) were to consider physical applications of conformal maps and their generalization, and

to study the construction of conformal maps with a view to determining the possible applicability of

high-speed electronic computing machines in this direction. This symposium was arranged by a

conMoittee consisting of E. F. Beckenbach (chairman), C. Lanczos, A. Ostrowski, and W. Seidel.

The Proceedings of this Symposium was ediited by E. F. Beckenbach and was published as NBS

Applied Mathematics Series 18. The title page and table of contents are listed in Appendix D. The

program of the Symposium can be found by looking at this table of contents.

The "Symposium on the Monte Carlo Method" was jointly sponsored by the Rand Corporation and the Institute for Nimierical Analysis with the cooperation of the Oak Ridge National

Laboratory. The committee on arrangements consisted of J. H. Curtiss, H. H. Germond, A. S.

Householder, C. C. Hurd, and R. P. Peterson. The Symposium was held on June 29, 30, and July 1,

1949.

The Monte Carlo method can be described quite generally as the representation of a physical or mathematical system by a sampling operation whose expectation or variance gives the behavior of

the system under scrutiny. Thus, for example, the nimierical integration of partial differential equations of a certain type can be accomplished by building up a large sample of trials of certain

stochastic processes whose probability functions asymptotically satisfy the partial equations. In

certain physical situations formally represented by such equations, the physicist may prefer to place

primary emphasis on the stochastic processes and the associated sampling operations, which he will

regard as new mathematical models to be used in place of the continuous models of classical applied

mathematics. J. von Neumann and S. M. Ulam are credited as being the originators of Monte Carlo

methods. John Curtiss was a strong proponent of these methods and initiated a program of studies

of Monte Carlo methods at INA. In his IBM paper (see Appendix B) he discussed the distinguished

pedigree of Monte Carlo methods. Under his leadership, INA pioneered in certain applications of

Monte Carlo methods, especially in finding eigenvalues of SchrOdinger's equation. INA also was

concerned with developing methods for generating random nimibers. The generation of pseudorandom numbers with uniform distribution by number-theoretical methods was studied by D. H.

Lehmer and Olga Todd. Forsythe and others also considered the case of other distributions. Testing

of the quality of these pseudo-random numbers was done in collaboration with the Statistical Engineering Laboratory of the AMD. John Todd made various controlled experiments on the Monte

Carlo solution of the Dirichlet problem in various dimensions.

Unfortunately, today the expression "Monte Carlo Method" is loosely applied to any empirical

sampling solution of a probability or statistical problem, a much older and less sophisticated technique (see D. Teichroew [112]).

A. S. Besicovitch

C. F. Davis

J. J. Gilvary

Casimir Kuratowski

G. Brown

H. H. Germond

C. Hastings

Sir Harold Spencer Jones

13
The purposes of this Symposium were to interchange information concerning the usefulness of

the method, to stimulate discussions relative to its limitations, and to indicate directions in which

further theoretical research is needed. The program of the Symposium can be found in the Proceedings of this Symposium, published as NBS Applied Mathematics Series 12 in 1951, edited by A. S. Householder, G. E. Forsythe, and H. H. Germond. The title page and table of contents are reproduced in Appendix D.

We mention here the later (1956) Gainesville Symposiimi on Monte Carlo Methods to which

INA contributions were made by J. H. Curtiss, T. S. Motzkin, O. Taussky-Todd and J. Todd. It was

supported by the Air Force and the Proceedings [130] were published by Wiley.

Cooperation with other scientific organizations was a central part of administrative policy of

INA. In the Spring of 1949 a Seminar on Stochastic Processes was established jointly with the Rand

Corporation as part of the Monte Carlo project of INA. The seminar met regularly every Thursday,

alternating between the Institute building and the Rand building in Santa Monica. It was continued

during the sxmimer under the direction of W. Feller.

A second cooperative venture was a conference on establishing a west coast mathematical

journal. This conference was held on June 27, 1949. Some 50 or 60 west coast mathematicians were

invited to the conference by John Curtiss, the Acting Chief of the Institute. They represented all the

west coast imiversities, the Rand Corporation, and the National Bureau of Standards. A. W. Tucker,

of Princeton University, was chairman of the conference, and, in addition, served as an official

representative of the American Mathematical Society. A proposal of the National Bureau of Standards to establish a new journal under cooperative editorship, with publication by the Government

Printing Office, was placed before the group. After a lengthy discussion it was decided that a

journal should be established under the sponsorship of the universities on the west coast with the

cooperation of the Institute for Numerical Analysis. An exploratory committee was formed imder

the chairmanship of E. F. Beckenbach. This led to the establishment of the Pacific Journal of

Mathematics under the sponsorship of the west coast universities and INA. The headquarters of the

Pacific Journal was located at INA-UCLA. Beckenbach became its first Editor-in-Chief. He employed Elaine Barth to be his Administrative Assistant. Initially, she was in charge of preparing the

manuscripts for photo offset. At present she is still responsible for the preparation of all manuscripts

to be published in the Pacific Journal of Mathematics.

Another journal in which NBS took an important part was Mathematical Tables and other Aids

to Computation, originally established in 1943 by the National Research Council. In 1960 its name

was changed to Mathematics of Computation. Publication was taken over by the American Mathematical Society in part in 1962 and wholly in 1966. Among those connected with NBS-INA, who

served as editors, were E. W. Cannon, D. H. Lehmer, J. Todd, and C. B. Tompkins.

The NBS has its own Journal of Research and many important papers written by INA members

are published there. Curtiss insisted that this should not be a vehicle for publishing papers not

acceptable in outside journals. The only deviation from this policy was to cover cases of papers

involving tables and/or experimental computing.

The early tables of the Mathematical Tables Project were published by the Government Printing Office, beginning in 1939. A later series was published commercially by Columbia University

Press. In 1948 John Curtiss organized the Applied Mathematics Series. The series now includes 63

volumes, including tables, handbooks, symposium proceedings, and lecture notes. The volumes,

particularly relevant to NBS-INA, were 12, 15, 18, 29, 39, 42, 49, and 55. They are referred to in the

text and in the appendices. AMS 55 was particularly successful. It is a Handbook of Mathematical

Functions, edited by M. Abramowitz and I. A. Stegim and published in 1964 by the Government

Printing Office (later also by Dover and the 1972 edition was reprinted in 1984 by Wiley). G.

Blanch, U. Hochstrasser, and T. H. Southard of INA contributed chapters in this book. C. B.

Tompkins and J. Todd were on the National Research Council Committee, chaired by P. M. Morse,

which provided technical guidance on this volume.

It is appropriate to mention here the NBS-INA contributions to the Handbook ofPhysics which

was conceived by E. U. Condon when he was Director of NBS. Originally planned as an NBS

publication, it was actually published commercially by McGraw-Hill in 1964 (with a second edition

14
in 1967) with Hugh Odishaw as co-editor. The writing of Part I: Mathematics, was assigned to Olga

Taussky-Todd. She wrote chapters on Algebra, Ordinary Differential Equations, and Operators.

M. Abramowitz wrote on Integral Equations, F. L. Alt on Fundamentals, J. Todd on Analysis,

F. John on Partial Differential Equations, A. J. Hoffman on Geometry, E. U. Condon on Vector

Analysis, C. Lanczos on Tensor Calculus, C. B. Tompkins on Calculus of Variations, C. Eisenhart

and M. Zelen on Elements of Probability, and W. J. Youden on Statistical Design of Experiments.

In the second edition, the chapters on Operators and on Integral Equations were absorbed in a new

chapter by J. L. B. Cooper. A new chapter on Numerical Analysis by J. Todd was added.

As part of its educational program, the Institute xmdertook an experimental program for graduate students during Summer 1949 at the suggestion of H. F. Bohnenblust and Morgan Ward of

Caltech. The participants were:

Harold Gruen

Robert J. Diamond

Robert C. Douthitt

Ernest S. Elyash

Hans F. Weinberger

James G. C. Templeton

Lloyd K. Jackson

(UCLA)

(Caltech)

(UCB)

(Cornell)

(Carnegie Tech)

(Princeton)

(UCLA)

Special courses were arranged for these students.

15

CHAPTER IV

THE PERIOD SUMMER 1949 THROUGH SUMMER 1950

John Curtiss continued as Acting Director of INA during Summer 1949. Early in September,

J. Barkley Rosser became Director of the Institute. The members of the Senior Research Staff for

Summer 1949-Spring 1950 were:

J. Barkley Rosser (Director, Cornell)

Forman S. Acton (INA)

Lars V. Ahlfors (Harvard, Summer 1949)

Edwin F. Beckenbach (UCLA, Summer 1949)

Gertrude Blanch (INA)

Monroe D, Donsker (Cornell, Summer 1949)

Aryeh Dvoretzky (Hebrew University, Spring 1950)

William Feller (Cornell, Summer 1949)

George E. Forsythe (INA)

Samuel Herrick (UCLA, Summer 1949)

Magnus R. Hestenes (UCLA Representative)

Harry D. Huskey (INA)

Mark Kac (Cornell)

William Karush (Chicago)

Cornelius Lanczos (INA)

Alexander M. Ostrowski (University of Basel, Switzerland)

Otto SzSsz (Cincinnati)

Stephen E, Warschawski (Minnesota, Summer 1949)

Wolfgang R. Wasow (INA)

AJfredo Baiios and David S. Saxon of the UCLA Physics Department continued to serve as consultants at INA.

The members of the Jxmior Research Staff were:

George Gourrich

Lloyd K. Jackson

Raymond P. Peterson, Jr. Marvin L. Stein

Harold Gruen

Harold Luxenberg

Robert Sehnert

In Spring 1950 they were joined by Richard E. Cutkosky and Stuart L. Fletcher. Upon receiving his Ph. D., Raymond P. Peterson became a member of the Senior Staff.

The following visitors at INA gave Colloquium Lectures at INA during this period:

E. T. Benedict

R. P. Feynman

O. Helmer

R. Isaacs

S. Lefschetz

W. Prager

A. W. Tucker

M. Fekete

R. A. Fisher

J. O. Hirschfelder

I. Kaplansky

E. Penney

G. Szegd

17
Wolfgang Wasow became the third regular member of the research staff concerned primarily

with research. He was a specialist in ordinary and partial differential equations. He approached

these equations not only from the classical point of view but also through the use of Monte Carlo

methods. He became interested in Monte Carlo methods due to a suggestion made by John Curtiss.

(See [124-126].) Although he had no previous experience in numerical methods for solving differential equations, he learned quickly and soon became an expert in this field. Wasow received his

Ph. D. from New York University in 1942.

During Summer 1949 considerable progress was made on the problem of numerically constructing conformal maps and determining eigenvalues of linear operators. Ahlfors proposed two

numerical methods of constructing the mapping function which maps a polygon on a circle or

half-plane. Ostrowski weakened the restrictions on the applicability of the Theodorsen-Garrick

method of constructing conformal maps. Warschawski made a comprehensive review of results

dealing with the variation of the mapping function corresponding to deformation in the mapped

region. Beckenbach worked full time on the assembling and editing of the "Proceedings of the

Symposium on Conformal Mapping" [5].

The Monte Carlo method was demonstrated for the first time to be a useful tool in solving

eigenvalue problems. Mark Kac and M. D. Donsker developed a method for finding the lowest

eigenvalue and the corresponding eigenfunction of the SchrOdinger equation by the Monte Carlo

technique. Feller and Forsythe also participated in this program. Lanczos continued his studies on

his special methods for deading with eigenvalue problems. Hestenes, Karush, and Stein pursued the

problem of obtaining nimierical solutions of optimization problems. Herrick and SzSsz continued

the programs that they introduced earlier.

Considerable numerical experiments were made to test theoretical results even though the

high-speed computer was not yet available. P. L. Morton and D. H. Lehmer of Berkeley visited

INA and gave a favorable report of the progress made in the construction of the SWAC.

As remarked earlier, J. Barkley Rosser became the Director of INA in September 1949. Although his early training at the University of Florida was in physics, he received his Ph. D. at Princeton in 1934 in mathematical logic. He was a Proctor Fellow at Princeton, 1933-35, and a

National Research Council Fellow at Harvard, 1935-36. In 1936 he joined the Department of

Mathematics at Cornell University. During 1944-46 he was chief of the Theoretical Ballistics Section at Allegany Ballistics Laboratory in West Virginia. He served as a consultant for the Applied

Physics Laboratory, Johns Hopkins University, 1945-1963. Rosser was well versed in both pure and

applied mathematics. He was an expert in logic, analytic theory of numbers, infinite series, and

classical analysis. He had done a lot of computation (hand) in connection with Fermat's Last

Theorem and in connection with the tabulation of functions related to the error-function that turned

up in ballistics. Rosser was exceptionally well qualified to be the Director of INA. Although Rosser

served as Director of INA for only one year, his impact on INA was great. The programs he

initiated were basic and continued to be an important part of the research program at INA. As

noted below, he returned to INA for Summer 1951 at which time he participated in a symposium on

numerical algebra. In 1953 Rosser served on an evaluation committee of the National Bureau of

Standards (see Chapter VII). In addition he served on an advisory panel for NBS, 1962-1968.

During the period 1959-61, Rosser was the Director of the Conmiunications Research Division of

the Institute for Defense Analyses. He served as Chairman of the Mathematics Division of the

National Research Council, 1960-62, and was a member of the Conference Board of the Mathematical Sciences, 1962-64. In 1964-66, Rosser was a member of the Space Technology Panel of the

President's Science Advisory Committee. In 1963 Rosser left Cornell to accept the directorship of

the Mathematics Research Center at the University of Wisconsin, where he served with distinction.

He was always active in various professional organizations and served as president of the Society

for Industrial and Applied Mathematics (SIAM) and of the Association for Symbolic Logic.

During this period special emphasis was placed on iterative methods for solving systems of

linear equations and for finding eigenvalues of matrices. To this end, Rosser instituted a weekly

seminar concerned primarily with these topics. This seminar was attended principally by Rosser,

Forsythe, Hestenes, Karush, Lanczos, and Stein. Rosser and Forsythe were chiefly responsible for

18
the study of systems of linear equations. Forsythe, in particular, undertook the task of classifying the

various known methods for solving systems of linear equations. Hestenes, Karush, and Stein y/ere

responsible for the study of methods for finding eigenvalues of matrices with particular emphasis on

gradient methods, power methods, and inverse power methods. Lanczos continued the study of his

orthogonalization techniques for finding eigenvalues of matrices and for solving linear systems. He

also devised techniques for using Chebyshev and other orthogonal polynomials to obtain good

estimates of their solutions.

Of interest was a numerical experiment on finding the eigenvalues and eigenvectors of a symmetric matrix with close eigenvalues. The purpose was to test the methods devised by Lanczos and

by Hestenes and Karush. Rosser constructed an 8 x 8 syrmnetric matrix with close eigenvalues. Five

were bimched near 1000. Three of these were very close. The other two were equal. The negative

of one of these was also an eigenvalue. One of the two remaining eigenvalues was zero and the other

was very close to zero. Each eigenvector was computed to a 10-digit accuracy. The two methods

were equally successful. Time comparisons for the two methods could not be made because the

Lanczos method was carried out on a desk computer while the method of Hestenes and Karush was

carried out on an IBM CPC. Considerable time was spent in developing techniques for separating

close eigenvalues. After developing a suitable technique, it took about 11 hours to solve the problem

on the CPC by the gradient method developed by Hestenes and Karmh. At the present time these

computations could be carried out in a few minutes.

Another interesting computational episode occurred during this period. A group in industry

was having difficulties in inverting a certain 10 x 10 matrix. They asked INA for help. We too had

difficulties although our computations were carried out imder the supervision of G. Blanch, a

nationally recognized expert in Numerical Analysis. The computations were made on a desk calculator using a standard elimination method with pivoting. After spending too much time and money

on the project, Rosser inquired about the origin of this matrix. He found that it was a scaled

orthogonal matrix. Accordingly, its inverse could be obtained by a suitable rescaling of the transpose of the matrix, an operation that required very little computation. It should be pointed out that

even finding the inverse of an orthogonal matrix by an elimination procedure can require lots of

pivoting and that, on a desk calculator, pivoting can be a very time consuming operation- This

episode shows that matrix inversion is not a trivial matter even with a "good" matrix.

Research programs started previously were continued. Studies on the Monte Carlo method

were carried out by Forsythe, Dvoretzky, Kac, and Wasow. Improved methods for generating

random numbers were devised by Forsythe and Rosser. Lanczos devised a new method for inverting Laplace Transforms with applications to network analysis. Methods for improving rates of

convergence of series were considered by Rosser and Sz^z.

SUMMER 1950

As before, the research staff was enlarged during the Summer Session. The following re- searchers joined the Senior Staff for Summer 1950:

Paul Erdas

Richard P. Feynman

Fritz John

Edward J. McShane

WUliam E. Milne

Theodore S. Motzkin

Raymond P. Peterson, Jr. David S. Saxon

(Hungary)

(ComeU, IAS)

(New York)

(Virginia, IAS)

(Oregon)

(INA)

(INA)

(UCLA)

19
The new Graduate Fellows for this period were:

Harold P. Edmundson

William G. Hoffman

Robert K. Golden

In addition Lloyd Jackson participated in the program of Conformal Mapping. Upon obtaining

the Ph. D. he accepted a position at the University of Nebraska.

The following visitors of INA gave colloquium lectures during Summer 1950:

Feynman and McShane were supported by the Institute for Advanced Study. They were a

stimulus to the group.

David Saxon, a member of the UCLA Physics Department, was an important addition to the

staff. He strengthened our program on applications to physics. He also served as a consultant to the

mathematical research staff, thereby giving guidance in the choice of problems of interest to physicists. He later became President of the University of California.

The addition of Fritz John strengthened our program on numerical solutions of partial differential equations. When Rosser returned to Cornell in September, John became the Director of INA.

Theodore S. Motzkin was a new addition to the regular research staff at INA. He was a very

versatile and knowledgeable mathematician, who specialized in combinatorics, linear inequalities,

linear programming, and approximation theory. He started university study when not yet 16, and

studied at GOttingen and Berlin, and finally at Basel where he studied and collaborated with

A. OstrowskL His thesis, imder Ostrowski, finished in 1934, developed a theory of linear inequalities. He is credited with being one of the originators of the theory of linear inequalities and linear

programming. His first academic position was at Hebrew University in Jerusalem from 1935 to

1948. During World War II he was a cryptographer for the British Government in Palestine.

Motzkin came to the United States in 1948. After 2 years at Harvard and Boston College, he came

to NBS and UCLA. He collaborated with visitors and with other members of the research staff and,

in particular, with Forsythe and Wasow. He was a valuable addition to our research staff. An early

contribution of Motzkin deserves mention. We have already described Ostrowski's work on optimal

character of the Homer process for the evaluation of a polynomial p(x) for a single value of x. Suppose instead we want to evaluate/? (x) for many values of x. Is it possible to improve on the n

multiplications by a preliminary investment? Motzldn showed that, surprisingly, about n 12 multiplications would suffice. Essentially this depends on an algebraic identity which represents a polynomial of degree /z as a polynomial of degree of about n /2 in another variable. Motzkin also studied,

often in collaboration with J. L. Walsh of Harvard, a variety of problems in polynomial approximation. His Selected Papers have been published [67].

The U.S. Air Force, towards the end of World War II, set up a project "Scientific Computation of Optimal Programs" to reduce the time taken to "plan programs." Experimental calculations,

theoretic investigation and the specification and procurement of appropriate computers were in

progress at NBS Washington. Program Planning was the title of the INA Project (P&P April /June

1950) organized by Motzkin. The main activity at INA was concerned with Linear Programming

and methods to solve linear programs, including the Simplex Method. For historical accounts of this see the publications of George B. Dantzig, in particular his articles in AMS 15 and in [131].

M. Kac suggested to Olga Todd the study of pairs of matrices A, B with the L-property, that

is, pairs A, B such that the characteristic roots of a A+ feB are a Xj + p,;, where {Xj and { |Xi} are the

characteristic roots of A and B in some order, independent of a and b . Pairs of matrices of this type

had arisen in a statistical context connected with a theorem discovered by C. C. Craig and studied

by H. Hotelling of which Olga Todd [107] gave a new proof, using the L-property. A fruitful

A. A. Albert

J. Charney

J. C. P. Miller

G. R. Boulanger

L. Fox

A. Weinstein

20
collaboration between Olga Todd and Motzkin developed [66]. One result was a new basic theorem

in perturbation theory: "If every aA+bB is diagonalizable for all ratios a lb (including «) except

perhaps for one value, then the pair A, B has the L-property." "his result is the best possible.

Further if there is no exceptional ratio, then A and B commute. Another proof of this theorem was

given by Tosio Kato in his books [44,45] on Perturbation Theory for Linear Operators. H. Wielandt

also investigated the L-property.

Related to this work is the characterization of normal matrices by A. J. Hoffman and Olga

Todd [34]. Various problems connected with commutativity and its generalizations were studied by

Olga Todd and Kato [48].

Perhaps the most important event that occurred during this period, was the dedication, on

August 17, 1950, of the SWAC, the National Bureau of Standards Western Automatic Computer.

This machine was sponsored by the Office of Air Research of the Air Material Command, USAF.

It was designed and built by Harry Huskey and his staff at INA. The speakers at the dedication

ceremony were E. U. Condon, Director of NBS; Colonel F. S. Seller, Chief of the Office of Air

Research, Department of the Air Force; L. N. Ridenour, Dean of the Graduate School of the

University of Illinois; J. H. Curtiss, Chief of the National Applied Mathematics Laboratories; and H.

D. Huskey, Chief of the INA Machine Development Unit. The dedication was followed, on August

18, by a symposiimi on applications of digital computing machines to scientific problems. A description of this symposium is given in Appendix D.

At the time of the dedication of the SWAC the members of the Machine Development Unit

were:

These and other temporary personnel were responsible for the building of the SWAC. Among those

who made important contributions later were Ragnar Thorensen and E. W. Cannon, who was

Assistant Chief of AMD for computer development and was in residence at INA during a critical

period in the development of SWAC.

The research program for Summer 1950 was, in the main, a continuation of the programs

started earlier. R. Fortet of the University of Paris joined Dvoretzky, Kac, and Wasow in the study

of Monte Carlo methods for solving certain partial differential equations. Milne resimied work on

his monograph on numerical solutions of differential equations. About one-third of the book was

devoted to the numerical solution of partial differential equations. Forsythe, Hestenes, Karush,

Lanczos, Rosser, and Stein continued their studies on finding eigenvalues and solving linear systems. In particular, Rosser developed a method for the exact solution of linear systems with integral

coefficients. Rosser also studied the acceleration of the convergence of slowly converging series.

Harry D. Huskey

Edward L. Lacey

Roselyn S. Lipkis

Brent H. Alford

Arnold Dolmatz

Blanche C. Eidem

Sidney S. Green

Biagio F. Ambrosio

David F. Rutland

Harold Luxenberg

Harry T. Larson

Michael J. Markakis

John L. Newberger

James W. Walsh

21

CHAPTER V

THE PERIOD FALL 1950 THROUGH SPRING 1951

Fritz John became the Director of INA in September 1950. Fritz John received his Ph. D. at the University of GOttingen in 1933, specializing in the Calculus of Variations. He was a Research

Scholar at Cambridge University, 1934-35. John was a member of the Department of Mathematics

at the University of Kentucky, 1935-42. During World War II, he served as a Mathematician at the

U.S. War Department. In 1946 he joined the staff at the Courant Institute, New York University.

Fritz John is an expert on Partial Differential Equations, Nonlinear Elasticity, Analysis, and Geometry. His wide background enabled him to make significant contributions to our program on

Differential Equations at INA. John served as Director of INA for 1 year. Upon returning to the

Courant Institute, he continued to make significant contributions and received many honors, including the George David Birkhoff Prize in Applied Mathematics and membership in the National

Academy of Science. He gave the Gibbs Lecture of the American Mathematical Society in 1975.

See John [40]. His collected papers have, been published by Birkhauser, Boston in 1985.

Senior Research Personnel for this period were:

Fritz John (Director, NYU)

Forman S. Acton (INA)

Gertrude Blanch (INA)

Milton Dandrell (UCLA)

Aryeh Dvoretzky (Hebrew University)

George E. Forsythe (INA)

Robert Fortet (Paris, France)

John W. Green (UCLA)

Magnus R. Hestenes (UCLA Liaison)

Harry D. Huskey (INA)

William Kanish (Chicago)

Cornelius Lanczos (INA)

Hans Lewy (UC, Berkeley)

William E. Milne (Oregon)

Theodore S. Motzkin (INA)

Lowell J. Paige (UCLA)

David S. Saxon (UCLA)

Wolfgang R. Wasow (INA)

Members of the Junior Research Staff were:

Harold P. Edmimdson

Harold Gruen

William C. Hoffman

James P. Wesley

Robert K. Golden

Robert M. Hayes

Marvin L. Stein

When Stein received his Ph. D. in January he became a member of the senior research staff. In

his Ph. D. thesis. Stein justified the use of gradient methods for variational problems and for self

adjoint boundary value problems.

23
Colloquium lectures were given by the following visitors of INA:

N. Aronszajn

E. Gerjuoy

G. Polya

J. J. Stoker

A. Erdelyi

K. Knopp

S. Sherman

D. van Dantzig

Dvoretzky, Fortet, Karush, and Lewy returned to their universities at the end of Fall 1950.

Hestenes served only in a limited capacity. He was in charge of the contract that NBS had with

UCLA. Paige, an algebraist from UCLA, joined the group in January.

The weekly seminar initiated by Rosser and now led by John turned its attention to the study

of numerical methods for solving partial differential equations. Fritz John discussed the conver- gence of solutions of a finite-difference equation which approximated a parabolic partial differential

equation with some non-linear terms, as the mesh size tended to zero while keeping a fixed "shape."

As was often the case at INA, numerical studies were made in parallel to the theoretical ones; in

particular in this connection G. Blanch completed a substantial contribution. Monte Carlo methods

continued to be pursued by Dvoretzky, Fortet, and Wasow. More classical approaches were emphasized by John, Green, Lewy, and Milne. Forsythe participated in both programs. He also made

numerical experiments on various schemes for solving systems of linear equations. He foimd that, in

ill-conditioned cases, the optimal gradient method tended to "bog down." This suggested that an

acceleration method was needed. Forsythe and Motzkin suggested one acceleration scheme and

Hestenes suggested a simpler one. Considerable improvements were gained in the gradient method

and in other iterative methods by the use of these acceleration schemes. These experiments eventually led to the formulation, by Hestenes, of the conjugate gradient method, a method which will be

discussed later. M. L. Stein foimd that acceleration schemes were useful also in calculating eigenval- ues of matrices. Experiments also showed that high precision arithmetic was needed for the preservation of significant figures in the matrix computations.

Research on the theory of Program Planning expanded considerably due to the activities of

Motzkin, who invented some promising new methods for solving systems of inequalities, and

through a special seminar sponsored jointly by INA and Rand.

So far we have been concerned mainly with the activities of the research staff. From the

beginning, the computational unit under Blanch and Yowell was actively engaged in solving computational problems for the research staff, for various governmental agencies, and for contractors of

the Federal Government. In addition the computational facilities of INA were made available to

departments of west coast imiversities. In particular, the Departments of Astronomy, Chemistry,

Geophysics, Meteorology, Physics, and Psychology at UCLA made full use of these facilities. They

were among the first to carry out research in which machine computations played a significant role.

By 1951 the demands for computational assistance were so great that it was difficult for the Computational Unit to fulfill its obligations. Accordingly, a new unit of INA, called the Mathematical

Services Unit, was formed imder the supervision of Harry Huskey. It was funded by the United

States Air Force. One of the piuposes of this unit was to encourage Federal Government contractors to learn how to use electronic computers. Accordingly, computational services using SWAC

were made available to them. Many of these contractors made use of this service. Effectively, the

NBS offer to these contractors was to augment their contracts by providing free computational

services of a type which was not as yet available elsewhere. It is interesting to note that when IBM

announced that they would build a "Defense Calculator" if they could get 12 orders, 6 of the 12

orders came from INA's computer customers. The "Defense Calculator" became the IBM 701 —

their entry into the "Electronic Computer Age."

At the end of June 1951 the members of the Mathematical Services Unit were:

Arnold D. Hestenes (Head)

Marvin Howard

Harold Luxenberg

Thomas H. Southard

Frederick H. Hollander

Roselyn S. Lipkis

Robert R. Reynolds

Everett C. Yowell

24
As noted earlier, this group formed an effective liaison between INA and various government

agencies and government contractors.

A. D. Hestenes was the chief liaison officer. He was charged with the administration of the

funds assigned to the Services Unit. He was responsible for "educating" various government agencies and contractors of these agencies on the facilities available to them at INA and on how

electronic computing machines could be used to solve some of their problems.

Initially, Southard was on a sabbatical leave from Wayne State University. He found the

activities at INA so challenging that he decided to accept a regular position at INA. Primarily, his

duties were to do the "numerical" analysis part of the large variety of problems submitted to INA

for solution. He was actively engaged in the educational fimctions of INA and, from time to time,

taught numerical analysis at UCLA and at UCLA-Extension. Southard was interested in getting

people together and organized social activities for members of INA and friends. Later he helped

organize a National Meeting of the Association for Computing Machinery. He became involved in

organizing the first chapter of SIAM (the Society for Industrial and Applied Mathematics) west of

the Mississippi, and subsequently served as National Vice President (1954-6) and National President

(1956-8).

Reynolds also did the "numerical analysis" part of problems to be solved at INA. In addition he

wrote an extensive research paper on the Numerical Integration of the Rolling Pullout Equations

for an Airplane.

Throughout the existence of INA, Yowell was in charge of machine computations and was a

principal advisor to users of machines. He also wrote several research papers jointly with members

of the research group.

At the request of the Office of Air Research, a 3-month training program for 18 Air Force

Cadets was initiated. The purpose was to train USAF airmen in computing methods, the logical

design of high-speed digital computers, and in coding and programming techniques for the SWAC,

IBM, and hand machines. This course was taught by Acton, Reynolds, Milne, Luxenberg, Lipids,

and Yowell. This was part of the educational program sponsored by INA. From time to time

members of the staff at INA gave courses in numerical analysis and in computational techniques at UCLA and UCLA-Extension. Graduate courses and seminars in fields of their own interests were

given by:

J. H. Curtiss P. ErdOs

W. Feller R. P. Feymnan

G. E. Forsythe H. D. Huskey

F. John M. Kac

C. Lanczos T. S. Motzkin

A. Ostrowski H. A. Rademacher

J. B. Rosser D. S. Saxon

I. J. Schoenberg C. B. Tompkins

J. van der Corput W. R. Wasow

The Mathematical Services section was responsible for a very large variety of projects. This is evidenced by the following sample of the projects which were on their "books" in the Quarterly

Report, April-June 1951. Not all of these projects were active at the same time. Many other

projects were attacked later.

Computing services for the research staff

The determination of the periods and amplitudes of the light variations of certain stars Nimierical studies of a non-linear parabolic differential equation

Mathieu Functions II Special tables of Bessel functions

Tables for rocket and comet orbits

Punched card library

Determination of orbits of comets, minor planets, and satellites

Computation relating to air flight design

25
Analysis of circular shell-supported frames

Meteorological means

Earth tides

Evaporation computations

Boundary layer

Rolling pullout equations of motion

Raydist data analysis

Equations of pressure systems

Range error computation

Computations in connection with program analysis

Statistical smoothing rocket grain burning

Equations of combustion

Solution of sets of algebraic equations

Conversion of hexadecimal nimibers

Airplane windshield deicing and defogging

Computations in connection with lattice arrangements

Tables of the bivariate normal distribution function

Pressure fields of potential flow past a body of revolution

Simplified rolling pullout equations

Frequency response study

Low moments of normal order statistics

Three non-linear differential equations

Probability distribution of Kolmogorov statistic

Reduction of hydrographic data

26
CHAPTER VI

THE PERIOD SUMMER 1951 THROUGH SPRING 1952

During Summer 1951 the research group was unusually large, in part, because a symposium

was held late in August. Fritz John returned to New York University and Derrick Lehmer became

the new Director. Rosser returned for the summer to participate in the program concerned with

methods for solving linear equations and finding eigenvalues of matrices. Senior Research Personnel

for the period July 1951 to June 1952 were:

Derrick H. Lehmer (Director, UC Berkeley)

Forman S. Acton

Shmuel Agmon (Hebrew University, Summer 1951)

Lioman Bers fNYU. Summer 195 1") Gertrude Blanch (INA)

Leonard M. Blumenthal fMissouri^

B. Vivian Bowden (England, Fall 1951)

Alfred T. Brauer (North Carolina, Summer 1951)

William E. Bull (UCLA. Summer 1951")

Paul S. Dwyer (Michigan, Summer 1951)

Gaetano Fichera (Italy, Summer 1951)

Donald E. Fogelquist (Sweden, Summer 1951)

George E. Forsythe QNA)

Jerry W. Gaddum (Missouri)

Herman H. Goldstine (Institute for Advanced Study,

Summer 1951)

Magnus R. Hestenes (UCLA Liaison)

Gilbert A. Hunt (Cornell, Fall 1951)

Harry D. Huskey (INA)

William Karush (Chicago, Summer 1951)

Tom Kilbum (England, Fall 1951)

Cornelius Lanczos (INA)

Theodore S. Motzkin (INA)

Francis J. Murray (Columbia)

Victor A. Oswald, Jr. (UCLA)

Lowell J. Paige (UCLA)

J. Barkley Rosser (Cornell, Summer 1951)

David S. Saxon (UCLA)

Isaac J. Schoenberg (Pennsylvania)

Eduard L. Stiefel (ETH, Switzerland)

Marvin L. Stein (INA)

Philip Stein (South Africa, Summer 1951)

J. G. van der Corput (Holland, Summer 1951)

Joseph L. Walsh (Harvard, Summer 1951)

Wolfgang R. Wasow (INA)

Alexander Weinstein (Maryland, Summer 1951)

Maurice V. Wilkes (England, Fall 1951)

27
During Summer 1951 the special Junior Researchers and Graduate Fellows were:

Donald G. Aronson

John H. Gay

Harold P. Edmimdson

Robert K. Golden

Kenneth E. Iverson

Richard H. Lawson

William H. Warner

Mollie Z. Wirtschafer

Robert M. Hayes

Thomas E. Kurtz

Theodore D. Schultz

James P. Wesley

Gloria Zinderman

During Fall 1951 through Spring 1952, the Junior Researchers were:

Charles E. Africa, Jr. Daniel B. Ray

Richard H. Lawson

James A. Ward

and the Graduate Fellows were:

Richard G. Cornell

Robert M. Hayes

Michael J. Moracsik

Anthony Ralston

Stephen G. Gasiorowicz

Urs W. Hochstrasser

Thomas Neill, Jr.

Hochstrasser was a student from Switzerland who received his Ph. D. under the direction of Stiefel.

In recent years Hochstrasser has been the Director of the Swiss Atomic Energy Commission. It is interesting to note that Iverson is now known as the originator of the programming language APL.

Ralston is now a leading computer scientist.

Speakers at the INA Colloquium during Summer 1951 were:

During the period Fall 1951 to Spring 1952, we were fortunate in having the following visitors as colloquium speakers:

Derrick H. Lehmer was bom in Berkeley, CA. His father was Professor of Mathematics at UC-Berkeley, specializing in Number Theory. After undergraduate studies at UC-Berkeley and

graduate studies at Brown, he was a National Research Fellow at Caltech. After a short stay at Lehigh, he became Professor of Mathematics at UCB in 1940. One of the high points of his career was his selection by the American Mathematical Society to be the Gibbs Lecturer in 1965 [56]. In

1928 he married Emma Trotskaia. She, too, is a distinguished number theorist.

One of his early interests was in a mechanical number sieve operated photoelectrically. He later

programmed sieves for general purpose machines and then built a special purpose electronic number

sieve.

Lehmer had been involved in the dramatic investigation of the partition function by Hardy,

Ramanujan, MacMahon, and later by Rademacher. Hardy and Ramanujan obtained an asymptotic

I. J. Schoenberg

L. Bers

J. B. Rosser

E. Stiefel

J. G. van der Corput

J. H. Curtiss

J. L. Walsh

G. Fichera

F. J. Murray

L. M. Blumenthal

S. P. Frankel

W. T. Reid

A. J. Hoffman

Y. L. Luke

I. J. Schoenberg

R. Isaacs

28
formula for p (n ), the number of partitions of n . For n = 200, p (n ) is a number with 13 digits, which

was computed by MacMahon. It turned out that 8 terms of the asymptotic series gavep{n) with an

error of 0.004. A refinement of the asymptotic formula used by Lehmer gave p(121) as an integer

(with 27 digits) +0.0041 suggesting strongly the exact value ofp(721). What Rademacher did was

to obtain an identity for p(n).

It was clear that sooner or later Lehmer would become the Director of INA. The "Oath

Situation" at UC and the availability of the SWAC were enough to attract the Lehmers to INA.

Too few computers are designed for or by number theoreticians. The exploitation of the usual

machines for number theory— even simple factorization routines— forces the user to develop programming tricks which are often of general value.

At INA Lehmer was concerned with the Ramanujan function t(«), with the Riemann Hypothesis, and with Fermat's Last Theorem, the latter in collaboration with H. S. Vandiver. Earlier some

work related to Fermat's Last Theorem had been carried out on the SEAC. Following a suggestion

by H. Hasse, Olga Todd encouraged J. C. P. Miller to make experimental computations on

consecutive pth power residues and on p itself as a pth power residue. This work was described in

the New York Symposium [129]. Several later papers by the Lehmers are concerned with this

problem.

As will be noted later some of the work on the SWAC was reported by Emma Lehmer at the

Santa Monica Symposium [16].

Lehmer's selected papers have been published in three volumes [56].

An important addition to the Machine Development Unit was the appointment of Ragnar

ThorensoiL He was in charge of adding a drum to the SWAC, thereby significantly enlarging the

memory of the SWAC.

Bull and Oswald were UCLA linguists who joined Harry Huskey in seeking methods for

language translation by use of computers. They were pioneers in this field. Some of their results are given in [69]. There was also work on a Russian translation by Ida Rhodes of the Applied Mathematics Division in Washington.

Research in the Mathematical Theory of Program Planning was carried out enthusiastically by

Motzkin, Agmon, Blumenthal, Gaddum, Schoenberg, and Walsh. During July and August a joint

seminar was held with Rand on "Linear inequalities and related topics." Invited speakers from

outside were: A. W. Tucker, R. W. Shepherd, J. M. Danskin, S. Karlin, and R. E. Bellman.

Studies in numerical integration of ordinary and partial differential equations were pursued

vigorously by Agmon, Bers, Fichera, and Wasow. F. John completed Part I of an important memoir "On the integration of parabolic equations by difference methods" [40]. Johannes G. van der

Corput continued his extensive research on methods of asymptotic expansions. Rosser investigated

the problem of computing low moments of normal order statistics. He also participated in the

program of finding effective methods for solving a system of linear equations.

Late in June or early in July, M. Hestenes devised a Conjugate Gradient Method for solving a system of linear equations. It is an n-step iterative routine. He gave three versions of this routine.

When Stiefel arrived at INA from Switzerland, the librarian gave him a paper describing this

routine. Shortly thereafter Stiefel came to Hestenes' office with this paper in hand and said, "Look!

This is my talk." It ttimed out that Stiefel had invented the same algorithm from a different point of

view. He looked upon it as a relaxation routine whereas Hestenes viewed it as a gradient routine on

conjugate subspaces. The term "conjugate gradient" was coined by Hestenes. Because they had

devised the same routine independently at about the same time, Stiefel and Hestenes decided to

write a joint paper describing the routine and its properties. Hestenes arranged it so that Stiefel

could remain at INA and UCLA for at least a 6-month period during which time the joint paper

would be written (see [30]). During this period Stiefel taught a graduate course at UCLA on the

Theory of Relativity.

Eduard L. Stiefel was a very versatile mathematician. He began his career as a topologist and

made notable contributions in this field. One of these was used by Olga Todd in her proof of the

impossibility of deriving the Laplace equation from the Cauchy-Riemann equations in dimensions

other than 1, 2, 4, or 8. While at INA, Stiefel gave a proof of her result using representation theory

.

29
In 1943 Stiefel was appointed to a professorship at ETH (Eidgenossischen Technischen Hochschule) in Zurich, Switzerland. He was well versed both in pure and in applied mathematics. He

anticipated the coming of high-speed digital computers and was instrumental in the development of

such a computer at ETH. This led him to an intensive study of numerical methods. His group at ETH became leaders in this field. They developed effective routines for solving large-scale systems

of linear equations and for finding eigenvalues. Stiefel wrote an outstanding introductory book [96]

on numerical analysis which was translated into several languages. Stiefel also made notable contributions to celestial mechanics (see, e.g., his book [97] with G. Scheifele). In addition, with the

collaboration of his students, he wrote two other books [98,99]. Stiefel received many honors and

participated in the city government of Zurich.

The conjugate gradient algorithm has, as a subroutine, an algorithm which is equivalent to an

orthogonalization routine developed by Lanczos. The conjugate gradient routine therefore can be

derived from the results given by Lanczos. Using his orthogonalization routine, Lanczos devised a "Method of Minimized Iterations" for solving a system of linear equations. This method is a variant

of the original conjugate gradient routine. Credit should also be given to Rosser, Forsythe, Karush,

and Paige for the development of the conjugate gradient routine, because the routine was also an outgrowth of their efforts. Rosser and Stiefel presented the conjugate gradient method at the August 23-25, 1951 Symposium on Simultaneous Linear Equations and the Determination of Eigenvalues, which was a part of the Semicentennial Celebration of NBS.

The technical organization of the symposium was mainly the responsibility of Olga TausskyTodd, who, with assistance of L. J. Paige, edited the proceedings of the meeting. The proceedings

were published by the National Bxireau of Standards as Applied Mathematics Series 29. The symposium consisted of invited reports from various mathematicians, on selected topics, followed by a

roundtable discussion. The reports on simultaneous linear equations were confined to finite systems,

while those on the determination of eigenvalues dealt with both the discrete and continuous cases, and included reports specifically on the determination of boimds for eigenvalues. This symposium is considered by many to be the forenmner of the famous Gatlinburg Symposia. It was followed by

one organized by Wallace Givens at Wayne State University, Detroit, in 1957 and then the Gatlinburg Series began, initially organized by A. S. Householder. They took place in 1961, 1963, 1966,

and 1969. Later meetings took place at Los Alamos (NM) (1972), Bavaria (1976), Asilomar (CA)

(1977), Oxford (England) (1981), and Waterloo (Canada) (1984). In his paper [35] Householder has

written of . . the tremendous influence of INA in the early years of the development of modem

numerical analysis. It would be hard to exaggerate this influence, and it would be harder to say,

how much further along the subject might be now if the Institute had not been brutally cut off in its prime."

The papers presented at the 1951 conference are listed in Appendix D in the table of contents

of AMS 29.

In this symposium the contributions made at INA were described by Forsythe, Hestenes, and

Rosser. Forsythe presented a classification (and bibliography) of known methods for solving linear

equations. Rosser described the contributions in this area by the staff at INA. Hestenes discussed

gradient methods and Lanczos' method for finding eigenvalues and eigenvectors of matrices. The

bibliography by Forsythe extended work begun by Ostrowski and the Todds. Others at INA,

particularly Motzkin, collaborated in it. In a supplementary paper of the conference, Wallace

Givens introduced his tridiagonalization method for finding eigenvalues and eigenvectors of symmetric matrices, a method which proved to be very useful and which became very popular.

There is a second set of important papers closely related to the papers described above, written

imder the sponsorship of NBS. These papers were published in NBS Applied Mathematics Series 39

in 1954, edited by Olga Taussky-Todd. The book is entitled "Contributions to the Solutions of

Systems of Linear Equations and the Determination of Eigenvalues." Its contents are listed in

Appendix D. A. I. and G. E. Forsythe described numerical experiments with accelerated gradient

methods for solving linear equations. R. M. Hayes gave iterative methods for solving linear problems on a Hilbert space. J. Todd described the condition of finite segments of the Hilbert matrix.

30
This volume was followed by AMS 49, the title page and contents of which are listed in

Appendix D. Two of the three papers in this volume were written by E. Stiefel and P. K. Henrici

who were associated with INA.

As remarked above, Stiefel was given a joint appointment with INA and UCLA so that he and

Hestenes could write an extensive exposition on the Conjugate Gradient Method for solving systems of linear equations. Several versions of the method were given. Numerical experiments were

carried out by R. M. Hayes, Urs Hochstrasser, M. L. Stein, and W. Wilson. Wilson was a member

of the computational staff. Unfavorable as well as favorable situations were considered. Even singular systems were studied. In each case the numerical results were consistent with theoretical ones when the effects of roundoff errors were taken into accoimt. During this period Lanczos wrote up

his version of this routine which he called a method of "Minimized Iterations." In addition he

devised a special iteration for solving large systems of linear equations. This iteration was based on

properties of Chebyshev polynomials. Variations of standard gradient methods with or without

accelerations were studied experimentally and theoretically by Forsythe and M. L. Stein. Lehmer

gave simple explicit expressions for the inverse, the characteristic polynomial, and any positive

power of certain character matrices.

R. M. Hayes and M. L. Stein studied gradient and Rayleigh-Ritz methods for variational

problems. In particular, Hayes was concerned with quadratic variational problems. Using techniques developed by Hestenes, he studied them in a Hilbert Space setting. Accordingly he developed a large class of iterative processes for solving linear self adjoint elliptic boundary value

problems. One of these was a generalization of the conjugate gradient routine of Stiefel and Hestenes. In each case he established rates of convergence. His results were published in AMS 39 described earlier.

Forsythe initiated a study of Russian mathematical progress which led to the publication of a

bibliography of Russian mathematics books [24]. Pertinent articles by Russians were collected and

selected ones were translated into English by C. D. Benster under the editorships of Forsythe and

Blanch. Some translations were published commercially [20,42]. Several appeared as NBS Reports

[49,59,90]. An informal, but important, result of this program was the initiation of a class in Russian

for mathematicians at UCLA and INA.

Studies in the mathematical theory of program pl anning and linear inequalities were continued

vigorously by Motzkin, Schoenberg, Blximenthal, and Gaddum. A systematic treatment of the entire

field of linear inequalities and their applications was imdertaken in a graduate course by Motzkin at UCLA. Blimienthal continued his studies on metric methods in abstract algebra. Schoenberg pursued his theory of splines, a theory that has many useful applications.

Lehmer developed a practical method for obtaining the so-called Kloosterman Sums and investigated their properties. A series of tests for primality of Mersenne numbers were made on the

SWAC, using a code sent in by R. M. Robinson of UC-Berkeley. Robinson wrote the Mersenne

Code with a minimum of guidance. That the code was without error was (and still is) a remarkable

feat. These tests showed a high degree of reliability in SWAC operation, and gave some imexpected

information on the distribution of primes of the form 2"* — 1. It was with a great deal of excitement

that two new primes were discovered, namely, those for which p = 521 and p = 607. These were by

far the largest known primes to this date. Later it was shown that p = 1279 also yields a new larger

prime. In addition, Lehmer and Golden made a numerical study of the periods in the continued

fraction expansion of square roots of integers.

Studies in theoretical physics were carried out by Saxon in cooperation with members of the

Physics Department and other departments at UCLA. The topics considered are illustrated by the

following titles of papers that they produced: "Modes of vibration of a suspended chain"; "Distribution of electrical conduction currents in the vicinity of thunder storms"; "Radiation characteristics

of a turnstile antenna shielded by a metallic tube at one end"; "An optical model for nucleon-nuclei

scattering"; "Variational calculation of scattering cross-section"; and many others. It is interesting

to note that the paper, "Modes of vibration of a suspended chain," answered a query by E. U.

Condon, the Director of the National Bureau of Standards, who had noted that the traditional

solution required the end supports to restrain infinite pulling forces.

31

CHAPTER VII

THE PERIOD SUMMER 1952 THROUGH SPRING 1953

As usual there was increased activity in research during the summer period. Senior Research

Personnel and seminar participants for the period Summer 1953 through Spring 1953 were:

Derrick H. Lehmer

Forman S. Acton

Adrian A. Albert

Edward W, Barankin

Gertrude Blanch

Leonard M. Blumenthal

Truman A. Botts

Johannes G. van der Corput

George E. Forsythe

Jerry W. Gaddum

Dick W. Hall

E. H. Hanson

G. A. Hedlund

Magnus R. Hestenes

Gilbert A. Hunt

Harry D. Huskey

Mark Kac

Irving Kaplansky

William Karush

Cornelius Lanczos

Richard A. Leibler

Harold Levine

Arvid T. Lonseth

Theodore S. Motzkin

Victor A. Oswald Jr. Lowell J. Paige

David S. Saxon

Isaac J. Schoenberg

Julian Schwinger

Dan Teichroew

Charles B. Tompkins

Joseph L. Walsh

James A. Ward

Wolfgang R. Wasow

Charles Wexler

J. Wolfowitz

(Director, UC Berkeley)

(INA)

(Chicago, Summer 1952)

(Berkeley)

(INA)

(Missouri, Summer 1952)

(Virginia, Summer 1952)

(Holland, Summer 1952)

(INA)

(Missouri, Summer 1952)

(Maryland)

(Texas State, Summer 1952)

(Yale, Summer 1952)

(UCLA Liaison)

(Cornell, Summer 1952)

(INA)

(Cornell, Summer 1952)

(Chicago, Summer 1952)

(Chicago, Simmier 1952)

(INA)

(Washington, DC, Summer 1952)

(Harvard, Summer 1952)

(Oregon State, Summer 1952)

(INA)

(UCLA, Summer 1952)

(UCLA)

(UCLA)

(Pennsylvania, Summer 1952)

(Harvard, Summer 1952)

(INA)

(INA)

(Harvard)

(Utah, Summer 1952)

(INA)

(Arizona State, Summer 1952)

(ComeU)

Junior Researchers and Graduate Fellows were:

John W. Addison, Jr. Fred Baskin

Walter I. Futterman

Charles E. Africa Jr. Richard G. Cornell

Stephen G. Gasiorowicz

33
Robert M. Hayes

Richard A. Lawson

Michael J. Moravcsik

Urs W. Hochstrasser

Stanley W. Mayer

Thomas Neill

Daniel B. Ray

Marion I. Walter

Anthony Ralston

John Selfridge

Roger D. Woods

Lectures in the Numerical Analysis Colloquium Series were given by the following visitors:

Robert Hayes received his Ph. D. upon writing a thesis imder the direction of M. R. Hestenes.

His thesis was concerned with iterative methods for solving linear problems in Hilbert Space [27].

During his stay at INA he became interested in information storage and retrieval. This eventually

led him to the use of computers in libraries. He became a pioneer in this field and is now the Dean

of the Library School at UCLA.

During Summer 1952 a Seminar on Finite Projective Planes was held under the supervision of

C. B. Tompkins. The principal participants in this seminar were Albert, Botts, Hall, Hanson, Hedlund, Himt, Kaplansky, Leibler, Paige, Tompkins, Ward, and Wexler. Lectiires were given by C. B.

Tompkins, A. M. Gleason, J. A. Ward, L. K. Frazier, D. W. Hall, L. J. Paige, D. H. Lehmer, A. A.

Albert, and M. Hall, Jr. This seminar was headed by S. S. Cairns of the University of Illinois. It was

a forenmner of the SCAMP Project which was set up during the following year. Related work was

carried out jointly by A. J. Hoffman, M. Newman, E. Straus, and Olga Taussky-Todd. Olga

Taussky-Todd discussed topics in this area in her 1952 lecture in Palermo (see J. Research NBS 65

(1961), 15-17) using work of G. Pall and M. Newman.

Lehmer introduced a new project designed to investigate the logical theory of digital computing machines, particularly of automatic type, and to discover new techniques for using such machines. The projects introduced earlier were continued in the usual manner. Barankin joined

Motzkin in his project on linear inequalities. Schwinger gave a series of lectures on qiiantum dynamics. Lanczos devised four routines for using Chebyshev polynomials in solving large-scale systems

of linear equations. Lonseth continued his research on the numerical solution of integral equations

by an adaptation of least square techniques. Southard and Yowell devised an alternative "predictorcorrector" process for the numerical integration of ordinary differential equations with initial conditions. A new Mersenne prime was found with /7 = 2203. Numerical experimentation on solving

linear equations and finding eigenvalues was continued on the SWAC.

Acton, Huskey, and Lanczos were on leave of absence during fiscal year 1953. During his leave

of absence, Huskey was employed by Wayne University for the purpose of setting up a computation

laboratory. As usual, Hestenes and Paige participated on a part-time basis.

C. B. Tompkins became a regular member of the INA staff. He was a very versatile mathematician who was well versed in applications of mathematics and in the engineering aspects of computing machines. He received his M.A. degree at the University of Maryland in chemistry, physics, and

mathematics. He took his Ph. D. at the University of Michigan in 1935. After a year as an instructor

at Maryland, he spent 2 years as a National Research Fellow at Princeton University and the

Institute for Advanced Study, followed by 3 years as instructor at Princeton. Diu-ing these years

Tompkins collaborated with Marston Morse in his work on the Calculus of Variations in the Large.

He also held a Visiting Lectureship at the University of Wisconsin. During World War II Tompkins

A. M. Ostrowski

G. A. Hunt

D. Ray

J. G. van der Corput

E. W. Barankin

A. Dvoretzky

M. M. Schiffer

R. De Vogelaere

A. T. Lonseth

M. Kac

H. Levine

G. P61ya

P. ErdcJs

J. Wolfowitz

M. Minorsky

S. Bochner

34
served as a naval communications officer in the Pacific, rising to the rank of Lieutenant Commander. His experiences there impressed upon him the need for the development of high-speed computers. In 1946, he became one of the founders of Engineering Research Associates in St. Paul, MN, a

firm which played an important role in the development of the United States computer industry. In

1949 he organized, on behalf of the Office of Naval Research, the Logistics Research Project at George Washington University. He served as consultant and member of advisory panels to various

governmental agencies. In this connection, he played a major role in the development Project

SCAMP which we refer to later. This project later became an integral part of the Institute for

Defense Analyses in Princeton, NJ. He also served as a consultant to various computer corporations. As noted earlier, Tompkins served as consultant to Mina Rees and John Curtiss in the

establishment of the National Applied Mathematics Laboratories at NBS, of which INA was an

integral part.

In Spring 1953, Tompkins instituted a Seminar on Numerical and Computational Aspects of

Linear problems: Games, Linear Equalities, Linear Inequalities, Programming, etc. In this seminar

R. B. Horgan described SWAC coding of a search for a solution to a large set (128) of simple linear

inequalities in many (80) variables. M. Weber and E. C. Yowell described their SWAC experience

on finding the eigenvalues of a 45 x 45 matrix. Horgan, Weber, and Yowell were members of the

Mathematical Services Unit at INA with Yowell in charge of machine computations.

Because Harry Huskey was on a leave of absence during fiscal year 1953, Ragnar Thorensen

was in charge of the Machine Engineering and Development Unit. By the end of the year SWAC

was completed with the successful installation of a magnetic dnmi. Thorensen also redesigned the

Williams tube circuitry of the SWAC, thereby improving its reliability. The SWAC was now

working with greater efficiency. This increased the performance capabilities of the Mathematical

Services Unit and enabled the research staff to experiment with "larger" problems.

Studies in matrix inversion and in matrix eigenvalue problems were continued. Numerical

integration of differential equations was investigated further. Work it the mathematical theory of

program planning continued at approximately the same level. An investigation of variational methods applied to quantxmi mechanical scattering problems gave valuable results. Translation of interesting Russian mathematical articles was continued. The SWAC was used on a low priority basis to

investigate interesting topics in pure mathematics. A new Mersenne prime with p =2281 was discovered.

Several disturbing incidents occurred at INA during this period. One was the suspension of an

administrative clerk by the Commerce Department Loyalty Board No. 2, an action that appeared to

be unjmtified by the evidence as presented in the abstract of the hearing. Other personnel were

similarly investigated but cleared. These actions had an extremely negative effect upon the morale

of the entire institution. Equally distiu'bing was the unsettling relationship of the National Bureau of

Standards with other governmental agencies as a result of the "Battery Additive" case. By the end

of this period an "austerity" program had to be initiated at INA.

It is appropriate to digress for a moment and describe the situation that arose at NBS which led

to the austerity program at INA and finally to its separation from NBS. The NBS Applied Mathematics Division (AMD) was chronically in a precarious financial status in that it depended largely

on transfer of funds. In the late forties money was plentiful and AMD flourished. An important

factor was the central location of NBS in Washington, DC. It was easy for John Curtiss and the

senior members of his staff to keep in contact with the mathematical establishment as its members

visited Washington on committee business. These meetings generated goodwill toward AMD, and

in particular, simplified recruiting. (The present suburban location makes casual contacts less easy.)

To indicate the extent of the dependence of the AMD on transfer of funds, we note that, in

Fiscal Year 1953, out of its total budget of $1,546,200, only $128,400 came from NBS appropriations. In that year there were 163 employees in the Division.

During these, on the whole, good times there were occasional singularities. We mention some.

It would have been too much to expect that the staff could all avoid the security investigations

of the McCarthy era. John Curtiss saw to it that any of his staff involved got the best advice. One

of us (J. T.) recalls calling with Curtiss on a tennis friend to get advice from his father, who was an

important figure in Washington legal circles, on how to proceed.

35
The continued harassment of Condon contributed to his decision to resign and join the Coming

Glass Company as Director of Research in 1951. He later returned to academic life at Washington

University, St. Louis, and then at the University of Colorado, where he died in 1974. Condon was

succeeded as Director of NBS by Allen V. Astin, who was very sympathetic to the AMD, though

not quite as close as Condon.

Another great blow to the AMD was the resignation of John Curtiss in mid-1953. In Appendix

A we reproduce an evaluation of his work by John Todd [115]. F. L. Alt, who had been with the

AMD in various capacities, took over as Acting Chief and deserves great credit for holding the

organization together in very distressing circumstances until E. W. Cannon was appointed Chief in

late 1954. He remained in this position until his retirement in 1972. Cannon had been Assistant Chief

of the AMD in various capacities since its formation, and was on leave of absence during 1953-54

to serve as head of the George Washington University Logistic Research Project.

Another conflict was with the business sector. The NBS was supposed not to be in competition

with private enterprise. However, on occasion a Government contractor, who needed computational help which NBS alone could give efficiently at that time, would arrange for funds to be

diverted from his contract, say with the Department of Defense, to the NBS. There were a few

complaints that the INA, using Government supplied equipment, was imdercutting the small bxisi- ness computer firms which were then springing up.

However, the critical blows came from elsewhere. It was a consequence of the AD-X2 (battery

additive) controversy in which the NBS was involved from 1948 to 1953.

Part of the statutory responsibility of the NBS from the beginning was to render advisory

services to Government agencies on scientific and technical problems. Thus from the early 1920's

the NBS had tested a great many commercial battery additives, principally for the Post Office in

connection with mail fraud and for the Federal Trade Commission (FTC) with reference to public

advertising. All were of much the same composition, and distributed with more or less the same

claims. The NBS found none to be beneficial. NBS Circular 504, issued in 1951 after a new series of

such tests, warned people to be sxispicious of all such products.

The proprietor of a small California company proclaimed that his product, the AD-X2, had a

special composition and beneficial effect. The company encouraged satisfied users to flood Congress and the NBS with testimonial letters and, by the end of 1951, had gained the support of 28

Senators and 1 Congressman. Early in 1952, the NBS performed further tests of AD-X2 at the

request of the FTC, the Post Office, and the Small Business Committees of the House and Senate.

No beneficial effect of AD-X2 treatment was found. In the hope of finally resolving the issue, the

NBS performed a "blind" comparison test of matched groups of treated and untreated batteries

approved by the manufacturer following a procedure mutually agreeable to him and the NBS. The

comparison test, carried out carefully by staff of the Electrochemistry Section of the Electricity

Division, with meticulous analysis of the resulting measurements by the Statistical Engineering

Laboratory of the AMD showed that the product had no beneficial effect.

About the same time laboratory tests of this product were made by a group at a major university. The group's report did not claim that any practical benefits were associated with the observed

effects but the Senate Select Committee on Small Business was informed by a consultant thereto

that these tests completely supported the claims of the manufactxirer, and told this to the world by

press release in December 1952.

In January 1953 Eisenhower succeeded Truman as President. Sinclair Weeks became Secretary

of Commerce and Charles E. Wilson became Secretary of Defense. On March 24, 1953, Dr. Astin

was asked to resign by the Assistant Secretary of Commerce for Domestic Affairs to whom NBS

reported. Weeks explained "... I think that the National Bureau of Standards has not been sufficiently objective, because they discount entirely the play of the market place . . ." [Senate Small

Business Committee hearing on AD-X2, March 31, 1953; Hearings vol. p. 3; Washington Post, April

1, 1953, p. 1]. Astin's resignation was submitted on March 30, and was accepted by President

Eisenhower on April 2, to be effective April 18.

It is easy to imagine what alarm and despondency the news of Dr. Astin's forced resignation

brought to the NBS community. The first public announcement was probably in Drew Pearson's

36
column in the Washington Post of March 31. Among the actions considered was a mass resignation

in sympathy with Dr. Astin [Evaiing Star, April 17, p. 1]. The NBS commimity was relieved by the

outbursts in the press which included incisive cartoons. The nation's scientific community erupted

in overwhelming support of Dr. Astin and the integrity of the NBS.

In response to a suggestion of Sinclair Weeks, Secretary of Defense Wilson issued an order on

April 4 stating, among other things, that "Effectively immediately, no funds shall be obligated by

any Department or Agency of the Department of Defense for research and development to be

performed by a Government agency outside the Department of Defense without specific approval

of the Secretary or Deputy Secretary of Defense ..." [Date of order given and order quoted in

Washington Post, April 24, 1953, p. 8]. This caused considerable anxiety at INA, which was supported almost entirely by the Office of Naval Research and the Air Force.

Late on Friday, April 17, 1953, the day before Astin's resignation was to take effect. Weeks

announced (1) that the National Academy of Sciences would establish two committees; one (the

Jeffries Committee) to appraise the quality of the Bureau's work in relation to AD-X2 and to

determine whether additional testing was needed, and another (the Kelly Committee) to evaluate

the functions and operations of the NBS in relation to the nation's needs; and (2) that he had asked

Dr. Astin, and Dr. Astin had agreed, to continue to serve at least imtil the Academy Committees

had completed their work, but "no question is involved of Dr. Astin's permanent retention."

Although no mathematician was on the Kelly Committee initially, J. Barkley Rosser was

appointed to it subsequently. Mathematical statisticians, W. G. Cochran and S. S. Wilks, served on

the Jeffries Committee.

It happened that Astin had been invited to address the American Physical Society on May 1,

1953. In this memorable address he enimciated the beliefs of the Director of NBS and its staff,

beginning as follows:

"The Bureau staff believes first of all in the importance of scientific research

as a means of intellectual and scientific advancement, as the foundation of our

technological economy and high standard of living and as the bulwark of our

national security.

We believe in the teachings of Galileo that theory and hypothesis must con- form to the results of experimentation and observation. We believe in the philosophy of Lord Kelvin, that basic understanding in science depends on

measurement— the reductions of observation to nxraibers."

Astin's address was presented to an overflow audience which gave the speaker a standing

ovation.. We recommend the study of the whole address. It appeared in Physics Today 6 (1953), #6,

12-13 and is reprinted in the Astin Memorial Symposium booklet (NBS Special Publication 690,

January 1985).

The Kelly Committee submitted its initial findings late in July 1953, and its formal report on

October 15, 1953. Its appraisal of the functions and operations of the NBS was in the main favorable. Its principal recommendation was that the Bureau's weaponry development program be transferred to the Department of Defense.

On August 21, a Friday afternoon. Dr. Astin was reinstated. One of us (J. T.) recalls rrunors about the decision on that day and being depressed when there was no news at the close of business.

However, we remained in our office until Churchill Eisenhart later brought us the good news

which had been held up to prevent any demonstrations. The NBS was put imder the supervision of

the Assistant Secretary of Commerce for Administration.

On September 27, 1953, four major divisions of the NBS with about 60 percent of the personnel

of the NBS were transferred to the Department of Defense. The three working on proximity fuzes

and related materials became the Diamond Ordnance Fuze Laboratories of the Army Ordnance

Corps. The Bureau's Missile Development Division at Corona, CA, became the Naval Ordnance

Laboratories (Corona).

37
Summaries of the Kelly Committee report are given in Science , 119 (1954), 195-200, and in

Physics Today 6 (1953) #11, 17, #12, 4-11. While the Kelly Report highly praised the AMD activities, it questioned the appropriateness of the INA as part of the NBS program but left the decision

on the future of INA to the AMAC. After much deliberation it was decided that the NBS had to

give up the sponsorship of INA on Jime 30, 1954. The dissolution of INA is described in the next

chapter.

The Jeffries Committee fully supported the decisions of NBS on AD-X2. Summaries of its report are given in Science, 118 (1953), 683-5 and Physics Today 6 (1953) #6, 12 and #12, 26. There

is also a detailed account in [53] Chapter 3, 16-60, "AD-X2: The difficulty in proving a negative,"

Another accoimt is given by Samuel A. Lawrence [54]. A non-technical account of "The AD-X2

affair" is given on pages 12-21 of the Allen V. Astin Symposium booklet [71].

38
CHAPTER VIII

THE PERIOD SUMMER 1953 THROUGH SPRING 1954

D. H. Lehmer returned to the University of California at Berkeley in Augmt and C. B. Tompkins took over the Directorship of INA. Senior Research Personnel, including participants of the

SCAMP Project were:

Derrick H. Lehmer

Charles B, Tompkins

Gertrude Blanch

Harry D. Huskey

Forman S. Acton

Alfredo Bahos Jr. Richard H. Bruck

Kenneth A. Bush

Randolph Church

Robert P. Dilworth

George E. Forsythe

Richard A. Good

John W. Green

Dick Wick Hall

Marshall Hall Jr. Magnus R. Hestenes

Erwin Kleinfeld

Harold W. Kuhn

Cornelius Lanczos

Jacob Marschak

Theodore S. Motzkin

Lowell J. Paige

Gordon Pall

William A. Pierce

David S. Saxon

Jonathan D. Swift

Daniel Teichroew

Joseph L. Walsh

Wolfgang R. Wasow

(Director, UC Berkeley)

(Director, UCLA)

(INA)

(On leave)

(On leave)

(UCLA, Summer 1953)

(Wisconsin, Summer 1953)

(Illinois, Summer 1953)

(Monterey, Summer 1953)

(Caltech, Summer 1953)

(INA)

(Maryland, Summer 1953)

(UCLA, Summer 1953)

(Maryland, Summer 1953)

(Ohio State, Summer 1953)

(UCLA Liaison)

(Ohio State, Summer 1953)

(Bryn Mawr, Summer 1953)

(On leave)

(Chicago, Summer 1953)

(INA)

(UCLA, Summer 1953)

(Illinois Tech., Summer 1953)

(Syracuse, Summer 1953)

(UCLA, Summer 1953)

(UCLA, Summer 1953)

(INA)

(Harvard, Summer 1953)

(INA)

Acton, Huskey, and Lanczos remained on leave. S. S. Cairns was in charge of Project SCAMP,

a classified project sponsored by the Department of Defense. It was generally believed that it was

concerned with problems of communications.

The Graduate Fellows were:

John W. Addison Jr. Eugene Levin

Stanley W. Mayer

Mervin Muller

John Selfridge

Roger D. Woods

Walter I. Futterman

Genovevo C. Lopez

Edwin Mookini

Lloyd Philipson

Marion I. Walter

39
Most of the Gradiiate Fellows and many of the members of the Computing Staff obtained

Ph. D.'s in their field of specialization. Most of these hold university positions. Others are in industry and in government service. They are scattered throughout the United States and elsewhere.

Their contributions have been significant. In the present group, for example, John Addison is Professor of Mathematics at UC-Berkeley. Edwin Mookini (now deceased) became Vice President

of the University of Hawaii and was the top candidate for the Presidency at the time of his death.

Mervin Muller, now at Ohio State was in charge of computing at the World Bank. John Selfridge

has held professorships at variom imiversities and was a very successful Managing Editor of Mathematical Reviews during the period of its computerization.

The speakers at the Numerical Analysis Colloquiimi Series were:

The National Bureau of Standards was a co-sponsor with the American Mathematical Society

of a Symposium on Numerical Analysis held at Santa Monica City College, August 26-28. John H.

Curtiss was the chairman of the organizing committee. The symposium was entitled "American

Mathematical Society Sixth Symposium in Applied Mathematics: Numerical Analysis." The proceedings were published in 1956 with John Curtiss as editor. The title page and table of contents is given in Appendix D.

The papers presented in this symposium were concerned mainly with the following topics: discrete variable problems, combinatorial problems, number theory, eigenvalues, solutions of

linear systems, partial differential equations, linear and dynamic programming, and approximation

theory. A large number of the participants had been associated with NBS and INA as visiting

scientists. NBS and INA were represented by C. B. Tompkins, Olga Taussky-Todd, Emma Lehmer,

M. R. Hestenes, T. S. Motzkin, and W. R. Wasow. M. R. Hestenes presented a generalized conjugate gradient routine for solving linear systems. It was shown that every n-step procedure can be

interpreted to be a conjugate gradient routine of this type.

During Summer 1953 research on projects introduced earlier was continued. A special n-step

process for solving linear systems was devised by Motzkin. Walsh joined Motzkin in the study of

approximating polynomials. Studies in linear programming were continued by Motzkin, Kuhn, and

Marschak. Tompkins was concerned mainly with discrete variable problems. He was joined by

Kleinfeld and the Lehmers. Of course, number theory was pursued by various members of the

group.

David Saxon returned to his position in the Department of Physics at UCLA. He had a distinguished career in research and in administration. In 1975 he became President of the University of

California. In 1983 he was appointed Chairman of the Corporation of Massachusetts Institute of

Technology.

We now begin the final period of existence of INA. Research went on as usual. Numerical

experiments were made in finding the inverse of a matrix A. This led to the following problem.

Given an estimate X of the inverse of a matrix A, how do you improve this estimate? The Newton

algorithm of replacing X by (21 — XA)X was tried. In this algorithm the calculations of the matrices

E = 21 - XA and EX were crucial. When high (triple) precision was used, the algorithm was highly

successful for the matrices studied. When single precision was used throughout, the estimates became worse and worse. This phenomenon has been investigated deeply by J. H. Wilkinson.

Methods of preconditioning a symmetric matrix A by pre- and post-multiplication of A by a

diagonal matrix D were studied in [23] by Forsythe in cooperation with E. G. Straus of UCLA.

Precise characterizations of "best conditioned" matrices were obtained. Preconditioning tends to

reduce roundoff errors in matrix computations.

Studies in numerical solutions of certain partial differential equations were carried out by

Blanch, Motzkin, Philipson, Tompkins, and Wasow. Incidentally, Philipson wrote his Ph. D. dissertation under the supervision of Wasow.

G. Pall

J. Marschak

A. Bafios Jr.

S. Lefschetz

R. Radner

F. G. Foster

40
The Seminar in Numerical Analysis attracted many visitors from local industries and from

UCLA and nearby universities. It was held twice a week and was devoted to applications of

numerical analysis to mathematics, physics, engineering, and economics.

The decision of Secretary of Defense, Charles E. Wilson, to no longer permit a non-DOD

Government agency to serve as administrator of projects carried out at a imiversity but supported

entirely, or in large part, by DOD funds, caused the National Bureau of Standards to give up its administration of INA by June 30, 1954. The University of California was invited to take over this

administration. The university was not in a position to take over all sections of INA. However,

UCLA agreed to take over the administration of the research group, the SWAC and its maintenance, and the Library. This responsibility was assumed under contracts with the National Bureau

of Standards, the Office of Naval Research, and the Office of Ordnance Research. The contract

with NBS provided for the loan of the equipment which had been at the Institute. The question of

faculty status of INA members was to be dealt with after the takeover had been accomplished. The

new organization was to be called Nimierical Analysis Research (NAR). The local negotiation team

was comprised of C. B. Tompkins, M. R. Hestenes, Dean Paul Dodd, and Dean Vem Knutson.

At this time, a Numerical Analysis section was set up at NBS in Washington with John Todd

as Chief and with, on a smaller scale, a mission similar to that of INA. Milton Abramowitz became

the Chief of the Computation Laboratory, much reduced in size because of the restriction on

transferred funds. An account of "Numerical Analysis at the NBS" in the first 25 years 1946-71 was

given by John Todd in SIAM Review [116]. In 1957 John and Olga Todd joined the Department of

Mathematics at California Institute of Technology.

From the beginning of INA, John Curtiss worked diligently to build a first class research

library at INA. He was ably assisted by Forsythe with the help of Motzkin and Tompkins. They

were very successful. When INA was transferred to UCLA, the Department of Mathematics became the custodian of the Library. Tompkins and Hestenes were successful in obtaining funds for

maintenance and further development of the Library. Due to the efforts of L. J. Paige, the Library

was eventually transferred to UCLA. This gift enabled the Department of Mathematics to build a

first class Departmental Library. In appreciation of this gift, the books and journals received are

listed as belonging to the NAR Collection.

John Curtiss, working in conjunction with the various directors of INA, always encouraged

the departments of UCLA and neighboring universities to make use of the computing facilities at INA. Accordingly, there were a large number of unofficial members of INA comprised of faculty

and students who carried out pioneering research in various fields. Notable among these were

Michel Melkanoff of the Physics Department; James McCullough, Robert Sparks, and Kenneth

Trueblood of the Chemistry Department; Andrew Comrey of the Psychology Department; Yale

Mintz of the Meteorology Department; and Russell O'Neill of the Department of Engineering.

According to David Saxon "The work that was done on the optical model in those early days was

a genuine departure on the theoretical side of physics because it did permit the attack on an unusually complicated problem which would have been otherwise impossible, and for some time the

program which was written first by Roger Woods and then by Melkanoff was the most complicated

computational program of any that we knew about in the world of theoretical physics." This

program, developed under the leadership of David Saxon, became a world-wide standard for analyzing experiments on scattering of all kinds of particles (protons, neutrons, mesons, alphas) and

became a key tool for experimental physicists. Sparks and Trueblood wrote, "Digital computers

have revolutionized the practice of crystallography in the last thirty years One of the greatest

triumphs of early computers was SWAC's contribution to the solution of the hexacarboxylic acid

derivative of the vitamin B12."

INA attracted many distinguished visitors such as, John von Neumann, Solomon Lefschetz,

Edward Teller, Norbert Wiener, and many others, including researchers from neighboring imiversities. These were in addition to the many visitors who were given temporary appointments at INA.

This made INA an exciting place to be, not only for the regular members of INA, but also for the

visitors and the unofficial members of INA.

41
The engineers resigned on November 1, 1953 and accepted positions with the Magnavox

Corporation. The responsibility for maintaining the SWAC was assumed eventually by Fred H.

Hollander.

By June 30, 1954, various members of INA had accepted positions in industry and in various

departments of universities. For example, B. Handy, A. D. Hestenes, M. Howard, and E. C. Yowell

were employed by National Cash Register. S. Marks and A. Rosenthal went to the Rand Corporation. These and others continued to make significant contributions in applications of computers.

During his leave of absence from INA, Lanczos was employed by North American Aviation as a specialist in.computing. In 1954 at the invitation of Eamon de Valera, who was at that time Prime

Minister of the Republic of Ireland, Lanczos accepted the post of Senior Professor in the School of

Theoretical Physics of the Dublin Institute for Advanced Studies. There he had stimulating discussions with not only the senior staff but also with the junior staff of the Institute. He published many

scientific papers and six standard works written in a very personal style. These standard works

are: Applied Analysis, 1956; Linear Differential Operators, 1961; Albert Einstein and the Cosmic

World Order, 1965; Discourse on Fourier Series, 1966; Numbers Without End, 1968; and Space

Through the Ages, 1970. His last book, Einstein Decade: 1905-1915, was published in 1974. The

number of his scientific papers was about one hundred. Lanczos received many honors, such as. Membership of the Royal Irish Academy, 1957; the award of the honorary D. Sc. by Trinity

College, Dublin in 1962; the honorary degree of D. Sc. by the National University of Ireland in

1970; the honorary degree of Dr. Nat. Phil, from the Johann Wolfgang Goethe University, Frankfort, in 1972; and the honorary degree of D. Sc. from the University of Lancaster in 1972. Although

his work in his later years was concerned mainly with mathematical physics, he kept his interest in

numerical analysis and his influence on researchers in this field was great. Lanczos was awarded the

Chauvenet Prize of the Mathematical Association of America for his 1958 Monthly paper "Linear

systems in self adjoint form." He died in 1974.

In 1954 Harry Huskey accepted a faculty position at UC-Berkley, where he continued to make

significant contributions in the computer field. In 1967 he moved to UC-Santa Cruz to serve as Professor of Computer and Information Science. There he set up the USCS Computer Center and

served as its Director from 1967-1977. Internationally, he was in great demand as a consultant to

various computer centers, e.g., centers in India, Pakistan, Burma, Brazil, and Jordan. He spent an

extended period in Kanpur, India, where he directed the setting up of a computer center. Huskey

has served as a consultant to industrial and governmental agencies. He has been very active in

professional organizations on computers. He has received many honors. In particular, he and the

members of the SWAC Staff were honored at the Pioneer Session of the National Computer

Conference held in Anaheim, CA for their early work on the SWAC.

George E. Forsythe was one of the senior members of INA who remained with NAR. He soon

was given a faculty appointment in the Department of Mathematics, where he was in charge of the

educational program in numerical analysis. At his suggestion, Peter Henrici of Zurich, who had

been earlier attached to NBS and INA, was invited to join the Department. Henrici accepted,

thereby strengthening the program in numerical analysis at UCLA. Henrici was an expert in numerical solutions of differential equations by finite difference methods, as well as an authority on the

theory of functions of a complex variable. In 1957 Forsythe received a very attractive offer of a

faculty position at Stanford University, which would enable him to set up a program of his own. He

accepted this offer. In 1961 he became Professor of Computer Science and Chairman of the Department of Computer Science. Under his leadership, this department became the most influential one in

the country, attracting almost as many National Science Foundation Fellows as all other such

departments combined. Forsythe was for 2 years a member of the Board of Trustees of SIAM. He,

as well as Huskey, served a term as President of the Association of Computing Machines (ACM),

and has been otherwise active in these and other professional organizations. His wife, Sandra (Alexandra), was for some time a member of the st^ at INA and assisted him in various significant

computational programs. Later, at Palo Alto, she made significant contributions to the teaching in

computer science at the high school level. Forsythe died in 1972. A tribute to him is given by A. S.

Householder in the SIAM Journal on Numerical Analysis [36]. See also the article by J. Varah in

[131].

42
Wolfgang Wasow was also a senior member of INA who remained with NAR. However, he

held Fulbright Scholarships in Rome, Italy (1954-55), and in Haifa, Israel (1962). He spent the

academic year 1956-57 at the Army Mathematics Research Center in Madison, WI. In 1957 "Wasow

joined the Department of Mathematics at the University of Wisconsin, where he has had a distinguished career. In 1960 Forsythe and Wasow published a very successful book entitled Finite-Differ- ence Methods for Partial Differential Equations. It was translated into Russian, Japanese, and Chinese.

Theodore S. Motzkin became a member of the Department of Mathematics at UCLA. He

attained an international reputation for his contributions in the fields of linear inequalities, combinatorics, convexity, and approximation theory. He wrote significant papers in various fields, such as, algebraic geometry, algebra, number theory, function theory, and graph theory. He was a stimulus

to his colleagues. Motzkin was sought as visiting professor, as organizer and principal speaker of

many institutes, including two on combinatorics and inequalities, and as editor of journals and

symposia. Motzkin died in 1970.

Charles B. Tompkins became a member of the Department of Mathematics at UCLA. He was

in charge of the NAR Project and continued to be active in the SCAMP Project. He continued to

make the computing facility available to all interested faculty and students. In 1961 the Computing

Facility was set up as a separate organization with Tompkins as its first Director. Michel Melkanoff

played a significant role in the setting up of this organization. Tompkins was continually advocating

the broadening of the mathematical curriculum and organizing seminars. He was convinced that

mathematics and computers would have an increasing role to play in many new fields of research,

and, accordingly, he enthusiastically organized and conducted interdisciplinary colloquia. Tompkins died in 1971. A short simmiary of his career was published in Mathematics of Computation [122].

The paper [121] is representative of one area of his work.

Thomas H. Southard also remained with NAR. He became a member of UCLA-Extension and

contributed significantly to its applied mathematics program with special emphasis on numerical

analysis. Later he accepted the Chairmanship of the Department of Mathematics at the California

State College at Hayward, where he continued to emphasize the need for a broad mathematical

curriculum.

By and large the policies of INA became policies of NAR. The research program was continued much in the same manner as before with the support of ONR. Visitors continued to be invited

and to be supported by NAR. Qualified graduate students were supported. The SWAC remained

available for research computations for interested departments at UCLA and neighboring institutions. The SWAC was officially retired in 1967. In the meantime a separate computing facility at UCLA was set up to serve the entire university with the help of a more modem IBM electronic

computer, a gift from IBM. Earlier, the Business School, in cooperation with IBM, set up the

Western Data Processing Center to serve the business schools of the west coast imiversities. Later, a Computer Science Department was organized by the Engineering Department. Many members of

this Department had early training on the SWAC. A separate computing facility supported by the

National Institute of Health was set up in the UCLA medical school for medical research. Tompkins and Hestenes played a significant role in the establishment of these computing facilities at UCLA. The Institute for Numerical Analysis therefore enabled UCLA to be a pioneer in the use of

computers for research. In addition graduates of INA became leaders in numerical research elsewhere.

At 12 o'clock on Wednesday, June 30, 1954, INA held a "Gala Party and Luncheon" in honor

of the transfer of INA to UCLA, the people leaving INA, and the persons staying in NAR. At least

77 persons were present including David Saxon, the recent President of the University of California.

Shirley Marks, a long-time employee of INA, composed and read the following poem:

43
Looking back may be a sign of age

But, happily, may also be a gauge

Of what's to come, of futxire's promise—

And so reassiwe the doubting Thomas.

So let's return to the year '48

And see what, together, we can recreate

Of the life and times of INA—

Before they, old-soldierly, fade away.

Remember the summer of B. A. C?

Before Air-Conditioning: and the DDT

On desks and clothes; and Rusty the Rabbit

Whose tribe increased, as if by habit?

And the ping-pong tables we never made.

And the welfare meetings forever delayed

By erudite speeches on cookies and cokes,

And the cakes Mrs. Milne baked, and shaggy-dog jokes?

The visitors in summer of world-wide renown

Who politely inquired if the SWAC was still down?

And the picnics, and parties, and two hours for lunch.

And Rudolph the Reindeer, and lime-sherbet pimch?

Dr. Lanczos exhibiting his thespian powers.

The Lehmers code-checking into wee hours,

SWAC in the movies. Scrabble, Hearts, good friends?

On this happy past INA's future depends.

And though it begins with a change of name,

(Appropriate in June), the face is the same. And so, all good wishes to those who depart,

And to those who remain—best of luck—from the heart.

With this poem the saga of INA ended and NAR was bom.

44
CHAPTER IX

SOME TYPICAL PROJECTS

In this chapter we shall discuss, with a little more technical detail, some typical projects that

were carried out at INA. This will indicate the flavor of the work done at INA. We have chosen

projects in which we have had a special interest. These projects were carried out, in part, with the

cooperation of the group at NBS-Washington. At that time we were at the beginning of modern

numerical analysis, and so we were concerned with basic problems, such as, solutions of systems of

linear equations, inversion of matrices, solutions of characteristic value (eigenvalue) problems, and

related topics. There was much collaboration and even team work as the depth and magnitude of

the problems became understood. There was an increasing amount of experimental computation as machines became available. There was considerable international cooperation, with surveys of the

literature and translations when desirable. Work on many of these problems continues to this day.

See, e.g., [11], [13].

This chapter is divided into three parts. In the first part we are concerned with properties of

special matrices such as the Hilbert matrix. In the second part we discuss Gerschgorin theorems for

characteristic values of matrices. In the final part we give a detailed account of the program at INA

on solutions of systems of linear equations.

Hilbert IVlatrices and Related Topics

Historically, one of the first projects was the study of the Hilbert matrix. It all began with a

letter, dated October 3, 1947, to Olga Todd from George Temple (in whose department at King's

College, London, John Todd worked). He pointed out that the (British) Royal Aircraft Establishment (RAE) was interested in properties of the n xn Hilbert matrix

H„ = [l/(/+;-l)] (/J = l,2,...,n)

which had been studied by Hilbert in 1894. W. W. Sawyer, R. A. Fairthome and J. C. P. Miller had

computed the dominant characteristic value X„ of H„ for small values of n . In particular

X2«1.27, X3«1.41, X5«1.57

X6«1.62, X8«1.70, Xio«1.75, X2o«1.91.

Olga Todd showed that the asymptotic behavior of X„ is

X„=ir+0(l/logn).

She also considered the corresponding infinite Hilbert matrix H = H«. It is known (see Hardy,

Littlewood, P61ya: Inequalities) that the (infinite) quadratic form

Q(x) = XX a,,XiXj,

where a</ = l/(i+j — 1) and both simimations are over the positive integers, satisfies the relation

Q(x)<t:X X? when 0<Xxi^< oo

and this result is the best possible. Thus there is no nonzero vector x = (a:i, Xz, . . .) in /2 (Le., having

45
X < 00 ) such that Hx= irx. She raised the question in the Bulletin of the AMS as to the existence

of a vector x, not in for which Hx = ttx. We shall return to this point later.

The matrix H„ was notorious for its "ill condition," i.e., its inverse was hard to compute due to

cancellations in the computation and to roimd-off. The matrix had turned up in practice, e.g., in

least squares fitting of polynomials. The value of the determinant, det H„, was known to Cauchy

and the exact inverse (all elements integral) had been computed by E. Lukacs and I. R. Savage [111,

105-108]. About this time a serious study of "ill-condition" was beginning, e.g., by Turing and by

von Neumann and Goldstine. In this connection John Todd evaluated various condition-numbers of

special matrices which occurred in practice. These condition-numbers were figures-of-merit which,

when large, indicated that computational difficulties might be expected in inversion. He found that,

while the condition-numbers of matrices associated with the discretization of differential equations

were of polynomial growth in n, those for the Hilbert matrix were exponential. See AMS 29, 39,

and a survey article "On Condition Numbers" by John Todd [118].

Other aspects of condition-nimibers were studied at INA, such as, the effect of symmetrization

on condition-nimibers, by Olga Todd [110], an extension of Gauss's work on improving the condition of linear systems by G. E. Forsythe and T. S. Motzkin [132], and best conditioned matrices by

G. E. Forsythe and E. G. Straus [23].

Because of this ill condition many variants of the Hilbert matrix have been used as test matrices

for inversion programs and for programs for solving systems of linear equations.

We now return to the characteristic value/vector problem for the Hilbert matrix H„. The

Perron-Frobenius theory of positive matrices guarantees the existence of a (strictly) dominant characteristic value X„, with a corresponding positive characteristic vector. These objects can therefore

be calculated by the power method. G. E. Forsythe and M. Ascher computed the following values

of X, on the SWAC

X5o«2.08, X65'=2.11, X75«'2.16

Xioo**2.18, Xi25=2.21, X2oo**2.27

Other calculations were made by S. Schechter on a UNIVAC. This slow approach of X„ to ir was

intrigiiing and, as Littlewood [57, vol. 1, 22] wrote (in the context of good approximation), "Few

mathematicians can resist striking approximations even in a trivial context." There was therefore

considerable activity, not only for the Hilbert matrix, but for the important class of Toeplitz matrices to which it belongs.

The Hilbert matrix is also a totally-positive matrix and as such its characteristic vectors exhibit a regular sign pattern: the dominant one has no change in sign, the next one has a single change in

sign, the next one has two sign changes, and so on. This pattern can be used for testing programs for

the solution of the complete characteristic value problem.

There was further work at RAE by a German aeronautical engineer, Peter F. Jordan. By

interesting heuristic methods he obtained approximations to the characteristic values of H„. Despite

various attempts no rigorous treatment was found. Jordan's conjectures are stated as Problem 23 in

J. E. Littlewood's book Some problems in real and complex analysis. Heath 1965. Littlewood re- marks: "All imrigorous but it is quite a problem." [It is stated in this reference that the conjectures

were published in Nature, March 4, 1949, but we have not been able to trace this: perhaps Jordan's

letter was submitted at that time.]

Contributions to the study of H were also made by W. Magnus at about this time. See items 25,

26 in [61].

Considerable time elapsed before further significant progress was made. The first advance was

made at NBS and came to pass in the following way. There had been an interest in expanding the

international visitor program from Europe to Asia. No suitable candidate was apparent until Olga

Todd learned from George Temple of the work of Tosio Kato, a Japanese theoretical physicist,

who had made novel estimates of the first sub-dominant characteristic value. After long negotiations Kato was appointed to NBS and attacked the problem. Kato has had a distinguished career in

mathematics at UC-Berkeley after temporary appointments at the Courant Institute, NYU.

46
Kato's first result, depending on monotony considerations, was that there was indeed a nonzero

vector X such that Hx = 'irx, thus answering the problem of Olga Todd. He later showed that, if Af(9) is the Hilbert bound of the infinite matrix

H(e)=[i/(/+j-e)],

so that

M(e) = Tr cosec irO, -2<e^-3/2, M(e) = 'Tr, -3/2^6,

then every X with X>Af(0) is a characteristic value of H(9) with a corresponding positive characteristic vector and there is no characteristic value X<M(9) of H(9) with a positive characteristic

vector.

The next development was due to Marvin Rosenblum, now at the University of Virginia, who

first showed that every complex number with positive real p£irt is a characteristic value of H(9), 9

fixed and positive, by exhibiting the corresponding characteristic vector in terms of special functions. Rosenblum later determined the complete spectrum of H(9) using the Titchmarsh-Kodaira

theory of singular differential operators.

The story ends with the work of N. G. DeBruijn and H. S. Wilf who showed, in 1966, that

K='rr-i it' (log n)-2+0((log log n) (log n)"^

using results of Widom.

For a history of this problem to 1960 see John Todd, "Computational problems concerning the

Hilbert matrix," [117]. For a systematic account of related matters see H. S. Wilf [127].

The Hilbert matrix continues to be an object of interest to various groups of mathematicians as evidenced by the recent paper by M. D. Choi, "Tricks or treats with the Hilbert matrix" [11].

Gerschgorm Theorems and Related Results

So far we have been concerned largely with the asymptotics of the dominant characteristic

value of H„. We now turn our attention to less precise estimates of all the characteristic values of a

matrix. It is important to have inclusion regions, that is, sets 2 of points in the Euclidean plane

which include cr, the set of all the characteristic values of a matrix. In particular, it is often important to know when a matrix is convergent, i.e., when a is in U, the unit disk jz | < 1, or when it is stable, Le., when <t is in L, the left-hand half plane Rz <0. These problems are, however, essentially

equivalent since the Cayley map

w = (z-l)/(z + l)

transforms the interior of the imit circle into the left half plane.

We begin with the Gerschgorin theory. Recall that a matrix M= [my] has a strictly dominant

diagonal if

|mjj I >2'|/ny I, for all z.

Here 2' excludes j = i. The fact that a matrix with a strictly dominant diagonal is nonsingular was

known from the 1880's and is associated with many scientists— this is what Olga Todd called "the

recurring theorem" in a Monthly article [108]. Applying this to the matrix M=A-2l shows that, if

\z-aii\>y\aij\, alii,

then M is nonsingular and z is not a characteristic value of A. All the characteristic values of A

therefore lie inside or on the boimdary of the union of the n disks

\z-aii\<X'\aij\=ri.

47
These are the Gerschgorin disks or circles; they form an inclusion region 2. This theorem of

Gerschgorin [25] had been pointed out to Olga Todd by N, Aronszajn. She had used this and

various refinements of it in her war work (at NPL for the Ministry of Aircraft Production) in

connection with flutter of aircraft. One of the refinements was to note that, imder certain circumstances (irreducibility), it was not necessary to have strict diagonal dominance. Another was the

remark that since a similarity does not change the characteristic values, new, in general different,

inclusion regions can be obtained by applying the results to SAS"^ and using the fact that intersections of inclusion regions are again inclusion regions. Particularly important in practice is the case of

diagonal similarities. An important fact is that if a connected subset of r(< n) Gerschgorin disks is isolated from the complementary set, it necessarily contains r characteristic values. The use of

diagonal similarities in tiie case r = 1 was exploited successfully by Olga Todd in practical cases and

later studied theoretically by P. Henrici, John Todd [120], R. S. Varga [123], and others.

Among others connected with NBS or INA who worked in this general area were Ky Fan,

A. J. Hoffinan, and H. Wielandt. Hoffman, with Camion, proved a converse to the Gerschgorin

theorem [34].

We discuss further representative results briefly. Since the characteristic values of a matrix and

its transpose coincide, we can get column Gerschgorin circles with the same centers and (possibly

different) column radii Cy = 2'|fly |. Ostrowski mixed the rows and colunms. He showed that the

imion of the circles with centers au and radii q^"^ for any |3, O^p^l includes all the characteristic values of A. Instead of circles, A. T. Brauer [111, 101-106] used the Cassini Ovals defined by the

relations |z —au \ \z —Ojj \ ^<ry. Olga Todd prepared a bibliography on boimds for the characteristic

values of finite matrices [105], which was much in demand.

We have already noted the valuable contributions made to the INA program by European and

Asian visitors, thanks to the enlightened hiring policies of the NBS and the U.S. Civil Service

Commission. There was a notable contribution from S. Africa. In the early post war years Philip

Stein, together with a pupil, R. L. Rosenberg, submitted a paper to the London Mathematical

Society. This paper was concerned with comparison of the two classical iterative methods for the

solution of linear systems associated with the names of Jacobi and Gauss-Seidel. Olga Todd, as

referee, noted its importance and novelty and gave them detailed advice and made sure that it was

published. This paper [93] has become a classic and the Stein-Rosenberg Theory is a standard

chapter in courses on iterative matrix analysis. During his visit to NBS and later. Stein worked on

other problems suggested by Olga Todd, e.g., in Gerschgorin Theory and in Lyapxmov Theory,

with which his characterization of matrices C such that C-*0, is closely connected.

R. S. Varga and his many students built up a large body of work in this area. There were many

generalizations, e.g., to infinite dimensional problems, and developments from all over the world

from the early 1950's to the present day.

Solutions of Systems of Linear Equations

One of the main projects pursued at INA was the study of methods for solving a system of

linear equations by machine methods. A systematic study of this problem was imdertaken in 1949

when Barkley Rosser became the Director of INA. It should be remembered that, in 1949, the only

machine available to us was the IBM CPC. The SWAC was being built and could not be used. Of

course, desk machines were available, but we were interested in using "automatic" computing

machines. Apart from the CPC, we had to imagine what it would be like to me such machines. At

that time (as well as at the present time) a standard method for solving a system of linear equations

was some version of the Gaussian Elimination Method. Everett Yowell had already devised a

satisfactory elimination procedure for the CPC. He pivoted by permutation of rows. Variants of his

method are easily adapted to any "automatic" computing machine. For this reason we were con- cerned mainly with other methods for solving linear equations.

In 1949 Barkley Rosser organized a weekly seminar on the study of methods for solving

systems of linear equations and for finding eigenvalues of matrices. The regular members attending

this seminar were Barkley Rosser, Gertrude Blanch, George Forsythe, Magnus Hestenes, William

48
Karush, Cornelius Lanczos, and Marvin Stein. It was decided that Rosser and Forsythe should be

responsible for the study of methods for solving linear equations. Hestenes, Kanish, and Stein

should emphasize methods for finding eigenvalues of matrices. Lanczos already had a program of

study of methods for finding eigenvalues and eigenvectors of matrices and so would continue

working on his program. Blanch was busy running the Computational Unit and acted as an advisor

for the group. We also called upon Yowell for advice. All members of the group participated in

both programs. Forsythe undertook the task of reviewing the literature on these subjects. This led

to the publication (in AMS 29) of his important paper in which he classified methods for solving

linear equations and gave an extensive bibliography of this subject.

Let us try to recall some of the ideas presented in this seminar. It will be convenient to present

these ideas in a slightly more general form than they originally occurred to us. Consider the problem of finding the solution x of the linear equation

Ax=k, (1)

where A is a nonsingular n xn -matrix and k is an n -dimensional column vector. Its solution is

z=A-^k, (2)

where A~^ is the inverse of A. We assume that A"^ is unknown. For a given estimate x of z, we use

the size of the residual vector

r=k-Ax=A(z-x) (3)

as a measure of the accuracy with which x estimates z. Qearly, x = z if and only if r= 0. We shall use

the superscript T on vectors and matrices to denote the transpose. Thus, A^ is the transpose of A.

Recall that A is said to be symmetric if A^ = A. We can measure the size of r by looking at its largest

component or by looking at the length |r| of r. Alternatively, we can look at the value of the

function

f(x) =i r''Kr= \ x'^Bx - h'^x +ik'^Kk, (4a)

where K is a positive definite symmetric matrix and

B=A'^, h = A'^ (4b)

The matrix B is a positive definite symmetric matrix. When K=I, the identity, then F = ||r|^ The

solution x = z of Ax = k is the minimizer of F. The negative gradient of F is g=h-Bx=A^=B(z-x). (5)

Qearly x = z if and only if g=0, that is, if and only if x solves the equation Bh=h. The equation

Ax =k therefore can be replaced by an equation Bx=h in which the matrix B is a positive definite

symmetric matrix. Geometrically, the point x=z is the center of the (n — l)-dimensional ellipsoid

defined by the equation

/•(x) = 7

where -y is a positive number. When A is a positive definite symmetric matrix, the choice K=A~^

yields the relation B = A. In this case x solves Ax = k if and only if x is the minimizer of the function

/(x) = ^x'^Ax-k'^x+c,

where c is an arbitrary constant. The value of c is immaterial. For a given point Xi, the (n — l)-dimensional ellipsoid E defined by the equation

49
/W=/(Xi)

passes through Xi and the negative gradient ri = k—Axi of/at Xi is an inner normal of JB. In our studies of iterative methods for solving Ax=k, we soon came to the conclusion that

most of our iterations were equivalent to an iteration of the following type. Let Xi be an initial estimate of the solution z of Ax = k. Having obtained an i -th estimate Xj of z, compute an (/ + 1) — st estimate of z by a formula of the form

Xi+i = Xi - H. (Ax. - k) = x< + H, rj , (6)

where the matrix Hj is determined by some rule. In many cases the matrix Hj is a fixed matrix H. It

is easily verified that the sequence {x, } will converge linearly to z at a rate p, when

^=limsupj_«||I-H,AI| <1.

Here 1|M|| is a norm of M, which we choose to be the maximum of the length of Mx for all unit

vectors x. There are a variety of routines for solving Ax=k which are equivalent to an algorithm of the

form (6) with = H, a fixed matrix. One such routine proceeds as follows. We begin with an

estimate Xj of the solution z of Ax= k together with a set of m >n nonnull vectors Ui , . . . , i]„, which

span our space. We next select vectors Vi , . .

. , Vm such that the numbers

= v/Auy (j = l,...,m) (7)

are not zero. For example, we can select Vy =KAny, where K is a positive definite symmetric matrix.

Having obtained an i-th estimate x^ of z, we find an improved estimate Xi+i of z by the following

subroutine.

Set yi = Xi. Compute yi, . .

. , ym+i successively by the formulas

y;+i = yy+fl/Uy, aj=yj\k-Ayj)/dj. (8)

Then select x<+i = y„+i. Terminate the algorithm at the end of the i-th step if the residual

k—AXi+i is so small that x^+i can be taken as a suitable estimate of the solution z of Ax=k.

It should be noted that the scalar Oj in (8) is obtained by solving the equation

VyT[A(yy 4- ayUy) - k] = VyT(Ayy - k) +Gjdj = 0, (9)

that is, we choose Oj so that the residual k—Ayy+i is orthogonal to Vy. Observe that, because k=Az, Formula (8) can be rewritten in the form

yy^i-z = Wy(yy-z), (10)

where

Wy=I-UyVyTA/dy. (11)

Setting

W=W„W„_i . . . W2W1 (12)

we see, by (10) and the relations Xi+i=y„+i and yi=Xj, that

Xi^i-z = W„W„-i . . . W2Wi(x,-z)=W(x,-z). (13)

50
Defining

H = (I-W)A-^ (14)

we see that W=I-HA so that, with Az = k, (13) becomes

Xi+i — z = (I - HA)(x, - z) = X, - z - H(Ax< - k)

so that our routine is equivalent to one of the form

X.+1 = X. - H(Ax, - k), (15)

as was proved.

Normally the rate of convergence is improved when we introduce a relaxation factor p so that

Algorithm (8) takes the form

y/+i = y/ + Pa;Uy, ay=v/(k-Ay^)/dfy. (16)

The formula for Wy is then

Wy = I-pu,v/A/d';

and Algorithm (15) becomes

x<^i = x,-HO)(Ax,-k), (17)

where now H is a function of a positive number p. We have underrelaxation when P<1 and

overrelaxation when p > 1. In all cases we required p to be on the interval 0< p <2. Incidentally it can be shown that, when we select vy to be of the form Vy=KAuy, Algorithm (17) will always

converge when p is restricted in this manner.

In our seminar we did not begin with the general algorithm described above. We began with

the (forward) Gauss-Seidel routine. This is the special case in which m —n and Uy =Vy = «^, where

ei, . .

, , e„ are the imit coordinate vectors. In this event the matrix H(p) can be shown to be of the

form

H(P)=p(D-pL)-^ (18)

where D is the diagonal matrix and L is the strictly lower triangular matrix which together with a

strictly upper triangular matrix U are such that

A = L+D + U. (19)

We also considered the (backward) Gauss-Seidel routine in which case Uy=Vy = e„_y+i and

H(p)=p(D-pU)-\ (20)

Alternating the relaxed forward and backward Gauss-Seidel routines we obtain an Algorithm (17)

having

H(p)= p(2- p)(D- PU)-^D(D- PL)-^ (21)

This algorithm is equivalent to the routine generated by (8) when m = 2n — 1 and Uy = Vy are chosen

to be the vectors d, e2, . .

. , e^,-!, e„, e;,_i, . .

. , ei, ei successively. We normally chose p = 1 in these

51
three algorithms and did not attempt to find an optimal choice for |3. Convergence is assured when

A has sufficiently dominant main diagonal and when A is a positive definite symmetric matrix. In the

study of relatively simple examples, we found that it was not imusual for the convergence number

\L to be very close to one. Values of \l, such as jjl=0.99 and p,= 0.9999, appeared. This led us to

conclude that we could not expect to have rapid convergence except in very favorable cases. We also considered other special cases of Algorithm (8). In particular, we studied the case in

which m =n,\j = ej, and Uy is the i -th colimm of A^. This is called the Kaczmarz routine and has a geometrical interpretation in terms of projections on the hyperplanes associated with the equation

Ax = k. We omit this interpretation. By means of the transformation x=A^w, it is seen this routine

is equivalent to the application of the Gauss-Seidel routine to the equation

AATw=k.

Convergence is assured because the matrix AA^ is a positive definite symmetric matrix. Alternatively, if we select Uj = ej and Vy to be the ;-ith column of A we obtain a convergent routine which

is equivalent to applying the Gauss-Seidel algorithm to the equation

A^Ax=A^k.

In the general case with m=n. Algorithm (8) is equivalent to applying the Gaiiss-Seidel routine to

the equation

V^AUw=V'^ (22)

where Vi, . .

. , v„ are the column vectors of V, Uj, . . . , n;, are the column vectors of U, and x= Uw.

When U and V are chosen so that the matrix

D = V'^AU (23)

is a diagonal matrix, the solution z of Ax = k is obtained in one cycle (in steps) of Algorithm (8). One

can construct n's and v's of this type by a biorthogonalization process. However, such a process is basically equivalent to an elimination procedure. We did not pursue these ideas further at that time.

However, we did study the extension of Algorithm (8) in which the u's formed an infinite sequence

and discussed its properties including the question of convergence. In particular, at the suggestion

of Lanczos, we considered the case in which ui = k and n+i = Alt otherwise. In later years Tompkins

made effective use of the Kaczmarz routine. We next turned our attention to the study of gradient

methods for solving linear equations. Simultaneously, Rosser [83] devised a method for computing

the exact inverse of a matrix with integer coefficients.

In the study of gradient methods, we first considered the case in which our matrix A was

positive definite. Then the solution x = zofAx=kis the imique minimizer of the quadratic function

/(x)=|x'^Ax-k'fx. (24)

The residual r=k—Ax is the negative gradient of/at x, and is therefore in the direction of steepest

descent. The optimal gradient method proceeds as follows:

Select an initial poiat Xi and compute ri = k—Axi. Having found x^ and compute x<+i and r^+i

by the formulas

Xj+i = Xj +flir,, rj+i = iv-fl/Ai/,

ai = Ci/di, Ci=\ri\\ J,=ri^Ar<.

(25a)

(25b)

Terminate at the end of the i-th step if Vi+i is so small that x^^.i can be taken as a reasonable

estimate of the solution z.

52
I

This routine is called optimal because f =0, is the minimizer off on the line x = Xi+tTi. However, we quickly learned that normally it was not the best gradient routine. Forsythe constructed a random 6x6 positive definite matrix A and a vector k=Az with z prescribed. Starting with an

arbitrary initial point Xi distinct from z, he applied the optimal gradient routine to the system

Ax=K. In each case he failed to obtain a good estimate of z after a reasonable number of steps. The

iteration simply "bogged down." He then replaced equations (25a) by the relaxed equations

Xi+i=Xi + pa/ri, r,+i = i;-pfl<Ar<,

and obtained far better convergence when p was chosen to be one of the values 0.7, 0.8, or 0.9. The

choice p = 1, of course, gives us the original optimal gradient routine. Further experiments of this

type were carried out by M. L. Stein with essentially the same results. Hestenes and Karush had

similar experiences in the application of gradient methods to obtain eigenvalues of symmetric matrices. Motzkin suggested an alternative method for improving the optimal gradient routine. He

suggested that an acceleration step be introduced at appropriate times. This step consisted of minimizing/on the line joining the points x,_i and x,+i. The algorithm was then restarted with this linear

minimum point as the initial point. This too was successful. It was found that an acceleration routine

of this type had been introduced earlier by A. C. Aitken. We tried minimizing / on 2-planes or

3-planes instead of on lines. But these routines were cumbersome and did not yield significantly

better results. Hestenes and Stein prepared an extensive report of minimization methods for solving

linear equations. This report remained unpublished xmtil 1973 when it was published as a Historical

Paper in the Journal of Optimization Theory and Applications [31]. Publication was held up because,

in the meantime, Hestenes had devised an n -step conjugate gradient method which was based on the

concepts of relative gradients and of conjugacy, concepts which he had used in his studies of

variational theory. Hestenes and M. L. Stein established convergence properties for a large class of

iterative processes in the real and complex cases, including a generalized gradient routine of the

following type. As before, let A be a real positive definite matrix and let /(x) be the associated

quadratic function given by formula (24). Its minimum point z is the solution of Ax = k. Let H be a second positive definite symmetric matrix. We call the vector

g=-Hr=H(Ax-k)

a generalized gradient of/for reasons that will be explained below. Tht generalized optimal gradient

method proceeds as follows: Choose an initial point Xi and perform the iteration defined by the

equations

ri=k-Ax,, pt=Hr<

a, = d Idi , Ci =Pi ^ti, di = p'^Ap,

,

(26a)

(26b)

x,+i = x< +a/Pi, that is, Xi+i = Xj +aiHr<. (26c)

Observe that, if H = A~\ then fli= 1 and X2= A~*k, giving us the solution in one step. This suggests

that, if we have a rough estimate H of A~S Algorithm (26) should be very effective. We did not test

this conjecture numerically. To explain why we call g= — Hr a generalized gradient of/, let us

recall some concepts used by Hestenes in his studies of Variational Theory. Let /(x) be a function

on an inner product space possessing a directional derivative

/i(x;y)=(|y(^+'y)l-o,

which is linear in y. There is a vector g, called a "generalized" gradient of /at x, such that

/i(x;y)=<g,y>

53
for all vectors y. Here <z,y> is our inner product. In our case, with <z,y> =z^H"V, we have

/:(x;y) = (Ax-kfy= -i^y= <g,y> =fH'^

so that g= — Hris our "generalized" gradient. When H = I, we have the usual gradient. M. L. Stein

[91] used the generalized gradient concept in his Ph. D. dissertation to establish convergence theorems for gradients methods and Newton's method for variational problems. His thesis was sponsored by INA.

His study of the concepts of gradients and of conjugacy in variational theory led Hestenes to

the development of the theory of conjugate gradients. Early in July 1951 he used these ideas to

develop a conjugate gradient algorithm for solving Ax = k. In describing the method of conjugate

gradients, we continue with the assimiption that the matrix A in the equation Ax=k is a positive

definite symmetric matrix. Two vectors p and q are said to be conjugate if

p^Aq=0.

The term "A-orthogonal" is also used for this concept. An / -dimensional plane tti is conjugate to a

vector p if Ap is normal to tti or equivalently if every vector q in it/ is conjugate to p. On tti the

direction of steepest descent of

/(x)=|x^Ax-k''x

at a point x in tt/ is the negative gradient of / relative to ir^ and is accordingly the orthogonal

projection p on tti of the negative gradient r=k—Ax off at x. For X>0, we call Xp a "conjugate

gradient" of/ at x on ttj. The term "conjugate" is a geometrical term associated with ellipsoids. For

example, in the two-dimensional case, the midpoints of parallel chords of an ellipse lie on a line

"conjugate" to these chords and passing through the center for the ellipse, as shown in the following figure.

Figure 1

In this figure, x and y are respectively midpoints of parallel chords C and D. The chord

through X and y is conjugate to C and D and has as its midpoint the center z of the ellipse. This

resxilt, which we have described for an ellipse, holds in general. The midpoints of parallel chords of an / -dimensional ellipsoid lie in an / -plane tti "conjugate" to these chords and passing through the

54
center z of . Moreover, if is the intersection of the (n — l)-dimensional ellipsoid

/(x)=/(x.)

with an (/ + l)-plane tti+u then x, is on Ei and the orthogonal projection pi on tti+i of the negative

gradient r, =k—Ax, of /at Xj is an inner normal of at x,. This inner normal pi gives the direction

of steepest descent of /at x, relative to the (i + l)-plane ir.+i and is the "conjugate gradient" off at

X, on TTi+i. To find the midpoint m of a chord C of observe that m is the minimum point of / on this

chord and hence also on the line L containing the chord as a line segment. If x is a point on L and

p is a direction vector for L, the point x+fp is on L for all values of /. Moreover,

f(x+tp)=f(x)-a+\dt\

where

c=r^p, d=p^Ap, r=k—Ax.

The minimizer of /(x+Zp) with respect to f is t=c/d. Consequently, the point m=x+flp, with

a=c/d,is the minimum point of / on L and is therefore the midpoint of C.

The geometric algorithm for finding the center z of an ellipse £, shown in figure 1, can be put

in algebraic form as follows. Referring to figure 2, let Xi be a point on an ellipse E and let pi be a

vector pointing towards the interior otE at Xi. The minimizer X2 of /on the line

Li: x = xi+rpi,

with f as a parameter, is given by the formula

X2=xi+flapi,

where Oi is chosen in the manner described above. Select a second point yi = X2+ Sz which is not on

Li. The line

55
is parallel to Li. The minimum point off on L2 is

y2=yi+fciPi

for a suitable choice of bi. The vector

P2=y2-X2=S2 + fclPl

is conjugate to pi. The line

L3: X = X2 + fp2

passes through y2. The minimum point

Z = X2 + fl2P2

offonLi is the center z of the ellipse £. The method just described is a conjugate direction method

for finding the center z of an ellipse £. "When pi is an inner normal of £ at Xi and S2 is orthogonal

to pi, this method becomes the conjugate gradient method for finding the center z of £. This choice

of pi and S2 can be made by selecting

Pi = ri = k—Axi, S2=r2=k— Ax2.

In this event we have the formula

P2=r2+fcipi, fci= |r2lV|riP

for the vector pz conjugate to pi. The vector p2 is the A-orthogonal projection of T2 on the line L3.

It should be noted that, when pi = ti, the line

L4: x=X2+rr2

is parallel to the tangent T of £ at Xi. It follows that the miniTrmm point Zi offonL^ lies on the line

L5 joining Xi to z. Thus, we can reach the center z of £ by minimizing / successively on the lines Li,

L4, and Li, as pictured in figure 3. The points X2, Zi, and z are the midpoints of the chords in which

these lines cut the ellipse £

.

56
Figure 3

In the general case the method of conjugate gradients can be described as follows. In this

description it will be convenient to omit all references to ellipsoids and their inner normals. This

algorithm can be rephrased in terms of ellipsoids and will be done so later.

After selecting an initial point Xi, we compute the steepest descent vector pi of / at Xi and

obtain the minimum point X2 of/on the line Li through Xi in the direction pi. The (n — l)-plane iTn-i

through X2 conjugate to pi contains the minimnm point z off, so that the dimension of our space of

search has been reduced by one. We repeat the process restricting ourselves to the (n — l)-plane

tTb-i. We select the steepest descent vector p2 of /at X2 in Tr„-i and obtain the minimum point X3 of

/ on the line L2 through X2 in the direction p2. The (n — 2)-plane •ir„_2 in ir„_i through X3 and

conjugate to pj contains the minimum point z of/, so that at the next step we can limit our search

to ir„_2, a space of one lower dimension. This process is continued, decreasing the dimension of our

space of search by one in each step. In the i-th step we select a steepest descent vector pt at x, in an (n—i + l)-plane Vn-i+i and find the minimum point Xj+i of/ on the line L, through x^ in the direction

Pi. Our next space of search continuing z is the (n — /)-plane iTn-j in tt„-,+i through x^+i and conjugate to Pi. After m

^

steps we obtain a point x„+i which coincides with the minimum point z off,

the solution z of Ax=k.

The description of the conjugate gradient routine just given can be restated in terms of ellipsoids as follows: Select an initial point Xi. Let £„_i be the (n — l)-dimensional ellipsoid

/W=/(xi).

Its center z is the solution of Ax = k. Let pi be an inner normal of £„_i at Xi. Let X2 be the midpoint

of the chord of £„_i emanating from Xi in the direction of pi. The midpoints of the chords of

parallel to pi determine an (n — l)-plane ttb-i, which contains the point X2 and the center z of E„-i.

The (n — l)-plane ir„_i intersects the ellipsoid

/W=/(X2)

57
in an (n — 2)-dimensional ellipsoid E„-2 having z as its center. We repeat this process with E„_i

replaced by E„-2 and Xi replaced by X2. Select an inner normal p2 of E„-2 at X2. Let X3 be the

midpoint of the chord of £„_2 emanating from X2 in the direction of p2. The midpoints of the chords

of £„_2 parallel to p2 determine an (n — 2)-plane Tr„_2 in iTn-i containing X3 and the center z of E„_2.

The (n — 2)-plane ir„-2 intersects the ellipsoid

/(X)=/(X3)

in an (n — 3)-dimensional ellipsoid £„_3 passing through X3 and having z as its center. We next select an inner normal ps of £„_3 at X3 and repeat the construction relative to £„_3. Proceeding in this

manner we arrive at the (n — l)-step at a point x„_i on a one-dimensional ellipsoid Ej, an ellipse. We

select an inner normal p„_i of £1 at x„_i and find the midpoint x„ of the chord of Ei emanating from

x„_i in the direction of p„_i. The midpoints of the chords of Ei parallel to p„_i determine a line tti, a 1-plane, containing x„ and the center z of Ei. The line tti intersects the ellipsoid

in two points, whose midpoint x„+i is the center z of Ei and of E„_i, the solution x=z of Ax=k.

The description of the conjugate gradient method given above is somewhat involved. Fortunately, in applications, we need not explicitly determine the planes ir„_i, Tr„-2, ... or the ellipsoids

E„_i, E„_2, .... All we need to know are the formulas

ri+i=k-Ax,+i, pi+i = r<+i+fcir<, |r,+i|V|r,

determining the direction ps+i of steepest descent of / at x^+i on the (n — i)-plane iTn-j conjugate to

the vectors Pi, . . . , previously chosen. It is interesting to note that the 2-plane tr through x,+i,

containing the vectors ft and r<+i, cuts the (n — l)-dimensional ellipsoid

in an ellipse having ft as an inner normal at Xj. Since tt is conjugate to pi, . . . , ft, the problem of

finding ft+j becomes a two-dimensional problem of the type described earlier.

In view of the formula for ft+i given above, the conjugate gradient routine for solving Ax—

k

can be stated as follows: Initially select a point Xi. Compute pi = ri = k—Axi. Having obtained x<, r^, and ft compute x<+i, Ti+i, and ft+i by the formulas

a,=ci/di, di=Pi^Api, c<=ft'^ii or c<=|ri|2 (27a)

x<+i = x,+fljft, r<+i = r, -fliAft (27b)

fc,= -ftTAWrf, or fc,= |r,^i|V|r,p (27c)

ft+i = ri+i+i?/ft (27d)

Terminate at the m-th step if r„+i = 0. Then and Xm+i = z, the solution of Ax=k and the

TninimiiTTi point of /.

Observe that we have given alternative formiilas for bi and Ct . It can be shown that the residuals

Ti, . . . , Fm are mutually orthogonal and that the conjugate gradients pi, . . . , ft„ are mutually

conjugate. Also the residual r^+i is orthogonal to the vectors pi, . .

. , ft . In addition

ft /Ci = Ti/Ci + r2/C2 + ... + Ti /C,

.

The vectors pi, i>2, • • • , ftn satisfy the relations

58
P2=(l+fci)pi-fliApa, (28a)

Pi+i = (l+i>,)p,-fl,Api-fci_ipt_i (/>1). (28b)

Similarly, the residuals rj, rj, . . . , t„ satisfy the ternary relations

r2=ri-fliAri, (29a)

r,+i = (1

+

bt.i)ri - OiAVi - bi-iti-i (i > 1), (29b)

where

bi-i = atbi-i/ai-i. (29c)

In his original paper Hestenes used these ternary relations to give an alternative version of the

conjugate gradient algorithm. It is now known as Gradient Partan. It is obtained by rewriting

equations (29) in the following form

r2=ri-piAri, (30a)

r<+i = (Ti - Ar, - 8^.ir,.i)/(l - 8,_a), (30b)

where

3,= |ri|WAr„ 8,_i = r<_aV,-p,ArO/|r,_i|l (30c)

The corresponding formulas for the points are

X2=xi + Piri, (30d)

x<+i = (Xi + p^r, - 8,_ix,_i)/(l - 8._i). (30e)

The coefficients Pi and 8j_i are chosen so that r^+i is orthogonal to Ti and rj_i respectively. As a

result the residuals ri, r2, . .

. , r/ are mutually orthogonal so that i cannot exceed n . Algorithm (30) can be rewritten in the form

Select an initial point Xi and compute

ri = k-Axi, Pi=|ri|Vri^Ari, (31a)

X2= Xi + piFi, r2= Fi - PiAri. (31b)

For z = 2, 3, . . . compute

P,= |i;|Vr,^Aii, (31c)

yi+i = x< + p<ri, Si+i^Ti-^iATi, (31d)

8,_a=r,_l^^l/|^,_l|^ (31e)

X,,, = (yi+i - 8i_ix,_i)/(l - 8,_i), (31f)

ri+i = (si+i- 8i-ar<_i)/(l - 8,_a). (31g)

59
The point y,+i is obtained from x, by a gradient minimization step. The point Xj+i is obtained by

minimizing / on the line joining X/_i, and y,+i. The last operation is an acceleration procedure of the

type suggested by Motzkin. In this form the conjugate gradient routine consists of alternating a

gradient minimization step and an acceleration step.

There is another set of points Zi = Xi, z-i, . . . ,%„ associated with the conjugate algorithm (27).

Observe that, for i , the vector Pi is of the form

Pi = k—Azi, (32a)

pj=/n,(k-Az,), (32b)

ft+i=r<+i+Z><ft=m<+i(k-AZi+i) (32c)

so that

m,+i = H-i>jm,, mi = l, (32d)

zi = xi, z,+i = (x<+i + fcim,z,)//n,+i. (32e)

The conjugate gradient p, is therefore a scaled negative gradient of/ at the point z,. In addition, it can be shown that the point z,+i minimizes the square |rP of the negative gradient r of / on the

/-plane passing through the points Xi, X2, . . . , x^+j.

So far we have assumed that the matrix A was a positive definite matrix. The conjugate

gradient routine is valid when A is a nonnegative symmetric matrix. In this case the algorithm

terminates when either r„ = 0 or when Ap„, = 0. In the first case x„ solves Ax = k. In the second case

z„ is a least square solution of Ax = k. It is the shortest least square solution when Xj = 0 is the initial point. The conjugate gradient routine also solves Ax =k when A is symmetric and indefinite unless

one encounters the situation in which d, = 0 before the solution is obtained. To insure the effective- ness of our algorithm, we excluded the case in which A is indefinite.

To solve the eqiiation Ax = k for an arbitrary matrix A, we used a least square routine. That is, we applied the conjugate gradient algorithm to the equation

A^KAx=A^ (33)

where K is a positive definite symmetric matrix. Initially, we chose K=I, the identity. The associated function to be minimized is

F{x) = r^Kr= (k- Ax)'^K(k- Ax). (34)

g=A^K(k-Ax). (35)

Anticipating a generalization made later by Hestenes, we introduce a generalized negative gradient

Hg of F, where H is a positive definite symmetric matrix. Using generalized gradients, the conjugate gradient routine (27) for F can be put in the form:

Select an initial point Xi and perform the iteration defined by the following formulas

ri = k-Axi, gi=A'^Kri, i>i = Hgi, (36a)

c.=ps^gi, s,=Ar, di=Si'^KSi, ai=Ci/di, (36b)

Xi+i=x,+fl,pi, r<+i=r,-fl,Sj, g,+i=A'^i+i, (36c)

p,+i = Hg+i+fcift, fc,=g+i^H&+i/c,. (36d)

60
The generalized gradients gi, g2, . . . are H-orthogonal and the conjugate gradients pi, pz, • . • are B-orthogonal, where B=A^KA. It can be shown that, if Xi, X2, . . . , x„+i = z are the vertices of a nondegenerate simplex, matrices H and K can be chosen so that Algorithm (36) with Xi as its initial point reproduces the points xi, X2, . .

. , x„+i = z in the given order. It follows that the estimates of

z generated by an n -step algorithm can be reproduced by a conjugate gradient algorithm of the type

(36), provided that these estimates are independent.

When A is a positive definite symmetric matrix, the choice K=A"\ H = I in (36) yields the

original conjugate gradient Algorithm (27). The various versions of the conjugate gradient al- gorithm (with K= H = I) given above can be found in the original 1951 NBS report by Hestenes, in

which he also discussed the complex case. This report remained xmpublished until 1973 at which

time it was published as a Historical Paper [29] in the Journal of Optimization Theory and Applications. It happened that Eduard Stiefel of Zurich, Switzerland had devised the conjugate gradient

algorithm independently at about the same time or perhaps even a little earlier. When he arrived at UCLA to present his results at a Conference on Solutions of Linear Equations, he was given the

NBS report by Hestenes and found that we at INA had devised the same algorithm. Because of this

situation, it was decided that we should present our results in a joint paper. This led to the extensive

report by Stiefel and Hestenes on Conjugate Gradients published in 1952 [30]. It should be noted that the ternary relations (29) for the residuals are equivalent to the ternary

relations devised by Lanczos in his program for finding the eigenvalues of matrices. Lanczos presented these relations in the seminar sponsored by Rosser. It occurred to none of us at that time that

these relations could be used effectively in an algorithm for solving linear equations in n steps. We

were not aware of this connection until the conjugate gradient routine had been devised by geometrical considerations. It is clear therefore that the conjugate gradient algorithm is an easy conse- quence of results given by Lanczos. This led Lanczos to devise an alternative version of the

conjugate gradient algorithm, which he called a "Method of Minimized Iterations." This method

was published [51] in 1952 shortly before the joint paper of Stiefel and Hestenes appeared in the

same journal. Consider the case in which the matrix A is a positive definite symmetric matrix. Then

the routine proposed by Lanczos for solving Ax = k proceeds as follows:

Select Si = qi = k. Then compute vectors S2, Sj, . . . and qz, q3> • - • successively by the following

algorithm

s,+i=PiSi+Aq,, qi+i=<T<qi + si+i, (37a)

'yi=|s»T, Si = s<''Aq„ ^i = -hihi (37b)

<yi = --yi+i/Si, Mo=l, jJii = fH-i/Pi- (37c)

Terminate when sf„+i = 0. Then m

^

and the solution x=zofAx=kis

z= -M,iqi-M<2q2- • • • "M^qm- (37d)

Algorithm (37) is connected with the conjugate gradient Algorithm (27) by the relations

Ti = \ii.iSi, Pi = jii-iq<, xi = 0, (38a)

fli=-l/p< = -M</M.-i. bi = aj^i (38b)

Algorithm (37) is called a method of minimized iterations because s^+i is the shortest vector expressible in the form

Sj+i = (mo+/niA+ . . . + mi_iA'"^+A')k- (40)

One obtains the ternary relations of Lanczos by eliminating the q's in the relations (37). In his paper

61
Lanczos gave a detailed account of procedures for reducing the effects of round-off errors, including a discussion of how to employ Chebyshev polynomials for this purpose. His papers on this and

related subjects are very significant and contain many practical suggestions. Lanczos is widely

quoted by researchers in this field.

As suggested by Formula (40) for sj+i, every version of the conjugate gradient routine has

associated with it a significant set of polynomials. For example, in the conjugate routine (27), the

residuals r, and conjugate gradients p< are expressible in the form

r,=i?,_a(A)k,pj=P,-i(A)k, (41)

where (X.) and P,_i (X.) are polynomials in X of degree / - 1 having jRj_i (0)=P,_i (0) = 1. These

polynomials are generated by the algorithm

Ro=Po=l (42a)

Ri=Ri-i-ai\Pi.u Pt=Rt+biPi.i, (42b)

where Oi, . . . ,a„ and bi, . .

. , b„-i are the numbers appearing in Algorithm (27). The polynomial

R„{\) is a factor of the characteristic polynomial of A and is the characteristic polynomial of A

when m=n. These polynomials therefore can be used to find the eigenvalues of A in the manner

developed by Lanczos [50] or by some other means. The conjugate gradient routine belongs to a class of routines which we call coi^ugate direction

routines. In these routines we have given or we generate a set of n mutually conjugate vectors pi, pz,

. . . , by some means. Then the solution x=z of Ax=k can be obtained by the algorithm

Xi arbitrary, ri = k—Axi, (43a)

ai=Ci/di, </i=ps'^Aft, Ci=pi^ri=]^^ru (43b)

Xi+i = x< + fl,pi , ri+i = rj -fl, Api . (43c)

The vector x„+i is the solution z of Ax=k. In addition the inverse of A is given by the formula

A"' = PlPlV</l + P2P2^/t/2+ . . . +PnPn''/dn. (44)

It is generated by the iteration

Bo=0, B,=B,_i4-ptp,Vf/<, / = 1, ...,n (45)

We have B„ = A"^ Mutually conjugate vectors are generated by the conjugate gradient algorithm

(27). If we construct mutiially conjugate vectors pi, . .

. , p, in the following manner, we obtain a new routine, which we call a Coiyugate Gram Schmidt Process.

Select a set of linearly independent vectors Ui d„. Then compute pi, . .

. , p, as follows

Pi = ni, (46a)

P+i = H+i-fcaPi-6/2P2- • • • -biiPi, (46b)

for / = 1, . . . , n — 1, where, fory = 1, . .

. , i,

bij=pj'^Ani+i/dj, dj=pj^Apj=pj^Anj. (46c)

62
Introducing the commuting matrices

Wi=I-ptPi"^Aldi for 1=1J • • > (47a)

Co=I, Q=W,Q_i for/ = I (47b)

we see that Algorithm (46) can be put in the matrix form

(48)

for i = 1, . . . , n. Algorithm (46) can be rewritten in many ways and was done so by Stiefel and

Hestenes [30]. They showed that, when we choose Uj, . . .

, u„ to be the unit coordinate vectors,

Algorithm (46)-(43) is equivalent to a Gaussian Elimination method. Consequently, when A is a

positive definite symmetric matrix, a version of the Gauss Elimination for solving Ax=h can be

viewed to be a Conjugate Gram Schmidt Process. It should be noted that, in Algorithm (46)-(43),

the vector u, need not be chosen before the /-th step. If, in the i-th step, we choose u, to be the

residual r^, our algorithm becomes a conjugate gradient algorithm with a "built in" procedure for

reducing roimd-off errors and reduces to the conjugate gradient algorithm (27) when no roxmd-off

errors occur. The Conjugate Gram Schmidt Algorithm and hence also Gaussian Elimination can be approached from a geometrical point of view. We term this geometrical approach, the method of

parallel displacements, for reasons that will become evident. We proceed with the minimization with

a positive definite quadratic function

as follows. We begin with n + 1 independent points Xi, Xh, x^, . . . , x^, that is, n + 1 points which

do not lie in an (n — l)-plane. Equivalently, we begin with a point Xi and n linearly independent

vectors Ui, . .

. , ii„ and set x^ = Xj + n, . Having obtained these points, we set pi = xu — Xi and minimize

/on the lines x=X/i+rpi for7 = l, . . . , n. This yields n points X2=xi2, X22, . .

. , x„2. which lie in an

(n — l)-plane iTn-i conjugate to pi and containing the minimum point z of /. We have thereby

reduced our space of search by one. Accordingly, we repeat the process in the (/i — l)-plane tTb-i.

We setp2=X22— X2 and minimize/on the parallel lines x=3^2+'P2for7 =2, . .

. , n. This yields n — 1

points X3 = X23, X33, . .

. , x„3 which lie in an (n — 2)-plane 'ir„_2 in tTn-i conjugate to p2 and containing

the minimum point z of/. Because 'ir„_2 is in it„_i, it is conjugate to pi as well as to p2. Proceeding

in this way we find, in the z-th step, that we have n—i+2 points x^, x^/, . .

. , x„< points lying in an (n-i + l)-plane iTn-i+i which contains the minimum point z off and is conjugate to the vectors pi,

P2, . .

. , p-i. We continue by setting p = X// — x< and minimizing / on the parallel lines x = x^/ + fp for

j=i, . . . , n. This yields n—i + 1 points determining an (n - l)-plane tTh-i containing z. The point

x„+i is the minimum point z off. The vectors Pi, P2, • . , Pn are mutually conjugate.

The case n = 3 is shown schematically in the following diagram.

/(x)=|x'^Ax-k^x+c

63
Figure 4

The geometrical procedure just described follows readily from the formulas given by Stiefel

and Hestenes, although they did not explicitly give this interpretation.

When the vectors Ui, . .

. , i]„ are mutxially orthogonal, the Conjugate Gram Schmidt Algorithm

(46) can be put in the following matrix form. Select Go=I, the identity, and perform the following

iteration

p,=G,_iH, s,=Aps, q, = Gi_iS<, (49a)

d,=Pi\, 7/=q.'^s«, ^i = 1ildi, (49b)

For each /, the matrix G/ is a positive definite symmetric matrix, which is also given by the formula

where Bj and Q are the matrices (45) and (47b) respectively. Because Q, =0 and B„ =A"S we have

G„ = A~^ The solution of Ax =k is therefore z = G„k. Algorithm (49), with suitable modifications, is the basis for the variable metric routines that have been developed for nonquadratic as well as quadratic optimization.

Conjugate direction routines and, in particular, the conjugate gradient routine have many

interesting and useful properties. For example, in the conjugate gradient algorithm (27), the function

/(x) and the error function |x— z | are diminished at each step. The conjugate gradient routine (27) can terminate in fewer than n steps. This occurs when k is orthogonal to some of the eigenvectors

of A. It also occurs when A has fewer than n distinct eigenvalues, unless round-off errors intervene.

If the eigenvalues of A are clustered about m values, the vector x,„+i will be a good estimate of the

solution z of Ax = k. This means that often a good estimate of z is obtained in fewer than n steps. In

1951, in a physical application, Steifel and Hochstrasser obtained a satisfactory estimate of z in 90

steps with n = 106 [94]. Numerical experiments were carried out at INA by Hayes, Hochstrasser,

Stein, Wilson, and others. Ill-conditioned as well as well-conditioned matrices were used, including

Gi = G,-i - (piqJ + qiPi'^/di + (1 + POftP^'^M (49c)

Gj — QCj^+Bi, (50)

64
the Hilbert matrices. Singular matrices were also studied. We considered nonsymmetric as well as symmetric matrices, using an appropriate conjugate gradient algorithm in each case. In all cases we

obtained excellent results when high precision arithmetic was used. Because of machine limitations,

we restricted ourselves to relatively small (n <12) matrices. As a result we did not make an in-depth

study of roundoff errors. Because of round-off errors, the conjugate gradient routine frequently

does not yield the exact solution in n steps. If a satisfactory estimate of the solution is not obtained

after n , n + 1, or n + 2 steps, the algorithm should be restarted with the last estimate of the solution

as the new initial point. Alternatively, we can use the methods proposed by Lanczos for dealing

with round-off errors. Some have used a preconditioning scheme on the matrix A. Many of these are equivalent to using Algorithm (36) with a wise choice of H and K Others have combined the

conjugate gradient method with other routines to obtain effective means for obtaining solutions of

systems of linear eqxiations arising in the study of linear (and nonlinear) partial differential equations.

At present, extensions of the conjugate gradient method play an important role in numerical applications of nonquadratic as well as quadratic optimization.

An interesting application of conjugate gradients was made by Dr. David Sayre of IBM. He

sought to find a least squares solution of a system of more than 10,000 nonlinear equations in about

5,000 unknowns. His method was to linearize the equations, find a least squares estimate of the

linearized equations, use the estimate to obtain a new linearization, and repeat the operation. To

obtain his least square estimate of the linearized equations, he used only five steps of a conjugate

gradient algorithm of the type (36). The results are described in the following excerpt.

10,000 Equations and 15 Hours Later

It usually takes microseconds—perhaps as long as several seconds— for the computer to solve most problems.

However, Dr. David Sayre of IBM waited almost 15 hours

as a System/360 Model 91 computed the answer to his particular problem— a problem a team of scientists once worked two years to complete.

A mathematician at the T. J. Watson Research Center,

Sayre has been engaged in refining the structure of a

protein called rubredoxin. Beginning with an X-ray map at a resolution of 2.5 angstroms, Sayre used the Model 91 to process a complex system of more than 10,000 non-linear

equations— one of the largest equation systems ever pro- grammed for computer solution. A critical technique he

employed, called conjugate gradients, was developed by

Professor Magnus Hestenes of UCLA and was adopted by

Sayre at the suggestion of a research center colleague, Dr.

Philip Wolfe.

With the computation completed, and the resolution mathematically refined to 1.5 angstroms—about the dis- tance between the centers of neighboring atoms— Sayre was able to identify some 400 of the protein's 424 non- hydrogen atoms. In significantly reducing the computation time required

for such high-resolution studies, Sayre's work may lead to

improved imderstanding of molecules like DNA and

RNA, key elements La the reproduction of human cells.

Dr. Sayre examines the structure of the protein, rubredoxin, on this electron-density map.

For an account of the later history of the conjugate gradient and Lanczos algorithms see [133].

Reprinted with permission. Infosystems, August 1974, p. 15. ©Hitchcock Publishing Company,

Carol Stream, IL.

65

REFERENCES

Abramowitz, Milton; Stegun, Irene A., eds., Handbook of mathematical functions, Natl. Bur,

Stand. (U.S.), Appl. Math. Ser. 55, 1964 June, 1046 p. [This volume was reprinted many

times. It was published also by Dover in 1966. Since 1982 it has been published by J. Wiley

and Sons.]

Acton, Forman S., Numerical methods that work. New York, NY, Harper & Row, 1970.

Aitken, Alexander C, Gallipoli to the Somme, Recollections of a New Zealand Infantry Man,

Oxford, 1963. [See the obituaries: Proc. Edinburgh Math. Soc. (2) 16, 15-176, 1968-9; J. M.

Whittaker and M. S. Bartlett, Biog. Memoirs of Fellows of the Royal Society 14, 1-14,

1968.]

Beckenbach, Edwin F.; Walter, Wolfgang, eds., General inequalities, 3, ISNM 64, Basel

Birkhauser Verlag, 1983; 563 p. Beckenbach, Edwin F., ed.. Construction and applications of conformal maps, Natl. Bur.

Stand. (U.S.), Appl. Math. Ser. 18, Washington, DC, U.S. Government Printing Office,

1952; 280 p. [A list of contents is reproduced in Appendix D.]

Blanch, Gertrude; Rhodes, Ida, Table making at NBS, 1-6 in [88].

Blanch, Gertrude, On the numerical solution of parabolic partial differential equations, J. Res. Natl. Bur. Stand. (U.S.) 50, 343-356, 1953.

Brillhart, John; Lehmer, D. H.; Selfridge, J. L.; Tuckerman, Bryant; Wagstaff, S. S. Jr.,

Factorization of b°-l, b = 2, 3, 5, 6, 7, 10, 11, 12, up to higher powers. Providence, RI,

American Math. Soc, Contemp. Math 22, 1983, 178 p. [A second edition appeared in 1988].

de Bniijn, Nicolaas G.; Wilf, Herbert S., On Hilbert's inequality inn dimensions. Bull. Amer.

Math. Soc. 68, 70-73, 1962.

Camion, P.; Hoffman, Alan J., On the nonsingularity of complex matrices. Pacific J. Math.

17, 211-216, 1966.

Choi, Man-Duen, Tricks or treats with the Hilbert Matrix, Amer. Math. Monthly 90, 301-

312, 1983.

Cochrane, Rexmond C, Measures for progress: A history of the National Bureau of

Standards, U.S. Government Printing Office, 1966; 703 p.

Concus, Paul; Golub, Gene H.; O'Leary, Dianne P., A generalized conjugate gradient

method for the numerical solution of elliptic differential equations, p. 178-198 in Studies in

Mathematics, vol. 24, Golub, Gene H., ed., Washington, DC, Math. Assoc. America, 1984.

Curtiss, John H., The Institute for Numerical Analysis of the National Bureau of Standards,

8-17 of Monthly Research Report, May 1951, Office of Naval Research, Washington, DC.

[Reproduced in Appendix B (2); this includes a detailed bibliography of the earlier work of

NBSINA.]

Curtiss, John H., Sampling methods applied to differential and difference equations. Seminar

on Scientific Computations, IBM, 1949; 23 p. [Reproduced as Appendix B (1).]

Curtiss, John H., ed.. Numerical Analysis, Amer. Math. Soc. Proceedings Sixth Symp. in

Appl. Math., McGraw Hill, New York, NY, 1956; 303 p. [See Appendix D.]

Darwin, Charles G., On Weber's Function, Quart. J. Mech. Applied Math. 2, 311-320, 1949.

Eisenhart, Churchill, Edward Uhler Condon, Dimensions, The Technical Bulletin of the

NBS 58, 151, 1974.

Erdfilyi, A.; Todd, John, Advanced instruction in practical mathematics, Nature 158, 690-2,

1946.

Faddeeva, Vera N., Computational methods of linear algebra. Translated by C. D. Benster,

New York, NY, Dover, 1959; 252 p.

Forsythe, George E., Solving linear equations can be interesting. Bull. Amer. Soc. 59, 299-

329, 1953.

Forsythe, George E., Tentative classification of methods and bibliography on solving linear

equations, 1-28 in [70].

Forsythe, George E.; Straus, Ernst G., On best conditioned matrices, Proc. Amer. Math.

Soc. 6, 340-345, 1955.

Forsythe, George E., Bibliography of Russian books. New York, Chelsea, 1956; 106 p. Gerschgorin, Semon A., tJber die Abgrenzung der Eigcnwerte cincr Matrix, Izv. Akad.

Nauk, USSR 7, 719-754, 1931.

67
Hartree, Douglas R., Differential analysers. Report 17-302 London Ministry of Supply 1946,

XXV p. [A copy of this rare volume, donated by J. N. L. Connor, has been deposited in the

Caltech Library.]

Hayes, Robert M., Iterative methods for solving linear problems in Hilbert Space, p. 71-103

in [111].

Henrici, Peter, Bounds for the eigenvalues of certain tridiagonal matrices, J. Soc. Indust.

Appl. Math. 11, 281-290, 1963 and 12, 497, 1964.

Hestenes, Magnus R., Iterative methods for solving linear equations, J. Optim. Theory Appl.

11, 322-334, 1973. [This Historical Paper is a reprint of an NBS Report 52-9, 1951.] [See also

Hestenes, Magnus R., Conjugate direction methods in optimization. Springer-Verlag, New

York, 1980; 318 p.]

Hestenes, Magnus R.; Stiefel, Eduard, Methods of conjugate gradients for solving linear

systems, J. Res. Natl. Bur. Stand. (U.S.) 49, 409-436, 1952.

Hestenes, Magnus R.; Stein, Marvin L., The solution of linear equations by minimization, J. Optim. Theory Appl. 11, 335-359, 1973. [This Historical Paper is a reprint of a paper written

at INA in 1951.]

Hestenes, Magnus R., The conjugate gradient method for solving linear equations, 83-102 in

[16].

Hinsley, F. H. et al., British intelligence in the second World War, 3 vols.. New York, Cambridge University Press, 1979, 1981, 1984 (Part 1), 1988 (Part 2), 601, 850, 693, 1038 pp.

Hoffman, Alan J.; Taussky, Olga, A characterization of normal matrices, J. Res. Natl. Bur.

Stand. (U.S.) 52, 17-19, 1954.

Householder, Alston S., The Gatlinburgs, SIAM Review 16 (3), 340-343, 1974 July.

Householder, Alston, George E. Forsythe, SIAM J. Numer. Anal. 10, part 2, VIII-XI, 1972.

Huskey, Harry D., The National Bureau of Standards Western Automatic Computer, Ann

Hist. Comput. 2, 1980, p. 419-431. [Reproduced as Appendix C]

Huskey, Harry D., The SWAC: The National Bureau of Standards Western Automatic

Computer, 419-431 in History of Computing in the Twentieth Century, eds., Metropolis, N.,

Hewlett, J., and Rota, G. C, New York, Academic Press, 1980.

John, Fritz, A priori estimates, geometric effects and asymptotic behavior. Bull. Amer.

Math. Soc. 81, 1013-1023, 1975.

John, Fritz, On the integration of parabolic equations by difference methods I. Linear and

quasi-linear equations for the infinite interval. Comm. Pure Applied Math. 5, 155-211, 1952.

Kemps, H. J. L.; van Lint, J. H., The football pool problem for five matches, J. Combin.

Theory 3, 315-325, 1967.

Kantorovitch, L. V.; Krylov, V., Approximate methods of higher analysis. Translated by

C. D. Benster, New York, Interscience, 1955.

Kantorovitch, L. V., Fimctional analysis and applied mathematics. Translated by C. D.

Benster, NBS Report #1509, 1952.

Kato, Tosio, Perturbation theory for linear operators, New York, NY, Springer, 1966; 592 p. Kato, Tosio, A short introduction to perturbation theory for linear operators. New York, NY,

Springer, 1982; 161 p. Kato, Tosio, On the Hilbert matrix, Proc. Amer. Math. Soc. 58, 78-81, 1957.

Kato, Tosio, On positive eigenvalues of infinite matrices. Comm. Pure Appl. Math. 11,

573-586, 1958.

Kato, Tosio; Taussky, Olga, Commutators of A and A*, J. Washington Acad. ScL 46, 38-40,

1956.

Kupradze, V. D., Fimdamental problems in the theory of diffraction, NBS Report #2008,

1952.

Lanczos, Cornelius, An iteration method for the solution of the eigenvalue problem of linear

and differential operators, J. Res. Natl. Bur. Stand. (U.S.) 45, 255-282, 1950.

Lanczos, Cornelius, Solution of systems of linear equations by minimized iterations, J. Res.

Natl. Bur. Stand. (U.S.) 49, 33-52, 1952.

Latimer, Claiborne G.; MacDuffee, Cyrus C, A correspondence between classes of ideals

and classes of matrices, Ann. Math. 34, 313-316, 1933.

Library of Congress, Congressional Research Service. Technical information for Congress.

Report to the subcommittee on Science Research and Development of the Home Committee

on Science and Astronautics. Chapter 3, p. 16-60, AD-X2: The difficulty in proving a

68
negative. Washington, DC, 1971.

Lawrence, Samuel A., The battery additive controversy. The Inter-University Case Program

#68, University of Alabama Press, 1962; 33 p. Lehmer, Derrick, Mechanized Mathematics, Bull. Amer. Math. Soc. 72, 739-750, 1966.

Lehmer, Derrick. H., Selected Papers, 3 vols., D. McCarthy, ed., St. Pierre, Manitoba,

Canada, Charles Babbage Research Centre; 1135 p. Littlewood, John E., Collected Papers, 2 vols., Oxford Clarendon Press, 1982; xxxviii+ 1644

P-Littlewood, John E., Some problems in real and complex analysis, Lexington, Heath 1968; 57

P-Lopschitz, A. M.; Gavurin, M. K.; Birman, M. S.; Azbelev, N.; Vinograd, R., Four articles

in numerical matrix methods, Translated by C. D. Benster, NBS Report #2007, 1952.

Lowan, Arnold N., The Computation Laboratory of the NBS. Scripta Math. 15, 33-63,

1949.

Magnus, Wilhelm, Collected Papers, New York, N.Y., Springer-Verlag, 1984; 726 p.

Miller, J. C. P., ed.. Tables of Weberparabolic cylinder functions giving solutions of the differential equation (d^/dx^ + (x^— a)y = 0. Computed by Scientific Computing Service Ltd.,

Mathematical Introduction by J. C. P. Miller. London, Her Majesty's Stationery Office,

1955; 233 p.

Milne, William E., Numerical solutions of differential equations. New York, NY, John Wiley

and Sons, Inc., 1953; 275 p. Morse, Phillip M., E. U. Condon, Biographical Memoirs, National Academy of Sciences 48,

125-161, 1976.

Motzkin, Theodore S.; Taussky, Olga, Pairs of matrices with property L, Trans. Amer.

Math. Soc. 73, 108-114, 1952.

Motzkin, Theodore S.; Taussky, Olga, Pairs of matrices with property L, II, Trans. Amer.

Math. Soc. 80, 387-401, 1955.

Motzkin, Theodore S., Selected Papers, eds., Cantor, David G.; Gordon, Basil; Rothschild,

Bruce, Boston, BirkhSuser, 1953; 530 p.

Ostrowski, Alexander M., Solutions of equations and systems of equations. New York, NY,

Academic Press, 1st ed. 1960; 2nd ed. 1966; 3rd ed. 1973.

Oswald, Victor A.; Fletcher, Stuart L., Proposals for the Mechanical Resolution of German

Syntax Patterns, Modem Language Forum 36, 1-24, 1952.

Paige, Lowell J.; Taussky, Olga, eds., Simultaneous linear equations and the determination of

eigenvalues, Natl. Bur. Standards (U.S.), Applied Mathematics Series 29, Washington, DC,

U.S. Government Printing Office, 1953; 126 p. [List of contents reproduced in Appendix D.]

Passaglia, Elio, Allen V. Astin: A commemorative career biography, Natl. Bur. Stand.

(U.S.), Special Publication 690: Science: evidence, truth & integrity, 1985, 34 p.

Picone, Mauro, What INAC has given and is able to give, INAC, Rome, 1951; lip.

Picone, Mauro, On the mathematical work of INAC during the first quarter century of its existence , INAC, Rome, 1953; 37 p.

Rademacher, Hans A., Zur Theorie der Dedekindsche Strnmien. Math. Zeitschr. 63, 445-

463, 1956.

Rees, Mina S., Mathematics and the government: the post-war years as augury of the future,

in The Bicentennial Tribute to American Mathematics 1776-1976, Dalton Tarwater, ed.. Math.

Assoc. Amer. 1977; 101-118.

Rees, Mina S., The Federal Computing Machine Program, Science 112, 731-736, 1950.

[Reproduced with new introduction. Arm. Hist. Computing 7, 151-168, 1985.]

Rees, Mina S., The Mathematical Sciences and World War II, Amer. Math. Monthly 87,

607-621, 1980.

Rees, Mina S., The computing program of ONR, Ann. Hist. Comput. 4, 102-120, 1982.

Reid, Constance, Courant in Gottingen and New York, New York, NY, Springer-Verlag,

1978; 371 p. Rosenblum, Marvin, On the Hilbert matrix I, Proc. Amer. Math. Soc. 9, 137-140, 1958.

Rosenblum, Marvin, On the Hilbert matrix II, Proc. Amer. Math. Soc. 9, 581-585, 1958.

Rosser, J. Barkley, Rapidly converging iterative methods for solving linear equations, in

[70].

69
[83] Rosser, J. Barkley, A method for computing exact inverses of matrices with integer coefficients, J. Res. Natl. Bur. Stand. (U.S.) 49, 349-358, 1952.

[84] Rosser, J. Barkley, Mathematics and mathematicians in World War II, Notices Amer. Math.

Soc. 29, 509-515, 1982.

[85] Rhodes, Ida, A new approach to the mechanical translation of Russian, Natl. Bur. Stand.

(U.S.), Report 6295, 1959, 10 p. [86] Sadler, Donald H.; Todd, John, Admiralty Computing Service, Math. Comp. (formerly

Math. Tables Aids Comput.) 2, 289-297, 1947.

[87] Sadler, Donald H.; Todd, John, Mathematics in government service and industry. Nature

157, 571-573, 1946.

[88] Scaife, B. K. F., ed.. Studies in numerical analysis. Papers in honor of Cornelius Lanczos,

Dublin, Royal Irish Academy and New York, NY, Academic Press, 1974; xxii+333 p.

[89] Schwarz, H. R.; Rutishauser, H.; Stiefel, E., Numerical analysis of symmetric matrices. Prentice Hall, 1973; 243 p.

[90] Semendiaev, K., The determination of latent roots and invariant manifolds of matrices by

means of iterations. Translated by C. D. Benster, NBS Report #1402, 1952.

[91] Stein, Marvin L., Gradient methods in the solution of systems of linear equations, J. Res.

Natl. Bur. Stand. (U.S.) 48, 407-413, 1952.

[92] Stein, Philip, Some general theorems on iterants, J. Res. Natl. Bur. Stand. (U.S.) 48, 82-83,

1952.

[93] Stein, Philip; Rosenberg, R. L., On the solution of simultaneous equations by iterations, J. London Math. Soc. 23, 111-118, 1948.

[94] Stiefel, Eduard, Ueber einige Methoden der Relaxationsrechnung, Z. angew. Math. Phys. 3,

1-33, 1952.

[95] Stiefel, Eduard, Some special methods of relaxation technique, 43-48, in [70].

[96] Stiefel, Eduard,^ introduction to numerical mathematics. New York, NY, Academic Press,

1963; 286 + 30 p. [German edition 1961, 1976; French edition 1967; Spanish edition 1966;

Hungarian edition 1973.]

[97] Stiefel, Eduard; Scheifele, G., Linear and regular celestial mechanics, Berlin, Springer, 1971.

[98] Stiefel, Eduard; Kirchgraber, U., Methoden der analytischen Storungsrechnung und ihre

Anwendungen, Stuttgart, Teubner, 1978.

[99] Stiefel, Eduard; Fassler, A., Gruppentheoretische Methoden und ihre Anwendungen , Stuttgart,

Teubner, 1979.

[100] SzSsz, Otto, Collected Mathematical Papers, ed., H. D. Lipsich, New York, NY, Hafner,

1955; 1432 p.

[101] Szegd, Gabor, Collected Papers, ed., R. A. Askey, 3 vols., Boston, BirkhSuser, 1982;

857 + 863 + 880 p.

[102] Sflss, Irmgard, The Mathematical Research Institute Oberwolfach through critical times,

13-17 in [4].

[103] Silss, Wilhelm, ed., Field Information Agencies Technical, Review of German Science,

1939-1946. Pure Mathematics, 2 vols., Wiesbaden, Germany, Dieterich'sche Verlagsbuchhandlung 1948; 316 + 265 p.

[104] Taussky, Olga, A method for obtaining bounds for characteristic roots of matrices with

applications to flutter calculations, London, Aeronautical Res. Council Report 10, 508, 1947.

[105] Taussky, Olga, Bibliography on bounds for characteristic roots of finite matrices, Natl. Bur.

Stand. (U.S.) Report 1162, 1951.

[106] Taussky, Olga; Todd, John, On covering theorems for groups, Ann. Soc. Polon. Math. 21,

303-305, 1948.

[107] Taussky, Olga, On a matrix theorem of A. T. Craig and H. Hotelling, Nederl. Akad. Wetensch. Proc. 61A, Indag. Math. 20, 139-141, 1958.

[108] Taussky, Olga, A recurring theorem on determinants, Amer. Math. Monthly 56, 672-675,

1949.

[109] Taussky, Olga, Research Problem 12, Bull. Amer. Math. Soc. 60, 290, 1956.

[110] Tausslqr, Olga, Note on the condition of matrices. Math. Tables Aids Computation 4, 111-

112, 1950.

[Ill] Taussky, Olga, ed.. Contributions to the solution of systems of linear equations and the

determination of eigenvalues, Natl. Bur. Stand. (U.S.), Appl. Math. Ser. 39, Washington,

70
DC, U.S. Government Printing Office, 1954, 139 p. [List of contents reproduced in

Appendix D.]

[112] Teichroew, Dan, Distribution sampling with high speed computers, Natl. Bur. Stand. (U.S.)

Working paper INA 53-11, 1953, 167 p. [See also J. Amer. Statist. Assoc. 60, 27-49, 1965.]

[113] Thomson, Sir George, C. G. Darwin (1887-1962), Biographical Memoirs of Fellows of the

Royal Society 9, 69-85, 1963.

[114] Todd, John, Oberwolfach 1945, 19-22 in [4].

[115] Todd, John; John Hamilton Curtiss, 1909-1977, Ann. Hist. Comput. 2 (2), 104-110, 1980.

[Reproduced as Appendix A.]

[116] Todd, John, Nimierical Analysis at the National Bureau of Standards, SIAM Rev. 17, 361-

370, 1975.

[117] Todd, John, Computational problems concerning the Hilbert matrix, J. Res. Natl. Bur. Standards 65B, 19-22, 1961.

[118] Todd, John, On condition numbers, 141-159, Colloques intemationaux du CNRS No. 165.

Programmation en math&matiques mimeriques, Besanjon, 1966, Paris, CNRS, 1968.

[119] Todd, John, ed.. Experiments in the computation of conformal maps, Natl. Bur. Stand.

(U.S.), Appl. Math. Ser. 42, Washington, DC, U.S. Government Printing Office, 1955, 61 p. [List of contents reproduced in Appendix D.]

[120] Todd, John, On the smallest isolated Gerschgorin disks for eigenvalues, Nimier. Math. 7,

171-175, 1965.

[121] Tompkins, Charles B., Machine attacks on problems whose variables are permutations, 195-

211 in [16]. [This article discusses many experiments made on the SWAC at INA.]

[122] Varga, Richard S., On smallest isolated Gerschgorin disks for eigenvalues, Niraier. Math. 6,

366-376, 1964.

[123] Walther, Alwin, ed., Field Information Agencies Technical, Review of German Science,

1939-1946, Applied Mathematics, 5 vols., Wiesbaden, Germany, Dieterich'sche Verlagbuchhandlung, 1948, 307 + 161 +202 + 116 + 263 p. [124] Wasow, Wolfgang R., Random walks and the eigenvalues of elliptic difference equations, J. Res. Natl. Bur. Stand. (U.S.) 46, 65-73 1951.

[125] Wasow, Wolfgang R., On the duration of random walks, Ann. Math. Statistics 22, 199-216,

1951.

[126] Wasow, Wolfgang R., On the mean duration of random walks, J. Res. Natl. Bur. Stand.

(U.S.) 48, 462-471, 1951.

[127] Wilf, Herbert S., Finite sections of some classical inequalities [Ergeb. d. Math. u. ihrer

Grenzgebiete, 52], New York, NY, Springer-Verlag, 1970, 83 p.

[128] Massey, Harrie S. W., Leslie John Comrie, 1893-1950. Obituary Notices of Fellows of the

Royal Society 8, 97-105, 1952.

[129] Bellman, Richard; and Hall, Marshall, Jr., eds., Combinatorial Analysis. Proc. Tenth Symposium in Applied Math., American Math. Soc. Providence, RI 1960, 311. p.

[130] Meyer, H. A., ed., Symposium on Monte Carlo Methods. New York, NY, John Wiley and

Sons, 1956, 382 p.

[131] Proceedings ACM Conference on the History of Scientific and Numeric Computation,

Princeton, NJ, May 1987. Preliminary version distributed at Conference. Final version to be

published by Addison-Wesley.

[132] Forsythe, George E.; Motzkin, Theodore S., An extension of Gaiiss transformation for

improving the condition of systems of linear equations, Math. Tables Aids Computation 6,

3-17, 1952.

[133] Golub, Gene H.; O'Leary, Dianne P., Some history of the conjugate gradient and Lanczos

methods, 1948-1976. SIAM Rev. 31, 50-102, 1989.

[134] Curtiss, John H., The National Applied Mathematics Laboratories— A Prospectus, Ann.

Hist. Comput. 11, 13-30, 1989. Unpublished internal report. National Bureau of Standards,

Washington, DC, 1947.

[135] Curtiss, John H., The National Applied Mathematics Laboratories of the National Bureau of

Standards: A progress report covering the first five years of its existence, AniL Hist. Comput. 11, 69-98, 1989. Unpublished manuscript. National Bureau of Standards, Washington,

DC, 1953.

[136] Aspray, William and Gunderloy, Michael, Early computing and numerical analysis at the

National Bureau of Standards, Ann. Hist. Comput. 11, 3-12, 1989.

71

PHOTOGRAPHS AND CARTOONS

Edward U. Condon, Fourth Director NBS 1945- John H, Curtiss, First Chief, Applied Mathematics

1951 Division 1947-1955

NBS-INA, Building 3U on UCLA Campus Mina Rees, John H. Curtiss, and Olga Todd

73
Hans A, Rademacher

74
Douglas R. Hartree, Acting Director INA 1948 (using a Brunsvign B20 hand

computer) [From obituary notice by C. S. Darwin, Biographical Notices of Fellows of the Royal Society 4 (1958), p. 102-116.]
Unknown, George E. Forsythe, and William E. Robert E. Greenwood and Robert H. Cameron

Milne

Olga Todd

76
John H. Curtiss, Acting Director INA 1948-1949

Tobias Dantzig and Alexander M. Ostrowski George E. Forsythe, John Todd, and John W.

(Tobias Dantzig's son, George B. Dantzig, who Green

created linear programming, was closely associated

with NBS and in particular with relevant parts of

INA activity.

77
J. Barkley Rosser, Director INA 1949-1950

Group including: Mark Kac, Edward J. McShane, J. Barkley Rosser, Aryeh

Dvoretzky, George G. Forsythe, Olga Todd, Wolfgang R. Wasow, and Magnus R.

Hestenes

78
Mrs. and Mr. Theodore S. Motzkin Wolfgang R. Wasow and Magnus R. Hestenes
George E. Forsythe and Isaac J. Schoenberg

Everett C. Yowell and Thomas H. Southard Arnold D. Hestenes and Everett C. Yowell

80
Fritz John, Director NBS-INA 1950-1951 Allen V. Astin, Fifth Director NBS 1951-1969
Charles B. Tompkins, Director NBS-INA 1953-1954

Derrick H. Lehmer, Director NBS-INA 1951-1953 J. G. van der Corput

with Mrs. Lehmer (left) and Mrs. Arnold D.

Hestenes (right)

82
83

APPENDIX A

JOHN HAMILTON CURTISS 1909-77

We are greatly indebted to John Curtiss for his leadership and foresight in fostering the development of mathematics pertinent to machine computation. He had the ability to interest prominent

mathematicians to join him in this new development. The present book describes some of the

mathematicians whom he enticed to devote their time and efforts in this direction. It is most appropriate that we include in this book the story of John Curtiss himself. Fortimately, such a story has

already been written by John Todd. We present this article now as our story of his life.

A-1

John Hamilton Curtiss, 1909-1977

JOHN TODD

lot John Hamilton Curtiss was chief of the Applied

Mathematics Division of the National Bureau of

Standards from 1946 to 1953. He was largely

responsible for the planning and construction of

SEAC and SWAC and for the procurement of the

first UNIVACs for federal establishments.

Keywords and phrases: automatic computer,

SEAC, SWAC, National Bureau of Standards,

John Curtiss, obituary

CR category: 1.2

Annals of the History of Computing

Vol 2, No 2

April 1960

1 Vita

John Hamilton Curtiss was a man of many talents:

first and always a mathematician, but also a highly

able administrator, musician, and tennis player.

During the years 1946-1953 he was at the National Bureau of Standards (NBS) and played a

vital role in the development, procurement, and

widespread application of computers in the

United States. I was with him at NBS in

1947-1948 and 1949-1953. and I shall discuss his

contributions to the computer field as I remember

them.

John Curtiss was born on December 23,' 1909,

into an academic environment. His father, D. R.

Curtiss (1878-1953), was professor of mathematics at Northwestern University, was president of

the Mathematical Association of America in

1935-1936, and wrote a standard introduction to

complex variable theory (1926), still in print. His

uncle, Ralph H. Curtiss, was professor of astronomy at the University of Michigan.

© 1980 l)v the Amt-rican Kideration t)f Intormalion Frocc-ss- inp SotietifS, Ini . Permission to ( ()py without It'c all or part ot

this inali-rial is granted provided that the lopit-s are not made

or distributed tor dirett (oininenial advantage, the AFIPS

topyriglit iiolici' and the title ol the public aiion and its date

apfx-ai , and n()ti( e is given that (opying is by permission oi the

American Federation ol Inlorniation PnKessiiip; So(ieti<-s.

Ini . To topy otherwise, oi to republish, re(|uires spei ifii pei -

mission.

Author's address; Department ol Mathematic s 2.5S-37. Cali- fornia Institute of Tec hnology, Pasadena, (iA 91 12,").

' December 2S. incidentally, is ol gieat importance in tlie his- tory oi mathematics. R. Frieke wrote in the t.nryklopiidit d. Math. Wiss. (II. B3, p. 183) that jacobi said it was the birthday

ol elliptic func tions. Neither B. i.. (Carlson nor I has been able to check this statement, but F.uler reported on the tundameiital work of Fagnano to the Berlin Ac ademy on this dale in

17,51 , and it was on this date in 1799 that Ciauss completed the proofOf the expression for the arithmetic-geometric mean in terms ol an elliptic integral.

© 1980 AFIPS 0164-1239/80/020104-1 10$01 00/0

After graduating with highest honors from

Northwestern University in 1930, John (Curtiss

obtained an M.S. degree in statistics under H, L. Rietz at the University of Iowa, then one of the

leading centers in the Midwest for mathematical

training. Two of Curtiss's fellow graduate stu- dents there were S. S. Wilks and Deane Montgomery, who went on to distinguished mathematical

careers at Princeton University and the Institute

for Advanced Study, respectively.

John Curtiss then went to Harvard University,

where he earned his Ph.D. in 1935 under Professor J. L. Walsh. His first job after obtaining the

doctorate was an instructorship in mathematics at Johns Hopkins University in 1935-1936. In 1936

he joined the mathematics faculty at Cornell University, where he taught until entering the U.S.

Navy in January 1943. He was stationed in Washington, DC with the quality control section of the

Bureau of Ships until April 1946, when he was

discharged with the rank of Lt. Commander.

He immediately joined the NBS as an assistant

to the director, E. U. Condon, and was initially

responsible for statistical matters. On July 1, 1947, he was apjX)inted chief of a new division of

the NBS initially called the National Applied

Mathematics Laboratories; later the designation

Applied Mathematics Division (AMD) was used

and I shall keep to this. John Curtiss remained at the NBS until mid- 1953, except for a semester as

visiting lecturer at Harvard University in 1952.

He sf)ent a year at the Courant Institute of New

York University, and was executive director of the

American Mathematical Society (AMS) in Providence, RI from 1954 to 1959. In 1959 he became

professor of mathematics at the University of

Miami, Coral Cables, where he worked intensively

on one of his first areas of interest: approximation

theory in the complex domain. One by-product of

this p>eri()d was a graduate text on complex variable theory (1978). He died of heart failure at Port Angeles, WA on August 13, 1977, while en

route to the AMS summer meeting in Seattle.

While I cannot recall ever seeing John Curtiss

at the console of a computer—he always said that

I was involved in the salt mines of computing—his

interest in numerical analysis was considerable.

He wrote one paf>er on numerical algebra ( 1 954^)

and edited the proceedings of an important symposium (1956). Although the main body of his

work on approximation theory is peripheral to practical computation, as a statistician he was

deeply interested in "Monte Carlo." One pap>er on

this subject (1950), delivered at an IBM confer-
John Curtiss • J Todd

!()

Annals ot the History of Computing

Vol 2 No 2

April 1980

I'luc in 1949, was much acclaimed, and another

(1954cO was iranslaied into Russian. He gave

courses on numerical algebra at NYU in 1953-

19r)4 and on numerical analysis at the University

of Miami. During his time in Providence he made

a caref ul analysis of the book-sales {polic ies of the

AM.S.

John (lurtiss was <|uite sure that the nascent

computing fraternity had to become a national

.soc iety with publications of its own if it were to develop appropriately and be able to exert influence. Thus he help>ed enthusiastically in the

organization of the Eastern Association for (Computing Machinery (ACM), which dropped the

regional adjective from its title in 1948. He was

the first president in 1947 and always encouraged

his staff to participate in the work of this and

other professional organizations. Of these, Franz

L. Alt, Harry D. Huskey, and George E. Forsythe

later became presidents of ACM and Thomas H.

Southard was president of SI AM. (>urtiss also saw- that publications were supported, in particular

Mathematical Tables and Other Aids to Computation

and the Pacific Journal of Mathematics.

I will restrict myself fairly severely to the time

period 1946-1953 and to activities involving machines. But it is appropriate to point out here that

John Curtiss's contribution to the development of

modern numerical mathematics cannot be overestimated. He realized that any exf)erienced pure

mathematician could find attractive, challenging,

and important problems in numerical mathematics, if the person chose to do so. From the death

of Gauss in 1855 to 1947 the field of numerical

mathematics was, with a few exceptions, cultivated by nonprofessional mathematicians whose

real interests lay elsewhere. Accordingly Curtiss

recruited professionals from far and near to take

part in the programs he had envisaged. Indeed,

nearly all of his recruits contributed significantly

to his program.^

John Curtiss once documented the remarkable

success of the operation he planned by counting

the pap>ers presented at the 1952 International

Congress of Mathematicians by various organizations. NBS was in the middle of the top seven; the

others were the University of California at Berkeley, the University of Michigan, the University of

-Among these was J. C. P. Miller, from the Cambridge University Mathematical Laboratory. Happily, the adjective late used in a recent issue ot the Annals (Volume 1, Number 2. October 1979, p. 99) does not apply. Miller continues to make

disiiM^(uished contributions to niimei ital tiiaihematics.

John Curtiss in 1976.

Chicago, the Institute for Advanced Study,

Harvard University, and the University of Pennsylvania—all organizations founded long before

the Applied Mathematics Division of NBS.

For further details of the numerical mathematics programs at the NBS see Lowan (1949),

Blanch and Rhodes (1974), and Todd (1975).

2 "The Prospectus"

NBS was no stranger to computing equipment. It had been responsible for the Mathematical Tables

Project of the Works Progress Administration in

New York since 1938. This group was supported

during World War II by the Applied Mathematics

Panel of the Office of Scientific Research Development and from 1946 by the organization

that later developed into the Office of Naval Research (ONR). Led by Arnold N. Lowan, the

group included Milton Abramowitz (later chief

of the Computation Laboratory), Ida Rhodes,

Gertrude Blanch, Herbert E. Salzer, and Irene A.

Stegun.

While John Curtiss was first concerned with statistical matters within NBS, he soon had a national

resf>onsibility. Several incidents led to this. In

1945, Eckert and Mauchly, who were largely re- sponsible for the ENIAC, approached the Census

Bureau (which, like NBS, was part of the U.S.

Department of Commerce) with the suggestion

that a computer could facilitate its work—in the

coming 1950 census, for example. This suggestion was discussed by the Science Committee of

the Department of Commerce, which asked NBS

for technical advice. The final agreement (April

1946) was that the Census Bureau would transfer

funds to NBS, which would select a suitable computer and purchase it. The Army Ordnance
John Curtlas • J Todd

106

Annals of the History of Computing

Vol 2, No 2

April 1980

Department also transferred funds lo the Kk(

tronics Division of NBS for the development of

computer components.

At this time (London instructed Curtiss to sur- vey the federal needs for computers and for a national computing center. This investigation had

its source in ONR, and Rear Admiral H. G.

Bowen suggested that NBS and ONR should

jointly establish such a center, to develop as well as use computers. Funds for this purpM)se were

transferred in September 1946. In other countries, similar plans were being considered (see

Todd 1975, p. 362).

Curtiss's investigations led to a broadening of

the program. He realized very early, for example,

that the mathematics needed to exploit the new

John Todd is currently Professor of Mathematics at

the California Institute of Technology. He was

attached to the National Bureau of Standards in

1947-1948 and 1949-1957. His interest m

computing matters began during World War II when

he was a Scientific Officer in the British Admiralty.

He hcL<i written extensively on numerical mathematics.

computers had also to be developed, an opinion

shared by Mina Rees of ONR (1977). The program he formulated was described in the prospectus issued in February 1947. The AMD was to have four sections:

1. Institute for Numerical Analysis (IN A)—to

be a field station at UCLA (the University of

California at Los Angeles).

2. Compulation Laboratory (CL).

3. Statistical Engineering Laboratory (SEL).

4. Machine Development Laboratory (MDL).

The last three were to be in Washington, DC. The

nucleus of the CL was to be the Lowan group.

The program of the AMD to be guided, within

NBS operations, by an Applied Mathematics Executive Council consisting of representatives of

various federal agencies and some outside experts. Later the title of this group was changed to Applied Mathematics Advisory Council (AMAC).

A total staff of about 100 was contemplated and of

that only about 30 were on the NBS payroll when

the AMD was founded.

The AMD came into being on ) uly 1, 1947. The

prospectus had been a remarkable doc ument insofar as there was little need to change its c onients

as time passed. Curtiss undertook a massive recruitment |)rograin to impleme nt the aims of the

AMI). He was highly c|waliliecl for this activity,

with his outgoing personality and many ac ademic

contacts. Fortunately, too, the time was opportune for sue h an expansion, because many mathematicians were being clemobili/ed fiom their

World War II activities—a tunnl)er of them fresh

from some experience with applied mathematic s. Ihe soundness of the original stiiictiiie of

AMD is c lear f rom the f ac t that des|)ite a succession of NBS direc tors and se\ei al reorgani/alions.

the present (1979) organization is essentially un- changed, apart f rom the transfer of the MDL activities elsewhere. The original division has been

elevated to a center and now has four divisions:

Mathematical Analysis, Operations Research,

Scientific C-omputing, and Statistical Engineering.

Curtiss supported his staf f with an administrative and intellectual e limate eonduc ive to scientific

development. In this context let me mention an

instance of his loyalty to his staf f . On one occ a- sion, I was unexpectedly asked to clef end an awkward personnel action, and my superiors took

advantage of (Uirtiss's absence at IN A to "chew

me up." I complained to Curtiss by telephone,

and he immediately composed a seven-page

memorandum to those concerned, essentially a paper on Classification and Recruitment Problems Peculiar to a Mathematical Researc h I! nit. Beginning with the assumption that the federal

government wanted a mathematic s activity, he ex- plained how to operate it eeonomiealK and

efficiently. In my remaining years at NBS, I had

no further problems in this connection.

The years I am discussing were not the happiest

in Washington and some of our staf f were involved in loyalty and security problems. John

Curtiss saw to it that they got the best legal advice.

3 Procurement Problems

By late 1946, once certain legal situations were

resolved, NBS had funds available for two computers. The first was the UN I VA(- for the Census

Bureau, contracted for with the Fx kert-Mauc hly

organization in 1946, and the second was the NBS

computer (financed by ONR), contrac ted for with

the Raytheon (Company early in 1947.

The terms of the.se eontrae ts were discussed by

various eonnnittees, bv advisors (notal)lv, C.

Stibitz), and by the AMAC. There were many

complications, both administrative and tec hnic al. During these discussions the one l'\l\'A('. became three; one for the Air Comptrollei and

another foi the Army Map Service were adclcd.
John Curtias • J Todd

1 07 It is appropriate to mention here the division of

responsibilities between the Applied Mathematics

Division and the Electronics Division of NBS. The

AMD was responsible for the logic design of the

computers and their suitability for the jobs envisaged, and for initial liaison with the contractors.

The Electronics Division was respH)nsible for the

soundness of the design of components and for

all engineering matters. Once the development

was complete, the divisions were to share the liai- son and documentation duties.

The chief of the MDL from 1946 was E. W.

Cannon, who succeeded John Curtiss as chief of

the AMD in 1953. Ida Rhodes, originally with the

Mathematics Tables Project, was active in ensuring the suitability of projX)sed designs and later in

educating the coders and programmers.

Early in 1948, as it became clear that none of

these machines would be completed on schedule,

two enormously significant events took place. The

Air Comptroller, while awaiting the delivery of

the UNI VAC, realized that a small "interim" computer to be developed at NBS would provide useful exp>erience. This led to SEAC, discussed in

Section 4 below. The Air Materiel Command

wanted two computers, one for Wright Field and

one for IN A, but no supplier could be found.

Consequently, it accepted a profxjsal for a modest

machine to be developed at INA by Harry D.

Huskey. This led to SWAC; see Section 5. To end this historical sketch, the Census UNIVAC was completed early in 1951 and was dedicated on June 16, 195 1. The UNI VAC for the Air

Comptroller was completed in February 1952,

and the one for the Army Map Service was completed in April 1952. The Raytheon machine for

NBS was never completed, but a related machine

was delivered to the Naval Air Missile Test Center

at Point Mugu, CA in 1952.

4 SEAC

The NBS Interim Computer, later called SEAC

(Standards Eastern Automatic Computer), was

constructed for the Air Comptroller by a group in

the NBS Electronics Division (led by S. N.

Alexander) beginning in the fall of 1948. The

MDL collaborated in the design, and it was agreed

that as soon as the computer became operational

it would be moved to the CL. At that time I was

the chief of the CL, and we had a considerable

Annals ot grouD (led by Alan J. Hoffman) working for the

the History of Air Comptroller on linear programming, a subvor2^No^2 ject just being develop>ed by G. B. Dantzig and his

April 1980 associates.

In about 15 months SEAC became productive.

On April 7, 1950, my hand held by R.J. Slutz, I ran my first program: solving the Diophantine I equation ax + by = I. Actually a and b were originally taken to be the largest pair of consecutive

Fibonacci numbers that fittea into the machine , (<2'"); this was chosen to give the slowest Eucli- ' dean algorithm. The day before, Franz Alt had

run a factorization program using a small sieve.

SEAC was dedicated on June 20, 1950. Originally it had a 512-word delay-line memory, but

512 words of electrostatic memory were added.

The original teletyf>e input/output was supplemented by magnetic wire. For more technical information, see NBS (1947, 1950, 1951, 1955),

Greenwald et al. (1953), Shupe and Kirsch (1953),

and Leiner et al. (1954).

It was originally quite a distance from the CL to SEAC's building, and my coders/ojierators had to cross and recross the NBS grounds to use the

computer. On a visit to Los Alamos in 1 95 1 , after

I had described (p>erhaps too enthusiastically) the

current state of our operations, the laboratory authorities there decided that SEAC was just the

thing they needed for their weapon-related computations. Accordingly they preempted SEAC,

providing their own crew (for security reasons and for educating them in the use of computers).

Even less time was then available for developmental work, and pleas to move the machine to the CL, where we now realized how odd minutes

could be used effectively, were rejected on the

grounds that the delicate equipment might not

survive the trip. John Curtiss finally negotiated

with the AEC for the construction of a cinderblock building abutting the SEAC building, and

those of us in direct contact with the machine

moved into the new structure. A few years later

the machine was moved to the CL, where it operated until it was retired on April 23, 1964.

5 SWAC

In Section 3 I noted the origin of the Air Materiel

Command machine, later called SWAC (Standards Western Automatic Computer). This project began from scratch in January 1949, and the

first Williams tube machine to be completed in the

United States was dedicated on April 7, 1950. Just

as the British ACE was designed by a mathematician (A. M. Turing), the SWAC was designed by

Harry D. Huskey, who was trained as a mathematician. It was built among and for mathematicians.

There was a rather long period of debugging,

but in due course all troubles were overcome and
John Curtiss • J Todd

SWAC became a reliable machine with many

significant accomplishments. When the INA

ofjeration closed in 1954, SWAC was transferreci

to UCLA and remained in ojseration until 1967.

See the accompanying article about SWAC by

Huskey; also see NBS (1947, 1950, 1951, 1955),

Huskey (1950), and Huskey et al. (1953).

6 Conclusion

John Curtiss, with the full support of NBS

director E. U. Condon, and with modest encour- agement from various federal agencies, accelerated the progress of the United States toward a preeminent jX)sition in the construction of computers and their exploitation for scientific computations. I have already mentioned the use of

SEAC by the AEC; rocket and comet orbits were

computed on commercial equipment at INA

(Herrick 1973), and perhaps the first automatically computed earth-moon trajectory was done

on SEAC (Froberg and Goldstein 1952).

Had Curtiss been able to stay at NBS and had

support for INA been continued,^ there is no

doubt that the mathematical development would

have kept up with the enormous achievements of

the engineers. We who remained did what we

could to carry on the work for which Curtiss laid

solid foundations. Those of us who were with him

at NBS enjoy getting together and recalling the

exciting times of 1946-1953 and are grateful to

have had the privilege of working with a fine

American mathematician.

7 Postscript—Some Personal Reminiscences

My own first contact with John Curtiss was a letter

dated early in 1947, enclosing the prospiectus and

inviting me to consider joining INA. During

World War II, I had been active in organizing an Admiralty Computing Service in Great Britain.

Later, with my colleagues A. Erdelyi and D. H.

Sadler, I suggested the formation of a National

Mathematical Laboratory, later established as a division of the National Physical Laboratory. During that time my wife, Olga l aussky, and 1 had

many contacts with American mathemaiic ians sta- tioned in or visiting Europe, especially H. P. Robertson, H. M. MacNcille, G. Balcy Price, R.

Courant, and J. von Neumann. They were aware

of my activities, and 1 was in corresjjonderu e with

members of the Applied Mathematics Panel.

Some of these p>eoplc probably suggested my

name to John Curtiss. We arrived in New York on

a troop ship late in September 1947. Our fiist contact with the computer world outside

Washington was al the Aberdeen Meeting of

ACM on December 1 1-12, 1947.

John Curtiss was a bachelor who enjoyed fast cars and plenty of good food and drink. In introducing us to Washington society he asked me to

arrange a sherry party in his apartment. I provided sherries of varying cjuality and served them

according to his evaluation of the guests, reserv- ing the Bristol Cream for the director. For the

benefit of the many visitors to INA he compiled a

list of restaurants labeled according to the c ivil service gradings PI to P8.

He did not find the Civil Service regime too

convenient, and much of his activity was sjX'nt

maintaining contacts with other agencies, often

after regular hours. He dictated a diary late at night; a transcription was circulated to his staf f

the next day so that we were aware of what commitments he had made.

He was not happy on planes and did not travel

to Eurojje until 1976. A letter from him dated

May 11, 1976, from the Mathematical Research

Institute at Oberwolfach is addressed to me as "The Savior of Oberwolfach" (As British naval

officers, G. E. H. Renter and I were able, in 1945,

to prevent the dissolution of an institution that

has since made great contributions to mathematics, including formal languages, complexity

theory, many asfjects of numerical analysis, and,

for instance, computerized tomography.) He

complained about staying in "magnificent old fire

traps" and characterized one of the famous Lon-

'In 1952 there was a change of administration and the NBS whole of the AMD was at thai liuu- in a [)ri( ai loiis tinaiK i il was involved in the ADX2 (battery additive) controversy. Ciov- situation due in part to a decision of the Depanniciii <>l Dc- ernment scientists were reprimanded for not taking account fense not to permit transfers of funds lo NRS—and ilu AMI)

of the play of the marketplace. The NBS director was asked to had relied very heavily on sue h transfers from its l)e);iniunn.

resign, but was reinstated, and the NBS decisions were fullv The Universiiv of (California was nol in a position to take over supported by a special committee appointed by the National the operation of INA, but it ke pt llie SW AC and the \alual)lt

Annals of Academy of Science. One consequence of this was an investi- library and continui-d a nunu ru al analysis iesear< li pioji ( i mi

the History of gation of the NBS as a whole by an ad hoc committee chaired a modest scale. Private indusir\ and universities bc iu liied Computing M.J. Kelly. In the report of this committee (submitted to from the well-trained staff tluv \%ere able to recruit fiom tin Vol 2. No 2 the Secretary ofCommerce on October \5. \9ri3) the AMD closing of the IN.A operation and heavy reduitions in i' April 1980 was one of two divisions that were spec ially commended. The Washington workforce.
John Curtias • J Todd

109

Annals of the History of Computing

Vol 2, No 2

April 1980

don clubs as "the awfullest fire trap of all, but

interesting." He indicated two remembrances of

Kngland, the first "an infinite series of near-headon-collisions," and the second musical: "I recently

got a record of KIgar's organ music including the

Sonata in C played on the organ at Colston Hall,

Bristol, which we inspected amid chaotic preparations for a Salvation Army C>horal concert. Then

we heard the Sonata itself, included by coincidence in a noon recital in Hertford Chapel in

Oxford (and played too slowly)."

My last meeting with John Curtiss was at the

1976 Los Alamos Research Conference on the

History of Computing. He said then that he

thought that historians, so far, had not fully appreciated the contribution of the National Bureau

of Standards in the field. I hojje this essay will

begin to put things in balance.

8 Acknowledgments

I am indebted to John Curtiss's sister Alice (Mrs.

Edwin F. Beckenbach) for much p>ersonal information. I am also grateful to Nancy Stern for

careful criticism of a draft of this paper and to

Churchill Eisenhard and Henry Tropp for comments on a second version.

I have had access to a five-year progress report

Curtiss wrote in 1953. All the material I have

used, including a file of the jjeriodical rep)orts of

the AMD (1947 ), will be deposited at an appropriate center.

REFERENCES

Blanch. Gertrude, and Ida Rhodes. 1974. "TableMaking at the National Bureau of Standards." S<i«it^i

in Numerical Analysis, Papers in Honor of Cornelius

Lancws (B. K. P. Scaife, ed.). New York: Academic

Press, pp. 1-6.

Curtiss, D. R. 1926. Analytic functions of a complex

variable. Carus Mathematical Monographs, No. 2, Math.

Assoc. of America.

Froberg, C. E., and A. A. Goldstein. 1952. A collision

path from the earth to the moon in the restricted

problem of three bodies. Kung. Fysiografiska

Sdllskapets i Lund Forhandlingar 22, 14. Greenwald, S., R. C. Haueter, and S. N. Alexander.

1953. SEAC. Proc. IRE 41, 1300-1313.

Herrick, S. 1953. "Rocket and Comet Orbits." Applied

Math. Series #20. Washington: Government Printing Office.

Huskcy, H. D. 1950. "Characteristics of the Institute

for Numerical Analysis Computer." Mathematical

Tables and Other Aids to Computation 4, 30, 103-108.

Huskey, H. D., R. Thorensen, B. F. Ambrosio, and E. C. Yowell. 1953. The SWAC—design features and

operating experience. Proc. IRE 41, 10, 1294-1299.

Leiner, A. L., W. A. Notz, J. L. Smith, and A. Weinberger. 1954. System design of the SEAC and

DYSEAC. IRE Trans. Electronic Computers, 8-23.

Lowan, A. N. 1949. The Computation Laboratory of

the National Bureau of Standards. Scripta Math. 15,

33-63.

NBS. February 1947. Technical News Bulletin.

NBS. September 1950. Technical News Bulletin.

NBS. May 1951. Technical News Bulletin.

NBS. 1955. "Computer Development (SEAC and

DYSEAC) at the National Bureau of Standards."

NBS Circular 551. Washington: Government Printing Office. (This volume contains reprints of several

of the more hardware-oriented papers in this list of

references.)

Rees, Mina S. 1977. "Mathematics and the Government: The Post-War Years as Augury of the Future."

The Bicentennial Tribute to American Mathematics

1776-1976 (D. Tarwater, ed.), Washington: Math.

Assoc. of America, pp. 101-116.

Shupe, P. D., Jr., and R. A. Kirsch. 1953. SEAC—

review of three years of operation. Proc. East. Joint

Computer Conf., 83-90.

Todd, John. 1975. Numerical analysis at the National

Bureau of Standards. SIAM Review 17, 361-370.

PUBLICATIONS OF JOHN H. CURTISS

Administrative Papers

1948. A review of government requirements and activities in the field of automatic digital computing

machinery. Theory and Techniques for Design of Electronic Digital Computers (C. C. Chambers, ed.). Lectures delivered July 8 to August 31, 1946, Moore

School of Electrical Engineering. Philadelphia: Univ.

of Pennsylvania, pp. 29.1-29.32.

February 1947. "The National Applied Mathematics

Laboratories—A Prosf>ectus." 46 pp.

1948. A federal program in applied mathematics. Sci- ence 107, 257-262.

1949. Some recent trends in applied mathematics.

Amer. Scientist 37 , 1-5.

September 9, 1950. The program of a large computation center. Unpublished address, ACM, Wasnington, DC.

May 1951. The Institute for Numerical Analysis of the

NBS. Monthly Research Report, Office of Naval Research, 8-17. Another version of this appeared in ATner. Math. Monthly 55 (1951), 372-379.

April 1, 1953. The National Applied Mathematics

Laboratories of the NBS. Unpublished progress re- port covering the first five years of its existence.
John Curtlss • J Todd

Books

1978. Introduction to Functions of a (Complex Vartnhlt'. New

York; Dekker.

1956. "Numerical Analysis." Pror. Symp. Applied Math. 6. New York: McGraw-Hill.

Technical Papers

1935. Interpolation in regularly distributed points. Trans. Amer. Math. Soc. 38, 458-473.

1936. A note on the degree of polynomial approximation. Bull. Amer. Math. Soc. 42. 873-878.

1937. A note on the Cesaro method of summation. Bull. Amer. Math. Soc. 43. 703-708.

1940. On extending the definition of a harmonic function. Amer. Math. Monthly 47, 225-228.

1941. Generating functions in the theory of statistics. Amer. Math. Monthly 48, 374-386.

1941. Necessary conditions in the theory of interpolation in the complex domain. Ann. ofMath. , Ser. 2, 42,

634-646.

1941 . On the distribution of the quotient of two chance

variables. Ann. Math. Statistics 12, 409-421.

1941 . Riemann sums and the fundamental polynomials

of Lagrange interpolation. DuA? Math.f. 8, 525-532.

1941. On the Jacobi series. Trans. Amer. Math. Soc. 49,

467-501.

1942. A note on the theory of moment generating functions. Ann. Math. Statistus 13, 430-433.

1943. Convergent sequences of probability distributions. Amer. Math. Monthly 50, 94-105.

1943. On transformations used in the analysis of vari- ance. Ann. Math. Statistics 14, 107-122.

1946. A note on sonre single sampling plans requiring

the inspection of a small number of items. Ann. Math.

Statistics 17, 62-70.

1947. Acceptance sampling by variables, with sp>ecial

reference to the case in which quality is measured by

average or dispersion. y. Research Nat. Bur. Standards

39, 271-290.

1950. Sampling methods applied to differential and

difference equations. Proc. Seminar on Scientific Computation, November 1949. New York: IBM, pp.

87-109.

1953. Elements of a mathematical theory of probability.

Math. Mag. 26, 233-254.

1954a. "Monte Carlo" methods for the iteration of lin- ear operators. y. Math. Physics 32, 209-232. (Russian

translation in: Uspehi Mat. Nauk (N. S.) 12, 5(77)

(1957), 149-174.]

I954A. A generalization of the method of (onjugate

gradients for solving systems of linear algebraic

e(|uations. Mathematical Tables and Other Aids to Computation 8, 189-193.

1954<'. A theoretical comparison of the efficiencies of

two classical methods and a Monte C^arlo method for

computing one tomponent of the solution of a set of

linear algebraic ecjuations. Symposium on Monti' (.arlv

Methods, University of Florida. New York: John Wiley

& Sons, pp. 191-233. (1956, London: Chapman and

Hall.)

1960. Interpolation with harmonic and complex polynomials to boundary values. J. Math. Mech. 9,

167-192.

1 96 1 . A stochastic treatment of some classical interpolation problems. Proc. 4th Berkeley Sympos. Math. Statist. Ind. Prob., Vol. II. Berkeley: Univ. of Calif. Press, pp.

79-93.

1962. Solution of the Dirichlet problem by interpolating harmonic polynomials. Bull. Amer. Math. Soc. 68,

333-337.

1962. Interpolation by harmonic polynomials, y. Soc. Indust. Appl. Math. 10, 709-736.

1962. Limits and bounds for divided differences on a Jordan curve in the complex domain. Pacific f. Math.

12, 1217-1233.

1962. Polynomial interpolation in points equidistributed on the unit circle. Pacific J. Math. 12, 863-877.

1964. Harmonic interpolation in Fejer points with the

Faber polynomials as a basis. Math. Z. 86. 75-92.

1965. Convergence of complex Lagrange interpolation

polynomials on the locus of the interpolation points.

Duke Math. J. 32, 187-204.

1966. The transfinite diameter and extremal {joints for

harmonic polynomial interpolation, y. Analyse Math.

17, 369-382.

1966. Solutions of the Dirichlet problem in the plane by

approximation with Faber polynomials. SIAM J. Numer. Anal. 3, 204-228.

1969. The asymptotic value of a singular integral re- lated to the Cauchy-Hermite interpolation formula.

Aequationes Math. 3, 130-148.

1969. Transfinite diameter and harmonic polynomial

interpolation. y. Analyse Math. 22, 371-389.

1971. Faber f>olynomials and the Faber series. Amer.

Math. Monthly 78, 577-596.

1972. Corrections to "Faber f>olynomials and the Faber

series." /l»7wr. Math. Monthly 79, 313,

1972. Over determined harmonic px)lynomial interpolation. y. Approx. Theory 3, 149-175.

Annals of the History of Computing

Vol 2. No 2

April 1980

APPENDIX B

TWO PAPERS BY JOHN H. CURTISS

The first of the following two papers is one of the early papers on the Monte Carlo method and

had a great influence in the development of this method. Though it was never formally published in a scientific journal, the paper was presented at the Seminar on Scientific Computation, November

1949, IBM Corporation. Because of its significance we are pleased to present this contribution at

this time. The second paper is a report on the progress of the Institute for Numerical Analysis

1947-51. It contains a bibliography of the work accomplished during the first 3 years. Attention is, in particular, invited to the concluding paragraphs.

B-1
I
Sampling Methods Applied to Differential

and Difference Equations

JOHN H. CURTISS

National Bureau of Standards

THE PURPOSE of this paper is to present, as simply as possible, a part of the theory underlying

the use of probability sampling techniques in the

numerical solution of boundary value problems. The

sampling procedures under discussion consist of re- peated trials, or realizations, of a type of stochastic

process which physicists call a random walk (or

random flight) with absorbing barriers. This type

of process has recently become familiar to statisticians in connection with sequential analysis.

In practice, the trials are actually to be carried

out by computing machines or hand computers. However, the problem of instructing machines or hand

computers as to how best to execute and check the

computations will be touched on only briefly in this

paper. Another limitation on scope is that eigenvalue

problems will be avoided by considering equations

in which only the derivatives or differences of the

unknown function appear, not the function itself.

Still another limitation is that the basic asymptotic

theorems are only sketched.

An attempt has been made to state , with reasonable

generality, a number of facts which have heretofore

been presented only for special cases, or which have

never been formally published, although they may be

common knowledge among specialists. Some of the

material in Sections 11 to 17 inclusive appears to be

new. An effort has been made to compile a bibliography which is extensive enough to serve at least as an

introduction to the literature bearing upon the connection between differential equations (particularly

of the type to be considered here) and stochastic

processes.

This introduction would not be complete without

an acknowledgement of the author's debt to his colleagues in the National Bureau of Standards; in

particular, to Dr. W. WasowandDr. R. P. Peterson, Jr. , with whom a number of conversations on

the subject matter were held. Dr. Wasow and Dr . Forman Acton read a first-draft of the manuscript

and made many helpful suggestions . The development

in Section 17 is almost entirely due to Dr. Wasow,

and part of the exposition in Section 13 closely follows a discussion which he transmitted to the author in a

letter. The author is grateful to Dr. Wasow for permission to include this material.

The article undoubtedly shows traces of the influence of Prof. W. Feller, who conducted seminars at

the Institute for Numerical Analysis of the National

Bureau of Standards in the summer of 1949. The

manuscript of Dr. Feller's forthcoming book on

probability, which will probably become a standard

reference, became available at the Institute after the

present paper was largely completed. Unfortunately,

limitations of time schedule made it impossible to

consult this manuscript in the course of the preparation of this article, although the paper would probably

have thereby benefited considerably.

Historical and Bibliographical Notes 2

It has lately become fashionable to apply a rather

picturesque name, the "Monte Carlo Method", to any procedure which involves the use of sampling

devices based on probabilities to approximate the

solution of mathematical or physical problems. The

Monte Carlo Method has, so far, largely been used

in connection with functional equations and quadratures. No comprehensive technical exposition of the

method has as yet appeared. A stimulating philosophical introduction to the Monte Carlo Method has

been given by Metropolis and Ulam'** , and a number
of interlaboratory reports have appeared. In such

publications, and in word of mouth transmittal of

computing techniques, it is natural to omit references

to any existing background literature. Consequently,

misconceptions regarding the novelty of the method

have arisen from time to time among persons interested in the applications rather than the history.

The use of functional equations in connection with

probability problems arising in games of chance dates

back to the earliest beginnings of probability theory,

and was familiar to de Moivre. Lagrange, and Laplace. The connection between probabilities and the

differential equations of mathematical physics is also an old story to theoretical physicists. During the

early years of the present century, it was studied

particularly extensively by Einstein. Smoluchowski,

Lord Rayleigh. Langevin. and many others. Good

bibliographies will be found at the end of References

3, 24 and 30. More specifically, in 1899 Lord Rayleigh^^ proved that a random walk in one dimension

(of the type considered below in Section 4 but without

absorbing barriers) yields an approximate solution

of a parabolic differential equation. The relationship

between random flights (i.e. , random walks in more

than one dimension) and the first boundary value

problem, or Dirichlet problem, for elliptic difference and differential equations was established by

Courant, Friedrichs, and Lewy in 1928^. Mathematical interest was stimulated by the publication in

1931 of a celebrated paper by Kolmogorov^"* dealing

with the relationship between stochastic processes

of the Markoff type and certain integro-differential

equations. Shortly afterwards, the well-known book

of Khintchine appeared" , followed by papers by

Feller (for example, Ref. 9) and others.

This is, of course, not intended to be an exhaustive survey. Enough has been said to indicate that

the theory of the Monte Carlo Method has a rather

distinguished pedigree . The novelty which the method

possesses lies chiefly in its point of view. With few

exceptions, the authors cited above proceed from a problem in probabilities to a problem in functional

equations, whose solution is then obtained, or at

least proved to exist, by classical methods and furnishes the answer to the probability problem. In the

Monte Carlo Method, the situation is reversed. The

probability problem (whose solution can always be

approximated by repeated trials) is regarded as the

tool for the numerical analysis of a functional equation. Or alternatively, in a physical problem which,

classically, would call for an analytic model, the

equivalent probability problem is regarded as an

adequate model in itself, and derivation of an analytic

equivalent is considered to be superfluous. The use

of probability theory in this way to solve physical

problems was apparently first suggested by von Neumann and Ulam.

But it is worth noting that the Monte Carlo Method

is not at all novel to statisticians, as was pointed out i by Prof. John Wishart at a symposium on the Monte ! Carlo Method held in 1949 at Los Angeles, Calif." . ! For more than fifty years, when statisticians have

been confronted with a difficult problem in distribu- i tion theory, they have resorted to what they have

sometimes called "model sampling". This process

consists of setting up some sort of urn model or system, or drawings from a table of random numbers,

whereby the statistic whose distribution is sought ' can be observed over and over again and the distribution estimated empirically. The theoretical distribution in question is usually a multiple integral

over a peculiar region in many dimensions, so, in

such cases, "model sampling" is clearly a Monte i Carlo Method of numerical quadrature. In fact, the i distribution of "Student's t" was first determined in

this way. Many other examples can be found by

leafing through the pages of Biometrika and the other

statistical journals*

Most of the technical literature referred to above

on the relation between probabilities and functional

equations deals with equations which can loosely be I described as of parabolic type. The amount of pub- \ lished material on the elliptic case is relatively

meager, and for that reason this case has been

chosen as the central topic of this paper. The basic

references at the moment seem to be 5, 13, 16, 20, ' 23, and the ever-growing literature on sequential ' analysis, summarized to 1946 by Waldos. *> Several

other references will be given as the exposition ' proceeds.

I A General Remark 3

J

1 It was observed by Courant, Friedrichs, and i

I

!

a. It should also be mentioned that the computation i of important constants such as tt by statistical

| sampling methods is an old trick mentioned in

many textbooks. An interesting computation of tt of this type was given by Lexis in Ref. 15, pp.

| 161-163, where the computation is based on statistical data on the sex ratio at birth. There, '

Lexis gives a reference to Fechner who did the

same thing earlier using psychological observa- | tions.

b. An interesting expository treatment of problems ! related to those considered in this paper has been

given by Polya^^ . It is notable that Polya also

considered there a game of chance leading to a

hyperbolic partial differential equation - a matter

which has to date received practically no attention

j in the literature. The author is indebted to Prof. I Polya for calling his attention to this reference

and to the one in the preceding footnote. J

A'.
Lewy^ that the type of stochastic problem appropriate

to elliptic differential equations is superficially quite

different from the one which is naturally associated

with parabolic equations. The reason is that, in the

typical elliptic case, interest is centered on determining an unknown function throughout a bounded

region in terms of known values given on a closed

boundary. It is, therefore, to be expected that the

boundary will somehow play a unique and peculiar

role in the probabilistic formulation.

But actually, the peculiarity of the elliptic case

is more apparent than real. The usual problem in

stochastic processes may be described as follows:

A process described by its transition probabilities runs on for m steps, or for a time t, where m or t are fixed in advance. Then the process is suddenly

terminated. It is required to find the probabilities

of the various terminal states. However, it is not

considered to be out of the spirit of the game to place

traps (e. g. absorbing barriers) along the way, which

terminate the process automatically if it somehow

falls into them. In the elliptic case, all possible

terminal states consist of traps placed on the boundary

of the region in question, and no time limit is set at

all. Thus the elliptic problem is merely a special

caseof the usual problem with m or t infinite. Physically, it corresponds to the existence of a steady

state.

These intuitively phrased considerations will be

illustrated mathematically in Section 7 below.

The Gambler's Ruin 4

The starting point of the discussion will be a classical problem in games of chance, which, according

to Uspensky,* was first solved by Huygens.

The problem is this: Two players G and G' play

a game consisting of a series of turns. The probability that G wins a turn is constant throughout the

game. The stake in each turn is $h. What is the

probability that G ruins G' before he himself is

ruined ?

The problem can be restated graphically. Let p

be the probability that G wins a turn, and q the probability that G' wins a turn; p + q= 1. Let $g be the

wealth of G at the start of the game and let $ g' be

that of G' ; it is assumed that g and g' are integral

multiples of h. Plot the points g and b = g + g' on

the X axis and divide up the interval (0, b) into equal

subintervals of length h.

a- See Ref. 25, pp. 139, where existence of the solution is proved and formulas for the solution are

derived for the case h = 1.

A particle whose position represents the wealth of

G at any moment in the game starts at g and performs

a random walk in which the probability of a step to

the right is always p and to the left is always q. Each

step is of length h. What is the probability that the

point reaches the right hand endpoint of the interval

before it reaches the left hand endpoint ?

Probabilities in infinite series of trials can be

tricky, and so the existence of a solution requires

attention first. Let x be the fortune of G at any point

in the game, and let vjx) denote the probability that

he ruins G' before getting ruined himself in, at most,

m games, where m is for the moment fixed. This

probability certainly exists, by any reasonable definition of probability. It can be broken down into

two mutually exclusive cases: (1) G wins the first

turn and ruins G' within the next m - 1 turns; (2) G

loses the first turn and ruins G' within the next m - 1 turns. This leads to the difference equation

vjx) = p v„_i(x + h) + q v„_i(x - h) 0<x<b ,

vjO) = 0 , vjb) = 1 . (1)

This is a simple special case of the fundamental

Chapman-Kolmogorov integral equation" of Markoff

stochastic processes, which forms the starting point

for many of the recent mathematical researches re- ferred to in Section 2.

Now

VTM. i(x) - vjx) = p[ vjx + h) - v„.i(x + h) ]

+ q [ vjx - h) - v^.i(x - h) ] ,

so if we can show that vjx) ^ v^_i(x) for some

mg and for all x, then it follows that this is true for

all m>mo. But Vo(x) = 0, 0<x<b, and

Vo(x) = 1, x^ b, and by (1),

so the inductive chain is started with m,, = 1. Therefore, Vq, v^, Vj ,

. . . , is a monotonic sequence

with unity as an upper bound. Thus, by a familiar

result in analysis, lim vjx) = v(x) exists, 0 < x ^ b.

It is this limit which we call the probability of ruin.

It is a fact that if G does not ruin G' in a game,

then he will get ruined himself with probability 1;

b. It can be written as

where F(||x) denotes the probability that a single

step starting at x in the random walk will make

the particle reach the interval ^ ^ x.
that is, the probability that the game will go on forever is zero. We shall defer the proof until Section

11.

Taking the limit on both sides of (1) we obtain

v(x) = p v(x + h) + q v(x - h) , v(0) = 0, v(b) = 1 .

(2)

The solution of this boundary value problem can be

found by the usual methods of the theory of difference

equations. It is:

-x/h

v(x) =- (p/q)

or

1 - (p/q)

v(x) = x/b ,

^bTh

q = P

=

(3a)

(3b)

The answer to the original question is obtained by

letting x = g in these formulas. It is seen that if the

wealth of G is large compared to that of G' , then G

will have a good chance of ruining G' , even if each

turn is disadvantageous to G.

Passage to the Differential Equation 5

It is natural to wonder what happens if the stake

h in each turn is decreased while the initial wealth

of each player is held constant. In the case p = q = a, formula (3b) seems to indicate that this would have

no effect on the probability of ruin. If p ^ q, a glance at (3a) reveals that, if the stake is very small,

the gambler to whom the turns are unfavorable almost surely is going to be ruined. To obtain interesting results in the case, the risks must be modified as the stakes grow smaller, so as to make them

more nearly equal.

The situation is reflected in the difference equation

(2) when passage to the corresponding differential

equation is attempted. The equation can be written

in terms of difference quotients, as follows:

A^v(x) + P^"^^ Av(x) = 0 , 0<x<b (4)

where

L v(x) = [ v(x + h) - v(x) ] /h and

A2v(x) = [ Av(x) - Av(x - h) ] /h

= [ v(x + h) + v(x - h) - 2v(x) ] /h2 .

Since formally as h — 0 , Av(x)-^ and A^v(x)-^ ,

the difference equation goes formally into a differential equation if the coefficient of A v(x) approaches

a limit.

An obvious way to arrange this is to stipulate that

the odds shall be connected with the stakes in such a way that p - q and h are of about the same magnitude.

That is, let

p - q = h Q! + o(h) (5)

where is a constantand o(h) denotes an infinitesimal of higher order than h. By using the relation

p + q = 1, it is easy to show that (5) implies that

-E* = 1 + 2 a h + o(h)

q

(6)

Then the difference equation (4) goes over formally

into

dfu

dx2

o du . + 2 a — = 0 dx (7)

u(0) = 0 , u(b) = 1 . The solution of (7) is easily found to be

1 -2Q!x

u(x) = -2ab

1 - e Substituting (6) into (3a) and letting h approach zero,

it is seen that v(x) = u(x) . h-0

Therefore, not only does the difference equation

formally go over into the differential equation, but

the solution of one rigorously approaches that of the

other.

In the case p = q, the second term in (7) is absent

and the solution of the difference equation and differential equations are identical. This is to be

anticipated from the fact that the solution of the difference equation was independent of h.

A More General Ordinary Differential Equation 6

The development in the preceding section proceeded in the classical way, i. e. , from a probability

problem to a functional equation, which was then

solved analytically. But in this paper, the goal is

to investigate the possibility of reversing the argument. It is clear that the probability problem gives

an exact solution of the boundary value problem (2) or (4) and an approximate solution of the boundary

value problem (7) if h is small and p and q are nearly

equal. Moreover, if the game were played over and

over many times, and the relative frequency of times

in which G' was ruined rather than G were recorded,

this relative frequency would be an estimate of the

solution at the point x = g. Thus, the game clearly provides a method for

approximate numerical quadrature of (7). The re- mainder of the paper is devoted to study of the method

.

Consider, now, the boundary value problem given

by

U(a) = A , U(b) = B ,

(8)

i
where a =a(x) and /3 = |3(x) are uniformly bounded

functions of x with ]3(x) > d> 0 for a < x < b .* Let a lattice or mesh of equally spaced points now

be superimposed on the x-axis, dividing it into subintervals of length h. (The number h is called the

mesh constant of the lattice . ) A problem in difference

equations corresponding to (8) is

L(V)= ^ A^V + 2a AV = 0 , a<x<b , (9)

V(x) = {

A, X < a

B, X > b . Rearranging the terms and solving for V(x), we obtain

V(x) = p(x) V(x + h) + q(x) V(x - h) , a < x < b , (10)

where

p(x) =

^(x) + 2h a (x)

D(x)

q(x) = D(x)

D(x) = 2 ^(x) + 2h a(x) .

(If a = a (x) is sometimes negative, we assume that

2h lal< d, a < X < b.)

It will be noted that p(x) + q(x) = 1 and that (10)

looks very much like (2). This suggests that a

probability model analogous to the problem of the

gambler's ruin exists for (10), and that p and q can

again be interpreted as the transition probabilities

in a random walk.

In fact, consider the following situation: A particle starts at a point Xo lying on the lattice and

inside the interval [a,b], and performs a random

walk on the lattice. The conditions of the walk are

that if the particle is momentarily at the point x, the

probability of a step of length h to the right is p(x)

and the probability of a step of length h in the other

direction is q(x), where p and q are given by the

above formulas. If the particle reaches the interval

X < aor the interval x 5^ b, the walk is stopped. In

the former case, a score A is tallied; in the latter

case, a score B is tallied. What is the mean value

of the score?

Let vjx,b) be the probability of reaching the interval X 3^ b before arriving at x ^ a if the walk is

allowed to continue for only m steps, and let vjx,a)

be the probability of reaching x < a before reaching x 3^ b in, at most m steps. Let Vjx) be the mean

a. It is assumed whenever a differential equation is

written down in this paper that the coefficients

possess sufficient regularity for the solution to

exist.

value if, at the end of the walk, B is tallied forx ^ b,

and A is tallied for x < a, and 0 is tallied if the

particle is still in the interval [ a,b] after m steps.

Then

V„(x) = Bv„(x,b)+AvJx,a) . The function vjx,b) satisfies a Chapman-Kolmogorov equation:

vjx, b) = p(x) v„.i(x + h, b) + q(x) v„_i(x - h, b), (11)

a<x<b ,

- , , , 0, X < a vjx,b) = {j; ^ and a similar one is satisfied by vjx,a). Since

Vjx) is a linear combination of functions satisfying

the linear equation (11), VJx) itself satisfies a similar Chapman-Kolmogorov equation:

V» = p(x) V„., (x+h) + q(x) V„., (x-h), (12)

a < X < b,

V (X) 4^' ^

Just as in Section 4, it can now be shown that

lim v„(x,b) = v(x,b)

m — 00

exists, and lim vjx,a) = v(x,a)

m — 00

exists, and, therefore,

lim Yjx) = V(x) = Bv(x,b)

m— 00

+ Av(x, a) exists.

Therefore, taking the limit in (12), it is seen that

V(x) satisfies (10) and (9).

The mean value V(x) considered in the preceding

paragraph is the mean value of scores A, B and 0,

tallied respectively if the particle reaches x ^ a before

X 5^b, or x^h before x < a, or never reaches either

X ^a or X 5^ b. But it will be shown in Section 11

that the last case occurs with zero probability, so

the only cases which enter into the mean value are

the first two.

Thus, it is seen that the mean value of the tally

in the random walk problem furnishes a solution to

the boundary value problem (9) at the point Xq.

Several comments are now in order.

1. As in the limiting case of the gambler's ruin,

it will be noticed that p(x) - q(x) is of the same order

as h; in fact,

p(x) -q(x) =h +o(h).

It is to be noted that the difference equation (9) when

placed in the form (10), seems automatically to select the probabilities of the appropriate equivalent
game. This is true also for the cases of higher

dimensionality.

2. It is of interest to observe that, inasmuch as

V(x) is the weighted mean of V(x + h) and V(x - h), it must lie between these two values. Therefore, it can have no relative maxima and minima inside the interval [a, b], and its value must always lie in the interval

[A,B] or [B,A] defined by the boundary values. This

means that if A = B = 0, then V(x) = 0. Thus there

cannot be two solutions V(x) and V*(x), V(x) ^Y*{x),

both satisfying (10) with the same boundary values,

because their difference would satisfy (10) with zero

boundary values, and would then be identically zero.

3. In carrying out the successive approximations

defined by (12), it is actually possible to start with

any initial approximation Vq (x) (see Ref. 23, pp.

184-185). This can be used as the basis for a nu- merical procedure known as the Liebmann Method

,

which is particularly useful in a higher number of

dimensions. The Liebmann Method, of course, does

not involve probability consideration.

4. There is another interpretation of (10) which

can also be used as the basis of a non-probabilistic

numerical technique. It is as follows: A mass of

substance concentrated originally at x begins to diffuse along the x-axis. During the first second, p(x)

of the material goes to x + h, and q(x) goes to x - h.

During the second second, p(x t h) of the material at x + h goes to x+ 2h, and q(x + h) goes to x + h - h = x. Also p(x - h) of the material at x - h goes to x and

q(x - h) goes to x - 2h. This continues until all the

material has left the interval a < x < b. The proportion reaching x 5^ b is noted; let this be P,,. The

proportion reaching x ^ a is also noted; let this be

. Then V(x) = A P^ + B Pi,. When reduced to

computing practice this method has been called the

"census method"^^.

A Related Parabolic Differential Equation 7

The case in which the random walk is restricted

to, at most, m steps will now be considered again.

A time variable t is now introduced by the expedient

of letting t = m X, where A is interpreted as the time

interval for a step of the walk. Let V(x,t) = Vjx).

It is instructive to write (12) in the equivalent form

V(x,t + X) - V(x,t)

X

= ^ rv(x + h,t) + V(x - h,t) - 2V(x,t)1 (13)

D L ^ J

+ 2^ [V(x + h,t) - V(x,t)], t> 0, a <x<b,

/'A, X ^ a, t ^ 0

V(x,t) =< B, X ^ b, t ^ 0

I 0 , a < X < b, t = 0.

If X = h^, and then X — 0, this equation goes formally over into the parabolic differential equation

ay_i 3^ a(x) 3V

9t "

2 3x2 +^(x) ax • (14)

It can, in fact, be proved under suitable restrictions

that the solution of (14) satisfying the boundary conditions stated above does exist, and that the limiting

solution of (12) or (13) is the solution of this equation

(see Ref. 13, pp. 24-31 and Ref. 9). The equation

is closely related to the Fokker -Planck equation of

mathematical physics. In the diffusion interpretation

of the random walk with A = B = 1, the equation yields an expression for the rate |^ at which the material

disappears into the boundary of the infinite spacetime strip a<x<b, t ^ 0,as time goes on.

If, now, in (13), we let t oc first, with a fixed

X = h^, then the left side approaches zero, and,

in the limit, the equation is identical with (9). The

formal limit as X — 0 is now the differential equation

(8). It is again possible to prove that the form has

substance: as X — 0 the solution of (9) does approach

the solution of (8). This will be discussed more fully

in the next section.

In later sections it will be seen that the natural

generalization of the problem considered in Section

6 to several space variables x, y, z, . . . , consists

in the replacement of the expression on the left side

of (8) (or the right side of (14)) by an elliptic differential operator. The developments of the present

section, although purely formal, show why the problems considered in this paper may be considered as

limiting cases of the possibly more familiar probability problems associated with parabolic operators.

The relationship was previously discussed in general

terms in Section 3.

Generalization of the Stochastic Process and Passage

to the Differential Equation 8

It can be proved in various ways that, as h approaches zero, the solution of the difference equation

(9) approaches that of the differential equation (8).

(See Ref. 10, pp. 160-166.) A completely general

proof will not be given in this paper, although Section

13 below contains the essence of such a proof.

In considering the asymptotic characteristics of

the random walk of Section 6, it is natural to inquire

whether it provides the only probabilistic approach

to the solution of (8). The question was answered in

the negative by Petrowsky^", who established an asymptotic connection between a broad class of stochastic processes and (8). His result is, in a sense,

fundamental to this paper. Although limitations of

space preclude giving the details of his proof here

(it is rather complicated, but is readily accessible

in Ref. 13), it, nevertheless, seems to be desirable.
at least, to state his theorem and show that it incidentally establishes that the solution of (9) approaches

that of (8).

A parameter X is again introduced, which may

be considered to represent the time interval between

observations on the state of the process. The asymptotic character of the process will now be made

to depend entirely on X, and the lattice previously

used to set up the difference equation in Sections 4

and 6 will not play a role here at all. The states will

again be represented by the successive abscissas of

amoving particle on the x-axis, and thus the process

maybe considered as a general form of random walk.

The length and direction of the kth step taken by the

particle will be denoted by a random variable X^.

It will be assumed that the distribution of depends only on X and on the position x of the initial

point from which the step is taken, but not on k nor

in any other way on the past history of the process.

This is the characteristic property of a Markoff

process.

The distribution of is called the transition distribution of the stochastic process. The distribution

function of the transition distribution will be denoted

by F^(^lx); this is the probability that X^ is less

than or equal to £ - x if the kth step started at x. We

assume (merely to simplify statements) that F^(f I x)

is defined for all x, a < x < b.

The first and second moments about the point x

and third absolute moment of the transition distribution will be denoted respectively by M^(x), D^(x),

T^(x); they are given by the formulas

M^{x) = /_'^(^-x) df F^(flx) ,

X -co A

T^(x) =/_'^U-xl3df F^(^lx) .

Now let the random walk start at a point x on the

interval a < x < b and proceed until it reaches the

interval x » b or the interval x < a. If it reaches the

interval x$ a first, a quantity A is tallied. If it reaches X ^ b first, a quantity B is tallied. Let V(x)

be the mean value of the tally, assumed for the moment to exist. It is to be noted that since the transition distribution depends on X, so does V(x).

The theorem is then as follows:

Theorem 1:' Let M_AAA (x), D (x), T (x) exist for

a < x < 0 and let the distribution of X^ be such that

M (x) = X k(x) a(x) + o(X)

A

D^(x) = X k(x) ^(x) + o(X)

T^(x) = o(X),

uniformly in x, where oi(x), /3(x) and k(x) are continuous functions and ^(x) > 0, k(x) > 0, a ^ x ^ b.

Furthermore, let there exist positive numbers e and 77 independent of x such that

A

1 - FXX(x + e I X) = Prob (X, > e IX, = x) ^ ^7^.", (b)

A A

Then as X — 0, the mean value V(x) approaches the

solution U - U(x) of the boundary value problem

MX) ^-1?. 2a (X) 3^ = 0, dx

U(a) = A, U(b) = b,

uniformly in the interval a ^ x =e b.

It is easily seen that the random walk considered

in Section 6 satisfies the condition of the theorem,

provided that X = h^ and a(x) and ^(x) have the

smoothness properties required by the theorem.

For

M^(x) = h p(x) - h q(x) = 1^ = h^ - . a + oCh^),

D^(x) = h2 p(x) + h2 q(x) = h2 =h^-j-P

Tx(x) = h3 p(x) + h3 q(x) = h^ = o(h2).

Setting X = h2 and letting k(x) = 1/^ (x), we see

that the condition (a) in the statement of the theorem

is satisfied. Also

I'

which is bounded- from zero for all x because the

numerator of p(x) is supposed to have that property.

Therefore, condition (b) is satisfied with

6^ = h/2 = x/2 and tj^

= p(x) = (

|3 + 2^Xa )/(2/3 + 2^Xa )

.

It will be noted that the starting point of the walk in

Section 6 was not restricted to any particular denumerable sequence of lattice points, but rather was

Prob (X, > ^Ix) = p(x).

a . Petrowsky ^ proves the theorem with the condition on the third moment given here replaced by a Lindeberg condition, which is weaker than the

present one. He considers only the case in which

k(x) = 1, A = 0, B = 1; but the extension to the

present case is trivial.

b. By Prob. (MIN) is meant the conditional probability of the event M, given that N has happened.

Condition (b) means that the probability is at least

in any step that starts at x, in which the abscissa of the moving particle is increased by more

than e ^ .
permitted to be any point of the interval. This implies

that for each h the transition distribution function of

the process is defined for all values of x in [a,b].

Let V(x, t) be the expected tally if the random walk

is limited to, at most, t = mX steps, and if A is

scored when the region x < a is reached, B is scored

when X 5^ b is reached, and 0 is scored if the particle

is still in the interval a < x < b at time t. Then , exactly as in Section 6, a Chapman-Kolmogorov

equation is obtained:

V(x,t+X) = /.«V(^,t) d^F^aix),

t » 0, a < X < b,

(15)

{A, t > 0, X < a

B, t ^ 0, X ^ b

0, t = 0, a <x< b.

The existence of the limit of V(x)» as t - oo follows

by induction in the same way as before. It will be

shown in Section 11 that the probability of remaining

in a< x <b is zero. The limit function satisfies an

equation similar to (15), and the proof of Theorem 1 consists in showing that this equation is asymptotically equivalent to the differential equation.

Equation (15) may be written as follows:

- V(x,t)]d^F^(|lx).

In this form it is a direct analogue of (13). Under

suitable conditions (see Ref. 13, pp. 24-31) as X — 0

the integral equation will go formally over into the

parabolic differential equation (14) and the solution

of the one will equal that of the other.

General Dirichlet Problems for Differential and

Difference Equations 9

The results of the preceding sections are of some

theoretical interest, but, probably, have little significance for practical numerical analysis. Fortunately, however, they all generalize immediately

to spaces of two or more dimensions, where their

importance for computational techniques is much

greater because of the well-known difficulties of attacking partial differential equations in many variables by numerical methods.

For simplicity, the exposition here will be restricted principally to two dimensions. The arguments

a. It should be noted that both V(x) and V(x,t) for

fixed t depend on X because the transition distribution does.

and results carry over into spaces of higher dimensionality without essential change.'

The central problem will be the following one,

which we shall call Problem D:

Let C denote a Jordan curve of the finite plane;

let R denote the region'' interior to C;

let R = R + C; let <(> (x,y)

be a given function piecewise continuous on C. Required, to find for all points in R that solution U(x,y)

of the elliptic differential equation

dxdy

au

(16)

+ 2a^^ + 2a 8U

ax ""2 ay

= 0

for which U(x,y)= (/)(x,y), (x,y) on C, where jS^,

^i2»/322>"i»^2> a-re functions of x and y continuous

and with continuous first and second derivatives in a

region containing R, with

^ii> 0, /322 > 0, /3ii )322 -

|3f2 > 0 in R.

This is the Dirichlet problem, or first boundary

value problem of potential theory, for a homogeneous

elliptic differential equation of a rather general type.

A rectangular lattice of points of the (x, y) plane

is now selected, consisting of the vertices of squares

of side h formed by the system of lines

X = Xo+ jh, y = yo + jh, j = 0,± 1, ±2,...

Here Xg and y,, are chosen conveniently, and h is the

mesh constant.

If h is sufficiently small, (and we henceforth as- sume this to be the case) then the lattice points in R

will form a single, uniquely determined lattice region

R^, with the property that any point of R^ can be connected to a fixed preassigned point of R^ by a chain

of lattice points lying in R^. By a chain of lattice

points is meant a sequence of lattice points in which

each point (after the first) follows one of its four

neighbors (i.e., the four points at distance h)?

a. It is true that there are important differences in

the theory in two dimensions and in higher dimensions when infinite regions are considered, as is

pointed out in (21) . But here we shall be concerned

only with bounded regions.

jj That is, a one-to-one continuous transform of a circumference.

c. By region is meant an open connected set.

d. It is not assumed that the line segments joining

successive points in a chain must lie entirely in

R, although this will be true if h is small enough.
Certain neighbors of some of the points in will not

be included in R^. Furthermore, the point (x + h, y + h)

will not be in R^ for all (x,y) in R^. Let be the

set of missing neighbors plus the missing points

(x + h, y_+ h), belonging to all points in R^. Then

the set Rh= Rh+ has the following properties:

1. Every lattice point in R is included in R^.

2. All points of lie on or outside of C.

3. Each point (x,y) of R^ is an interior point, in

the sense that all four of its neighbors and the point

(x + h, y + h) are present in R^h- C^. '

4. Any point in R^ can be connected to any point

in R^ by a chain of neighbors, which lie in R except

possibly for the last point in the chain.

It is of interest to note that, given any point (x,y),

if h is sufficiently small, a uniquely determined lattice region R^ containing (x,y) with properties (1),

(2), (3), (4) exists for all smaller values of h." If the point (x,y) is restricted to lie on an arbitrary but

fixed closed point set interior to R, then there is a minimum value of h > 0, such that this situation holds

for all h < ho uniformly for every point on the closed

point set.

Differences in the x and y directions will be symbolized as follows:

A_V = V(x + h,y) - V(x,y)

V(x + h.v) + V(x-h.v)-2V(x.v)

^ „ V(x + h,y + h)-V(x + h,y)- V(x,y + h)+V(x,y)

V(x,y + h) - V(x,y)

„_ V(x,y + h) + V(x,y-h)-2V(x,y)

h^

The argument (x,y) will frequently be denoted by P

or Q. The neighbors of P in counter-clockwise order,

starting with (x + h, y) will be denoted by subscripts

1, 2, 3, 4 and the point (x t h, y + h) by the subscript

5.

^ Some of the points of may be interior points

in this sense also.

b. A proof of the existence of R^ with properties

(1) - (4) for some h sufficiently small and for all

values of h smaller, can be given along the lines

indicated in Ref. 29, pp. 7-9, where it is shown

that the region R can be approximated arbitrarily

closely by a simply connected region formed out

of squares of side h.

Finally, we let

L(V)=|3,,A,,V+2^,,A,,V + ^3,A„V

+ 2a, A3 V+ 2a^ A, V , where the coefficients of the differences have the

same properties as the coefficients of (16).

In the difference equation formulation, Problem

D now becomes the following, which we call Problem

Let 0(x,y) be given on C^. Required to solve the

difference equation

L(V) = 0 (17)

for (x,y) on R^, subject to the boundary conditions

that

V(x,y) = (p(x,y) on q.

Rearranging the terms of (17) and solving for V(P),

we obtain

V(P) ^ R(P) V(P,) + P2(P) V(P2) + P3(P) V(P3)

+ p,(P) V(P,) + p,(P) V{P,) , where

(18)

P2(P) = 22-

D

2^ 12 + 2ha2

P3(P)

P4(P) =^

P5(P) = and

D

D = D(P) = 2^,1+ 2^22 - 2/3,2 + 2h(a, + a^)

As in the one dimensional case j

^ ) = 1

» but here

Pi(P) or P2(P) may be negative if 2/3i2> /3ii or

2/3i2> ^22 • therefore, assume in the sequel

^^^^

2/3,2< and 2^,2 < ^22 for P in R,^

It is also supposed that h is so small that if a , and

03 are sometimes negative, the quantities Pj(P) are

nevertheless all non-negative in R^.

Consider now the following problem : Let the points

of C^ be denoted by Q, , j = 1, 2, . . . . A particle

starts at the point Pq on R h and performs a random

walk (or random flight) on R^. The conditions of the

a. It is to be noticed that in any case, /3fo< P22

implies that 2/3,2 < /3ii + ^22 ^he inequality

for the geometric and arithmetic means.
walk are that if the particle is momentarily at the

arbitrary point P, the probabilities of stepping to

P„ P2, P3, P„ P^arepilP), P2(P), P3(P), p,(P),

and P5(P) respectively. When the particle reaches

a point Q. the walk is terminated and ^(Q,) is tallied.

What is the mean value of the score ?

Let 00 be the absolute value of a lower bound for

0(Qj) for Qj on C^. The quantity 0(Qj) + 0o is never

negative. The required mean value, if it exists, is

the difference of the corresponding mean values if the tally at is 0(Qj) + 0o, and if the tally at Q. is

00 , provided that these mean values can be shown to

exist. Thus it suffices to establish the existence of

the mean value for a non-negative set of boundary

values, say i//(Qj), which will now be done.

First, let the walk be limited to m steps at most.

Let VJP) be the mean value if 4/{Qj) is tallied if the

walk reaches Qj, j = l, 2, . . . , and zero if the

particle is still on R^^ after m steps.

The function VJP) obviously satisfies for each P

on a Chapman-Kolmogorov equation:

VJP) = Pi(P) V„.,(PJ + P3(P) V„.i(Pj (19)

+ P3(P) V„.i(P3) + P,(P) V„.,(PJ

+ P5(P) V„.i(PJ , PonR,,

VJQ,) = «//(Q,) , j = l, 2,... .

Then

V„.i(P) - VJP) = 2 P,(P)[VJP^) - V„.,(^)] ,

1 = 1

so, if for any m, V„^i^ V„for all P on R^, the inequality will be true for all greater values of m. But

V„(P) = 0 for P on R,, and Vo(Q^) = i//(Q,) , j = 1, 2,

. . . . Since the coefficients Pj, Pg, P3, P4, Pj, are

non-negative, it will follow that Vi(P) ^ Vo(P) for

allPonR^. AlsoO^ bound ;//(Qj). Therefore,

the sequence { V^} is a monotonic bounded sequence

and must have a limit for every P on R^. Taking the

limit in (19), it is seen that the limiting function V(P)

satisfies (17) and (18).

It will be shown in Section 11 that if m = oo, the

probability of tallying a zero is, itself, zero. Therefore, the mean value of the tally in the random walk

problem with the number of steps unlimited furnishes a solution to Problem E^.*

a. McCrea and Whipple" have derived explicit ex- pressions for V(P) in thecase in which R^ is rectangular, 0(QJ = 1, 0(QJ = 0, j = 2, . . . , L(V)= AV = ^„V + AyyV. They also give explicit

expressions for the mean number of visits to a givenpoint P'onR^- The three-dimensional case

and the case of an unbounded are considered.

All of the comments made in Section 6 are again

applicable, with slight changes in the wording to take

care of dimensionality. In particular, V(P) can have

no relative maximum nor minimum on R^ and its

value must always lie in the closed interval bounded

bythe greatestand leastvalue of 0(Qj). If 0(Qj)= 0,

then V(P)= 0; the solution of Problem is accordingly unique.

Generalization of the Stochastic Process and Passage ! to the Partial Differential Equation 10

| As in the one-dimensional case, the function V(P),

j under certain conditions, approaches the solution of

Problem D as h—0. A standard proof is the one ] given by Courant, Friedrich and Lewy In this re- ! ference, the details are given only for the case in

which L(V)= AV= A„V + AyyV , but the proof generalizes to any second-order elliptic difference equation. In more than two dimensions it establishes, 1 only, that the boundary value 0(x, y) are approximated I in the mean.

Feller (see Ref. 23, pp. 187-194) reproduces a | proof given in 1941 by Petrowsky for the case

j L(V)= AV. The proof proceeds^ by showing, first,

that for a sequence of values of h approaching zero,

the sequence of solutions of AV = 0 is equicontinuous j in R^; but the Petrowsky proof then goes on to es- [ tablish, in a way which is valid for any dimension- I ality, that the limit function actually assumes the

j given boundary values as P approaches a point Q onC. I Both of these proofs have the mathematical ad- | vantage that they furnish proofs of the existence of a

j solution to Problem D. But if the point of view is 1 adopted that the existence and uniqueness of the so- j lution to Problem D has already been proved by one 1 method or another, and that the problem is to decide j when a random walk will approximate the solution,

then a formulation of the asymptotic situation can be

given which is the direct generalization of Theorem 1 1 to two or more dimensions. This was done in 1933 ! by Petrowsky.^ His result is a generalization of

' an investigation made a year or two earlier by

Liineberg.

j The essence of the Petrowsky result in twodimen- I sions is this: The states of a stochastic process are

represented by the coordinates of a particle moving

in the (x,y) plane, whose position is observed at time \ interval X. The transition distribution describing

the passage from one state to another is defined by 1 a distribution function F^(^,T]lx,y) which given 1 the conditional probability that the x and y coordinates

of the particle do not exceed ^ and 77 respectively i after a step which started at the point (x,y). This

distribution depends only on (x,y) and X, and not

j on past history, so the process is Markoffian. \ As in Theorem 1 , it is assumed that the first and

second moments of F about x and y are of the order I

I
of ^- A further condition, such as the one on

in Theorem 1 , is added to insure that the tails of the

transition distribution approach zero with A at a

suitable speed. Also, a restriction analagous to

condition (b) in Theorem 1 is imposed on so as

to insure that the probability of the particle remaining in R indefinitely is zero (see Theorem 2 in the

next section).

It is assumed that 0(x,y) is uniformly bounded and

continuous everywhere in the complement of R, or

is at least piecewise continuous there with simple

discontinuities along non-intersecting arcs connecting C to infinity.

Suppose, now, that the random walk starts at a

point P in R and proceeds until it first reaches a

point Q, not in R, whereupon (^(Q) is tallied. Let

V(P) be the mean value of the tally; this mean value

depends on X because the transition distribution does.

The theorem of Petrowsky asserts that under the

conditions here outlined, V(P) approaches the solution of Problem D as X—0.

It turns out, as in the one -dimensional case, that

the special random walk considered in Section 9

satisfies the hypothesis of the theorem.

Thus, there are an infinity of random walk problems equivalent asymptotically to the one in Section

9. In fact, given any Markoffian random walk in a bounded region whose boundaries are treated as absorbing barriers, if the first two moments of the

transition distribution are both of the order of the

time interval required for a single transition, and

if the "value" of each walk corresponds to where the

walk ends on the boundary, then the mean value of

the walk will ordinarily approximate the solution of a Dirichlet problem for the region.

The theorem is proved in two dimensions in Ref

.

20 with the added hypothesis that C consists of a

finite number of arcs, each with continuous curvature. In the two-dimensional case, this restriction

on C can be removed. However, in the case of three

and more dimensions, it is well-known that Problem

D does not have a solution unless the boundary C is

suitably restricted beyond the mere requirement that

it bea one-to-one continuous map of the surface of a sphere. A sufficient condition (see Ref. 12, p. 329)

is that each point Q of C is the vertex of any right

circular cone, which has no points in the part of R

lying in a suitably small spherical neighborhood of Q.

Inspection of Petrowsky's proof reveals that it can

be modified so as to require no more than this condition on C in three and more dimensions.

The Analogue of Green's Function 11

The notation used in the preceding sections for

the one and the two dimensional cases will now be

consolidated. In one dimension, R will denote the

interval a<x<b, R^will denote the points of the

lattice described in Section 6 which lie in the interval a < X < b,

will consist of the two points of the lattice not in

a< X <b but nearest to the interval (one in x =s a and

one in x 5^ b) , P will denote any point of a<x<b

and Q any point of the complement of this interval

with respect to the x-axis, C will consist of the points

X = a and x = b; and the boundary value function

(t> = (/)(x) will have only two values: 0(x) = A, x ^ a,

0(x) = B,x 5^ b. Problem D will mean the one given

by (8) and Problem the one given by (9) . Also it is to be understood in the one dimensional case that

Pi(P) = p(x), P2(P) = q(x), P3= P4= P5= 0. On the

other hand, in the two-dimensional case a and b will

be defined to mean respectively the greatest lower

bound of x for (x,y) in R and the least upper bound of

X for (x, y) in R. The random variable will denote

the X component of the kth step of the walk.

Let the steps in a random walk of any of the types

considered in the preceding sections be numbered

serially, and let the random variable n denote the

serial number of the step which ends the walk. The

mean value of n will be denoted by E(n|Po); it depends on X or h as well as on the starting point P^

of the walk.

Let k be the integral part of the number 1 +(b-a)/€

where e is identified in the statement of the following

theorem.

Theorem 2: If for fixed X or h, positive numbers

e and 77 exist which are independent of P, such that

Prob (X^>e| P) > 77 (20)

for all P in R, then

(a) Prob (n > km) < (1 - tj'^) m = 0, 1 , . . . ,

(b) Prob (n =00) = 0 ,

(c) E(nlP(,) < k/ry" , and is therefore uniformly

bounded for all P^ in R.

To prove the theorem, first consider

Prob (n < k) = 1 - Prob (n > k).

This is the probability that the walk leaves R in, at

most, k steps. Let 6 be the distance from the starting point P of the walk to the nearest boundary point as measured in the direction of increasing x along a

line drawn through P parallel to the x-axis. Then

since 6 < b - a,

Prob (n « k) ^ T?^"^^*^/^ ^ >T7''.

Therefore

Prob (n > k) < 1 - . NowProb(n> mk), m > l,is equal to the probability

that the walk continues inside R for k steps, multiplied by the probability that it does so for k more,
etc. , and so on for m factors. That is

Prob (n > mk) < (1 - 77'')",

as was to be proved for (a)

.

Since Prob (n > j) decreases monotonically with

j, (b) follows at once Finally,

00 00 E(n|Po) = Z j Prob (n = j) = 2 Prob (n > j)

j = 0 i = 0

00 00 2 k Prob (n > kj) < k 2 (1 - 77*^)^ j=o j = 0

Unfortunately, the bound for E(n|Po) given by the

theorem is not in general a good one.**

In the case of a random walk on a lattice R^, condition (20) need be valid only for all P on for the

theorem to be true. In two dimensions, condition

(20) could, of course, be stated in terms of the y

component of a step in the walk instead of the x component, provided that the meanings of a and b were

properly adjusted.

The random walk on R^ considered in Sections 6

and 9 will now be further studied. Let P and P' be

two points of R^, and let G(P, P' ) be the mean value

a. Proved by different methods under less general

conditions, and in one dimension, by Wald.^^'

.

The result is of importance in sequential analysis

because it implies that sequential tests satisfying

very weak conditions always terminate with probability one.

b. It should be noted that the function V(x, mX) of

Section 8 with A = B = 1 gives the probability of

leaving (a, b) in at most m steps, and therefore

Prob (n> m) = 1 - V(x,mX) . Therefore

v(x, km X) > 1 - (1 - tj'')" .

It is also of interest to note that 1 - V(x,mX) is

equal to the probability that the maximum and

minimum of the sums

K

X + 2 Xj, K= 1, 2, . . . ,m

lie within the interval a < x < b. The asymptotic

properties of this probability as m — 00 have

been investigated at some length recently by Erdbs

and Kac,^ Wald^^ and Chung^ in cases in which

the length of the interval (a, b] increases with m.

The fact that V(x,mX) satisfies equation (15) suggests a possibly fruitful approach to such problems.

of the number of times the particle visits P' in the

course of the random walk if it started at P. Let

6(P,P') = 0 if P 7^ P'and6(P,'P) = 1 . Theorem 3: The function G(P,P') satisfies the

difference equation in P,

L(G)=^5^.6(P,P')

with G(Q,P') = 0 for Q on C^.

Here D(P') is the function D defined in connection

with (10) in one dimension and (18) in two. The proof

consists of showing that G(P,P') exists and satisfies

the equation

G(P,P') -6(P,P') = 2 p^(P) G(Pj,P') , 1 which is the same as the equation in the theorem.

Forthe proof, let Pj(P,P') denote the probability

that the random walk ends on P' in exactly i trials,

i = 0, 1, 2, . . .. We note that P, (Q,P') = 0 for Q

on C^. Then

00 G(P,P')= 2 Pj(P,P') , i=o (21)

provided that this series can be shown to converge.

Now Pn,k(P,P') is certainly not greater than the

probability that the walk is still inside the interval

after mk steps. That is,

P„,(P,P') < Prob (n> mk) < (1 - rj^^ , by Theorem 2. This shows that the series in (21) is

dominated by a convergent geometric progression

and therefore must converge.

The probability of ending on P' in the ith step can

be broken down into the usual cases, giving the equation,

P,{P,P')=h p,(P)P,_i(P3,P'), i = l,2,. . . .

Therefore

and

5 ®

P,(P,P') = 2 Pj(P) 2 Pi.,(Pj,P')

G(P,P') =Po(P,P') + 2 Pj(P)G(Pj,P') ,

with G(Q,P') =0, Q on Cj,. Transposing and noting

that Po(P,P') =6(P,P') , we obtain Theorem 3. Theorem 4: The solution of problem with the

equation L(V) = 0 replaced by L(V) = F(x, y) , is given

by

V(P) = W(P) - h2 - G(P,P') F(P')

P'CR, D(P')

where W(P) is the solution of L(W) = 0 with boundary

values W(Q) = (^(Q), Q on C^.

The proof is by direct substitution into the equation

L(V) = F(P), using Theorem 3.
It is evident from the theorem that G(I^P) is analogous to the Green's function of Problem D. It is possible to extend the results discussed in Sections 8 and

10 to show that as h- 0, the solution of L(V) = F

given above approaches the solution of Problem D

with the zero on the right side of (16) replaced by

F(x,y).

A concept closely related to the mean number of

visits to P' , and one which has received a good deal

of attention in the physical literature, is that of the

probability of return to a point which has once been

visited. Let r(P') denote the probability of at least

one return to P' after a visit. Let v(P,P') denote

the probability of reaching P' from P before first

reaching the boundary. Then it is easily shown that

1 v(P,P') l-r(P') = G(P',P') G(P,P')

• (22)

The demonstration will be postponed to Section 15.

It is to be noted that the third member of the equation

is independent of P.*

The Mean Length of the Random Walk 12

Consider the random walk on Rh of Sections 6 and

9, and let the random variable M(P,P') denote the

number of visits to a point P' on R^if the walk starts

at P on R^. The mean value of M(P,P') is G(P,P').

Now the total number of visits to all points of Rh,

counting the start of the walk as a visit to P, is

P'CR, ^^P'P')-

This is, of course, equal to n, the number of steps

in the walk. The sum has only a finite number of

terms, so its mean value is the sum of the mean

values of the terms. Therefore

E(nlP) = P'CR. G(P,P')

With an eye on the statement of Theorem 4, this

formula can be rewritten as follows:

E(nlP) = - h2 P'CR.

D(P')

h2

G(P,P')

D(P')

Theorem 4 then implies the following result:

Theorem 5: The mean value E(nlP) as a function

of P, is a solution of the problem

L[E(n|P)] = -^, PonR,,

E(nlQ) = 0, Q on .

(23)

a. The problem of the limiting value of v(P,P') as

the size of the region R increases is discussed

by Polya^^ .

It is possible to extend the results discussed in

Sections 8 and 10 to show that in the limit as h -» 0,

h2 E(n|P) approaches for each P on R^, the solution

of Problem D with 0 = 0 and with the zero on the right

side of (16) replaced by -2^(x) in one dimension and

by 2 i3i2(x,y) - 2 /3u(x,y) - 2 /3 22(x,y) intwodimensions.*

An important special case is that in which the

coefficients of L(V) are constants. Suppose first

that a ^0, OT + ^ 0. It can then be verified

by direct substitution that the solution of (23) is

E(nlx) = 2§5- [V*(x) - x] (24)

in one dimension, and

E(n|x,y) = 2h^a,+a,) ^^^^^^y) ' + y)] (25)

in two dimensions, where V* is the solution of Problem with boundary values

(/>(x) = X, or 0(x,y) = x + y."

IfOf^^O, then an alternative formula to (25), which

is valid whether or not o{ + = 0, is

D

E(n|x,y) = 21#a,

[V*(x,y) - x]

where now V* is the solution of Problem with

boundary values 0(x,y) = x. There is, of course, a

similar formula involving instead of a^.

If a = 0, or aj = = 0» it can be verified that

the solutions of (23) for one and two dimensions are,

respectively,

E(nlx) = V2 . (26)

E(n|x,y) =

h2

V**(x, y) - (x^ + y^ - xy)

(27)

where V** is the solution of Problem with0(x) = x^ or 0(x,y) = x2 + y2 - xy.

It is interesting to put (24) and (25) in another

form. In the two dimensional case, suppose that all

motion in the random walk takes place in directions

a. Theorem 5 and the statement of this paragraph

recently were generalized by W. Wasow to the

more general random walks considered in Sections

8 and 10. Recently David® has proved that in one

dimension, with identically distributed steps

whose distributions are independent of x and possess a continuous density function, E(nlP) considered as a function of = b - a and Zj = b - x

satisfies a hyperbolic partial differential equation

in Zi and Zg.

b. The distribution of n in case the variables are

independent and have identical bounded discrete

distributions was derived by Wald^'^s in one

dimension and Blackwell and Girschick^ in two.
parallel to the axes; no short cuts are allowed. Then

the term on the right side of each equation outside

of the square bracket is the reciprocal of the mean

displacement in a single step. (This has been established explicitly in one dimension in Section 8,

and follows in two dimensions by inspection of the

definitions of Pj , . . . , Pj given in connection with

(18).) The expression" inside the square bracket"is

the mean value of the total displacement S„ in the

walk because V* is the mean value of the terminal

value of x or X + y. Thus, if we denote the displacement in a single step hy /u , and if we let E(S„|P) be

the mean total displacement if the walk starts at P,

then (24) and (25) become

E(S„IP) =/«E(nlP). (28)

A similar interpretation of (26) and (27) is possible

in terms of squared displacements.

The equation (28) has been studied in one dimension in connection with determining the average

sample number in sequential analysis. Wald^^ established its validity in the case in which the variables

have identical discrete distributions possessing

moment generating functions, and Blackwell ^ proved

that it holds generally if either (a) the are identically distributed or (b) the X^ are all bounded and

have the same mean.'

These results have practical significance in the

solution of differential equations by random walks,

inasmuch as they permit an a priori estimate to be

made of E(n|P) by using the fact that V* and V**

must assume their maxima and minima on (see

Section 13 below), or by using estimates for the solutions of the differential equations satisfied in the

limit. The more general derivations of (28) given

by Wald and by Blackwell suggest that in practice the

use of a more complicated random walk such as those

considered in Sections 8 and 10 in place of the one

on chosen by the difference equation L(V) = 0

itself, might not be profitable because it would not

in general reduce the mean number of steps before

reaching the boundary. However, the more complicated walk might provide a better approximation

to the solution of Problem D for a given value of X.

Simple explicit formulas can be given for E(nlP)

in the one dimensional case in which a and ^ are

constant and the random walk is that of Section 6.

Let a' and b' be the terminal points of the walk in

X « a and x b respectively. By direct substitution

into (23) it can be verified that if a ^ Q, the solution

is

^ ^ x/h a'/h

E(n|x) = t^jp-

[ (b - a ) . - X + a J ;

and if a =0, the solution is

E(n|x) = [ . [ b'- X

]

Thus, for example, if a = a' - 0, b = b' = 50, and

if a random walk with p = q = 1/2 were to be set up on a lattice with a mesh constant of h = 1, and finally

if the starting point x were chosen in the middle of

the interval, then the mean number of steps would

be 25 X 25= 625. This would be less if p q.

The Degree of Approximation of the Solution of the

Difference Equation to that of the Differential Equation". 13

Let U(P) be the solution of Problem D with ^:(U) = 0

replaced by «J! (U) = F(P). The implication in the

statement of Problem D is that U is defined only in

R. Its definition will now be extended by requiring

it to assume the values (/)(Q) on C^, so that the dif-.

ferences in L(U) are defined for every point on R^.

A function € (P) is next introduced, defined on R^ by

the equation

L(U) =^(U) -€(P) .

If U satisfies the equation ,2f (U) = F(P) , then it clearly satisfies the difference equation

L(U) = F(P) - € (P) on R,.

Therefore, by Theorem 4,

U(P)=W(P) -h^p.^j^

+ h2 P'CR,

G(P,PO F(P')

D(P')

G(P,P') 6 (P')

D(P')

~b7R aTh"

e - e

where W(P) is the solution of L(W) = 0 with boundary

values 0(Q). But the first two terms on the right

constitute, by Theorem 4, the solution of L(V) = F(P)

with boundary values 0(Q) on C^. That is,

TTMDN XT(^\ h2 ^ G(P,P')6(P')

U(P) - V(P) = h^ ^ , and . (29)

|U(P)-V(P)U h^e^^ ^l^^

where ^ max is the maximum of e (P) for P on R^.

Another form of this inequality is obtained by noticing

E(n|P)=p,^j^ G(P,P'):

IU(P) - V(P)|< h2 6'^^^E(n|P) ,

where ^'max is the maximum of € (P)/D(P) on R^.

These inequalities are fundamental in studying the

degree of approximation of the solution of Problem

a. The statistical writers do not seem to have noticed

the connection between E(n|P) and the difference

equation (23).

a. Much of the material in this Section was developed

independently by Wasow.
to that of Problem D. They make such a study

depend on estimates of the difference e (P) between ^ (U) and L(U), and on estimates of E(nlP) or the

summation in (29).

The problem of estimating e (P) will be postponed

for a moment.

Two general approaches to the estimation of E(n| P) or of the summation in (29) are possible. The first

is, so to speak, an a posteriori method; that is, the

actual value of E(n|P) can often be estimated statistically with sufficient accuracy for present purposes

after several hundred random walks have been performed. The second method is to use mathematical

a priori estimates. This will now be discussed in

more detail.

In Sectior 12, explicit formulas were given for

E(niP) in certain cases. From these, and from the

fact that the values of V* and V** must lie between

the maxima and minima of the boundary values of

these functions, bounds for E(n|P) can easily be

found as was previously mentioned in Section 12.

For example, in the case of formula (27),

The first two properties, by Theorem 4, yield the

relation

E(n|x,y)

max (x'2+ y'2- x'y')-(x2+ y2 - xy)] ^

(x^y') on Ch

The summation in (29) multiplied by h^ is, by

Theorem 4, the solution of L(V) = -1 with boundary

values 0(x,y) = 0 on C^. Although it maybe difficult

to arrive at a reasonable bound for the solution of

L(V) = - 1 in general cases, nevertheless the following device is often feasible: Find a function z(P)

such that L(z) ^ > 0 on R. . Then

P'CR.

G(P,P^)

D(P')

max

Q on C,

lz(Q) - z(P)|

(30)

If, for instance, a^{P)'> 0 on , then a possible

z(P) would be simply z(P) = z(x, y) = x. For in this

case, L(x) = 2a^{x,y), and Lq can be taken as the

minimum of 2a ^{P) on R^. Or again, if oi i{x,y) is

positive for x > x,, and negative for x < x^, then z(P) can be taken as (x - Xo)^. Many such functions z(P) can usually be found in any given case. The goodness

of the resulting estimate depends on a skillful choice

of z(P).

The proof of (30) is as follows: Let Z(P) denote

the solution of L(Z) = 0 with the boundary values z(Q)

on C^. Then Z*(P) =Z(P) - z(P) has these properties:

(1) Z*(Q) = 0, Q on C,.

(2) L(Z*) = -L(z), where L(z) ^ L„ > 0 on R,.

(3) iz*(P)i ^cfif„v - • \>i on

Z*(P) = h2 2

P'CR

5 Lo h2 2

G(P,P') . L(z(P'))

D(P')

G(P,P')

P CR. D(P')

Combining this with the third property, (30) is obtained at once.^

The problem of estimating ^ max (29) will now

be considered. It is known that under the conditions

in the statement of Problem D in Section 9, and under

the further condition that F(P) has continuous first

and second derivatives in a region containing R, the

solution of »<'(U) = F(P)will have continuous first and

second derivatives in R. Furthermore, in the one

dimensional case, if the lattice is so arranged that = C, then the solution of /(U) = F(x) is continuous

in a closed interval containing R^ and has continuous

first and second derivatives in the corresponding

open interval. It also takes on the same boundary

values at the same points specified in Problem . In the remainder of this section it is assumed that

the one-dimensional lattice has been so arranged.

The analogous situation in two and more dimensions is not so simply achieved. Without further

discussion of the mathematics involved, it will be

assumed henceforth that 0(x, y) and C are such that

(1) 0 is defined in the complement of R, or at least

in an annular closed region surrounding R, and (2).

K the definition of U, the solution of /(U) = F(P) in

R, is extended by making U identically equal to 0

wherever 0 is defined exterior to R, then U has continuous first and second derivatives in a_region containing R (and, therefore, containing R^ for all h

sufficiently small). The statement of Problem

for L(V) = F(P) given in Section 9 and Theorem 4

now implies that U and V coincide on • Both this

and the conUnuity of the derivatives of U in a region

containing R^ are essential for what follows. ^ The demonstration will be given in two dimensions

;

specialization to one dimension is immediate. By

an application of Taylor's Theorem with remainder,

a. The exposition in this paragraph and the preceding

one follows closely a discussion given by Wasow

in a letter to the author.

b. It would be possible, of course, to redefine R^ so

that it lies entirely in R, and then to extend the

definition of 0 into the interior of R appropriately.

The procedure in the text of having lie outside

R has been followed merely for convenience of

exposition in indicating the relationship with

Petrowsky's theorem.
it is easily shown that, for example,

A.. U =

3x2 61, (P) , P on R, , where le^^l is less than or equal to the maximum

absolute deviation of on the line segment P3P1

3x2

- ^ 3^ from its value at P. Similar expressions for the

other differences of U can be derived. Operating

now on U with L, it is found that the quantity

6 (P) =^(U) - L(U)

can be written as follows:

€ (P) = i3ii + 2^12 €12 + ^22 ^22 + 20(1 €10

where c^^, e^^, etc., are functions of P whose

respective bounds for each individual P on are

listed in Table I. The abbreviation "max. dev. "in

Function Bound for absolute value

max. dev.T—5- on P.P,

3x2 3

1

€12

1/ 32U . slmax. osc.g^g in square

PP.P^PJ +i(max.osc.|^

in same square) + max. dev.

32U .in same square.

3x3 y

^22 3^ U

max. dev. 5- on P. P, 3y2 4 2

max. dev.^ on P P

dx

£01

max. dev. on P P,

3y 2

Table I

the second column of this table means the maximum

possible absolute deviation of the value of the designated function from its value at P, and "max. ocs. " means the maximum oscillation of the functions;

i.e., for a function f(P) the maximum of If (P') - f(P")|

for P' and P" on the designated set of points.

An upper bound for e (P) on R^, and therefore for

^max' can be obtained by replacing the e 's in the

expression for € (P) by the overall maxima on R^ of

their respective bounds as given in the table, and

replacing the coefficients in € (P) by their maxima

on R.

It is now an obvious consequence of the assumptions on the continuity of the derivatives of U that

lim

h«o ^max = 0

uniformly on R. If the existence and uniform continuity of the second derivatives of U in some region

containing R (or merely in the interval a < x^ b in

the one dimensional case) is considered an acceptable

assumption, and if an a priori estimate of E(n|P) or

of the summation in (29) is at hand, which, when

multiplied by h2 , is bounded as h approaches zero , then the discussion in this section maybe considered

to provide a proof that as h approaches zero, the

solution of Problem approaches that of Problem

D. These assumptions are certainly not all fulfilled

in certain cases admitted in the original statement

of Problem D, as, for instance, when 0(x,y) is only

piecewise continuous. Limitations of space prevent

further discussion of such matters, and the material

in this section is presented accordingly from the

point of view of providing a basis for estimating the

degree of approximation of V(P) to U(P) in special

cases, rather than from that of providing a definitive

asymptotic treatment of the solution of Problem . Such a treatment is, in fact, furnished by Petrowsky's

work. 2°

The Dispersion of the Statistical Estimate of V(P)

and the Number of Samples Necessary to Achieve a Given Accuracy. 14

The Monte Carlo solution of Problem D consists

in (1) assuming that for a fine enough mesh, or a small enough value of X, the theoretical mean value

V(P) gives a satisfactory approximation to the solution

of the differential equation, and (2)- estimating V(P)

by one of the usual estimators employed in statistics.

The most common estimator is the simple average,

or arithmetic mean, of the N tallies obtained in N

random walks, and we shall consider here only this

estimator.

The argument which follows applies to the general

random walks considered in Sections 8 and 10.

Let 01,02, .••.0N be independent, identically

distributed, random variables representing the tally

after each of the N walks, which are all assumed to

start from the fixed point P on R^. Then for j = 1,

. . . , N

E(0.)=V(P) ,

1

1

i

a. Throughout this paper, the symbol E(X) will be

used for the mean value of the random variable X.
and the variance * 0^ ) is given by

a2(0^) = E[0. - V(P)]2

(31) ^ E [ 0j - M]2 ^ bound [ 0(Q) - M ]2 ,

where Q in the last member is any point exterior to

R on which the walk can end with non- zero probability,

and M is any real number whatsoever. The inequality

follows from the fact that the second moment of any

distribution is least about the arithmetic mean.

The statistical estimate of V(P) will be, a priori,

f = ^ •

Its mean value is, of course, V(P), and its variance

is given by

bound [0(Q) - M]^

N

(32)

The estimator 0 will be nearly normally distributed

for N 5^ 100, by the Central Limit Theorem of probability theory, so Prob {|0 - V(P)1> 2a(0)}

will be about 0.05. If this is considered a small

enough margin of error, and if it is desired to compute V(P) to, say, m decimal places, then an upper

bound for the necessary number of samples can be

determined as follows: Choose M so as to make the

last member of (32) as small as possible, and let

bound Ml

10 m+l

This yields the formula

N = 16- 102"> bound [0(Q) - M]^ (33)

It is worthwhile writing out the distribution of the

endpoint of the walk and the formulas for E(0. ) and

a^( 0j) in the case of the random walk on discussed

in Sections 6 and 9. This can be done with the aid

of the function v(P, P' ) introduced at the end of Section 11, but with P' now on C^. In that case, v(P,P')

is the solution of L(V) = 0 with 0(Q) = 0, Q on C^ , Q^P', and 0(P') = 1. In other words, v(P,P') is

the probability of reaching the particular point P'

before reaching any other point exterior to R.

The variance of a distribution or of a random

variable is defined to be the second moment of the

distribution about the mean value. The square

root of the. variance is called the standard deviation, or standard error. The notation a^(X) will

always be used for the variance of the random

variable X.

The probability distribution of the endpoint of the

random walk can now be written out as shown in

Table n, in which Q^, . . . represent the individual points of C^

.

Position of

endpoint Qi Q2 Q3

Probabilities v(P,Q,) v(P,Q2) v(P,Q3)

Table II

Then

E(0,)= 2 0(QJ v(P,Qj) = V(P)

1 and (31) becomes

aM(/)i)=2:[0(Q.) - V(P)]2 v(P,QJ

1 _

(«J4) < bound [0 (QJ - M]^.

i Of course (32) remains unchanged.

In one dimension, these formulas yield the following expression for the variance of 0:

a^(0) = (B - A)^

v(x,b')(^-v(x,b-))

(35^

This is greatest when v(x,b')= |

we get the upper bound

in which case

a2(0) ^

(B - A)^

4N

This is exactly the value that would have been obtained by letting M = (A + B)/2 in (32), so it follows

that with proper choice of M, the bound given by (32)

under certain very special circumstances is the best

one possible. Formula (33) now becomes

N = 4-102-"(B - A)2 . (36)

Unless the boundary values are nearly constant, or

unless only low accuracy is needed, it is obvious

that the number of samples required as estimated

by (33) or (36) will be appallingly large. Less pessimistic estimates can be obtained by working directly

with the first two members of (31), or of (34), or

with (35), using knowledge as to the approximate

value of v(P,Q). The technique of sequential analysis'^ can also be used profitably to cut down the

sample size, the maximum possible saving over

techniques requiring a fixed number of samples is

of the order of about 50%. Other techniques possibly

leading to a much greater reduction in the overall

amount of computing will be considered in Section 16.

The Dispersion of the Statistical Estimate of G(P,P')

15

We now calculate the variance of the estimate of
and the variance* is given by

a2(0,) = E[0, - V(P)]2

(31) ^ E [ 0j - M]2 $ bound [ 0(Q) - M ,

where Q in the last member is any point exterior to

R on which the walk can end with non- zero probability,

and M is any real number whatsoever. The inequality

follows from the fact that the second moment of any

distribution is least about the arithmetic mean.

The statistical estimate of V(P) will be, a priori,

(&1 + 02 +. • • + 0 N N

Its mean value is, of course, V(P), and its variance

is given by

aH^)-.£lilA. < bound [0(Q) - M]^ (32)

The estimator 0 will be nearly normally distributed

for N ^ 100, by the Central Limit Theorem of probability theory, so Prob {l0 - V(P)|> 2a(0)}

will be about 0.05. If this is considered a small

enough margin of error, and if it is desired to compute V(P) to, say, m decimal places, then an upper

bound for the necessary number of samples can be

determined as follows: Choose M so as to make the

last member of (32) as small as possible, and let

bound |0(Q) - Ml _ 5

This yields the formula

10

N = 16- 102" bound [0(Q) - M]^ (33)

It is worthwhile writing out the distribution of the

endpoint of the walk and the formulas for E(0j) and

0j) in the case of the random walk on discussed

in Sections 6 and 9. This can be done with the aid

of the function v(P, P' ) introduced at the end of Section 11, but with P' now on C^. In that case, v(P,P')

is the solution of L(V) = 0 with 0(Q) =0, Q on C^ , Q P', and 0(P') = 1. In other words, v(P,P') is

the probability of reaching the particular point P'

before reaching any other point exterior to R.

The variance of a distribution or of a random

variable is defined to be the second moment of the

distribution about the mean value. The square

root of the variance is called the standard deviation, or standard error. The notation ct^(X) will

always be used for the variance of the random

variable X.

The probability distribution of the endpoint of the

random walk can now be written out as shown in 1 Table n, in which Qi, Qg, • • • represent the individual points of C^

.

Position of

endpoint Qi Q2 Q3

Probabilities v(P,QJ v(P,Q2) v(P,Q3)

Table II

Then

E(0,)= Z0(QJ v(P,Q,) = V(P)

1 and (31) becomes

CTM0j)=2:[0(Q,)-V(P) ]2 v(P,QJ

< bound [0 (Q,) - M]^.

(34)

This is greatest when v(x,b') = |

we get the upper bound

(B - A)2

2 » in which case

4N

Of course (32) remains unchanged. «

In one dimension, these formulas yield the following expression for the variance of 0:

a2(0) =(B-A)2 ^(^-^')(^-^(^>^^>)

. (35)

This is exactly the value that would have been obtained by letting M = (A + B)/2 in (32), so it follows

that with proper choice of M, the bound given by (32)

under certain very special circumstances is the best

one possible. Formula (33) now becomes

N = 4-102°'(B - A)2 . (36)

Unless the boundary values are nearly constant, or

unless only low accuracy is needed, it is obvious

that the number of samples required as estimated

by (33) or (36) will be appallingly large. Less pessimistic estimates can be obtained by working directly

with the first two members of (31), or of (34), or

with (35), using knowledge as to the approximate

value of v(P,Q). The technique of sequential analysis can also be used profitably to cut down the

sample size, the maximum possible saving over

techniques requiring a fixed number of samples is

of the order of about 50% . Other techniques possibly

leading to a much greater reduction in the overall

amount of computing will be considered in Section 16.

The Dispersion of the Statistical Estimate of G(P,P')

15

We now calculate the variance of the estimate of
Instead of p^, >

• • -'Ps' a new set of transition

probabilities p^ *(P), P2*(P), • • • ,P5 *(P) is chosen

at each point P on such that no one is zero and

2 P,*(P) = 1.

Equation (18) is replaced by the equivalent one:

V(P) = 2 P3*(P)-

P,(P)

P,*(P)

V(P,) (38)

This is interpreted in probability language as follows:

A particle starts at the point P on and performs

the random walk described in Section 9 but with the

transition probabilities Pi* , . . . , p^* . If the first

step is to Pj,j = l,2, ... ,5, the weight Pj(P)/Pj*(P)

is attached to the particle. Then if the next step is

to P^, i = 1, 2, ... ,5, this weight is multiplied

by Pi(Pj)/P i*(Pj), and so on, until a point Q of

is reached. The cumulated product of weights is then

multiplied by 0(Q) and tallied as the final score for

the walk.

It is easy to see, in various ways, that V(P) as

previously defined must also be the mean value of

the final tally. For one thing, the mean value of the

tally clearly satisfies (38) with the proper boundary

values. Another way to look at the matter is to consider the contribution to V(P) arising from a particular path from P° in R^ to Q in C ^. We suppose

that this path proceeds via the particular points

p'', k = 1, 2, ... . The probability that the particle

took this path is n *(po)

. p. *(pi)

. p. (P2)..., (39)

Jo J

1

where each subscript

numbers

path is

1, 2,

0(Q)

P. *(P=^)...,

J 2

jk is some one of the five

5. The tally at the end of the

Pi (P°) P^ (P^)

p. *{P°) . p, *(P^)

(40)

Jo

The mean value of the tally for this particular path

is then simply the product of (39) and (40), or

0(Q) . TT p. (p*) , k

but this is exactly the mean value which would have

been obtained from this path if the walk had proceeded according to the transition probabilities Pj(P)

instead of Pi*(P).

In applying the method to estimate V(P°), the

estimator will be the average of N tallies obtained

in N walks. Let z bea random variable whose values

are all of the various possible tallies and whose

distribution is given by the corresponding probabilities of the tallies. Then

E(z) = V(P°) = Z 0(Q) 2 Y

P. (P"^)

Jk D *(PM

Jk

and

a2(z)= -[V(P°)]2+ 2 [0(Q)]2 2 TT

QcC, k

Pi (P"^)

Jk P -^(P")

Jk

Pi *(P^) ) , Jk

(41)

where the inner summation is extended overall possible paths leading from P° to Q. The estimate of

V(P) based on z, will be

z = — + z. N

where z^, Zg ,

. . . denote separate independent

observations on z. The variance of the estimator is

(z) = a2(z)/N.

Formula (41) has two interesting consequences.

In the first place, it is clear that if some one of the

p* 's is made to approach zero and if the corresponding

p is not zero, then ct^(z) becomes infinite. Therefore, it is evidently possible to make a choice of the

p*'s which will make the standard error of the estimate of V(P) very much more unfavorable than it would have been if the natural probabilities p^, . . . , Pghad been used.

In the second place, a good choice of the p*'s may

result in a reduction in the standard error. Indeed,

if there is a priori knowledge concerning the solution

V(P), the variance a^(z) can be very greatly reduced.

Consider the extreme case in which V(P) is already

known on R^. Suppose V(P) is always positive on R^.

(This can always be brought about by merely .adding a constant to the boundary value 0(Q).) Now let

Pi*(P) =P,(P) • ' j = i' 5 .

In view of (18), 2 p,*(P) = 1.

1

It is now quite easily seen by substitution that the

double summation in (41) has the value

2 [0(Q)?

V(P°)

0(Q)

2 TT p. (P'^) =[V(P°)]2

It Jk
and so a^(z) = o^iz) = 0.

The existence of transition distributions leading

to zero standard errors of estimates was pointed out

by Kahn^^ in connection with mechanical quadrature

and the integration of certain integro-differential

equations of diffusion theory."

Of course, in practice, VCP) will not be known,

but a suitable approximation can be used instead,

perhaps based on physical theory or intuition. The

author has seen cases in which the number of samples

required to achieve a given accuracy has been re- duced by a factor of 1/100 by this device.

Another way of using the technique of alteration

of transition probabilities consists in making the new

probabilities constant and, in particular, all equal.

In the two-dimensional case, with /3 12 5^0, the

individual values, if equal, would be 1/5. This

technique might be convenient for certain types of

computing machines as the handling of the random

numbers is simpler. The effect on a^(z) niight be

unfortunate however, and, if possible, an analysis

of (41) should be made in advance.

The non-homogeneous equation L(V) = F(x,y),

when rearranged in the form (18), becomes

V(P)= Z p.(P) V(P.) F(P) .

From the meaning of G(P,P'), the direct statistical

estimate of the second term in the solution as given

in Theorem 4 is the average of all the values of

h^ F(P)/D(P) observed at each resting place in R^,

counting the value at the starting point into the average, and using the transition probabilities Pi(P),

. . . , P5(P). But it can be seen by repeating the

argument used in connection with (39) and (40) that

G(P,P') can also be considered as the mean value

of the various weights which the particle has attached

to it on its various visits to P' (with a weight of unity

added on if P' = P). Therefore, it is possible to use

the following procedure for estimating the solution

of the non-homogeneous equation: Using the altered

transition probabilities Pi*,---,P5*, calculate

at each interior resting point P' of the walk the

proper weight w(P') as previously directed, and also

a. See also the paper by Kahn and Harris. The

method is called that of "importance sampling"

by these writers. Dr. W. Edwards Deming pointed out to the author that the method has been known

for many years in various forms to sampling

experts working in the social sciences, who have

sometimes called it "sampling with probability

in proportion to size". (See Ref . 7, pp. 92-93.)

calculate the quantity T = - w(P')h2 F(P')/D(P') and

add it to the cumulative sum of all previous quanti- '

ties T so calculated in the course of the walk. The

cumulative sum is to start with the term

^ -h2 F(P)/t)(P).

The final tally is the product of 0(Q) into the cumulated

product- weight at arrival, plui the cumulative sum | of the values of T upon arrival. (Zero is added to

T at Q.)

Bookkeeping Procedures 17 1

1

It is advantageous in many cases to extract as \ much information from a single random walk as possible. This matter has already been touched upon

in Section 15, where it was suggested that if G(P,P')

were to be calculated for a fixed P, and for every

point P' on R h, then a natural procedure might be i to observe all of the values of n(P,P') for each P' 1 on R^ in each individual random walk, and average ; these parallel values over N walks. This entails ' keeping a complete record of the sequence of events

in each random walk. Such a diary may become quite

bulky, and the decision as to how complete it should

be will be affected by the storage facilities of the

computing machinery to be used. j The bookkeeping approach will probably often be 1 convenient when the value of the solution V(P) of ' Problem is to be estimated at several points of

R^. Two procedures now suggest themselves. Suppose that the solution is to be obtained at the points

P and P' , that N random walks on R^ are started at i P, and that N' of these arrive at P* before reaching |

. The first procedure consists in treating each I one of these N' random walks after the first visit to

| P' as a single new random walk starting at P' . The | meannumberof walks thus gained for P' is Nv(P,P') ] where v(P,P') in the function used in Section 15 and j elsewhere. In one dimension, at least, this will

usually be a substantial fraction of N.

The second procedure consists in regarding each

visit to P' as starting a new random walk at P'. ; Thus in the notation of Section 15, each random walk

startingfrom P generates n(P,P') new random walks

| at P'. This second procedure, clearly, also has j significance as providing a new estimation procedure

at P itself; this is the case in which P' coincides ' with P. ' However, Wasow has found that the second pro- , cedure is disadvantageous, as it increases the dis- i persion of the statistical estimate of V(P'). His

demonstration (which was communicated to the author

informally in a letter) runs essentially as follows:

Consider the first procedure for a moment. As , in Section 14, let 0i, (^2>---»0vr- be identically i
distributed independent random variables representing the tally after each of the N' walks started from

P' , so E(Oj)= V(P'). The estimate of V(P') in this

procedure is,a priori,

0

N'

J

Although if the walks were started at N, then N'

must be a random variable, it will, nevertheless,

now be assumed that N' is fixed; that is, we shall

study only the conditional distribution of 0, given

N'. It is assumed naturally that N'> 0. (Of course

if P' coincides with P, then N' = N.)

As before, we denote the variance of 6 in the first

procedure by

oH0) = aH 0^)/N'.

In the second procedure, the estimate of V(P'

)

corresponding to 0 is, a priori,

N'

Z n^0^

0* = N

where n^ =nj(P,P'), is a random variable representing the number of visits to P' in the jth random

walk, under the hypothesis that there is at least one

such visit in that walk. (The distribution of n.(P,P')

is given by Table III as it stands if P = P' , and if P / P ' it is given by Table III but with the value 0

deleted and the probabilities all divided by 1 - v.)

The 2N' variables 0^ and n^ are all independent.

The function 0* is probabilistically more complicated than 0, as it contains random variables in

both numerator and denominator. Its mean value is,

nevertheless, still V(P); it may be calculated in this

way:

E(0*) = E [E (0*1 n.)]

FN'

2 V(P')

= E

n. N'

2 n.

V(P') E^ (1)

= V(P') ,

where E denotes the unconditional mean value with

respect to the variables ni, n^,. .. ,n^-, and

E^ (0*|n. ) signifies the conditional mean value of

0* with respect to 0i, 02,

values of the n^.

In the same notation,

for fixed

(t2(0*) = E[0* - V(P')]2

E^ {E^ [(0*-V(P'))Mn.] }

(42)

But Schwarz's inequality states that

N'

2 1-n,

N' N' N'

^2 12 2 n.2 = N' 2 n. (43)

with the equality holding only if n ^ = n 3 = . . . =n- ^ -

The joint distribution of n^, n^, . . . ,n

N'

N'

can be

concentrated at a single point in N' -space only in

the trivial case in which there is only one lattice

point on R^. Disregarding this case, and substituting

(43) into (42), we obtain the strong inequality

a2(0,) oH 0*)>

N'

= a2(0) , which was to be proved.

Wasow has further pointed out that an upper bound

for a2(0*) now lies close at hand. For

N'

2 n^ ^ N'

1 and the n. are identically distributed; therefore, by

the computation used to find ct ^ [n(P , P ') ] in Section 1 5

,

aM0*)^ 4^ E„(n,2) N

a^(0,)

N

n,

a^(0,)

N

V 1 -t- r

(1 - r)2

1 + r

, P /P'

(1 - r)2

, P = P' .
It is clear that bookkeeping procedures of the type

discussed in this section can be applied to the sampling methods discussed in Section 16. In this case,

the bookkeeping will involve calculating in parallel a number of scores.

Further Problems 18

The theory presented in the paper is relatively

complete for the equation L(V) = F(P) in bounded

regions. The eigenvalue case is being explored actively by various research workers as this is being

written.

There are, however, a number of questions in

connection with the problems treated in this paper

which have not as yet been adequately answered. One

of these is that of the degree of approximation of

V(P) for a given h or x to the solution of Problem D.

The demonstration given in Section 13 of the present

paper depends upon finding a suitable a priori es- timate for the derivatives of the solution of problem

D in terms of the given boundary value function 0 and

of C. Just how this is to be done in complicated

cases is not at all clear. In this connection, furthermore, it would be interesting to obtain a useful bound

for E(nlP), or for the right side of (29) valid for the

most general operator L. Neither the proof of Petrowsky's Theorem, nor the standard treatments of

Problem 5, 10,15 throw any light on this matter.

Another open problem consists of the extension of

the methods of this paper to boundary value problems

involving differential equations of order higher than

two, and to non-linear differential equations. Further open problems are encountered when unbounded

regions are considered. A considerable amount of

additional research probably will be needed to give

fully satisfactory answers to these questions.

REFERENCES

1. D. Blackwell, "On an Equation of Wald ", Annals

of Math. Stat., Vol. 17, (1946).

2. D. Blackwell and M. A. Girshick, "On Functions

of Sequences of Independent Chance Vectors with

Applications to the Problem of Random Walk in k

Dimensions", Annals of Math. Stat., Vol. 17

(1946), pp. 310-317.

3. S. Chandrasekhar, "Stochastic Problems in Physics and Astronomy", Reviews of Modern Physics,

Vol. 15 (1943), pp. 1-89.

4. Kai Lai Chung, "On the Maximum Partial Sums of

Sequences of Independent Random Variables",

Trans. Am. Math. Soc. , Vol. 64 (1948), pp. 205-

233.

5. R. Courant, K. Friedrichs, and H. Lewy, "Uber I die Partiellen Differenzengleichungen derMathe- ! matischen Physik", Math. Annalen, Vol. 100 ! (1928), pp. 32-74.

6. Herbert T. David, "A Note on Random Walk", i Annals of Math. Stat. , Vol. 20(1949), pp. 603-608.

7. W. Edwards Deming, "Some Theory of Sampling", i New York, (1950).

8. P. Erdos and M. Kac, "On Certain Limit Theorems of the Theory of Probability", Bull. Am. I Math. Soc, Vol. 52 (1946), pp. 292-302.

'

9. W. Feller, "Zur Theorie der Stochastischen I Prozesse", Math. Annalen, Vol. 113, (1936),'

pp. 113-160. I 10. T. Fort, "Finite Differences and Difference Equations in the Real Domain", Oxford, 1948.

11. Herman Kahn, "Stochastic (Monte Carlo) Attenuation Analysis ", RAND Corporation, Santa Monica,

Calif. , Report No. R-163, June 14, 1949.

12. O. D. Kellogg, "Foundations of Potential Theory",

Berlin, 1929.

13. A. Khintchine, "AsymptotischeGesetzederWahr- | scheinlichkeitsrechnung", Berlin, 1933. j 14. A. Kolmogorov, "Uber die Analytischen Methoden

j in der Wahrscheinlichkeitsrechnung", Math. An- I nalen, Vol. 104 (1931) pp. 415-458.

15. W. Lexis, "Abhandlungen Zur Theorie derBevolterungs-und Moralstatistic " , Jena, 1903, pp. j 161-163.

16. R. Luneberg, "Das Problem der Irrfahrt ohne

Richtungsbeschr'ankung und die Randwertaufgabe

der Potentialtheorie", Math. Annalen, Vol. 104 , (1931), pp. 700-738.

17. W. H. McCrea and F. J. W. Whipple, "Random

Paths in Two and Three Dimensions", Proc . 1 Royal Soc. Edinburgh, Vol. 60(1940), pp. 281-298. i 18. N. Metropolis and S. Ulam, "The Monte Carlo I Method", Jour. Am. Stat. Assoc. , Vol 44 (1949),

j pp. 335-341. i 19. The Monte Carlo Method: Proceedings of a Sym- I posium held in Los Angeles, Calif. , on June 29, } 30 and July 1, 1949. To be published by the Na- i tional Bureau of Standards. ; 20. I. Petrowsky, "Uber das Irrfahrtproblem", Math.

Ann., Vol. 109 (1933), pp. 425-444.

21. George Polya, "Surla Promenade au Hasard dans i un Reseau de Rues", Actualites Scientific et In- j dustuelles. No. 734: Colloque consacre a la

j theorie des probabilites I: Conferences d'intro- 1 duction pp. 25-44. Paris. 1938. 1 22. Lord Rayleigh, "On James Bernoulli's Theorem \ in Probabilities", Phil. Mag., Vol. 47 (1899),

pp. 246-251. I 23. J. D. Tamarkin and W. Feller, "Partial Differ- : ential Equations". Mimeographed lecture notes

j for a course given at Brown University, June 23 - i September 13, 1941. (Chapters I-IE, J. D. Ta-

'

markin. Chapters IV-VH, W. Feller. )

|
I

24. G. E. Uhlenbeck and L. S. Ornstein, "On the

Theory of the Brownian Motion", Phys. Rev. , Vol. 36 (1930), pp. 823-841.

25. J. V. Uspensky, "Introduction to Mathematical

Probability". New York and London, 1937.

26. A. Wald, "On Cumulative Sums of Random Variables", Ann. Math. Stat., Vol. 15 (1944), pp.

283-296.

27. A. Wald, "Limit Distribution of the Maximum

and Minimum of Successive Cumulative Sums of

Random Variables", Bull. Am. Math. Soc. , Vol. 53 (1947). pp. 142-153.

28. A. Wald, "Sequential Analysis", New York, 1947.

29. J. L. Walsh, "Interpolation and Approximation

by Rational Functions in the Complex Plane", New

York, 1935.

30. M. C. Wang and G. E. Uhlenbeck, "On the Theory

of the Brownian Motion n". Rev. Mod. Phys. , Vol. 17 (1945), pp. 323-342.

31. W. Wasow, "Random Walks and the Eigenvalues

of Elliptic Difference Equations". To appear in

the Journal of Research of the National Bureau of

Standards.

DISCUSSION

MR. ISAACSON: It occurs to me that the elliptic

equation may be regarded as an equation representing

the steady state solution which might be approached

by the solution of the parabolic equation, in which

case some of the methods which were described by

Dr. Thomas might be applicable and might yield an

estimate of the accuracy of your probabilistic approach to the problem. Have you considered that

view of the problem, rather than the random walk

method ?

DR. CURTISS: Yes. The connection between the

elliptic differential equation and the parabolic differential equation is this: for elliptic equations, n is a random variable; whereas for parabolic equations

there is a one-sided random walk (always going forward) as far as n is concerned. You introduce a time

variable which essentially varies as the reciprocal of

n. The moment that has been done, it becomes a parabolic differential equation, as Courant and Friedrichs

explicitly state in a footnote. The background of exactly your comment is contained in a rather extensive

footnote on the pages in which this equation is treated.

CHAIRMAN TAUB: An alternative to your method for solving the parabolic equation utilizes the

Khintchine theorem, which holds for parabolic differential equations as well, and this method for

solving parabolic equations, should be compared to

those of the first by Dr. Thomas.
I
Reprint from

of the

OFFICE OF NAVAL RESEARCH

Issue of

MAY 1951

OFFICE OF NAVAL RESEARCH

DEPARTMENT OF THE NAVY

WASHINGTON, D. C.
I
1'ltQ Un5tltutQ j^ot A/umetical -final

if

5i5

the A/ational bateau Standatd5

J. H. Curtiss, Chief

National Applied Mathematics Laboratories, NBS

Dr. Curtiss discusses the program of the Institute for Numerical Analysis

established for basic research and training in the types of mathematics which are

pertinent to the efficient exploitation and further development of high-speed automatic digital computing equipment. He has been head of the National Applied

Mathematics Laboratories, of which the Institute is one of four branches, since

NAML. was established in 1947. Prior to this Dr. Curtiss was with the Bureau of

Ships. He has also served on the faculties of Cornell, Harvard, and Johns Hopkins

Universities and is well known as a consultant on problems in mathematical statistics.

Suppose that a research worker suddenly

found that he had to solve 100 linear equations in

100 unknowns, and that he had access to a large

automatic digital computing machine with whose

coding and operation he was familiar. How

should he set about preparing the problem for

the machine?

This situation is by no means unrealistic.

Large sets of simultaneous equations are en- countered in many data-reduction problems (e.g.

in geodetic triangulation problems), in mathematical treatments of logistics and economics,

and in engineering and applied physics. In fact,

many numerical problems in applied mathematics, such as the numerical solution of integral

and differential equations, can be reduced in one way or another to solving large sets of simultaneous linear equations, although in individual cases this may not be the optimal procedure.

It may look easy. Any high school student

knows how to solve simultaneous linear equations. For example, why not write the various

solutions down directly as the quotients of pairs

of determinants and make the machine evaluate

the determinants directly from their definition?

This is a beautiful theoretical solution;

but each determinant has 100 rows and 100 columns. By definition, the value of each determinant

is the algebraic sum of 100 x 99 x 98 x. . . x 3 x 2x1 signed products, each consisting of 100

factors chosen from among the elements of the

determinant so that each row and each column

is represented. On this basis, for each determinant over 10157 products of 100 numbers each

would have to be formed and added together algebraically. If a multiplication could be done in

100 microseconds on the automatic computing

machine, and an addition in 10 microseconds,

and. if an infinitely extensive high-speed internal memory could be assumed, then each

1 Member of the staff of the NBS Institute for

Numerical Analysis.

determinant would be evaluated in something

over 10^^*^ years of machine time! And if all the

solutions to the equations are wanted, 101 of

these determinants will have to be evaluated.

It looks as if the machine will be busy for some

time to come.

So this method turns out to be ridiculous

for a problem of this size. By use of efficient

methods, each determinant can be evaluated in

something like 3 x 10^ multiplications and a comparable number of algebraic additions. For

the 101 determinants required, the straight

computing time would be something like one

hour. Another 3 to 6 hours must be allowed for

internal logical operations like changing the instructions within the machine. However, the as- sumption of unlimited memory is pure nonsense;

the really high-speed (i.e., readily accessible)

memories in the current crop of machines are

limited to about 1000 numbers of 10 to 12 decimal digits each. To carry out the procedure

it would be necessary constantly to bring data

in and out of the high-speed memory, a process

so slow that, by comparison, computing time can

be ignored. Finally, the twin problems of significant digits and round-off errors are so seri- ous as to make it quite unlikely that any ordinary

procedure suitable for 5 or 6 equations would

even roughly approximate the answer to our problem with 100 equations.

What about other methods? There is certainly no dearth of them to choose from. Dr.

George E. Forsythe has been working on a

classification and bibliography in which he distinguishes two main categories: direct methods,

like the method of determinants referred to above, and iterative methods, in which the solution is arrived at by successively closer approximations. Rapid recapitulation of some of the

short names of the methods will suggest the

variety of approaches available:

Direct Category : Chio and Aitken determinant methods; Bingham, R. Schmidt, Lanczos,i

and Frame characteristic equation methods;

- 8 -
Gauss elimination method and many abbreviations thereof; Cholesky (or square-root) and

"escalator" triangularization methods; GramSchmidt, Fox- Huskey^-Wilkinson, and Bodewig

orthogonalization methods; block elimination

methods; "Below-the-line" device, R. A. Fisher

technique, and other special methods.

Iterative category : about a dozen distinct

variants of the Wittmeyer type (some of them

associated with names like Gauss, Seidel, Jacobi,

and von Mises); a number of versions of the

least-squares type, including an entire subclass first characterized as such byRosser^ and

Hestenes^; the subclass of gradient methods

with several variants; iterated elimination methods; and "relaxation" methods.

Then too, there is a separate category of

miscellaneous special methods, such as "MonteCarlo" methods (i.e., sampling methods based on probability theory), and a method of H.

Lewy^ for solving equations when the solution is

known to be in integers.

This catalogue is quite probably incomplete, but even as it stands, it presents a truly

bewildering problem of selection. D. R.

Hartree^ (1) says "It is probably the case that

there is no one best method for the evaluation of

the solution of a set of simultaneous linear algebraic equations, but that the best method in

any particular case depends on the structure of

the set of equations concerned. This is certainly

true of methods for treating such equations without the use of an automatic machine, and may

well still be true when such assistance is available. For example, it may be that most of the

coefficients are non-zero, and are not small integers and moreover are known only approximately, either because they are derived from

measurements which are subject to experimental error or because they are results of previous

calculations and are subject to rounding-off

errors; or it may be that each equation involves

only a few of the variables, and these with coefficients which are small integers and are known

to be exact. It is quite likely the most appropriate methods in the two cases will be different."

But how should the choice be made ? Somehow, somewhere, guide lines must be laid down

which will permit intelligent selection of methods, with due regard to the peculiarities of a given system of equations and to the specifications of available computing equipment. The

literature is full of special methods, but the

overall guide book has not yet been written. In

fact, there seem to be many cases even now in

Former Director of Research of the NBS

Institute for Numerical Analysis.

^Consultant of the NBS Institute for Numerical Analysis.

which no known method gives feasible solutions.

Moreover, relatively little is known about how

round-off errors and other errors inherent in

computational work pile up in the many methods

listed above when they are applied to really

large sets of equations. Perhaps the only way

to resolve that particular sort of question will

be by extensive arithmetical experiments on

high-speed automatic machines.

Here is an important type of problem,

then, which recurs over and over again in applied mathematics. A lot is known about it, but

not nearly enough for the full and effective exploitation of automatic digital computing machinery, and what is already known needs to be

pulled together in a usable form. In other words,

this area of numerical analysis badly needs both

background research and foreground research,

together with laboratory experimentation, to maximize the nation's return from its

multimillion dollar investment in computing machinery.

It is to work on such problems that the

Institute for Numerical Analysis of the National

Bureau of Standards was established in 1947

with the support of the Office of Naval Research

and the cooperation of the University of California. The Institute is located on the campus of

the University of California at Los Angeles, and

maintains close liaison with the University. Its permanent staff numbers about 70 and includes

some 15 scientists (mathematicians and theoretical physicists) at the post-Fh.D. level. In

addition, a number of scientists have worked at the Institute at one time or another on temporary

appointments. Some of the outstanding mathematicians in the world have been associated for

various terms with the Institute, and the permanent research staff compares in strength and

productivity with the stronger academic mathematics departments.

The Institute is one of the four branches

of the National Applied Mathematics Laboratories (NAML) of the National Bureau of Standards, which ONR played a fundamental role in

establishing. The story of the Institute has been

set forth at some length in a readily accessible

article in Science (2), so only a brief outline is

necessary here.

In 1945 a study of Navy Department computing requirements by the Office of Research

and Invention, now ONR, resulted in a memorandum prepared by Lt. Comdr. James H.

Wakelin and circulated within the Naval establishments. It in effect recommended the establishment, with Navy participation, of a national

interagency computing center which would develop and use large-scale automatic machines.

Early in 1946, Rear Admiral H. G. Bowen, then

Chief of Naval Research, suggested to Dr. E. U.

Condon, Director of the National Bureau of

9
standards, that ONR and NBS should jointly

undertake to establish such a facility. It was

eventually agreed that NBS should be solely responsible for administration of the center, and

also that ONR's participation was conditional

upon commitments from other agencies to support the new activity.

It was natural that ONR should approach

NBS with such a proposal, since ONR had been

temporarily supporting the famous NBS, Mathematical Tables Project, originally a WPA project but financed during the war on OSRD funds.

Also, in the spring of 1946, the Census Bureau

had requested NBS to construct a large automatic digital machine suitable for the preparation

of census reports. Therefore the nucleus for

the sort of activity envisioned in Comdr. Wakelin's report was already present at NBS early

in 1946.

A year of cooperative study ensued, including various conferences with possible clients

and applied mathematical groups all over the

country. The Air Materiel Command became interested, and made certain commitments in connection with its machine development program.

This support, along with less substantial expressions of interest from other agencies, fulfilled the ONR requirements for outside participation. In the summer of 1947, NAML was

established as a new division of NBS. It was

organized into four branches: the Institute for

Numerical Analysis in Los Angeles; the Computation Laboratory in Washington (to carry on

the work of the Mathematical Tables Project but

with more emphasis on problem-solving and

less on tables); the Statistical Engineering Laboratory in Washington; and the Machine Development Laboratory in Washington.

The Institute was conceived as "the focal

point in the organization for basic research and

training in the types of mathematics which are

pertinent to the efficient exploitation and further

development of high-speed automatic digital

computing equipment. A secondary function is to provide a computing service for Southern

California and to give assistance in the formulation and analytical solution of problems in applied mathematics." The functions of the Institute are identified in more detail in a Prospectus

for the NAML issued early in 1947, which was

the basis of reference (2). They were specified

there as follows:

"a) Plans and conducts a program of re- search in pure and applied mathematics aimed

primarily at developing methods of analysis

which wiU permit the most efficient and general

Now reconstituted as the NBS Applied

Mathematics Advisory Council, The Council

is referred to again below.

use of high-speed automatic digital computing

machinery.

"b) Conducts training programs for personnel of industry, Government agencies, and

educational institutions, in the theory and

disciplines needed for the full exploitation of highspeed automatic digital computing equipment.

"c) Studies and formulates requirements

for the intelligence and internal organization

which high-speed automatic computing machinery should have; develops overall performance specifications for such machinery.

"d) Serves as a center at which competent

scholars can explore the usefulness of highspeed automatic digital computing machinery in

their own fields of interest.

"e) Formulates requirements for further

mathematical tables and other aids to computation; reviews the overall program of NAML

with regard to the production of such objects

and advises the Administration and Executive

Council* accordingly.

*f) Reviews, analyzes, and, as necessary, assists in the mathematical formulation of problems in applied mathematics

of the more complex and novel type arising in

outside laboratories.

"g) Provides a computing service containing both standard equipment and high-speed

automatic equipment (when available) for local

industries, educational institutions, and Government agencies.

"h) Assists, and conducts liaison with, related programs in local educational institutions.

"1) Maintains a consulting service on

special problems in applied mathematics.

"j) Prepares reports and monographs

giving the results of the research described

above; also prepares training manuals, bibliographies, and indices."

The Institute is divided into the Research

section and the Mathematical Services section.

The Research Section has from the beginning

been supported almost exclusively by ONR, but

now has some research for the Air Comptroller,

USAF, underway. Also, a small subsidy from

NBS pays for a (tart of the necessary experimental computing.

The Mathematical Services Section operates the computing service mentioned in the

quotation above, and has recently completed the

construction of a large-scale automatic digital

computing machine under the direction of Dr.

Harry D. Huskey. The machine, which is called

the NBS Western Automatic Computer (SWAC

for short), operates in the parallel mode and

uses an electrostatic memory system, employing the Williams principle. The electrostatic

memory was started with a 256-word capacity, soon to be increased to 512 words. A

rotating-drum magnetic memory of 10,000 word

capacity is also being added.

- 10
Fig. 1. The NBS western automatic computer (SWAC)

This machine was financed entirely by the

Office of Air lElesearch (OAR) , Air Materiel Command, following its decisive role in the planning

stages of NAML.

No computing service can be operated effectively from reimbursements obtained after

(and in the Government, this sometimes means

long after) problems are solved. Some form of

advance financing is quite essential, both to pay

for backlog work and to provide working capital.

This critical form of budgetary support has been

provided by OAR to the Institute's computation

laboratory ever since the beginning. In addition,

when OAR made its original decision to support

construction of the SWAC, it provided that a certain minimum amoimt of time should always be

available to. the ONR research program.

When taken with certain aspects in the activities of the Eastern branches of the NAML,

the interagency participation in the NAML program forms an ideal example of cooperation between Departments of the Executive Branch of

the Government, and as such, has received

favorable comment from the Bureau of the

Budget. An important element in this picture is

the NBS Applied Mathematics Advisory Council,

which reviews and guides the program.

To return now to the Institute: Computers

like to say there has been nothing new discovered

in numerical analysis since Gauss. There have

been many advances in other fields of mathematics, pure and applied, since that time. Can

it be because professional mathematicians' were

not interested in numerical analysis and the

field was left to amateur mathematicians to develop? Whatever the answer, it has been con- sidered essential to adopt a very fundamental

approach in setting up the program of the Institute. In particular, a policy of staffing the

Research Section mainly with competent professional mathematicians has been rigorously

followed. It has been found desirable, however,

to have at least one theoretical physicist and

one expert in classical applied mathematics on

the staff, so that advice can readily be obtained

as to profitable directions in which to work.

Every care has been taken to provide at- tractive working conditions. For example, most

of the senior members of the research staff have

private offices; experienced typists are on hand

for typing mathematical manuscripts; desk computing machines are readily available for those who

need them; and a conscious effort is made to insulate the scientists from administrative red tape.

- 11 -
The Institute staff members have faculty

privileges in the excellent UCLA library nearby,

but a book in hand is worth two on somebody's

desk in a fraternity house six blocks away, and

it was soon clear that a good working collection

should be built up in the Institute. The aim has

been to make it pre-eminent in all items bearing

directly on numerical analysis, both ancient and

modern, and to make it outstanding in up-to-date

material in mathematical analysis and applied

mathematics. Secondary emphasis has been

placed on theoretical physics and certain

branches of pure mathematics, such as abstract

algebra, although outstanding new books in such

fields are ordered. The Institute Library has

been fortunate in picking up back editions of the

standard mathematical journals; for example, it has about all volumes of Mathematische Annalen

back to vol. 1 in 1869.

The computing equipment of the Institute,

in addition to the SWAC, consists of hand machines and punched -card machinery. The

punched card installation includes an IBM Card

Programmed Calculator (this is an electronic

computing machine with a substantial internal

memory), a 604 Calculating Punch, two 602A

multipliers, a tabulator, and various supporting

items. The research mathematicians have easy

access to the machines at all times (there is no

restricted area) , and unless an emergency arises

they usually get prompt action on any experimental computing.

The significance of the Institute as a training center ranks high, and care has been taken

to implement this aspect of the plans. Shortterm appointments of senior research workers

have introduced scholars in various fields to the

facilities of the Institute (see Item (d) in the list

of functions) . There have also been many visitors

to the Institute whose travel and expenses were

paid for by their own institutions. Scientists are

always welcome, and the visitors' book looks

like a *Who's Who' of the mathematical world.

Like the Institute for Advanced Study in the

Bast, the Institute for Numerical Analysis in the

West is sure to be visited by mathematical

travelers in this country.

More formal educational programs are

conducted, too. The Institute collaborates with

UCLA on various courses. A particularly ambitious program of graduate mathematical instruction in modern numerical analysis is being

given cooperatively by the Institute and UCLA in

the summer of 1951. Fellowships of two types are also offered: summer studentships aimed

at familiarizing graduate students of mathematics and physics with the Institute, and thesis

fellowships lasting one or more years, to enable a graduate student to complete a Ph.D thesis.

A number of symposia, colloquia, and short, intensive courses on modern automatic computing

machinery have also been organized from time

to time.

Credit for recognizing the need for fundamental research by the Institute should go

particularly to Dr. Mina Rees, Director of the

Mathematical Sciences Division of ONR. She

saw that the great activity in developing automatic digital computing machines in this country

was not being adequately paralleled by theoretical

investigation aimed at finding out how best to use them. Indeed, it seemed that many of them

would probably stand idle altogether too much

of the time while mathematicians catch up

with them.

The ONR research program is, and always

has been, the heart of the Institute. The type of

problem described at the beginning has had a great deal of attention in the past, but further

studies are necessary to evaluate the older results in their relation to modern automatic

machinery. Moreover, there are many significant types of problems, for which it is doubtful

if appropriate methods exist at all, so that

really difficult cases may demand completely

novel approaches.

A type of problem which has received

much attention at the Institute is that of finding

the lowest eigenvalue of the time-independent

Schrbdinger equation

^

Au - Vu + Xu = O , (1)

where V and the unknown u are functions of

several independent variables, A is theLaplacian

operator, and X is a constant. The boundary

conditions are that u tends to zero as any coordinate approaches infinity, that the squares

of the first partial derivatives of u are eachintegrable over the entire plane, and the first

partial derivatives have simple discontinuities

at the origin. The problem is important; in the

theory of wave mechanics, the eigenvalues of

this equation represent the energy states of a system whose potential energy function is V.

Moreover, if suitable methods could be developed

for this equation, they might be applicable to

more complicated equations arising in the study

of nuclear forces.

Discussion is restricted to one dimension,

for simplicity of notation. A classical way of

finding the lowest eigenvalue A.j of such an

equation is the so-called Rayleigh-Ritz method,

which is based on the theorem from the calculus

of variation that

+ V(x)(v(x))2ldx

X. - min — —. 1. ,

where v(x) is any function belonging to a class

- 12
of admissible functions. To apply this theorem

calls for a grand search among all classes of

admissible functions for the one that will minimize the expression on the right. The search

can be systematized and abridged (theRayleighRitz method does that), but is still apt to be

lengthy, particularly in many dimensions.

For the helium atom there are six independent variables x. The lowest eigenvalue was

found with great accuracy by E. A. Hylleraas (3)

in 1930, using the Rayleigh-Ritz method, after a celebrated and back-breaking computation. For

more complicated atoms, this method seems to be practically out of the question on currently

available computing machinery.

In another category of classical methods

for solving such problems, the fundamental technique consists in replacing the differential equation by a corresponding one infinite differences

and finding the eigenvalues of the latter. A good

example of this attack will be found in (4). This

essentially changes the problem to one of finding

the eigenvalues of a matrix; but in may dimensions that in itself can be a most difficult task.

Then too, very little is known about the relations

between the eigenvalues of the difference equation and those of the differential equation. All

in all this seems to be a problem for which it is

worthwhile searching for new methods.

A fruitful hunting ground for such methods

is in probability theory. It has been known for

many years [for a brief history, see (5)] that the

distribution functions associated with certain

random walks provide exact solutions of certain

difference equations, and asymptotic solutions of

the related differential equations. One of the

first applications of Monte -Carlo methods to finding eigenvalues was made at the Institute for

Numerical Analysis by Professor Mark Kac

during his tour of duty there in the summer of

1949. He made use of the following relation derived in his study of certain Wiener functionals:

r

X

-

lin, '°gE j exp(-./o V(X(T))dT[

1 t — °: t where E means expected (i.e. mean) value, and

X(t) is a Wiener process with mean zero*. The

expected value appearing in the formula is then

evaluated by the following procedure: A random

walk starting at the origin with a pre-assigned

i.e., X(t) for a single value of t is a random

variable with the density function

andfortwo or more values oft, say tj < t2 < t3,

the joint distribution of the corresponding ran- dom variables is such that X(ti) is statistically

independent of X(t3) - X(t2).

number n of elementary steps is executed and a score is tallied after each step according to the

following rule: If Sj^ represents the position of

the walk at the k-th step, the score V{S^//n) is

tallied after the k-th step. After nt steps, the scores are all added and divided by n to obtain

afinal score. By another theorem ofDr.Kac(6),

the distribution of the final scores approaches

that of the integral in eq. 2. Therefore to evaluate

approximately the right-hand side of eq. 2, it is

only necessary to perform the random walk and

get the final score many times, then raise e to the power of each final score, and average

the results.

Clearly there are several sources of error

in this approximation. There is the statistical error in the final average; there is the error in

using the random walk approximation to the

Wiener functional, and there is the approximation involved in using eq. 2. In the experiments

of 1949, good results were obtained for the case

V(x) = x2 (the harmonic oscillator); these are

reported in (7). The method has worked satisfactorily for the hydrogen atom, but the more

difficult case of helium calls for further re- search into the nature of the approximations.

Another random walk method which avoids

the theory of Wiener functionals was developed

by Dr. W. Wasow (8) of the permanent staff of

the Institute. It is so simple that up to a certain

stage the errors of approximation can be studied

analytically more easily than in the Kac method.

However, the Wasow method belongs in a sense

to the category of difference equation methods;

it replaces (1) by the corresponding difference

equation and approximates its eigenvalues, not

those of eq. 1. Thus the method has the disadvantage shared by all difference equation

methods and identified above, i.e., the dubious- ness of the relation between the eigenvalues of a differential equation and those of the corresponding difference equation.

Dr. Wasow sets up the simplest possible

symmetric random walk in the space of the independent variables, very much as Dr. Kac does,

and proceeds to cumulate a multiplicative score

as the walk goes on. If P and Q are two points

of the lattice, and the walk starts at P, then he

lets gn(P,Q) be the mean value of the score at Q

if the walk arrives at Q in exactly n steps.

Under certain conditions, the quantity G(P,Q) = r gj^(P,Q) exists; this is analogous to the

Green's function of the original differential

equation. Obviously gn(P,Q), and, in fact,

g *(p) = 2 g (P,Q), can easily be estimated by

n Q

"

performing the walk many times and keeping

track of the score at each step. Wasow then

sets up a generating function,

CC ir(P. r) = L r" gn'(P) .

13
and shows that its poles have a very simple direct relation to the sought-for eigenvalues. It then remains only to estimate the poles, knowing

the coefficients of \p*. In the case of a bounded

fundamental region, this is facilitated by the fact

that \p* is a rational function of r. Experiments

on the Wasow method are now commencing.

Another interesting method, perhaps closer

to Kac than to Wasow, was proposed by Dr.

Robert Fortet (9) in a recent tour of duty at the

Institute. He considers the analogous problem in

integral equations (it is well known that (1) can be

reduced to an integral equation if the Green's

function of (1) is known). Dr. Fortet's method is

based on an interesting theorem which was first

proved by Kac and Siegert, and which he was the

first to prove in all generality. It runs as follows:

Consider the Fredholm integral equation

u{x) = f(x) + X J

C3(x,£)u<£)(£.

a Let D(a) be its Fredholm determinant. (The solutions of D(X)=0 are the desired eigenvalues.)

Let X(t) be a real Gaussian process whose co- variance function is G. Then

/•b , .1 E [exp(ivl X^(t)dt)] = [D(2iv)] 2 , y real,

•'a All of these results depend on really sophisticated mathematical theory. It is not only

the search for completely novel methods, however, that requires deep mathematical theory.

For example, the Institute has devoted much

thought to developing new methods of finding the

eigenvalues of finite matrices. Although the

methods studied at the Institute have generally

fallen into the classical categories, nevertheless

some serious mathematics has been required in

proving the required convergence theorems.

[See for example (10)]. In addition to solving large systems of

linear equations, and calculating eigenvalues, a

third and now principal area of Institute research

is the development of techniques for numerically

solving differential equations. The orientation

came from certain difficult problems in nonlinear parabolic differential equations brought

into the Mathematical Services Section by nearby

Naval scientific laboratories, notably the Naval

Ordnance Test Station, China Lake and Pasadena.

Monte- Carlo methods have been experimented

with at some length in certain simpler cases,

notably in the Dirichlet problem and conformal

mapping. [A general exposition of the available

techniques is given in (5)].

Problems worked on so far by the Institute are described periodically in quarterly reports

to ONR and in the widely available public quarterly report of theNAML, entitled "Projects and

Publications of the National Applied Mathematics Laboratories." (Copies will be sent

regularly on request to any Naval activity having a need for the information.)

Detailed results have been prepared as papers, of which more than 100 have been submitted to the standard journals since 1 January

1948 for publication. A bibliography of the publications of the Institute to date will be found at the end of this article.

The Institute is an example of a successful

venture of the Government into fundamental re- search. What is its usefulness in a time of more

or less complete mobilization for defense? The

question was foreseen during the planning days

of the NAML and was answered in the Prospectus

for the organization, referred to earlier. Of

the four main guidelines for the program of the

proposed center, the last was identified as follows:

'It should undertake to maintain a reservoir of personnel trained in applied mathematics

which can be drawn on in case of a national

emergency, and should at the same time develop

disciplines and tools to facilitate the conversion

of the nation's peace-time scientific manpower

to emergency uses."

This, of course, applies as much to the Institute for Numerical Analysis as to the larger

organization. The permanent staff, the substantial body of alumni, and the general good-will

among scientists which the Institute has built up,

will all serve as an important asset of scientific

manpower if and when needed. Special problems of immediate importance may then be

undertaken in addition to comprdi ens ive, fundamental attacks on problem types, and some of

the work will, of course, become classified.

The familiarity which the research personnel

and the visitors have obtained with computing

machinery in their experimental work will be

put to good use. But the economical and advantageous solution in mathematics is the one which

applies to awhole class of problems and not just

to the special one in hand. It is hoped that, however pressing the demands are on the Institute

for immediate results of the "quick and dirty"

kind, the organization wUl always continue to find time to do work of broad usefulness in the

science of modern high-speed computing.

(Unclassified)

- 14 -
REFERENCES

(1) Calculating Instruments and Machines, D. R. Hartree. The University of Illinois Press, Urbana, 1949. (2) A Federal Program in Applied Mathematics, J. H. Girtiss. Science, 107 (1948) 257-262.

(3) Zeits. f. Physik, E. A. Hylleraas, 65 , 209 (1930).

(4) Eigenvalue Problems related to the Laplace Operator,

D. A. Flanders and G. H. Shortley. Proceedings of

Seminar on Scientific Computation held by IBM Corp.

64 (November 1949).

(5) Sampling Methods applied to Differential and Difference Equations, J. H. Curtiss. Proceedings of Seminar on Scientific Computation held by lEM Corp. 87 (November 1949).

(6) On distributions of certain Wiener functionals, M. Kac. Transactions of American Mathematical Society,

65, 1-13 (1949).

(7) A sampling method for determining the Icnrest eigenvalue and the principal eigenfunction for Schrbdinger'

s

equation, M. Kac and M. D. Dmsker. ^BS Journal of

Research, 44, 551 (May 1950).

(8) Rendooi Walks and the Eigenvalues of Elliptic Differ- ence Equations, W. Wasow. NES Journal of Research,

46, 65 (January 1951).

(9) On the estimation of an eigenvalue by an additive

functional of a stochastic process, with special

reference to the Kac-Donsker method, R. Fortet. To appear in the Journal of Research of the National

Bureau of Standards.

(10) The method of gradients for the calculation of the characteristic roots and vectors of a real synmetric

matrix, M. R. Hestenes and W. Karush. To appear in the Journal of Itesearch of the National Bureau of Standards.

Appendix

Bibliography of the Institute for Numerical Analysis

1. L. L. Bailin. An analysis of the effect of the discontinuity in a bifurcated circular guide upon plane

longitudinal waves. Accepted for publication in the NBS Journal of Research.

2. E. F. Beckenbach. On subharmonic, harmonic and linear

functions of two variables. Accepted for publication

inRevista (UniversidadNacional deTucuman, Argentina).

3. E. C. Beckenbach, and E. W. Graham. Oh subordination

in complex variable theory. To be included in "The

constnicticm and applications of conformal maps: Pro- ceedings of a Symposium held June 22-25, 1949," at the NBS Institute for Numerical Analysis, Los Angeles,

California, to be published in the NBS ^plied Mathematics Series.

4. E. F. Beckenbach, W. Seidel and 0. Szasz. Recurrent

determinants of orthogonal polynomials. Accepted for publicaticoi in the Duke Mathematical Journal.

5. E. F. Beckenbach. A class of mean value functions.

Am. Math. Mo. LVII, 1-6 (Jan. 1950).*

6. E. F. Beckenbach. Metric differential geometry.

Math. Mag. 23, 143-152 (Jan-Feb 1950).'

7. G. Blanch and R. Siegel. Table of modified Bernoulli

polynomials. NBS J. Res. 44, 103-107 (Jan. 1950),

RP2060.

8. R. H. Cameron. A "Simpson's Rule" for the numerical

evaluation of Wiener's integrals in function space. Accepted for publication in the Duke Mathematical

Journal

.

9. M. L. Cartwright. Forced oscillations in nonlinear

systems. NBS J. Res. 45, 514-518 (Dec. 1950). PP2166.

10. J. H. Curtiss. Sampling methods applied to differential and difference equations, with special reference to equations of the elliptic type. Proceedings, Scientific Computation Forum, held by the International

Business Machines Corporation, Endicott, N. Y.

,

November 1949, p. 87-109.'

11. R. E. Cutkosky. A Monte Carlo method for solving a class of integral equations. Accepted for publication

in the NBS J. Res.

12. M. n. Donsker and M. Kac. A sampling method for

determining the lowest eigenvalue and the principal

eigenfunction of Schrodinger' s equation. NBS J. Ffes. 44, 551-557. (May 1950), RP2102.

13. A. Dvoretzky, A. Wald, and J. Wolfowitz. Relations

among certain ranges of vector measures. Accepted

for publication in the Pacific Journal of Mathematics.

14. A. rVoretzky, A. Wald and J. Wolfowitz. Elimination

of randomization in certain statistical decision

procedures and zero-sum two-person games. Sukmitted

to a technical journal for publication.

15. A. Dvoretzky and P. Erdos. Some problems on random

walk in space. To be included in the Proceedings of

the Berkeley Second Symposiun on Mathematical Statistics and Probability, University of California, 1950.

16. P. Erdbs. Qi a recursion formula and on seme Tauberian

theorems. Accepted for publication in the NBS Journal

of Research.

17. G. E. Forsythe. Second order determinants of Legendre

polynomials. Accepted for publication in the Duke

Mathematical Journal.

18. G. E. Forsythe. Exact particle trajectories for non- viscous flow in a plane with a constant coriolis parameter. J. Meteorology 6, 337-346 (Oct. 1949).

19. G. E. Forsythe. Solution of the telegrapher's equa- tion with boundary conditions on only one characteristic. NBS J. Res. 44. 89-102 (Jan. 1950), RP2059.

20. G. E. Forsythe -uid R. A. Leibler. Matrix inversion by

a Monte Carlo method. ^^AC IV, 127-129 (July 1950).*

21. G. E. Forsythe and J. W. Tukey. The extent of n randan unit vectors. Submitted to a technical journal

for publication. ^ 22. W. Feller and 0. E. Forsythe. New matrix transformations for obtaining characteristic vectors. Accepted

for publication in the Quarterly of Applied Mathematics.

23. G. E. Forsythe. Generation and testing of random

digits at the National Bureau of Standards, Los

Angeles. To be included in 'The Monte Carlo method: Proceedings of a Symposium held June 22-25, 1949 at the NBS Institute for Numerical Analysis, Los Angeles,

California," to be published as NBS Applied Mathematics Series 12. 24. R. Fortet. Random determinants. Accepted for publication in the NBS J. Res.

25. B. Fortet. Oi some functionals of Laplacian processes. .Submitted to a technical journal for publication.

26. R. E. Greenwood. Numerical integration for linear

suns of exponential functions. Ann. Math. Stat. 20 , 608-611 (Dec. 1949).*

27. D. R. Hartree. Note on systematic rounding-off errors

in numerical integration. An appendix in J. Res. 42, 57 (Jan. 1949).

28. S. Herrick. 1949 Ephemeris of Jupiter's 9th satellite. Pub. Astronomical Soc. Pac. 61, No. 360, 136 (June 1949).

29. M. R. Hestenes. An elementary introduction to the calculus of variations. Math. Mag. XXIII, 249-268

(May-June 1950).'

30. M. R. Hestenes. Applications of the theory of quadratic forms in Hilbert space to the calculus of

variations. Submitted for publication to a technical

journal.

31. M. R. Hestenes and W. Karush. A method of gradients

for the calculation of the characteristic roots and vectors of a real synmetric matrix. Accepted for

publication in the fiGS J. Res.

32. M. R. Hestenes and W. Karush. The solutions of Ax = XBx. Submitted to a technical journal.

- 15 -
33. H. D. Huskey. Oh the precision of a certain procedure

of numerical integration. J. Res. 42, 57-62 (Jan.

1949) , RP1950.

34. H. D. Huskey. Characteristics of the Institute for Numerical Analysis computer. MTAC IV, 103-108

(Apr. 1950).*

35. H. D. Huskey. Electronic digital computing in England.

MTAC III, 213 (July 1948).

36. H. D. Huskey. Klectronic digital computing in the United States. Report of a Conference cn High-Speed

Automatic Calculating Machines, 22-25 June 1949, University Mathematical Laboratory, Cambridge, England,

p. 109-112.

37. H. D. Huskey. Semi-automatic instruction on the

Zephyr. To appear in the Proceedings of a Symposiun

on Large-Scale Digital Calculating Machinery, Harvard

University, September 1949.

38. H. D. thiskey and V. R. Huskey. A new era in calculating machines. Office Appliance Trade Journal

(Jan. 1949).

39. H. D. Huskey and V. R. Huskey. High speed comuting and accounting. Submitted to the Journal of

ccountancy.

40. M. Kac. Ch some connection^ between probability theory

and differential and integral equations. To appear

in the Proceedings of the Berkeley Second Symposium

on Probability, university of California, 1950.

41. W. Karush. An iterative method for finding characteristic vectors of a symnetric matrix. Submitted to a technical journal for publication.

42. W. Karush. Determination of the extreme values of

the spectrum of abounded self-adjoint operator. Submitted to a technical journal for publication.

43. C. Lanczos. An iterati've method for the solution of

the eigenvalue problem of linear differential and

integral operators. NBS J. Res. 45, 255-282 (Oct.

1950) , RP2133. —

44. C. Lanczos. Introducticm to the Tables of the Chebyshev polynomials CJ, (x) and S^(x). For inclusion in the NBS Confutation Laboratory's volume of the tables, to be published as NBS Applied Mathematics Series 9. 45. E. Lukacs and 0. Szasz. Certain Fourier transforms

of distributions. Accepted for publication in the Canadian Journal of Mathematics.

46. W. E. Milne. Note on the Runge-Kutta method. ^BS J. Res. 44. 549-550 (May 1950), PP2101.

47. W. E. Milne. The remainder in linear methods of

approximation. NBS J. Res. 43, 501-522 (Nov. 1949),

RP2041.

48. W. E. Milne. A note on the numerical integration of

differential equations. NBS J. Res. 43, 537-542

(Dec. 1949), nP2046.

49. W. E. Milne. Nimerical determination of characteristic numbers. NBS J. Res. 45, 245-254 (Sept. 1950),

RP2132.

50. W. E. Milne. Numerical methods associated with

Laplace's equation. To appear in the Proceedings of

a Symposium on Large Scale Digital Connuting Machinery, Harvard University, September 1949.

51. T. S. Motzkin. Two consequences from the transposition theorem on linear inequalities. Subnitted to a technical journal for publication.

52. A. M. Ostrowski. On some generalizations of the

Cauchy-Frullani integral. Prox. Nat. Acad. Sci. 35, 612-616 (Oct. 1949).

53. A. M. Ostrowski. Note on Vincent's theorem. Ann. Math. 52, 702-707 (Nov. 1950).'

54. A. M. Ostrowski. (jeneralization of a theorem of

Osgood to the case of continuous approximation. Ac- cepted for publication in the Bulletin of the Ameri- can Mathematical .Society.

55. A. M. Ostrowski. Note on an infinite integral. Sub- mitted to a technical journal for publication.

56. A. M. Ostrowski. On two problems in abstract algebra

connected with Homer's rule. Submitted to a techni- cal journal for publication.

57. A. M. Ostrowski. On a discontinuous analogue of

Theordorsen 's and (jarrick's method. To appear in "The

construction and application of conformal maps: Pro- ceedings of a Symposium held at the rCS Institute for

Numerical Analysis, Los Angeles, California, June

1949," to be published in the NBS Applied Mathematics

Series.

58. A. M. Ostrowski. On the convergence of Theordorsen 's and (jarrick's method. To appear in "Hie construction

and application of conformal maps: Proceedings of a Symposium held at the NBS Institute for Numerical

Analysis, Los Angeles, California, June 1949," to be published in the NBS Applied Mathematics Series.

59. (A. M. Ostrowski) Ch the remainder of the Runge-Kutta

formula in the theory of ordinary differential equations. Connunicated by L. Bieberbach; revised and

edited by A. M. Ostrowski. To be published in Zeitschrift fur angewandte Mathematik und Physik.

60. P. G. Hoel and R. P. Peterson. A solution to the problem of optimum classification. Ann. Math. Stat. XX, 433-438 (Sept. 1949).

61. J. B. Rosser. Transformations to speed the convergence of series. NBS J. Res. 46, 56-64 (Jan. 1951),

RP2175. ~

62. J. B. IVisser. A method of computing exact inverses

of matrices with integer coefficients. Accepted for publicaticHi by the J. Res.

63. J. B. Rosser. Real roots of real Dirichlet L-series.

NBS J. Res. 45, 505-514 (Dec. 1950), FP2165.

64. J. B. Rosser, C- Lanczos, M. R. Hestenes, and W. Karush. The separation of close eigenvalues of a real symmetric matrix. Accepted for publication in the NBS J. Res.

65. W. Seidel. A bibliography of numerical methods in conformal mapping. To appear in "Tlie construction and applications of conformal maps: Proceedings of a Symposium held at the NBS Institute for Numerical

Analysis, Los Angeles, California, June 1949," to be published in the Applied Mathematics Series.

66. W. Seidel and 0. Szasz. Cki positive harmonic functions

and ultraspherical polynomials. Accepted for publication in the Journal of the London Mathematical Society.

67. M. Stein. On methods for obtaining solutions of fixed end point problems in the calculus of variations.

Doctorate thesis, in cooperation with the Uiiversity

of California at Los Angeles.

68. 0. Szasz. Gibbs' phenomenon for Hausdorff means.

Trans. Am. Math. Soc. 69, 440-456 (Nov. 1950).

69. 0. Szasz. Ch a sunnation method of 0. Perron. Math. Zeitschrift 52, 1-7 (1949).'

70. 0. Szasz. Inequalities concerning ultraspherical

polynomials and Bessel functions. Proc. Am. Math.

Soc. 1. 256-267 (Apr. 1950).*

71. 0. Szasz. Sumnation of slowly convergent series with

positive terms. .J. Math. Phys. 28, 272-279 (Jan.

1950).* ~

72. 0. Szasz. A generalization of S. Bernstein's polynomials to the infinite interval. NBS J. Res. 45,

239-245 (Sept. 1950), RP2131. ~

73. 0. Szasz. Ch the Gibbs' phenomenon for Euler means. Acta Scientiarum Mathematicarum 12, 107-111 (1950).*

74. 0. Szasz. Ch some trigonometric transforms. Submitted

to a technical journal for publication.

75. 0. Szasz. Oi the relative extreme of the Hermite

orthogonal functions. To appear in BoUetino della

Unione Matematica Italian (Firenze).

76. 0. Szasz. Ch a Tauberian theorem for Abel sunnability.

Accepted for publication in the Pacific Journal of

Mathematics.

77. 0. Szasz. Tauberian theorems of suinnability (Ri).

Accepted for publication in the American Journal of

Mathematics.

78. 0. Szasz. Ch the relative extreme of Bessel functions.

Accepted for publication in Bolletino della Unione

Matematica Italian (Firenze).

79. 0. Szasz. Identities and inequalities concerning

orthogonal polynomials and Bessel functions. Submitted to a technical journal for publication.

80. 0. Szasz. On the Gibbs' phenomenon for a class of

linear transforms. To be published in "Publications

de rinst. Math, de I'Acad. Serbe des Sciences,"

vol. IV.

81. 0. .Szasz and J. Todd. The convergence of CauchyRiemann sums to Cauchy-Riemann integrals. Accepted

for publication in the NBS J. Ifes.

- 16
82. 0. Taussky. A recurring theorem on determinants. Am. Math. Mo. LVI, 672-676 (Dec. 1949).*

83. 0. Taussky. A remark concerning the characteristic

roots of the finite segments of the Hilbert matrix.

Oxford Qu. J. i, 8-83 (June 1949).

•

84. 0. Taussky. Bounds of characteristic roots of matrices. Duke Math. J. 15, 1043-1044 (Dec. 1948).*

85. 0. Taussky. Ch a theorem by Latimer and Macduffee.

Canadian J. Math. J,, 300-302 (1949).*

86. 0. Taussky. Notes on numerical Analysis, 2: Note

on the condition of matrices. ^f^AC IV, 111-112

(Apr. 1950).'

87. 0. Taussky. Classes of matrices and quadratic fields. Accepted for publication in the Pacific Journal of

Mathematics.

88. 0. Taussky. Bounds for characteristic roots of

matrices, II. NBS J. Res. 46. 124-125 (Feb. 1951), RP2184.

89. 0. Taussky and L. A. Wigglesworth. Note on a theorem' in n-diinensional geonetry. Am. Math. Mo. 55 , 492-494

(Oct. 1948).* ~

90. J. Todd. On the relative extreme of the Laguerre

orthogonal functions. Bolletino della Unione Mate- matica Italiana (Firenze) Ser. Ill, Ann. V., 122-125

(1950).'

91. J. Todd. Table of Arctangents of rational nunbers. To

be published as NBS Applied Mathematics Series 11. 92. J. Todd. A problem on arctangent relations. An. Math.

Mo. 56, 517-528 (Oct. 1949).'

93. J. Todd. Notes on modem nunerical analysis, I. MIAC

IV, 39-44 (Jan. 1950).'

94. J. Todd and S. D. Chowla. The density of reducible

integers. Canadian J. Math. _1, 297-299 (1949).

95. S. E. Warschawski. On conformal mapping of variable

regions. To appear in "The construction and applications of conformal maps: F'roceedings of a Symposium

held at the rfiS Institute for Nunerical Analysis, Los Angeles, California, June 1949," to be published in the NBS Applied Mathematics Series.

96. S. E. Warschawski. On the degree of variation in conformal mapping of variable regions. Trans. Am. Math. Soc. 69, 335-356 (Sept. 1950).'

97. W. Wasow. On the mean duration of randooi walks. Ac- cepted for publication in the ^BS J- Res98. W, Wasow. Randcm walks and eigenvalues of elliptic

difference equations. J. Res. 46, 65-73 (Jan. 1951),

RP2176.

99. W. Wasow. On the duration of randcm walks. Subnitted to a technical journal for publication.

100. E. C. Yowell. Numerical solution of partial differential equations. To appear in the Proceedings of a Scientific Coi^utation Forum held by the International

Business Machines Corporation, Endicott, N. Y.

,

December 1949.

101. E. C. Yowell. A Monte Carlo method for solving

Laplace's equation. To appear in the Proceedings of

a Scientific Computation Forum held by the Inter- national Business Machines Corporation, Endicott,

N. Y., December 1949.

102. The Monte Carlo method: Proceedings of a Symposim

held on June 29, 30, and July 1, 1949, in Los Angeles,

California, under the sponsorship of the RAND Corporation, and the National Bureau of Standards, with

the cooperation of the Oak Ridge National Laboratory.

To be published as I^GS ^plied Mathematics Series 12. 103. Problems for the numerical analysis of the future. Four papers presented at a Symposium on Numerical

Analysis and Automatic Calculating Machinery, at the NBS Institute for Numerical Analysis, June 1948. To

be published as NBS Applied Mathematics Series 15. 104. The construction and applications of conformal maps: Proceedings of a Symposiun held June 22-25, 1949, at the Institute for Numerical Analysis, Los Angeles,

California. Edited by E. F. Beckenbach. To be published in the NBS Applied Mathematics Series.

105. Tables relating to the Mathieu functions. In press, Columbia University Press.

106. (G. E. Forsythe) Various techniques used in connection

with randcm digits, by J. von Neumann. Sunnary by

G. E. Forsythe. To appear in "The Monte Carlo

method: Proceedings of a Symposiun held June 22-25,

1949, at the NBS Institute for Numerical Analysis, Los Angeles, California," to be published in the NBS

Applied Mathematics Series.

Items to be published, now In Manuscript

1. S. Derrick. Tables for rectilinear and nearly

rectilinear ('nearly parabolic") orbits.

2. Tables of Ej^(z), second quadrant.

3. Special table of Bessel functions (Bessel -Clifford).

4. L. Ahlfors. Numerical methods in conformal mapping.

5. Conformal representation of simply- and multiplyconnected regions, by L. Kantorovitch and others; translation from the Russian by W. Seidel.

Note: Reprints of items published in the I^GS Journal

of Research are for sale by the Superintendent of

Documents, U. S. Government Printing Office, Washington 25, D. C. Reprints of starred items are available from the National Bureau of Standards, Institute

for Numerical Analysis, UCLA 405 Hilgard Avenue,

Los Angeles 24, California.

(Ihclassified)

- 17 -
APPENDIX C

THE SWAC

The building of the SWAC was one of the important accomplishments of INA. So far in the

text we emphasized the mathematical accomplishments of INA and said very little about the construction of the SWAC. To complete the story of INA we now include an article by Harry D.

Huskey describing the story of the SWAC. To obtain a comprehensive overview of computing

machines developed at about this time and earlier, we refer the reader to the important paper by

Mina Rees entitled "The Federal Computing Machine Program," Sdence 112 (1950), 731-736.

C-1

A HISTORY OF COMPUTING IN THE TWENTIETH CENTURY

The SWAC: The National Bureau of Standards

Western Automatic Computer

HARRY D. HUSKEY

1. Background

The SWAC had its beginning at the 19 October 1948 meeting of the Applied Mathematics Executive Council held at the National Bureau of Standards (NBS) in Washington. D C. This Executive Council served as an advisory body to the National Applied Mathematics Laboratories, which was a

division of the NBS.

The Mathematics Laboratories had been established in 1945 through a "suggestion" by the Navy Department to the Director of the Bureau of

Standards. Dr. Edward U. Condon. The Navy hoped that the Bureau would

establish a centralized national computation facility, equipped with highspeed automatic machinery, to provide computing service to other government agencies and to play an active part in the further development of computing machinery. Dr. Condon complied, setting up the National Applied

Mathematics Laboratories, with Dr. John Curtiss as Chief. The Laboratories were to have four main parts: the Computation Laboratory, the Machine Development Laboratory, and the Statistical Engineering Laboratory,

all in Washington. D.C.. and the Institute for Numerical Analysis (INA), a

field station to be located near some university in California.

The success of the ENIAC had excited mathematicians and other scientists to the possibilities now opening before them. No company was yet tum419

Copynght cl 1980 by Academic Press. Inc.

All hghis of reproduction in any form reserved. ISBN 0-12-49I650-3

Copyright © 1980 by Academic Press, Inc. Reprinted by permission

of the publisher and reprinted with permission by Harry D. Huskey.
420 HARRY D. HUSKEY

ing out electronic computers, but several had become interested in trying.

University scientists, encouraged by the University of Pennsylvania's success in the field, were also attempting to build computers for their own

use. Government agencies, quick to see the potentials of an electronic computer, were eager to acquire one. However, the field was new, there was no

background of experience, and no one was absolutely certain what type of

computer would best suit his purpose, or even what company was most

likely to build a workable computer within a reasonable time. Therefore,

government agencies were glad to ask the NBS to assist them in negotiating

with computer companies. In early 1948. the Bureau had begun negotiating

with the Ecken-Mauchly Computer Corporation and the Raytheon Corporation, and later with Engineering Research Associates.

The computers were slow in being developed. New techniques were

being tried and often they did not work as well, or as soon, as had been first thought, or hoped. The personnel of the Applied Mathematics Laboratories

became impatient with this slow development, and decided that they could

build one faster with the help of the Electronics Laboratory at the Bureau.

Also, it had become clear that in order to be able to judge effectively the

probability of a new technique working they would need more "hands-on"

expertise. Dr. Edward Cannon and the author convinced Dr. Curtiss that

this "gamble" was worth trying, and Dr. Mina Rees of the Office of Naval

Research backed them up. This was in spite of the advice of a committee,

consisting of Dr. George Stibitz, Dr. John von Neumann, and Dr. Howard

Aiken, which had been asked by Dr. Curtiss to consider the Bureau's role in

the computer field. Their advice had been that the NBS shouldn't really

work on computers, but should confine its work tp improving components.

In May 1948. the decision was made at the Executive Council to build a machine for the Bureau's own use in Washington. At that meeting it had also

been decided that the Bureau should buy three UNIVAC's which were being

developed by the Eckert-Mauchly Computer Corporation. One of these

was to go to the Census Bureau, one to the Air Materiel Command in Dayton, Ohio, and the third to the INA. Later, due to a security problem that

had arisen in the company, it was decided that the military funds could not

be used to purchase UNIVACs. This reopened the question of procuring a computer for the Air Materiel Command and for the INA. Thus, at the October 1948 meeting of the Executive Council it was decided that the Bureau

should build a second computer at the Institute for Numerical Analysis,

which had by now been located in a reconverted temporary building on the

campus of the University of California at Los Angeles. This machine was to

be built under the direction of the author, who had joined Curtiss's group in

January 1948. He had spend the previous year at the National Physical Laboratory in Teddington, England, working under Alan Turing with James Wilkinson and others on the Automatic Computing Engine (ACE) project. He
SWAC 421

had been offered the job there on the recommendation of Professor Douglas

Hartree, whom he had met while working on the ENIAC project.

2. The Institute for Numerical Analysis

In December 1948, the author transferred to the Institute for Numerical

Analysis, and in January 1949. work started on the INA computer. The computer at the NBS in Washington followed the EDVAC (University of Pennsylvania) design using mercury delay lines for memory. The Executive

Council, which included representatives of the U.S. Air Force and the Office of Naval Research, felt that the NBS should not build the same type of

machine that others were already building, or planning to build. The author

had become interested in the possibility of using cathode-ray tubes for stor- age while in England, where he had seen the work being carried on at Manchester University under Professor F. C. Williams, and the proposal was

made that this type of computer be built at INA. As a precaution, the Council wanted it to be designed in such a way that if it didn't work it could be

converted to a magnetic drum computer. Of course, we had no doubt that it would work.

Finances were tight. Three hundred thirty thousand dollars had been

transferred to the NBS to cover the cost of both the INA computer and the

one for the Air Materiel Command (which was to be contracted to the

Raytheon Corporation). Dr. Curtiss noted in his 1953 progress report on the

Mathematics Laboratories, "The project was handicapped throughout by

having much too tight an annual budget' "[1].

At INA an empty room was given to the author and he was told to "go

ahead." Not only did personnel have to be recruited to assist in designing

and constructing the computer, but also machine shop equipment and supplies had to be procured from scratch. Fortunately, the Bureau had just completed a study contract with Eckert-Mauchly so we asked for all the machine tools acquired on that contract to be shipped to Los Angeles. There

was also the race against time. After all, part of the justification for the

NBS's building its own computers was the slowness of the would-be commerical suppliers in putting workable machines on the market. Consequently, construction of the computer began even while the study of the general machine organization and logical system was still under way. The

development of the machine system, circuitry, and building techniques proceeded simultaneously with the actual construction of the machine.

The assembly of the computer was completed by July 1950. It was formally dedicated on 17-19 August. The opening session on the afternoon of

the 17th featured speeches by Dr. Condon; Colonel F. S. Seller. Chief of the

Office of Air Research, USAF: Dr. L. N. Ridenour, Dean of the Graduate
422 HARRY D. HUSKEY

School at the University of Illinois: Dr. Curtiss: and the author. The real

highlight of the program, of course, was a demonstration of the computer.

The second day consisted of a symposium, opened by Dr. Condon, on

the applications of digital computing machinery to scientific problems. Paul

Armer. Leland Cunningham. Samuel Herrick. Stanley Frankel, Derrick

Lehmer. and Jerzy Neyman were among the speakers. The third day was

spent mainly in demonstrations of the SWAC.

3. Naming the Computer

The name of the computer had undergone several changes during its construction. In a talk that the author gave at the Second Symposium on LargeScale Digital Calculating Machinery, held at the Computation Laboratory at Harvard University on 13-16 September 1949. the machine was called the

ZEPHYR. This name had been chosen to emphasize the modest nature of

the effort, "a gentle wind from the west." in contrast to other projects carrying names such as TYPHOON. HURRICANE, WHIRLWIND. Good-natured rivalry with the group at NBS in Washington caused them to suggest

SIROCCO (a hot wind from the desert) as a substitute for ZEPHYR.*

In retrospect it was clear that ZEPHYR was not a strategic choice of

name. Hence, the very prosaic name. Institute for Numerical Analysis Computer, was used for a time, and appears in an article published in [3]. Early

in 1950, someone in the administration at NBS suggested that the names of

the computers being built at the Laboratories in Washington and at INA

be tied together to the glory of NBS. Hence came the name National Bureau

of Standards Western Automatic Computer, which in the style of the times

lent itself nicely to being shortened to the name SWAC. Similarly, the NBS

computer in Washington was initially called the NBS Interim Computer.

Later it became the National Bureau of Standards Eastern Automatic

Computer or SEAC. It was constructed under the direction of Dr. Samuel

Alexander.

4. Project Organization and Staff

The project was divided into three major pans: memory, arithmetic, and

control. B. Ambrosio, with the help of Harry Larson, was to handle the

memory. Bill Gunning visited from Rand Corporation and helped with the

memory until he broke a leg skiing. Edward Lacey worked on the arithmetic

unit, making use of the type of circuitry developed at MIT on the WHIRLWIND project. David Rutland worked on the control unit, and we pro-

' Independently. H. Zamenek of Vienna. Austria, named the machine built under his direction MAILUEFTERL (gentle wind, also as a joking comparison).
SWAC 423

ceeded to put together a computer. R. Thorensen joined the group later and

worked on the magnetic drum. Other parts such as power supply and inputoutput, all being more straightforward (requiring less development), were

handled by the group in a less explicit way

By July 1950. the staff consisted of three engineers, three junior engineers, and four technicians, in addition to the author. Everyone knew that

what was being done was pioneer work on a new frontier and excitement

abounded. A good-natured comradely rivalry developed between the two

projects at the Bureau, as well as with similar projects elsewhere. There was

a good deal of open exchange of information and techniques, and no one

doubted that significant progress was being made on all fronts. The staff

worked long and irregular hours uncomplainingly, spurred on by the eager

interest of the distinguished internationally known scientists working at the

INA. All of them were eagerly looking forward to having such a tool as the

SWAC at their disposal.

The computer was to be parallel, using for its memory standard commerically available cathode-ray tubes (in contrast to some projects that were ex- perimenting with specially built tubes). This decision was made both in the

interests of time and of money. In fact, components that were mass produced commerically were used wherever possible in the SWAC. The

reasoning behind this decision was that they could be easily replaced, were

relatively economical, and, in general, could be expected to have more reliability. To further assist in servicing and to decrease the down time for maintenance, it was also decided to have all circuitry on removable plug-in

chassis, with spare plug-in units for about 80% of the computer. Thus, in case of the failure of some component the faulty chassis could be removed

and replaced by a spare one. This type of construction was especially

important in those early days when no one knew just how much reliability

one could expect from the components. Laboratory test equipment was constructed so that faulty chassis could be repaired in the laboratory without

use of computer time. The majority of the plug-in chassis contained an aver- age of 10 tubes each; however, the magnetic drum circuits, which were built

later, were smaller, usually having a single tube per plug in unit.

Besides the 37 cathode-ray tubes used in the memory, the SWAC

contained 2600 tubes and 3700 crystal diodes. The average tube life was

between 8000 and 10,000 hours. Most tube failures resulted from low emission and intermittent shorts, and not from heater failure.

5. Memory

As noted earlier, the author had visited F. C. Williams and his staff at Manchester University, and observed there the method of storing information in cathode-ray tubes. One of the main appeals of the Williams system

was the high speed of computation possible because of random access to
424 HARRY D. HUSKEY

memory locations. Numbers could be transferred in parallel instead of by

serial pulse trains. However, the machine being designed at Manchester was

serial by bit in operation, which meant that information was transferred to or from one tube at a time. In order to take full advantage of the possible speed

of the Williams tube storage the SWAC was designed to operate in parallel.

That is, information would be transferred in and out of all memory tubes

simultaneously. This may have increased the headaches during construction

but it did pay off in speed. At the time of its dedication in August 1950. the

SWAC was the fastest computer in existence, being able to do 16,000 threeaddress additions or 2600 such multiplications per second.

The main disadvantage of the cathode-ray type of storage on the SWAC

was its relatively small size. The SWAC stored 256 words of 37 binary digits

(bits) each. To do this 37 cathode-ray tubes were used, with the various bits

of a particular number being stored in corresponding positions of each of the

tubes. At first it had been hoped that it would be possible to push the highspeed memory above 256 words (to 512 or even 1024 words). However,

memory difficulties prevented this from happening. In this type of memory

the individual digits of information were stored as spots of charge that existed over small areas inside the face of the tube. These spots were arranged

in a rectangular array on the face of the tube. Two different charge distributions, providing the two states needed to represent a binary digit, could be

produced at each spot. These spots had either a dot or a dash appearance,

the dot corresponding to a 0 and the dash to a 1 . A monitoring tube mounted

on the console showed the dot-dash pattern in terms of zeros and ones on

any one of the 37 memory tubes depending on an appropriate switch setting.

This type of memory required regeneration. Unless this was done the

original charge pattern would tend to disappear over a period of time as the

charged spots collected stray electrons. We always knew this would happen,

so the memory cycle was made 16 fxsec long, with an 8-/xsec action cycle followed by an 8-/xsec restore cycle. During the action cycle operands were

transferred from the memory to the arithmetic unit, results were transferred

back to the memory, or the next instruction was transferred to the control

unit. In the restore, or regeneration, cycle one of the spots was restored to

its initial value. Thus, in 256 restore cycles (about 4 msec) the whole memory

would have been regenerated. Under these conditions, information could be

stored in the memory indefinitely. A crystal-controlled oscillator regulated

both the rate of regeneration of the memory and the synchronization of the

control circuitry.

There were two main features of the Williams-tube type of memory that

gave us considerable trouble, and prevented us from storing more information

on each tube. One of these was the presence of so-called flaws on the inside of

the face of the tube, which prevented it from storing information satisfactorily. We found that each particular tube had three or four such spots on it.
SWAC 425

These spots turned out to be small carbonized particles of lint. The problem

was that we had to build 37 tubes, deflecting all the 256 bits on the faces in

unison so that no one of the bits would land on any of these carbonized

spots. A storage spot might be alongside one of the flaws, and any slight drift

might bring the spot onto it. It was only after it was too late to look for an- other supplier that we learned that our tubes were being manufactured in a reconverted mattress factory! That no doubt had increased our difficulties

substantially; however, all commerically manufactured cathode-ray tubes

had some flaws in them. To minimize this difficulty tubes were carefully se- lected, and the location of the memory array was adjusted so as to avoid bad

spots. We also spent a lot of effort producing extremely stable power supplies, so that we could control drift for reasonable periods of time in order to

avoid trouble from such spots.

Another major problem with the memory was "spill-over."" or redistribution of charge. Each memory access generated a cloud of secondary electrons that "rained down"" on the neighboring charged spots. This limited the

number of times the neighboring spots could be read before the spot in question had to be regenerated. The term read-around ratio indicated how many

times one could look at a given spot before the neighbors were ruined. A

program might access spots in a given area of the tube many times before the

regeneration occurred. One was between the devil and the deep blue sea—if one adjusted parameters in one direction flaws were less troublesome, but

the read-around ratio would collapse. If one tried to improve the read around

ratio with sharper focus, then flaws were more of a problem. It was spillover that kept us from increasing the memory size above 256 words.

Since it was not practical to enlarge the size of the high-speed memory, it became imperative to have auxiliary storage. It was decided to use a magnetic drum for this purpose. At first it was planned to use a magnetic drum built

by Professor Paul Morton at the University of California. Berkeley. In those

early days drums were made from extruded aluminum tubing. There are

stresses in the metal and, as time passes, the material flows. This happened

to our drum. Although it was round when delivered to us, it was no longer

round by the time we had built circuitry and tried to connect it to the computer. There was much folklore around, such as "haul the tubing around in

the trunk of your car for six months so that it would be relaxed."" Ultimately

we purchased another drum from a commerical supplier and connected it to

the computer. The drum used had 4096 words of memory. Average access

time per word v/as about 500 ptsec, in contrast to the cathode-ray tube access

time of 16 /u,sec. Transfers to and from the drum were most efficiently handled in blocks of 32 words, which was the number of words on any track on

the drum. The transfer would begin with the first number that became available and persisted for precisely one revolution of the drum. The drum ro- tated at 3600 revolutions per minute, so 32 words were transferred in ap-
426 HARRY D. HUSKEY

proximately 17.000 /isec. Since there was no wait time in this mode, the access time per word was about 500 /xsec. The drum transfer instruction also

allowed for transfers of 8 or 16 words. However, these took the same time as a full 32-word transfer.

6. Arithmetic

The arithmetic unit used three 37-bit registers. A memory buffer register

supplied operands from memory. There were 37 binary adders that could

add this operand to the contents of the accumulator (the second register).

The third register (called R) stored the multiplier and parts of the product

during multiplication.

The maximum carry propagation time (through 37 bits) was 6 /isec. Negative numbers were stored in memory as sign and absolute value. Negative

operands were complemented in the memory buffer. The timing was synchronous with sufficient time being allowed for maximum carry propagation.

This timing very nicely matched the 8-/xsec memory access cycle.

At the time of its dedication it was the fastest computer in existence,

being able to do a three-adress (C = A + B) addition in 64 fxsec and a similar

multiplication in 384 /xsec.

High-current low-impedance circuits were used in the arithmetic unit to give a fast carry so that multiplication would take the minimum time. Multiplication was by repeated addition with shifting occurring on an 8-/i,sec cycle.

The least significant bits were processed first so that carry was at most

across 37 bits. The use of high-current circuits caused substantial change in

power supply current when operands were all ones versus all zeros. This

change was a much as 15 amperes. A neat diagnostic test was discovered

after some months of operation. It consisted of multiplying ones by ones for a tenth of a second, then running the diagnostic program the next tenth.

Since the power supply filtering was designed only for three phase full wave

(equivalent to 360 cycles) rectification, this change in load caused changes in

voltage levels of about 20 volts. When the diagnostic routines ran under

these conditions, no known failures occurred on general problems.

7. Input and Output

Initial input and output were by typewriter and punched paper tape. The

typewriters (Flexowriters) and the tape stations required substantial mainte- nance and were terribly slow. Therefore, we soon connected an IBM collator (077) and a card punch (513) for input and output. The collator read cards

at 240 per minute and the punch punched at 80 cards per minute. Seventyeight bits (2 words) were read in from each row on the card and 1 1 rows were

used. Substantial computing could be done between rows on the card.
SWAC 427

8. Control

The SWAC used eight basic instructions: add: subtract; multiply

rounded; product (two-word answers); compare; extract; input; and output.

There was a variation of the compare instruction, which compared absolute

values.

Two principles were followed in deciding upon this list of basic instructions. The first was that there should be as few instructions as practical in order

to simplify the electronic circuitry of the computer, and to permit speedy

construction. The second was that the instructions should be sufficiently

general to do scientific computation. The SWAC used a four-address instruction. A floating-point interpretive system called SWACPEC was later developed, which made it easier for users to write programs.

9. Other Details

The SWAC was small in size compared to most of the computers of its time. The units were mounted in three connected cabinets, which were made

to order by a local manufacturer. The memory and control were on one side

and back-to-back with it was the arithmetic unit. The total size of these cabinets was approximately 12 ft wide, 5 ft deep, and 8 ft high. For esthetic reasons the cabinets had glass (shower) doors. The building was made of

wood and had fire sprinklers mounted in the ceiling. This led to a good deal

of kidding from the INA staff saying that there was something wrong in the

relative location of .he sprinklers and the shower doors.

The operating console was an ordinary desk with specially built panels

mounted on its top surface. The actual operation of the SWAC took place

from it. In addition to the memory monitoring tube already mentioned, there

were neon lights and another cathode-ray tube that indicated the address in

memory involving memory accesses during action cycles. The desk and cabinets were a tan color. The entire computer was located in a room 40 by 30 ft. with power supplies, a motor generator, and an air conditioner located

elsewhere.

In the course of operation, certain logical facilities were added to make

the operation more efficient. One of the more interesting of these was the

addition of a loudspeaker and plug-in arrangement allowing the operator to

"listen" to any of the instructions in a problem. For example, an alternate

succession of add and subtract instructions produced an 8-kHz note. One of

the problems run on the SWAC involved the generation of pseudorandom

digits. The corresponding sequence of tones was christened the Random

Symphony.

Being afraid of trouble from power line transients caused by other activities on the UCLA campus, from the beginning we expected to use a motor generator on the power line. The Engineering Department offered us a spare alternator. For some reason that we never quite understood (the
428 HARR'l D. HUSKEY

Figure 1

alternator was alleged to have the wrong number of poles) the output of the

power supplies when using the alternator had more noise then when we con- nected directly to the power line. Ultimately, we replaced it with a larger

regulated system with much more satisfactory results.

We had expected to cool the computer with the Los Angeles air. However, the germanium diodes turned out to be much more temperature

sensitive then we had expected, so we added a cooling unit to the air system.

By this time the total computer was very well integrated with the building.

This only became a problem later when it was decided to move the computer

to the UCLA Engineering Building.

10. Applications

By mid-1953, the SWAC was producing useful results 70C> of the time

that it was turned on. and was doing over 53 hours of useful computing per

week. This was before the installation of the magnetic drum.

The SWAC was used in a research computing environment, and therefore the problems run on it tended to be quite large. St^lution times as high as 453 hours were reported during its early operation.
SWAC 429

Figure 2

The INA was an exciting place as many scientists of international reputation spent a few days, weeks, months, and occasionally a year or more

there. It was a crossroads for numerical analysts and early computer scientists. Some of the early problems included the search for Mersenne primes,

the Fourier synthesis of x-ray diffraction patterns of crystals, the solution of

systems of linear equations, and problems in differential equations. In addition to problems originating from the INA staff, the computer was also used

to do problems for other government agencies.

One of the exciting problems in pure mathematics on which the SWAC

worked was the study of Mersenne numbers, that is, numbers of the form

2"-!, where p is a prime. These numbers, when prime, are related to the

"perfect numbers" of the Greeks, numbers that are the. sum of all their integral divisors excluding themselves. The list of values of p that yielded prime

numbers up to that time wasp = 2, 3, 5, 7, 13, 17, 19, 31, 61, 89, and 107.

Everyone was greatly excited when the SWAC added 521 to the list, and

there was a real celebration when 607 was added about an hour later. In all,

by June 1953, five values had been added to the list with the use of the

SWAC as a result of systematic testing of all primes up to 2297.

Applied problems, such as the study of large-scale circulation patterns in

the earth's atmosphere, were also run on the SWAC. In this problem 750,000
430 HARRY D HUSKEY

Figure 3

pieces of data were processed to yield a similar number of answers. SWAC

spent 325 hours on this problem.

When the NBS ceased to support the IN A in 1954. the SWAC was transferred to the University of California at Los .Angeles and moved to the Engineering Building. There it continued in useful operation until it was retired in

December 1967. at the age of 17. Parts of the SWAC are on exhibit in the

Museum of Science and Industry in Los Angeles.

.ACKNOWLEDGMENT

The author vv ishes lo express his appreciation to Velma R. Huskey for assisting in the preparation of ihis paper.

REFERENCES

1. .1. H. Cuniss. Progress Report on the National Applied Mathematics Laboratories of the Nalional Bureau of Standards, p. 73 (April L 19.^3).

2. H n. HuNke\ . Semiautomatic instruction on the ZEPHYR. Frm . S\ inf' l.in uf-Scale
SWAC 431

Diffiial Calf. Math.. 2nd. pp. 83-90, Harvard Univ. Press, Cambridge, Massachusetts,

1951.

3. H. D. Huskey, Characteristics of the Institute for Numerical Analysis computer. Math.

Tables Other Aids Compiit. 4, No. 30. pp. 103-108 (1950).

4. H. D. Huskey, R. Thorensen, B. F. Ambrosio, and E. G. Yowell, The SWAC-design fea- tures and operating experience, Proc. IRE 41, No. 10, pp. 1294-1299 (1953).

DIVISION OF NATURAL SCIENCES

UNIVERSITY OF CALIFORNIA

SANTA CRUZ. CALIFORNIA

APPENDIX D

SYMPOSIA AND TABLES

In this appendix we give the title pages and tables of contents of the volumes of the NBS

Applied Mathematics Series which were written as a result of the activities at NBS-INA. A number

of these are proceedings of various symposia held at INA, thereby giving a description of the

program of these symposia. Others are outgrowths of these symposia. In addition, three of them

describe important tables which were compiled at INA.

D-1
I

I

i:

; I

Ij

I

I

jj
PROCEEDINGS OF

SYMPOSIA IN APPLIED MATHEMATICS

VOLUME VI

NUMERICAL ANALYSIS

McGRAW-HILL BOOK COMPANY, INC.

KEW YORK TOEONTO LONDON

1956

FOR THE AMERICAN MATHEMATICAL SOCIETY

80 WATERMAN STREET, PROVIDENCE, RHODE ISLAND
PROCEEDINGS OF THE

SIXTH SYMPOSIUM IN APPLIED MATHEMATICS

OF THE AMERICAN MATHEMATICAL SOCIETY

Held at the Santa Monica City College

August 26-28, 1953

COBPONSOBED BT

THE ATIONAL BUREAU OF STANDAUDS

John H. Curtiss

EDITOB

EDITOEIAL COMMITTEE

R. V. Churchill

Eric ReLssner

A. H. Taub

Copyright ® 1956 by the McGraw-Hill Book Company, Inc. Printed in the

United States of America. All rights reserved. This book, or parts thereof, may not be reproduced in any form without permission of the publishers.

Library of Congnu Catalog Card Number 50-1183
CONTENTS

Editob'b Pbxtace V

Computational Proble.mw in the Theory of Dynamic Programming 1 Bt KICEABD Bri.T.MAN

Some Methods for Bolutiona of Boundary-value Problems of Linear Partial Differential

Equations 11 Bt Stetan Be&ouan

Computational Aspects of Certain Combinatorial Problems 31

Bt R. H. Bbucx

Numerical Results on the Shock Configuration in Mach Reflection 45

Bt D. R. Cltttterham and A. H. Taub

Some Applications of Gradient Methods 59

Bt Joszph W. Fibchbace

Some Qualitative Comments on Stability Considerations in Partial Difference Equations 73

Bt S. p. Frankzl

Approximations in Numerical Analysis—A Report on a Study 77

Bt Cecil Hastings, Jb., Jeanne Hatwabd, and James P. Wong, Je.

The Conjugate-gradient Method for Solving Linear Systems 83

Bt Magnus R. Hestenes

Number Theory on the SWAC 103

Bt £u3£a Lebuze

The Assignment Problem 109

Bt T. S. MoTzaN

The Method of Steepest Descent 127

Bt p. C. Rosenbloou

Function Spaces and Approximation 177

By Arthub Sard

Some Computational Problems in Algebraic Number Theory 187

Bt Ouja Taubskt

Machine Attacks on Problems Whose Variables Are Permutations 195

Bt C. B. Tompkins

Best-approximation Polynomials of Given Degree 213

Bt J. L. Walsh

iii
iv CONTENTS

Recent Results in Numeric&l Methods of ConfomuJ M&pping 219

Bt S. E. Wxbschaweki

On the Asymptotic Transformation of Certain Distributions into the Normal Distribution 251

Bt Wouqano Wasow

Error Bounds for Eigenvalues of Symmetric Integral Equations 261

Bt Helmut Wielandt

On the Solution of Linear Systems by Iteration 283

Bt David Youno

Index 299
UNITED STATES DEPARTMENT OF COMMERCE • Charies Sawyer, Secretary

NATIONAL BUREAU OF STANDARDS • B. U. C—J—. Director

Table of Arctangents

of Rational Numbers

John Todd

National Bureau of Standards

Applied Mathematics Series* 11

iMued March 30, 1951

(Reprinted January 4, 19C5)

UNITED STATES GOVERNMENT PRINTING OFFICE, WASHINGTON i 1951

FOR SALE BY THE SUPERINTENDENT OF DOCUMENTS, VS. GOVERNMENT PRINTING OFFICE

WASHINGTON, D.C, SCMOl - PRICE 70 CENTS (PAPES COVER)
Contents

Parr Introduction _ v

1. Origin and use of the tables v

2. Description of the tables _ _ vi

3. Theoretical basis for the computational process vi

4. Computational process viii

5. Preparation of tables _ _ ^ ix

6. Printing _ _ x

7. Checking after typing x

8. References x

Table 1. Arctan —» Arccot — > and reductions of Arctan — 0 for <m<n^100 1

Table 2. Reductions of Arctan n, for reducible n^2089 93

NOTATION

By Arctan x and Arccot z, where z is « finite real number, we invariably understand

the principal values of the many-valued quantities arctan z and arccot z, i. e., the

values which satisfy— }t < Arctan z < i», — 1» < Arccot z < }«.

In the tables, for integral values of z, we write

(z)<= Arctan z.

Ill
UNITED STATES DEPARTMENT OF GOMMERCE • Charies Sawyer, Secretary

NATIONAL BUREAU OF STANDARDS • E. D. Cemitm, Dirmetm

Monte Carlo Method

Proceedings of a symposium held June 29, 30, and

July 1, 1949, in L>06 Angeles, California, under the

sponsorship of the RAND Corporation, and the

National Bureau of Standards, with the cooperation of the Oak Ridge National Laboratory

National Bureau of Standards

Applied Mathematics Series • 12

Iwued June U, 1951

UNITED STATES GOVERNMENT PRINTING OFFICE, WASHINGTON : 1951

FOR SALE BY THE SUPERINTENDENT OF DOCUMENTS, U. S. GOVERNMENT PRINTINC OFFICE, WASHLNGTON 25. D. C

PRICE SO CENTS
Contents

Page

Preface in

foreword - . v

1. Showers produced by low-enei^ electrons and photons _ 1

Robert R. Wilson

2. An alignment chart for Monte Carlo solution of the transport

.

problem 4

B. I. Spinrad, G. H. Goertzel, W. S. Sktder

3. Neutron age calciilations in water, graphite, and tissue 6

Alston S. Householder

4. Methods of probabilities in chains applied to particle transmission

through matter 9

Wentjell C. DeMarctjb, Lewis Nelson

5. Stochastic methods in statistical mechanics 12

Gilbert W. King

6. Report on a Monte Carlo calculation performed with the Eniac 19

Maria Mayer

7. Calculation of shielding properties of water for high energy neutrons _ 21

Preston C. Hammer

8. A Monte Carlo technique for estimating particle attenuation in

bulk matter 24

B. A. Shoor, Lewis Nelson, Wendell DeMarctjs, Robert L.

Echols

9. Estimation of particle transmission by random sampling 27

Herman Kahn, T. E. Harris

10. History of RAND's random digits—Summary 31

George W. Brown

11. The mid-square method of generating digits 33

Preston C. Hammer

12. Generation and testing of random digits at the National Bureau of

Standards, Los Angeles 34

George E. Forsythe

13. Various techniques iised in connection with random digits 36

John von Neumann

14. Round table discussion 39

H. H. Germond

VII
UNITED STATES DEPARTMENT OF COMMERCE • Charles Sawyer, Secretary

NATIONAL BUREAU OF STANDARDS • E. U. C—dop, Dtrmetar

Problems for the Numerical Analysis

of the Future

National Bureau of Standards

Applied Mathematics Series •IS

iMued June 29, 1951

UNITED STATES GOVERNMENT PRLNTING OFFICE, WASHINGTON t 1951

FOR SALE BY THE SUPERINTENDENT OF DOCUMENTS. U. S. GOVERNMENT PRINTING OFFICE. WASHINGTON 25. D. C

PRICE 20 CENTS
Contents

Foreword--

1. Some unsolved problems in numerical analysis

Douglas R. Hartree

2. Numerical calculations in nonlinear mechimics

S. Lefschetz

3. Wave propagation in hydrodynamics and electrodynamics—.

Bernard Friedman

4. Linear programing

George B. Dantzig
UNITED STATES DEPARTMENT OF COMMERCE • Charies Sawyer, Seeretaty

NATIONAL BUREAU OF STANDARDS • A. V. AMia, Dirwctor

Construction and Applications

of Conformal Maps

Proceedings of a Symposium

Held on June 22-25, 1949. at the

Institute for Numerical Analysis

of the

National Bureau of Standards

at the

University of California, Los Angeles

Edited by E. F. Beckenbach

Natiojnal Bureau of Standarids

Applied Mathematics Series • 1 8

Issued December 26, 1952

FOB SALE BY THE SUPEHINTENDENT OF DOCUMENTS. U. S. GOVERNMENT PMNTmC OFFICE. VASHINGTON tS.D.C. PRICE 12^5 (Bnckram)
Contents

Page

Preface m

1. On network methods in conformal mapping and in related problems 1 R. V. MisES

2. Flow patterns and conformal mapping of domains of higher topological

structure 1 7

R. COUBANT

3. Some industrial applications of conformal mapping 15

H. PORITSKT

4. Conformal maps of electric and magnetic fields in transformers and similar

apparatus 31

G. M. Stein

5. Conformal mapping applied to electromagnetic field problems 59

Ernst "Weber

6. On the use of conformal mapping in two-dimensional problems of the theory

of elasticity 71

I. S. SOKOLNIKOFF

7. Conformal mapping related to torsional rigidity, principal frequency, and

electrostatic capacity 79

G. SZEGO

8. Fluid dynamics, conformal mapping, and numerical methods 85

Andrew Vazsonti

9. The use of conformal mapping in the study of flow phenomena at the free

surface of an infinite sea 87

Eugene P. Cooper

10. An application of conformal mapping to problems in conical supersonic flows. 91

R. C. F. BARTELS and O. Laporte

11. On the Helmholtz problem of conformal representation 105

Alexander Weinstein

12. Some generalizations of conformal mapping suggested by gas dynamics 117

LiPM.\N Bers

13. The use of conformal mapping to determine flows with free streamlines 125

David M. Young

14. Conformal mapping in aerodynamics, with emphasis on the method of successive conjugates . 137

I. E. Garrick

15. On the convergence of Theodorsen's and Garrick's method of conformal

mapping 149

A. M. OSTROWSKI

16. On a discontinuous analogue of Theodorsen's and Garrick's method 165

A. M. OsTROWSKI

17. On conformal mapping of variable regions 175

S. E. Warschawski

18. A variational method for simply connected domains 189

A. C. ScHAEFFER and D. C. Spencer
P*ge

19. Some remarks on variational methods applicable to multiply comiected

domains - _ 193

M. ScHiFrER and D. C. Spencer

20. The theory of Kernel functions in conformal mapping _ 199

S. Bergman and M. Schiffer

21. A new proof of the Riemann mapping theorem 207

P. R. Garabedian

22. The Kernel function and the construction of conformal maps... 215

Zeev Nehaei

23. An approximate method for conformal mapping _ 225

Lee H. Swinford

24. On the effective determination of conformal mapping.. _' 227

Max Shiffman

25. The difference equation method for solving the Dirichlet problem 231

P. C. ROSENBLOOM

26. Relaxation methods as ancillary techniques 239

Richard Southwell

27. Conformal invariants. _ 243

Lars V. Ahlfors and A. Beurling

28. On subordination in complex variable theory 247

E. F. Beckenbach and E. W. Graham

29. Asymptotic developments at the confluence of boundary conditions 255

Hans Lewt

30. Monodiffric functions 257

RuFus Isaacs

31. Recent contributions of the Hungarian school to conformal mapping. 267

G. SZEGO

32. Bibliography of numerical methods in conformal mapping... 269

"W. Seidel

vi
'I

I

1
UNITED STATES DEPARTMENT OF COMMERCE • Sinclair Weeks, Secretary

NATIONAL BUREAU OF STANDARDS • A. V. Aatia, Dir^tor

Tables for Rocket and Comet Orbits

Samuel Herrick

National Bureau of Standards

Applied Mathematics Series • 20

iMued March 9, 19S3

UNITED STATES GOVERNMENT PRINTING OFFICE, WASHINGTON « 1953

FOR SALE BY THE SUPERINTENDENT OF DOCUMENTS. U. S. GOVERNMENT" PRINTING OFFICE, WASHINGTON IS, D. C. PRICE 91.75
Contents

Pace

Introduction _ v

1. Rectilinear motion ; origin of the functions v

2. Direct interpolation (examples, rectilinear naotion) vii

3. Inverse interpolation (examples, rectilinear motion) ix

4. Nearly rectilinear or "nearly parabolic" motion xi

5. Position and velocity from time (examples, parabolic and nearly rectilinear motion) - _ - xiv

6. Time from position and velocity (examples, nearly rectilinear motion) xviii

7. Method of computation of the tables xx

8. Acknowledgments xxii

9. References- xxiii

Schedule A. Ranges of values for various n's xxiv

Tables _ 1 Parabola: }iD' and D with argument %IP=10-**u,

all values of n 3

Ellipse: 1—cos E, E with argument sin E=\0~**u, and

Hyperbola: cosh F—\, sinh F, Fvath argument sinh F—F=\0~**u

n>5 - 3

n=4 12

n=3 --- 30

n=2 48

71=1 - .-- 66

n=0 84

In all tables, except where n=0 for ellipse aud hyperbola, u=0.15(.005)0.34(.01)

0.74,8D; t/ = 0.74f.02)1.5(.05)3.4(.l)7.4(.2)15(.5)34(l)74(2)150,7D.

Where n=0, for ellipse and hyperbola, u=0.l5(.005)0.34(.01)0.74,8D; for ellipse, u = 0.74C.01)3.15,7D; for hyperbola, u=0.74(.02)1.5(.05)3.4,7D; w = 3.4(.l)4

(.2;8(.5) 12(1)20(2)80(5) 170(10)300,fiD.

Everett interpolation coefficients 99
UNITED STATES DEPARTMENT OF COMMERCE • Sinclair Weeks, Secretary

NATIONAL BUREAU OF STANDARDS • A. V. Aatin. Dirmctor

Simultaneous Linear Equations

and the

Determination of Eigenvalues

Edited by L. J. Paige and Olga Taussky

Proceedings of a sympoBium held August 23-23, 1951,

in Los Angeles, California, under the spoasorshlp

of the National Bureau of Standards, in cooperation

• with the Office of Naval Research

National Bureau of Standards

Applied Mathematics Series • 29

Issued August 31, 1953

For ulr by tlir Superintendent of Document*, U. S. Government Printing Office, Vaahinrton 23. D. C. Price tl.50
Contents

Page

Foreword ni

1. Tentative classification of methods and bibliography on solving systems of linear equations, by George E. Forstthe 1 2. Simultaneous systems of equations, by A. M. Ostrowski 29

3. The geometry- of some iterative methods of solving linear systems,

by Alston S. Householder 35

4. Solutions of linear systems of equations on a relay machine, by

Carl-Erik Froberg 39

5. Some special methods of relaxation technique, by Eduard Stiefel. . 43

6. Errors of matrix computations, by Paul S. Dtvyer 49

7. Rapidly converging iterative methods for solving linear equations,

by J. Barkley Rosser 59

8. Some problems in aerodjTiamics and structural engineering related

to eigenvalues, by R. A. Frazer 65

9. Inclusion theorems for eigenvalues, by H. Wielakdt 75

10. On general computation methods'for eigenvalues and eigenfunctions,

by Gaetano Fichera 79

11. Variational methods for the approximation and exact computation

of eigenvalues, by Alexander TVeinstein _ 83

12. Determination of eigenvalues and eigenvectors of matrices, by

Magnus R. Hestenes 89

13. New results in the pertairbation theorj- of eigenvalue problems, by

F. Rellich _ _ _ 95

14. Bounds for characteristic roots of matrices, by Alfred Brauer 101

15. Matrix inversion and solution of simultaneous linear algebraic equations with the IBM 604 electronic calculating punch, by George

W. Petrie, III -_ 1. _- 107

16. Experiments on the inversion of a 16 X 16 matrix, by John Todd„ 113

17. A method of computing eigenvalues and eigenvectors suggested by

classical results on s^Tnmetric matrices, by Wallace Givens 117

18. Computations relating to inverse matrices, by Jack Sherman 123

19. Resvilts of recent experiments in the analysis of periods carried out

in the Istituto Kazionale per le Applicazioni del Calcolo, by

Gaetano Fichera 125

IV
UNITED STATES DEPARTMENT OF COMMERCE • Sinclair Weeks, Secretary

NATIONAL BUREAU OF STANDARDS • A. V. AMia, DInelor

Contributions to the Solution of

Systems of Linear Equations and the

Determination of Eigenvalues

Edited by Olga Taussky

National Bureau of Standards

Applied Mathematics Series • 39

Is«u«d September 30, 1934

^or ulc by the Superintendent of Doeumcota, U. S. Govenunent Printing OIBee, WubiBfton 25, D. C. ........ Priee t2.M .
Contents

Preface II

1. Practical solution of linear equations and inversion of

matrices, by L. FoX 1

2. Punched-card experiments with accelerated gradient methods for

linear equations, by A. I. FORSYTHE and G. E. FORSYTHE 55

3. Iterative methods of solving linear problems on Hilbert space,

by R. M. Hayes 71

4. Tables of inverses of finite segments of the Hilbert matrix,

by I. R. Savage and E. LuKACS 105

5. Ihe condition of the finite segments of the Hilbert matrix,

by John Todd 109

6. Lower bounds for the rank and location of the eigenvalues of

a matrix, by K. Fan and A. J. HOFFMAN 117

7. Inequalities for eigenvalues of Hermitian matrices, by Ky Fan.. 131

(lll>
UNITED STATES DEPARTMENT OF €X>MMERCE • Sinclair Week*, Secretary

NATIONAL BUREAU OF STANDARDS • A. V. Aatia, Dirmetmr

Experiments in the Computation

of Conformal Maps

Edited by John Todd

National Bureau of Standards

Applied 3Iatliematics Series • 42

Issued Januar>- 20, 19S5

For sale by the Superintendent of Document*, U. S. Goiemment Printing Office, Waabington £S, D. C rrioc 40 eenta
Contents

P«ge

Preface _ _ ii

1. Conformal mapping of a special ellipse, on the unit circle by A. M.

OSTROWSKl _ 1

2. Theodorsen's and Garrick's method for conformal mapping of the \mit

circle into an ellipse, by A. M. Ostrcwski _ 3

3. On the solution ol the Lichtenstein-Gershgorin integral equation in con- formal mapping: I. Theorj^ by S. E. Warschawski 7

4. On the solution of the Lichtenstein-Gershgorin integral equation in con- formal mapping: II. Computational experiments, by John Todd

and S. E. Warschawski 31

5. Two numerical methods in conformal mapping, by L. Ahlfors 45

6. Computation of harmonic measure by L. Ahlfors' method, by G. Blanch

and L. K. Jackson _ 53

m
UNITED STATES DEPARTMENT OF COMMERCE • Sinclair ^Teclu, Secretary

NATIONAL BUREAU OF STANDARDS • A. V. Aatia, Dirmetor

Further Contributions to the Solution of

Simultaneous Linear Equations and the

Determination of Eigenvalues

National Bureau of Standards

Applied Mathematics Series • 49

IsHued January 15, 1958

ale bv the Siiprrinlrndenl of Document*, l'. S. Co%ernmpnt Printing Office, Vuhinirlon 25, D. C. • • - - - - Price SO cent*
Contents

Page

Preface _ _ n

1 . Kernel pohnomials in linear algebra and their numerical applications,

by E. L. Stiefel- _ 1

2. The quotient-difference algorithm, by P. Hexrici 23

3. Solution of eigenvalue problems with the LR-irausformation, hy H.

RUTISHAUSER 47

lU
UNITED STATES DEPARTMENT OF COMMERCE • Lewia L. StraoM, Secretary

NATIONAL BUREAU OF STANDARDS • A. V. Aatln, DUwetm^

Tables of the

Bivariate Normal Distribution Function

and Related Functions

Compiled and Edited

by the

National Bureau of Standards

National Bureau of Standards

Applied Mathematics Series • 50

iMued June 15, 1959

ulc by the SupviotoMleat of DoeuBMaU, U. S. Gorcramat Printlac OA«a, Taahlnctoo SS, D. C.
Contents

Preface ii

I. Introduction, by Gertrude Blanch v

1.1. Source and scope of the tables v

1.2. Mathematical properties of the functions vi

1.3. Interpolation _ vni

1.4. Method of computation and preparation of manuscript xv

II. Applications of the tables, by D. B. Owen xvii

2.1. Introduction xvn

2.2. Applications to measurement errors, a calibration system, double

sampling, percentage changes in a sample mean xix

2.3. Applications to polygons xxviii

2.4. Applications to the range and noncentral ^-distribution xxxiu

2.5. Applications to a heat flow problem _ _ xxxvi

2.6. Applications to problems in selection and to estimation of correlation _ XXXVII

III. Bibliography . XLm

IV. List of errors in Pearson's tables of L{h,k,r) and L(h,k,—r) xlv

Tables

I. L(hM 1 [;i,Ar=0(.l)4; r=0(.05)0.95(.01)l ; 6D]

II. L{h,k,-r) 127

{A,it=0(.l)^„it:.; r=0(.05)0.95(.01)l; 7D,

where Z(A„A:,—r)<i-10~' if A, and it, are both less than 4].

III. V(hM) 215

[X=0.1(.l)l; A=0(.01)4(.02)4.6(.l)5.6, 7D]

IV. VOJi,h) - 235

IX=0.1(.1)1; A=0(.01)4(.02)5.6, 7D1

V. y=(arcsinr)/(2T) 257

[r=0(.01)l; 8D; second central differences]

m
APPENDIX E

EDUCATIONAL ACTIVITIES

From the beginning there was a broad educational program at INA. We have seen already, in

Appendix D and in the text, that a series of symposia was sponsored by INA jointly with UCLA

and/or other institutions. Members of INA were encouraged to participate in seminars and colloquia sponsored by UCLA and neighboring institutions. A large nimiber of informal lectures and

seminars were conducted by visitors of INA as well as by regular members of INA. These lectures

and seminars normally dealt with topics related in a broad sense to numerical analysis and machine

computations. Students and faculty of UCLA and members of other institutions were welcome to

attend these meetings. As stated in the text, INA also sponsored a series of colloquia on a broad set of topics of interest to the group. The speakers could choose their own topics. Most of the speakers

were visitors to INA. We have compiled the following chronological list of visiting colloquium

speakers:

1948-49

A. S. Besicovitch

C. F. Davis

J. J. Gilvary

Casimir Kuratowski

C. Brown

H. H. Germond

C. Hastings

Sir Harold Spencer Jones

1949-50

E. T. Benedict

R. P. Feynman

O. Helmer

R. Isaacs

S. Lefschetz

W. Prager

A. W. Tucker

M. Fekete

R. A. Fisher

J. O. Hirschfelder

I. Kaplansky

E. Penney

G. SzegO

1950-51

N. Aronszajn

E. Gerjuoy

G. P61ya

J. J. Stoker

I. J. Schoenberg

L. Bers

J. B. Rosser

E. Stiefel

J. G. van der Corput

Y. L. Luke

I. J. Schoenberg

R. Isaacs

A. Erd61yi

K. Knopp

S. Sherman

D. van Dantzig

J. H. Curtiss

J. L. Walsh

G. Fichera

F. J. Murray

L. M. Blimienthal

S. P. Frankel

W. T. Reid

A. J. Hoffman

1952-53

A. M. Ostrowski

G. A. Hunt

D. Ray

J. G. van der Corput

E. W. Barankin

A. Dvoretsky

A. T. Lonseth

M. Kac

H. Levine

G. P61ya

P. Erdds

J. Wolfowitz

N. Minorsky

S. Bochner

M. M. Shiffer

R. de Vogelaere

E-1
1953-54

G. PaU

J. Marschak

A. Bafios, Jr.

S. Lefschetz

R. Radner

F. A, Foster

INA also sponsored regular courses and seminars for its members. They were conducted by

INA staff members for no additional compensation. They fall into two classes:

1) Research activities and within-training courses for the INA Staff to which UCLA staff

members and other interested persons were usually invited. These were part of the regular workday of the instructors.

2) Formal UCLA courses in numerical analysis and closely related subjects, attended by INA

staff members and UCLA students. The work days of the instructors of these courses were adjusted so that teaching them was in addition to the regularly assigned duties.

The following INA staff members and visitors acted as instructors of these courses at various

times.

F. S. Acton

S. S. Cairns

P. Erdos

R. P. Feymnan

D. R. Hartree

F. John

R. S. Lipids

W. E. Milne

A. Ostrowski

J. B. Rosser

I. J. Schoenberg

R. Thorensen

J. G. van der Corput

E. C. Yowell

G. Blanch

J. H. Curtiss

W. Feller

G. E. Forsythe

H. D. Huskey

C. Lanczos

H. Luxenberg

T. S. Motzkin

H. A. Rademacher

D. S. Saxon

T. Southard

C. B. Tompkins

W. R. Wasow

In addition to these courses, there were a second set of coiirses taught by INA staff members

on their own time, not as a function of the INA, and ordinarily for additional compensation. These

courses were sponsored by UCLA, UCLA Extension Division, and the University of Southern

California. Some of the INA participants were:

B. F. Ambrosio

F. R Hollander

H. D. Huskey

E. E. Osborne

J. Schwinger

R. Thorensen

J. L. Walsh

M. Weber

A. D. Hestenes

M. Howard

C. Lanczos

D. S. Saxon

T. H. Southard

C. B. Tompkins

W. R. Wasow

E. C. Yowell

Some members of INA held joint appointments at INA and UCLA. They usually gave a

graduate course at UCLA in their field of specialization. In addition members of INA, such as, Forsythe, Motzkin, Tompkins, and Wasow participated in the Ph. D. theses programs at UCLA and

use.

The Educational Program at INA led to the writing of lecture notes in nxmierical analysis and

related topics. These lecture notes were issued from time to time as working papers or as NBS

Reports. Among these are:

G. E. Forsythe: Nimierical methods for elliptic partial differential equations

Theory of selected methods of finite matrix inversion and decomposition.

J. G. van der Corput: Asymptotic expansions

E-2
C. Lanczos: Approximation by orthogonal polynomials

H. Rademacher: Elliptic and modular functions

T. S. Motzkin: Chebyshev polynomials and extremimi problems

1. J. Schosnberg: Partial differential and difference equations

J. Schwinger: Quantum Dynamics

W. Wasow: Introduction to the asymptotic theory of ordinary linear differential

equations

Many of these were later developed into books. The preparation of these notes was often part

of the duties of the graduate fellows.

Members of INA and NAR also participated in writing articles forming the "The Tree of

Mathematics" edited by Glenn James of UCLA and published in 1957 in Mathematics Magazine.

Their contributions were

E. F. Beckenbach: Complex Variable Theory; Metric Differential Geometry

J. H. Curtiss: Elements of a Theory of Probability

J. W. Green: Theory of Functions of a Real Variable

M. R. Hestenes: An Elementary Introduction to the Calculus of Variations

Olga and John Todd: Applications of Systems of Equations, Matrices, and Determinants

Members of INA and NAR also participated in a series of lectures sponsored by the UCLA

Extension Division. Their lectures were published in a book entitled "Modem Mathematics for the

Engineer" edited by E. F. Beckenbach and published in 1956 by McGraw-Hill. The INA-NAR

participants were J. W. Green, M. R. Hestenes, G. E. Forsythe, D. H. Lehmer, and C. B. Tompkins.

E-3
iif

liIIi jiiIiI
APPENDIX F

PERSONNEL

A large number of persons were officially connected with INA. We have compiled a list of

these members in this appendix. We make no attempt to list the many visitors who came to INA as

unofficial members. Among these were distinguished scientists, such as J. von Neimiann, R.

Courant, E. Teller, S. Lefschetz, Norbert Wiener, and many others. We had frequent visitors from

abroad, particularly from England. Then, too, there were unofficial members of INA comprised of

faculty and graduate students from UCLA and neighboring imiversities.

We now present the official list of personnel of INA.

Chief, AMD

(Located at NBS, Washington, DC)

John Hamilton Curtiss 1947-53

Franz L. Alt 1953-54

Assistant Chief, AMD

(Located at NBS, Washington, DC)

Edward W. Cannon 1947-54

Consultant to the Divisim

(Located at NBS, Washington, DC)

Olga Taussky-Todd 1949-54

She was an initial member of INA 1947-48.

Chief of the Conqiutation Laboratory at NBS-Wadiington

John Todd 1949-54

Todd and his group collaborated with the staff at INA. Todd was an initial member of INA

1947-^.

Directors of INA

Douglas R. Hartree Spring and Summer 1948

John H. Curtiss 1948-49

J. Barkley Rosser 1949-50

Fritz John 1950-51

Derrick H. Lehmer 1951-53

Charles B. Tompkins 1953-54

Assistant Director and UCLA Liaison Officer

Magnus R. Hestenes 1950-54

In this capacity he was ably assisted by William T. Puckett of UCLA. During this period,

Hestenes was also the Chairman of the Department of Mathematics at UCLA.

Chief of Machine Develoiment and Mathematical Services

Harry D. Huskey 1948-54

F-l
Assistant to the Director

Gertrude Blanch

Albert S. Cahn, Jr. Wilbur W. Bolton, Jr.

1948-54 (Numerical Analysis)

1948-53 (Administration)

1948-49 (Administration)

Senior Researchers, Regular Members

The dates given are the initial dates of employment.

Forman S. Acton 1949

Gertrude Blanch 1948

George E. Forsythe 1948

Magnus R. Hestenes 1949

Harry D. Huskey 1948

Qsmelius Lanczos 1949

Theodore S. Motzkin 1950

David S. Saxon 1950

Dan Teichroew 1952

Charles B. Tompkins 1952

Wolfgang R. Wasow 1949

(Part time)

(UCLA-Physicist)

(Members of the Mathematical Services Unit also carried out research.)

Senior Researchers, Temporary Members

The dates given are the initial dates of employment. Some members served during several

periods. Their permanent affiliations are enclosed in parentheses. Except where noted, they were

Mathematicians.

Lars V. Ahlfors 1949 (Harvard)

Adrian A. Albert 1952 (Chicago)

Alfredo Bafios, Jr. 1952 (UCLA-Physicist)

Edward W. Barankin 1952 (Berkeley)

Edwin F. Beckenbach 1948 (UCLA)

Leonard M. Blumenthal 1951 (Missouri)

Truman A. Botts 1952 (Virginia)

B. Vivian Bowden 1951 (England)

Alfred Brauer 1951 (North Carolina)

Richard H. Bruck 1953 (Wisconsin)

William E. Bull 1951 (UCLA-Linguist)

Kenneth A. Bush 1953 (Illinois)

Robert H. Cameron 1948 (Minnesota)

Randolph Church 1953 (Monterey)

Milton Dandrell 1950 (UCLA)

Robert P. Dilworth 1953 (Caltech)

Monroe D. Donsker 1949 (Cornell)

Aryeh Dvoretsky 1950 (Israel)

Paul S. Dwyer 1951 (Michigan)

Paul Erdds 1950 (Hungary)

Richard P. Feynman 1950 (Cornell)

Gaetano Fichera 1951 (Italy)

Donald E. Fogelquist 1951 (Sweden)

Robert Fortet 1950 (France)

William FeUer 1949 (Cornell)

Jerry W. Gaddum 1951 (Missouri)

Herman H. Goldstine 1951 aAS)

Richard A. Good 1953 (Maryland)

F.2
jouii w. vjrccn lyju

XxUUCri C VJICCIIWLIIJU 1Q4.R ^ X CAUSJ TiirV Wirlr Hall 1 ividryidiiu 1 Mar<thall Hall Jr

JCi. n. ndllDOIl ^ 1 CAoS dLaic)

fr A Hedlund 1952 I X dlW 1 1948 (^X-.'CUliL'l f^JimhriHDA ILl^Wj Fnolanrl^ X^llglallU.1

Ooiiiuci XTcrriC'&. ft Tff^T A • A Ctf*rtT^<^**^ A^ 1 ^ KJ v-'Ij/^'/vsLronoiiicr i rrilhprt A TTnnt 1951 ^rr»rn<»11\

nilZ, JUIiU

IVlCUJIL XV, 1949 1 V-AJi llWll 1

II* xvdiJicLLioii.

y

1952 1 V^lllWagU 1

Wllilalil ^aiUdil 1949 f f n 1 o ftrt ^V^lllL'agU 1 T'nm irilHiim 1951 ^x^U^laliU 1 T^rwin dwill TTlpifif^lfl X\_LWilli 1953 miipaon ^x^uica^ij} OV»ir\ ^tat#»^ WllliJ OLdLCI

irioi uiu w • xwiiiiii 1953 yX>ljrii ividwr I J-/dllwfL XX* J.jwlllllwl 1951 iRf^rlrf^l lX->wl A.dwyv^1 TJipKarH XVIWIKU u A TXwwlUlwl p'lVjlpr 1952 t T T ni>l 1 II 1^ IVJll) xy 1 1952 ^TTarvafH-Pfi IX XCU V CU U*X Uvcif*ict^ Y dlC-l3L

xxdiia x^ww y 1950 ixjci jwcicy 1 ./^VlU 1. J_/UlloCLll 1952 IvyiCgUU OLaLCi

JaCUU iVicLrCilclK. # 1 /T/^_ W/>/%n/%tm/>c I ^V^IUCagO'xlwOUUIZllCSy

x^uwaiu ja iviChjiioiic 1950 I V 11 gliUal

William TT it I lOI 1 1 FX-fa Milne IVXUUC 1948

x^icuiik. ja ivxuiiay 1951 ^^^r\liiTTiViia\ ^^.^iUXllUXaf l^WCil J. X olgC Vyjx)

frnrHnn VJCIl UVJIL XPall all 195"^ mi T#»rh ^

xvajrixiuuu x* x wtdstjii) Ji* 1950 niPT A"^ W^illiam TT Itl Kllll A Pi^rp^ X IWl L/C 195^^

^(3J

1 awlidC 1 ^VlwAdUUwl IVXa X^aLl^JWo&J. 1949 Lyty IDddCli O Wll^CllcUlUI

vicior >\a wswaiQ, jTa 10^1

TTanc XXOild A DaH^mapli^r XvaUwiUawUwl 194X ^P^nnQvlvaTiia^

J a XJCUJ^Wy XXUddwX 1949 X^*t7 I \-^l llwll 1

XdClA^ *l a kJdl\JwllL'&l ^

1951 ±yjx ^P**titiQvl IX cuiid Y 1vatiia^ V 0.1110

Till lOT^ Coll 1X7 in nt^r JUIlali owiiwiugcr 1952 ^xxol Vdl U*x liy Mwl3LJ YVlaUlIllir OCIUCI 194R ^ yy ayiicf

FHwarH T ^tipfpl 1951 1 vJ witT^rlatiH^ WlLZiWllallU 1

xiiiiip Philtn OICIU *\tc\n 1951

JOIlalllall U, oWlIX Lyjj nici A^

IQdR LyrO IV >I11L1I1I1^U

J Willi X vJiiu 1947 1 X^vXlUwllf X^ll^lOilU

Olaa TaiiQQlrv-TnHrl 1947 n1 X^VIllLlwlly ^TiHr\n FtnylanH^ * .IIV i<^IIH 1 T n i/an Hat ^^/mtiii^ J a vja YcUl ucr v.AJipui 1951 1 XXUliCtllU 1 JOScpIl J-(a Waisn IQ^I o^^f or/1

^nalVaru^

Tam^c A AX/ai*H 1952 1 Loll 1 dLCpnCIl da WdTSCliaWSlkl lyty ^iVUIlIiCoU ulf /\icAaiiucr yy cixiaiciii 1951 1 lYXai y loULi 1

V-'ilariCS WcXlCi lyj^ I A Ti7/^no CtotA 1 ^/MiZUIia dutlC^

iTXCim M^aiiricft Aww VT a 'Willces TT AXXkWo 1951

Jacob Wolfowitz 1952 (CorneU)

Junior Researchers and Graduate Fellows

The dates listed are the initial dates of employment.

John W. Addison, Jr. 1952

Charles E. Africa, Jr. 1951

Donald G. Aronson 1951

F.3
Fred Baskin 1952

Richard G. Cornell 1951

Richard E. Cutkosky 1950

Robert J. Diamond 1949

Robert C. Douhitt 1949

Harold P. Edmundson 1950

Ernest E. Elyash 1949

Stuart L. Fletcher 1950

Walter I. Futtennan 1952

Stephen G. Gasiorowicz 1951

John H. Gay 1951

Robert K. Golden 1950

George E. Gourrich 1949

Harold Gruen 1949

Robert M. Hayes 1950

Urs W. Hochstrasser 1951

William C. Hoffman 1950

Kenneth E. Iverson 1951

Lloyd K. Jackson 1949

Thomas E. Kurtz 1951

Richard H. Lawson 1951

Eugene Levin 1953

Genevevo C. Lopez 1953

Howard W. Luchsinger 1948

Harold Luxenberg 1949

Stanley W. Mayer 1952

Edwin Mookini 1953

Michael J. Moracsik 1951

Mervin Muller 1953

Thomas Neill, Jr. 1951

Raymond P. Peterson, Jr. 1948

Lloyd Philipson 1953

Anthony Ralston 1951

Daniel B. Ray 1951

Theodore D. Schultz 1951

John Selfridge 1952

Robert H. Senhart 1948

Marvin L. Stein 1948

James G. C. Templeton 1949

Marion I. Walter 1952

William H. Warner 1951

Hans F. Weinberger 1949

James P. Wesley 1950

Mollie Z. Wirtschafer 1951

Roger D. Woods 1952

Coiiq>iitation and Mathematical Services Unit

The dates listed are the initial dates of employment.

Gertrude Blanch 1948

Louis L. Bailin 1948

Arnold D. Hestenes 1951

Frederick H. Hollander 1951

Marvin Howard 1949

Roselyn Siegel Lipids 1948

Harold Luxenberg 1950

Leslie H. Miller 1951

F^
>

William C. Randels 1950

Robert R. Reynolds 1950

Thomas H. Southard 1951

Everett C. Yowell 1948

Computing Staff

The dates listed are the initial dates of employment.

Helen Arens 1948

Warren A. Bailv 1952

<J won XA* x^ox&uu 1952

Leonus T Batiste 1951

Patricia Burton Bremer 1948

Patricia L Childress 1952

TientiKon A Curtis*; 1952

1950

Leola Cutler 1948

Richard L Dunn 1952

Alexandra Forsvthe 1950

Eileen M PevdeT 1948

PHwarH r> Fi«ht»r

Lillian Porthal 1949

Gladvs P. Franklin 1950

J TliiAj^ell Pranks 1948

1952

Fannie M Gordon 1948

1952

Beniamin P Handv Jr 1952

^ii^ai/cui fxcuuuiK

Hohert XXV/L/WX k MXTXa Haves XXOT WO 1950

rjiiiiid ivi.* xiwULiwi dwii 19^1 X7«^X

Donald ' ** ^ 1\ Henlev XXwXXXWT 1949

Delia Herhio 1948

Ruth XXULXL B.Xfa Horaan XXWX^CXU 1948

VVlUlalll JUdLlvC) Jl.

WilHam W 'M'**'" PX • ITpatino .^CaLIil^ 1Q57

rrprald vjcroiu Ww > xwjxuuic ICimhle

jyxaxy IVl* AaUdC

Piipene T^evin 1952

Barhara X^cU L/CU CX T.evine X^w T XUw 1952

X^UXxUa X>w^wl

Nancv X^CUlVJr XXwI.'L'XXXO BohHins M^ann XTXCUXXX 1949

^liirlAV T A^^fiflrc OJliilCjr -L^* iTicu&d 194R

X^ldliA. Xa 1V1CC&. 195'^ X7*/^

xxCiCU V • XV1CC&. Owen R. Mock 1951

Elmer E. Osborne 1951

William O. Paine, Jr. 1948

John A. Postley 1948

Mary M. Y. Quan 1952

Everett A. Rea 1948

Nan N. Reynolds 1951

La Verne E. Rickard 1951

Leon C. Robbins, Jr. 1948

Albert H. Rosenthal 1949

Philip H. Sayre 1952

F.5
William C. Schultz 1948

M. Winfred Smith 1949

Jerome A. Sneck 1951

Herbert Snow 1949

Seymour Soil 1951

Louise M. Straus 1951

Rose Tishman 1948

Mary Tudor 1949

Gladys P. Walker 1953

Maria Weber 1952

Marian E. Williams 1953

Lindley L. Wilson 1950

Boyd C. Zacharias 1948

Machine Develoinnent Unit

The dates listed are the initial dates of employment.

Harry D. Huskey 1948

Biago F. Ambrosio 1949

William Arsenault 1952

Edward I. Lacey 1949

Harry T. Larson 1949

Roselyn Siegel Lipids 1948

Norman L. Loretz 1950

David S. Rutland 1949

Robert R. Schmidt 1953

Ragnar Thorensen 1951

Electronic Laboratory Sta£F

The dates listed are the initial dates of employment.

Brent H. Alford 1949

Robert V. Bamett 1949

Charles M. Brauer 1949

Arnold Dolmatz 1949

Blanche C. Eidem 1949

John P. Francis 1951

Allen E. Garfein 1952

Sidney S. Green 1949

Milton B. Grier 1949

Gaylord W. Jones 1951

Louis E. Justice 1951

Charles H. Keller 1952

John D. Mach 1949

Michael J. Markakis 1949

Howard K. Marks 1951

Edward D. Martinolich 1950

Kenneth J. Millikin 1951

Charles A. MitcheU II 1951

Lyle E. Mitchell 1951

Wallace J. Moore 1951

John L. Newberger 1949

Geraldine Orr 1949

Eugene M. Rector 1951

Dean W. Slaughter 1951

Sol Scope 1952

James W. Walsh 1949

F-6
Administrative Staff and Secretaries

The dates listed are the initial dates of employment. Initial classifications are given within

parentheses except for secretaries. Some of the secretaries were given administrative posts later.

Many served only for a short period.

Plsie A Aho 1949

James T) Allen 1952 fQerk) 1 ^^A A A^ f Lfiuis H Adelizzi 1952 {Office Machine Ooerator^

Wilbur W. Bolton 1948 (Administrative Officer)

Jean S Booth 1950

Gloria E. Bosowski 1950

Eva Cassirer 1951 fTranslator'i T A A %4>AAAJA%A A V Harriet W. Cotton 1951

Edna S. Cruzan 1948

Mildred L. Dalton 1951

Betty E. Dean 1951

Selma S. Doumani 1950

Marv G. Eskridce 1952

Gloria E. Estes 1951

Irene M Everett 1949

Albert H. Feldman 1951 (Office Machine Operator)

Ellma Franz 1948 (Library Assistant)

Mildred S. Goldberg 1948

Sandra Goldberg 1952

Vendla T WJ 1% J 1 C* HJL A* Gordanier f 1 Mill IIV*! 1951 X^^ X Margaret E. Gould 1949

Eleanore J. Harris 1949

Mark H Hennes 1951 ^Office ^^achine Onerator^

Dorotliv Wihhard 1949

Miller A. Holt 1952 fStock Clerks

Velma Sr*l 1 1 iTl RAX> Huskev A AUOJVw T 1949 rPhv^ical Science Fditor^

Elsi ft L HusTnan 1948

Dora Kaolan A^«AL^AVAAA 1948

Earle A^UA A V W.TT Kimball AmA J *A fc.^l*AA 1950 fStock 1 k>X^ W^A Clerk'^ ^^A WA ^A f Svlvia C. Krasell 1948

Cecelia R. Leonard 1950

Dora K Madnff AdmtniQt'Tativp AcQiQtatit^

l^ilHrf^jl T A^artinolipli 1Q52

A wOO A^ • ^TACU^wX 1949

1948 ^ Administrative A^'^istant^

Lucv A. Moore 1951

1949

Leo Ai^%rX^ Moskowitz AvAVLY^AX^ TT AbArf 1952 rProcurement Clerks

Anne B. Oates 1952

Sata Ohashi 1949

Patricia Kellv Peed 1948 (Librarian)

Gertrude WA trA hA%AW Mm^t Z X^WA^AWA Reider 1948

1951

Ruth XX. UklA AAAa Stafford W LCIXXUA U 1950

Estelle H. Strauss 1952 (Library Assistant)

Joyce R. Toy iy4o

Reve J. Vineyard 1950

Mildred Vulkmanic 1952 (Administrative Assistant)

James R. Walker 1951 (Office Machine Operator)

Kathrine C. Warren 1950

Mildred B. Webb 1952 (Administrative Qerk)

Madeline V. Youll 1950 (Library Assistant)

F-7
]

i

j

1

I
ABRAMOWITZ, Milton (1915-1958) 4, 14, 41

ACTON, Forman S. 25

AD-X2 (See battery additives) 36

AITKEN, Alexander C. (1895-1967) 1, 53

ALT, Franz L. 15, 36

AMAC Applied Mathematics Advisory Council 5, 38

AMD Applied Mathematics Division 1, 6

American Mathematical Society 40

AMS Applied Mathematics Series 14

approximation 9, 20

ASTIN, Allen V. (1904-1984) 36, 37, 38

battery additives 36

BECKENBACH, Edwin F. (1906-1982) 5, 10, 13, 14

BLANCH, Gertrude 2, 7, 14

de BRUUN, Nicolaas G. 47

CAHN, Albert S. Jr. 7

Caltech 5, 15

CANNON, Edward W. 21, 36

cartoons 83

Cauchy-Riemann equations 29

Cauchy-Riemann integral 10

characteristic values and vectors 9, 45, 46, 47, 48

COCHRANE, Rexmond C. (1912-1986) 1 COMRIE, LesUe J. (1893-1950) 1, 2

CONDON, Edward U. (1902-1974) 4, 14, 36

conformal mapping 12, 13

conguate gradient method 29, 30, 54

COOPER, Jacob L. B. (1915-1979) 15 van der CORPUT, Johannes G. (1890-1975) 3, 29

COURANT, Richard (1888-1972) 1 CURTISS, John H. (1909-1977) 5, 14, 41, App. A, App. B

DARWIN, Charles G. (1887-1962) 4

differential equations, ordinary 9, 12, 15, 34

differential equations, partial 9, 12, 15, 24, 40

EISENHART, Churchill 4, 15, 37

ERDELYI, Arthur (1908-1977) 4

Federal Trade Commission 36

FELLER, Willy K. (1906-1970) 14

FORSYTHE, Alexandra I. (1918-1980) 42

FORSYTHE, George E. (1917-1972) 10, 19, 42

FOX, LesUe 4

FRIEDMAN, Bernard (1915-1966) 8

GERSCHGORIN, Semen A. (1901-1933) 12, 48

GOODWIN, Edward T. 4

Handbook of Mathematical Functions (AMS 55, Abramowitz and Stegim) 4, 14

Handbook of Physics (Condon and Odishaw) 14

HARTREE, Douglas R. (1897-1958) 2, 8, 9

HENRia, Peter (1924-1987) 31, 42, 48

HERBLOCK 83

HERRICK, Samuel (1911-1981) 5, 7

HESTENES, Arnold D. 25

HESTENES, Magnus R. (1906-1991) 4, 19, 53, 59

HILBERT, David (1862-1943) 9, 45

HOFFMAN, Alan J. 15, 21, 48

HUSKEY, Harry D. 8, 11, 42, App. C

ind-l
International Computing Center 3

inversion 19, 31, 40

JOHN, Fritz 15, 20, 23, 27

JORDAN, Peter F. 46

Journal Research 14

KAC, Mark (1914-1984) 19, 20, 21

KACZMARZ, Stephan (1895-1939) 52

KARUSH, William 18, 19, 21

KATO, Tosio 46

KELLEY, Marvin (1896-1971) 37, 38

LANCZOS, Cornelius (1893-1974) 2, 11, 12, 18, 30, 42, 61

LEFSCHETZ, Solomon (1884-1972) 8

LEHMER, Derrick H. 13, 28, 29

LEHMER, Emma T. 28, 29

linear programming 4, 20, 29, 30, 35

LITTLEWOOD, John E. (1885-1977) 1, 46

LOWAN, Arnold N. (1898-1962) 2

Mathematical Association of America vii matrix 9, 12, 18, 20, 30, 45, 47, 48

MILLER, Jeffrey C. P. (1906-1981) 4, 29, 45

MILNE, William E. (1890-1971) 9, 21, 24

Monte Carlo method 13, 18, 19, 21, 24, App. B

MORSE, Phillip M. (1903-1985) 4, 14

MOTZKIN, Theodore S. (1908-1970) 20, 24, 43

MTAC Mathematics Computation 14

NAML (later AMD) National Applied Mathematics Laboratories 1, 5, 6

NAR Numerical Analysis Research 41

NBS National Bureau of Standards 1, 40

NPL National Physical Laboratory (Ghreat Britain) 4 von NEUMANN, John (1903-1957) 8, 13, 46

NEWMAN, Morris 34

ODISHAW, Hugh (1916-1984) 15

OLVER, Frank W. J. 4

ONR Office of Naval Research 5

OSTROWSKI, Alexander M. (1893-1986) 12, 18

Pacific Journal cf Mathematics 14

PAIGE, Lowell J. 24, 41

PETERSON, Raymond P. Jr. (1925-1960) 10, 17

PICONE, Mauro (1885-1976) 2

polynomials 9, 10, 20

Post Office Department 36

projective planes 34

Prefects and Publications of the NAML 6

quadrature 10

RADEMACHER, Hans A. (1892-1969) 7, 29

REES, Mina vii, 5

RHODES, Ida (1900-1986) 2, 29

ROSENBLUM, Marvin 47

ROSSER, J. Barkley (1907-1989) 18, 19, 27, 37, 48

ind-2
SADLER, Donald H. (1908-1987) 2, 4

SAXON, David S. 11, 20, 31, 40, 41

SOUTHARD, Thomas H. 14,25,43

STEGUN, Irene A. 4, 14

STEIN, Marvin L. 23, 31, 54

STEIN, PhiUp (1890-1974) 19, 48

STIEFEL, Eduard L. (1909-1978) 29, 61

SWAC 8, 21, 31, 35, 41, App. C

Sweden 3

SZASZ, Otto (1884-1952) 7, 9

SZEGO, Gabor (1895-1985) 10

tables 7, 14, App. D

TAUSSKY, Olga, also TAUSSKY TODD, Olga vii, 14, 30,

TEICHROEW, Daniel 13

TEMPLE, George F. J. 45, 46

TODD, John 2, 4, 7, 9, 45

TOMPKINS, Charles B. (1912-1971) 5, 15, 34, 43

TURING, Alan M. (1912-1954) 11, 46

UCLA, University of California Los Angeles passim

ULAM, Stanislow M. (1909-1984) 13

USSR 3

WARSCHAWSKI, Stephen E. (1904-1989) 18

WASOW, Wolfgang R. 18, 40, 43

WEEKS, Sinclair (1893-1972) 36

WILE, Herbert S. 47

WILKINSON, J. H. (1919-1986) 4, 40

WILSON, Charles E. (1890-1961) 41

WOMERSLEY, John R. (1907-1958) 4

YOUDEN, WaUam J. (1900-1971) 15

YOWELL, Everett C. 10, 24, 25

ind-3
I
NIST.1 14A U.S. DEPARTMENT OF COMMERCE

(REV. 3-90) NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

BIBLIOGRAPHIC DATA SHEET

1. PUBUCATION OR REPORT NUMBER

NIST/SP-730

2. PERFORMING ORGANIZATION REPORT NUMBER

3. PUBUCATION DATE

August 1991

4. TITLE AND SUBTITLE

NBS-INA—The Institute for Numerical Analysis—UCLA 1947-1954

5. AUTHOR(S)

Hestenes, M. R. [UCLA]; Todd, J. [CALTECH]

6. PERFORMING ORGANIZATION (IF JOINT OR OTHER THAN NIST, SEE INSTRUCTIONS)

U.S. DEPARTMENT OF COMMERCE

NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

GAITHERSBURG, MD 20899

7. CONTRACT/GRANT NUMBER

8. TYPE OF REPORT AND PERIOD COVERED

Final

9. SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (STREET, CITY, STATE, ZIP)

The Mathematical Association of America

1529 Eighteenth Street, N.W.

Washington, DC 20036

10. SUPPLEMENTARY NOTES

11. ABSTRACT (A 200-WORD OR LESS FACTUAL SUMMARY OF MOST SIGNIFICANT INFORMATION. IF DOCUMENT INCLUDES A SIGNIFICANT BIBUOGRAPHY OR

UTERATURE SURVEY, MENTION IT HERE.)

This is a history of the Institute for Numerical Analysis (INA) with special emphasis

on its research program during the period 1947 to 1956. The Institute for Numerical Analysis

was located on the campus of the University of California, Los Angeles. It was a section

of the National Applied Mathematics Laboratories, which formed the Applied Mathematics

Division of the National Bureau of Standards (now the National Institute of Standards and

Technology), under the U.S. Department of Commerce.

This history of the program at INA is concerned primarily with the development of

mathematics pertinent to solving numerical computations. This development could happen only

if some mathematicians were proficient in handling the electronic digital computers. To

insure that there would be some, INA was constituted. It was well funded, and could attract

first class mathematicians to take a year off for research at INA. There they were in the

midst of people solving problems of considerable difficulty using the digital computers.

They were thus enticed into using them. When this happened, many important developments

emerged. This history is centered around these people and asks "Who were there?", "What

were their interests?", "What did they do?".

12. KEY WORDS (6 TO 12 ENTRIES; ALPHABETICAL ORDER; CAPITAUZE ONLY PROPER NAMES; AND SEPARATE KEY WORDS BY SEMICOLONS)

differential equations; digital computers; history; linear programming; matrix;

numerical analysis; UCLA

13. AVAILABIUTY

1

X

UNUMITED

FOR OFFICIAL DISTRIBUTION. DO NOT RELEASE TO NATIONAL TECHNICAL INFORMATION SERVICE (NTIS).

ORDER FROM SUPERINTENDENT OF DOCUMENTS, U.S. GOVERNMENT PRINTING OFFICE,

WASHINGTON, DC 20402. ORDER FROM NATIONAL TECHNICAL INFORMATION SERVICE (NTIS). SPRINGFIELD, VA 22161.

14. NUMBER OF PRINTED PAGES

182

IS. PRICE

ELECTRONIC FORM

lylJL LJ M. Technical Publications

Periodical

Journal of Research of the National Institute of Standards and Technology— Reports NIST

research and development in those disciplines of the physical and engineering sciences in which

the Institute is active. These include physics, chemistry, engineering, mathematics, and computer

sciences.

Papers cover a broad range of subjects, with major emphasis on measurement methodology and

the basic technology underlying standardization. Also included from time to time are survey

articles on topics closely related to the Institute's technical and scientific programs. Issued six

times a year.

Nonperiodicals

Monographs — Major contributions to the technical literature on various subjects related to the

Institute's scientific and technical activities.

Handbooks — Recommended codes of engineering and industrial practice (including safety codes)

developed in cooperation with interested industries, professional organizations, and regulatory

bodies.

Special Publications — Include proceedings of conferences sponsored by NIST, NIST annual

reports, and other special publications appropriate to this grouping such as wall charts, pocket

cards, and bibliographies.

Applied Mathematics Series — Mathematical tables, manuals, and studies of special interest to

physicists, engineers, chemists, biologists, mathematicians, computer programmers, and others

engaged in scientific and technical work.

National Standard Reference Data Series — Provides quantitative data on the physical and chemical

properties of materials, compiled from the world's literature and critically evaluated. Developed

under a worldwide program coordinated by NIST under the authority of the National Standard

Data Act (Public Law 90-396). NOTE: The Journal of Physical and Chemical Reference Data

(JPCRD) is published bi-monthly for NIST by the American Chemical Society (ACS) and the

American Institute of Physics (AIP). Subscriptions, reprints, and supplements are available from

ACS, 1155 Sixteenth St., NW., Washington, DC 20056.

Building Science Series — Disseminates technical information developed at the Institute on building

materials, components, systems, and whole structures. The series presents research results, test methods, and performance criteria related to the structural and environmental functions and the

durability and safety characteristics of building elements and systems.

Technical Notes— Studies or reports which are complete in themselves but restrictive in their

treatment of a subject. Analogous to monographs but not so comprehensive in scope or definitive

in treatment of the subject area. Often serve as a vehicle for final reports of work performed at NIST under the sponsorship of other government agencies.

Voluntary Product Standards — Developed under procedures published by the Department of

Commerce in Part 10, Title 15, of the Code of Federal Regulations. The standards establish

nationally recognized requirements for products, and provide all concerned interests with a basis

for common understanding of the characteristics of the products. NIST administers this program

as a supplement to the activities of the private sector standardizing organizations.

Consumer Information Series — Practical information, based on NIST research and experience,

covering areas of interest to the consumer. Easily understandable language and illustrations

provide useful background knowledge for shopping in today's technological marketplace.

Order the above NIST publications from: Superintendent of Documents, Government Printing Office,

Washington, DC 20402.

Order the following NIST publications— FIPS and NISTIRs—from the National Technical Information

Service, Springfield, VA 22161.

Federal Information Processing Standards Publications (FIPS PUB) — Publications in this series

collectively constitute the Federal Information Processing Standards Register. The Register serves

as the official source of information in the Federal Government regardmg standards issued by

NIST pursuant to the Federal Property and Administrative Services Act of 1949 as amended,

Public Law 89-306 (79 Stat. 1127), and as implemented by Executive Order 11717 (38 FR 12315,

dated May 11, 1973) and Part 6 of Title 15 CFR (Code of Federal Regulations).

NIST Interagency Reports (NISTIR) — A special series of interim or final reports on work

performed by NIST for outside sponsors (both government and non-government). In general,

initial distribution is handled by the sponsor; public distribution is by the National Technical

Information Service, Springfield, VA 22161, in paper copy or microfiche form.
U.S. Department of Commerce

National Institute of Standards and Technology

Gaithersburg, MD 20899

Official Business

Penalty for Private Use $300