i I I

CSI REPORT

No. 12

Evaluating Historical Materials

by

Dr. Larry D. Roberts

COMBAT

STUDIES

INSTITUTE

90-3784
CSI Reports are short-term research papers prepared in

response to official inquiries. They are based mainly on

secondary sources and provide basic information on the

subject under consideration. The views expressed in a

CSI Report are those of the author and not necessarily

those of the Department of the Army or the Department of

Defense.

Combat Studies Institute

Missions

The Combat Studies Institute was established on 18 June 1979

as a department-level activity within the U.S. Army Command and

General Staff College, Fort Leavenworth, Kansas. CSI has the

following missions:

1. Conduct research on historical topics pertinent to the doctrinal

concerns of the Army and publish the results in a variety of formats

for the Active Army and Reserve Components.

2. Prepare and present instruction in military history at

USACGSC and assist other USACGSC departments in integrating

military history into their instruction.

3. Serve as the U.S. Army Training and Doctrine Command's

executive agent for the development and coordination of an

integrated, progressive program of military history instruction in the

TRADOC service school system.
CSI Report No. 12

Evaluating Historical Materials

by

Dr. Larry D. Roberts

Combat Studies Institute

U.S. Army Command and General Staff College

Fort Leavenworth, Kansas 66027-6900

90-3784
In this publication, when the masculine gender is

used, both men and women are included.
Many who read history make the mistake of accepting at face

value all the information an author presents. This is true of those

reading a single history book and those conducting historical

research and analysis. In reality, only part of any historical work is

uncontested fact; the rest is the author's interpretation or opinion of

how and why things happened. In addition, errors of fact can easily

be found in most books, articles, or documents due to the author's

error or errors in the documents he used in his research. The

realization of these failings often causes some readers to go to the

extreme of doubting all that they read. History can be of immense

value to any reader, including the professional soldier, if these

shortcomings are recognized and the facts are carefully sifted. To

do this, the reader must critically evaluate the book he is reading or

the materials he is using in his analysis.

This CSI Report deals with ideas and techniques the professional

soldier can use to assess historical materials critically. The soldier

can apply this evaluation process to individual history books or the

assorted documents, articles, and .monographs used in historical

research. There is nothing complex or even magical about this

procedure. The techniques are basically the same commonsense

rules or guides the average person uses every day. Essentially, the

reader, much as a consumer, is concerned with the quality of the

materials used in making the item, the craftsmanship of the person

or persons who did the work, and the purpose for which the item

was designed to serve.

Areas of Evaluation

For any historical writing, the first point to examine is the expertise

or qualifications of the author. The writer's breadth and depth of

knowledge, skill in finding and using sources, and ability to marshal

facts to support his interpretations are essential in the reliability and

value of a historical work. The author's competence or ability to do

these things is generally based on one or more of three general

backgrounds: academic preparation, extensive personal research,

and/or personal experience. Each has its own value and limitations.
The completion of a graduate program in history or one of the

associated disciplines means that the individual has attained a basic

level of knowledge in his field of specialization and has developed,

to a degree, research and analysis skills. The reader should not

accept an author's expertise in a subject simply because the author

has a master's degree or even a doctorate. If the subject of the

historical work falls within the author's principal area of academic

study, then the reader can accept the writer's competency to deal

with the material. For example, a historian who spent his formative

education studying German operations in World War II is probably

able to write intelligently about the Wehrmacht on the Eastern Front.

However, a book on Soviet infiltration tactics written by an individual

with a German intellectual background may be suspect. It is

possible, however, that a trained historian can reach beyond the

areas of study he dealt with in school. But the extended study of a

subject and a close familiarization with sources add unquestioned

value to a book.

Graduate education is not the only background that qualifies a

writer to address a subject with authority. Neither Barbara

Tuchman nor Cornelius Ryan had formal training as historians. Yet

they have authored some of the best books written on World War I

and the Normandy invasion. Tuchman and Ryan based their books

on extensive archival research and, in Ryan's case, hundreds of

interviews with participants. For these authors, depth and breadth

of research gave every indication that the authors' stories were

accurate and their conclusions sound. However, gauging an

author's credibility or competency by the size of the footnotes or

bibliography in his book is difficult. The reader has to decide

whether the writer has looked at enough of the right material to

substantiate the text. If the reader is unfamiliar with the subject, he

is normally unable to judge the completeness of the research.

The last thing that can reveal the expertise of an author is

personal experience. The accuracy and value of Charles

MacDonald's Company Commander stem largely from his personal

experience as a company commander in World War II. He was
able to describe the world of the combat infantry far better than

anyone else who had not experienced conflict at that level.

MacDonald, as with others who write of their own experiences, is

the equivalent of an eyewitness. In a court of law, eyewitness

testimony is the most decisive and compelling of all. However, the

ability to speak as one who was there does not mean the author is

totally reliable. For example, the infantryman who landed on Utah

Beach on 6 June 1944 would view the overall Normandy landings in

a different light from a comrade who had to claw his way across the

sand at Omaha. Similarly, the view of events of a V Corps staff

officer would differ from that of a man in the first wave to come

ashore. In other words, the value of personal experience can be

offset by the limitations of that experience when applied to a

broader question.

Researchers and analysts have the same requirement to consider

authorship as those reading historical books, monographs, and

articles. Reports, memorandums, staff summaries, and the like

have authors, and the author's background or experience, proximity

to the action, and duty position influence his credibility in preparing

these documents. For example, a summary of operations written

by one of the participating company commanders would be good,

at least for the area that involved that company. A similar summary

written by a division staff officer might be a better overall report but

would not necessarily be based on personal experience. The

further removed from the event by time and distance, the less likely

the author's account is completely accurate.

The writer's background, professional training, depth and breadth

of research, or personal experience does not determine the value of

a written historical work. However, they are indicators the reader or

researcher can use to evaluate the historical material in question.

Of equal importance to the author's qualifications is his purpose

for writing the material in the first place. Few historical works exist

simply to tell a story. Some are an attempt to retell a story in light

of new evidence; others are an effort to correct what is perceived to
be errors or incorrect interpretations in earlier works. Memoirs are

historical works in which the author tells his side of an issue or

story. Sometimes, his memoirs are a response to unfavorable

treatment in other books or articles.

It is important for a reader to determine an author's purpose, for it

can reveal the author's biases or slants. Those advancing a

particular point of view marshal all the facts that support their point

and often disregard evidence that runs counter to their thesis. Even

the most objective historian tends to describe and analyze events

through the rose-colored glasses of his own experiences or values.

The knowledge of the book's purpose and potential slant allows the

reader to place the evidence presented in the appropriate light,

possibly disregarding some information due to its obvious bias.

The purpose of a document is normally quite evident, sometimes

being stated in the title. Hence, we have "reports" on various

topics, "summaries of operations," and "memorandums for the

record" on specific issues. Historians generally give greater value

to these types of documents as they are supposed to be more

objective and have less inherent biases. However, the reader

cannot assume that these types of material are totally factual. Army

historian Oliver L Spaulding noted the questionable accuracy of

General Zachary Taylor's reports during the Mexican War:

His [Taylor's] reports were written by an acute military student a

clever and discreet artist in words; it has been said that they

"never lied and never told the truth." In every single battle, from

Palo Alto to Buena Vista, the report is so phrased that, while it is

difficult to challenge any specific statement of fact, the impression

is precisely that one most creditable to the commander. Certain

facts are played up, others toned down, still others quietly

omitted.*

Unfortunately, Taylor's reports were not the only historical

documents written more to reflect favorably on the commander than

to report accurately the operations of the command.

*Colonel Oliver L Spaulding, Books: How to Judge Them and How to Use Them. Lecture

delivered at the U.S. Army War College, Washington Barracks, DC, 2 October 1922, 10.
Having established the author's credentials or qualifications and

the purpose for writing the text, the reader must evaluate the

sources the author used in preparing his book or article. Sources

are the equivalent of raw materials and, as such, affect the text just

as the quality of the materials used in making a car affect the car's

quality. The number and type of sources used in a work reflect

directly the breadth and depth of research. If the sources are poor

or insufficient to support the book, the value of the book suffers.

Historians place the greatest value on "primary sources," which

are reports, personal accounts, interviews, and other written

material that are concurrent with the event under study. As

variations of the eyewitness category of material, these types of

information have great value because they reflect either a firsthand

account of an event or action or an interview with a participant. The

words and descriptions have not been filtered through the mind of a

person distant from the event in space and time. Because of this,

historians consider primary sources to be important, if not crucial, to

establishing accuracy. However, these materials must be used with

caution.

Readers and writers must be careful to distinguish independent

accounts from dependent accounts. For example, a battalion

commander's report of the location of his companies, his S3 report,

and the company commanders' reports are simply variations of the

same information. The first two probably depend on the latters'

statements. However, if a neighboring battalion commander reports

the same information or if captured enemy documents identify

companies at those locations, we have independent verification of

the companies' positions.

In this respect, the breadth of sources is as important as the type

of sources. The description of a battalion's operations cannot be

based on a single individual's observations. A soldier sees only a

narrow part of an action. His view is physically limited to the area

he occupies, and he generally has little firsthand knowledge of

what's happening on his flanks or rear. Even the forces opposing
him are a matter of impression. A heavy volume of fire over an

extended area might lead him to conclude he is facing a reinforced

company or battalion minus. The reality might be a platoon on an

extended front with several automatic weapons. Only the enemy's

documents or accounts could confirm or invalidate the

commander's perception.

Among the most valuable sources a writer can use are official

reports and personal accounts, be they in an interview, diary, or

published account. However, these items can be extremely

misleading if not checked against other reports, messages, orders,

and so forth. An author who relies on only one source or one class

of sources when others are available to him does not have the

complete story. His narrative or analysis should be used with

caution.

The need for breadth of sources is even more important when the

writer relies on "secondary sources." These items are generally

other historical writings, books, and articles that deal with the

subject in question. They constitute secondhand information as the

primary sources of information have been filtered through the

thoughts, impressions, and possibly biases of the writers. In some

respects, reports written long after the action or by the successor of

the actual commander are more appropriately secondary sources,

even though they date from the same general time period. For

example, the Secretary of War's Annual Report for 1877, which

deals with the Nez Perce campaign, was written long after the

conclusion of hostilities and was based on information that had

passed through a number of command headquarters.

In the final analysis, the type and breadth of sources is driven

largely by the purpose of the work. An author attempting to

describe in general terms the history of World War II can use a

number of secondary works as the basis for his story. If, however,

the purpose is to analyze combat operations for a corps, division,

or subordinate units, the author must use as many primary sources
as possible. The depth and accuracy of analysis depends on the

depth and breadth of sources.

We have examined the author's qualifications; the purpose for

writing the book, article, or document; and the sources used as the

basis for that work. The last area for evaluation is the work itself,

the body of the book or article. In this assessment, the reader must

ultimately answer a two-part question: does the author support his

major points, and does the work achieve the purpose or objective

for which it was written? Even the simplest statement of purpose or

thesis has several supporting ideas or elements. Much like a piece

of furniture, if too many of these supporting "legs" are absent or not

well constructed, the whole thing falls apart. The reader must

decide if the key supporting arguments or concepts are validated by

evidence and whether there are enough supporting arguments to

support the author's main idea. Finally, if the book or written work

does not achieve the author's intended purpose, the author has

failed. The author's inability to support his major points or to

achieve his goal is much like a commander failing to carry out his

mission.

The reader must be cautious, however, not to make an

all-or-nothing, good-or-bad judgment about the material as a whole.

There are often jewels of historical facts, stories, or important

insights even in the worst publications. A reader should be

sensitive to these and credit the author with some contribution,

even if it is by accident. This goes back to the basic premise that a

reader can neither accept at face value all that an author says nor

reject it totally.

The Evaluation Process

The actual process of evaluating historical material has three

phases. In the first phrase, the preliminary assessment, the reader

determines the qualifications of the author, establishes the purpose

of the book, and surveys the sources used. Beginning the
evaluation with the preliminary phase has the additional benefit of

possibly saving the reader some time.

Most readers take several hours to digest a book of 400 to 500

pages. Given the requirements of the job, the need to spend time

with the family, and other social or community obligations, few

soldiers have more than an hour or two each evening, if that, for

professional or leisure reading. If a history book proves to be of

little value, the reader has probably wasted a number of hours.

Therefore, a preliminary evaluation can give the user some idea of

whether the book will justify the investment in time that will be

required.

The preliminary analysis begins with the title page, which tells the

reader the title, author, place and date of publication, and the name

of the publisher. The title gives the general subject of the book and

can indicate the purpose of the book. However, titles can be

misleading. A number of authors who have no difficulty in writing

hundreds of pages of text struggle with a few words in the title.

Failing to match title with subject can lead the reader to expect

something he will not get. A glance at the book's table of contents

should give a closer approximation of the book's true coverage.

Since there are few tables of contents for articles, the reader must

trust the author's judgment and choice of words. As a rule, the

reader will find out quickly if the title and actual subject of the article

are mismatched. In most cases, the titles of most documents are

normally accurate and often identify the purpose of the material. A

"summary of operations" is a narrative that has as its purpose the

concise description of what a unit or organization did.

Publication information is of mixed value. Experienced historians

know the major publishing companies with reputations for printing

quality works. These firms often refer manuscripts to other

historians for comments and recommendations on whether to print

them or not. The average reader, however, seldom has this much

familiarity with publishers. The date of publication, however, can
provide some information. A book on World War II, written by a

participant and published shortly after the war, would have some

advantages. The memories of that participant are comparatively

fresh, and the author may still have contact and collaboration with

the men he served with. On the other hand, the reader should

exercise caution when reading a similar book written thirty years

after the war. The converse may be true of a historian's work on

the same subject. A history of that war written in the early 1950s

could not have contained information that was still classified. Thirty

years later, that material is probably much more available. Recently,

several books on the Enigma and Ultra secrets have appeared, and

these books simply could not have been written twenty years ago.

Authorship is the final point to examine on the title page. Again,

the objective is to determine the author's qualifications, and there

are several ways to do this. The book's dust jacket provides some

information, most often describing the author in the most flattering

terms. This must be taken with a few grains of salt. At a minimum,

the jacket will give the reader some idea of the author's educational

background, work experience, and other published books. Other

sources that often comment on an author's professional

background are reviews of the book that are published in

professional journals and periodicals such as Book Review Digest

In a number of ways, book reviews provide an invaluable source of

information on a publication. A dictionary of authors or a guide to

scholars gives such information as education, previous job

experience, and other published works, sometimes including

articles. Once more, the reader must be cautious not to place too

much stock in academic degrees or the number of books written.

Virtually anyone, given enough time, can write several bad books.

The value of personal experience must be determined in association

with the book's subject and purpose.

If the work is a book, the next step in the preliminary assessment

is to glance at the preface. Quite often, the author will state his

reasons for writing the book and also include some discussion of

his use of sources. The statement of purpose or thesis is important
to note, even to the extent of writing it down. If the item under

scrutiny is an article, the reader should look at its first few

paragraphs. Good writers will have their thesis statement

somewhere in the beginning of the work. When dealing with

documents, be they report summaries or analyses, the title of a

document generally states the purpose of the material. Once more,

the purpose is important because the reader must ultimately

determine if the author achieved his goal.

The final step in the preliminary evaluation, that dealing with

sources, takes the reader to the bibliography. Here, the reader

examines the number and types of sources used. Given the

author's subject and purpose, the reader must decide whether the

author has the right kind of sources and enough of them to address

the topic adequately. The reader should beware of a flaw of some

historians, which is to "pad" the bibliography-that is, adding a

number of books or articles that the author did not consult in order

to give the appearance of extensive research. Readers can check

this by quickly comparing the chapter footnotes with the

bibliography. The footnotes identify what sources the author

actually used in his book or article. In the actual reading of the text,

the reader will get an even clearer picture of the kind of sources the

author examined or should have examined.

If, after the preliminary evaluation, the reader decides that the

book, article, or other material is worth the time to read, he begins

the second part of the evaluation, the concurrent evaluation phase.

Throughout this process, the individual must read critically and look

for the author's proof of his key points. However, the reader must

also be as objective in weighing the author's key points and

conclusions as he expects the author to be in amassing the

evidence.

A particularly useful technique a reader can use during the

concurrent evaluation phase is nothing more than maintaining a

small notebook for jotting down the overall purpose of the work,

interesting points in the text, questions, and so forth. Without a

10
notebook, the reader must rely on his memory to answer basic

questions. If an individual reads other books or articles on the

same subject, the collected notes can be used to determine areas

of conflict or agreement. Some who have done considerable

professional reading have organized these notes according to

subjects, such as leadership, tactics, logistics, and so on.

The concurrent evaluation process has two parts. First, while the

reader is moving through the text, he must stop from time to time to

ask some basic questions:

• Is the material I am reading appropriate to the subject of the

book or article? Some authors tend to wander off the path and

add material that has little relation to the subject. In many

cases, they found a particularly interesting story or point and

put it in the text even though it is irrelevant.

• Has the author supported his main points with references to

sources or by logic? A book or article without footnotes is a

minefield. Useful information of insight may be found in it, but

the danger of error, misunderstanding, and bias is

unquestioned. The reader should, in short, expect the author

to prove his points using as many independent sources as

possible.

• Is there anything the writer should be addressing in this section

or chapter that he has not? If there are questions the text

raises, the reader should jot them down in his notebook.

Successive paragraphs or chapters may address them later. If

not, the reader should note this in his final evaluation of the

book or article.

The second part of the concurrent evaluation process occurs

when the reader has finished the book, article, or document. The

reader must decide whether the author achieved his purpose or

objective in writing the item and determine whether the author

adequately supported his position or interpretation of events with

the right kind and number of supporting sources. By answering

11
these questions, the reader actually determines the value of the

historical material he has read. Even if the item has little value

overall, the reader must objectively decide if there are good points

or ideas in the text.

The final assessment of a historical work takes place over time.

The subsequent evaluation of a book occurs when it is measured

against other books or articles on the same subject. A reader

interested in a facet of history, be it a battle or a dimension of

warfare such as logistics or leadership, cannot develop a full

understanding from a single book. He must look at other

interpretations or studies, much as a writer must consult a number

of sources to get the clearest understanding of what took place or

why. The reader should expect to see differences in authors'

opinions and possible disagreement over what took place. In the

end, he must decide which writer bestproves his case.

The notebook that was mentioned earlier is invaluable in this

process. Few readers have the ability to remember the key points

of several books on the same subject. After a while, the reader

would be hard pressed to say who wrote what. A glance at his

notebook will often refresh his memory and help him weigh the

comparative value of what he has read.

In the evaluation process, the reader must be cautious not to

commit three mistakes:

• First, the best books are not necessarily the ones that agree

with the reader's preconceived ideas about a subject. The best

historical works might actually be those that challenge the

reader's assumptions and force him to reevaluate his own

beliefs or opinions.

• Second, a reader must be careful in comparing books with

different purposes. A general history of World War II has

different structure and source requirements from a text that

analyzes infantry operations in the same war. Similarly, the

12
criteria by which we gauge "qualified" authors may be different

for these two texts.

• Finally, the reader must remember that newer does not

necessarily mean better. Books referred to as "classics" have

stood the test of time. The interpretations or descriptions

found in military history classics have remained sound even in

light of the discovery of new information.

Reading and studying history can be of immense value to the

professional soldier. History can provide the kind of insights into

battle, leadership, tactics, and so forth that could not otherwise be

gained. Even combat experience is limited to the circumstances of

particular battles and the level of military operations that existed at

the time. For example, those blooded in Vietnam know well that

kind of war. However, those experiences would be of a limited

value if applied to the mechanized war that happened in 1944 on

the plains of Germany. History allows the soldier to learn from the

actions of soldiers as distant in time as the Greeks and Persians

and as close as the British in the Falklands.

Unfortunately, a soldier must invest some time to benefit from the

experiences of those who have come before him. The key is to

make that time as productive as possible. Getting the absolute

most out of the books that are read is productive. The ability to

evaluate historical materials, whether they are read or used in

research and analysis, enhances the basic value of the material

itself. These techniques can assist the soldier in learning about his

profession and what that profession must be able to do in time of

war.

The following works can assist a reader in understanding the

nature of historical materials and the best way to use them:

Barzun, Jacque and Henry Graff. The Modem Researcher. 4th

edition. New York: Harcourt, Brace, and Jovanovich, 1985.

13
Cantor, Norman F., and Richard I. Schneider. How to Study

History. Arlington Heights, IL: AHS Publishing Corporation,

1967.

Johnson, Allen. The Historian and Historical Evidence. New York;

Charles Scribner's Sons, 1926.

Gray, Wood, et al. Historian Handbook: A Key to the Study and

Writing of History. 2d edition. Boston: Houghton Mifflin Co.,

1964.

Elton, G. R. The Practice of History. New York: Thomas Y.

Crowell, 1967.

14

90-3784-CGSC-1000-29 June 90